Image shelter segmentation method, system and device based on low-rank adaptive fine tuning

Abstract

The invention discloses an image shelter segmentation method, system and device based on low-rank adaptation fine tuning, and belongs to the technical field of image analys.The method comprises the following steps that feature extraction is conducted on an input image through a feature extraction assembly of an image encoder, and the feature extraction assembly comprises a self-attention mechanism with LoRA branches; and segmenting the image through the mask decoder and the features to obtain a segmented image of the shielding object. The LoRA branch allows the SAM to only update a small part of parameters in the training process of the medical image, so that the calculation and storage overhead is saved, the segmentation performance is ensured, and the fine adjustment difficulty of the model is reduced; the improvement on the SAM ensures that automatic segmentation can be completed without depending on any prompt.

Description

Technical Field

[0001] The present invention relates to the field of image analysis technology, and in particular to an image occluder segmentation method, system and device based on low-rank adaptive fine-tuning. Background Art

[0002] Pelvic fracture is a serious trauma involving complex orthopedic surgery and has always been a huge challenge for orthopedic surgeons. With the development of medical technology, minimally invasive pelvic surgery has gradually become an emerging method for the treatment of pelvic fractures due to its advantages such as less trauma and faster recovery. Ensuring accurate positioning and navigation is crucial to the success of minimally invasive pelvic surgery and patient safety. The positioning and navigation of minimally invasive pelvic surgery usually rely on 2D / 3D registration technology to provide real-time and accurate three-dimensional positioning for minimally invasive surgery. CT is usually used as preoperative 3D data and X-ray is used as intraoperative 2D data. The main task is to find the best 3D transformation to minimize the misalignment between the 2D image and the 3D body.

[0003] Due to individual differences among patients and the use of essential medical devices during surgery, the patient's preoperative CT and intraoperative X-ray images may have occlusions formed by implanted screws or surgical instruments, causing image distortion and reduced image quality, which in turn affects the subsequent image analysis and registration in minimally invasive surgery. Therefore, it is necessary to segment the occlusions on the preoperative CT and intraoperative X-ray images to provide high-quality image data for subsequent image processing, doctor diagnosis, and surgical navigation.

[0004] Early surgical instrument segmentation was based on traditional image processing technology, mainly relying on morphological methods, prior knowledge-based methods, filter methods, etc. However, these methods are usually designed for a specific task or scenario and lack the ability to adapt to complex and dynamically changing scenarios. With the development of deep learning, surgical instrument segmentation has gradually evolved from traditional methods to modern deep learning-based methods.

[0005] Current deep learning-based occluder segmentation methods rely heavily on a large amount of annotated data for model training. However, obtaining annotated data is very time-consuming and laborious, and the lack of large-scale annotated datasets makes it difficult to guarantee model accuracy. Existing occluder segmentation methods need to be improved in terms of segmentation accuracy and speed. Summary of the invention

[0006] In view of the above-mentioned technical problems existing in the prior art, the present invention provides an image occluder segmentation method, system and device based on low-rank adaptive fine-tuning, and performs occluder segmentation based on the SAM model fine-tuned by LoRA, so as to obtain better results by fine-tuning a small number of parameters, thereby greatly reducing the consumption of computing resources.

[0007] The present invention discloses an image occluder segmentation method based on low-rank adaptive fine-tuning, comprising the following steps: extracting features from an input image through a feature extraction component of an image encoder, wherein the feature extraction component comprises a self-attention mechanism with a LoRA branch; segmenting the image through a mask decoder and features to obtain a segmentation map of the occluder.

[0008] Preferably, the projection layer of the self-attention mechanism is provided with a LoRA branch.

[0009] Preferably, the Q projection layer and the V projection layer of the self-attention mechanism are respectively provided with LoRA branches.

[0010] Preferably, the LoRA branch includes two linear layers, and the second weight matrix and output of the LoRA branch are expressed as: ; in, Represented as the output of the LoRA branch, Expressed as the second weight matrix of the LoRA branch, F is represented as a sequence of tokens of input, W is the first weight matrix of the projection layer, A and B They are the linear layers of the LoRA branch respectively.

[0011] Preferably, the output projection of the self-attention mechanism is expressed as: ; in, Att(Q,K,V) Denoted as the output projection, W q , W k , W v They are Q , K and V The first weight matrix of the projection layer; A q , B q , A v and B v They are the trainable parameters of the LoRA branch, C out The number of channels represented as the output token sequence.

[0012] Preferably, the segmentation network of the occluder of the two-dimensional X-ray image includes an image encoder and a mask decoder, and the feature extraction component of the image encoder includes a transformer module. The transformer module includes a first normalization layer, a self-attention mechanism with a LoRA branch, a second normalization layer, and a multi-layer perceptron; The transformer module extracts the features of two-dimensional X-ray images by: The embedded block of the two-dimensional X-ray image is aligned with the embedded block after the first normalization layer and the self-attention mechanism to obtain the first fusion feature; The first fused feature is aligned with the first fused feature after passing through the second normalization layer and the multi-layer perceptron to obtain the output feature.

[0013] Preferably, the mask decoder comprises a plurality of decoding units connected in sequence, the decoding unit comprises an upsampling layer and two decoding components connected in sequence, and the decoding component comprises a nonlinear activation function, a normalization layer BN and a convolution layer connected in sequence.

[0014] Preferably, the segmentation network of occluders in a three-dimensional CT image includes an image encoder and a mask decoder, wherein the image encoder is composed of a block embedding module, a position embedding module, and a plurality of feature extraction components based on an attention mechanism; The block embedding module is used to segment the input CT image into small 3D patches and embed each patch into a high-dimensional feature vector to obtain an embedding vector; The position embedding module is used to add the 3D position encoding to the embedding vector to preserve the spatial position and information; The feature extraction component is used for feature extraction and consists of a multi-head self-attention mechanism and a feedforward network; The mask decoder includes a transformer module, a transposed convolution module, and a multi-layer perceptron; The transformer module is used to receive the features from the encoder and further process the feature information in combination with its own input; The transposed convolution module is used for the transposed convolution operation, which upsamples the feature information, restores the low-resolution features to high-resolution features, and gradually restores them to the original resolution; The multi-layer perceptron MLP is used to further process and fuse high-resolution features and output an occlusion segmentation map.

[0015] The present invention also discloses a system for implementing the above-mentioned image occluder segmentation method, comprising a feature extraction module and a decoding module. The feature extraction module is used to extract features from an input image through a feature extraction component of an image encoder, wherein the feature extraction component includes a self-attention mechanism with a LoRA branch; The decoding module is used to segment the image through the mask decoder and the features to obtain a segmentation map of the occluder.

[0016] The present invention also provides a device, comprising an input unit, a processing unit and an output unit. The input unit is used to obtain an input image; The processing unit is used to execute the code. When the processing unit processes the code, the image occlusion segmentation method is executed. The output unit is used to output a segmentation map of the occluder.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: the LoRA branch allows SAM to update only a small part of the parameters during the training process of medical images, which not only saves computing and storage overheads, but also reduces the difficulty of model fine-tuning while ensuring segmentation performance; the deletion of the prompt encoder module allows the model to complete automatic segmentation without relying on any prompts. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the image occluder segmentation method based on low-rank adaptive fine-tuning of the present invention; Figure 2 It is the architectural diagram of the two-dimensional model; Figure 3 This is a schematic diagram of the attention mechanism based on LoRA; Figure 4 It is a three-dimensional model architecture diagram; Figure 5A is the first input image; Figure 5B This is the first segmentation effect diagram; Fig. 6A is the first input image; Figure 6B This is the first segmentation effect diagram; Fig. 7A is the first input image; Figure 7B This is the first segmentation effect diagram; Fig. 8A is the first input image; Figure 8B This is the first segmentation effect diagram; Fig.9A is the first input image; Fig. 9B This is the first segmentation effect diagram; Fig. 10A is the first input image; Fig. 10B This is the first segmentation effect diagram; Fig.11 It is a system logic block diagram of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The present invention is further described in detail below in conjunction with the accompanying drawings: Overview: SAM (Segment Anything Model) is a general image segmentation model, including an image encoder, a prompt encoder, and a mask decoder. The image encoder is used to encode the input image and convert the image into a feature representation that can be processed and understood by the computer for subsequent analysis and processing. The prompt encoder is responsible for converting various prompt information provided by the user, such as text descriptions, clicks, box selections, etc., into prompt codes so that the model can understand the user's intentions. The mask decoder is used to combine the image code and the prompt code, perform decoding operations, predict the area in the image that needs to be segmented, and generate the corresponding segmentation mask.

[0021] The Transformer Block is an attention-based encoder and the core component of the SAM model. It consists of two parts: the Attention Mechanism and the Feed-Forward Neural Network. Projection is the core component of the self-attention mechanism, which can help the model capture contextual relationships when processing sequence data, while the Feed-Forward Neural Network is used for feature extraction and representation learning after the attention mechanism.

[0022] Low-Rank Adaptation (LoRA) fine-tuning is an efficient model fine-tuning method. It is a lightweight model fine-tuning technology that can improve the performance of the model without significantly increasing the complexity of the model. Through LoRA, the SAM model can be fine-tuned while keeping the model lightweight. Since LoRA updates some parameters of the model by learning the low-rank matrix instead of directly updating the original weight matrix, it can achieve better results by fine-tuning a small number of parameters, greatly reducing the consumption of computing resources and being introduced into various large model fine-tuning tasks.

[0023] In the present invention, it should be understood that terms such as "include" or "have" are intended to indicate the existence of features, numbers, steps, behaviors, components, parts or a combination thereof disclosed in the specification, and do not exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts or a combination thereof exist or are added.

[0024] The first aspect of the present invention provides an image occluder segmentation method based on low-rank adaptive fine-tuning, such as Figure 1 , including the following steps: Step 101: Perform feature extraction on the input image through a feature extraction component of an image encoder, wherein the feature extraction component includes a self-attention mechanism with a LoRA branch.

[0025] Among them, the input features of the self-attention mechanism are aligned and fused with the output features of the self-attention mechanism after the LoRA branch. Considering that the native SAM is applied to natural images, the occluder segmentation based on LoRA can be trained using a small amount of annotated pelvic dataset.

[0026] Step 102: Delete the prompt encoder.

[0027] Step 103: Segment the image using the mask decoder and the features to obtain a segmentation map of the occluder.

[0028] The LoRA branch allows SAM to update only a small number of parameters during the training of medical images, which not only saves computing and storage overhead, but also reduces the difficulty of model fine-tuning while ensuring segmentation performance. Considering the safety and real-time nature of intraoperative operations, the Prompt encoder module of SAM is removed, and automatic segmentation is completed without relying on any prompts. Segmentation of occluded objects does not rely on any containment prompt information.

[0029] In the specific embodiment, how to effectively segment the occluders on 2D and 3D pelvic images is discussed to provide the preconditions for image restoration and subsequent image registration algorithms. Experiments are set up for detailed analysis to verify the feasibility of the proposed method. However, due to the differences between 2D and 3D images, the same model cannot be used directly. Here, the backbone network SAM is used to adapt images of different dimensions, which are called OS-LMA-2D and OS-LMA-3D respectively.

[0030] See also Figure 2 ,The architecture for occluder segmentation in 2D X-ray pelvic images consists of an image encoder adapted from SAM,and a mask decoder.

[0031] Given a medical X-ray image with an occluder (Masked Image), where H and W are the height and width of the image respectively, and C is the number of channels of the image. The goal of segmentation is to predict the segmentation map (mask) corresponding to its occluder. The image encoder includes at least one transformer block with a LoRA branch, specifically including: the first normalization layer Layer Norm, the self-attention mechanism with a LoRA branch, the second normalization layer Layer Norm, and the multi-layer perceptron MLP (also known as a multi-layer fully connected network). The feature extraction methods include: Step 201: The embedded patches of the two-dimensional X-ray image are aligned with the embedded patches after the first normalization layer and the self-attention mechanism to obtain the first fusion feature.

[0032] Step 202: The first fused feature is aligned with the first fused feature after passing through the second normalization layer and the multi-layer perceptron to obtain an output feature.

[0033] like Figure 3 The self-attention mechanism with LoRA branch is shown, and the input token sequence is defined as , the output token sequence (tokens) is defined as , the first weight matrix of the projection layer , C in is represented as the number of channels of the input token sequence, C out It is represented as the number of channels of the output token sequence. Assume W The update should be gradual, so a low-rank matrix can be used to approximate this gradual update.

[0034] First, freeze the first weight matrix of the projection layer of the self-attention mechanism that was originally pre-trained W， Then add LoRA branches in the projection layer to achieve low-rank approximation. Since the multi-head self-attention mechanism determines the area of ​​attention by cosine similarity, it is reasonable to apply LoRA to the projection layer of Q, K or V to affect the attention score. Specifically, the first LoRA branch and the second LoRA branch are added to the query (Q, proj.Q) and value (V, proj.V) projection layers respectively.

[0035] The LoRA branch consists of two linear layers ,in r is a low rank value, The updated second weight matrix and output of the LoRA branch can be expressed as: ; The self-attention mechanism processing strategy / output projection (Proj.o) is expressed as: ; in, W q , W k , W v The first weight matrix of the Q, K or V projection layer respectively; A q , B q , A v , B v are the trainable parameters of the LoRA branch, softmax() It is an activation function.

[0036] In the two-dimensional architecture, the mask decoder can also be reconstructed. Considering that the segmentation task in this chapter requires two-class segmentation, namely a background class and a foreground class, the mask decoder is reconstructed according to the characteristics of the segmentation task. Figure 2 After reconstruction, the mask decoder includes multiple decoding units, which are composed of upsamplers and two decoding components connected in sequence. The decoding components include nonlinear activation functions RELU, normalization layers BN, and convolution layers Conv connected in sequence. The new mask decoder can be obtained by superimposing the decoding units.

[0037] The occluder mask is decoded using a simple model structure, and the final occluder-aware segmentation map is obtained. The design of this architecture enables OA-LMA-2D to perform occluder segmentation in real time and accurately in a surgical environment, providing an innovative solution for medical image segmentation tasks.

[0038] like Figure 4 The occluder segmentation architecture of 3D CT pelvic images includes an image encoder and a mask decoder. The image encoder includes three modules: a block embedding module (3D Patch Embedding), a position embedding module (3D PositionalEmbedding), and multiple feature extraction components (Attention Block) based on the attention mechanism.

[0039] Taking 3D CT as input, the block embedding module is used to segment the input CT image into small 3D patches and embed each patch into a high-dimensional feature vector. At this time, each patch is represented as an embedding vector of fixed size. The position embedding module is used to add 3D position encoding to the patch embedding to retain spatial position and information. The feature extraction component performs feature extraction and consists of a multi-head self-attention mechanism and a feedforward network. The output contains long-range dependency information and can capture the relationship between different patches in the 3D image. The LoRA branch is added to the attention mechanism of the feature extraction component. The LoRA branch method is consistent with OS-LMA-2D. Specifically, the LoRA branch is added to the V and Q projection layers. More specifically, it includes a three-dimensional normalization layer Layer Norm 3D, a multiplication function MatMul, a Scale operation, a three-dimensional sequential position encoding 3D RelPE, MatMul, a multiplication function Layer Norm, and a feedforward layer Feed-forward.

[0040] The 3D mask decoder includes a transformer block, a transposed convolution block, and a multi-layer perceptron MLP. The transformer block is used to receive features from the encoder and further process the feature information in combination with its own input. As the core component of the decoder, its structure is similar to the 3D Attention Block in the encoder.

[0041] The transposed convolution module is used for the transposed convolution operation to upsample the features, restore the low-resolution features to high-resolution features, and gradually restore the resolution of the original 3D CT image.

[0042] The multi-layer perceptron MLP is used for further processing and feature fusion, and outputs the final occluder segmentation result.

[0043] In the occluder segmentation task, the selection of loss function and evaluation index is crucial to the training and performance evaluation of the model. In order to make the method of the present invention more accurate, a mixed loss of BCE (cross entropy loss) and Dice loss can be used as the loss of the overall model training: ; in, CombinedLoss Expressed as mixed loss, are weight coefficients, BCE(p,y) Expressed as cross entropy loss, DiceLoss(p,y) Denoted as Dice loss.

[0044] In order to intuitively display and visually evaluate the experimental results, Dice Coefficient (DC), Intersection over Union (IoU), and Hausdorff Distanc (HD) are used as evaluation indicators. Among them: DC is used to measure the similarity between the segmentation result and the true label. The closer the value is to 1, the better the segmentation effect.

[0045] ; IoU is used to measure the ratio of the intersection and union of the predicted segmentation area C and the true segmentation area D. The larger the value, the better the segmentation effect of the model.

[0046] ; HD is used to calculate the maximum distance between the predicted boundary and the true boundary, which is used to evaluate boundary matching. The smaller the value, the better the segmentation effect of the model.

[0047] ; In a specific test, Figure 5A , 6A , 7A, 8A, 9A and 10A respectively show the input images of X-ray, and 5B, 6B, 7B, 8B, 9B and 10B respectively show the corresponding segmentation effect images.

[0048] The present invention improves the native SAM model, designs LoRA fine-tuning, and quickly and accurately completes the occluder identification and positioning, providing a good prerequisite for image restoration and subsequent image registration tasks. The 2D and 3D occluder segmentation methods are designed separately, and SAM is improved according to the task characteristics, including improving the image encoder, reconstructing the occluder mask decoder, removing the prompt encoder, etc., which improves the accuracy of SAM in the pelvic 2D / 3D image occluder segmentation task. The introduced LoRA fine-tuning module fine-tunes the larger SAM model, further improving the accuracy and rationality of the pelvic image occluder segmentation, and providing better quality pelvic images for clinical image restoration, surgical registration or doctor diagnosis.

[0049] The occluders can be segmented before or during surgery. To ensure the safety of surgical operations, it is possible to avoid relying on any prompts for segmentation, and real-time segmentation of occluders can be achieved during surgery.

[0050] A second aspect of the present invention provides a system for implementing an image occlusion segmentation method, such as Fig.11 It includes feature extraction module 1 and decoding module 2. The feature extraction module 1 is used to extract features from the input image through a feature extraction component of an image encoder, wherein the feature extraction component includes a self-attention mechanism with a LoRA branch; The decoding module 2 is used to segment the image through the mask decoder and the features to obtain a segmentation map of the occluder.

[0051] The system further comprises a fine-tuning module 3, wherein the fine-tuning module 3 is used to fine-tune the feature extraction component of the feature extraction module based on low-rank adaptation.

[0052] The third aspect of the present invention also provides a device, including an input unit, a processing unit and an output unit, wherein the input unit is used to obtain an input image; the processing unit is used to execute code, and when the processing unit processes the code, it executes the above-mentioned image occlusion segmentation method to obtain a segmentation map; the output unit is used to output the segmentation map of the occlusion.

[0053] The processing unit may be a server, such as a single server, a server cluster, a cloud server, etc., or it may be a terminal device or software for user operation. The terminal may be a smart phone, a tablet computer, a laptop computer, or a desktop computer; it may also be software running on the above devices.

[0054] The present invention provides an occluder segmentation method for 2D and 3D images. By improving SAM according to task characteristics, including improving the image encoder, reconstructing the occluder mask decoder, removing the prompt encoder, etc., the accuracy of SAM in the pelvic 2D / 3D image occluder segmentation task is improved. The introduced LoRA fine-tuning module fine-tunes the larger SAM model, further improving the accuracy and rationality of the pelvic image occluder segmentation, and providing a better quality pelvic image for clinical image restoration, surgical registration or doctor diagnosis. It should be pointed out that the image occluder segmentation method, system and device of the present invention are not limited to the processing of pelvic images.

[0055] The technical solution provided by the present invention is implemented using large model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than ten trillion model parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability. The pre-trained model can be fine-tuned with a small number of samples, so that the large model can be applied to different tasks.

[0056] It should also be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0057] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for image occlusion segmentation based on low-rank adaptive fine-tuning, characterized in that: The following steps are involved: Extracting features from an input image by a feature extraction component of an image encoder, wherein the feature extraction component includes a self-attention mechanism having a LoRA branch, and one or more projection layers of the self-attention mechanism are provided with a LoRA branch; The image is segmented through the mask decoder and features to obtain the segmentation map of the occluded object.

2. The image occluder segmentation method according to claim 1, characterized in that: The Q-projection layer and V-projection layer of the self-attention mechanism are respectively set with LoRA branches.

3. The image occluder segmentation method according to claim 2, characterized in that: The LoRA branch includes two linear layers.

4. The image occluder segmentation method according to claim 3, characterized in that: The second weight matrix and output of the LoRA branch are expressed as: ； in, Represented as the output of the LoRA branch, Expressed as the second weight matrix of the LoRA branch, F is represented as a sequence of tokens of input, W is the first weight matrix of the projection layer, A and B They are the linear layers of the LoRA branch respectively.

5. The image occluder segmentation method according to claim 4, characterized in that: The output projection of the self-attention mechanism is expressed as: ； in, Att(Q,K,V) Denoted as the output projection, W q , W k , W v They are Q , K and V The first weight matrix of the projection layer; A q , B q , A v and B v They are the trainable parameters of the LoRA branch, C out The number of channels represented as the output token sequence.

6. The image occluder segmentation method according to claim 1, characterized in that: The segmentation network of 2D X-ray image occlusions includes an image encoder and a mask decoder. The feature extraction component of the image encoder includes a transformer module. The transformer module includes a first normalization layer, a self-attention mechanism with a LoRA branch, a second normalization layer, and a multi-layer perceptron; The transformer module extracts the features of two-dimensional X-ray images by: The embedded block of the two-dimensional X-ray image is aligned with the embedded block after the first normalization layer and the self-attention mechanism to obtain the first fusion feature; The first fused feature is aligned with the first fused feature after passing through the second normalization layer and the multi-layer perceptron to obtain the output feature.

7. The image occluder segmentation method according to claim 6, characterized in that: The mask decoder includes multiple decoding units, each of which consists of an upsampling layer and two decoding components connected in sequence. The decoding components include a nonlinear activation function, a normalization layer BN, and a convolutional layer connected in sequence.

8. The image occluder segmentation method according to claim 1, characterized in that: The segmentation network of occluded objects in 3D CT images includes an image encoder and a mask decoder. The image encoder consists of a block embedding module, a position embedding module, and multiple feature extraction components based on the attention mechanism. The block embedding module is used to segment the input CT image into small three-dimensional patches and embed each patch into a high-dimensional feature vector to obtain an embedding vector; The position embedding module is used to add the 3D position encoding to the embedding vector to preserve the spatial position and information; The feature extraction component is used for feature extraction and consists of a multi-head self-attention mechanism and a feedforward network; The mask decoder includes a transformer module, a transposed convolution module, and a multi-layer perceptron; The transformer module is used to receive the features from the encoder and further process the feature information in combination with its own input; The transposed convolution module is used for the transposed convolution operation, which upsamples the feature information, restores the low-resolution features to high-resolution features, and gradually restores them to the original resolution; The multi-layer perceptron is used to further process and fuse high-resolution features and output an occlusion segmentation map.

9. An image occlusion segmentation system, characterized in that: Used to implement the image occluder segmentation method according to any one of claims 1 to 8, the system comprises a feature extraction module and a decoding module, The feature extraction module is used to extract features from an input image through a feature extraction component of an image encoder, wherein the feature extraction component includes a self-attention mechanism with a LoRA branch; The decoding module is used to segment the image through the mask decoder and the features to obtain a segmentation map of the occluder.

10. An image occlusion segmentation device, characterized in that: It includes input unit, processing unit and output unit. The input unit is used to obtain an input image; The processing unit is used to execute the code. When the processing unit processes the code, the image occluder segmentation method according to any one of claims 1 to 8 is executed. The output unit is used to output a segmentation map of the occluder.

Images