Medical small target segmentation method and system based on multi-scale feature fusion and two-stage joint learning

Abstract

A method for segmenting small medical targets based on multi-scale feature fusion and two-stage joint learning combines the different purposes and respective advantages of the two stages. Different multi-scale fusion strategies are adopted for the coarse and fine segmentation networks, respectively. A spatial transformation module is introduced to effectively transfer the loss gradient from the fine segmentation network to the coarse segmentation network. Finally, a two-stage joint optimization method is used to adjust the parameters of the coarse and fine segmentation networks. A system for segmenting small medical targets based on multi-scale feature fusion and two-stage joint learning is also provided. This invention effectively solves the problems of sparse pixel information and unclear features faced by small targets in medical images, achieving excellent performance in segmenting small medical targets.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a segmentation method and system for small objects in medical images. Background Art

[0002] Early detection and diagnosis are crucial for serious illnesses, especially when it comes to identifying small lesions and detecting brain tumors. Accurate medical image segmentation and recognition can help doctors detect tiny abnormalities, enabling early treatment and improving treatment outcomes and survival rates. Specifically, there's a close correlation between the identification of small lesions and brain tumor size and survival rates. Many known tumors have a cure rate of up to 90% if detected early. However, neglecting early detection can lead to progressive disease progression, resulting in a series of cascading effects on the body. Currently, most patients are diagnosed in the middle or late stages of the disease, significantly missing the optimal treatment window. Consequently, the cure rate plummets to just 15%. Furthermore, accurate identification and localization of small, critical structures such as blood vessels and neural canals in medical images are crucial for assisting surgical procedures. For example, precise segmentation of the location and course of the mandibular neural canal in oral cone-beam computed tomography (CBCT) images can help dentists better plan surgical procedures before dental implant surgery, avoiding accidental damage to neural pathways during surgery (which is virtually impossible to repair), thereby improving surgical safety.

[0003] Many medical images suffer from an imbalanced distribution of classes. For example, when segmenting the mandibular canal or brain tumor, the target often occupies only a small portion of the entire image, while normal tissue or background occupies the vast majority. Although a number of convolutional neural network-based segmentation methods have been proposed and successfully applied to medical image segmentation tasks, small targets occupy little pixel information, have unclear features, and have blurred boundaries. Traditional convolutional neural networks (CNNs) are typically designed to process medium or large-sized targets, making it difficult for CNNs to capture the details and features of small targets. In particular, during the encoding and decoding process, the edge information of small targets in the image is easily lost, resulting in blurred edge details. In recent years, PraNet (Parallel reverse attention network for polyp segmentation) has improved segmentation accuracy by combining the global features of the parallel partial encoder and decoder and using a reverse attention module to extract boundary information. CaraNet (Context Axial Reverse Attention Network for Segmentation of Small Medical Objects) uses Res2Net (A New Multi-Scale Backbone Architecture) as the backbone network. It introduces the axial attention module and the channel feature pyramid (CFP) module based on PraNet. The axial attention module can extract the global dependency graph and local feature information by calculating the attention values ​​in the horizontal and vertical directions respectively. The channel feature pyramid obtains multi-scale feature information by adding different expansion rates to each channel. The above method further improves the accuracy of segmentation by modeling semantic information at different levels. Although global scale information and multi-level semantic information are crucial to improving the performance of segmentation tasks, how to more effectively model the global context and perform semantic reasoning still requires further research and exploration.

[0004] To address the class imbalance problem in medical images, two-stage segmentation strategies that align with the idea of ​​gradual segmentation in clinical practice often achieve better segmentation accuracy. In clinical practice, doctors first roughly locate the region of interest and then extract the target outline based on detailed features. Two-stage segmentation methods employ a similar strategy, consisting of two stages: candidate box extraction and pixel-level classification. The goal of the candidate box extraction stage is to generate a set of candidate regions that may contain the small targets to be segmented. Common methods include region generation-based methods and deep learning-based methods, which can effectively generate potential target candidate regions and provide input for the next stage of pixel-level classification. In the pixel-level classification stage, each pixel is assigned to a corresponding category. Convolutional neural networks are typically used for pixel-level classification. These networks can learn high-level image features and perform detailed classification, improving the accuracy and robustness of small target segmentation.

[0005] However, in general two-stage methods, the connection between the two stages is not well examined. After generating pixel-by-pixel segmentation results in the coarse stage, only the boundary information of the coarse segmentation results is used to locate the target in the fine stage, resulting in a waste of network resources. For example, Y.Zhou et al. proposed a two-stage segmentation method from coarse to fine (A fixed-point model for pancreas segmentation in abdominal CT scans), in which the coarse segmentation stage provides coarse positioning and the fine adjustment stage obtains precise segmentation. This method has achieved relatively good performance in pancreas segmentation, but there is still a problem of inconsistent training and test results. That is, during training, coarse segmentation and fine adjustment are independent and do not minimize the global ability function, but during testing, it is assumed that coarse segmentation and fine adjustment cooperate with each other in the iterative process. Therefore, sometimes fine adjustment will produce worse segmentation accuracy than coarse segmentation. Summary of the Invention

[0006] In order to overcome the shortcomings of the existing technology, the present invention provides a medical small target segmentation method and system based on multi-scale feature fusion and two-stage joint learning. Combining the different purposes and respective advantages of the two stages, different multi-scale fusion strategies are adopted for the coarse and fine segmentation networks respectively, and a spatial conversion module is introduced to effectively transfer the loss gradient from the fine segmentation network to the coarse segmentation network, so that the target positioning information of the coarse segmentation stage can be better transferred to the fine segmentation stage. Finally, the parameters of the coarse and fine segmentation networks are adjusted in the form of two-stage joint optimization, thereby better solving the problems of scarce pixel information and unclear features faced by small targets in medical images, and achieving excellent medical small target segmentation performance.

[0007] The technical solution adopted by the present invention to solve its technical problem is:

[0008] A method and system for medical small target segmentation based on multi-scale feature fusion and two-stage joint learning, comprising the following steps:

[0009] Step 1: Collect and preprocess the medical images to be segmented. Divide the medical images annotated with target masks into a training set D1 and a validation set D2 according to a preset ratio. Adjust each image in D1 using a multi-scale strategy of {0.75, 1.0, 1.25}.

[0010] Step 2: Use each image X in the training set D1 to train a coarse segmentation network, with the backbone network being Res2Net;

[0011] Step 3: Use the transformation module to connect the coarse segmentation stage and the fine segmentation stage, and shrink the input original image X by cropping;

[0012] Step 4: Use the cropped image X F Train the fine segmentation network, with the backbone network being Res2Net;

[0013] Step 5: Use the joint optimization method to adjust the parameters of the coarse segmentation network and the fine segmentation network simultaneously until the maximum number of iterations is reached, and save the coarse segmentation network parameters, fine segmentation network parameters, and spatial transformation parameters with the best performance on the validation set D2;

[0014] Step 6: Input the medical image X without target mask annotation into the trained coarse segmentation network to obtain the coarse segmentation output probability map Pc;

[0015] Step 7: Through the conversion and cropping module, Pc is converted into the spatial weight Xc of each pixel of the original image X, and the cropping operation is performed with Pc as the reference image r to obtain the cropped image X F ;

[0016] Step 8: X F Send it into the trained fine segmentation network to obtain the segmentation result of small targets.

[0017] The process of step 2 is as follows:

[0018] 2.1 Multi-scale feature extraction, using parameter W C Res2Net extracts features of each level {f i |i=1,2,...,L}, the resolution of each feature map is

[0019] 2.2 Applying partial decoders to aggregate high-level features {f i |i=3,...,L}:

[0020]

[0021] where f i ' is the high-level feature after aggregation. When i = L, f' L =f L , where L = 5, U p (·) is upsampling (magnification is 2 j-i , i∈[3,L-1], j∈[i+1,L]), Conv(·) is a 3×3 convolution, Π represents the concatenation operation, and ⊙ represents the element-by-element multiplication;

[0022] 2.3 For the high-level features of different scales after aggregation {f i '|i=3,...,L}, upsampling-concatenation operations are performed in sequence, and then two convolutional layers are used to obtain the global feature map Fc, which is resized to H×W; the two convolutional layers are 3×3 convolution and 1×1 convolution respectively;

[0023] 2.4 Take Sig(Fc) as the output probability map Pc of coarse segmentation, where Sig(﹒) is the activation function sigmoid.

[0024] The process of step three is as follows:

[0025] 3.1 The conversion module converts the coarse segmentation prediction probability Pc into the spatial weight of each pixel of the original image X:

[0026] X c =X⊙g(P c ;η) (2)

[0027] Where ⊙ is the element-by-element multiplication, g(P c ; η) is a convolution operation that maintains the same size of input and output. The spatial weight of each output point is determined by the coarse segmentation prediction probability Pc in its small neighborhood, and its parameter is η;

[0028] 3.2 In the early stage of training, the gold standard is used as the reference image r, and in the later stage, the coarse segmentation probability map P is used. c As the reference image r;

[0029] 3.3 Binarize the reference image r into image Z, find all foreground pixels in image Z, and calculate the minimum 2D bounding box containing these pixels;

[0030] 3.4 Increase the size of the bounding box by K pixels in all four directions to obtain an enlarged rectangular box, and use it to crop the image X c If the enlarged rectangle exceeds the image X c The missing values ​​are filled with 0, and the cropped image is recorded as X F .

[0031] The process of step 4 is as follows:

[0032] 4.1 Multi-scale feature extraction, using parameter W F Res2Net, extract image X F Features at all levels {f i |i=1,2,...,L};

[0033] 4.2 Through the background denoising module, the deep features with larger receptive fields are used to assist the shallow layers in removing background noise, resulting in a feature map with a relatively clear background:

[0034] f i '=σ(U p (f′ i+1 ))⊙f i ,i∈{L-1,L-2} (3)

[0035] Among them U p (·) represents the upsampling operation, σ(·) is the activation function ReLU, ⊙ is the element-wise multiplication, and the L-th layer feature f′ L =f L , where L = 5, the feature map f′4 of other layers gets the guidance information of f′5, and f′3 gets the guidance information of f′4;

[0036] 4.3 The three feature maps of different scales {f′3, f′4, f′5} are fused through the splicing module and upsampled using bilinear interpolation to restore the fusion result to the size H×W of the original input image;

[0037] 4.4 After Sig(﹒), it is used as the output probability map of fine segmentation, where Sig(﹒) is the activation function sigmoid.

[0038] In step 5, the loss function of the joint optimization is as follows:

[0039] Loss total =αLoss(P c ,Y)+(1-α)Loss(P F ,Y) (4)

[0040] Where α is the weight coefficient of the coarse segmentation loss, P c and P F are the predicted probabilities of coarse segmentation and fine segmentation respectively, Y is the gold standard, and Loss(·) adds focus loss F-Loss on the basis of binary cross entropy BCE loss and IoU loss:

[0041] Loss(P,Y)=λ1(BCE(P,Y)+IoU(P,Y))+λ2F_Loss(P,Y) (5)

[0042]

[0043]

[0044]

[0045]

[0046] Where P is the predicted probability of the input image, p i ∈P and y i ∈Y are the predicted probability and label of image pixel i, N is the number of pixels in the image, λ1 and λ2 are the weight coefficients of the loss term, both set to 1.

[0047] A medical small target segmentation system based on multi-scale feature fusion and two-stage joint learning. The preprocessing module is used to collect and preprocess the medical images to be segmented. The medical images annotated with target masks are divided into a training set D1 and a validation set D2 according to a preset ratio. Each image in D1 is adjusted according to the multi-scale strategy of {0.75, 1.0, 1.25}.

[0048] The coarse segmentation training module is used to train the coarse segmentation network using each image X in the training set D1, with the backbone network being Res2Net;

[0049] The conversion training and cropping module uses the conversion module to link the coarse segmentation stage and the fine segmentation stage, and shrinks the input original image X by cropping;

[0050] Fine segmentation training module for using the cropped image X F Train the fine segmentation network, with the backbone network being Res2Net;

[0051] The joint optimization module is used to adjust the parameters of the coarse segmentation network and the fine segmentation network simultaneously until the maximum number of iterations is reached using the joint optimization form, and save the coarse segmentation network parameters, fine segmentation network parameters, and spatial transformation parameters with the best performance on the validation set D2;

[0052] The coarse segmentation module is used to input the medical image X without target mask annotation into the trained coarse segmentation network to obtain the coarse segmentation output probability map Pc;

[0053] The conversion and cropping module is used to convert Pc into the spatial weight Xc of each pixel of the original image X, and perform the cropping operation with Pc as the reference image r to obtain the cropped image X F ;

[0054] Fine segmentation module, used to convert X F Send it into the trained fine segmentation network to obtain the segmentation result of small targets.

[0055] The above modules correspond to the contents of step 1 to step 8 of the method of the present invention respectively.

[0056] The technical concept of the present invention is: by applying a two-stage segmentation strategy, in the coarse segmentation stage, with the help of convolutional neural networks and multi-scale fusion strategies, multi-scale feature fusion is performed with the deep layer with stronger semantic expression ability as the main driver to achieve efficient target positioning. A conversion module is introduced to associate the coarse segmentation stage and the fine segmentation stage, and the output result of the coarse segmentation is transferred to the fine segmentation stage with the spatial weight of each pixel. Then, the input original image is further shrunk by the cropping function. In the fine segmentation stage, the deep layer is used to assist the shallower layer background denoising, and then the denoised multi-scale features are fused to achieve pixel classification, thereby effectively realizing the segmentation of small medical targets.

[0057] The medical small target segmentation system of the present invention consists of preprocessing, coarse segmentation, conversion and cropping, and fine segmentation. The preprocessing unit performs preprocessing such as resizing on the input medical image. The coarse segmentation unit inputs the preprocessed medical image into a convolutional neural network (Res2Net) for feature extraction, and uses a partial decoder to aggregate only high-level features to achieve efficient and accurate target positioning. The conversion and cropping unit first converts the coarse segmentation prediction probability into a spatial weight for each pixel of the original image to better utilize the coarse-scale segmentation results and ensure consistency between the training and testing stages; then the coarse segmentation probability map is used as a reference image, and the input original image is further shrunk by a cropping operation. The fine segmentation unit uses another Res2Net to extract features of the input image, and introduces a background denoising module, using deep features with a larger receptive field to assist the shallow layer in removing background noise. The three feature maps of different sizes are then fused to finally obtain an accurate segmentation result. This method effectively solves the problem that small targets in medical images occupy little pixel information, have unclear features, and are easily affected by complex and changing background interference.

[0058] The beneficial effects of the present invention are as follows: the present invention solves the problem of poor performance in the segmentation of small medical targets, and uses a two-stage segmentation method to effectively generate potential target candidate areas in the coarse segmentation stage to provide input for the pixel-level classification in the next stage. At the same time, a multi-scale fusion strategy is added to perform multi-scale feature fusion with the deep layer having stronger semantic expression ability as the main driver to achieve efficient target positioning. A conversion module is introduced to associate the coarse segmentation stage with the fine segmentation stage, so that the target positioning information of the coarse segmentation stage can be better passed to the fine segmentation stage. In the fine segmentation stage, a convolutional neural network is used for pixel-level classification. The network can learn the high-level features of the image and adopt another multi-scale fusion strategy to use the deep layer to assist the shallower background denoising, and then fuse the denoised multi-scale features to achieve accurate pixel classification. It alleviates the problem of low segmentation accuracy caused by the small pixels occupying the image, unclear features, and susceptibility to complex and changeable background interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is the algorithm block diagram of the present invention.

[0060] Figure 2 Flowchart of the present invention.

[0061] Figure 3 Comparison of Dice values ​​for mandibular nerve canal segmentation performance indicators of different size ratios DETAILED DESCRIPTION

[0062] The present invention will be further described below with reference to the accompanying drawings.

[0063] Reference Figures 1 to 3 , a medical small target segmentation method based on multi-scale feature fusion and two-stage joint learning, taking the segmentation of extremely small objects such as the mandibular nerve canal as an example, Figure 3 The method of the present invention is compared with the CaraNet method in the segmentation performance of the mandibular nerve canal at different scales. The method includes the following steps:

[0064] Step 1: Collect and preprocess the medical images to be segmented. After preprocessing several oral cone beam CT (CBCT) images according to the optimal window width and window position, all coronal images are symmetrically cut along the vertical direction (i.e., the y-axis) to form two images of equal size. The images are uniformly resized to 352×352. The foreground and background regions of the mandibular nerve canal are annotated. The coronal image data with the target mask annotated is divided into a training set D1 and a validation set D2 according to a preset ratio. Each image in D1 is resized according to the multi-scale strategy of {0.75, 1.0, 1.25}.

[0065] Step 2: Use each image X in the training set D1 to train a coarse segmentation network, with the backbone network being Res2Net. The process is as follows:

[0066] 2.1 Multi-scale feature extraction, using parameter W C Res2Net extracts features of each level {f i |i=1,2,...,L}, the resolution of each feature map is

[0067] 2.2 Applying partial decoders to aggregate high-level features {f i |i=3,...,L}:

[0068]

[0069] where f i ' is the high-level feature after aggregation. When i = L, f' L =f L , where L = 5, U p (·) is upsampling (the ratio is 2j-i, i∈[3,L-1], j∈[i+1,L]), Conv(·) is a 3×3 convolution, Π represents the concatenation operation, and ⊙ represents the element-by-element multiplication;

[0070] 2.3 For the high-level features of different scales after aggregation {f i '|i=3,...,L}, upsampling-concatenation operations are performed in sequence, and then two convolutional layers are used to obtain the global feature map Fc, which is resized to H×W; the two convolutional layers are 3×3 convolution and 1×1 convolution respectively;

[0071] 2.4 Take Sig(Fc) as the output probability map Pc of the coarse segmentation, where Sig(﹒) is the activation function sigmoid;

[0072] Step 3: Use the transformation module to link the coarse segmentation stage and the fine segmentation stage, and shrink the input original image X by cropping. The process is as follows:

[0073] 3.1 The conversion module converts the coarse segmentation prediction probability Pc into the spatial weight of each pixel of the original image X:

[0074] X c =X⊙g(P c ;η) (2)

[0075] Where ⊙ is the element-by-element multiplication, g(P c ; η) is a convolution operation that maintains the same size of input and output. The spatial weight of each output point is determined by the coarse segmentation prediction probability Pc in its small neighborhood, and its parameter is η;

[0076] 3.2 In the early stage of training, the gold standard is used as the reference image r, and in the later stage, the coarse segmentation probability map P is used. c As a reference image r to complete the coordination of the coarse and fine segmentation networks;

[0077] 3.3 Binarize the reference image r into image Z, find all foreground pixels in image Z, and calculate the minimum 2D bounding box containing these pixels;

[0078] 3.4 Increase the size of the bounding box by K pixels in all four directions to obtain an enlarged rectangular box, and use it to crop the image X c , where K is set to 20, if the enlarged rectangle exceeds the image X c The missing values ​​are filled with 0, and the cropped image is recorded as X F ;

[0079] Step 4: Use the cropped image X F Training the fine segmentation network, with the backbone network being Res2Net, the process is as follows:

[0080] 4.1 Multi-scale feature extraction, using parameter W F Res2Net, extract image X F Features at all levels {f i |i=1,2,...,L};

[0081] 4.2 Through the background denoising module, the deep features with larger receptive fields are used to assist the shallow layers in removing background noise, resulting in a feature map with a relatively clear background:

[0082] f i '=σ(U p (f′ i+1 ))Πf i ,i∈{L-1,L-2} (3)

[0083] Among them U p (·) represents the upsampling operation, σ(·) is the activation function ReLU, Π is the element-wise multiplication, and the L-th layer feature f′ L =f L , where L = 5, the feature map f′4 of other layers gets the guidance information of f′5, and f′3 gets the guidance information of f′4;

[0084] 4.3 The three feature maps of different scales {f′3, f′4, f′5} are fused through the splicing module and upsampled using bilinear interpolation to restore the fusion result to the size of the original input image (H×W);

[0085] 4.4 After Sig(﹒), it is used as the output probability map of fine segmentation, where Sig(﹒) is the activation function sigmoid;

[0086] Step 5: Using the joint optimization method, adjust the parameters of the coarse segmentation network and the fine segmentation network simultaneously until the maximum number of iterations is reached. Save the coarse segmentation network parameters, fine segmentation network parameters, and spatial transformation parameters with the best performance on the validation set D2. The loss function of the joint optimization is as follows:

[0087] Loss total =αLoss(P c ,Y)+(1-α)Loss(P F ,Y) (4)

[0088] Where α is the weight coefficient of the coarse segmentation loss, set to P c and P F are the predicted probabilities of coarse segmentation and fine segmentation respectively, Y is the gold standard, and Loss(·) adds focus loss F-Loss on the basis of binary cross entropy BCE loss and IoU loss:

[0089] Loss(P,Y)=λ1(BCE(P,Y)+IoU(P,Y))+λ2F_Loss(P,Y) (5)

[0090]

[0091]

[0092]

[0093]

[0094] Where P is the predicted probability of the input image, p i ∈P and y i ∈Y are the predicted probability and label of image pixel i, N is the number of pixels in the image, λ1 and λ2 are the weight coefficients of the loss term, both set to 1;

[0095] Step 6: Input the medical image X without target mask annotation into the trained coarse segmentation network to obtain the coarse segmentation output probability map Pc;

[0096] Step 7: Through the conversion and cropping module, Pc is converted into the spatial weight Xc of each pixel of the original image X, and the cropping operation is performed with Pc as the reference image r to obtain the cropped image X F ;

[0097] Step 8: X FSend it into the trained fine segmentation network to obtain the segmentation result of small targets.

[0098] This embodiment also provides a medical small object segmentation system based on multi-scale feature fusion and two-stage joint learning, including a preprocessing module, a coarse segmentation training module, a transformation training and cropping module, a fine segmentation training module, a joint optimization module, a coarse segmentation module, a transformation and cropping module, and a fine segmentation module. Each of these modules corresponds to steps 1 to 8 of the method of the present invention, respectively.

[0099] As described above, the specific implementation steps of this patent make the present invention clearer. Any modifications and changes made to the present invention within the spirit of the present invention and the scope of protection of the claims fall within the scope of protection of the present invention.

Claims

1. A small medical target segmentation method based on multi-scale feature fusion and two-stage joint learning, characterized by: The method comprises the following steps: Step 1: Collect and preprocess the medical images to be segmented. Divide the medical images annotated with target masks into a training set D1 and a validation set D2 according to a preset ratio. Adjust each image in D1 using a multi-scale strategy of {0.75, 1.0, 1.25}. Step 2: Use each image X in the training set D1 to train a coarse segmentation network, with the backbone network being Res2Net; Step 3: Use the transformation module to connect the coarse segmentation stage and the fine segmentation stage, and shrink the input original image X by cropping; Step 4: Use the cropped image X F Train the fine segmentation network, with the backbone network being Res2Net; Step 5: Use the joint optimization method to adjust the parameters of the coarse segmentation network and the fine segmentation network simultaneously until the maximum number of iterations is reached, and save the coarse segmentation network parameters, fine segmentation network parameters, and spatial transformation parameters with the best performance on the validation set D2; Step 6: Input the medical image X without target mask annotation into the trained coarse segmentation network to obtain the coarse segmentation output probability map Pc; Step 7: Through the conversion and cropping module, Pc is converted into the spatial weight Xc of each pixel of the original image X, and the cropping operation is performed with Pc as the reference image r to obtain the cropped image X F ; Step 8: X F Send it to the trained fine segmentation network to obtain the segmentation result of small targets; The process of step three is as follows: 3.1 The conversion module converts the coarse segmentation prediction probability Pc into the spatial weight of each pixel of the original image X: X c =X⊙g(P c (h) (2) Where ⊙ is the element-by-element multiplication, g(P c ; η) is a convolution operation that maintains the same size of input and output. The spatial weight of each output point is determined by the coarse segmentation prediction probability Pc in its small neighborhood, and its parameter is η; 3.2 In the early stage of training, the gold standard is used as the reference image r, and in the later stage, the coarse segmentation probability map P is used. c As the reference image r; 3.3 Binarize the reference image r into image Z, find all foreground pixels in image Z, and calculate the minimum 2D bounding box containing these pixels; 3.4 Increase the size of the bounding box by K pixels in all four directions to obtain an enlarged rectangular box, and use it to crop the image X c If the enlarged rectangle exceeds the image X c The missing values ​​are filled with 0, and the cropped image is recorded as X F .

2. The method for segmenting small medical objects based on multi-scale feature fusion and two-stage joint learning according to claim 1, wherein the process of step 2 is as follows: 2.1 Multi-scale feature extraction, using parameter W C Res2Net extracts features of each level {f i |i=1,2,...,L}, the size of the input image X is H×W, and the resolution of each feature map is 2.2 Applying partial decoders to aggregate high-level features {f i |i=3,...,L}: where f i ' is the high-level feature after aggregation. When i = L, f' L =f L , where L = 5, U p (·) is upsampling, the magnification is 2 j-i , i∈[3,L-1], j∈[i+1,L], Conv(·) is a 3×3 convolution, ∏ represents the concatenation operation, and ⊙ represents the element-by-element multiplication; 2.3 For the high-level features of different scales after aggregation {f i '|i=3,...,L}, upsampling-concatenation operations are performed in sequence, and then two convolutional layers are used to obtain the global feature map Fc, which is resized to H×W; the two convolutional layers are 3×3 convolution and 1×1 convolution respectively; 2.4 Take Sig(Fc) as the output probability map Pc of coarse segmentation, where Sig(﹒) is the activation function sigmoid.

3. The medical small target segmentation method based on multi-scale feature fusion and two-stage joint learning according to claim 1, characterized in that: The process of step 4 is as follows: 4.1 Multi-scale feature extraction, using parameter W F Res2Net, extract image X F Features at all levels {f i |i=1,2,...,L}; 4.2 Through the background denoising module, the deep features with larger receptive fields are used to assist the shallow layers in removing background noise, resulting in a feature map with a relatively clear background: f i '=σ(U p (f′ i+1 ))⊙f i ,i∈{L-1,L-2} (3) Among them U p (·) represents the upsampling operation, σ(·) is the activation function ReLU, ⊙ is the element-wise multiplication, and the L-th layer feature f L '=f L , where L = 5, the feature map f4' of other layers gets the guidance information of f5', and f3' gets the guidance information of f4'; 4.3 The three feature maps of different scales {f3', f4', f5'} are fused through the splicing module and upsampled using bilinear interpolation to restore the fusion result to the size H×W of the original input image; 4.4 After Sig(﹒), it is used as the output probability map of fine segmentation, where Sig(﹒) is the activation function sigmoid.

4. The medical small target segmentation method based on multi-scale feature fusion and two-stage joint learning according to claim 3, characterized in that: In step 5, the loss function of the joint optimization is as follows: Loss total =αLoss(P c ,Y)+(1-α)Loss(P F ,Y) (4) Where α is the weight coefficient of the coarse segmentation loss, P c and P F are the predicted probabilities of coarse segmentation and fine segmentation respectively, Y is the gold standard, and Loss(·) adds focus loss F-Loss on the basis of binary cross entropy BCE loss and IoU loss: Loss(P,Y)=λ1(BCE(P,Y)+IoU(P,Y))+λ2F_Loss(P,Y) (5) Where P is the predicted probability of the input image, p i ∈P and y i ∈Y are the predicted probability and label of image pixel i, N is the number of pixels in the image, λ1 and λ2 are the weight coefficients of the loss term, both set to 1.

5. A system for implementing the medical small target segmentation method based on multi-scale feature fusion and two-stage joint learning as claimed in claim 1, characterized in that: The system comprises: The preprocessing module is used to collect and preprocess the medical images to be segmented. The medical images annotated with target masks are divided into a training set D1 and a validation set D2 according to a preset ratio, and each image in D1 is adjusted according to the multi-scale strategy of {0.75, 1.0, 1.25}; The coarse segmentation training module is used to train the coarse segmentation network using each image X in the training set D1, with the backbone network being Res2Net; The conversion training and cropping module uses the conversion module to link the coarse segmentation stage and the fine segmentation stage, and shrinks the input original image X by cropping; Fine segmentation training module for using the cropped image X F Train the fine segmentation network, with the backbone network being Res2Net; The joint optimization module is used to adjust the parameters of the coarse segmentation network and the fine segmentation network simultaneously until the maximum number of iterations is reached using the joint optimization form, and save the coarse segmentation network parameters, fine segmentation network parameters, and spatial transformation parameters with the best performance on the validation set D2; The coarse segmentation module is used to input the medical image X without target mask annotation into the trained coarse segmentation network to obtain the coarse segmentation output probability map Pc; The conversion and cropping module is used to convert Pc into the spatial weight Xc of each pixel of the original image X, and perform the cropping operation with Pc as the reference image r to obtain the cropped image X F ; Fine segmentation module, used to convert X F Send it into the trained fine segmentation network to obtain the segmentation result of small targets.

Images