A progressive boundary region optimization medical image small sample segmentation method

Abstract

The application discloses a medical image small sample segmentation method with progressive boundary region optimization, and utilizes a double-branch feature extraction network and a prototype decoding module based on prototype learning to obtain a rough segmentation mask, introduces a boundary optimization module to enhance the boundary information representation of high-level features, uses a dense comparison module to decode the enhanced features to obtain an edge segmentation mask, and finally uses the edge segmentation mask as a supplement to the rough segmentation mask to correct the region with inconsistent boundary prediction. Through repeated use of the boundary optimization module and the dense comparison module, the rough prediction mask is refined into an accurate and complete segmentation prediction mask, effectively solving the problem of unclear boundary region segmentation caused by complex segmentation target morphology, category prototype bias and insufficient local region attention, and greatly improving the segmentation accuracy of small sample medical images.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision technology, specifically to a small-sample segmentation method for medical images with progressive boundary region optimization. Background Technology

[0002] Few-shot segmentation of medical images is an important branch of few-shot learning. It uses a small number of labeled medical samples to extract category representations, and then uses these representations to predict new medical category samples without retraining the model. This technique allows segmentation models trained on single-category data to be directly applied to segmentation targets of other categories of medical data. The segmentation results provide clinicians with the anatomical structural information needed for disease diagnosis, and have significant application value in assisting clinical diagnosis and treatment. However, due to the small contrast differences between target objects in medical image data, the significant influence of organ motion, and the small size of labeled samples, building high-performance medical segmentation models with small sample sizes remains a challenge.

[0003] To date, numerous small-sample segmentation methods for medical images have been proposed. Based on the granularity of support features, these methods can be divided into two categories: The first category employs an attention mechanism to extract information for segmentation of the query image. It uses squeezing and excitation modules to fuse information from the support images into the query image to guide segmentation branches. Its advantage is the use of a parameterized attention mechanism, which more effectively obtains the importance of each feature map of the support features, while having fewer weight parameters. However, the presence of complex lesion or organ image features in medical images can significantly reduce the accuracy of the model's segmentation. The second category uses class average pooling, averaging support features from different categories as prototypes to classify query pixels. Its advantage is the use of a parameterless prototype metric to measure the distance between feature prototypes and features of different categories. This method can filter out much interference information, has strong interpretability and noise resistance. However, the global category prototype ignores much local information and suffers from prototype bias, meaning the prototypes obtained by this method sometimes cannot effectively guide segmentation.

[0004] While current methods offer various solutions to improve the performance of small-sample segmentation of medical images, the segmentation of medical images still faces challenges such as unclear boundary region segmentation due to factors like low contrast, significant variations in tissue features, and complex shapes of tissues or lesions. This invention proposes a progressive boundary region optimization method for small-sample segmentation of medical images. This method not only focuses on the feature representation of local regions but also enhances the feature representation of the segmentation target boundary regions, significantly improving the segmentation accuracy of boundary regions. Summary of the Invention

[0005] The purpose of this invention is to provide a progressive boundary region optimization method for small-sample segmentation of medical images. It utilizes a feature extraction module and a prototype decoding module to obtain a coarse segmentation mask, then introduces a boundary optimization module to enhance the boundary information representation of high-level features, and finally uses a dense comparison module to decode the enhanced features to obtain a precise segmentation mask. This effectively solves the problem of unclear boundary region segmentation caused by complex target morphology, class prototype bias, and insufficient attention to local regions in existing algorithms.

[0006] To achieve the above objectives, this invention employs the following technical solution: First, a superpixel generation algorithm is used to divide the image into multiple correlated local regions by capturing the local similarity between pixels. Then, a region is randomly selected as a pseudo-label, and the pseudo-label and the original unlabeled data are used as training samples in the few-shot segmentation model training phase. Next, a coarse segmentation mask is obtained using a prototype-based dual-branch feature extraction network. Then, an edge segmentation mask is obtained using a boundary optimization module and a dense comparison module. Finally, the edge segmentation mask is used as a supplement to the coarse segmentation mask to correct regions with inconsistent boundary predictions. By refining the coarse segmentation mask three times, the boundary effect is optimized.

[0007] A progressive boundary region optimization method for small-sample segmentation of medical images includes the following steps:

[0008] Step 1: Preprocessing medical images:

[0009] Step 1.1: For unlabeled images The pseudo-label image is generated using a superpixel algorithm. ;

[0010] Step 1.2: Transfer unlabeled images With pseudo-label images Pairing is performed to obtain support set image pairs ;

[0011] Step 1.3: For support set image pairs Transform to obtain the query set image pair ;

[0012] Step 1.4: Support set image pairs Image pairs with query set Pairing is performed to obtain input data for the training phase. .

[0013] Step 2: Obtain a coarse segmentation mask using a dual-branch feature extraction network:

[0014] Step 2.1: Support for image sets and query set images Input into the feature extraction network to obtain the features of the last three high-level layers. ;

[0015] Step 2.2: High-level features With support set tags Perform masked average pooling to output the prototype feature vector. ;

[0016] Step 2.3: Convert the prototype vector With query set feature map The input is fed into the prototype network to obtain the query set image. coarse prediction mask .

[0017] Step 3: Obtain the edge segmentation mask using the boundary optimization module and the dense comparison module:

[0018] Step 3.1: Apply the coarse prediction mask and high-level characteristics The input is fed into the boundary optimization module to obtain the feature map after boundary enhancement. ;

[0019] Step 3.2: Enhance the feature map after boundary optimization module With prototype vector The input dense comparison module is used for decoding to obtain the boundary supplement map. .

[0020] Step 4: Use the edge segmentation mask as a supplement to the coarse segmentation mask:

[0021] Step 4.1: Complete the boundary map With coarse prediction mask Adding them together yields the segmentation prediction mask. ;

[0022] Step 4.2: High-level characteristics and Perform the same operation as in Step 3 to obtain the boundary supplementary maps. and ;

[0023] Step 4.3: Use the cross-entropy loss function and the Dice loss function to constrain the training of the model.

[0024] This invention leverages the enhanced regional feature representation capabilities of an attention-based boundary optimization module to propose a progressive boundary region optimization method for small-sample medical image segmentation. This method repeatedly utilizes the boundary optimization module and the metric network module to correct inconsistent regions in the prediction results, refining the coarse prediction mask into an accurate and complete segmentation prediction mask. Simultaneously, a parameterless metric network module performs point-to-point dense comparisons between the extracted support set prototype and query set features, filtering out interference information in the image while focusing on the feature representation of local regions. Compared to previous class average pooling methods, this method avoids prototype bias issues and maintains segmentation accuracy even with complex lesion or organ image features. This method significantly improves boundary segmentation accuracy by enhancing the feature representation of the segmentation target's boundary regions. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method of the present invention;

[0026] Figure 2 This is a model diagram of the method of the present invention;

[0027] Figure 3 This is a diagram illustrating the medical image preprocessing procedure used in this invention;

[0028] Figure 4 This is a structural diagram of the prototype network module involved in this invention.

[0029] Figure 5 This is a structural diagram of the boundary optimization module involved in this invention.

[0030] Figure 6 This is a structural diagram of the dense comparison module involved in this invention.

[0031] Figure 7 This is a diagram showing the result of the method of the present invention. Detailed Implementation

[0032] This invention is achieved using the following technical means:

[0033] A progressive boundary region optimization method for small-sample segmentation of medical images is proposed. First, the medical images are preprocessed: pseudo-labeled images are generated using a superpixel algorithm, and then paired with unlabeled data. The resulting data serves as input to the network model. Next, a coarse segmentation mask is obtained through a bi-branch feature extraction network. Then, an edge segmentation mask is obtained using a boundary optimization module and a parameterless metric network module. Finally, the edge segmentation mask is used as a supplement to the coarse segmentation mask to correct regions where boundary predictions are inconsistent.

[0034] The flowchart of the method described in this invention is as follows: Figure 1 As shown, the model diagram is as follows Figure 2As shown, the specific steps include:

[0035] Step 1: Preprocess the medical images. The specific process is shown in the attached document. Figure 3 As shown:

[0036] Step 1.1: For unlabeled images The pseudo-label image is generated using a superpixel algorithm. ;

[0037] For unlabeled images The superpixel algorithm is used to divide the image into multiple correlated local regions, and then a region is randomly selected as the pseudo-label image. .

[0038] Step 1.2: Transfer unlabeled images With pseudo-label images Pairing is performed to obtain support set image pairs ;

[0039] Randomly selected pseudo-label images It is an unlabeled image. The tags are paired to form support set image pairs. .

[0040] Step 1.3: For support set image pairs Transform to obtain the query set image pair ;

[0041] For support set image pairs Support set images With support for pseudo-labeled images Perform the same random geometric transformations and intensity conversions to generate query set images with new shapes and intensities. With query set pseudo-label image The two are paired to form a query set image pair. .

[0042] Step 1.4: Support set image pairs Image pairs with query set Pairing is performed to obtain input data for the training phase. .

[0043] Will support image pairs Image pairs with query set After pairing As training samples during the training phase of the few-shot segmentation model.

[0044] Step 2: Obtain a coarse segmentation mask using a dual-branch feature extraction network:

[0045] Step 2.1: Support for image sets and query set images Input into the feature extraction network to obtain the features of the last three high-level layers. ;

[0046] A support set image and a query set image The input is processed by the ResNet dual-branch feature extraction network, which extracts features through five convolutional layers and outputs the last three high-level feature layers. .

[0047] Step 2.2: High-level features With support set tags Perform masked average pooling to output the prototype feature vector. ;

[0048] Features of the last three high-rise buildings Tags respectively with support set Perform masked average pooling to output the prototype feature vectors of these three layers. The formula for calculating the prototype vector of a certain category through masked average pooling is as follows:

[0049] (1)

[0050] in, It supports feature maps extracted from images. It represents a certain category. The prototype of a category can be obtained through average pooling of the mask. These are the position coordinates of the pixels on the feature map. It is an indicator function that outputs 1 if the parameter is true, and 0 otherwise.

[0051] Step 2.3: Convert the prototype vector With query set feature map The input is fed into the prototype network to obtain the query set image. coarse prediction mask .

[0052] Use the prototype network to obtain the feature map of the query set. A probability graph of vectors belonging to a certain prototype along the channel axis. The prototype network module structure diagram is attached. Figure 4 As shown. Prototype vector Includes foreground and background prototypes, representing the class centers of the foreground and background in high-dimensional space. The prototype vector group is represented as... For one of the prototypes Specifically, the formula for calculating the prototype probability map is as follows:

[0053] (2)

[0054] Among them, query set feature map Indicated as Distance function It is the cosine distance. It is the scaling factor of the distance function. Set it to 20.

[0055] The prediction mask for the corresponding category is a probability map. The part with the highest probability corresponds to The calculation formula is as follows:

[0056] (3)

[0057] Among them, the predicted mask of the query set image It is a coarse prediction mask. .

[0058] Step 3: Obtain the edge segmentation mask using the boundary optimization module and the dense comparison module:

[0059] Step 3.1: Apply the coarse prediction mask and high-level characteristics The input is fed into the boundary optimization module to obtain the feature map after boundary enhancement. ;

[0060] The structure diagram of the boundary optimization module is attached. Figure 5 As shown, the feature representation used to enhance boundary context information has the following calculation formula:

[0061] (4)

[0062] in, To query the high-level features of the image set , For the prediction mask of the previous stage , Features of the query set after boundary enhancement.

[0063] The query set prediction mask is represented as , using maximum operation Obtain the corresponding foreground image Background image and boundary area map The calculation formula is as follows:

[0064] ,

[0065] ,

[0066] (5)

[0067] All pixels of the aggregated region map and the input feature map Based on the information, the feature vectors of the foreground image, background image, and boundary region image are calculated using the following formula:

[0068] , , (6)

[0069] in, Represents pixels in spatial dimensions. Represents the foreground eigenvector. Represents the background feature vector. This represents the boundary feature vector.

[0070] Then, calculate each feature vector. , , With input feature map The similarity between each pixel in the image is calculated using the following formula:

[0071] ,

[0072] ,

[0073] ,

[0074] , , ,

[0075] (7)

[0076] Similarity scoring , , For representing vectors , , We perform a weighted summation to obtain the enhanced feature map. Each pixel The calculation formula is as follows:

[0077] (8)

[0078] in, , , and yes convolution.

[0079] Feature Enhancement Map and input feature map Connect along the channel axis, via Convolution reduces dimensionality and outputs a feature map with enhanced boundaries. That is, feature map .

[0080] Step 3.2: Enhance the feature map after boundary optimization module With prototype vector The input dense comparison module is used for decoding to obtain the boundary supplement map. .

[0081] Decoding is performed using a dense comparison module, the structure of which is shown in the attached diagram. Figure 6 As shown.

[0082] prototype vector Including foreground prototype vector and background prototype vector Both are copied and extended to the query set feature map. The same size, respectively represented as and , and Connect them along the channel axis to generate a new feature map, then use a... Dimensionality reduction is performed using convolution, and the output feature map is obtained. The calculation formula is as follows:

[0083] (9)

[0084] in, It is a concatenation operator.

[0085] New feature map After a The convolution generates a probabilistic map of the query set image, which is then passed through a... The activation function outputs the final prediction mask. The calculation formula is as follows:

[0086] (10)

[0087] Among them, the prediction mask Supplementing the boundary map .

[0088] Step 4: Use the edge segmentation mask as a supplement to the coarse segmentation mask:

[0089] Step 4.1: Complete the boundary map With coarse prediction mask Adding them together yields the segmentation prediction mask. ;

[0090] Use boundary supplementary map To erase and fill in the foreground boundaries, and to replace inaccurate boundary prediction masks. Refined into accurate and complete segmentation prediction masks The calculation formula is as follows:

[0091] (11)

[0092] Step 4.2: High-level characteristics and Perform the same operation as in Step 3 to obtain the boundary supplementary maps. and .

[0093] Will With coarse prediction mask Adding them together yields the segmentation prediction mask. And then With coarse prediction mask Adding them together yields the segmentation prediction mask. , This is the final segmentation prediction mask. The calculation formula is as follows:

[0094] (12)

[0095] (13)

[0096] Step 4.3: Use the cross-entropy loss function and the Dice loss function to constrain the training of the model.

[0097] The weighted sum of the cross-entropy loss function and the Dice loss function constitutes the combined loss function. The calculation formula is as follows: (14)

[0098]

[0099]

[0100] (15)

[0101] (16)

[0102] in, Represents the cross-entropy loss function. For Dice's loss, This is a hyperparameter, set to 0.5. It is the Ground Truth of the label. This refers to the number of pixels in the query set images.

[0103] This concludes the description of the specific implementation process of the present invention.

Claims

1. A progressive boundary region optimization method for small-sample segmentation of medical images, characterized in that, First, the medical images are preprocessed: pseudo-labeled images are generated using a superpixel algorithm, and then paired with unlabeled data. The resulting data is used as input to the network model. Then, a coarse segmentation mask is obtained through a two-branch feature extraction network. Next, an edge segmentation mask is obtained using a boundary optimization module and a parameterless dense comparison module. Finally, the edge segmentation mask is used as a supplement to the coarse segmentation mask to correct regions where the boundary prediction is inconsistent. The edge segmentation mask obtained using the boundary optimization module and the dense comparison module includes: Step 3.1: Apply the coarse prediction mask and high-level characteristics The input is fed into the boundary optimization module to obtain the feature map after boundary enhancement. ; The boundary optimization module is used to enhance the feature representation of boundary context information. The calculation formula for this module is as follows: (4) ， in, To query the high-level features of the image set , For the prediction mask of the previous stage , Features of the query set after boundary enhancement; Step 3.2: Enhance the feature map after boundary optimization module With prototype vector The input dense comparison module is used for decoding to obtain the boundary supplement map. ; Decoding is performed using a dense comparison module; prototype vector Including foreground prototype vector and background prototype vector Both are copied and extended to the query set feature map. The same size, respectively represented as and , and Connect them along the channel axis to generate a new feature map, then use a... Dimensionality reduction is performed using convolution, and the output feature map is obtained. The calculation formula is as follows: (9) ， in, It is a concatenation operator; New feature map After a The convolution generates a probabilistic map of the query set image, which is then passed through a... The activation function outputs the final prediction mask. The calculation formula is as follows: (10) ， Among them, the prediction mask Supplementing the boundary map .

2. The progressive boundary region optimization method for small-sample medical image segmentation according to claim 1, characterized in that, Specifically, the following steps are included: Preprocessing of medical images includes: Step 1.1: For unlabeled images The pseudo-label image is generated using a superpixel algorithm. ; For unlabeled images The superpixel algorithm is used to divide the image into multiple correlated local regions, and then a region is randomly selected as the pseudo-label image. ; Step 1.2: Transfer unlabeled images With pseudo-label images Pairing is performed to obtain support set image pairs ; Randomly selected pseudo-label images It is an unlabeled image. The tags are paired to form support set image pairs. ; Step 1.3: For support set image pairs Transform to obtain the query set image pair ; For support set image pairs Support set images With support for pseudo-labeled images Perform the same random geometric transformations and intensity conversions to generate query set images with new shapes and intensities. With query set pseudo-label image The two are paired to form a query set image pair. ; Step 1.4: Support set image pairs Image pairs with query set Pairing is performed to obtain input data for the training phase. ; Will support image pairs Image pairs with query set After pairing As training samples during the training phase of the few-shot segmentation model.

3. The progressive boundary region optimization method for small-sample medical image segmentation according to claim 1, characterized in that, Obtaining a coarse segmentation mask through a two-branch feature extraction network includes: Step 2.1: Support for image sets and query set images Input into the feature extraction network to obtain the features of the last three high-level layers. ; A support set image and a query set image The input is processed by the ResNet dual-branch feature extraction network, which extracts features through five convolutional layers and outputs the last three high-level feature layers. ; Step 2.2: High-level features With support set tags Perform masked average pooling to output the prototype feature vector. ; Features of the last three high-rise buildings Tags respectively with support set Perform masked average pooling to output the prototype feature vectors of these three layers. The formula for calculating the prototype vector of a certain category through masked average pooling is as follows: (1) ， in, It supports feature maps extracted from images. It represents a certain category. The prototype of a category can be obtained through average pooling of the mask. These are the position coordinates of the pixels on the feature map. It is an indicator function that outputs 1 if the parameter is true, and 0 otherwise. Step 2.3: Convert the prototype vector With query set feature map The input is fed into the prototype network to obtain the query set image. coarse prediction mask ; The feature map of the query set is obtained using the prototype network module. A probability graph of vectors belonging to a certain prototype along the channel axis. ; prototype vector Includes foreground and background prototypes, representing the class centers of the foreground and background in high-dimensional space. The prototype vector group is represented as... For one of the prototypes Specifically, the formula for calculating the prototype probability map is as follows: (2) ， Among them, query set feature map Indicated as Distance function It is the cosine distance. It is the scaling factor of the distance function. Set to 20; The prediction mask for the corresponding category is a probability map. The part with the highest probability corresponds to The calculation formula is as follows: (3) ， Among them, the predicted mask of the query set image It is a coarse prediction mask. .

4. The progressive boundary region optimization method for small-sample medical image segmentation according to claim 1, characterized in that, Using edge segmentation masks as a supplement to coarse segmentation masks includes: Step 4.1: Complete the boundary map With coarse prediction mask Adding them together yields the segmentation prediction mask. ; Use boundary supplementary map To erase and fill in the foreground boundaries, and to replace inaccurate boundary prediction masks. Refined into accurate and complete segmentation prediction masks The calculation formula is as follows: (11) ， Step 4.2: High-level characteristics and Perform the same operation as in Step 3 to obtain the boundary supplementary maps. and ; Will With coarse prediction mask Adding them together yields the segmentation prediction mask. And then With coarse prediction mask Adding them together yields the segmentation prediction mask. , This is the final segmentation prediction mask. The calculation formula is as follows: (12) ， (13) ， Step 4.3: Use the cross-entropy loss function and the Dice loss function to constrain the training of the model.

Images