Semi-supervised medical image segmentation method based on suspicious pixel mutual correction

Abstract

The invention discloses a semi-supervised medical image segmentation method based on suspicious pixel mutual correction. The method comprises the following steps: 1, preprocessing and dividing a data set; 2, a semi-supervised medical image segmentation network model SPMC is constructed and comprises a network A, a network B, a CDPS module and an SPLAT module, and the network A and the network B adopt the same encoder and decoders with different up-sampling modes; the CDPS screens annotation data prediction results output by the network A and the network B, identifies common trapping pixels which are difficult to predict accurately, and corrects the common trapping pixels by referring to existing tags; the SPLAT screens unlabeled data prediction results output by the network A and the network B, performs pseudo supervision on suspicious pixels in the network B by using credible pixels generated by the network A, calculates total loss and updates parameters; 3, training a network model SPMC; 4, evaluating SPMC performance and optimizing parameters; and 5, inputting a medical image to be segmented into the trained SPMC, and outputting a segmentation result. According to the invention, the prediction accuracy of complex and fuzzy difficult areas in the image is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular is a semi-supervised medical image segmentation method based on mutual correction of suspicious pixels. Background Art

[0002] Medical image segmentation aims to accurately identify and separate anatomical structures such as organs and tumors in the original image, provide reliable auxiliary information for clinical diagnosis and treatment, and help doctors diagnose diseases more accurately, plan treatment plans, and monitor treatment effects. With the continuous advancement of deep learning technology, a variety of network structures based on convolutional neural networks have been proposed one after another, and have achieved remarkable results in the field of automated image segmentation. However, image segmentation methods based on fully supervised learning rely on a large amount of labeled data. Due to the particularity of medical images, public data sets are relatively scarce, and the labeling of medical images requires doctors with professional knowledge. The labeling process is time-consuming and labor-intensive, and it is difficult to promote on a large scale.

[0003] In contrast, semi-supervised learning can train models based on a small amount of labeled data and a large amount of unlabeled data, achieving segmentation results comparable to or even better than fully supervised learning. The semi-supervised learning method not only significantly reduces the dependence on a large amount of labeled data and reduces the labeling burden on doctors, but can also efficiently solve complex clinical segmentation tasks and promote the further development of medical image analysis technology. The application of semi-supervised learning in medical image segmentation has laid a solid technical foundation for precision medicine and intelligent diagnosis, and provided new research directions and solutions.

[0004] Common semi-supervised learning methods can be divided into five categories: generative adversarial methods, contrastive learning, consistency regularization, pseudo-labeling methods, and hybrid methods. The consistency regularization and pseudo-labeling methods are the most widely used. Consistency regularization is based on the smoothness assumption. The core idea is that the model should produce similar results when predicting different perspectives of the same image. According to the way of introducing disturbances, consistency regularization can be divided into data perturbation, feature perturbation, and model perturbation. Data perturbation increases diversity by transforming the input image. For example, Yun et al. proposed to enhance the diversity of training samples by mixing different images, but this data augmentation method is only effective in labeled or unlabeled data. Internal operations ignore the problem of mismatch in empirical distributions between the two; feature perturbation introduces transformations on the feature maps of the segmentation network. For example, Ouali et al. add noise or perform random occlusion in the feature space to enable the main decoder and the auxiliary decoder to generate consistent predictions under different perturbations. However, this method relies on multiple auxiliary decoders, which increases the complexity of the model; model perturbation is achieved by transforming the network structure or parameters. For example, Chen et al. train two networks with the same structure but different initialization parameters to force them to generate consistent predictions for the same image. However, this method does not fully consider the potential impact of noise on model performance.

[0005] The pseudo-labeling method is based on the clustering hypothesis. Its core idea is to first train the model with labeled data, then use the trained model to generate pseudo-labels for unlabeled data, expand the data set and retrain the model. Since the quality of pseudo-labels is crucial to the training of semi-supervised learning, Lee et al. proposed to expand the data set by only selecting predictions with higher confidence as pseudo-labels for unlabeled data. In order to further simplify the model and improve the generalization ability, Yang et al. selectively used reliable images with higher overall prediction stability for retraining. In addition, there are some methods that combine pseudo-labels with consistency regularization. For example, Sohn et al. encourage the model's prediction results on strongly enhanced data to be consistent with the high-confidence pseudo-labels generated by weakly enhanced data. In semi-supervised learning, the uncertainty analysis of the model helps to make full use of the information in the pseudo-labels. The commonly used uncertainty estimation method is Monte Carlo Dropout. This method estimates uncertainty by applying Dropout multiple times in the inference stage and using the variance of the generated prediction results. For example, Yu et al. estimate uncertainty through Monte Carlo Dropout and only use pixels with lower uncertainty for model training to improve performance. Although these methods can improve the quality of pseudo-labels by using pixels with higher confidence or lower uncertainty, pixels with lower confidence or higher uncertainty still contain a lot of valuable information, and direct filtering may lead to waste of information.

[0006] Due to the complex anatomical structure of medical images and the blurred boundaries of tissues and organs, they are easily affected by artifacts during the imaging process. Therefore, there are significant differences in the difficulty of prediction in different areas of the image. If the areas where prediction is more difficult are not processed separately, the prediction accuracy of the difficult areas will be seriously affected. Therefore, mainstream semi-supervised learning methods face two major challenges when processing complex medical images: first, the complexity differences between different areas within the image are usually ignored; second, there is a lack of a mechanism to separately process challenging areas in the image. Summary of the invention

[0007] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a semi-supervised medical image segmentation method based on mutual correction of suspicious pixels, which improves the prediction accuracy of complex and fuzzy difficult areas in the image.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] A semi-supervised medical image segmentation method based on mutual correction of suspicious pixels comprises the following steps:

[0010] Step 1: Crop the medical images in the medical image datasets from different sources and divide them into training sets, test sets and validation sets;

[0011] Step 2: Construct a semi-supervised medical image segmentation network model SPMC;

[0012] The network model SPMC includes network A, network B, CDPS module and SPLAT module. Network A and network B are both Vnet or Unet networks. Network A and network B use the same encoder E. Network A and network B use decoders D with different upsampling methods. a and D b ;

[0013] Step 3: Train the network model SPMC

[0014] Step 3.1, initialize the parameters of the network model SPMC;

[0015] Step 3.2: Input the training set into network A and network B respectively, and output the corresponding feature maps through encoder E. The feature map output by encoder E of network A is directly input into decoder D. a , output the prediction results of the labeled data And the prediction results of unlabeled data Add random noise perturbation to the feature map F output by the encoder E of network B to obtain a feature map with perturbation The feature map containing the perturbation Input decoder D b, output the prediction results of the labeled data And the prediction results of unlabeled data

[0016] Step 3.3: Screening using the CDPS module and Identify the common pixels that are difficult for both network A and network B to accurately predict, and correct the common pixels with reference to the existing label Y; use the SPLAT module to filter and Use the trusted pixels generated by network A to pseudo-supervise the suspicious pixels in network B, thereby improving the utilization efficiency of unlabeled data and the generalization ability of the model;

[0017] Step 3.4: Calculate the total loss

[0018] Step 3.5, update the parameters of the network model;

[0019] Step 4: Use the validation set to evaluate the performance of the network model SPMC and optimize the parameters;

[0020] Step 5: Input the medical image to be segmented into the trained network model SPMC and output the segmentation result.

[0021] Furthermore, the medical image datasets from different sources in step 1 include a left atrium dataset LA, a brain tumor segmentation dataset BraTS, and an automatic heart challenge dataset ACDC.

[0022] Furthermore, in step 2, when the medical image dataset is LA and BraTS, network A and network B are both Vnet networks; when the medical image dataset is ACDC, network A and network B are both Unet networks.

[0023] Furthermore, the parameters of the network model SPMC are initialized in step 3.1, including: selecting the SGD optimizer to train the network model SPMC, setting the weight decay factor to 10 -4 , the momentum is set to 0.9, and the initial learning rate is set to 10 -2 , the number of iterations is set to 30k;

[0024] When the medical image dataset is LA, the batch size is set to 8; when the medical image dataset is BraTS, the batch size is set to 4; when the medical image dataset is ACDC, the batch size is set to 16.

[0025] Furthermore, in step 3.3, the CDPS module is used to screen and The specific process of identifying the common pixels that are difficult for both network A and network B to accurately predict and correcting the common pixels with reference to the existing label Y is as follows:

[0026] Step 3.3.1.1. Calculation and , use the threshold τ to filter out the reliable pixels with high confidence, retain the suspicious pixels with low confidence, and generate the corresponding mask and Respectively expressed as:

[0027]

[0028] In the formula, is the indicator function;

[0029] Step 3.3.1.2, take and The intersection of network A and network B is obtained by It is expressed as:

[0030]

[0031] Step 3.3.1.3, use Screening and Co-trapped pixels and It is expressed as:

[0032]

[0033] In the formula, ⊙ indicates that the product operation is performed one by one on the elements at the corresponding positions;

[0034] Step 3.3.1.4: In order to obtain the correct supervision signal, use Screen out the pixels with the label Y and The corresponding label Y cdp , expressed as:

[0035]

[0036] Step 3.3.1.5: Use mean square error loss Common Pixels and Make corrections so that network A and network B focus on labeled data and The unpredictable areas in the , and provide correct guidance, are expressed as:

[0037]

[0038] Where MSE() is the mean square error loss function.

[0039] Furthermore, in step 3.3, the SPLAT module is used to screen and The process of using the trusted pixels in network A to pseudo-supervise the suspicious pixels in network B is:

[0040] Step 3.3.2.1: Use the probability sharpening function to and Convert into soft pseudo labels SPL respectively a and SPL b ,make and The probability distribution difference is more obvious, and the probability sharpening function is expressed as:

[0041]

[0042] Where T is the hyperparameter that controls the output distribution of the model, and P represents or

[0043] Step 3.3.2.2: Use threshold τ to filter out and Trusted pixels and soft pseudo labels SPL in a and SPL b The suspicious pixels in , get different masks, take the intersection of different masks, and get and It is expressed as:

[0044]

[0045] Step 3.3.2.3. Use corrected pseudo-supervision loss For unlabeled data and The suspicious pixels are used for guidance, which can be expressed as:

[0046]

[0047] In the formula, MSE() is the mean square error loss function, and ⊙ represents the product operation of the corresponding position elements one by one.

[0048] Furthermore, the value of T in step 3.3.2.1 is 0.1.

[0049] Furthermore, when the medical image dataset is LA and ACDC, the value of τ is 0.7; when the medical image dataset is BraTS, the value of τ is 0.65.

[0050] Furthermore, the total loss in step 3.4 is It is expressed as:

[0051]

[0052] In the formula, and Respectively expressed as:

[0053]

[0054] In the formula, is the cross entropy loss, For the loss of Days, is the mean square error loss, and They are and The corresponding modified pseudo-supervision loss, represents the supervision loss, represents the unsupervised loss, β is a hyperparameter, λ is a time-varying Gaussian warm-up function, PL b and PL a are the generated pseudo labels, respectively expressed as:

[0055]

[0056] The λ is expressed as:

[0057]

[0058] In the formula, t represents the current iteration number, t max Indicates the maximum number of iterations.

[0059] Furthermore, the value of β is 0.5.

[0060] Compared with the prior art, the present invention has the following technical effects:

[0061] The present invention introduces a regional complexity evaluation and screening mechanism to enable the network to focus on challenging image regions, thereby improving the overall performance of the model in image segmentation tasks. Specifically, on the one hand, the CDPS strategy is introduced to solve the problem of inaccurate predictions caused by insufficient model learning of challenging regions. Unlike the mainstream method of treating all pixels in the image equally, CDPS extracts suspicious pixels with high uncertainty in the prediction results of the two networks separately, and screens out suspicious pixels that exist in both prediction results, namely, co-encumbered pixels, to guide the network to focus on these co-encumbered pixels and correct them, thereby improving the accuracy of the model's predictions for complex and fuzzy regions; On the other hand, the SPLAT strategy aims to address the potential impact of noise in pseudo-labels. Different from the mainstream method that only uses credible pixels with low uncertainty as pseudo-labels for pseudo-supervised learning, SPLAT also mines information on suspicious pixels with high uncertainty. In order to make full use of these suspicious pixels, SPLAT uses credible pixels in the prediction of network A to perform selective pseudo-supervised learning on suspicious pixels in the prediction of network B. At the same time, the idea of ​​soft pseudo-labels is introduced to enhance the stability of credible pixels and provide auxiliary teaching signals for the network, which effectively improves the learning efficiency of the model for unlabeled data, further improves the quality of pseudo-labels, and indirectly improves the prediction accuracy of complex and blurred difficult areas in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 : Flow chart of the present invention;

[0063] Figure 2 : Flow chart of the CDPS module of the present invention;

[0064] Figure 3 : Flow chart of the SPLAT module of the present invention;

[0065] Figure 4 : Visual comparison of segmentation results of different methods on the left atrium dataset;

[0066] Figure 5 : Visual comparison of segmentation results of different methods on the brain tumor segmentation dataset. DETAILED DESCRIPTION

[0067] The specific contents of the present invention are further explained in detail below in conjunction with embodiments.

[0068] like Figure 1 As shown, a semi-supervised medical image segmentation method based on mutual correction of suspicious pixels includes the following steps:

[0069] Step 1: Randomly crop the medical images in the medical image datasets from different sources and divide them into training sets, test sets and validation sets;

[0070] The medical image datasets from different sources include the left atrium dataset LA, the brain tumor segmentation dataset BraTS and the automatic heart challenge dataset ACDC, wherein: the left atrium dataset LA consists of 100 three-dimensional gadolinium-enhanced MRI scan images and their labels, with an isotropic resolution of 0.65mm×0.65mm×0.65mm, 80 samples for training, and 20 samples for verification. During the training process, the image data is randomly cropped to 112×112×80 as the input of the network; the brain tumor segmentation dataset BraTS consists of preoperative MRI scan images of 335 patients with gliomas and their labels, including 259 high-grade gliomas HGG and 76 low-grade gliomas LGG. The MRI scan of each patient includes Four modes: T1, T1Gd, T2 and T2-FLAIR, and the labels are checked pixel by pixel by experts, of which 250 samples are used for training, 25 samples are used for validation, and 60 samples are used for testing. During the training process, the data is randomly cropped to 96×96×96 as the input of the network, and a sliding window with a step size of 64×64×64 is used in the inference stage; the automatic heart challenge dataset ACDC consists of cardiac dynamic MRI scans of 100 patients and their labels, which include background, right ventricle, left ventricle and myocardium. 70 samples are used for training, 10 samples are used for validation, and 20 samples are used for testing. During the training process, the 3D scans of the dataset are converted into 2D slices, and the size of all slices is adjusted to 256×256 as the network input;

[0071] Step 2: Construct a semi-supervised medical image segmentation network model SPMC;

[0072] like Figure 1 As shown, the network model SPMC includes network A, network B, Figure 3 The CDPS modules shown in Figure 4 The SPLAT module shown in the figure, when the medical image datasets are LA and BraTS, both network A and network B are Vnet, when the medical image dataset is ACDC, both network A and network B are Unet networks; network A and network B use the same encoder E, and network A and network B use decoders D with different upsampling methods a and D b ;

[0073] Step 3: Train the network model SPMC

[0074] Step 3.1, initialize the parameters of the network model SPMC;

[0075] Use SGD optimizer to train the network model SPMC and set the weight decay factor to 10 -4 , the momentum is set to 0.9, and the initial learning rate is set to 10 -2 , the number of iterations is set to 30k;

[0076] When the medical image dataset is LA, the batch size is set to 8, and each batch includes 4 labeled data and 4 unlabeled data; when the medical image dataset is BraTS, the batch size is set to 4, and each batch includes 2 labeled data and 2 unlabeled data; when the medical image dataset is ACDC, the batch size is set to 16, and each batch includes 8 labeled data and 8 unlabeled data;

[0077] Step 3.2: Input the training set into network A and network B respectively, and output the corresponding feature maps through encoder E. The feature map output by encoder E of network A is directly input into decoder D. a , output the prediction results of the labeled data And the prediction results of unlabeled data Add random noise perturbation to the feature map F output by the encoder E of network B to obtain a feature map with perturbation The feature map containing the perturbation Input decoder D b , output the prediction results of the labeled data And the prediction results of unlabeled data

[0078] Step 3.3: Since the distribution of suspicious pixels in the prediction results of two different networks is usually different, the prediction results after feature perturbation usually contain more suspicious pixels. When both networks have difficulty predicting pixels at the same position, it is necessary to perform additional learning on them. The common pixels are areas that are difficult for both networks to self-correct, so they need to be corrected using more accurate supervisory signals, such as Figure 2 As shown, the CDPS module is used to screen and Identify the common pixels that are difficult for both network A and network B to accurately predict, and correct the common pixels with reference to the existing label Y. The specific process is as follows:

[0079] Step 3.3.1.1. Calculation and , use the threshold τ to filter out the reliable pixels with high confidence, retain the suspicious pixels with low confidence, and generate the corresponding mask and Respectively expressed as:

[0080]

[0081] In the formula, is the indicator function. When the medical image dataset is LA and ACDC, the value of τ is 0.7; when the medical image dataset is BraTS, the value of τ is 0.65;

[0082] Step 3.3.1.2, take and The intersection of network A and network B is obtained by It is expressed as:

[0083]

[0084] Step 3.3.1.3, use Screening and Co-trapped pixels and It is expressed as:

[0085]

[0086] In the formula, ⊙ indicates that the product operation is performed one by one on the elements at the corresponding positions;

[0087] Step 3.3.1.4: In order to obtain the correct supervision signal, use Screen out the pixels with the label Y and The corresponding label Y cdp , expressed as:

[0088]

[0089] Step 3.3.1.5: Use mean square error loss Common Pixels and Make corrections so that network A and network B focus on labeled data and The unpredictable areas in the , and provide correct guidance, are expressed as:

[0090]

[0091]

[0092] In the formula, MSE() is the mean square error loss function;

[0093] like Figure 3 As shown, the SPLAT module was used to screen and Use the trusted pixels in network A to pseudo-supervise the suspicious pixels in network B and fully exploit their predictions and The difference between the two networks is used to encourage suspicious pixels in the prediction results of each network to learn only from the credible pixels in the corresponding position of the other network, and the idea of ​​soft pseudo-labels is introduced to provide more reliable supervision signals for suspicious pixels in the network prediction results. In this way, high-quality interactive learning between the two networks is achieved, thereby improving the generation quality of pseudo-labels and reducing the noise interference caused by pseudo-labels, thereby improving the utilization efficiency of unlabeled data and the generalization ability of the model. The specific process is as follows:

[0094] Step 3.3.2.1: Use the probability sharpening function to and Convert into soft pseudo labels SPL respectively a and SPL b ,make and The probability distribution difference is more obvious, making pixels with high confidence become more "confident". The probability sharpening function is expressed as:

[0095]

[0096] In the formula, T is a hyperparameter that controls the output distribution of the model, with a value of 0.1, and P represents or

[0097] Step 3.3.2.2: Use thresholds to filter out or Trusted pixels and soft pseudo labels SPL in a and SPL b The suspicious pixels in , get different masks, take the intersection of different masks, and get and It is expressed as:

[0098]

[0099] Step 3.3.2.3. Use corrected pseudo-supervision loss For unlabeled data and The suspicious pixels in the network are used for guidance to correct the deviation of the two networks themselves, which is expressed as:

[0100]

[0101] In the formula, MSE() is the mean square error loss function, and ⊙ represents the product operation of the corresponding position elements one by one;

[0102] Step 3.4: Calculate the total loss It is expressed as:

[0103]

[0104] In the formula, and Respectively expressed as:

[0105]

[0106] In the formula, is the cross entropy loss, For the loss of Days, is the mean square error loss, and They are and The corresponding modified pseudo-supervision loss, represents the supervision loss, represents the unsupervised loss, β is a hyperparameter with a value of 0.5, which is used to balance the mean square error loss / corrected pseudo-supervised loss and other losses, λ is a Gaussian warm-up function that varies with time and is used to balance the supervised loss and the unsupervised loss, PL b and PL a are the generated pseudo labels, respectively expressed as:

[0107]

[0108] The λ is expressed as:

[0109]

[0110] In the formula, t represents the current iteration number, t max Indicates the maximum number of iterations;

[0111] Step 3.5, update the parameters of the network model;

[0112] Step 4: Use the validation set to evaluate the performance of the network model SPMC and optimize the parameters;

[0113] Step 5: Input the medical image to be segmented into the trained network model SPMC and output the segmentation result.

[0114] This embodiment is implemented using the Pytorch 1.7 framework and NVIDIA (GeForce RTX 3090 GPU. Four indicators Dice, Jaccard, 95HD and ASD are used to evaluate the performance of the network model SPMC. These four indicators are divided into two categories: region-sensitive indicators and edge-sensitive indicators, where: region-sensitive indicators include Dice similarity coefficient (Dice) and Jaccard similarity coefficient (Jaccard), which are used to measure the degree of overlap between two sample areas. Therefore, the higher the value, the better the segmentation effect; edge-sensitive indicators include 95% Hausdorff distance (95HD) and average surface distance (ASD). 95HD represents the maximum distance between the nearest points between two samples, and ASD represents the average distance between the surface points of two samples. Therefore, the lower the values ​​of 95HD and ASD, the better the segmentation effect.

[0115] On the LA dataset, the performance of the image segmentation method proposed in this embodiment is compared with the existing methods MT, DTC, CPS, MC-Net, URPC, SS-Net, MCF, AC-MT and UG-MCL. For fair comparison, network A is used for inference. The results are shown in Table 1. At the same time, the indicators obtained by training V-Net with 10%, 20% and 100% of the labeled data are used as the baseline and upper limit of the reference performance. It can be observed from Table 1 that compared with the baseline model, the four evaluation indicators of the network model SPMC of this embodiment have been significantly improved. Specifically, when using 10% of the labeled data and 90% of the unlabeled data for training, Dice is improved from 76.51% to 90.33%; when using 20% ​​of the labeled data and 80% of the unlabeled data for training, Dice is improved from 84.78% to 91.07%, which is close to the upper limit of 91.55% for fully supervised learning (that is, using 100% of the labeled data to train V-Net); in addition, compared with the most advanced methods, the network model SPMC of this embodiment still maintains the best performance in the two training settings, especially when using 10% of the labeled data for training, compared with the suboptimal method (AC-MT), Dice is improved by 2.69%, and has higher segmentation accuracy.

[0116] from Figure 4 It can be seen that the network model SPMC of this embodiment can more finely segment the branches or edge areas of the left atrium that are difficult to segment, as shown in the circled part in the figure.

[0117] Table 1 Comparison of segmentation results of different methods on the left atrium dataset

[0118]

[0119] In order to further verify the generalization ability of the network model SPMC of this embodiment on different data sets, the performance of the image segmentation method proposed in this embodiment and the existing methods MT, DTC, CPS, MC-Net, URPC, SS-Net, MCF, AC-MT and UG-MCL are compared on the BraTS data set, and the results are shown in Table 2.

[0120] Table 2 Comparison of segmentation results of different methods on brain tumor segmentation dataset

[0121]

[0122] Compared with the LA dataset, brain tumor segmentation is more challenging, but the network model SPMC of this embodiment still has a good performance. It can be seen from Table 2 that when only 10% of the labeled data and 90% of the unlabeled data are used for training, the Dice is improved by 4.65% compared with the baseline model; in addition, compared with the most advanced methods, the network model SPMC of this embodiment still has an advantage, and the Dice is improved by 1.03% compared with the suboptimal method (UG-MCL).

[0123] from Figure 5 It can be seen that the segmentation effect of the network model SPMC in this embodiment is closer to the true label, especially the edge segmentation of the tumor is more complete.

[0124] In addition to segmentation experiments on binary classification datasets such as LA and BraTS, the performance of the image segmentation method proposed in this embodiment is compared with the existing methods MT, DTC, CPS, MC-Net, URPC, SS-Net, MCF, AC-MT and UG-MCL on the four-classification dataset ACDC. The results are shown in Table 3. When trained with 10% labeled data and 90% unlabeled data, the Dice of the network model SPMC of this embodiment is improved by 7.04% compared with the baseline model; when the number of labeled data is increased to 20%, the Dice is improved by 4.59%. Compared with the most advanced methods, the network model SPMC of this embodiment still maintains the best performance in the segmentation tasks of multiple organs; the network model SPMC of this embodiment is trained with 10% and 20% labeled data, and the Dice is improved by 1.68% and 1.02% respectively compared with its corresponding suboptimal methods (SS-Net, UG-MCL).

[0125] Table 3 Comparison of segmentation results of different methods on the automatic heart challenge dataset

[0126]

Claims

1. A semi-supervised medical image segmentation method based on mutual correction of suspicious pixels, characterized in that: The steps include: Step 1: Crop the medical images in the medical image datasets from different sources and divide them into training sets, test sets and validation sets; Step 2: Construct a semi-supervised medical image segmentation network model SPMC; The network model SPMC includes network A, network B, CDPS module and SPLAT module. Network A and network B are both Vnet or Unet networks. Network A and network B use the same encoder E. Network A and network B use decoders D with different upsampling methods. a and D b ; Step 3: Train the network model SPMC Step 3.1, initialize the parameters of the network model SPMC; Step 3.2: Input the training set into network A and network B respectively, and output the corresponding feature maps through encoder E. The feature map output by encoder E of network A is directly input into decoder D. a , output the prediction results of the labeled data And the prediction results of unlabeled data Add random noise perturbation to the feature map F output by the encoder E of network B to obtain a feature map with perturbation The feature map containing the perturbation Input decoder D b , output the prediction results of the labeled data And the prediction results of unlabeled data Step 3.3: Screening using the CDPS module and Identify the common pixels that are difficult for both network A and network B to accurately predict, and correct them with reference to the existing label Y; Screening using the SPLAT module and Use the trusted pixels generated by network A to pseudo-supervise the suspicious pixels in network B, thereby improving the utilization efficiency of unlabeled data and the generalization ability of the model; Step 3.4: Calculate the total loss Step 3.5, update the parameters of the network model; Step 4: Use the validation set to evaluate the performance of the network model SPMC and optimize the parameters; Step 5: Input the medical image to be segmented into the trained network model SPMC and output the segmentation result.

2. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 1, characterized in that: The medical image datasets from different sources in step 1 include the left atrium dataset LA, the brain tumor segmentation dataset BraTS, and the automatic heart challenge dataset ACDC.

3. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 2, characterized in that: In step 2, when the medical image dataset is LA and BraTS, network A and network B are both Vnet networks; when the medical image dataset is ACDC, network A and network B are both Unet networks.

4. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 2 or 3, characterized in that: Initializing the parameters of the network model SPMC in step 3.1 includes: selecting the SGD optimizer to train the network model SPMC, setting the weight decay factor to 10 -4 , the momentum is set to 0.9, and the initial learning rate is set to 10 -2 , the number of iterations is set to 30k; When the medical image dataset is LA, the batch size is set to 8; when the medical image dataset is BraTS, the batch size is set to 4; when the medical image dataset is ACDC, the batch size is set to 16.

5. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 2 or 3, characterized in that: In step 3.3, the CDPS module is used to screen and The specific process of identifying the common pixels that are difficult for both network A and network B to accurately predict and correcting the common pixels with reference to the existing label Y is as follows: Step 3.3.1.

1. Calculation and , use the threshold τ to filter out the reliable pixels with high confidence, retain the suspicious pixels with low confidence, and generate the corresponding mask and Respectively expressed as: In the formula, is the indicator function; Step 3.3.1.2, take and The intersection of network A and network B is obtained by It is expressed as: Step 3.3.1.3, use Screening and Co-trapped pixels and It is expressed as: In the formula, ⊙ indicates that the product operation is performed one by one on the elements at the corresponding positions; Step 3.3.1.4: In order to obtain the correct supervision signal, use Screen out the pixels with the label Y and The corresponding label Y cdp , expressed as: Step 3.3.1.

5. Use mean square error loss Common Pixels and Make corrections so that network A and network B focus on labeled data and The unpredictable areas in the , and provide correct guidance, are expressed as: Where MSE() is the mean square error loss function.

6. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 5, characterized in that: In step 3.3, the SPLAT module was used to screen and The process of using the trusted pixels in network A to pseudo-supervise the suspicious pixels in network B is: Step 3.3.2.1: Use the probability sharpening function to and Convert into soft pseudo labels SPL respectively a and SPL b ,make and The probability distribution difference is more obvious, and the probability sharpening function is expressed as: Where T is the hyperparameter that controls the output distribution of the model, and P represents or Step 3.3.2.2: Use threshold τ to filter out and Trusted pixels and soft pseudo labels SPL in a and SPL b The suspicious pixels in , get different masks, take the intersection of different masks, and get and It is expressed as: Step 3.3.2.

3. Use corrected pseudo-supervision loss For unlabeled data and The suspicious pixels are used for guidance, which can be expressed as: In the formula, MSE() is the mean square error loss function, and ⊙ represents the product operation of the corresponding position elements one by one.

7. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 6, characterized in that: The value of T in step 3.3.2.1 is 0.

1.

8. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 6, characterized in that: When the medical image dataset is LA and ACDC, the value of τ is 0.7; when the medical image dataset is BraTS, the value of τ is 0.

65.

9. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 6, characterized in that: The total loss in step 3.4 It is expressed as: In the formula, and Respectively expressed as: In the formula, is the cross entropy loss, For the loss of Days, is the mean square error loss, and They are and The corresponding modified pseudo-supervision loss, represents the supervision loss, represents the unsupervised loss, β is a hyperparameter, γ is a time-varying Gaussian warm-up function, PL b and PL a are the generated pseudo labels, respectively expressed as: The λ is expressed as: In the formula, t represents the current iteration number, t max Indicates the maximum number of iterations.

10. The semi-supervised medical image segmentation method based on mutual correction of suspicious pixels according to claim 9, characterized in that: The value of β is 0.5.

Images