Semi-supervised medical image segmentation method based on single-cycle regularization

Abstract

The invention belongs to the field of medical image processing, and particularly relates to a semi-supervised medical image segmentation method based on single-loop regularization. Comprising the following steps: learning a preprocessed labeled image by adopting a plurality of parallel sub-networks, and calculating supervised loss of a model; single-cycle consistency regularization is adopted to enable the sub-networks to share the learned information, and regularization loss of the model is calculated; selecting a pseudo label of each sub-network by adopting a pseudo label competition mechanism; each sub-network uses a pseudo label to learn the preprocessed unlabeled image, and the unsupervised loss of the model is calculated; calculating the total loss of the model according to the supervised loss, the regularized loss and the unsupervised loss; adjusting model parameters according to the total loss of the model to obtain a trained medical image segmentation model; performing image segmentation by using the trained model; according to the method, related knowledge can be better extracted from the label-free image, the influence caused by noise is avoided, and the accuracy of image segmentation is improved.

Description

Technical Field

[0001] The invention belongs to the field of medical image processing, and in particular relates to a semi-supervised medical image segmentation method based on single-cycle regularization. Background Art

[0002] Medical images play an increasingly important role in the diagnosis process of modern medicine. When medical staff diagnose and treat, medical images can reflect the internal structure of the human body, reflect the details of lesions and injuries, and help doctors to further diagnose and analyze diseases. Existing studies have shown that good segmentation effects can be obtained by using a large amount of labeled data and existing image segmentation algorithms; however, in the field of medical image processing, it is often very expensive and time-consuming to obtain a large amount of labeled medical image data. Therefore, semi-supervised medical image segmentation using only a small amount of labeled data and a large amount of unlabeled data has important practical significance and application value. It reduces the demand for manually labeled data by extracting information from unlabeled images, and this information can also improve the generalization ability and accuracy of the model. At the same time, other technologies such as data enhancement and transfer learning can be combined to further improve the performance of the model. Semi-supervised medical image segmentation has broad application prospects in the field of medical image processing, and can be applied to disease diagnosis, surgical navigation, medical research and many other aspects.

[0003] The current mainstream semi-supervised medical image segmentation mainly uses deep learning models and pseudo-labeling, consistency learning, transfer learning and other methods to learn the features of unlabeled images. At the same time, it is necessary to avoid the interference caused by the model's incorrect predictions. The semi-supervised medical image segmentation method can further improve the model's generalization ability and accuracy when there are only a small number of labeled images. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a semi-supervised medical image segmentation method based on single-cycle regularization, the method comprising: obtaining a medical image to be predicted, and inputting it into a trained medical image segmentation model to obtain a medical image segmentation result;

[0005] The training process of the medical image segmentation model includes:

[0006] S1: Obtain a medical image dataset and preprocess it to obtain preprocessed medical images; wherein the medical images include labeled images and unlabeled images;

[0007] S2: Use multiple parallel sub-networks to learn labeled images, and each sub-network outputs its own image segmentation results; calculate the supervised loss of the medical image segmentation model;

[0008] S3: Single-cycle consistency regularization is used to share the learned information between sub-networks, and the regularization loss of the medical image segmentation model is calculated;

[0009] S4: According to the image segmentation results output by each sub-network, a pseudo-label competition mechanism is used to select the pseudo-label of each sub-network;

[0010] S5: Each sub-network uses pseudo labels to learn unlabeled images and calculate the unsupervised loss of the medical image segmentation model;

[0011] S6: Calculate the total model loss based on the supervised loss, regularization loss and unsupervised loss; adjust the model parameters based on the total model loss to obtain a trained medical image segmentation model.

[0012] Furthermore, the process of preprocessing medical images includes:

[0013] Resize the labeled image and its label to 256×256×3 and 256×256×1 respectively;

[0014] Resize the unlabeled image to 256×256×3;

[0015] Randomly select an image enhancement method from brightness adjustment, contrast adjustment, Gaussian blur, and random cropping to process all medical images to obtain preprocessed medical images.

[0016] Preferably, the formula for calculating the supervised loss of the medical image segmentation model is:

[0017]

[0018] Among them, L sup represents the supervised loss of the medical image segmentation model, represents the supervised loss of subnetwork i, α represents the first weight coefficient, L dice represents dice loss, L ce represents the cross entropy loss, f i (x) represents the image segmentation result of subnetwork i for input image x, y represents the label of input image x, and N represents the number of subnetworks.

[0019] Preferably, the formula for calculating the regularization loss of the medical image segmentation model is:

[0020]

[0021] Among them, L con represents the regularization loss of the medical image segmentation model, represents the regularization loss of sub-network i, α represents the first weight coefficient, L dicerepresents dice loss, L ce represents the cross entropy loss, f i (x) represents the image segmentation result of sub-network i for the input image x, f i (x) represents the image segmentation result of sub-network i for the input image x, and N represents the number of sub-networks.

[0022] Preferably, the process of selecting pseudo labels for each subnetwork includes: using the image segmentation result output by the current best subnetwork as the pseudo label for the remaining subnetworks, and using the image segmentation result output by the second-best subnetwork as the pseudo label for the best subnetwork; wherein, the smaller the supervised loss, the better the subnetwork.

[0023] Preferably, the process of calculating the unsupervised loss of the medical image segmentation model includes:

[0024] Calculate the sum of the unsupervised losses of all sub-networks except the best sub-network

[0025]

[0026] Calculate the unsupervised loss of the best subnetwork

[0027]

[0028] Among them, β represents the second weight coefficient, L dice represents dice loss, L mse represents the mean square error loss, f i (x) represents the image segmentation result of sub-network i for the input image x, represents the image segmentation result of sub-network b1 for input image x, b1 represents the label of the best sub-network, represents the image segmentation result of the sub-network b2 for the input image x, and b2 represents the label of the sub-optimal sub-network;

[0029] according to and Compute the unsupervised loss for medical image segmentation models.

[0030] Furthermore, the formula for calculating the unsupervised loss of the medical image segmentation model is:

[0031]

[0032] Among them, L unsup represents the unsupervised loss of the medical image segmentation model, σ represents the sigmoid function,

[0033] represents the supervised loss of subnetwork b1, represents the supervised loss of sub-network b2.

[0034] Preferably, the formula for calculating the total loss of the model is:

[0035] L=L sup +λ j (L con +L unsup )

[0036]

[0037] Among them, L represents the total loss of the model, L sup represents the supervised loss of the medical image segmentation model, L con represents the regularization loss of the medical image segmentation model, L unsup represents the unsupervised loss of the medical image segmentation model, λ j represents the third weight during the j-th round of training, and epoch represents the total number of training rounds.

[0038] The beneficial effects of the present invention are as follows: the present invention further improves the model performance by adopting a plurality of sub-network integration methods while using a designed single-loop consistency regularization module for feature extraction and information fusion; the competitive pseudo-label generation mechanism designed by the present invention can further extract information from unlabeled images while avoiding the influence of errors caused by long-term learning of a single sub-network output on the final result; the present invention can better extract relevant knowledge from unlabeled images, avoid the influence of noise, and improve the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a training flow chart of the medical image segmentation model of the present invention;

[0040] Figure 2 This is a structural diagram of the medical image segmentation model in the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The present invention proposes a semi-supervised medical image segmentation method based on single-cycle regularization, which includes the following contents:

[0043] Obtain the medical image to be predicted and input it into the trained medical image segmentation model to obtain the medical image segmentation result.

[0044] like Figure 1 , Figure 2 As shown in Figure 1, the training process of the medical image segmentation model includes:

[0045] S1: Obtain a medical image dataset and preprocess it to obtain preprocessed medical images; wherein the medical images include labeled images and unlabeled images.

[0046] Obtain a medical image dataset and preprocess it. Specifically, adjust the image size and the corresponding label size to 256×256×3 and 256×256×1 for labeled images, and adjust the image size to 256×256×3 for unlabeled images. Then randomly select an image enhancement method from brightness adjustment, contrast adjustment, Gaussian blur, and random cropping and apply it to the image currently waiting for training to enhance data diversity and obtain preprocessed medical images.

[0047] S2: Use multiple parallel sub-networks to learn labeled images, and each sub-network outputs its own image segmentation results; calculate the supervised loss of the medical image segmentation model.

[0048] The model uses a sub-network with the same structure but different parameter initialization. The sub-network is a deep learning network that can achieve image segmentation. In the training network, parallel sub-networks with the same structure but different initialization parameters are used to perform supervised learning on labeled images at the same time, and each sub-network outputs its own image segmentation results.

[0049] The supervised loss of subnetwork i in labeled images during training is:

[0050]

[0051] in, represents the supervised loss of subnetwork i; α represents the first weight coefficient, preferably, α is set to 0.5; L dice represents dice loss, L ce represents the cross entropy loss, f i (x) represents the image segmentation result of subnetwork i for input image x, y represents the label of input image x, and N represents the number of subnetworks.

[0052] After training, the supervised loss of each sub-network is calculated, and the total supervised loss of the model is

[0053]

[0054] Among them, L suprepresents the supervised loss of the medical image segmentation model, and N represents the number of sub-networks.

[0055] S3: Single-cycle consistency regularization is used to share the learned information between sub-networks, and the regularization loss of the medical image segmentation model is calculated.

[0056] The present invention designs a single-loop consistency regularization structure for information exchange between sub-networks; prediction results of different sub-networks on labeled images are input into the single-loop consistency regularization structure to perform single-loop consistency regularization learning, in which a sub-network will only learn from another sub-network that has not been learned by other sub-networks, thus forming a single-loop structure; for example: if there are 3 sub-networks, in this structure, sub-network 3 learns from sub-network 2, sub-network 2 learns from sub-network 1, and sub-network 1 learns from sub-network 3, thus forming a single-loop structure, which can ensure that a single sub-network will not be interfered by noise from too many sub-networks, and can also ensure that the sub-network has direct or indirect connections to other sub-networks.

[0057] The regularization loss of this structure is:

[0058]

[0059] The total regularization loss of the medical image segmentation model is:

[0060]

[0061] Among them, L con represents the regularization loss of the medical image segmentation model, represents the regularization loss of subnetwork i, α represents the first weight coefficient, and f i-1 (x) represents the image segmentation result of sub-network i-1 for the input image x, f N (x) represents the image segmentation result of sub-network N for the input image x.

[0062] S4: According to the image segmentation results output by each sub-network, a pseudo-label competition mechanism is used to select the pseudo-label of each sub-network.

[0063] The image segmentation result output by the current best sub-network is used as the pseudo label of the remaining sub-networks, and the image segmentation result output by the second-best sub-network is used as the pseudo label of the best sub-network; the smaller the supervised loss, the better the sub-network.

[0064]

[0065]

[0066] Among them, b1 represents the label of the best sub-network, and b2 represents the label of the second best sub-network.

[0067] For example, if the losses of the three subnetworks are 0.127, 0.186, and 0.093 respectively, then subnetwork 3 is the best subnetwork and subnetwork 1 is the second best subnetwork. In this example, both subnetwork 1 and subnetwork 2 use the output of subnetwork 3 as pseudo labels, and the best subnetwork, i.e., subnetwork 3, uses the output of subnetwork 1 as pseudo labels.

[0068] S5: Each sub-network uses pseudo labels to learn unlabeled images and calculate the unsupervised loss of the medical image segmentation model.

[0069] Calculate the sum of the unsupervised losses of all sub-networks except the best sub-network

[0070]

[0071] Calculate the unsupervised loss of the best subnetwork

[0072]

[0073] Among them, β represents the second weight coefficient, L mse represents the mean square error (MSE) loss, represents the image segmentation result of subnetwork b1 on the input image x, Represents the image segmentation result of subnetwork b2 for input image x. The reason why the present invention uses MSE instead of CE is that MSE can generate a smoother error value and reduce the influence of pseudo-label noise.

[0074] according to and Compute the unsupervised loss for a medical image segmentation model:

[0075]

[0076] Among them, L unsup represents the unsupervised loss of the medical image segmentation model, σ represents the sigmoid function, represents the supervised loss of subnetwork b1, represents the supervised loss of sub-network b2.

[0077] S6: Calculate the total model loss based on the supervised loss, regularization loss and unsupervised loss; adjust the model parameters based on the total model loss to obtain a trained medical image segmentation model.

[0078] The formula for calculating the total loss of the model is:

[0079] L=Lsup +λ j (L con +L unsup )

[0080]

[0081] Among them, L represents the total loss of the model, L sup represents the supervised loss of the medical image segmentation model, L con represents the regularization loss of the medical image segmentation model, L unsup represents the unsupervised loss of the medical image segmentation model, λ j represents the third weight during the j-th round of training, and epoch represents the total number of training rounds.

[0082] During the training process, the Adam optimizer was used, the BatchSize of a single card was set to 8, the total number of training rounds was set to 6000, the learning rate was divided by 10 every 1000 rounds, and training was performed on two Nvidia GeForce RTX A6000s. The network parameters were updated using the back propagation algorithm. When the loss function converged or reached the maximum preset number of iterations, the training was stopped, the model parameters were saved, and the trained medical image segmentation model was obtained.

[0083] After obtaining the trained model, the image to be predicted is input into the training network. For the predictions of the obtained sub-networks, the results with the same prediction are taken as the final prediction results, and the output with higher prediction confidence is selected as the final prediction result for the parts with different prediction results.

[0084] In summary, the present invention utilizes multiple different sub-networks during training to enable the model to view the problem from different angles, so as to extract more information from limited data. At the same time, a single-loop regularization structure is designed. While the model is learning labeled image pairs, this structure can also be used to exchange information from different angles learned by each sub-network, so as to enhance the recognition and generalization capabilities of the model. The use of a single-loop structure can avoid the situation where the model fluctuates due to excessive noise from different sources while learning the outputs of multiple sub-networks at the same time. At the same time, the loop structure can also be used to indirectly learn the outputs of other sub-networks, and complete the information exchange between sub-networks. In the subsequent training process, supervised learning can also further reduce the impact of noise in the single-loop consistency regularization, and further improve the model performance. The traditional learning method for unlabeled images is to use a teacher-student model. Such a model has a disadvantage: since the network that generates pseudo-labels is a specified specific sub-network, as the training progresses, the errors in the sub-network used to generate pseudo-labels may be continuously transmitted with the training errors and cannot be corrected in subsequent training. Therefore, the present invention adopts a competition mechanism to select the sub-network that generates pseudo-labels by using the learning results of labeled images. Through this competition mechanism, all sub-networks may generate pseudo-labels for other sub-networks to learn, which not only promotes information exchange between sub-networks, but also avoids the transmission of errors caused by long-term learning of the output of a single sub-network, thereby improving model performance. The present invention can better extract relevant knowledge from unlabeled images, avoid the impact of noise, and improve the accuracy of image segmentation.

[0085] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A semi-supervised medical image segmentation method based on single-cycle regularization, characterized in that: include: Obtain a medical image to be predicted and input it into a trained medical image segmentation model to obtain a medical image segmentation result; The training process of the medical image segmentation model includes: S1: Obtain a medical image dataset and preprocess it to obtain preprocessed medical images; wherein the medical images include labeled images and unlabeled images; S2: Use multiple parallel sub-networks to learn labeled images, and each sub-network outputs its own image segmentation results; calculate the supervised loss of the medical image segmentation model; S3: Single-cycle consistency regularization is used to share the learned information between sub-networks, and the regularization loss of the medical image segmentation model is calculated; S4: According to the image segmentation results output by each sub-network, a pseudo-label competition mechanism is used to select the pseudo-label of each sub-network; S5: Each sub-network uses pseudo labels to learn unlabeled images and calculate the unsupervised loss of the medical image segmentation model; S6: Calculate the total model loss based on the supervised loss, regularization loss and unsupervised loss; adjust the model parameters based on the total model loss to obtain a trained medical image segmentation model.

2. The semi-supervised medical image segmentation method based on single-cycle regularization according to claim 1, characterized in that: The process of preprocessing medical images includes: Resize the labeled image and its label to 256×256×3 and 256×256×1 respectively; Resize the unlabeled image to 256×256×3; Randomly select an image enhancement method from brightness adjustment, contrast adjustment, Gaussian blur, and random cropping to process all medical images to obtain preprocessed medical images.

3. The semi-supervised medical image segmentation method based on single-cycle regularization according to claim 1, characterized in that: The formula for calculating the supervised loss of the medical image segmentation model is: Among them, L sup represents the supervised loss of the medical image segmentation model, represents the supervised loss of subnetwork i, α represents the first weight coefficient, L dice represents dice loss, L ce represents the cross entropy loss, f i (x) represents the image segmentation result of subnetwork i for input image x, y represents the label of input image x, and N represents the number of subnetworks.

4. The semi-supervised medical image segmentation method based on single-cycle regularization according to claim 1, characterized in that: The formula for calculating the regularization loss of the medical image segmentation model is: Among them, L con represents the regularization loss of the medical image segmentation model, represents the regularization loss of sub-network i, α represents the first weight coefficient, L dice represents dice loss, L ce represents the cross entropy loss, f i (x) represents the image segmentation result of sub-network i for the input image x, f i (x) represents the image segmentation result of sub-network i for the input image x, and N represents the number of sub-networks.

5. The semi-supervised medical image segmentation method based on single-cycle regularization according to claim 1, characterized in that: The process of selecting the pseudo labels of each subnetwork includes: using the image segmentation result output by the current best subnetwork as the pseudo label of the remaining subnetworks, and using the image segmentation result output by the second-best subnetwork as the pseudo label of the best subnetwork; wherein, the smaller the supervised loss, the better the subnetwork.

6. The semi-supervised medical image segmentation method based on single-cycle regularization according to claim 1, characterized in that: The process of calculating the unsupervised loss for a medical image segmentation model involves: Calculate the sum of the unsupervised losses of all sub-networks except the best sub-network Calculate the unsupervised loss of the best subnetwork Among them, β represents the second weight coefficient, L dice represents dice loss, L mse represents the mean square error loss, f i (x) represents the image segmentation result of sub-network i for the input image x, represents the image segmentation result of sub-network b1 for input image x, b1 represents the label of the best sub-network, represents the image segmentation result of the sub-network b2 for the input image x, and b2 represents the label of the sub-optimal sub-network; according to and Compute the unsupervised loss for medical image segmentation models.

7. The semi-supervised medical image segmentation method based on single-cycle regularization according to claim 6, characterized in that: The formula for calculating the unsupervised loss for the medical image segmentation model is: Among them, L unsup represents the unsupervised loss of the medical image segmentation model, σ represents the sigmoid function, represents the supervised loss of subnetwork b1, represents the supervised loss of sub-network b2.

8. The semi-supervised medical image segmentation method based on single-cycle regularization according to claim 1, characterized in that: The formula for calculating the total loss of the model is: L=L sup +λ j (L con +L unsup ) Among them, L represents the total loss of the model, L sup represents the supervised loss of the medical image segmentation model, L con represents the regularization loss of the medical image segmentation model, L unsup represents the unsupervised loss of the medical image segmentation model, λ j represents the third weight during the j-th round of training, and epoch represents the total number of training rounds.

Images