Semi-supervised medical image segmentation method and device, equipment and storage medium

Abstract

The invention provides a semi-supervised medical image segmentation method and device, equipment and a storage medium, and relates to the technical field of deep learning, and the method comprises the steps: obtaining a medical image in a historical time period; obtaining a target medical image; obtaining a semi-supervised image segmentation model; and obtaining a prediction segmentation result of the target medical image. The optimal transmission and central moment consistency regularization method is put forward from the global perspective, on one hand, the optimal transmission is adopted to select an area with similar semantics of marked data and unmarked data for label propagation, and the optimal transmission has the advantages of considering global distribution information and searching for optimal mapping between two distributions; on the one hand, the method is more suitable for label propagation than a graph structure or a similar matrix which depends too much on local information and is easy to converge to local optimum, on the other hand, central moment consistency regularization is adopted to constrain the model to pay attention to global geometric features of unmarked data, and compared with a traditional convolutional neural network which usually learns local information of images, the method has the advantages that the algorithm is more accurate. And the global information learning capability of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a semi-supervised medical image segmentation method, device, equipment and storage medium. Background Art

[0002] Medical data usually has complex backgrounds and fuzzy boundaries, which greatly increases the difficulty of annotation, so it is unrealistic to obtain a large amount of labeled medical data. In recent years, semi-supervised learning attempts to make full use of labeled and unlabeled data to improve the performance of the model and reduce the demand for labeled data. Therefore, semi-supervised medical image segmentation algorithms have been widely studied.

[0003] We focus on pseudo-labeling methods and consistency regularization methods. On the one hand, pseudo-labeling methods use models to predict pseudo-labels for unlabeled data to further train the model. To improve the quality of pseudo-labels, some research methods combine conditional random fields (CRFs), level set methods, and weak boundary annotations. Other research methods use multiple models to jointly optimize the training process; however, these methods focus more on learning the distribution of unlabeled data rather than the association between unlabeled data and labels. On the other hand, consistency regularization-based methods encourage the network to produce the same predictions for the same samples under different perturbations (such as model perturbations, different tasks). In order to provide more information to the model from unlabeled data, researchers have proposed methods such as uncertainty mapping, multi-scale consistency, and constructing similarity matrices between labeled and unlabeled features. However, these methods mainly rely on the information provided by convolutional neural networks (CNNs), which usually learn local representations of images and lack global learning of unlabeled data. Summary of the invention

[0004] The purpose of the present invention is to provide a semi-supervised medical image segmentation method, device, equipment and storage medium to improve the above problems. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] In a first aspect, the present application provides a semi-supervised medical image segmentation method, comprising:

[0006] Acquire medical images within a historical time period, where the medical images include a first image and a second image, where the first image is an image carrying a label, and the second image is an image not carrying a label;

[0007] Acquire a target medical image, where the target medical image is an image without a label;

[0008] Training a preset semi-supervised learning model based on the medical image, and stopping the training when the number of training times meets a set condition to obtain a semi-supervised image segmentation model;

[0009] The target medical image is input into the semi-supervised image segmentation model to obtain a predicted segmentation result of the target medical image.

[0010] In a second aspect, the present application also provides a semi-supervised medical image segmentation device, comprising:

[0011] A first acquisition unit, configured to acquire medical images within a historical time period, wherein the medical images include a first image and a second image, wherein the first image is an image carrying a label, and the second image is an image not carrying a label;

[0012] A second acquisition unit is used to acquire a target medical image, where the target medical image is an image without a label;

[0013] A training unit, used for training a preset semi-supervised learning model based on the medical image, and stopping the training when the number of training times meets a set condition to obtain a semi-supervised image segmentation model;

[0014] The first input unit is used to input the target medical image into the semi-supervised image segmentation model to obtain a predicted segmentation result of the target medical image.

[0015] In a third aspect, the present application also provides a semi-supervised medical image segmentation device, comprising:

[0016] Memory for storing computer programs;

[0017] A processor is used to implement the steps of the semi-supervised medical image segmentation method when executing the computer program.

[0018] In a fourth aspect, the present application further provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned semi-supervised medical image segmentation method are implemented.

[0019] The beneficial effects of the present invention are:

[0020] The present invention proposes an optimal transmission and central moment consistency regularization method from a global perspective. On the one hand, the optimal transmission is used to select regions with semantically similar labeled data and unlabeled data for label propagation. The optimal transmission has the advantages of considering global distribution information and finding the optimal mapping between two distributions. It is more suitable for label propagation than graph structures or similar matrices that are overly dependent on local information and easily converge to local optimality. On the other hand, the central moment consistency regularization is used to constrain the model to focus on the global geometric features of unlabeled data. Compared with traditional convolutional neural networks that usually learn local information of images, the model's ability to learn global information is improved. .

[0021] Therefore, other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 It is a schematic diagram of the process flow of the semi-supervised medical image segmentation method described in an embodiment of the present invention;

[0024] Figure 2 Schematic diagram of a dual-branch architecture of a semi-supervised image segmentation network described in an embodiment of the present invention;

[0025] Figure 3 It is a schematic diagram of the process of the optimal transmission part described in an embodiment of the present invention;

[0026] Figure 4 Schematic diagram of the structure of the semi-supervised medical image segmentation device described in an embodiment of the present invention;

[0027] Figure 5 Schematic diagram of the structure of the semi-supervised medical image segmentation device described in an embodiment of the present invention.

[0028] Markings in the figure: 10, first acquisition unit; 20, second acquisition unit; 30, training unit; 40, first input unit; 800, semi-supervised medical image segmentation device; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0031] Embodiment 1:

[0032] This embodiment provides a semi-supervised medical image segmentation method.

[0033] See also Figure 1 , the figure shows that the method includes step S10, step S20, step S30 and step S40.

[0034] Step S10. Acquire medical images within a historical time period, where the medical images include a first image and a second image, where the first image is an image carrying a label, and the second image is an image not carrying a label;

[0035] Specifically, considering that the image segmentation algorithm based on deep learning requires a large amount of labeled data for training to obtain good segmentation performance, and medical images usually have complex backgrounds and blurred boundaries, which greatly increases the difficulty of labeling, it is unrealistic to obtain a large amount of labeled medical data. Therefore, semi-supervised learning can be used to try to make full use of labeled and unlabeled data to improve the performance of the model.

[0036] Normally, the annotations for any medical image are usually divided into foreground annotations and background annotations. The foreground annotated area is usually the area that needs to be studied in detail. The types of annotations required for different medical images may be different, and no special restrictions are made here.

[0037] Step S20. Acquire a target medical image, where the target medical image is an image without a label;

[0038] Specifically, the target medical image is a medical image that needs to be segmented.

[0039] Step S30. Training a preset semi-supervised learning model based on medical images, and when the number of training times meets the set conditions, stopping the training to obtain a semi-supervised image segmentation model;

[0040] Specifically, Figure 2 As shown, it is a schematic diagram of the dual-branch architecture of the semi-supervised image segmentation network in this application. This application is described in detail based on the architecture of the teacher branch and the student branch. The image samples used for training are input into the teacher branch and the student branch at the same time. Only the teacher branch passes through the mapping head to output the feature map F. The feature map F contains labeled features and unlabeled features. The student branch outputs the prediction map P for the unlabeled image samples. s There are five losses in the training network, supervision loss L s Contains Dice loss and cross entropy loss. The unsupervised loss contains the mean square error loss L of the two branches for unlabeled images. c , second-order central moment consistency loss L cmc and the information transmission loss L based on optimal transmission w-pseu , where the information transmission loss L based on optimal transmission w-pseu It is composed of feature map F and prediction map P s Calculated by the optimal transmission algorithm.

[0041] Specifically, step S30 specifically includes step S31, step S32, step S33, step S34, step S35, step S36, step S37, step S38, step S39, step S310 and step S311:

[0042] Step S31. Each medical image is input into the teacher branch to obtain a target feature map and a first prediction map;

[0043] Step S32. Input each medical image into the student branch to obtain a second prediction image;

[0044] Specifically, for a fixed number of labeled images and unlabeled images input in a preset round, random cropping and flipping are used to expand the number of images to avoid overfitting of the model. In this application, the preset round is 15,000 rounds, and a small amount of data is input in each round, which includes labeled images and unlabeled images. In the process of training the semi-supervised image segmentation network in each round, the teacher branch f of the semi-supervised image segmentation network t It will output the prediction map and feature map of each image, and the student branch f s Only the prediction map for each image is output.

[0045] Step S33. Perform transformation processing on the first prediction map and the second prediction map to obtain a first probability map and a second probability map;

[0046] Step S34. Calculate the Dice loss based on the second probability map of the first image and the first image to obtain a first loss;

[0047] Specifically, Dice loss is used to evaluate the similarity between the network's prediction results of the image and the true label in the image segmentation task. The Dice loss calculation formula is:

[0048]

[0049] Among them, L D For the first loss; Y is the predicted probability value of the i-th pixel in the second probability map of the first image; i is the label carried by the i-th pixel in the label map of the first image; n is the number of pixels divided in the first image.

[0050] Step S35. Based on the second prediction image of the first image and the first image, calculate the cross entropy loss to obtain a second loss;

[0051] Specifically, cross entropy loss is a commonly used loss in classification tasks, which is used to measure the difference between two probability distributions. The cross entropy loss calculation formula is:

[0052]

[0053] Among them, L E For the second loss; Y is the predicted value of the jth class predicted by the i-th pixel in the second prediction image of the first image; i is the label carried by the i-th pixel in the label map of the first image; n is the number of pixels divided in the first image.

[0054] Step S36. Based on the first prediction images of all second images and the second prediction images of the second images, calculate the mean square error loss and the second-order central moment consistency loss to obtain the third loss and the fourth loss;

[0055] Specifically, the mean square error loss is used to calculate the difference between the prediction graphs obtained by the teacher branch and the student branch, which can be used to measure the consistency of the outputs of the two branches. By minimizing the loss, the predictions of the two branches can be made more consistent, thereby improving the robustness and performance of the model. The mean square error loss calculation formula is:

[0056]

[0057] Among them, L c For the third loss; is the predicted probability that the i-th pixel of the first probability map belongs to the j-th class; is the predicted probability that the i-th pixel of the second probability map belongs to the j-th category; n is the number of pixels divided into the second image; c is the number of label categories.

[0058] Specifically, step S36 specifically includes step S361, step S362, step S363 and step S364:

[0059] Step S361. Calculate the centroid positions in the first probability map of the second image and the second probability map of the second image respectively to obtain a first position and a second position;

[0060] Step S362. Calculate the second-order central moment of the teacher branch based on the first position to obtain the first second-order central moment;

[0061] Step S363. Calculate the second-order central moment of the student branch based on the second position to obtain a second second-order central moment;

[0062] Step S364. Calculate the norm of the first second-order central moment and the second second-order central moment to obtain a fourth loss;

[0063] Specifically, in the prior art, in order to ensure the consistency of model predictions under different training conditions, it is necessary to rely on the information provided by the convolutional neural network, and the convolutional neural network usually focuses on learning the local representation of the image and lacks global learning of unlabeled data. Therefore, this application considers the second-order central moment consistency loss from a global geometric perspective, so as to learn more information about the unlabeled images.

[0064] The second-order central moment is calculated for the target area part in the unlabeled image probability map output by the teacher branch and the student branch, and the second-order central moment consistency of the two branches is constructed, where:

[0065]

[0066] Among them, P s is the second probability map; P t is the first probability map; is the predicted probability value of the cth category in the second probability map; is the predicted probability value of the cth category in the first probability map; c is the number of label categories.

[0067] c is the number of label categories, which actually represents the category of the object. For example, if the object to be predicted is a brain tumor, then the rest are background categories, brain tumor is the foreground category, and the number of label categories is 2; if multiple organs (for example, 3) are to be predicted, then the background category is added, and the number of label categories is 4.

[0068] The second-order central moment of the c-th target object can be:

[0069]

[0070] in, is the p+q+r order central moment of the cth category in the second probability map; is the p+q+r order central moment of the cth category in the first probability map; is the predicted value of the c-th category position (x, y, z) in the second prediction map; is the predicted value of the c-th category position (x, y, z) in the second prediction map; (x s ,y s ,z s ) is the centroid of the second prediction graph; (x t ,y t ,z t ) is the centroid of the first prediction graph; M is the maximum x-coordinate; N is the maximum y-coordinate; L is the maximum z-coordinate; c is the number of label categories.

[0071] The calculation formula for the second-order central moment consistency loss is:

[0072]

[0073] Among them, L cmc For the fourth loss; is the second-order central moment of the cth category in the second probability map; is the second-order central moment of the cth category in the first probability map; I(·) is the norm; c is the number of label categories.

[0074] Step S37. Based on the target feature map and the second probability map of the second image, calculate the information transfer loss to obtain a fifth loss;

[0075] Specifically, considering that the features of the labeled image and the unlabeled image obey the labeled feature distribution and the unlabeled feature distribution respectively, the transfer matrix after optimal transfer optimization represents the similarity between the two feature distributions, and the most similar labeled feature is selected for each unlabeled feature from the labeled feature distribution, and its pixel label is assigned to the pixel of the unlabeled feature as its pixel pseudo-label. At the same time, the probability value that the labeled feature is most similar to the unlabeled feature in the optimal transfer matrix is ​​selected as the pseudo-label weight to measure the reliability of the pseudo-label.

[0076] The existing technology usually adopts pseudo-labels combined with conditional random fields, level set methods, boundary annotations or joint optimization of multiple models. However, these methods focus more on learning the distribution of unlabeled image samples rather than the relationship between annotation features and labels. Current label propagation methods, such as cosine similarity and construction graph structure, rely on local information and easily converge to local optimality, which will cause inaccurate label propagation and weaken the transmission of effective information.

[0077] Therefore, in the embodiments of the present application, the relationship between labeled image samples and unlabeled image samples is fully considered, and label propagation based on optimal transmission is constructed. Compared with the prior art, optimal transmission has the advantages of considering global distribution information and finding the optimal mapping between labeled image samples and unlabeled image samples, and is more suitable for label propagation.

[0078] Second-order central moment consistency regularization and label propagation based on optimal transfer have complementary effects. The two methods learn unlabeled image samples in a complementary way: optimal transfer uses the label information in labeled image samples to help the network learn unlabeled image samples, while second-order central moment consistency regularization focuses on learning global geometric information from the unlabeled image samples themselves.

[0079] Specifically, step S37 specifically includes step S371, step S372, step S373, step S374, step S375, step S376, step S377, step S378 and step S379:

[0080] Step S371. Determine a plurality of target pixel features from the annotated features in the target feature map, where the target pixel features are pixel features whose predicted values ​​and label values ​​are the same in the prediction map;

[0081] Step S372. Store multiple target pixel features and corresponding label values ​​in a memory bank;

[0082] Step S373: Sampling is performed on the memory bank to obtain a first feature vector, in which the number of each type of pixel unit is the same, and the first feature vector is a vector carrying a label;

[0083] Step S374. Sampling the unlabeled features in the target feature map to obtain a second feature vector;

[0084] Step S375. Perform random sampling on the second probability map to obtain a probability vector, where the probability vector and the vector in the second feature vector correspond to the same pixel position one by one;

[0085] Step S376. Calculate the transmission matrix of the first eigenvector and the second eigenvector based on the optimal transmission algorithm to obtain the optimal transmission matrix;

[0086] Step S377. Based on the optimal transmission matrix and the label carried by the first eigenvector, assign a pseudo label vector and a corresponding pseudo label weight vector to the probability vector;

[0087] Step S378. Calculate the category probability vector based on the probability vector and the corresponding pseudo label vector;

[0088] Step S379. Based on the pseudo label vector, the pseudo label weight vector and the category probability vector, the information transfer loss is calculated to obtain the fifth loss;

[0089] Specifically, Figure 3 As shown, Figure 2 Schematic diagram of the optimal transmission process in the l and f u are the labeled features and unlabeled features output by the teacher branch, respectively, P s is the probability map predicted by the student branch for the unlabeled image samples, and D, H, and L are the spatial dimensions of the input image samples in the teacher branch and the student branch, respectively.

[0090] First, a memory bank (MB) is established to store the annotation features and corresponding labels from the teacher branch. The semi-supervised learning model stores the correct pixel part predicted by the annotation sample in MB. In order to calculate the optimal transfer matrix, p annotation features are randomly sampled from MB for each label category to form the first feature vector a=(a 1 ,a 2 ,...,a p·c ), where c is the number of label categories, and the corresponding label vector is y = (0, ..., 0, 1, ..., c-1, ..., c-1), where 0 represents the background class and 1 to c-1 represent the foreground class.

[0091] The unlabeled features f output from the teacher branch u Randomly select q unlabeled features to form the second feature vector b = (b 1 ,b 2 ,...,b q ), randomly select q vectors from the probability graph output by the student branch to form a probability vector P = (p 1 ,p 2 ,...,p q ). The probability vector P and the vectors in the unlabeled feature vector b correspond to the same pixel positions one by one, which facilitates the calculation of information transfer loss.

[0092] The specific calculation steps for optimal transmission are:

[0093] Calculate the cost matrix:

[0094] C = dist(a,b)

[0095] Where C is the cost matrix, dist(·) is the distance function, a is the first eigenvector, b is the second eigenvector,

[0096] Initialize the kernel matrix:

[0097]

[0098] Among them, K is the kernel matrix; C is the cost matrix; k is the adjustment factor;

[0099] Initialize the scaling factor:

[0100]

[0101] Among them, I p·c is a vector of all 1s with a length of p·c, I q is a vector of all 1s of length q;

[0102] Iterative Optimization:

[0103] Repeat the iteration until the total number of iterative optimization times is preset. The corresponding iterative calculation formula is:

[0104]

[0105] Among them, a is the first eigenvector; b is the second eigenvector; K T is the transposed matrix of the kernel matrix K; v (i-1) is the scaling factor at the i-1th iteration; u i is the scaling factor at the i-th iteration;

[0106] Calculate the transfer matrix:

[0107] Because vector a and vector b are sampling vectors of the labeled feature distribution and the unlabeled feature distribution, respectively, the optimal transmission of entropy regularization can iteratively obtain the optimal transmission matrix between vector a and vector b The optimal transfer matrix of entropy regularization can be expressed as:

[0108]

[0109] Where m is the total number of iterations; a is the first eigenvector; b is the second eigenvector; K is the kernel matrix; v m -1 is the scaling factor at the m-1th iteration; u m is the scaling factor at the mth iteration; diag(·) is a diagonal matrix.

[0110] Based on the optimal transmission matrix And the label vector y, determine the pseudo label of the unlabeled feature vector b and the corresponding pseudo label weight is defined as follows:

[0111]

[0112] in, is the pseudo label corresponding to the jth feature in the second feature vector; Wj is the weight corresponding to the jth feature in the second eigenvector; is the optimal transmission matrix The value of row i and column j in ; is the index of the maximum value of the optimal transmission matrix in the i-th row; is the maximum value of the optimal transmission matrix in the i-th row; i∈(1,2,...,p·c); j∈(1,2,...,q).

[0113] That is, y is the label vector of vector a, and through the optimal transfer matrix we transfer part of the label of y to vector b; that is, the probability vector P with the same pixel position as vector b has a pseudo label after optimal transfer. At the same time, our optimal transmission matrix also gives the weight W, which can measure reliability.

[0114] Next, use the above pseudo-label Supervised pixel probability of the unlabeled probability vector P: that is, at a given pixel, based on In the category of this pixel, the probability vector P is selected at this pixel position as The probability of the category is Right now Category cross entropy loss is used to calculate information transfer loss, and the calculation formula is as follows:

[0115]

[0116] Among them, L w-pseu For the fifth loss; is the category probability of the probability vector; is the pseudo label of the probability vector; W is the weight corresponding to the pseudo label of the probability vector;

[0117] Label propagation based on optimal transmission can associate the labels of labeled image samples with unlabeled image samples, guiding the model to learn the unlabeled image samples more accurately.

[0118] Step S38. Based on the first loss, the second loss, the third loss, the fourth loss and the fifth loss, a comprehensive loss is obtained;

[0119] Specifically, based on the above multiple losses, the semi-supervised learning model is optimized and trained. The comprehensive loss is mainly divided into the supervised loss on labeled image samples and the unsupervised loss on unlabeled image samples, which are defined as follows:

[0120] L=λ 1 L a +λ 2 L u

[0121] Where, L is the comprehensive loss; L a is the supervision loss of labeled image samples; L u is the unsupervised loss of unlabeled image samples; 1 and λ 2 L s and L u The weight ratio of .

[0122] Based on Dice loss and cross entropy loss, the labeled image samples are trained, and the loss calculated comprehensively is:

[0123] L s =L D +αL E

[0124] Among them, L s is the supervision loss of labeled image samples; L D is the first loss; L E is the second loss; α is the weight of the second loss;

[0125] The learning of unlabeled image samples is supervised based on mean square error loss, information transfer loss and second-order central moment consistency loss, which are defined as follows:

[0126] L u =β 1 L c +β 2 L w-pseu +β 3 L cmc

[0127] Among them, L u is the unsupervised loss of unlabeled image samples; L c For the third loss; cmc For the fourth loss; L w-pseu is the fifth loss; β 1 ,β 2 and β 3 L c , L w-pseu and L cmc The corresponding weight ratio.

[0128] Step S39. Adjust the parameters of the student branch based on the comprehensive loss, and adjust the parameters of the teacher branch based on the exponential moving average algorithm to obtain an updated semi-supervised learning model, where the semi-supervised learning model carries the number of updates;

[0129] Step S310. When a plurality of initial semi-supervised learning models are selected from a plurality of updated semi-supervised learning models, the number of updates carried by the initial semi-supervised learning models is the same as a plurality of preset values;

[0130] Step S311. Calculate and compare the evaluation indicators of multiple initial semi-supervised learning models respectively, and determine the initial semi-supervised learning model with the best evaluation indicator as the semi-supervised image segmentation model;

[0131] Specifically, the semi-supervised learning model is continuously updated and trained based on the above-mentioned comprehensive loss. In this application, the total training rounds are set to 15,000, and the corresponding semi-supervised learning model is saved every 200 training rounds. Finally, four metrics are used to determine the model with the best performance from the 75 saved models as the semi-supervised image segmentation model. The four metrics are: Dice coefficient, Jaccard coefficient, 95% Hausdorff distance and average surface distance. Among them, Dice coefficient and Jaccard coefficient, both indicators are used to measure the similarity of overlapping areas, and the higher the value, the better; 95% Hausdorff distance and average surface distance, these two indicators are used to measure the distance between boundary point sets, and the lower the value, the better.

[0132] Step S40. Inputting the target medical image into the semi-supervised image segmentation model to obtain a predicted segmentation result of the target medical image;

[0133] Specifically, by inputting the target medical image into the trained semi-supervised image segmentation model, the segmentation result of the target medical image will be obtained, that is, a predicted label will be assigned to each pixel position.

[0134] Embodiment 2:

[0135] like Figure 4 As shown, this embodiment provides a semi-supervised medical image segmentation device, the device comprising:

[0136] A first acquisition unit 10 is used to acquire medical images within a historical time period, where the medical images include a first image and a second image, where the first image is an image carrying a label, and the second image is an image not carrying a label;

[0137] A second acquisition unit 20 is used to acquire a target medical image, where the target medical image is an image without a label;

[0138] A training unit 30 is used to train a preset semi-supervised learning model based on medical images, and when the number of training times meets a set condition, the training is stopped to obtain a semi-supervised image segmentation model;

[0139] The first input unit 40 is used to input the target medical image into the semi-supervised image segmentation model to obtain a predicted segmentation result of the target medical image.

[0140] In a specific implementation disclosed in the present application, the training unit 30 includes:

[0141] A second input unit, used for inputting each medical image into the teacher branch to obtain a target feature map and a first prediction map;

[0142] a third input unit, configured to input each medical image into the student branch to obtain a second prediction map;

[0143] A conversion processing unit, used for performing conversion processing on the first prediction map and the second prediction map to obtain a first probability map and a second probability map;

[0144] A first calculation unit, configured to calculate a Dice loss based on a second probability map of the first image and the first image to obtain a first loss;

[0145] A second calculation unit is used to calculate a cross entropy loss based on a second prediction image of the first image and the first image to obtain a second loss;

[0146] A third calculation unit is used to calculate a mean square error loss and a second-order central moment consistency loss based on the first prediction images of all second images and the second prediction images of the second images, so as to obtain a third loss and a fourth loss;

[0147] a fourth calculation unit, configured to calculate an information transfer loss based on the target feature map and the second probability map of the second image to obtain a fifth loss;

[0148] A first obtaining unit is used to obtain a comprehensive loss based on the first loss, the second loss, the third loss, the fourth loss and the fifth loss;

[0149] An adjustment unit, used for adjusting the parameters of the student branch based on the comprehensive loss and adjusting the parameters of the teacher branch based on the exponential moving average algorithm to obtain an updated semi-supervised learning model, wherein the semi-supervised learning model carries the update number;

[0150] A screening unit, used to screen out a plurality of initial semi-supervised learning models from the plurality of updated semi-supervised learning models, wherein the number of updates carried by the initial semi-supervised learning models is the same as a plurality of preset values;

[0151] The first determination unit is used to respectively calculate and compare the evaluation indicators of multiple initial semi-supervised learning models, and determine the initial semi-supervised learning model with the best evaluation indicator as the semi-supervised image segmentation model.

[0152] In a specific implementation disclosed in the present application, the third computing unit includes:

[0153] A fifth calculation unit, used to calculate the centroid position in the first probability map of the second image and the second probability map of the second image respectively, to obtain the first position and the second position;

[0154] A second obtaining unit is used to calculate the second-order central moment of the teacher branch based on the first position to obtain a first second-order central moment;

[0155] A third obtaining unit is used to calculate the second-order central moment of the student branch based on the second position to obtain a second second-order central moment;

[0156] The sixth calculation unit is used to calculate the norm of the first second-order central moment and the second second-order central moment to obtain a fourth loss.

[0157] In a specific implementation disclosed in the present application, the fourth calculation unit includes:

[0158] A second determining unit is used to determine a plurality of target pixel features from the labeled features in the target feature map, where the target pixel features are pixel features whose predicted values ​​and label values ​​are the same in the prediction map;

[0159] A storage unit, used to store multiple target pixel features and corresponding label values ​​in a memory bank;

[0160] A fourth obtaining unit is used to perform sampling processing on the memory library to obtain a first feature vector, the first feature vector includes all kinds of pixel features, the number of each type of pixel feature in the first feature vector is the same, and the first feature vector is a vector carrying a label;

[0161] A first random sampling unit is used to perform random sampling processing from unlabeled features in the target feature map to obtain a second feature vector;

[0162] A second random sampling unit is used to perform random sampling processing on the second probability map of the second image to obtain a probability vector, and the probability vector and the vector in the second feature vector correspond to the same pixel position one by one;

[0163] A fifth obtaining unit, configured to calculate a transmission matrix of the first eigenvector and the second eigenvector based on an optimal transmission algorithm to obtain an optimal transmission matrix;

[0164] an allocating unit, configured to allocate a pseudo label vector and a corresponding pseudo label weight vector to the probability vector based on the optimal transmission matrix and the label carried by the first eigenvector;

[0165] A seventh calculation unit, configured to calculate a category probability vector based on the probability vector and the corresponding pseudo label vector;

[0166] The eighth calculation unit is used to calculate the information transfer loss based on the pseudo label vector, the pseudo label weight vector and the category probability vector to obtain a fifth loss.

[0167] It should be noted that, regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0168] Embodiment 3:

[0169] Corresponding to the above method embodiment, a semi-supervised medical image segmentation device is also provided in this embodiment. The semi-supervised medical image segmentation device described below and the semi-supervised medical image segmentation method described above can refer to each other.

[0170] Figure 5 8 is a block diagram of a semi-supervised medical image segmentation device 800 according to an exemplary embodiment. Figure 5 As shown, the semi-supervised medical image segmentation device 800 may include: a processor 801 and a memory 802. The semi-supervised medical image segmentation device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0171] The processor 801 is used to control the overall operation of the semi-supervised medical image segmentation device 800 to complete all or part of the steps in the above-mentioned semi-supervised medical image segmentation method. The memory 802 is used to store various types of data to support the operation of the semi-supervised medical image segmentation device 800, and these data may include, for example, instructions for any application or method operating on the semi-supervised medical image segmentation device 800, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules can be keyboards, mice, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the semi-supervised medical image segmentation device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 can include: Wi-Fi module, Bluetooth module, NFC module.

[0172] In an exemplary embodiment, the semi-supervised medical image segmentation device 800 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned semi-supervised medical image segmentation method.

[0173] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned semi-supervised medical image segmentation method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 802 including program instructions, and the above-mentioned program instructions can be executed by the processor 801 of the semi-supervised medical image segmentation device 800 to complete the above-mentioned semi-supervised medical image segmentation method.

[0174] Embodiment 4:

[0175] Corresponding to the above method embodiment, a readable storage medium is also provided in this embodiment. The readable storage medium described below and the semi-supervised medical image segmentation method described above can refer to each other.

[0176] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the semi-supervised medical image segmentation method of the above method embodiment.

[0177] The readable storage medium may specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or other readable storage medium that can store program codes.

[0178] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0179] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A semi-supervised medical image segmentation method, characterized in that: include: Acquire medical images within a historical time period, where the medical images include a first image and a second image, where the first image is an image carrying a label, and the second image is an image not carrying a label; Acquire a target medical image, where the target medical image is an image without a label; Training a preset semi-supervised learning model based on the medical image, and stopping the training when the number of training times meets a set condition to obtain a semi-supervised image segmentation model; The target medical image is input into the semi-supervised image segmentation model to obtain a predicted segmentation result of the target medical image.

2. The semi-supervised medical image segmentation method according to claim 1, characterized in that , based on the medical image, the preset semi-supervised learning model is trained, and when the number of training times meets the set conditions, the training is stopped to obtain a semi-supervised image segmentation model, wherein the semi-supervised learning model includes a teacher branch and a student branch, including: Inputting each medical image into the teacher branch to obtain a target feature map and a first prediction map; Inputting each medical image into the student branch to obtain a second prediction map; Performing conversion processing on the first prediction map and the second prediction map to obtain a first probability map and a second probability map; Calculate the Di ce loss based on the second probability map of the first image and the first image to obtain a first loss; Calculate a cross entropy loss based on a second prediction image of the first image and the first image to obtain a second loss; Based on the first prediction images of all second images and the second prediction images of the second images, calculating the mean square error loss and the second-order central moment consistency loss to obtain a third loss and a fourth loss; Calculating information transfer loss based on the target feature map and the second probability map of the second image to obtain a fifth loss; Obtaining the comprehensive loss based on the first loss, the second loss, the third loss, the fourth loss and the fifth loss; Adjusting the parameters of the student branch based on the comprehensive loss, and adjusting the parameters of the teacher branch based on an exponential moving average algorithm to obtain an updated semi-supervised learning model, wherein the semi-supervised learning model carries an update number; Selecting a plurality of initial semi-supervised learning models from the plurality of updated semi-supervised learning models, wherein the number of updates carried by the initial semi-supervised learning models is the same as a plurality of preset values; The evaluation indicators of multiple initial semi-supervised learning models are calculated and compared respectively, and the initial semi-supervised learning model with the best evaluation indicator is determined as the semi-supervised image segmentation model.

3. The semi-supervised medical image segmentation method according to claim 2, characterized in that ,Based on the first prediction map and the second prediction map corresponding to all the second images, the mean square error loss and the second-order central moment consistency loss are calculated to obtain the third loss and the fourth loss, including: respectively calculating the centroid positions in the first probability map of the second image and the second probability map of the second image to obtain a first position and a second position; Calculate the second-order central moment of the teacher branch based on the first position to obtain a first second-order central moment; Calculate the second-order central moment of the student branch based on the second position to obtain a second second-order central moment; The norms of the first second-order central moment and the second second-order central moment are calculated to obtain the fourth loss.

4. The semi-supervised medical image segmentation method according to claim 3, characterized in that , based on the target feature map and the second probability map of the second image, calculating the information transfer loss to obtain the fifth loss, including: Determine a plurality of target pixel features from the annotated features in the target feature map, wherein the target pixel features are pixel features in the prediction map having the same predicted value and label value; Store multiple target pixel features and corresponding label values ​​in a memory bank; Sampling the memory library to obtain a first feature vector, wherein the first feature vector includes all types of pixel features, the number of each type of pixel features in the first feature vector is the same, and the first feature vector is a vector carrying a label; Performing random sampling processing on unlabeled features in the target feature map to obtain a second feature vector; Performing random sampling processing on the second probability map of the second image to obtain a probability vector, wherein the probability vector and the vector in the second feature vector correspond to the same pixel position one by one; Calculate the transmission matrix of the first eigenvector and the second eigenvector based on an optimal transmission algorithm to obtain an optimal transmission matrix; Based on the optimal transmission matrix and the label carried by the first eigenvector, assigning a pseudo label vector and a corresponding pseudo label weight vector to the probability vector; Calculate a category probability vector based on the probability vector and the corresponding pseudo label vector; Based on the pseudo label vector, the pseudo label weight vector and the category probability vector, the information transfer loss is calculated to obtain the fifth loss.

5. A semi-supervised medical image segmentation device, characterized in that: include: A first acquisition unit, configured to acquire medical images within a historical time period, wherein the medical images include a first image and a second image, wherein the first image is an image carrying a label, and the second image is an image not carrying a label; A second acquisition unit is used to acquire a target medical image, where the target medical image is an image without a label; A training unit, used for training a preset semi-supervised learning model based on the medical image, and stopping the training when the number of training times meets a set condition to obtain a semi-supervised image segmentation model; The first input unit is used to input the target medical image into the semi-supervised image segmentation model to obtain a predicted segmentation result of the target medical image.

6. The semi-supervised medical image segmentation device according to claim 5, characterized in that: The training unit comprises: A second input unit, used for inputting each medical image into the teacher branch to obtain a target feature map and a first prediction map; A third input unit, used for inputting each medical image into the student branch to obtain a second prediction map; A conversion processing unit, used for performing conversion processing on the first prediction map and the second prediction map to obtain a first probability map and a second probability map; A first calculation unit, configured to calculate a Di ce loss based on a second probability map of the first image and the first image to obtain a first loss; A second calculation unit, configured to calculate a cross entropy loss based on a second prediction image of the first image and the first image to obtain a second loss; A third calculation unit is used to calculate a mean square error loss and a second-order central moment consistency loss based on the first prediction images of all second images and the second prediction images of the second images, so as to obtain a third loss and a fourth loss; a fourth calculation unit, configured to calculate an information transfer loss based on the target feature map and the second probability map of the second image to obtain a fifth loss; A first obtaining unit, configured to obtain the comprehensive loss based on the first loss, the second loss, the third loss, the fourth loss and the fifth loss; An adjustment unit, configured to adjust the parameters of the student branch based on the comprehensive loss and adjust the parameters of the teacher branch based on an exponential moving average algorithm to obtain an updated semi-supervised learning model, wherein the semi-supervised learning model carries an update number; A screening unit, used to screen out a plurality of initial semi-supervised learning models from a plurality of updated semi-supervised learning models, wherein the number of updates carried by the initial semi-supervised learning models is the same as a plurality of preset values; The first determination unit is used to respectively calculate and compare the evaluation indicators of multiple initial semi-supervised learning models, and determine the initial semi-supervised learning model with the best evaluation indicator as the semi-supervised image segmentation model.

7. The semi-supervised medical image segmentation device according to claim 6, characterized in that: The third computing unit comprises: A fifth calculation unit, configured to respectively calculate the centroid position in the first probability map of the second image and the second probability map of the second image to obtain a first position and a second position; A second obtaining unit is used to calculate the second-order central moment of the teacher branch based on the first position to obtain a first second-order central moment; A third obtaining unit is used to calculate the second-order central moment of the student branch based on the second position to obtain a second second-order central moment; The sixth calculation unit is used to calculate the norm of the first second-order central moment and the second second-order central moment to obtain the fourth loss.

8. The semi-supervised medical image segmentation device according to claim 7, characterized in that: The fourth calculation unit comprises: A second determining unit is used to determine a plurality of target pixel features from the annotated features in the target feature map, wherein the target pixel features are pixel features whose predicted values ​​and label values ​​are the same in the prediction map; A storage unit, used to store multiple target pixel features and corresponding label values ​​in a memory bank; a fourth obtaining unit, configured to perform sampling processing on the memory library to obtain a first feature vector, wherein the first feature vector includes all kinds of pixel features, the number of each kind of pixel features in the first feature vector is the same, and the first feature vector is a vector carrying a label; A first random sampling unit is used to perform random sampling processing from unlabeled features in the target feature map to obtain a second feature vector; A second random sampling unit is used to perform random sampling processing on the second probability map of the second image to obtain a probability vector, wherein the probability vector and the vector in the second feature vector have a one-to-one correspondence with the same pixel position; a fifth obtaining unit, configured to calculate a transmission matrix of the first eigenvector and the second eigenvector based on an optimal transmission algorithm to obtain an optimal transmission matrix; an allocating unit, configured to allocate a pseudo label vector and a corresponding pseudo label weight vector to the probability vector based on the optimal transmission matrix and the label carried by the first eigenvector; A seventh calculation unit, configured to calculate a category probability vector based on the probability vector and the corresponding pseudo label vector; An eighth calculation unit is used to calculate the information transfer loss based on the pseudo label vector, the pseudo label weight vector and the category probability vector to obtain the fifth loss.

9. A semi-supervised medical image segmentation device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the semi-supervised medical image segmentation method according to any one of claims 1 to 4 when executing the computer program.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the semi-supervised medical image segmentation method according to any one of claims 1 to 4 are implemented.

Images