Semi-supervised medical 3D image segmentation method and system based on stage similarity constraint

Abstract

The invention discloses a semi-supervised medical 3D image segmentation method and system based on stage similarity constraint, and the method comprises the steps: training an image segmentation model to segment a 3D image, and obtaining a segmentation result; when the segmentation model is trained, a sample group is input into the subnet A and the subnet B at the same time for feature extraction, and low-dimensional features extracted in the early stage and prediction results are obtained respectively; the early-stage feature similarity minimization module calculates the similarity based on the low-dimensional features and minimizes the similarity; a global similarity maximization module respectively calculates feature embedding of the subnet A and the subnet B and similarity matrixes of class prototypes, further calculates global similarity maximization loss and maximizes the similarity of the similarity matrixes of the subnet A and the subnet B; and finally, calculating the final model loss and carrying out gradient return. From the perspective of similarity constraint, loss functions based on similarity are respectively added in the early stage and the later stage of feature extraction of the model, so that the model learns knowledge which is also applicable to non-label data from label data.

Description

Technical Field

[0001] The present invention relates to the technical field of semi-supervised medical image processing, and in particular to a semi-supervised medical 3D image segmentation method and system based on stage similarity constraints. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Automatic segmentation of medical images is the basis and key component for developing effective computer-aided diagnosis (CAD) systems. Accurate segmentation results can quantitatively analyze the morphological properties of organ tissues and provide clinicians with valuable disease diagnosis information. With the development of deep learning, convolutional neural networks have achieved impressive improvements in a variety of automatic image segmentation applications. Despite the remarkable progress achieved in automatic medical image segmentation using deep learning, the scarcity of accurately annotated training data remains a significant obstacle to the widespread adoption of such techniques in clinical settings.

[0004] As a solution, the concept of semi-supervised segmentation was proposed to enable models to be trained using less labeled but abundant unlabeled data.

[0005] The dataset for semi-supervised medical image segmentation includes a small number of labeled images and a large number of unlabeled images. The task is to segment the unlabeled data and obtain better segmentation results by using the knowledge learned from the dataset containing a small number of labeled images and a large number of unlabeled images.

[0006] Medical 3D images not only have the characteristics of pixel-by-pixel annotation, which consumes a lot of manpower, but also have blurred edges and low regional contrast, which become obstacles to accurate segmentation. In recent years, semi-supervised learning methods have proposed a large number of methods to utilize unlabeled data in order to improve segmentation accuracy. In general, they can be roughly divided into two categories: consistency regularization and pseudo-labeling methods. Methods based on consistency regularization encourage the model to produce invariant predictions on the input image under small perturbations at the data, feature, and model levels. Specifically, they can add small perturbations to unlabeled samples and enforce consistency of model predictions on the original data and perturbed data, or directly use adversarial regularization on the entire unlabeled dataset to enforce similar prediction distributions. Pseudo-labeling-based methods mainly follow the self-training pipeline. The key to this method is to assign high-quality pseudo-labels to unlabeled data, then merge the labeled dataset with the pseudo-labeled dataset and retrain the model. In this case, low-quality pseudo-labels may have higher uncertainty, may contain more noise, and may potentially mislead the model during training. In addition, some recent methods try to combine the two to achieve good performance, that is, the model should produce similar pseudo-labels for the same unlabeled data when it is subjected to different degrees of perturbations. Although this method has achieved some results, it still does not fully consider the similarity difference between the features of labeled data and unlabeled data before and after feature extraction. In summary, the existing technology for solving the existing problems of semi-supervised medical 3D image segmentation is still imperfect and lacks a high-precision and efficient solution. Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a semi-supervised medical 3D image segmentation method and system based on stage similarity constraint. From the perspective of similarity constraint, an image segmentation model is constructed, and a similarity-based loss function is added in the early and late stages of feature extraction by the model, so that the model can learn knowledge on labeled data that is also applicable to unlabeled data.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention provides a semi-supervised medical 3D image segmentation method based on stage similarity constraint, comprising: Obtain labeled samples and unlabeled samples of medical 3D images and construct a sample group; An image segmentation model is constructed, and the sample group is input into the image segmentation model for training to obtain a trained image segmentation model; the image segmentation model includes parallel subnets A and B, and a pre-feature similarity minimization module and a global similarity maximization module are set; when the image segmentation model is trained, the sample group is simultaneously input into subnets A and B for feature extraction, and low-dimensional features extracted in the pre-feature and prediction results are obtained respectively; the pre-feature similarity minimization module calculates similarity based on the low-dimensional features extracted in the pre-feature and minimizes its similarity; the prediction results are embedded to obtain high-dimensional features in the post-feature extraction stage, and class prototypes and feature embeddings are obtained by screening; next, the global similarity maximization module calculates the feature embeddings of subnets A and subnet B and the similarity matrices of class prototypes respectively, and then calculates the global similarity maximization loss to maximize the similarity of the similarity matrices of subnets A and B; finally, the final model loss is calculated and the gradient is back-transferred; The medical 3D image to be segmented is segmented based on the trained image segmentation model to obtain an image segmentation result.

[0009] As a further technical solution, the early feature similarity minimization module calculates the cosine similarity of the low-dimensional features extracted in the early stage of subnet A and subnet B, and uses a loss function to minimize the low-dimensional feature similarity of subnet A and subnet B.

[0010] A further technical solution is to add sample self-attention and cross-sample attention between the encoder and decoder of the subnetwork B in sequence.

[0011] As a further technical solution, the image segmentation model adopts a dynamic pseudo-label generation strategy, selects a subnet for generating pseudo-labels based on the size of the supervised loss, and then obtains pseudo-labels corresponding to unlabeled samples.

[0012] In a further technical solution, the supervised loss is expressed as:

[0013] in, indicates that there is supervised loss, represents the loss weight, represents DIce loss, represents the true label of the labeled sample, represents the prediction result of subnet A on labeled samples, Represents the prediction results of subnet B on labeled samples.

[0014] As a further technical solution, the global similarity maximization module calculates the similarity of feature embedding and class prototype of subnet A and subnet B respectively as follows: Compare the pixel-by-pixel prediction categories of subnet A and subnet B for the same sample on unlabeled samples, and select pixels with the same predictions by subnet A and subnet B; For each predicted class, pixels with the same prediction are sorted according to the confidence score; Select the features of the top set number of pixels as unlabeled features; The cosine similarity between the unlabeled features and the selected class prototypes is calculated to obtain a similarity matrix.

[0015] In a further technical solution, the total loss function of the image segmentation model includes supervised loss and unsupervised loss, and the unsupervised loss is expressed as:

[0016] in, represents the similarity minimization loss, represents the unlabeled loss, represents the global similarity maximization loss, , , They represent the loss weights of different losses respectively.

[0017] In a second aspect, the present invention provides a semi-supervised medical 3D image segmentation system based on stage similarity constraint, comprising: A data acquisition module is configured to: acquire labeled samples and unlabeled samples of medical 3D images and construct a sample group; The model training module is configured to: construct an image segmentation model, input the sample group into the image segmentation model for training, and obtain a trained image segmentation model; the image segmentation model includes parallel subnets A and B, and sets a pre-feature similarity minimization module and a global similarity maximization module; when the image segmentation model is trained, the sample group is simultaneously input into subnets A and B for feature extraction, and low-dimensional features extracted in the pre-feature and prediction results are obtained respectively; the pre-feature similarity minimization module calculates similarity based on the low-dimensional features extracted in the pre-feature and minimizes its similarity; the prediction results are embedded to obtain high-dimensional features in the post-feature extraction stage, and class prototypes and feature embeddings are obtained by screening; next, the global similarity maximization module calculates the feature embeddings of subnets A and subnet B and the similarity matrices of class prototypes respectively, and then calculates the global similarity maximization loss to maximize the similarity of the similarity matrices of subnets A and subnet B; finally, the final model loss is calculated and the gradient is back-transferred; The model segmentation module is configured to segment the medical 3D image to be segmented based on the trained image segmentation model to obtain an image segmentation result.

[0018] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the semi-supervised medical 3D image segmentation method based on stage similarity constraint as described in the first aspect.

[0019] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the semi-supervised medical 3D image segmentation method based on stage similarity constraint as described in the first aspect are implemented.

[0020] One or more of the above technical solutions have the following beneficial effects: In terms of segmentation effect, this paper proposes for the first time a semi-supervised medical 3D image segmentation model based on stage similarity constraints. The two subnets make predictions for the same sample, which increases the diversity of model predictions. The early feature similarity minimization module further increases the diversity of model feature learning. The cross-sample attention establishes dense cross-sample correlations between a group of samples, realizing the migration of label prior knowledge to unlabeled data. The global similarity maximization module constrains the similarity matrices of the two subnets to remain consistent and regularizes the model. Compared with the traditional MT (Mean Teacher)-based method, the accuracy of model operation is improved.

[0021] In terms of practicality and scalability, the method is based on a pseudo-label method and consistency regularization, combined with a CNN network and an attention mechanism. In the semi-supervised medical 3D image segmentation method based on stage similarity constraints of the present invention, similarity-based loss functions are added in the early and late stages of feature extraction from the perspective of similarity, feature similarity minimization constraints are added in the early stage of feature extraction, and global feature similarity maximization constraints are added in the late stage of feature extraction, so that the model will learn knowledge on labeled data that is also applicable to unlabeled data.

[0022] In terms of computational efficiency, not all labeled and unlabeled pixels are fully correlated. On labeled data, features whose predictions from the two subnets are the same as the true labels are used to generate class prototypes, and a single memory bank is established for storage. When selecting unlabeled pixels and memory banks for full correlation calculations, only pixels with the same predictions from the two subnets are selected, which greatly improves computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0024] Figure 1is a structural diagram of an image segmentation model according to an embodiment of the present invention; Figure 2 This is a schematic diagram of cross-sample attention according to an embodiment of the present invention; Figure 3 is a schematic diagram of a global similarity maximization module according to an embodiment of the present invention; Figure 4 is an example diagram of the segmentation result of a medical 3D image according to an embodiment of the present invention; Figure 5 This is an example diagram of the true value of a semi-supervised medical 3D image based on stage similarity constraints in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0026] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0027] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0028] Embodiment 1 like Figure 1 As shown, this embodiment discloses a semi-supervised medical 3D image segmentation method based on stage similarity constraint, which includes the following steps: S1: Obtain labeled samples and unlabeled samples of medical 3D images and construct a sample group; In this embodiment, experiments are performed using datasets publicly available on the Internet, namely, The Left AtrialDataset and The NIH pancreas dataset.

[0029] S2: construct an image segmentation model, input the sample group into the image segmentation model for training, and obtain a trained image segmentation model; the image segmentation model includes parallel subnets A and B, and sets a pre-feature similarity minimization module and a global similarity maximization module; when the image segmentation model is trained, the sample group is simultaneously input into subnets A and B for feature extraction, and low-dimensional features extracted in the pre-feature and prediction results are obtained respectively; the pre-feature similarity minimization module calculates similarity based on the low-dimensional features extracted in the pre-feature and minimizes its similarity; the prediction results are embedded to obtain high-dimensional features in the post-feature extraction stage, and the class prototype and feature embedding are obtained by screening; next, the global similarity maximization module calculates the feature embedding and similarity matrix of the class prototype of subnets A and B respectively, and then calculates the global similarity maximization loss to maximize the similarity of the similarity matrix of subnets A and B; finally, the final model loss is calculated and the gradient is back-transferred; In this embodiment, the image segmentation model includes parallel subnets A and B, both of which are encoder-decoder structures, both of which are based on V-net but have different upsampling methods. On the basis of the backbone network, an early feature similarity minimization module and a global similarity maximization module with complementary performance are innovatively designed, which are located in the early and late stages of feature extraction, respectively, and a loss function is constructed from the perspective of similarity to constrain the training of the image segmentation model. In addition, the present invention selects a dynamic pseudo-label selection strategy in the selection of pseudo-labels, that is, compared to setting a complex screening strategy, a pseudo-label generation network is dynamically selected to reduce computational overhead while achieving more reliable label propagation.

[0030] The early feature similarity minimization module is located in the early stage of feature extraction. The low-dimensional features of the encoders in the two sub-networks after the second block are extracted as early features. By calculating the cosine similarity of the early features of the two sub-networks and using the loss function to minimize the similarity of the low-dimensional features of the two sub-networks, the two sub-networks can extract different features, thereby preventing the two sub-networks (sub-network A and sub-network B) from collapsing with each other and increasing the diversity of model learning features and predictions.

[0031] Specifically, Figure 1 As shown, the sample groups are input into subnet B and subnet A respectively. For labeled samples , extract its early features from subnet B , extract its previous features from subnet A , in the previous feature similarity minimization module, the minimum similarity between the two is calculated, and the loss is calculated ; For unlabeled samples , extract its previous features from subnet B , extract its previous features from subnet A , the loss is calculated as Next, and The average value of is used as the initial feature similarity minimization loss to participate in the final loss calculation.

[0032] The similarity loss calculation formula is expressed as:

[0033] in, represents the minimization loss of the previous feature similarity, represents the early features of the unlabeled samples or labeled samples extracted by subnet A, Represents the early features of the unlabeled samples or labeled samples extracted by subnet B.

[0034] Furthermore, in order to enable the image segmentation model to capture the global and local information in the input data in different ways and improve the feature expression ability of the model, the present invention sequentially adds sample self-attention and cross-sample attention between the encoder and decoder of subnet B to successively model the relationship in the sample and the relationship between samples, which are denoted as T1 and T2 respectively. Figure 2 Schematic diagram of cross-sample attention. This mechanism consists of two consecutive Transformer encoder layers, each of which includes a multi-head attention and an MLP block with layer normalization after each block. The first Transformer encoder layer performs intra-sample self-attention on the spatial dimension of each sample, while the second Transformer encoder layer performs inter-sample self-attention along the batch dimension to further realize information propagation between different samples. That is, along the sample dimension, pixels at the same spatial position in the sample are fed into the self-attention module to build cross-sample relationships. In this way, two self-attention modules installed sequentially along different dimensions are used to realize the attention between samples in a batch on the basis of realizing the attention of a single sample, thereby realizing efficient mutual attention calculation between all pixels. It should be noted that since the implementation of cross-sample attention requires more than one sample, the subnet with added cross-sample attention is not suitable for model verification and testing, so here cross-sample attention is only added in subnet B.

[0035] However, minimizing the feature similarity in the early stage may introduce too strong a perturbation to the model, so that the features extracted by the sub-network may contain less meaningful information for prediction, resulting in inconsistent and unreliable predictions of the two sub-networks. Therefore, it is necessary to ensure that the sub-networks make meaningful predictions. Specifically, for the labeled data, the true labels are used to supervise the training of the two sub-networks to generate semantically meaningful predictions. The DIce loss is calculated for the labeled data (samples), the true labels, and the predicted labels of the two sub-networks. and , represents the DIce loss of the true label, Represents the DIce loss of the predicted labels of the two subnets, and the supervised loss composed of the two as follows:

[0036] in, represents the true label of the labeled data, represents the prediction result of subnet A on labeled data, represents the prediction result of subnet B on labeled data, represents DIce loss, Represents the loss weight, which is used to balance the supervised loss.

[0037] For unlabeled data, the two subnets each generate a subnet pseudo label. The present invention uses a dynamic pseudo label generation strategy to take the accuracy of labeled data on the two subnets as an indicator of the segmentation accuracy of the two subnets, and selects the pseudo label generated by the subnet with higher segmentation accuracy as the final pseudo label, and constrains the prediction and final pseudo label generated by the subnet with poor accuracy. The corresponding loss function is constructed as follows:

[0038]

[0039]

[0040] in, represents the pseudo-label supervision loss, represents the pseudo-label supervision loss of subnet A, represents the pseudo-label supervision loss of subnet B, represents the mean square error loss, represents the final pseudo-label of the unlabeled data, represents the prediction result of subnet A on unlabeled data, Represents the prediction result of subnet B on unlabeled data. The dynamic pseudo-label generation strategy selects the subnet to generate pseudo-labels based on the size of the supervised loss and then obtains the pseudo-labels corresponding to the unlabeled samples. Compared with the fixed setting of a certain subnet as the network for generating pseudo-labels, this dynamic selection of subnets to generate the final pseudo-labels further avoids the problem of low quality of the generated pseudo-labels, which in turn causes the model to fall into cognitive bias.

[0041] Furthermore, the global similarity maximization module is as follows Figure 3The global similarity maximization module enables the model to learn from the features that both subnets make correct predictions in the labeled data, and further learns the information on the labeled data to improve the overall model segmentation effect.

[0042] The global similarity is calculated by the features obtained by passing the unlabeled data through the subnet and the class prototype representing the correct predicted features. A projection head is attached to each of the two subnets. After the unlabeled data passes through the subnet and then through the projection head, two sets of embeddings are obtained by sampling from the projected features extracted from the unlabeled samples. It should be noted that the embeddings are sampled from the embeddings corresponding to the same set of positions in the two subnets.

[0043] Unlike other methods that use clustering methods to determine class prototypes, since medical image segmentation is a binary classification problem, the present invention establishes a unified shared class prototype library for the two subnets to store class prototypes. When selecting class prototypes, the choice is to further make full use of labeled data, that is, to compare the predicted categories of the two subnets for the same data, and to select the features predicted by the two subnets in the labeled data that are the same as the true category of the image as the class prototype. However, as the training iteration cycle increases, the segmentation accuracy of the training set becomes higher and higher. It is impossible for the present invention to use each prototype that meets the above conditions as a class prototype to calculate the global similarity. Therefore, the present invention sets a fixed number of feature slots for the class prototype library. For each training sample, the class prototype library updates the feature slots corresponding to the labeled samples in the current small batch in a manner similar to the query. In the class prototype library, the present invention randomly samples a quantitative number of embeddings from each class to calculate the global similarity to increase its diversity.

[0044] In order to reduce the computational overhead of calculating global similarity, we cannot calculate the similarity between the embeddings of all unlabeled samples and the class prototypes of the corresponding categories. Here, we first compare the predicted categories of the two subnets for the same data pixel by pixel on the unlabeled samples, and select the pixels with the same predictions from the two subnets. Next, for each category, we sort the confidence scores of these pixels, and then select the features of the top i pixels as the sampled unlabeled features. Next, we calculate the cosine similarity between the top i pixels selected by the two subnets and the selected class prototypes. and Respectively represent the similarity matrix between subnet A, subnet B and class prototype, and the calculation method is as follows:

[0045] in, represents the number of prototype features sampled, Indicates the number of class prototypes, represents the temperature coefficient, represents unlabeled features, represents a high-quality class prototype. Next, the CE loss constraint is used and , while encouraging the two subnets to learn useful semantic information from the class prototype, and at the same time encourage the two subnets to produce consistent predictions, Represents the global similarity maximization loss, expressed as follows:

[0046] in, represents the number of sampled features, Denotes CE loss. CE loss is used to minimize the difference between the two similarity matrices to further align the predictions between the two subnetworks, reduce the noise in the pseudo-labels, and ultimately obtain a more accurate segmentation result.

[0047] Furthermore, the total loss function of the model is expressed as follows:

[0048] in, indicates that there is supervised loss, represents the unsupervised loss.

[0049] The total loss function consists of two parts: supervised loss and unsupervised loss. The supervised loss consists of three parts, which are expressed as follows:

[0050] in, represents the similarity minimization loss, represents the unlabeled loss, represents the global similarity maximization loss, , , They represent the loss weights of different losses respectively.

[0051] In this embodiment, the data path, model storage path, etc., as well as training parameters such as initialization, bias, regularization, initial learning rate, learning rate reduction method, optimization algorithm, number of iterations, and data enhancement method are used to achieve the training of the image segmentation model.

[0052] It should be noted that the structural difference between the two subnets is that only one subnet adds the cross-sample attention mechanism, and the upsampling methods of the two are also different, which further ensures the diversity of the predictions of the two subnets.

[0053] During the training process, each epoch randomly extracts N labeled samples from the training set of the dataset, and N is determined by the proportion of labeled samples in the entire training set. Each iteration extracts two samples from the selected labeled data and randomly extracts two samples from the remaining unlabeled data to form a new group of samples and send them to the two subnets respectively. After a group of samples are sent to the two subnets respectively, the early features of the two subnets for a group of samples are extracted for the calculation of the early feature minimization module. Next, the model selects the final pseudo-label based on the dice loss of the subnet on the labeled data and closes the prediction result and the final pseudo-label result. The prediction results generated by the two subnets are passed through the embedding module to obtain the later features, and then the class prototype and feature embedding are obtained. The similarity between the embedded features of the two subnets and the class prototype under the corresponding classification is calculated and calculated. . After calculating the final loss, the gradient is returned.

[0054] The model testing process is similar to the training process, including setting input images, using models, etc. In one embodiment, the input includes the test folder path, test model path, test model, number of test images, and test result output path. The final model obtained by training is tested on the entire test set, and the test result returns the average result of the test set. Finally, the test results are visualized to display the 3D medical image segmentation results generated by the model.

[0055] S3: Segment the medical 3D image to be segmented based on the trained segmentation model to obtain an image segmentation result.

[0056] In this embodiment, Figure 4 This is an example of the segmentation result of a medical 3D image to be segmented. Figure 5 is an example diagram of the real value of the medical 3D image to be segmented. By comparing the two, it can be seen that the method of this embodiment improves the segmentation result of the medical 3D image.

[0057] Embodiment 2 This embodiment discloses a semi-supervised medical 3D image segmentation system based on stage similarity constraint, including: A data acquisition module is configured to: acquire labeled samples and unlabeled samples of medical 3D images and construct a sample group; The model training module is configured to: construct an image segmentation model, input the sample group into the image segmentation model for training, and obtain a trained image segmentation model; the image segmentation model includes parallel subnets A and B, and sets a pre-feature similarity minimization module and a global similarity maximization module; when the image segmentation model is trained, the sample group is simultaneously input into subnets A and B for feature extraction, and low-dimensional features extracted in the pre-feature and prediction results are obtained respectively; the pre-feature similarity minimization module calculates similarity based on the low-dimensional features extracted in the pre-feature and minimizes its similarity; the prediction results are embedded to obtain high-dimensional features in the post-feature extraction stage, and class prototypes and feature embeddings are obtained by screening; next, the global similarity maximization module calculates the feature embeddings of subnets A and subnet B and the similarity matrices of class prototypes respectively, and then calculates the global similarity maximization loss to maximize the similarity of the similarity matrices of subnets A and subnet B; finally, the final model loss is calculated and the gradient is back-transferred; The model segmentation module is configured to segment the medical 3D image to be segmented based on the trained image segmentation model to obtain an image segmentation result.

[0058] Embodiment 3 The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of embodiment 1 when executing the program.

[0059] Embodiment 4 The purpose of this embodiment is to provide a computer-readable storage medium, a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method of embodiment 1 are performed.

[0060] The steps involved in the apparatus of the above embodiments 3 and 4 correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0061] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0063] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A semi-supervised medical 3D image segmentation method based on stage similarity constraint, characterized in that: include: Obtain labeled samples and unlabeled samples of medical 3D images and construct a sample group; An image segmentation model is constructed, and the sample group is input into the image segmentation model for training to obtain a trained image segmentation model; the image segmentation model includes a parallel subnet A and a subnet B, and a pre-feature similarity minimization module and a global similarity maximization module are set; when the image segmentation model is trained, the sample group is simultaneously input into subnet A and subnet B for feature extraction, and low-dimensional features extracted in the pre-feature and prediction results are obtained respectively; the pre-feature similarity minimization module calculates similarity based on the low-dimensional features extracted in the pre-feature and minimizes the similarity; the prediction result is embedded to obtain high-dimensional features in the post-feature extraction stage, and a class prototype and feature embedding are obtained by screening; Next, the global similarity maximization module calculates the feature embedding and similarity matrix of the class prototype of subnet A and subnet B respectively, and then calculates the global similarity maximization loss to maximize the similarity of the similarity matrix of subnet A and subnet B; finally, the final model loss is calculated and the gradient is back-propagated; The medical 3D image to be segmented is segmented based on the trained image segmentation model to obtain an image segmentation result.

2. The semi-supervised medical 3D image segmentation method based on stage similarity constraint according to claim 1, characterized in that: The early feature similarity minimization module calculates the cosine similarity of the low-dimensional features extracted in the early stage of subnet A and subnet B, and uses the loss function to minimize the low-dimensional feature similarity of subnet A and subnet B.

3. The semi-supervised medical 3D image segmentation method based on stage similarity constraint according to claim 1, characterized in that: Sample self-attention and cross-sample attention are added between the encoder and decoder of the subnetwork B in sequence.

4. The semi-supervised medical 3D image segmentation method based on stage similarity constraint according to claim 1, characterized in that: The image segmentation model adopts a dynamic pseudo-label generation strategy, selects a subnet to generate pseudo-labels based on the size of the supervised loss, and then obtains the pseudo-labels corresponding to the unlabeled samples.

5. The semi-supervised medical 3D image segmentation method based on stage similarity constraint according to claim 4, characterized in that: The supervised loss is expressed as: in, indicates that there is supervised loss, represents the loss weight, represents DIce loss, represents the true label of the labeled sample, represents the prediction result of subnet A on labeled samples, Represents the prediction results of subnet B on labeled samples.

6. The semi-supervised medical 3D image segmentation method based on stage similarity constraint according to claim 1, characterized in that: The global similarity maximization module calculates the similarity of feature embedding and class prototype of subnet A and subnet B respectively as follows: Compare the pixel-by-pixel prediction categories of subnet A and subnet B for the same sample on unlabeled samples, and select pixels with the same predictions by subnet A and subnet B; For each predicted class, pixels with the same prediction are sorted according to the confidence score; Select the features of the top set number of pixels as unlabeled features; The cosine similarity between the unlabeled features and the selected class prototypes is calculated to obtain a similarity matrix.

7. The semi-supervised medical 3D image segmentation method based on stage similarity constraint according to claim 1, characterized in that: The total loss function of the image segmentation model includes supervised loss and unsupervised loss, and the unsupervised loss is expressed as: in, represents the similarity minimization loss, represents the unlabeled loss, represents the global similarity maximization loss, , , They represent the loss weights of different losses respectively.

8. A semi-supervised medical 3D image segmentation system based on stage similarity constraints, characterized in that: include: A data acquisition module is configured to: acquire labeled samples and unlabeled samples of medical 3D images and construct a sample group; The model training module is configured to: construct an image segmentation model, input the sample group into the image segmentation model for training, and obtain a trained image segmentation model; the image segmentation model includes parallel subnets A and B, and sets a pre-feature similarity minimization module and a global similarity maximization module; when the image segmentation model is trained, the sample group is simultaneously input into subnets A and B for feature extraction, and low-dimensional features extracted in the pre-feature and prediction results are obtained respectively; the pre-feature similarity minimization module calculates similarity based on the low-dimensional features extracted in the pre-feature and minimizes the similarity; the prediction results are embedded to obtain high-dimensional features in the post-feature extraction stage, and the class prototype and feature embedding are obtained by screening; Next, the global similarity maximization module calculates the feature embedding and similarity matrix of the class prototype of subnet A and subnet B respectively, and then calculates the global similarity maximization loss to maximize the similarity of the similarity matrix of subnet A and subnet B; finally, the final model loss is calculated and the gradient is back-propagated; The model segmentation module is configured to segment the medical 3D image to be segmented based on the trained image segmentation model to obtain an image segmentation result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the semi-supervised medical 3D image segmentation method based on stage similarity constraint as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the semi-supervised medical 3D image segmentation method based on stage similarity constraint as described in any one of claims 1 to 7 are implemented.

Images