Semi-supervised medical image segmentation method combining feature similarity and reliable region enhancement

Abstract

This invention provides a semi-supervised medical image segmentation method combining feature similarity and reliable region enhancement. The method includes employing a dual-network model as the base network to perform preliminary feature extraction and model training on input labeled and unlabeled images. The deep features and segmentation features extracted from the dual-network model are then fed into a feature similarity module. In this module, a self-supervised method is used to mine deep feature information from the unlabeled data, associating the ability of deep features to predict ground truth labels with the segmentation features, thus mapping the similarity relationships of deep features onto the segmentation features. Based on the dual-network model, the advantages of mutual learning between the two networks are combined to obtain a high-confidence network model, forming a three-network model to generate pseudo-labels. A reliable region selection module then filters out reliable pseudo-labels to further constrain the dual-network model. This application can fully utilize unlabeled data information and efficiently utilize all pseudo-label information, improving model accuracy.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and more specifically to a semi-supervised medical image segmentation method that combines feature similarity and reliable region enhancement. Background Technology

[0002] Semantic segmentation is a crucial fundamental research area. In recent years, with continuous breakthroughs in semantic segmentation methods, such as FCN, GAN, and Vision Transformer, it has been widely applied to various downstream tasks, with medical image segmentation being a significant application area. Modern medical imaging technology allows for the complete scanning of slices of many organs, pathologies, and tissues; however, due to their complex representations, identification, segmentation, and annotation require highly specialized knowledge. Therefore, training a good segmentation model becomes crucial when only a limited amount of labeled data is available. To address this, researchers have proposed semi-supervised learning, currently falling into two main categories: methods based on consistent regularization and methods based on entropy minimization.

[0003] Semi-supervised learning trains models using only a small amount of labeled data and some unlabeled data. In 2013, Dong-Hyun et al. proposed a self-training method that uses ground truth labels and pseudo-labels to train the network. In 2017, Antti et al. proposed the Mean Teacher method to maintain consistency in predictions for the same image after different perturbations, thus achieving consistency regularization. In 2020, Yassine et al. proposed the CCT method, which achieves consistency constraints by perturbing the decoder. Among the recently proposed methods: 1) CPS (Cross-Pseudo-Label Supervision) constructs a dual-network model structure, simultaneously training and implementing the generation and utilization of pseudo-labels in the same stage, using pseudo-labels for cross-constraints, thus achieving both the utilization of pseudo-label information and the combination of consistency constraints; 2) GuidedMix-Net proposes a hybrid guided network to utilize labeled data information to guide the learning of unlabeled data; 3) UCC proposes uncertain guided cross-head joint training; 4) USRA proposes an "undifferentiated subclass regularization network," designing a threshold loss function for optimization; 5) RECO designs its network by utilizing the effective information of adjacent pixels and the relative information of different categories. These all provide new ideas for semi-supervised semantic segmentation and can achieve good results.

[0004] The methods described above typically begin by training a preliminary model using labeled data for supervised learning. Then, unlabeled data is combined with the preliminary model to extract usable information from the unlabeled images, such as generating pseudo-labels or using consistency in the results for training. However, with limited labeled data, the pseudo-labels generated by the model contain a large amount of noise, which can cause the model to learn in an undesirable direction. Therefore, reducing this noise has become an important task in semi-supervised learning. Current methods such as ST++, CCT, and CRST embody this idea. They primarily handle pseudo-labels in two ways: 1) directly using all pseudo-labels, but the large amount of noise in the pseudo-labels interferes with model learning; 2) only retaining pseudo-labels with high confidence, which leads to the discarding of effective information from low-confidence parts, failing to fully utilize all pseudo-label information. Therefore, the unlabeled data information extracted by existing methods has two main shortcomings: 1) unlabeled data information is usually not fully mined; 2) the obtained pseudo-label information contains a large amount of noise, making it impossible to efficiently utilize all pseudo-label information. Summary of the Invention

[0005] To address the technical problems of existing semi-supervised learning methods for medical image segmentation, such as the inability to fully extract unlabeled data and the presence of significant noise in the obtained pseudo-label information, which prevents efficient utilization of all pseudo-label information, this invention provides a semi-supervised medical image segmentation method that combines feature similarity and reliable region enhancement.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A semi-supervised medical image segmentation method combining feature similarity and reliable region enhancement includes:

[0008] A dual-network model is used as the base network to perform preliminary feature extraction and model training on the input labeled and unlabeled images.

[0009] The deep features and segmentation features extracted from the dual-network model are used to assist in segmentation. Specifically, the deep features and segmentation features extracted from the dual-network model are fed into the feature similarity module. In the feature similarity module, a self-supervised method is used to mine the deep feature information of the unlabeled data. The ability of the deep features to predict the ground truth label is associated with the segmentation features, so that the similarity relationship of the deep features is mapped to the segmentation features, thereby achieving the effect of consistency constraint between the deep features and the segmentation results, thus making full use of the deep feature information of the unlabeled data.

[0010] Based on the dual-network model, the advantages of mutual learning between the two networks are combined to obtain a high-confidence network model, forming a three-network model to generate pseudo-labels. Then, the reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels. The reliable pseudo-labels are then used to constrain the dual-network model again, thereby increasing the weight of reliable regions, reducing pseudo-label noise interference, and efficiently utilizing all pseudo-label information. The high-confidence network model and the reliable region filtering module together form a reliable region enhancement module.

[0011] Furthermore, specifically, the process of inputting the deep features and segmentation features extracted from the dual-network model into the feature similarity module includes:

[0012] The deep features and segmentation features in the dual-network model decoder are passed into the feature similarity module through a mapper.

[0013] Furthermore, the deep features and segmentation features in the decoder are selected from the U-net basic network. Specifically, the middle three layers of the U-net basic network are selected as deep features, and the top layer of the U-net basic network is selected as segmentation features.

[0014] Furthermore, the step of mining deep feature information of unlabeled data through a self-supervised method in the feature similarity module, and associating the ability of deep features to predict ground truth labels with segmentation features, so that the similarity relationship of deep features is mapped to segmentation features, includes:

[0015] First, in the feature similarity module, the deep features F of the two networks are calculated using the following formula. i Similarity between and segmentation features S i Similarity between

[0016]

[0017] in, They respectively represent in the feature space Two feature tensors with C channels and spatial dimensions (H,W) and (M,N); It is a feature similarity measure used to represent the feature similarity between two feature tensors f and g at positions (h,w) and (m,n) respectively, and when f = g, it represents the similarity between the same feature tensor at (h,w) and (m,n);

[0018] Then, based on the similarity between the deep features of their respective networks... The similarity between segmentation features is constrained by the correlation loss function. The correlation loss function table method is as follows:

[0019]

[0020] Where b is a hyperparameter used to adjust the distribution of similarity relationships, i∈{1,2};

[0021] Then, the similarity measure between the deep features F1 obtained through the first network decoder and the deep features F2 obtained through the second network decoder in the dual-network model is calculated. And measure this similarity The similarity relation is used to constrain the similarity measure between segmentation features S1 and S2. Thus, a hybrid similarity constraint is obtained in the dual-network model where the segmentation features and deep features of the two networks are similar. The loss function of the hybrid similarity constraint is as follows:

[0022]

[0023] Furthermore, based on the aforementioned dual-network model, a high-confidence network model is derived by integrating the advantages of mutual learning between the two networks to form a three-network model for generating pseudo-labels. Then, a reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels, including:

[0024] First, based on the trained dual-network model, calculate the accuracy on the validation set. Then, select the model that performs best on the validation set across all training epochs. Next, dynamically update the parameters of the high-confidence network model f(θ3) using the parameters of the best model according to the following formula:

[0025]

[0026] Where, θ i Let α(θ) represent the parameters of the current iteration model i, and let θ3 be randomly initialized. i ) represents the accuracy of the current iterative model i on the validation set, where i∈{1,2,3};

[0027] Next, the updated high-confidence network model f(θ3) is used to generate pseudo-labels, and then the reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels according to the following formula:

[0028]

[0029] in, This is a reliable pseudo-label for the i-th pixel in the entire image. Predict the overall entropy value of pixels for an unlabeled image; q i (c) represents Qi The value at channel c; This represents the softmax result of the i-th pixel obtained by the decoder of the high-confidence network model for the unlabeled image, where C is the number of channels; ε represents the value at the top α percentile of the entropy value after sorting the overall entropy values ​​of the input unlabeled image; NULL represents pixels that need to be ignored.

[0030] Furthermore, the overall entropy value of the predicted pixels for the unlabeled image is calculated using the following formula:

[0031]

[0032] Furthermore, the loss function for further constraining the dual-network model using the reliable pseudo-labels is as follows:

[0033]

[0034]

[0035] Where ω is the weight ratio of the reliable region enhancement module, and P i u The predicted segmentation result for model i. For reliable pseudo-labels of the entire image in the reliable region enhancement module, Let Dsc represent the loss of model i with reliable pseudo-labels.

[0036] Compared with existing technologies, the semi-supervised medical image segmentation method combining feature similarity and reliable region enhancement provided by this invention has the following advantages:

[0037] 1. This invention proposes a Feature Similarity Reliable Region Enhancement Network (FSRENet) that includes a feature similarity module and a reliable region enhancement module. It is a network that utilizes feature similarity relationships in semi-supervised learning and combines reliable region enhancement ideas with high accuracy.

[0038] 2. This invention proposes a Feature Similarity Module (FSM), which can be effectively combined with current semi-supervised models and can map the similarity relationship of deep features to the segmentation results, so that the similarity relationship between segmentation features is consistent with the similarity relationship between deep features. In this way, the ability of deep feature similarity to predict ground truth labels can be utilized, making fuller use of the information of unlabeled data and effectively improving the accuracy of the model.

[0039] 3. This invention proposes a Reliable Region Enhancement (REM) module, which can make full use of all pseudo-label information and reduce the impact of pseudo-label noise, thereby improving the accuracy of the model.

[0040] 4. Through ablation experiments and comparative experiments, this invention demonstrates that it proposes a highly effective semi-supervised learning model for medical image segmentation, and can achieve better segmentation results. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the feature similarity reliable region enhancement network model provided by the present invention.

[0042] Figure 2 This is a schematic diagram of selecting deep features and segmentation features in the U-net network provided by the present invention.

[0043] Figure 3 This is a schematic diagram of the reliable region enhancement network provided by the present invention.

[0044] Figure 4 This is a visual illustration comparing the prediction results of the feature similarity reliable region enhancement network model provided by this invention with those of various existing methods on 10% ACDC labeled data. Detailed Implementation

[0045] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below with reference to specific illustrations.

[0046] Please refer to Figure 1 As shown, this invention provides a semi-supervised medical image segmentation method that combines feature similarity and reliable region enhancement, comprising the following steps:

[0047] A dual-network model is used as the base network to perform preliminary feature extraction and model training on the input labeled and unlabeled images.

[0048] The deep features and segmentation features extracted from the dual-network model are used to assist in segmentation. Specifically, the deep features and segmentation features extracted from the dual-network model are fed into the feature similarity module. In the feature similarity module, a self-supervised method is used to mine the deep feature information of the unlabeled data. The ability of the deep features to predict the ground truth label is associated with the segmentation features, so that the similarity relationship of the deep features is mapped to the segmentation features, thereby achieving the effect of consistency constraint between the deep features and the segmentation results, thus making full use of the deep feature information of the unlabeled data.

[0049] Based on the dual-network model, the advantages of mutual learning between the two networks are combined to obtain a high-confidence network model, forming a three-network model to generate pseudo-labels. Then, the reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels. The reliable pseudo-labels are then used to constrain the dual-network model again, thereby increasing the weight of reliable regions, reducing pseudo-label noise interference, and efficiently utilizing all pseudo-label information. The high-confidence network model and the reliable region filtering module together form a reliable region enhancement module.

[0050] As a specific implementation method, the overall structure of the feature similarity reliable region enhancement network model proposed in this invention is as follows: Figure 1 As shown, the network model consists of two data streams and five modules. The two data streams include a labeled data stream (indicated by the red arrow) providing accurate information, and an unlabeled data stream (indicated by the black arrow) providing information to be mined. The five modules include an input module for receiving labeled and unlabeled images, a DoubleNet module (also known as a dual-network model) for cross-constraints and feature mapping of unlabeled data, a prediction module for providing validation information, a feature similarity module (FSM) for mining deep feature information from unlabeled data, and a reliable region enhancement module (REM) for efficiently utilizing all pseudo-label information. In the overall network model structure, labeled data serves as the primary source of effective information. This invention directly utilizes a dual-network model (each network consisting of an encoder and a decoder) to perform loss calculations on the segmentation results and ground truth (GT) for optimization. For unlabeled data, this invention not only uses the segmentation results generated by decoders Decoder1 and Decoder2 as pseudo-labels to constrain the dual-network model, achieving cross-constraints to provide information about unlabeled data, but also uses a projector to feed the deep feature information F extracted by the dual-network model and the segmentation feature information S into the feature similarity module. The similarity information of the deep features constrains the segmentation features, thereby combining the ability of deep features to predict the ground truth with the segmentation results, mining the deep feature information of the unlabeled data, and optimizing its segmentation results. Furthermore, to effectively utilize all pseudo-label information, this invention integrates the advantages of mutual learning between the two networks, designing a high-confidence network model to generate reliable pseudo-labels and further constrain the dual-network model. This achieves the goals of enhancing the weights of reliable regions, reducing pseudo-label noise interference, and utilizing all pseudo-label information. By combining these two methods, the present invention obtains a more accurate and reliable network model.

[0051] To link deep features with segmentation results, this invention first extracts deep features F from decoder i through mapper i. i and segmentation features S i The features are then fed into the feature similarity module. In a specific embodiment, the deep features and segmentation features in the decoder are selected from the existing U-net base network. Since U-net is the base network, its decoder features consist of five layers from bottom to top. Specifically, this invention selects the middle three layers (Feat-layer1, Feat-layer2, Feat-layer3) of the U-net base network as deep features and the top layer (Seg-layer) of the U-net base network as segmentation features, as detailed below. Figure 2 As shown.

[0052] As a specific embodiment, the step of mining deep feature information of unlabeled data through a self-supervised method in the feature similarity module, and associating the ability of deep features to predict ground truth labels with segmentation features, so that the similarity relationship of deep features is mapped to segmentation features, includes:

[0053] First, in the feature similarity module, the deep features F of the two networks are calculated using the following formula. i Similarity between and segmentation features S i Similarity between

[0054]

[0055] in, They respectively represent in the feature space Two feature tensors with C channels and spatial dimensions (H,W) and (M,N); It is a feature similarity measure used to represent the feature similarity of two feature tensors f and g at positions (h,w) and (m,n) respectively, and when f = g, it represents the similarity of the same feature tensor at (h,w) and (m,n).

[0056] Then, based on the similarity between the deep features of their respective networks... The similarity between segmentation features is constrained by the correlation loss function. The correlation loss function table method is as follows:

[0057]

[0058] Where b is a hyperparameter used to adjust the distribution of similarity relationships, i∈{1,2}; when the similarity of deep features... If the value is positive (or negative), then the corresponding similarity in the segmentation feature is... It should also be a positive (or negative) value, equivalent to the following: if two points at different positions in the deep features are similar (or dissimilar), then the corresponding two points at different positions in the segmentation features should also be similar (or dissimilar); however, and Cannot pass directly Constraints, therefore through Constrained by positive and negative values when If positive, then the constraint is applied. It approaches 1, and conversely, it is -1. However, due to the problem of collinearity, its minimum value is constrained to 0.

[0059] Then, the similarity measure between the deep features F1 obtained by the first network decoder (Decoder1) and the deep features F2 obtained by the second network decoder (Decoder2) in the dual-network model is calculated. And measure this similarity The similarity relationship is used to constrain the similarity measurement between segmentation features S1 and S2. Thus, a mixed similarity constraint (Mixsim) is obtained in the dual-network model, where the segmentation features and deep features of the two networks are consistent. Compared with the individual similarity constraints (Ahasim) of the two networks, it further embodies the idea of ​​consistent regularization. The loss function of the mixed similarity constraint is as follows:

[0060]

[0061] In the dual-network model, the same unlabeled image enters two models with different initializations and is cross-constrained using all the pseudo-labels generated by each model. This reflects both the idea of ​​entropy minimization and the idea of ​​consistent regularization. However, since the deep features obtained by the same image through different initialization networks are also different, in order to further combine the idea of ​​consistency constraints, this invention designed the above-mentioned consistency constraint loss that combines deep features and segmentation features.

[0062] Therefore, the overall loss function of the feature similarity module consists of three parts: the similarity constraints of the two individual networks and the mixed similarity constraints. The overall loss function is specifically expressed as follows:

[0063]

[0064] As a specific embodiment, a simplified diagram of the reliable region enhancement idea in a dual-network model is shown below. Figure 3(The gray background represents the dual-network model). Based on this idea, this invention designs a reliable region enhancement network. Extracting the reliable region is crucial; therefore, obtaining a high-performing model is a key step. Since the dual-network models continuously constrain and learn from each other during training, their performance dynamically changes. To integrate the advantages of both, a high-confidence network model f(θ3) is designed.

[0065] As a specific embodiment, based on the dual-network model, a high-confidence network model is derived by integrating the advantages of mutual learning between the two networks to form a three-network model to generate pseudo-labels. Then, a reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels, including:

[0066] First, based on the trained dual-network model, calculate the accuracy on the validation set. Then, select the model that performs best on the validation set across all training epochs. Next, dynamically update the parameters of the high-confidence network model f(θ3) using the parameters of the best model according to the following formula:

[0067]

[0068] Where, θ i Let α(θ) represent the parameters of the current iteration model i, and let θ3 be randomly initialized. i ) represents the accuracy of the current iterative model i on the validation set, i∈{1,2,3}.

[0069] After obtaining the high-confidence network model f(θ3), the overall entropy value of the predicted pixels of the unlabeled image is calculated using the following formula:

[0070]

[0071] Where, q i (c) represents Q i The value at channel c; This represents the softmax result of the i-th pixel obtained by the decoder of the high-confidence network model for the unlabeled image, where C is the number of channels.

[0072] Next, the calculated overall entropy values ​​are sorted, and the pixels with the highest entropy values ​​(α) are selected as reliable pixels to enhance reliable regions. Therefore, the updated high-confidence network model f(θ3) is used to generate pseudo-labels. Specifically, this invention uses the segmentation results generated by the decoder Decoder3 in the high-confidence network model f(θ3) as pseudo-labels. Then, the reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels according to the following formula:

[0073]

[0074] in, This is a reliable pseudo-label for the i-th pixel in the entire image. Predict the overall entropy value of pixels for an unlabeled image; q i (c) represents Q i The value at channel c; This represents the softmax result of the i-th pixel obtained by the decoder of the high-confidence network model for the unlabeled image, where C is the number of channels; ε represents the value at the top α percentile of the entropy values ​​after sorting the overall entropy values ​​of the input unlabeled image; NULL represents pixels that need to be ignored. Because the model has low accuracy at the beginning and becomes more accurate during training, α gradually increases with the number of iterations in this invention, starting from the minimum value α. min to the maximum value α max .

[0075] As a specific embodiment, after obtaining reliable pseudo-labels for each pixel in the entire image, since the confidence of the initial pseudo-labels is low and cannot be directly used for optimization, the weight ω of the pseudo-label confidence gradually increases with the increase of the number of iterations. The loss function of the reliable region enhancement module, which is used to constrain the dual network model again through the reliable pseudo-labels, is as follows:

[0076]

[0077]

[0078] Where ω is the weight ratio of the reliable region enhancement module, and P i u The predicted segmentation result for model i. The reliable pseudo-label for the entire image in the reliable region enhancement module can be obtained by calculating the pseudo-label for each pixel in the entire image. Let Dsc represent the loss of model i with reliable pseudo-labels.

[0079] In summary, the objective equation of the segmentation method proposed in this invention mainly consists of two parts: a supervised loss on labeled images and an unsupervised loss on unlabeled images. Labeled images are directly subjected to cross-entropy constraints and Dsc loss with the ground truth labels through a dual-network model, and the loss function is shown below:

[0080]

[0081]

[0082] in, The cross-entropy loss represents the difference between the predicted segmentation result of model i and the ground truth label. Let represent the Dsc loss between the predicted segmentation result of model i and the ground truth label, where i∈{1,2}.

[0083] Unlabeled images not only require cross-constraints after obtaining predicted segmentation results through a dual-network model, but also need to be jointly constrained by a feature similarity module and a reliable region enhancement module. The loss function is as follows:

[0084]

[0085]

[0086] in, From equation (4), From equation (8).

[0087] Therefore, similar to traditional semi-supervised learning, the overall loss function of the network in this invention is composed of labeled data loss. and unlabeled data loss The composition, and its overall loss function are shown in the following equation:

[0088]

[0089] Where λ is the weight that controls the loss of unlabeled data.

[0090] To verify the effectiveness of the FSRENet network as a whole and its individual modules, this invention conducted ablation experiments on 10% of the labeled ACDC dataset, testing the Ahasim and Mixsim methods in the feature similarity module and the reliable region enhancement module. This invention also found that Dsc loss can further improve the model's accuracy; therefore, ablation experiments were conducted on unlabeled data using cross-entropy loss and Dsc loss, and the experimental results are shown in Table 1 below.

[0091] Table 1

[0092]

[0093]

[0094] As shown in Table 1, the Feature Similarity Module and Reliable Region Enhancement Module proposed in this invention can be effectively combined and complement each other. When all modules are combined, FSRENet can improve the mDsc by 3.3% on the benchmark method. Furthermore, the model performance improves to varying degrees after adding Ahasim and Mixsim, indicating that the Feature Similarity Module (FSM) can effectively mine the information of unlabeled images, and constraining segmentation features with the similarity relationship of deep features is effective. Moreover, the hybrid feature consistency constraint can further realize the idea of ​​consistent regularization. Adding the Reliable Region Enhancement Module (REM) improves the performance on the benchmark method by 1.4%, indicating that REM can better utilize all pseudo-label information and effectively suppress noise, further enhancing the model's performance.

[0095] The FSRENet method was compared with nine existing methods, including Mean Teacher (MT), Adversarial Network (AD), Deep CoTraining (DCT), Entropy Minimization (EM), Uncertainty Aware Mean Teacher (UAMT), Interpolation Consistency Training (ICT), Regularized Dropout (RD), Cross Pseudo Supervision (CPS), and Cross Teaching Between CNN Transformer (CTBCT). The results of the comparative experiments on the ACDC dataset, using 5%, 10%, 20%, and 50% of the labeled data for three categories, are shown in Table 2 below.

[0096] Table 2

[0097]

[0098]

[0099] The table shows that FSRENet demonstrates a significant advantage in the mDsc metric across different labeled data ratios. This is attributed to the synergistic effect of the FSM and REM modules. FSM utilizes information more deeply, while the REM module further optimizes the information, resulting in more accurate segmentation across all results. Furthermore, FSRENet exhibits high accuracy across all three evaluation metrics with 5% and 10% labeled data. However, as the amount of labeled data increases, its advantage in Hd95 and ASD metrics is gradually overtaken by other methods, though it still outperforms the baseline method. This is likely because with limited labeled data, the additional model performance improvement primarily relies on the information from unlabeled data and the accuracy of pseudo-labels. As labeled data increases, the reliability of pseudo-labels also increases, and the noise contained in pseudo-labels decreases. Therefore, this demonstrates that FSRENet can fully extract information from unlabeled data even with limited labeled data and efficiently utilize all information while effectively reducing the impact of pseudo-label noise.

[0100] To provide a more detailed comparison of FSRENet with other methods, this paper also presents the results of comparative experiments on the three categories using 10% of the ACDC labeled dataset, as shown in Table 3 below. This table shows that FSRENet demonstrates superior performance in the detailed classification of all three categories when only a small amount of labeled data is available. This indicates that FSRENet is not only superior in classifying a single category but also possesses good generalization ability.

[0101] Table 3

[0102]

[0103]

[0104] To more intuitively demonstrate the advantages of this invention, the prediction results of the model trained on 10% labeled data and applied to the test set are also visualized, such as... Figure 4 As shown in the figure, the FSRENet method of this invention, compared with the other nine methods, can more finely segment the edge parts and has better performance in segmentation results, and can perform better segmentation when faced with noise.

[0105] Compared with existing technologies, the semi-supervised medical image segmentation method combining feature similarity and reliable region enhancement provided by this invention has the following advantages:

[0106] 1. This invention proposes a Feature Similarity Reliable Region Enhancement Network (FSRENet) that includes a feature similarity module and a reliable region enhancement module. It is a network that utilizes feature similarity relationships in semi-supervised learning and combines reliable region enhancement ideas with high accuracy.

[0107] 2. This invention proposes a Feature Similarity Module (FSM), which can be effectively combined with current semi-supervised models and can map the similarity relationship of deep features to the segmentation results, effectively improving the accuracy of the model.

[0108] 3. This invention proposes a Reliable Region Enhancement (REM) module, which can make full use of all pseudo-label information and reduce the impact of pseudo-label noise, thereby improving the accuracy of the model.

[0109] 4. Through ablation experiments and comparative experiments, this invention demonstrates that it proposes a highly effective semi-supervised learning model for medical image segmentation, and can achieve better segmentation results.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A semi-supervised medical image segmentation method combining feature similarity and reliable region enhancement, characterized in that, include: A dual-network model is used as the base network to perform preliminary feature extraction and model training on the input labeled and unlabeled images. The deep features and segmentation features extracted from the dual-network model are used to assist in segmentation. Specifically, the deep features and segmentation features extracted from the dual-network model are fed into the feature similarity module. In the feature similarity module, a self-supervised method is used to mine the deep feature information of the unlabeled data. The ability of the deep features to predict the ground truth label is associated with the segmentation features, so that the similarity relationship of the deep features is mapped to the segmentation features, thereby achieving the consistency constraint between the deep features and the segmentation results, thus making full use of the deep feature information of the unlabeled data. Based on the aforementioned dual-network model, a high-confidence network model is obtained by integrating the advantages of mutual learning between the two networks, forming a three-network model to generate pseudo-labels. Then, a reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels. These reliable pseudo-labels are then used to further constrain the dual-network model, thereby increasing the weight of reliable regions, reducing pseudo-label noise interference, and efficiently utilizing all pseudo-label information. The high-confidence network model and the reliable region filtering module together constitute a reliable region enhancement module. Specifically, the process of inputting the deep features and segmentation features extracted from the dual-network model into the feature similarity module includes: The deep features and segmentation features in the dual-network model decoder are passed into the feature similarity module through a mapper; The step of mining deep feature information of unlabeled data through a self-supervised method in the feature similarity module, and associating the ability of deep features to predict ground truth labels with segmentation features, so that the similarity relationship of deep features is mapped to segmentation features, includes: First, in the feature similarity module, the deep features F of the two networks are calculated using the following formula. i Similarity between and segmentation features S i Similarity between in, They respectively represent in the feature space Two feature tensors with C channels and spatial dimensions (H,W) and (M,N); It is a feature similarity measure used to represent the feature similarity between two feature tensors f and g at positions (h,w) and (m,n) respectively, and when f = g, it represents the similarity between the same feature tensor at (h,w) and (m,n); Then, based on the similarity between the deep features of their respective networks... The similarity between segmentation features is constrained by the correlation loss function. The correlation loss function is expressed as follows: Where b is a hyperparameter used to adjust the distribution of similarity relationships, i∈{1,2}; Then, the similarity measure between the deep features F1 obtained through the first network decoder and the deep features F2 obtained through the second network decoder in the dual-network model is calculated. And measure this similarity The similarity relation is used to constrain the similarity measure between segmentation features S1 and S2. Thus, a hybrid similarity constraint is obtained in the dual-network model where the segmentation features and deep features of the two networks are similar. The loss function of the hybrid similarity constraint is as follows: Based on the aforementioned dual-network model, a high-confidence network model is derived by integrating the advantages of mutual learning between the two networks to form a three-network model for generating pseudo-labels. Then, a reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels, including: First, based on the trained dual-network model, calculate the accuracy on the validation set. Then, select the model that performs best on the validation set across all training epochs. Next, dynamically update the parameters of the high-confidence network model f(θ3) using the parameters of the best model according to the following formula: Where, θ i Let α(θ) represent the parameters of the current iteration model i, and let θ3 be randomly initialized. i ) represents the accuracy of the current iterative model i on the validation set, where i∈{1,2,3}; Next, the updated high-confidence network model f(θ3) is used to generate pseudo-labels, and then the reliable region filtering module filters the pseudo-labels generated by the high-confidence network model to select reliable pseudo-labels according to the following formula: in, This is a reliable pseudo-label for the i-th pixel in the entire image. Predict the overall entropy value of pixels for an unlabeled image; q i (c) represents Q i The value at channel c; This represents the softmax result of the i-th pixel obtained by the decoder of the high-confidence network model for the unlabeled image, where C is the number of channels; ε represents the value at the first α percentile of the entropy values ​​after sorting the overall entropy values ​​of the input unlabeled image; NULL represents pixels that need to be ignored. The loss function used to further constrain the dual-network model using the reliable pseudo-labels is as follows: Where ω represents the weight percentage of the reliable region enhancement module. The predicted segmentation result for model i. For reliable pseudo-labels of the entire image in the reliable region enhancement module, Let Dsc represent the loss of model i with reliable pseudo-labels.

2. The semi-supervised medical image segmentation method combining feature similarity and reliable region enhancement according to claim 1, characterized in that, The deep features and segmentation features in the decoder are selected from the U-net basic network. Specifically, the middle three layers of the U-net basic network are selected as deep features, and the top layer of the U-net basic network is selected as segmentation features.

3. The semi-supervised medical image segmentation method combining feature similarity and reliable region enhancement according to claim 1, characterized in that, The overall entropy value of the predicted pixels for the unlabeled image is calculated using the following formula:

Images