A semi-supervised medical image segmentation method based on joint feature consistency of difficult pixels and effective images

Abstract

The application discloses a kind of semi-supervised medical image segmentation methods based on difficult pixel and effective image joint feature consistency, comprising: loading medical image data is preprocessed operation, wherein training set is divided into marked image and unmarked image;Medical image data input UFC semi-supervised twin network, network includes student model, teacher model, difficult pixel feature selector and class feature extractor, student model and teacher model are all included extractor and classifier, and weight sharing exists between student model and teacher model;For marked image, input student model and obtain output, calculate the output of student model and the supervision loss of label;For unmarked image, respectively input student model and teacher model and obtain output, and use the output of teacher model to detect the output of student model as consistency loss;Using the weighted optimization model parameters of supervision loss and consistency loss, and output after optimization multiple times, complete medical image segmentation.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and pattern recognition, specifically relating to a semi-supervised medical image segmentation method based on the joint feature consistency of difficult pixels and effective images. Background Technology

[0002] Medical image segmentation is a crucial task in computer vision and medical image analysis, aiming to accurately locate and segment various organs or lesions in medical images. Currently, deep learning-based medical image segmentation methods can be broadly categorized into two types: CNN-based methods and Transformer-based methods. CNN-based methods, such as CE-Net, 3D-UX-Net, LKAU-Net, SGU-Net, and PHNet, utilize hierarchical representations to obtain local feature representations of medical images. These methods typically introduce functional modules such as pyramid feature fusion, attention mechanisms, and depthwise separable convolutions to enhance the network's feature representation capabilities. Transformer-based medical image segmentation methods, such as TransUNet, UNETR, FAT-Net, ConvFormer, and FCT, process global information of the image through self-attention mechanisms, allowing the network to better capture long-range dependencies between pixels, thereby improving the accuracy and robustness of medical image segmentation. Although CNNs and Transformers are widely used in medical image segmentation, due to limited labeled data, their good performance is primarily achieved in fully supervised environments.

[0003] Conversely, semi-supervised learning can address the problem of scarce labeled data by utilizing a small amount of labeled data and a large amount of unlabeled data. Consistency learning, adversarial learning, self-training, and contrastive learning methods are widely used in the semi-supervised field. Among these, consistency learning methods are primarily used in semi-supervised medical image segmentation. Their core idea is to enhance pixel-level consistency in the network output through different perturbations. For example, UA-MT is an uncertainty-aware self-ensemble model that uses Monte Carlo Dropout to eliminate unreliable predictions. Furthermore, TCSM_v2 is a transform consistency self-ensemble model that utilizes transform consistency including rotation, flipping, and scaling operations. Additionally, FixMatch uses a simplified consistency regularization combined with pseudo-labels, forcing the model to produce consistency only for high-confidence predictions from image pairs augmented with strong and weak data. These methods typically eliminate predictions with high uncertainty, leading to damage to the structured semantic information of medical images, and they do not emphasize learning challenging pixels.

[0004] Representation learning refers to the process of learning useful high-level representations from raw data. Contrastive learning is generally considered a typical technique in representation learning, aiming to learn data representations by narrowing the distance between similar sample pairs while widening the distance between dissimilar sample pairs. Contrastive learning-based methods, such as MOCO, SimCLR, and SwAV, initially emerged in the field of self-supervised learning, but due to their superior performance in learning representations, they were quickly introduced into the field of semi-supervised medical image segmentation. The core idea of ​​these methods is to consider pixels with the same semantics as positive samples, while pixels with different semantics are considered negative samples. Although these methods have achieved better results by using contrastive learning, they often neglect image-level representation learning, thus affecting the quality of image segmentation. Summary of the Invention

[0005] To address the problems in existing technologies, this invention provides a semi-supervised medical image segmentation method based on the joint feature consistency of difficult pixels and effective images. This method improves the segmentation quality of consistency learning for challenging pixels in the semi-supervised medical field, enhances the learning ability of consistency learning for the overall semantic information of the image, and thus effectively improves the quality of semi-supervised medical image segmentation.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes:

[0007] Step 1: Load medical image data for preprocessing, where the training set is divided into labeled images and unlabeled images;

[0008] Step 2: Input medical image data into the UFC semi-supervised Siamese network. The UFC semi-supervised Siamese network includes a student model, a teacher model, a hard pixel feature selector, and a class feature extractor. Both the student and teacher models include extractors and classifiers, and weights are shared between the student and teacher models. For labeled images, the student model is input and its output is obtained. The supervised loss between the student model's output and the label is calculated. For unlabeled images, the student and teacher models are input separately and their outputs are obtained. The output of the teacher model is used to detect the output of the student model as the consistency loss.

[0009] Step 3: Optimize the model parameters using a weighted average of supervised loss and consistency loss, and output the optimized results after multiple optimizations to complete the medical image segmentation.

[0010] Furthermore, the preprocessing operation also includes data augmentation processing, which includes weak enhancement and strong enhancement. Weak enhancement includes one or more of random horizontal flipping, random rotation, and random cropping, while strong enhancement includes CutMix, Gaussian blur, and color dithering.

[0011] Furthermore, the medical image data includes M labeled images. and their corresponding tags and N unlabeled images When X U When weakly augmented, the unlabeled image is output by the teacher model, and backpropagation is not performed to update the network parameters; when X U When subjected to strong enhancement, X U The student model output is pseudo-labeled by the teacher model using cross-entropy loss. Supervision is performed, and X is obtained from the features extracted by the network extractor based on the predicted probability of the network output. U The hard pixel vectors and class feature vectors are obtained, and then mapped to the latent feature space, followed by a joint feature consistency loss. Perform network optimization; X L The output of the student model is determined by the true label Y. L By using cross-entropy loss and Dice loss Supervision was conducted, and the total supervisory loss was... The specific losses are expressed as follows:

[0012]

[0013]

[0014]

[0015] in, It is HPFC loss. This is CFC loss, and the total loss is ω sup ω un_ce 、 and ω ufc These are the weights for different losses.

[0016] Furthermore, in the method, the batch size is initially set to B and the number of input channels to C. in A height of H and a width of W For unlabeled medical images, and These correspond to strongly enhanced and weakly enhanced images of unlabeled medical images, respectively; then, the extractors from the student model and the teacher model are used to extract... and Features z s and z w Secondly, predictions are obtained through the corresponding classifier. Where C is the number of categories; finally, the following are obtained: and Maximum prediction probability and the corresponding pseudo-tags and For each pixel x i (1≤i≤B×H×W), the corresponding predicted probabilities from the student model and the teacher model are respectively represented as and The corresponding pseudo-tags are represented as follows: and Based on the high-confidence output of the teacher model, simple pixels and difficult pixels are defined as follows:

[0017]

[0018] Where 0 ≤ τ ≤ 1.

[0019] Furthermore, the method first sets and These are the hard pixels from the student model and teacher model extractors, respectively. The feature vectors; then execute and After that, I obtained Where φ represents the mapping head and ψ represents the prediction head; then calculate and Cosine similarity between Where <·> represents the inner product, and ||·||2 represents the l2 norm; the hard pixel distance between the student model and the teacher model is measured again. Finally, the HPFC loss is: in It is the number of difficult pixels in a batch.

[0020] Furthermore, the effective image in the method is first defined as follows:

[0021]

[0022] in Medical images from the teacher model χ b (1≤b≤B) is the maximum predicted probability of the pixel in the d-th row and q-th column. For a 3D dataset, effective images are selected using 2D slices obtained from three different dimensions of the 3D features extracted from the network extractor. For an effective medical image χ... v ∈X valid After strong enhancement and weak enhancement, the corresponding features z are obtained from the corresponding extractors. (s,v) and z (w,v) Then the classifier of the teacher model will classify z. (w,v) Predicted as set up p (w,v) The prediction for class c, from z (s,v)and z (w,v) Extracting image-level class feature vectors Z from each image (s,v,c) and Z (w,v,c) The details are as follows:

[0023]

[0024] in and They are z (s,v) z (w,v) and p (w,v,c) The element in the d-th row and q-th column.

[0025] Furthermore, in the method, firstly In obtaining and The calculations obtained afterwards and cosine similarity Then the distance between the class feature vectors of the student model and the teacher model is Finally, the ICFC loss is defined as follows: in It is the number of valid images in a batch.

[0026] Furthermore, the method includes unlabeled medical images. The feature map is Where C′ is the output dimension of the network extractor, using three different sized receptive regions for segmenting targets of various sizes: regions area and region For each sensing region block The information captured from a specific region in z is as follows:

[0027]

[0028] From the above formula, it can be seen that each sensing region block ρ has a total of (2 ρ+1 -1) 2 For each region-level feature, PCFC uses the same feature extraction operations and loss calculation to obtain... The total CFC loss is

[0029] Furthermore, in the method, the student model is applied to x. e ∈χ easy The prediction is expressed as The corresponding pseudo-label for the teacher model is By applying the cross-entropy loss function Consistency loss is defined as Where ε refers to the number of simple pixels in a batch.

[0030] Compared with existing technologies, this invention first classifies pixels in an image into two categories: easy pixels and difficult pixels, based on the confidence difference between the student model and the teacher model. Then, consistency learning is used in the latent feature space to impose strong constraints on difficult pixels, while consistency constraints are applied to the network's prediction results for easy pixels. Secondly, a class feature extractor is designed to extract effective image-level class feature vectors, which helps improve the network's global feature representation capability. This invention has the following advantages compared with existing technologies: 1. Unlike existing mainstream methods that treat all pixels in an image equally, this invention proposes a novel consistency learning strategy for difficult pixels in medical images, solving the problem of insufficient learning for challenging pixels. First, difficult pixels are selected based on the confidence difference between the student model and the teacher model. Then, they are mapped into the latent feature space to obtain better consistency constraints, thereby enhancing the model's ability to segment challenging regions and improving the model's robustness. 2. Unlike existing mainstream methods that remove pixels with high uncertainty, this invention proposes a novel image-level class feature consistency learning strategy. Without disrupting the structured semantic information of medical images, this invention utilizes the predicted probabilities of the teacher model to obtain image-level class feature vectors, thereby achieving effective learning of image-level semantic information. Experimental results show that on three publicly available datasets, the proposed UFC outperforms state-of-the-art (SOTA) methods, including ISIC (International Skin Imaging Collaboration), MoNuSeg (Multi-Organ Nuclei Segmentation), and LA (LeftAtrial). Furthermore, this invention, as a plug-and-play network module, can be directly integrated into any semi-supervised method with both student and teacher models. Attached Figure Description

[0031] Figure 1 This is a visual segmentation result of skin cancer images using the FixMatch method as an example in existing technologies;

[0032] Figure 2 This is a flowchart of the method of the present invention;

[0033] Figure 3 This is the overall network framework diagram of the method of the present invention;

[0034] Figure 4a It is a region / image selector diagram; Figure 4b It is a feature extractor graph; Figure 4c It is a difficult pixel mask acquisition image; Figure 4d It is a feature map;

[0035] Figure 5 This is a visualization comparison of the segmentation results of the method of this invention with existing mainstream methods on the ISIC and MoNuSeg datasets;

[0036] Figure 6 This is a visualization comparison of the segmentation results of the method of this invention with existing mainstream methods in the left atrial dataset LA. Detailed Implementation

[0037] The present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] While existing techniques have achieved success in medical imaging, they still face two major challenges. First, different regions in an image contain varying prior structural information, with regions possessing complex prior structures typically exhibiting greater segmentation challenges. However, most existing methods employ unweighted pixel-level consistency loss, ignoring the differences in pixel segmentation difficulty across different regions of a medical image, leading to significant underlearning of pixels within challenging regions. Second, the global prior structure of medical images, which contains image-level semantic information, is crucial, but most existing methods avoid including pixels with high uncertainty in consistency learning, thereby compromising the overall semantic integrity of the image and impairing the model's ability to learn image-level semantic information from the image itself. See also Figure 1 The FixMatch method is used as an example to demonstrate the segmentation results of a skin cancer image. Green, yellow, and red pixels represent predictions of high confidence. Medium confidence level and low confidence in This represents the probability of a pixel being predicted. The confidence map shows that pixels in different regions of the image have varying segmentation difficulties. Furthermore, it's clear that a large number of medium-confidence pixels and some low-confidence pixels are correctly predicted, but only high-confidence pixels are used for consistency learning, which severely damages the structured semantic information of the image.

[0039] Therefore, this invention proposes a semi-supervised medical image segmentation method based on the joint feature consistency of difficult pixels and effective images. It mainly involves theoretical knowledge of deep learning techniques, data augmentation techniques, consistency learning techniques, and representation learning techniques. This method can be well applied to any semi-supervised medical image segmentation method with student and teacher models, addressing the problems of current mainstream semi-supervised medical image segmentation methods neglecting the learning difficulty of different pixels and the overall semantic information of the image. The aim is to improve the segmentation quality of consistency learning for challenging pixels in the semi-supervised medical field, enhance the learning ability of consistency learning for the overall semantic information of the image, and thus effectively improve the quality of semi-supervised medical image segmentation. This invention employs a joint feature consistency (UFC) strategy based on difficult pixels and effective images for semi-supervised medical image segmentation. UFC includes HardPixel Feature Consistency Learning (HPFC) and Class Feature Consistency Learning (CFC). HPFC focuses on challenging difficult pixels, while CFC fully utilizes the structured global semantic information in the effective region and the effective image to achieve better medical image segmentation quality.

[0040] See Figure 2 The method of the present invention includes:

[0041] Step 1: Load medical image data for preprocessing, where the training set is divided into labeled images and unlabeled images;

[0042] Step 2: Input medical image data into the UFC semi-supervised Siamese network. The UFC semi-supervised Siamese network includes a student model, a teacher model, a hard pixel feature selector, and a class feature extractor. Both the student and teacher models include extractors and classifiers, and weights are shared between the student and teacher models. For labeled images, the student model is input and its output is obtained. The supervised loss between the student model's output and the label is calculated. For unlabeled images, the student and teacher models are input separately and their outputs are obtained. The output of the teacher model is used to detect the output of the student model as the consistency loss.

[0043] Step 3: Optimize the model parameters using a weighted average of supervised loss and consistency loss, and output the optimized results after multiple optimizations to complete the medical image segmentation.

[0044] The present invention will now be illustrated by specific embodiments.

[0045] (1) Medical Dataset Sources and Preprocessing: ① The ISIC dataset comes from the 2018 International Skin Imaging Collaboration Organization Skin Lesion Segmentation Challenge. The training set contains 2594 images, and the validation set contains 100 images. This dataset contains different types of skin lesions and different resolutions. To improve the computational efficiency of different models, all images were resized to 256×192. 10% (259 images) and 20% (519 images) of the training set were randomly selected as labeled data, and the remainder were used as unlabeled data. ② The MoNuSeg dataset comes from the 2018 Multi-Organ Nucleus Segmentation Challenge. The training set contains 30 histopathological images, and the validation set contains 14 images, all of which are H&E stained tissue images of multiple organs. The edges of the target and background tissues in these images are blurred and have low contrast. First, each original 1000×1000 kernel image was cropped into 16 non-overlapping 250×250 sub-images. Then, all sub-images were resized to a uniform resolution of 256×256. For semi-supervised learning, one image (containing 16 sub-images) and two images (containing 32 sub-images) are randomly selected from the training set as labeled data, and the remaining images are used as unlabeled data. ③ The LA dataset comes from the 2018 Left Atrium Segmentation Challenge and consists of 100 images with a resolution of 0.625×0.625×0.625mm. 3 The dataset consists of 3D MR images. 80 images were used for training, and 20 images were used for validation. During training, the left atrial data was randomly cropped into small data volumes of 112×112×80. For semi-supervised learning, 10% (8 images) and 20% (16 images) were used as labeled data, and the remainder were used as unlabeled data. To improve consistency learning, the medical images were augmented using the following data augmentation techniques: weak augmentation was used for all images; random horizontal flipping was used for the ISIC dataset, random horizontal flipping and random rotation (90 degrees, 180 degrees, and 270 degrees) were used for the MoNuSeg dataset, and random cropping was used for the LA dataset. Furthermore, three identical strong augmentations were used for all images: CutMix, Gaussian blur, and color jitter.

[0046] (2) The network training process of the method of the present invention: as follows Figure 3 As shown, a semi-supervised Siamese network using UFC is proposed. The student model and the teacher model share the same architecture and parameters. By reducing... and The weighted sum is used to update the student model. Especially in the calculation... In this process, the student model and the teacher model first need to map both the difficult pixel features and the effective image features to the latent feature space, and then use cosine similarity to measure their distance.

[0047] Suppose a medical image dataset contains M labeled images. and their corresponding tags and N unlabeled images When X U When weakly augmented, the network acts as a teacher; the output of the weak augmentation does not undergo backpropagation to update the network parameters. When X... U When the output becomes unreliable due to strong enhancements, the network acts as a student. X U The student model output is pseudo-labeled by the teacher model using cross-entropy loss. Supervision is performed. Simultaneously, based on the predicted probability from the network output, X is obtained from the features extracted by the network extractor. U The process involves generating hard pixel vectors and class feature vectors. These vectors are then mapped to a latent feature space and subjected to a joint feature consistency loss. Perform network optimization. Additionally, X L The output of the student model is determined by the true label Y. L By using cross-entropy loss and Dice loss Supervision was conducted, and the total supervisory loss was... The specific loss can be expressed as:

[0048]

[0049]

[0050]

[0051] in This is the HPFC loss proposed in this invention. This is the CFC loss proposed in this invention, with a total loss of... ω sup ω un_ce 、 and ω ufc Weights for different losses.

[0052] (3) HPFC in this invention: Since the segmentation accuracy of challenging pixels largely determines the model's segmentation ability, HPFC focuses on difficult pixels with high uncertainty in the low-dimensional latent feature space, thereby enhancing the model's ability to segment challenging regions. Let the batch size be B and the number of input channels be C. in A height of H and a width of W For unlabeled medical images, and These correspond to their strongly enhanced and weakly enhanced images, respectively. Then, the extractors from the student model and the teacher model are used to extract... Features z s z w Then, the prediction is obtained through the corresponding classifier. Where C is the number of categories. Finally, obtain... and Maximum prediction probability and their corresponding pseudo-tags and For each pixel x i (1≤i≤B×H×W), the corresponding predicted probabilities from the student model and the teacher model are respectively represented as and The corresponding pseudo-tags are represented as follows: and Compared to the student model, the teacher model's high-confidence output is more reliable. Therefore, based on the teacher model's high-confidence output, easy pixels and hard pixels are defined as follows:

[0053]

[0054] Where 0 ≤ τ ≤ 1. Difficult pixel mask acquisition is as follows: Figure 4c As shown, the difficult pixel mask is obtained by selecting pixels that simultaneously satisfy high confidence in the teacher model and low confidence in the student model as difficult pixels for HPFC. Based on the clustering assumption, most simple pixels in the teacher and student models have similar semantic information, so they are very close in the feature space. Therefore, performing consistency learning on simple pixels in the feature space leads to redundant computation. In contrast, difficult pixels with higher uncertainty in the student model exhibit significant differences in the semantic information extracted from the teacher and student models, making them the focus of consistency learning. Let... and These are the hard pixels from the student model and teacher model extractors, respectively. The feature vector, and then execute To obtain Where φ represents the mapping head and ψ represents the prediction head. The specific structures of φ and ψ are shown in the appendix. Figure 4d As shown, the feature mapping maps the acquired class features and hard pixel features to the latent space using the mapping head and prediction head designed in this invention to perform UFC (HPFC+CFC). Then, calculation... and Cosine similarity between Where <·> represents the inner product, and ||·||2 represents the l2 norm. Next, the hard pixel distance between the student model and the teacher model is measured. Finally, the proposed HPFC loss is in It is the number of difficult pixels in a batch.

[0055] (4) The CFC of the present invention: HPFC focuses on the segmentation quality of difficult pixels, significantly improving the model's ability to segment challenging regions. However, the definition of difficult pixels depends on the high confidence output from the teacher model. Furthermore, to address the shortcomings of pixel-level consistency learning strategies in improving the global feature representation of the network, this invention proposes image-level class feature consistency learning (ICFC). First, the effective image is defined as follows:

[0056]

[0057] in Medical images from the teacher model χ b (1≤b≤B) is the maximum predicted probability of the pixel in the d-th row and q-th column. For a 3D dataset, valid images are selected using 2D slices obtained from three different dimensions of the 3D features extracted from the network extractor. For a valid medical image χ... v ∈X valid After strong enhancement and weak enhancement, the corresponding features z are obtained from the corresponding extractors. (s,v) and z (w,v) Subsequently, the classifier of the teacher model will... (w,v) Predicted as set up p (w,v) The prediction for class c is given. The prediction from the teacher model is more reliable than that from the student model, based on z. (s,v) and z (w,v) Extracting image-level class feature vectors Z from each image (s,v,c) and Z (w,v,c) The details are as follows:

[0058]

[0059] in and They are z (s,v) z (w,v) and p (w,v,c) The element in the d-th row and q-th column is shown. It can be seen that the class feature vector contains the features of all pixels in the valid image, thus preserving the structured semantic information of the medical image. Although the valid image also contains low-confidence pixels, their number is far less than the number of high-confidence pixels, therefore these low-confidence pixels do not affect the effectiveness of the final extracted image-level class feature vector.

[0060] first, In obtaining and What was obtained afterward. Next, calculation. and cosine similarity Then, the distance between the class feature vectors of the student model and the teacher model is... Finally, the proposed ICFC loss definition is as follows: in This refers to the number of valid images in a batch. Optical medical images, such as skin cancer and cell nucleus images, typically exhibit significant spatial variability. Therefore, this invention employs a region-level class feature consistency learning method (PCFC) with a perceptual region adjustment strategy to enhance the model's perceptual ability to segment target regions, such as... Figure 4a and Figure 4b As shown, the region / image selector selects valid regions / images based on probability, and the class feature extractor performs class feature extraction on the selected valid regions / images to obtain image-level or region-level semantic information, performing CFC (Category-Focused Translation). Assuming an unlabeled medical image... The feature map is Where C′ represents the output dimension of the network extractor. To obtain more diverse feature information, three receptive regions of different sizes are used for segmentation targets of various sizes: regions Region 2∈ and region For each sensing region block The information captured from a specific region in z is as follows:

[0061]

[0062] From the above formula, it can be seen that each sensing region block ρ has a total of (2 ρ+1 -1) 2 Features at the region level. Similar to ICFC, PCFC uses the same feature extraction operations and similar loss calculations to obtain... Finally, the total CFC loss proposed in this invention is:

[0063] (5) Cross-entropy loss in consistency learning: By applying UFC after the extractor, strong feature representations can be effectively learned from unlabeled data, but UFC cannot directly optimize the classifier parameters. Furthermore, simple pixels exhibit high confidence in both the student and teacher models. Compared to the teacher model, the high-confidence predictions of the student model are more likely to be overconfident. Moreover, similar to difficult pixels, applying feature consistency learning to all simple pixels will significantly increase unnecessary computation. Therefore, this invention introduces a low-computation-cost consistency regularization method for simple pixels, which not only directly optimizes the classifier parameters but is also essential for learning simple pixels themselves. Applying the student model to x... e ∈χ easy The prediction is expressed as The corresponding pseudo-label for the teacher model is By applying the cross-entropy loss function Consistency loss is defined as Where ε refers to the number of simple pixels in a batch.

[0064] The effects of this invention can be further illustrated by the following experiments.

[0065] To verify the accuracy and superiority of the proposed method for medical image segmentation under semi-supervised conditions, the hardware experimental platform consisted of an NVIDIA GeForce RTX 3090Ti GPU with 24GB of VRAM; the software experimental platform utilized open-source software and frameworks such as PyTorch, Python, OpenCV, and NumPy. To fully utilize the VRAM, batch sizes were set to 32, 28, and 8, respectively, during training on the ISIC, MoNuSeg, and LA datasets. The Adam algorithm was used as the optimizer for all datasets. The learning rate was set to 0.001 for the ISIC and MoNuSeg datasets, and to 0.0005 for the LA dataset. and The loss weights are respectively set as ω sup =1, ω ufc =0.1. Weight of unsupervised loss. That is, as the number of training rounds increases, ω ufc The number of training rounds increases from 0 to 0.1, where t represents the current training round number. maxThis represents the total number of training rounds. In the definitions of simple and difficult pixels, this invention sets the threshold τ to a fixed value, i.e., τ = 0.95. In the definition of effective images / regions, the thresholds σ and η are also set to fixed values, i.e., σ = 0.8 and η = 90%. When comparing this invention with other methods, the Dice coefficient (DC) and Jaccard coefficient (JC) are used as the main evaluation metrics. For the ISIC and MoNuSeg datasets, accuracy (AC), specificity (SP), and sensitivity (SE) are also used. For the LA dataset, 95% Hausdorff distance (95HD) and average symmetric surface distance (ASD) are also used for performance evaluation.

[0066] For the ISIC and MoNuSeg datasets, this invention uses UNet++ as the segmentation network. For the LA dataset, this invention uses VNet as the segmentation network. Figure 3 The extractor in this context refers to all components except the classifier in the last layer of UNet++ / VNet, with 64 / 16 output channels. For large target segmentation tasks like skin cancer and left atrium, to reduce information redundancy and computational load during training, this invention downsamples the original feature sizes of 64×256×192 and 16×112×112×80 by a factor of two after the extractor, resulting in 64×128×96 and 16×56×56×40 respectively. Compared to skin cancer and left atrium, for small target segmentation tasks involving cell nuclei, accurate segmentation of each pixel is crucial; therefore, this invention does not employ downsampling but uses features of the original size. This invention is compared with several other mainstream methods, including MT, UA-MT, TCSM_v2, CPS, DTC, MC-Net, and ASE-Net. and FixMatch*, where FixMatch* indicates that FixMatch additionally uses the CutMix operation, all methods except The same segmentation network was used for all components except for DyBAC.

[0067] Segmentation visualization examples Figure 5 and Figure 6 As shown, Figure 5This document presents a visualization comparison of the segmentation results of the method described in this invention with other mainstream methods on the skin cancer dataset ISIC and the cell nucleus dataset MoNuSeg. Fully supervised training uses 100% labeled data, while label-only supervision uses only 259 / 2594 labeled data for skin cancer images and only 2 / 30 labeled data for cell nucleus images. Other methods are trained in a semi-supervised manner, with 259 / 2594 labeled data used for skin cancer images, 2 / 30 labeled data for cell nucleus images, and the remainder for unlabeled data. To better present the segmentation results of nuclei and cells, different colors are used for visualization. Green and red pixels represent predicted and ground truth labels, respectively, while yellow pixels represent the overlapping area between predicted and ground truth labels. Figure 6 This document presents a visualization comparison of the proposed method with other mainstream segmentation methods on the left atrial dataset (LA). Experimental results show that the proposed UFC outperforms state-of-the-art (SOTA) methods on three publicly available datasets: ISIC (International Skin Imaging Collaboration), MoNuSeg (Multi-Organ NucleiSegmentation), and LA (Left Atrial).

[0068] Table 1 compares the segmentation performance of the proposed method with that of current mainstream methods on the skin cancer dataset ISIC and the cell nucleus dataset MoNuSeg.

[0069]

[0070] Table 2 compares the segmentation performance of the proposed method and current mainstream methods on the left atrial dataset LA.

[0071]

[0072] Table 3 compares the effects of the method proposed in this invention when applied to other methods with student and teacher models.

[0073]

[0074] As shown in Tables 1 and 2, this invention generally outperforms other mainstream methods across all settings with different numbers of labeled images. For the ISIC dataset, particularly with training data of 259 / 2594 labels, this invention surpasses FixMatch* by 1.21% on DI. For the MoNuSeg dataset, particularly with training data of 2 / 30 labels, this invention surpasses TCSM_V2 by 1.07% on DI. For the LA dataset, particularly with training data of 8 / 80 labels, the method of this invention surpasses... 0.92%.

[0075] The joint feature consistency proposed in this invention is implemented based on the semi-supervised paradigm of FixMatch with strong and weak enhancements. As a plug-and-play network module, it can be directly applied to any semi-supervised method with teacher and student models. Therefore, this invention has been directly applied to semi-supervised methods such as MT, UA-MT, TCSM_v2, and ASE-Net, and ablation studies have been conducted on the MoNuSeg dataset, where only 1 / 30 of the training data was labeled. Table 3 shows the improvement in DI achieved by the above method, demonstrating a significant improvement.

[0076] This invention proposes a novel semi-supervised medical image segmentation method based on UFC (Unified Cross-Entropy). First, it distinguishes between easy and difficult pixels based on the confidence difference between the student and teacher models, focusing on difficult pixels by mapping them to a latent feature space for better consistency learning. Furthermore, it selects valid images using the output probability of the teacher model and obtains image-level class feature vectors from them. These vectors are then mapped to the latent feature space for image-level class feature consistency learning, thereby enhancing the model's ability to learn structured semantic information from the image itself. It not only focuses on the local segmentation quality of difficult pixels but also considers the global image-level segmentation quality of valid images by preserving the structured semantic information of medical images, significantly enhancing the feature representation capability of the segmentation network. Consistency regularization using cross-entropy loss is applied to easy pixels to directly optimize classifier parameters for higher-quality predictions, thus providing guidance for UFC and further improving the accuracy of medical image segmentation. Compared to current mainstream semi-supervised medical image segmentation methods, this invention demonstrates significant segmentation performance advantages on three publicly available medical image datasets.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semi-supervised medical image segmentation method based on the joint feature consistency of difficult pixels and effective images, characterized in that, include: Step 1: Load medical image data for preprocessing, where the training set is divided into labeled images and unlabeled images; The medical image data includes A tagged image and their corresponding tags and Unlabeled images ,when When weak enhancement is applied, the weakly enhanced unlabeled image is output by the teacher model, and backpropagation is not performed to update the network parameters; when When subjected to strong enhancement, The student model output is pseudo-labeled by the teacher model using cross-entropy loss. Supervision is performed, and probabilities are predicted based on the network output, while features extracted from the network extractor are obtained. The hard pixel vectors and class feature vectors are obtained, and then mapped to the latent feature space, followed by a joint feature consistency loss. Perform network optimization; The output of the student model is composed of real labels. By using cross-entropy loss and Dice loss Supervision was conducted, and the total supervisory loss was... The specific losses are expressed as follows: in, It is HPFC loss. This is CFC loss, and the total loss is , , ,and These are the weights for different losses; Step 2: Input medical image data into the UFC semi-supervised Siamese network. The UFC semi-supervised Siamese network includes a student model, a teacher model, a hard pixel feature selector, and a class feature extractor. Both the student and teacher models include extractors and classifiers, and weights are shared between the student and teacher models. For labeled images, the student model is input and its output is obtained. The supervised loss between the student model's output and the label is calculated. For unlabeled images, the student and teacher models are input separately and their outputs are obtained. The output of the teacher model is used to detect the output of the student model as the consistency loss. First set and These are the feature vectors of the hard pixels from the student model and teacher model extractors, respectively; then, the following is executed... ,and After that, I obtained ,in, Indicates the mapping header, This represents the prediction head; next, calculation is performed. and Cosine similarity between ,in Indicates the inner product. express Norm; measure the difficult pixel distance between the student model and the teacher model again. Finally, the HPFC loss is: ,in It is the number of difficult pixels in a batch; In the method, firstly In obtaining and The calculations obtained afterwards and cosine similarity Then the distance between the class feature vectors of the student model and the teacher model is Finally, the ICFC loss is defined as follows: ,in This is the number of valid images in a batch; The method includes unlabeled medical images. The feature map is ,in To define the output dimension of the network extractor, three receptive regions of different sizes are used for segmenting targets of various sizes: , and For each sensing region ,from The information captured for a specific area is as follows: As can be seen from the above formula, each sensing area Total For each region-level feature, PCFC uses the same feature extraction operations and loss calculation to obtain... The total CFC loss is ; Step 3: Optimize the model parameters using a weighted average of supervised loss and consistency loss, and output the optimized results after multiple optimizations to complete the medical image segmentation.

2. The semi-supervised medical image segmentation method based on joint feature consistency of difficult pixels and effective images according to claim 1, characterized in that, The preprocessing operation also includes data augmentation, which includes weak enhancement and strong enhancement. Weak enhancement includes one or more of random horizontal flipping, random rotation, and random cropping, while strong enhancement includes CutMix, Gaussian blur, and color dithering.

3. The semi-supervised medical image segmentation method based on joint feature consistency of difficult pixels and effective images according to claim 1, characterized in that, The method first sets the batch size to be... The number of input channels is Height is Width is of For unlabeled medical images, and These correspond to strongly enhanced and weakly enhanced images of unlabeled medical images, respectively; then, the extractors from the student model and the teacher model are used to extract... and Features and Secondly, predictions are obtained through the corresponding classifier. , ) ,in It is the number of categories; finally, each category is obtained. and Maximum prediction probability and the corresponding pseudo-tags and For each pixel The corresponding predicted probabilities from the student model and the teacher model are respectively represented as and The corresponding pseudo-tags are represented as follows: and Based on the high-confidence output of the teacher model, simple pixels and difficult pixels are defined as follows: in .

4. The semi-supervised medical image segmentation method based on joint feature consistency of difficult pixels and effective images according to claim 1, characterized in that, The effective image is first defined in the method as follows: in These are medical images from a teacher model. No. line, number The maximum predicted probability of column pixels, for a 3D dataset, uses 2D slices obtained from three different dimensions of 3D features extracted from the network extractor to select valid images. For a valid medical image... The features are obtained from the respective extractors after strong and weak enhancement. and Then the classifier of the teacher model will Predicted as ,set up express China to the first Class prediction, from and Extract image-level class feature vectors respectively and The details are as follows: in , and They are and The Middle line, number The elements of the column.

5. The semi-supervised medical image segmentation method based on joint feature consistency of difficult pixels and effective images according to claim 1, characterized in that, The method involves student models. The prediction is expressed as The corresponding pseudo-label for the teacher model is By applying the cross-entropy loss function Consistency loss is defined as ,in This refers to the number of simple pixels in a batch.

Images