Semi-supervised 3D image segmentation method based on quality-driven cross-learning

Abstract

This invention provides a semi-supervised 3D image segmentation method based on quality-driven cross-learning, comprising: acquiring a 3D image to be segmented; inputting the 3D image to be segmented into a pre-trained segmentation model for segmentation to obtain a segmentation result; wherein, the segmentation model includes a dual-network structure of a general network and an evidence network, which are complementary but have differences; the trained segmentation model is obtained by using sample weighting to reduce the influence of erroneous information on the segmented image during training, and using a directional weight learning method to determine the cross-supervised learning direction of the general network and the evidence network. Because the general network and the evidence network have appropriate divergence, the segmentation result obtained by both has higher accuracy and stability, thus enabling semi-supervised conditional training of the segmentation model with only a small number of labeled samples, resulting in stable and accurate 3D image segmentation.

Description

Technical Field

[0001] This invention belongs to the field of medical image segmentation technology, specifically relating to a semi-supervised 3D image segmentation method based on quality-driven cross-learning. Background Technology

[0002] 3D image segmentation, a core task in medical image processing and computer vision, has seen significant success with deep learning-based methods under fully supervised conditions. However, in practical applications, accurate annotation often requires specialized knowledge and considerable time, making it difficult to obtain a large number of pixel-level labeled samples. Semi-supervised learning, by combining limited labeled samples with a large number of unlabeled samples, effectively expands the model's learning ability and generalization.

[0003] Consistency regularization and pseudo-label self-training are two mainstream semi-supervised 3D image segmentation methods. Consistency regularization performs smooth predictions on the same data under different perturbations. For example, it applies consistency regularization to unlabeled samples under different feature perturbations, different model initializations, different contextual information, or different interpolation methods. Pseudo-label self-training methods use labeled samples to pre-train a model to generate labels for unlabeled samples, and then use both labeled and unlabeled samples to iteratively update the model.

[0004] The existing semi-supervised 3D image segmentation methods are as follows:

[0005] Reference 1 (Y.Wu, M.Xu, Z.Ge, J.Cai, L.Zhang, Semi-supervised left atrium segmentation with mutual consistency training, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2021, pp.297-306) proposes a semi-supervised 3D image segmentation method that improves segmentation accuracy through mutual consistency between two slightly different decoders. This method utilizes the concept of mutual consistency, where predictions from different networks should remain consistent with each other. This strategy helps the network learn from ambiguous regions and highly uncertain predictions, thereby improving the reliability of the segmentation.

[0006] Reference 2 (X. Luo, J. Chen, T. Song, G. Wang, Semi-supervised medical imagesegmentation through dual-task consistency, in: Proceedings of the AAAIConference on Artificial Intelligence, Vol. 35, 2021, pp. 8801-8809) proposes a semi-supervised 3D image segmentation method that enhances model performance by implementing prediction consistency between a standard regression network and the segmentation model. The core of this method lies in utilizing a dual-task learning framework, where two networks are trained on different tasks while maintaining consistent predictions for the same data. This approach can transfer useful information between multiple representations of the data, thereby improving segmentation accuracy.

[0007] Reference 3 (X. Luo, M. Hu, T. Song, G. Wang, S. Zhang, Semi-supervised medical imagesegmentation via cross teaching between CNN and transformer, in: International Conference on Medical Imaging with Deep Learning, PMLR, 2022, pp. 820-833) proposes a semi-supervised 3D image segmentation method that improves segmentation accuracy through cross teaching between convolutional neural networks (CNNs) and transformers. The core of this method lies in utilizing two different types of network structures to learn from and guide each other, thereby improving segmentation accuracy.

[0008] Reference 4 (L.Yu, S.Wang, X.Li, C.-W.Fu, P.-A.Heng, Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2019, pp.605-613) proposes an uncertainty-aware self-ensembling model for semi-supervised 3D image segmentation. This model uses Monte Carlo dropout to evaluate the uncertainty of the teacher model's predictions and filters out less reliable predictions, which helps improve the model's generalization ability to unlabeled samples.

[0009] Reference 5 (K. Wang, B. Zhan, C. Zu, X. Wu, J. Zhou, L. Zhou, Y. Wang, Tripled-uncertainty guided mean teacher model for semi-supervised medical imagesegmentation, in: Medical Image Computing and Computer Assisted Intervention MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part II 24, Springer, 2021, pp. 450-460) proposes a triple-uncertainty guided mean teacher model for semi-supervised 3D image segmentation. This method combines three different types of uncertainty—data uncertainty, model uncertainty, and class uncertainty—to guide the model's training. In this way, the model can achieve more robust segmentation performance on unlabeled samples.

[0010] However, existing segmentation methods have the following drawbacks:

[0011] Existing semi-supervised 3D image segmentation methods cannot effectively utilize sub-networks to extract information. For example, when there is insufficient divergence between sub-networks, the sub-networks trained under cross-supervision may reach consensus, reducing the efficiency of 3D image segmentation. When the divergence between sub-networks is too large, the stability of the sub-networks trained under cross-supervision decreases, leading to a reduction in the reliability of 3D image segmentation.

[0012] Furthermore, existing semi-supervised 3D image segmentation methods cannot distinguish the importance of different networks at different stages. They assume that subnetworks with poor segmentation performance have equal weights with subnetworks with good segmentation performance. This reduces the learning efficiency of subnetworks during the cross-supervised learning stage, resulting in a decline in overall segmentation performance.

[0013] Another key issue is the prediction collapse problem during pseudo-label generation, which is common with difficult samples that have complex structures and ambiguous boundaries. Introducing these samples into semi-supervised training leads to error accumulation, causing the model to learn incorrect information during training and reducing the overall performance of the segmentation model.

[0014] In summary, the aforementioned defects in existing semi-supervised 3D image segmentation methods result in poor 3D image segmentation performance. Summary of the Invention

[0015] To address the aforementioned problems in existing technologies, this invention provides a semi-supervised 3D image segmentation method based on quality-driven cross-learning. The technical problem to be solved by this invention is achieved through the following technical solution:

[0016] This invention provides a semi-supervised 3D image segmentation method based on quality-driven cross-learning, comprising:

[0017] S100, acquire the 3D image to be segmented;

[0018] S200, the 3D image to be segmented is input into the pre-trained segmentation model for segmentation to obtain the segmentation result; wherein, the segmentation model includes a general network and an evidence network with a dual network structure, the general network and the evidence network are complementary but have differences; the pre-trained segmentation model is obtained by using sample weighting during the training process to reduce the influence of erroneous information of training samples on the segmented image, and using the direction weight learning method to determine the cross-supervision learning direction of the general network and the evidence network.

[0019] Beneficial effects:

[0020] 1. The segmentation model of the present invention is a dual-network structure containing a general network and an evidence network. The general network and the evidence network have appropriate divergence and complementarity. The general network is optimized by cross-entropy loss and Dice loss, while the evidence network is optimized based on evidence theory, which can enhance the prediction divergence in semi-supervised training, thereby improving the accuracy and stability of 3D image segmentation.

[0021] 2. This invention uses a labeled sample set to evaluate the segmentation performance of the ordinary network and the evidence network in real time, with the evaluation metric being the Dice similarity coefficient. Based on the DSC evaluation results of the ordinary network and the evidence network, the directional weights w for cross-training are determined. v_e and w e_v This is to control the guidance direction in the cross-supervision process, reduce unreliable guidance in cross-training, and improve the reliability of the segmentation model.

[0022] 3. For unlabeled samples, the present invention dynamically adjusts their weight in the loss function based on their predicted DSC values. For samples whose predicted values ​​are lower than a preset threshold, they are temporarily excluded from training to avoid error accumulation, reduce the impact of difficult samples in semi-supervised training on model training, and thus reduce the impact of erroneous information on the segmentation model.

[0023] 4. This invention designs a total loss function that includes the supervision loss of labeled samples and the cross-supervision loss of unlabeled samples, and introduces a time-varying Gaussian slope rising function λ. t This balances the ratio of supervision loss and cross-supervision loss, thereby improving the accuracy of the segmentation model and enhancing the 3D image segmentation effect.

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0025] Figure 1 A flowchart of the semi-supervised 3D image segmentation method using a quality-driven deep cross-supervised learning network provided by the present invention;

[0026] Figure 2 The structure diagram of the quality-driven deep cross-supervised learning network provided by this invention;

[0027] Figure 3 The reasoning process of the evidence network provided by this invention;

[0028] Figure 4 A schematic diagram illustrating the process of training the segmentation model provided by this invention;

[0029] Figure 5 The direction learning weights w provided for this invention v_e and w e_v Workflow diagram;

[0030] Figure 6 A schematic diagram illustrating the visualization of 2D and 3D segmentation results of the pancreas on the Pancreas-CT dataset provided by this invention;

[0031] Figure 7 This is a schematic diagram illustrating the visualization of 2D and 3D segmentation results of the left atrium on the LA dataset provided by the present invention. Detailed Implementation

[0032] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0033] refer to Figure 1 This invention provides a semi-supervised 3D image segmentation method based on quality-driven cross-learning, comprising:

[0034] S100, acquire the 3D image to be segmented;

[0035] S200, the 3D image to be segmented is input into the pre-trained segmentation model for segmentation to obtain the segmentation result; wherein, the segmentation model includes a general network and an evidence network with a dual network structure, the general network and the evidence network are complementary but have differences; the pre-trained segmentation model is obtained by using sample weighting during the training process to reduce the influence of erroneous information of training samples on the segmented image, and using the direction weight learning method to determine the cross-supervision learning direction of the general network and the evidence network.

[0036] In practical use, this invention first uses a shared encoder to extract shared features from the 3D image to be segmented. Then, the output result is jointly determined by the ordinary network and the evidence network. Since the ordinary network and the evidence network have appropriate divergence, the segmentation result obtained by the two together has higher accuracy and stability.

[0037] This invention provides a semi-supervised 3D image segmentation method based on quality-driven cross-learning, comprising: acquiring a 3D image to be segmented; inputting the 3D image to be segmented into a pre-trained segmentation model for segmentation to obtain a segmentation result; wherein, the segmentation model includes a dual-network structure of a general network and an evidence network, which are complementary but have differences; the trained segmentation model is obtained by using sample weighting to reduce the influence of erroneous information on the segmented image during training, and using a directional weight learning method to determine the cross-supervised learning direction of the general network and the evidence network. Since the general network and the evidence network have appropriate divergence, the segmentation result obtained by both has higher accuracy and stability, thus enabling semi-supervised conditional training of the segmentation model with only a small number of labeled samples, resulting in stable and accurate 3D image segmentation.

[0038] In this embodiment, the overall network reference Figure 2 The segmentation model also includes a shared encoder, the output of which is connected to the inputs of the general network and the evidence network.

[0039] The shared encoder consists of alternating 5 convolutional modules and 4 downsampling modules. Each convolutional module contains k convolutional layers with a kernel size of 3 and a ReLU activation layer. The k values ​​for the five convolutional modules are 1, 2, 3, 3, and 3, respectively. The downsampling modules consist of convolutional layers with a kernel size of 2 and a stride of 2. The evidence network encoding structure is the same as that of the ordinary network. Both the ordinary network and the evidence network consist of alternating 4 deconvolutional modules and 4 convolutional modules. The deconvolutional modules consist of a deconvolution function and a ReLU activation function. The k values ​​for the four convolutional modules are 3, 3, 2, and 1, respectively.

[0040] The ordinary network calculates the prediction probability p. ik and real label y ik Cross-entropy loss between and Dice loss Joint optimization Defined as:

[0041]

[0042] The Dice loss Constructed based on the Dice similarity coefficient (DSC), it quantifies the overall similarity between segmentation predictions and ground truth labels, and is defined as:

[0043]

[0044] Where k is the category index, K is the number of categories, and i is the pixel index;

[0045] The loss function of the classifier in the ordinary network is:

[0046]

[0047] The classifier of the evidence network is based on Evidential Deep Learning (EDL), which extends Bayesian theory to subjective probabilities based on Dempster-Shafer evidence theory (DST), and collects predictive evidence of different categories. The predictive evidence corresponds to the parameter α. ik =e ik +1 Dirichlet distribution;

[0048] Among them, the intensity S of the Dirichlet distribution i The prediction probability p of voxels ik Defined as:

[0049]

[0050] Optimize using Dirichlet distribution The Bayesian risk is used to obtain a function for optimizing the evidence network. Represented as:

[0051]

[0052] In the formula: Let ψ(·) be a polynomial beta function, and let ψ(·) denote the double gamma function.

[0053] A KL divergence loss function is introduced, and the evidence for parameters related to the error category is reduced to zero through an adjustment formula, further reducing the contribution of these parameters. The adjustment formula is expressed as:

[0054]

[0055] In the formula: Γ(·) represents the gamma function, This represents the Dirichlet parameters after removing true evidence from the prediction parameters;

[0056] The reasoning process of evidence networks is as follows: Figure 3 As shown, the overall loss function of the evidence network is expressed as:

[0057]

[0058] Controlling the annealing coefficient λ kl In the case of min(1,t / 150), Gradually incorporated into the training process for stable learning, t represents the current iteration number.

[0059] refer to Figure 4 As an optional embodiment of the present invention, the training process of the trained segmentation model includes:

[0060] A. Obtain labeled and unlabeled sample sets, and initialize the ordinary network and the evidence network;

[0061] This invention designs and initializes two networks during training: a general network and the evidence network. The two networks exhibit appropriate divergence and complementarity. The general network is optimized using cross-entropy loss and Dice loss, while the evidence network is optimized based on evidence learning theory, enabling reasoning to address cognitive uncertainties during the network learning process.

[0062] B. Input the labeled samples from the labeled dataset into the ordinary network and the evidence network to obtain the first pseudo-label, and use the first pseudo-label to perform DSC evaluation on the performance of the ordinary network and the evidence network to obtain the DSC evaluation results of the two.

[0063] During training, the segmentation performance of the standard network and the evidence network is evaluated in real time using a labeled dataset, with the Dice similarity coefficient (DSC) as the evaluation metric. Based on the DSC evaluation results of the standard network and the evidence network, the directional learning weights w for cross-training are determined. v_e and w e_v This is to control the direction of guidance during the cross-supervision process.

[0064] Considering the difference in convergence speed between the ordinary network model and the evidence network model within each batch, this invention tends to use the segmentation model with better current segmentation performance to guide another segmentation model, enhancing the effectiveness and reliability of interactive guidance. Therefore, in each training batch, this invention estimates the segmentation performance of different models online based on the true values ​​of labeled samples. Assumptions This indicates the difficulty of segmentation prediction for batch-labeled samples by a standard network. This indicates that the evidence network provides difficult segmentation predictions for batch-labeled samples. Indicates the corresponding truth value. B l This indicates the batch size of the labeled samples. This invention calculates the DSC performance of different segmentation models in each batch; the DSC evaluation results are expressed as follows:

[0065]

[0066] in, This indicates the difficulty of segmentation prediction for batch-labeled samples by a standard network. This indicates that the evidence network provides difficult segmentation predictions for batch-labeled samples. B represents the corresponding truth value. l This indicates the batch size of the labeled sample, where H, W, and L represent the height, width, and length of the labeled sample, respectively.

[0067] C. Based on the evaluation results of the ordinary network and the evidence network, determine the directional learning weights for their respective cross-training.

[0068] The reasoning process for learning weights is as follows Figure 5 As shown. To determine the directionality of cross-supervised training, Dice was used. van and Dice enn The comparison involves comparing the pseudo-labels generated by the network with the higher Dice score in each training batch. In other words, the pseudo-labels generated by the network with the higher Dice score are used to supervise the learning of the network with the lower Dice score. This process is achieved by setting the learning direction weights w.v_e and w used for cross-supervised training e_v This is achieved through [the following]. The directional weights of the ordinary network and the evidence network are respectively expressed as:

[0069]

[0070] in, Indicates the indicator function, when w v_e Under the condition that = 1, w e_v =0, then the pseudo-labels generated by the ordinary network are used to guide the training of the evidence network in the current training batch; when w v_e Under the condition that = 0, w e_v =1, then the pseudo-labels generated by the evidence network are used to guide the training of the ordinary network in the current training batch.

[0071] D. Input the unlabeled samples in the unlabeled sample set into the ordinary network and the evidence network to obtain the second pseudo-label, and use the second pseudo-label to calculate the overlap of the sampling DSC performance between the ordinary network and the evidence network for the difficult prediction of unlabeled samples;

[0072] For unlabeled samples, their weights in the loss function are dynamically adjusted based on their predicted DSC values. Samples with predicted values ​​below a preset threshold are temporarily excluded from training to avoid error accumulation.

[0073] In most cross-supervised training frameworks, the consistency loss of unlabeled samples is typically minimized by simple averaging. However, this indiscriminate averaging may unintentionally emphasize hard samples, potentially leading to model overfitting due to noisy supervision. To overcome this problem, this invention proposes a simple and effective method to estimate the segmentation performance of each unlabeled sample in real time. This invention computes the hard prediction of unlabeled samples for a standard network model. Difficult prediction of unlabeled samples in evidence network models Sampling DSC performance

[0074]

[0075] The ordinary network makes difficult segmentation predictions for unlabeled samples. Difficult segmentation prediction of unlabeled samples using evidence network models Sampling DSC performance Represented as:

[0076]

[0077] In this invention, the greater the overlap between two segmentation models, the better. The larger the value, the better the segmentation performance for unlabeled samples. Therefore, this invention uses this metric as a weight to guide cross-supervised training. Furthermore, if... When the value is below a certain threshold, this invention considers it a case of unlabeled sample prediction failure. In this case, this invention temporarily excludes the sample from training and sets w... us Set to 0. Therefore, the weights of unlabeled samples are represented as:

[0078]

[0079] Where β is the threshold, and β is set to 0.5.

[0080] E, based on the overlap of the direction learning weights and the sampling DSC performance, guides the cross-supervised training process of the ordinary network and the evidence network to ensure the reliability and stability of using unlabeled samples.

[0081] F introduces a weighted control method that uses the prediction probability of one network as the supervision information of another network to perform weighted control on the cross-supervision training process;

[0082] To ensure the reliability and stability of unlabeled sample utilization, this invention proposes a quality-driven cross-supervised learning method that leverages the complementary strengths of different networks and introduces two weighted control methods. The predicted probability of one network serves as the supervision information for another network. Figure 2 As shown, firstly, this invention uses the predicted probability of an evidence network. To guide the output probability of ordinary networks This invention selects L1 loss as the consistency constraint. Furthermore, considering both batch model performance estimation and sample performance estimation, cross-supervision directional weights w are introduced. v_e w e_v And unlabeled sample weights w us w v_e and w e_v The cross-supervised learning weights are controlled to better transfer from better networks to worse networks, thereby guiding cross-supervised training, as shown in Equation (9). Specifically, w in e_v This determines the cross-supervision from the evidence network to the general network. Similarly, in The process determines the cross-supervision from the ordinary network to the evidence network. Weight w us Controlling the contribution of different samples to the loss function. Due to low w us This is often related to unreliable predictions of a single sample, which can lead to error accumulation and hinder the training process. Conversely, w us The higher the value, the more accurate the prediction for a sample.

[0083] Introducing cross-supervision directional weights w v_e w e_v and the weights w of unlabeled samples us Constructing a quality-driven perceptual cross-supervision loss as well as

[0084] Quality-driven perceptual cross-supervision loss This can mitigate the negative impact of noisy prediction samples, resulting in more robust and reliable semi-supervised training, and a quality-driven perceptual cross-supervised loss. for:

[0085]

[0086] The training process of the evidence network classifier is guided by the predicted probabilities of a regular network classifier, using a quality-driven perceptual cross-supervision loss. for:

[0087]

[0088] In the formula, It is the predicted probability of the evidence network. This is the predicted probability of a normal network, where N is the total number of voxels;

[0089] Using uncertainty-weighted loss Weighted control of the cross-supervised training process from the evidence network to the ordinary network;

[0090] Using uncertainty-weighted loss function Weighted control is applied to the cross-supervised training process from ordinary networks to evidence networks.

[0091] G, construct the total loss function that includes the supervised loss of labeled samples and the cross-supervised loss of unlabeled samples;

[0092] This invention constructs a total loss function that includes the supervised loss of labeled samples and the cross-supervised loss of unlabeled samples, and introduces a time-varying Gaussian slope ascending function λ. t To balance the proportions of supervised loss and cross-supervision loss, the loss function designed in this invention consists of two parts. The first part is the supervised loss for labeled data, including the loss for ordinary networks. and evidence networks The second part is the quality-driven perceptual cross-supervision loss. and The total loss is expressed as:

[0093]

[0094] In the formula, This refers to supervised loss for labeled data, including for both general networks and evidence networks. and evidence networks Quality-driven perceptual cross-supervision loss and It is a time-varying Gaussian rising function used to adjust the ratio between the supervision loss and the cross-supervision consistency loss.

[0095] H, optimize the parameters of the ordinary network and the evidence network, and apply cross-supervision loss to mine information from unlabeled samples;

[0096] I. Based on the DSC evaluation results, dynamically adjust the weights of unlabeled samples and repeat BH until the number of iterations is reached, and obtain the trained classification network.

[0097] The effects of this invention can be further illustrated by the following experiments:

[0098] (1) Experimental conditions

[0099] The hardware platform for the simulation experiment of this invention is an NVIDIA 4090RTX GPU with 24GB of video memory.

[0100] The simulation experiment framework of this invention is PyTorch.

[0101] (2) Simulation content

[0102] To verify the effectiveness of the semi-supervised image segmentation method of this invention, a series of simulation experiments were designed and implemented. The following is a detailed description of the experiments.

[0103] The experiment used two publicly available medical image datasets: the LA dataset and the Pancreas-CT dataset. The LA dataset contains 100 cardiac magnetic resonance imaging (MRI) scan samples, while the Pancreas-CT dataset contains 82 abdominal CT scan samples.

[0104] In the data preprocessing stage, this invention extracts 3D samples from the raw data, retains the target region, and normalizes it to zero mean and unit variance. Subsequently, this invention randomly crops 3D patches of size 112×112×80 and 96×96×96 from the LA dataset and Pancreas-CT dataset, respectively. The batch size is set to 4, including 2 labeled patches and 2 unlabeled patches. This invention uses an SGD optimizer with an initial learning rate of 0.1 and a multinomial learning rate decay strategy. The total number of training iterations is 30,000.

[0105] During the testing phase, this invention employs a sliding window strategy, with a fixed step size of 18×18×4 for the LA dataset and a fixed step size of 16×16×16 for the Pancreas-CT dataset. In this invention, a standard network is used as the backbone segmentation network.

[0106] Evaluation indicators:

[0107] In this invention, four evaluation metrics are used to assess performance, including DSC, JAC, average surface distance (ASD), and 95% Hausdorff distance (95HD). DSC and JAC values ​​are in the range [0,1], and better segmentation results are characterized by higher DSC and JAC values, defined as...

[0108]

[0109]

[0110] Here, TP, FP, and FN represent the number of true positives, false positives, and false negatives, respectively. 95HD and ASD are commonly used to measure the segmentation performance at the boundary. Let... and These represent the binary prediction and the binary basis truth, respectively. 95HD is represented as:

[0111]

[0112] In the formula: and These represent the sets of tissue surface voxels related to the prediction and the actual values, respectively. For the predicted surface voxels To real surface set The shortest Euclidean distance is defined as:

[0113]

[0114] and Following a similar approach to equation (18). Furthermore, ASD is defined as:

[0115]

[0116] Therefore, the smaller the values ​​of 95HD and ASD, the better the boundary segmentation performance.

[0117] Simulation results:

[0118] Table 1 presents the quantitative results for the Pancreas-CT dataset. Compared to the self-trained baseline with only 6 labeled samples, the proposed semi-supervised image segmentation method using a quality-driven deep cross-supervised learning network (QDC-Net) achieves improvements of 20.03%, 20.46%, 14.54%, and 4.69 in DSC, JAC, 95HD, and ASD, respectively. In the second set of experiments, QDC-Net improved these metrics by 8.51%, 10.52%, 12.06%, and 4.09, respectively, compared to self-training with 12 labeled samples. These results demonstrate that QDC-Net can effectively utilize unlabeled samples to improve segmentation performance. With 6 labeled samples and 56 unlabeled samples, QDC-Net outperforms nine other semi-supervised segmentation methods on DSC, JAC, and 95HD, particularly outperforming the blue method by 4.07% on DSC, 4.75% on JAC, and 5.48 voxels on 95HD. Among all semi-supervised methods, QDC-Net achieves a suboptimal ASD of 2.49. The DTC method, by introducing a boundary distance regression task, achieves the best ASD metric of 1.97. This invention also tested the performance of semi-supervised pancreas segmentation on 12 labeled samples and 50 unlabeled samples. QDC-Net achieved the best results on the three key metrics of DSC, JAC, and 95HD, and achieved suboptimal performance on the ASD metric, exceeding the suboptimal performance by 0.86%, 1.20%, and 0.79 voxels 250 on DSC, JAC, and 95HD, respectively. Notably, QDC-Net's DSC, 95HD, and ASD performance are comparable to its self-training performance on all labeled samples, at 81.23% vs 82.80%, 6.17 vs 5.58, and 1.68 vs 1.26, respectively. These results quantitatively demonstrate that QDC-Net effectively leverages the advantages of both ordinary and evidence networks in cross-supervised learning, stably complementing each other to extract valuable information from unlabeled data. Figure 5 Visualization to compare the experimental 2D and 3D pancreas segmentation results. From Figure 6 Rows 3 and 4 show that, compared to other methods, QDC-Net achieves anatomically reasonable segmentation results and maintains the integrity of pancreatic segmentation based on the actual situation. The lower half was implemented using 12 labeled samples and 50 unlabeled samples. Figure 5As can be observed in rows 5 and 6, the proposed QDC-Net displays boundaries consistent with the true boundaries in the 2D view. In the 3D view, QDC-Net obtains a more accurate 3D structure of the pancreas than other methods, as shown in rows 7 and 8. These visualization results support the ability of QDC-Net to achieve efficient and stable semi-supervised training by introducing model and sample quality estimation in cross-supervised training.

[0119] Table 1

[0120]

[0121]

[0122] Table 2 presents the quantization results for the LA dataset. This invention evaluated the semi-supervised segmentation method at 10% and 20% labeling rates. Compared to self-training using only 8 labeled samples, QDC-Net improves performance on DSC, JAC, 95HD, and ASD metrics by 5.32%, 9.72%, 2.20 voxels, and 0.79 voxels, respectively. Furthermore, QDC-Net outperforms other semi-supervised segmentation methods on all metrics, exceeding suboptimal performance by 0.61%, 0.98%, 1.03, and 0.06 on DSC, JAC, 95HD, and ASD, respectively. At 20% labeled samples, the proposed QDC-Net achieves comparable results to self-training with all labeled samples, showing only minor differences on DSC, JAC, 95HD, and ASD. Furthermore, QDC-Net achieved improvements of 5.41% and 8.22%, respectively. Under the same settings, QDC-Net also produced the best semi-supervised left atrial segmentation performance compared to other state-of-the-art semi-supervised methods. These quantitative results validate the superiority of the method presented in this invention and demonstrate the effectiveness of quality-aware cross-supervised learning in semi-supervised segmentation under low-labeled data conditions. Under different semi-supervised settings, [the following is a separate, unrelated sentence:] ... Figure 7The first four and last four rows visualize several 2D / 3D segmentation examples. As shown in the first two rows, the method of this invention generates consistent boundaries and accurate atrial segmentation based on ground truth under training conditions of 8 labeled samples and 72 unlabeled samples. The third and fourth rows show the 3D segmentation results for two samples. Some other semi-supervised segmentation methods, such as CPS and CPSCauSSL, generate 3D predictions with false negative regions in the background. In contrast, QDC-Net generates a more consistent 3D anatomical segmentation structure based on ground truth compared to other methods. The lower half provides segmentation results for 16 labeled samples and 64 unlabeled samples. The first two rows show the 2D left atrial segmentation results. The method of this invention achieves accurate tissue boundaries. The last two rows show the 3D left atrial segmentation results. The method of this invention achieves clear boundaries for clavicle segmentation. The accurate and stable segmentation predictions of MDSSL for different tissues under different noise ratios verify the scalability and reliability of MDSSL in handling noise.

[0123] Table 2

[0124]

[0125] In Tables 1 and 2, the literature for MT[1], UA-MT[2], SASSNet[3], DTC[4], URPC[5], FUSSNet[6], CPS[7], CPSCauSSL[8] and MC-Net+[9] is as follows:

[0126] [1] A. Tarvainen, H. Valpola, Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results, Advances in neural information processing systems 30 (2017).

[0127] [2]L.Yu,S.Wang,X.Li,C.-W.Fu,P.-A.Heng,Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation,in:International Conference on Medical Image Computing and Computer-AssistedInterven tion,Springer,2019,pp.605-613.

[0128] [3]S.Li,C.Zhang,X.He,Shape-aware semi-supervised 3d semanticsegmentation for medical images,in:Medical Image ComputingandComputerAssisted Intervention-MICCAI 2020:23rd International Conference,Lima,Peru,October 4-8,2020,Proceedings,Part I 23,Springer,2020,pp.552-561.

[0129] [4]X.Luo,J.Chen,T.Song,G.Wang,Semi-supervised medical imagesegmentation through dual-task consistency,in:Proceedings of the AAAIConference on Artificial Intelligence,Vol.35,2021,pp.8801-8809.

[0130] [5]X.Luo,W.Liao,J.Chen,T.Song,Y.Chen,S.Zhang,N.Chen,G.Wang,S.Zhang,Efficient semi-supervised gross target volume of nasopharyngeal carcinomasegmentation via uncertainty rectified pyramid consistency,in:InternationalConference on Medical Image Computing and Computer-Assisted Intervention,Springer,2021,pp.318-329.

[0131] [6]J.Xiang,P.Qiu,Y.Yang,Fussnet:Fusing two sources of uncertainty forsemi-supervised medical image segmen tation,in:International Conference onMedical Image Computing and Computer-Assisted Intervention,Springer,2022,pp.481-491.

[0132] [7]X.Chen,Y.Yuan,G.Zeng,J.Wang,Semi-supervised semantic segmentationwith cross pseudo supervision,in:Proceedings of the IEEE / CVF Conference onComputer Vision and Pattern Recognition,2021,pp.2613-2622.

[0133] [8] J.Miao, C.Chen, F.Liu, H.Wei, P.-A.Heng, Caussl: Causality-inspired semi-supervised learning for medical image segmentation, in: Proceedings of the IEEE / CVF International Conference on Computer Vision, 2023, pp.21426-21437.

[0134] [9] Y. Wu, Z. Ge, D. Zhang, M. Xu, L. Zhang, Y. Xia, J. Cai, Mutual consistency learning for semi-supervised medical image segmentation, Medical Image Analysis 81 (2022) 102530.

[0135] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0136] Although this application has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the appended claims in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality.

[0137] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A semi-supervised 3D image segmentation method based on quality-driven cross-learning, characterized in that, include: S100, acquire the 3D image to be segmented; S200, the 3D image to be segmented is input into a pre-trained segmentation model for segmentation to obtain a segmentation result; wherein, the segmentation model includes a general network and an evidence network with a dual-network structure, the general network and the evidence network being complementary but having differences; the pre-trained segmentation model is obtained by using sample weighting during training to reduce the influence of erroneous information from training samples on the segmented image, and by using a directional weight learning method to determine the cross-supervision learning direction of the general network and the evidence network; The training process of the trained segmentation model includes: A. Obtain labeled and unlabeled sample sets, and initialize the ordinary network and the evidence network; B. Input the labeled samples from the labeled dataset into the ordinary network and the evidence network to obtain the first pseudo-label, and use the first pseudo-label to perform DSC evaluation on the performance of the ordinary network and the evidence network to obtain the DSC evaluation results of the two. C. Based on the evaluation results of the ordinary network and the evidence network, determine the directional learning weights for their respective cross-training. D. Input the unlabeled samples in the unlabeled sample set into the ordinary network and the evidence network to obtain the second pseudo-label, and use the second pseudo-label to calculate the overlap of the sampling DSC performance between the ordinary network and the evidence network for the difficult prediction of unlabeled samples; E, based on the overlap of the directional learning weights and the sampling DSC performance, guides the cross-supervised training process of the ordinary network and the evidence network; F introduces a weighted control method that uses the prediction probability of one network as the supervision information of another network to perform weighted control on the cross-supervision training process; G, construct the total loss function that includes the supervised loss of labeled samples and the cross-supervised loss of unlabeled samples; H, optimizes the parameters of the ordinary network and the evidence network using the total loss function, and applies cross-supervision loss to mine information from unlabeled samples; I. Based on the DSC evaluation results, dynamically adjust the weights of unlabeled samples and repeat BH until the number of iterations is reached, and obtain the trained classification network.

2. The semi-supervised 3D image segmentation method based on quality-driven cross-learning according to claim 1, characterized in that, The segmentation model also includes a shared encoder, the output of which is connected to the inputs of the general network and the evidence network; The shared encoder consists of 5 convolutional modules and 4 downsampling modules alternately. Each convolutional module contains k convolutional layers with a kernel size of 3 and a ReLU activation layer. The k values ​​for the 5 convolutional modules are 1, 2, 3, 3 and 3, respectively. The downsampling modules consist of convolutional layers with a kernel size of 2 and a stride of 2. Both the ordinary network and the evidence network are composed of alternating 4 deconvolution modules and 4 convolution modules. The deconvolution module is composed of a deconvolution function and a ReLU activation function. The k values ​​of the 4 convolution modules are 3, 3, 2 and 1, respectively.

3. The semi-supervised 3D image segmentation method based on quality-driven cross-learning according to claim 2, characterized in that, The ordinary network calculates the prediction probability. and real labels Cross-entropy loss between and Dice loss Joint optimization Defined as: （1） The Dice loss Constructed based on the Dice similarity coefficient (DSC), defined as: （2） in, For category indexing, For the number of categories, For pixel index; The loss function of the classifier in the ordinary network is: （3）。 4. The semi-supervised 3D image segmentation method based on quality-driven cross-learning according to claim 3, characterized in that, The classifier of the evidence network is based on Evidence Deep Learning (EDL), which generalizes Bayesian theory to subjective probabilities based on Dempster-Shafer Evidence Theory (DST), and collects predictive evidence of different categories. The parameters corresponding to the predictive evidence are: The Dirichlet distribution; Among them, the intensity of the Dirichlet distribution prediction probability of voxels Defined as: （4） Optimize using Dirichlet distribution The Bayesian risk is used to obtain a function for optimizing the evidence network. , is represented as: （5） In the formula: For polynomial beta functions, Represents the double gamma function; Introduction KL The divergence loss function reduces the evidence for parameters related to the error category to zero through an adjustment formula, which is expressed as follows: （6） In the formula: Represents the gamma function. This represents the Dirichlet parameters after removing true evidence from the prediction parameters; The overall loss function of the evidence network is expressed as: （7） Controlling the annealing coefficient In this case, Gradually incorporated into the training process for stable learning, This indicates the current iteration number.

5. The semi-supervised 3D image segmentation method based on quality-driven cross-learning according to claim 4, characterized in that, The DSC evaluation results in B are expressed as follows: （8） in, This indicates the difficulty of segmentation prediction for batch-labeled samples by a standard network. This indicates that the evidence network provides difficult segmentation predictions for batch-labeled samples. Indicate the corresponding truth value, Indicates the batch size of the labeled sample. These represent the height, width, and length of the labeled sample, respectively.

6. The semi-supervised 3D image segmentation method based on quality-driven cross-learning according to claim 5, characterized in that, The directional weights of the ordinary network and the evidence network described in C are respectively expressed as follows: （9） in, Indicates an indicator function, when in Under the conditions, Then, pseudo-labels generated by ordinary networks are used to guide the training of the evidence network in the current training batch; when in Under the conditions, Then, pseudo-labels generated by the evidence network are used to guide the training of the ordinary network in the current training batch.

7. The semi-supervised 3D image segmentation method based on quality-driven cross-learning according to claim 6, characterized in that, The ordinary network described in section D provides a difficult segmentation prediction for unlabeled samples. Difficult segmentation prediction of unlabeled samples using evidence network models Sampling DSC performance Represented as: （10） The weights of unlabeled samples are represented as follows: （11） in, The threshold value is used.

8. The semi-supervised 3D image segmentation method based on quality-driven cross-learning according to claim 7, characterized in that, F include: Introducing cross-supervision directional weights , Weights of unlabeled samples Constructing a quality-driven perceptual cross-supervision loss as well as ; （12） （13） In the formula, It is the predicted probability of the evidence network. This is the predicted probability of a regular network. It is the total number of voxels; Using uncertainty-weighted loss Weighted control of the cross-supervised training process from the evidence network to the ordinary network; Using uncertainty-weighted loss function Weighted control is applied to the cross-supervised training process from ordinary networks to evidence networks.

9. The semi-supervised 3D image segmentation method based on quality-driven cross-learning according to claim 8, characterized in that, The total loss function in G is expressed as: （14） In the formula, It refers to supervised loss for labeled data, including for ordinary networks. and targeting evidence networks , Quality-driven perceptual cross-supervision loss and , It is a time-varying Gaussian rising function used to adjust the ratio between the supervision loss and the cross-supervision consistency loss.

Images