Semi-supervised bladder tumor medical image segmentation method based on supervised branches and uncertainty estimation

Abstract

The invention discloses a semi-supervised bladder tumor medical image segmentation method based on supervised branches and uncertainty estimation, and relates to the technical field of bladder tumor medical image segmentation. Comprising the following steps: S1, acquiring data; s2, data preprocessing; s3, data division; s4, constructing a model; s5, performing model training; and S6, carrying out segmentation identification. The invention provides a new semi-supervised task method, which introduces a supervision branch, combines a pseudo tag and uncertainty estimation based on information entropy, uses an average teacher model as a bottom layer architecture, and provides a supervision signal through a prediction result of a supervision branch network. Therefore, the bladder tumor segmentation model is supervised to generate a more accurate segmentation result.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation of bladder tumors, and in particular to a semi-supervised medical image segmentation method for bladder tumors based on supervised branches and uncertainty estimation. Background Technique

[0002] Bladder cancer is the 10th most common cancer globally, and its pathogenesis has not been fully elucidated yet. Factors such as advanced age, male gender, smoking, and a family history of bladder cancer are all closely related to the occurrence of bladder cancer. Since the mid-2000s, the decline rate of the incidence of bladder cancer has changed, accelerating from less than 1% per year initially to 1.4% per year during the period from 2015 to 2021. As a common type of cancer in the urinary system, bladder cancer has become an important public health issue. Its upward trend in incidence and high medical costs have imposed a heavy burden on society and patients.

[0003] To achieve early detection and effective management of bladder cancer, a variety of screening methods have emerged. Cystoscopy, as a commonly used method, can directly observe the bladder mucosa and detect potential tumors, and it has also been widely used in the field of artificial intelligence-assisted bladder cancer diagnosis. However, in the task of bladder tumor segmentation, a series of unique challenges still exist. First, there is a serious class imbalance problem in cystoscopy data. The tumor area appears relatively small against the background of a large amount of normal bladder tissue, which makes it difficult for machine learning algorithms to effectively learn tumor features during the training process and affects the performance of the model. At the same time, there are significant internal differences in the shape, size, and texture of tumors, and the class difference between the tumor area and the normal bladder wall is small, further increasing the difficulty of feature extraction. Traditional data augmentation methods are designed for general images and are difficult to effectively solve these problems in bladder tumor segmentation. Therefore, it is urgent to develop data augmentation techniques specifically applicable to bladder tumor segmentation. Second, most current segmentation methods rely on supervised learning and require a large amount of labeled data for training. However, obtaining pixel-level annotations of cystoscopy images is not only costly but also extremely time-consuming. In this case, semi-supervised learning methods have become potential solutions that can utilize unlabeled data to improve the performance of the model. Third, most existing semi-supervised learning methods only impose consistency constraints at the image level and ignore the importance of consistency at the feature level. To achieve more accurate segmentation results, it is necessary to consider consistency at different semantic levels.

[0004] In recent years, semi-supervised learning has made certain progress in the field of medical image segmentation. However, in the task of bladder cancer segmentation, existing methods still have significant deficiencies. For example, in dealing with tiny tumors, due to the lack of an effective recognition mechanism for small targets, only some relatively large tumors can be detected, resulting in a low recall rate of segmentation, which affects the accuracy of clinical diagnosis. Existing methods also face challenges. Tiny tumors account for a very small proportion of pixels in cystoscope images and have unclear features. Existing semi-supervised models are difficult to capture their features and are prone to misclassifying them as normal tissues. Moreover, the boundaries of some tumors in the mirror are blurred and there are high-brightness regions, which affect the model's capture of features. In addition, there are differences in the cystoscope imaging devices used in different hospitals, and the resolutions, contrasts, and noise levels of the images vary.

[0005] Therefore, it is an urgent problem for those skilled in the art to propose a semi-supervised medical image segmentation method for bladder tumors based on a supervised branch and uncertainty estimation to solve the difficulties existing in the prior art. Summary of the Invention

[0006] In view of this, the present invention provides a semi-supervised medical image segmentation method for bladder tumors based on a supervised branch and uncertainty estimation. By introducing a supervised branch, combining pseudo-labels and uncertainty estimation based on information entropy, and using the mean teacher model as the underlying architecture, the supervision signal is provided through the prediction results of the supervised branch network, so as to supervise the bladder tumor segmentation model to produce more accurate segmentation results.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A semi-supervised medical image segmentation method for bladder tumors based on a supervised branch and uncertainty estimation includes the following steps:

[0009] S1. Obtain data: Obtain bladder tumor image data;

[0010] S2. Data preprocessing: Perform preprocessing on the obtained bladder tumor image data;

[0011] S3. Data division: Divide the preprocessed bladder tumor image data into labeled data and unlabeled data;

[0012] S4. Model construction: Construct a bladder tumor segmentation model based on a supervised branch, a student model, and a teacher model;

[0013] S5. Model training: Train the bladder tumor segmentation model with labeled data and unlabeled data. After several trainings, obtain a trained bladder tumor segmentation model;

[0014] S6. Segmentation and Recognition: Input the bladder tumor image data of the patient to be tested into the trained bladder tumor segmentation model, and evaluate the bladder tumor segmentation and recognition results.

[0015] Optionally, in S3, a two-stream batch sampler is used to sample from the labeled data and unlabeled data simultaneously to construct a training batch, and both labeled data and unlabeled data are included in each training batch.

[0016] Optionally, in S4, the teacher model in the teacher network is used to implement the segmentation task, and the features extracted by the teacher model are supervised by the supervision branch. The encoder and decoder of the supervision branch have the same structure as the student model but independent parameters, which are represented by θ and θ′ respectively corresponding to the weight parameters of the student network and the teacher network. The weight parameters of the teacher network are updated through the EMA attention mechanism, that is, updated by combining the weight parameters θ of the student network and the smoothing coefficient α, where t represents the training time, specifically:

[0017] θ' t = αθ' t-1 +(1 - α)θ t .

[0018] Optionally, in S5, when training with labeled data, the labeled data is processed by the supervision branch network and the student network to extract target features, and the output results are constrained by the consistency loss and the supervision loss;

[0019] When inputting unlabeled data, the supervision branch, the student model, and the teacher model are processed simultaneously to generate segmentation results, and are constrained by the consistency loss and the cross-entropy loss among each other.

[0020] Optionally, in the supervision branch, uncertainty estimation is introduced, and the segmentation result is used to generate pseudo-labels and a mask, and the pseudo-labels are used to calculate the loss with the segmentation results generated by the student model and the teacher model under the constraint of the mask.

[0021] Optionally, the method of generating pseudo-labels and a mask from the segmentation result, and using the pseudo-labels to calculate the loss with the segmentation results generated by the student model and the teacher model respectively is:

[0022] Y g (u j ) = argmax(softmax(P g (u j )))

[0023] where P g (u j ) is the predicted probability distribution generated by the supervision branch, and the corresponding generated pseudo-label is Y g (u j ).

[0024] Optionally, the uncertainty estimation introduced in the supervision branch is:

[0025] U = -∑ j Y g (u j )log(Y g (u j ))

[0026] Set the dynamic threshold threshold:

[0027] T threshold = (0.75 + 0.25 * sigmoid_rampup(t, T))

[0028]

[0029] Select the area with higher reliability as the mask according to the threshold threshold:

[0030] mask = (U < T threshold )

[0031] where T is the total number of training times.

[0032] Optionally, the loss function includes the loss function of labeled data and the loss function of unlabeled data, and the total loss function is:

[0033] L totel = L1 + L2

[0034] where L1 is the loss function of labeled data and L2 is the loss function of unlabeled data;

[0035] The loss function of labeled data is:

[0036] L1 = l s (P g,T (x i ), y i ) + λl sc (P g (x i ), P T (x i ))

[0037] l s = L Dice (P g (x i ), y i ) + L Dice (P T (x i ), y i )

[0038] l sc= L CE (P g (x i ), P T (x i ))

[0039] Among them, l s is the supervision loss of the prediction results of the supervision branch and the student model and the true label. l sc is the consistency loss between the supervision branch and the prediction results of the student model. λ is the weight of the consistency loss. P g,T is the comprehensive representation of the prediction probability output results of the supervision branch and the teacher model. P g is the prediction result of the supervision branch. x i is the labeled data sample. y i is the true label corresponding to the labeled output sample. P T is the prediction result probability of the teacher model. L CE is the cross-entropy loss calculation. L Dice is the Dice loss calculation;

[0040] The loss function of the unlabeled data is:

[0041] L2 = l c + l p

[0042] l c = L CE (P g (u j ), P T (u j )) + L CE (P g (u j ), P S (u j )) + L CE (P S (u j ), P T (u j ))

[0043] Among them, l c is the consistency loss between the segmentation results of the supervision branch, the student model, and the teacher model. l p is the boundary information loss. u j is the unlabeled data sample. P S is the prediction result probability of the student model.

[0044] Optionally, six indicators including the Dice coefficient, mean intersection over union, sensitivity, specificity, mean absolute error, and accuracy are used in S6 to evaluate the bladder tumor segmentation and recognition results.

[0045] As can be seen from the above technical solutions, compared with the prior art, the present invention provides a semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation, which has the following beneficial effects:

[0046] (1) The present invention proposes a new semi-supervised task method, which introduces a supervised branch, combines pseudo-labels and uncertainty estimation based on information entropy, uses the mean teacher model as the underlying architecture, and provides the supervision signal through the prediction results of the supervised branch network, so as to supervise the bladder tumor segmentation model to produce more accurate segmentation results;

[0047] (2) The present invention proposes to generate a mask by combining pseudo-labels and uncertainty estimation based on information entropy to simplify model coupling and enhance the boundary information of the prediction results, thereby improving the model accuracy;

[0048] (3) The present invention introduces a loss to facilitate learning the reliability region of accurate segmentation predictions.

[0049] (4) After evaluation on the bladder tumor clinical medical image dataset, when the labeled data is limited to 15%, the Dice coefficient of the bladder tumor segmentation model for segmenting the target shape of the bladder tumor can reach up to 80.04%, the mIoU is up to 71.60%, and the accuracy reaches 94.90%. Compared with other methods, the Dice coefficient is increased by about 2%-11.3%, and the mIoU is increased by about 1.12%-16%. Comparative experiments on the public colonoscopy datasets Kvasir-SEG, ETIS_LaribPolypDB, and CVC-conlonDB also verify the effectiveness of the bladder tumor segmentation model. Finally, through extensive ablation studies, the effectiveness of the bladder tumor segmentation model is further verified, highlighting the superiority of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0051] Figure 1 It is a flowchart of a semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation provided by the present invention;

[0052] Figure 2 It is a framework diagram of the bladder tumor segmentation model provided by the present invention;

[0053] Figure 3Schematic diagram of the test results of each method provided by the embodiments of the present invention under the training weights of a 15% limited data set. Detailed implementation manners

[0054] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0055] Refer to Figure 1 and Figure 2 As shown, the present invention discloses a semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation, including the following steps:

[0056] S1. Obtain data: Obtain bladder tumor image data.

[0057] S2. Data preprocessing: Perform preprocessing on the obtained bladder tumor image data.

[0058] S3. Data division: Divide the preprocessed bladder tumor image data into labeled data and unlabeled data.

[0059] S4. Model construction: Construct a bladder tumor segmentation model based on a supervised branch, a student model, and a teacher model.

[0060] S5. Model training: Train the bladder tumor segmentation model with labeled data and unlabeled data. After several trainings, obtain a trained bladder tumor segmentation model.

[0061] S6. Segmentation recognition: Input the bladder tumor image data of the patient to be tested into the trained bladder tumor segmentation model, and evaluate the bladder tumor segmentation recognition result.

[0062] Further, in S3, a two-stream batch sampler is used to simultaneously sample from the labeled data and the unlabeled data to construct a training batch, and both labeled data and unlabeled data are included in each training batch.

[0063] Further, in S4, the teacher model in the teacher network is used to implement the segmentation task, and the features extracted by the teacher model are supervised by the supervised branch. The encoder and decoder of the supervised branch have the same structure as the student model but independent parameters, which are respectively represented by θ and θ′ corresponding to the weight parameters of the student network and the teacher network. The weight parameters of the teacher network are updated through the EMA attention mechanism, that is, updated by combining the weight parameters θ of the student network and the smoothing coefficient α, where t represents the training time. Specifically:

[0064] θ' t = αθ' t-1 + (1 - α)θ t 。

[0065] Furthermore, when training with labeled data in S5, the labeled data is processed by the supervised branch network and the student network to extract target features, and the output results are constrained by consistency loss and supervised loss;

[0066] When unlabeled data is input, the supervised branch, the student model, and the teacher model are processed simultaneously to generate segmentation results, and consistency loss and cross-entropy loss constraints are imposed among them.

[0067] Furthermore, in the supervised branch, uncertainty estimation is introduced. The segmentation result is used to generate pseudo-labels and a mask, and the loss is calculated for the pseudo-labels and the segmentation results generated by the student model and the teacher model under the constraint of the mask.

[0068] Furthermore, the method of generating pseudo-labels and a mask from the segmentation result, and the pseudo-labels and the segmentation results generated by the student model and the teacher model respectively are as follows:

[0069] Y g (u j ) = argmax(softmax(P g (u j )))

[0070] where P g (u j ) is the predicted probability distribution generated by the supervised branch, and the corresponding generated pseudo-label is Y g (u j ).

[0071] Furthermore, the uncertainty estimation introduced in the supervised branch is:

[0072] U = -∑ j Y g (u j )log(Y g (u j ))

[0073] Set a dynamic threshold threshold:

[0074] T threshold = (0.75 + 0.25 * sigmoid_rampup(t, T))

[0075]

[0076] Select regions with higher reliability as the mask according to the threshold threshold:

[0077] mask = (U < T threshold )

[0078] where T is the total number of training times.

[0079] Furthermore, the loss function includes the loss function of labeled data and the loss function of unlabeled data, and the total loss function is:

[0080] L totel = L1 + L2

[0081] where L1 is the loss function of labeled data and L2 is the loss function of unlabeled data;

[0082] The loss function of labeled data is:

[0083] L1 = l s (P g,T (x i ), y i ) + λl sc (P g (x i ), P T (x i ))

[0084] l s = L Dice (P g (x i ), y i ) + L Dice (P T (x i ), y i )

[0085] l sc = L CE (P g (x i ), P T (x i ))

[0086] where l s is the supervision loss of the prediction results of the supervision branch and the student model and the true label, l sc is the consistency loss of the prediction results of the supervision branch and the student model, λ is the weight of the consistency loss, P g,T is the comprehensive representation of the prediction probability output results of the supervision branch and the teacher model, P g is the prediction result of the supervision branch, x i is the labeled data sample, y i is the true label corresponding to the labeled output sample, P T is the prediction result probability of the teacher model, L CEFor cross-entropy loss calculation, L Dice is for Dice loss calculation;

[0087] The loss function for unlabeled data is:

[0088] L2 = l c + l p

[0089] l c = L CE (P g (u j ), P T (u j )) + L CE (P g (u j ), P S (u j )) + L CE (P S (u j ), P T (u j ))

[0090] where l c is the consistency loss between the segmentation results of the supervised branch, the student model, and the teacher model, and l p is the boundary information loss, u j is the unlabeled data sample, and P S is the probability of the prediction result of the student model.

[0091] Furthermore, in S6, six indicators including the Dice coefficient, mean intersection over union, sensitivity, specificity, mean absolute error, and accuracy are used to evaluate the bladder tumor segmentation and recognition results.

[0092] In a specific embodiment, it includes the following content:

[0093] (1) Dataset: The cystoscope dataset provided by the Affiliated Hospital of Anhui Medical University and the open-access polyp datasets Kvasir-SEG, CVC-ColonDB, and ETIS-Larib are used for comparative experiments to verify the stability of the bladder tumor segmentation model. These datasets contain real colonoscopy data, which have been manually annotated by doctors and verified by experienced experts.

[0094] (1) The bladder tumor dataset includes 1948 images with different bladder regions, and the resolution is 1920×1072.

[0095] (2) The Kvasir-SEG dataset includes 1000 images with different polyp regions, and the resolution ranges from 332×487 to 1920×1072.

[0096] (3) The ETIS_LaribPolypDB dataset contains 196 images, each with a resolution of 1225×966.

[0097] (4) The CVC-ColonDB dataset extracts 300 images with a resolution of 500×573 from 13 polyp video sequences, collected from 13 patients.

[0098] For the bladder tumor dataset, standard settings for partial polyp segmentation evaluation and evaluation metrics focusing on boundary information were adopted. For different comparison methods, a consistent distribution was maintained on the training, validation, and test sets. 60% of the images were used for training, 20% for validation, and 20% for testing. On the comparison dataset, the same evaluation criteria and training set division as those for the bladder tumor dataset were used.

[0099] (2) Six commonly used metrics were adopted to evaluate the segmentation performance, including the Dice coefficient (Dice), mean intersection over union (mIoU), sensitivity (sm), specificity (em), mean absolute error (MAE), and accuracy (acc).

[0100] (3) Using the PyTorch library, for the training set, each image was initially resized to 512×512 and augmented by random flipping and rotation. Then, the augmented patches were randomly cropped to 224×224 for training. For the validation and test sets, the image size was adjusted to 512×512. The SGD optimizer was adopted, and the learning rate was initialized to 0.001 during the training of the network for 500 epochs and decayed by a factor For the experimental setup, through the action of the two-stream batch sampler, each step processes two labeled data and two unlabeled data. Specifically, the student model and the supervised branch receive the labeled images for training, and the parameters of the student network are initially updated under the constraint of the true labels. After inputting the unlabeled data, the uncertainty of the prediction results of the supervised branch is estimated and a mask is generated to guide the student model and the teacher model to generate more reliable results. Finally, the teacher model is updated and the next iteration is performed. All experiments were run on the experimental cloud platform.

[0101] (4) Results

[0102] The semi-supervised segmentation framework of the present invention is compared and evaluated with several state-of-the-art semi-supervised learning methods. These methods include the standard Mean Teacher framework MT, the advanced Uncertainty-Aware Mean Teacher UA-MT, the Dual-Task Consistency strategy DTC that cleverly combines two complementary tasks, the Polyp-Aware Hybrid and Dual-Level Consistency Regularization PolypMixNet, and the Twin Student-Supervised Teacher DSST that updates the teacher model with a twin student model. These network framework comparisons are crucial for evaluating the performance of UDS-MT in the semi-supervised learning paradigm. In these experiments, 5%, 15%, and 30% of the data are randomly selected from the training data as labeled data. For the bladder tumor dataset and the training set of Kvasir-SEG, they include 187 labeled images and 374 labeled images respectively.

[0103] In addition to semi-supervised methods, it is also compared with fully supervised segmentation techniques. This comparison is necessary to evaluate the performance of the semi-supervised UDS-MT method of the present invention relative to fully supervised methods, which are generally regarded as benchmarks in the segmentation task. As a baseline, the present invention trains a fully supervised network, called Supervised, on 187 and 374 labeled images respectively using the PVT backbone. To demonstrate the effectiveness of semi-supervised learning, the segmentation performance under two supervised networks is tested. In addition, for a comprehensive evaluation, a fully supervised network is trained using the complete labeled training set with the pvt backbone, denoted as Supervised.

[0104] These methods are implemented using the published code and following the settings specified in the original literature. For data augmentation, the same techniques are always applied. For the bladder tumor dataset, only simple size transformation and grayscale conversion are performed, while for the comparison dataset, random flipping, rotation, and scaling are performed.

[0105] 4.1 Segmentation Results of Bladder Tumor Dataset

[0106] As Figure 3 shown, the present invention provides visualization results of multiple semi-supervised networks and backbone networks under a fully supervised network with a 15% limited dataset. Figure 3 The visualization results in

[0107] Table 1 Comparison results of the bladder tumor dataset under the fully supervised results of the PVT backbone network and different methods of UAMT, UAMT, DTC, PolypMix, dsst, and UDS-MT. The metric results are the test percentage results of the network on the test set

[0108]

[0109]

[0110] As shown in Table 1, when the labeled training data is limited, the method of the present invention achieves better performance than fully supervised algorithms. When the labeled data is only 15% of the bladder tumor training set, the minimum improvement on Dice is 2.81%, the minimum improvement on mIoU is 3.48%, sm and em are also improved to varying degrees, MAE is reduced to 5.18%, and the accuracy is also improved. There may be two main reasons for the improvement of the evaluation index effect.

[0111] First, deep learning-based methods rely heavily on the large quantity and high quality of the training set. Limited labeled data restricts the performance of fully supervised algorithms. The method of the present invention utilizes a large amount of unlabeled data. Through techniques such as pseudo-label generation and consistency regularization, the model can learn richer feature representations with limited labeled data. This strategy effectively alleviates the problem of insufficient labeled data, improves the utilization efficiency of the model for unlabeled data, and thus achieves a significant improvement in segmentation accuracy. However, in the absence of labeled data, overfitting may occur. By introducing uncertainty estimation and consistency loss, not only the segmentation accuracy is improved, but also the robustness of the model to noise and outliers is enhanced. This robustness enables the model to more stably output high-quality segmentation results when facing complex bladder tumor images, further enhancing the performance of key indicators such as Dice and mIoU. At the same time, the improvement of the Sm and Em indicators shows that the model also performs well in structure preservation and edge detection, while the reduction of MAE indicates that the model has been improved to a certain extent in prediction accuracy.

[0112] 4.2 Comparison of the segmentation results of the dataset

[0113] To verify the effectiveness of the method in the bladder tumor segmentation task, extensive comparative experiments were conducted on the publicly accessible polyp dataset. According to the same settings as the bladder tumor dataset, experiments were carried out using 15% and 30% of the training set respectively.

[0114] The Kvasir-SEG dataset contains 1000 high-quality endoscopic images, which show diverse polyp appearances and imaging conditions and is a challenging test platform. Specifically, 90 and 180 images were randomly selected from 600 training images as labeled data, and the remaining 510 and 420 images were used as unlabeled data. In this way, the actual situation of limited labeled data was simulated to evaluate the performance of the method of the present invention in the semi-supervised learning scenario. The experimental results are shown in Table 2.

[0115] Table 2 contains 600 polyp images. The test results of various methods under 15% limited dataset training and the test percentage results of full-supervised training with 15%, 30%, and 100% training data

[0116] Methods labeled unlabeled Dice↑ sm↑ em↑ mIoU↑ MAE↓ acc↑ supervised 600 0 91.86 92.46 93.95 90.18 1.97 97.15 supervised-15% 90 0 88.47 88.98 93.12 81.98 3.95 96.05 supervised-30% 180 0 90.52 90.90 94.95 84.92 2.86 97.14 MT 90 510 87.81 89.72 91.86 81.64 3.93 96.21 UAMT 90 510 87.92 89.71 91.96 81.60 3.74 96.45 DTC 90 510 80.01 83.84 89.19 75.71 8.80 94.95 PolypMix 90 510 85.69 88.72 90.77 79.50 4.67 95.61 DSST 90 510 86.27 89.21 91.15 80.53 4.37 96.09 UDS-MT (the present invention) 90 510 88.67 89.22 92.21 80.77 4.26 95.86

[0117] The ETIS_LaribPolypDB dataset contains 196 images. According to the same training set division as the bladder tumor dataset, the data participating in training consists of 118 images, among which 18 images are used as labeled data, and the other 100 images are used as the test set for testing. The test results are shown in Table 3

[0118] Table 3 contains 118 polyp images. The test results of various methods under 15% limited dataset training and the test percentage results of full-supervised training with 15%, 30%, and 100% training data

[0119] Methods labeled unlabeled Dice↑ sm↑ em↑ mIoU↑ MAE↓ acc↑ supervised 118 0 80.08 85.74 89.22 85.29 2.31 97.79 supervised-15% 18 0 38.77 63.06 67.28 30.97 6.06 93.94 supervised-30% 36 0 44.89 64.50 71.90 35.48 5.98 94.02 MT 18 100 55.21 67.51 68.48 56.18 21.77 78.33 UAMT 18 100 70.16 80.49 83.48 66.11 8.96 92.07 DTC 18 100 28.25 53.12 60.21 36.91 24.07 91.17 PolypMix 18 100 48.45 68.92 69.75 41.09 8.33 94.58 DSST 18 100 50.54 61.09 63.54 46.43 28.03 71.34 UDS-MT (the present invention) 18 100 53.39 63.36 68.19 49.13 27.32 73.56

[0120] The CVC-conlonDB dataset contains 300 images. According to the same training set division as the bladder tumor dataset, the data participating in training has 180 images, among which 27 images are used as labeled data, and the other 120 images are used as the test set for testing. The test results are shown in Table 4

[0121] Table 4 contains 180 polyp images. The test results of various methods under 15% limited dataset training and the test percentage results of full-supervised training with 15%, 30%, and 100% training data

[0122] Methods labeled unlabeled Dice↑ sm↑ em↑ mIoU↑ MAE↓ acc↑ supervised 180 0 86.0 88.62 89.98 87.84 2.12 98.82 supervised-15% 27 0 39.58 59.30 64.57 29.94 10.41 89.59 supervised-30% 54 0 49.21 66.58 73.26 40.42 7.36 92.64 MT 27 153 73.04 79.27 82.98 66.21 6.05 94.30 UAMT 27 153 63.58 71.75 79.11 56.61 15.53 94.73 DTC 27 153 29.57 51.25 42.47 26.09 33.94 72.87 PolypMix 27 153 72.39 79.75 83.87 64.20 6.09 94.16 DSST 27 153 67.80 76.66 74.64 60.06 9.18 91.79 UDS-MT (the present invention) 27 153 73.44 79.94 76.33 65.15 8.46 94.94

[0123] 4.3 Test results of 30% limited dataset

[0124] To improve the effectiveness of the verification of the method of the present invention, various semi-supervised methods are tested on the bladder tumor dataset and the Kvasir-SEG dataset under 30% limited annotation. The percentage results are shown in Tables 5 and 6

[0125] Table 5 30% bladder tumor results

[0126] Methods labeled unlabeled Dice↑ sm↑ em↑ mIoU↑ MAE↓ acc↑ MT 374 872 80.39 84.53 88.28 72.44 4.48 95.61 UAMT 374 872 80.49 84.39 88.06 72.66 4.61 95.43 DTC 374 872 77.65 79.91 82.82 67.65 7.32 94.64 PolypMix 374 872 76.85 82.42 84.83 68.97 5.57 94.57 DSST 374 872 80.85 84.88 88.01 73.10 4.60 95.50 UDS-MT (the present invention) 374 872 81.95 85.22 89.07 73.91 4.39 95.66

[0127] Table 6 30% Kvasir-SEG test results

[0128] Methods labeled unlabeled Dice↑ sm↑ em↑ mIoU↑ MAE↓ acc↑ MT 180 420 89.03 90.26 92.97 82.92 3.72 96.32 UAMT 180 420 89.34 90.58 92.98 83.41 3.41 96.64 DTC 180 420 86.78 87.21 91.87 81.14 6.36 96.06 PolypMix 180 420 89.51 89.02 90.83 80.27 4.36 95.81 DSST 180 420 88.33 89.97 92.19 82.16 3.91 96.22 UDS-MT (the present invention) 180 420 90.55 90.33 93.19 83.16 3.72 96.37

[0129] (V) Ablation analysis

[0130] Experiments were conducted by training models with some components deleted or altered. The ablation study was carried out using the bladder tumor dataset and in accordance with the settings outlined in the network framework.

[0131] To verify the effectiveness and necessity of each module, ablation experiments were conducted on each proposed module. The mean teacher framework was used as the baseline, where the teacher and student networks shared the same PVT backbone network structure. As shown in Table 7, adding only the supervision branch to the base network of the mean teacher increased Dice by 2.56% and mIoU by 2.35%; introducing only uncertainty estimation increased Dice by 0.28% while slightly decreasing mIoU. In addition, the present invention improved performance by combining uncertainty estimation with consistency regularization and the introduction of pseudo-labels, resulting in an increase in dice and mIoU by 3.27% and 3.28% respectively.

[0132] Table 7 Results of evaluating the ablation rate of the model of the present invention

[0133] Method Dice miou MAE acc Baseline 76.77 68.32 6.19 94.14 +guided 79.33 70.67 5.42 94.64 +uncertainty 77.04 68.12 5.80 94.24 +guided+uncertainty 80.04 71.60 5.18 94.90

[0134] Conclusion: The present invention proposes a new semi-supervised task method that introduces a supervision branch, combines pseudo-labels and uncertainty estimation based on information entropy, and uses the mean teacher model as the underlying architecture. The supervision signal is provided through the prediction results of the supervision branch network to supervise the segmentation network to produce more accurate anatomical results. At the same time, a mask is generated by combining pseudo-labels with uncertainty estimation based on information entropy to simplify model coupling and enhance the boundary information of the prediction results, thereby improving the accuracy. In addition, a loss is introduced to facilitate learning the reliability region of accurate segmentation predictions. Experimental verification conducted on the bladder tumor dataset, Kvasir-SEG, ETIS_LaribPolypDB, and CVC-conlonDB datasets shows that the method of the present invention is superior to other methods. In addition, ablation experiments were also carried out to prove the effectiveness of the strategy proposed by the present invention.

[0135] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0136] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation, characterized in that, It includes the following steps: S1. Data acquisition: Acquire bladder tumor image data; S2. Data preprocessing: Preprocess the acquired bladder tumor image data; S3. Data partitioning: Partition the preprocessed bladder tumor image data into labeled data and unlabeled data; S4. Model construction: Construct a bladder tumor segmentation model based on a supervised branch, a student model, and a teacher model; S5. Model training: Train the bladder tumor segmentation model with labeled data and unlabeled data. After several training sessions, obtain the trained bladder tumor segmentation model; S6. Segmentation and recognition: Input the bladder tumor image data of the patient to be tested into the trained bladder tumor segmentation model, and evaluate the bladder tumor segmentation and recognition results.

2. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation according to claim 1, wherein in S3, a two-stream batch sampler is used to sample from labeled data and unlabeled data simultaneously to construct a training batch, and both labeled data and unlabeled data are included in each training batch.

3. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation according to claim 1, wherein in S4, the teacher model in the teacher network is used to implement the segmentation task, and the features extracted by the teacher model are supervised by the supervised branch. The encoder and decoder of the supervised branch have the same structure as the student model but independent parameters, which are respectively represented by θ and θ′ corresponding to the weight parameters of the student network and the teacher network. The weight parameters of the teacher network are updated through the EMA attention mechanism, that is, updated by combining the weight parameters θ of the student network and the smoothing coefficient α, where t represents the training time, specifically: θ' t = αθ' t-1 + (1 - α)θ t .

4. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation according to claim 1, wherein in S5, when inputting labeled data for training, the labeled data is processed by the supervised branch network and the student network to extract target features, and the output results are constrained by the consistency loss and the supervised loss; when inputting unlabeled data, the supervised branch, the student model, and the teacher model are processed simultaneously to generate segmentation results, and the consistency loss and the cross-entropy loss are used for constraint among them.

5. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation according to claim 4, wherein in the supervised branch, uncertainty estimation is introduced, and the segmentation result is used to generate pseudo-labels and a mask. The pseudo-labels are used to calculate the loss with the segmentation results generated by the student model and the teacher model under the constraint of the mask.

6. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation according to claim 5, wherein the method of generating pseudo-labels and a mask from the segmentation result is as follows: Y g (u j ) = argmax(softmax(P g (u j ))) Among them, P g (u j ) is the predicted probability distribution generated by the supervision branch, and the corresponding pseudo-label generated is Y g (u j ).

7. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation according to claim 5, wherein the uncertainty estimation introduced in the supervised branch is: U = -∑ j Y g (u j ) log(Y g (u j )) Set the dynamic threshold threshold: T threshold = (0.75 + 0.25 * sigmoid_rampup(t, T)) Select the regions with relatively high reliability as the mask according to the threshold threshold: mask = (U < T threshold ) where T is the total number of training times.

8. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation according to claim 5, characterized in that The loss function includes the loss function of labeled data and the loss function of unlabeled data, and the total loss function is: L totel = L1 + L2 where L1 is the loss function of labeled data and L2 is the loss function of unlabeled data; The loss function of labeled data is: L1 = l s (P g,T (x i ), y i ) + λl sc (P g (x i ), P T (x i )) l s = L Dice (P g (x i ), y i ) + L Dice (P T (x i ), y i ) l sc = L CE (P g (x i ), P T (x i )) where l s is the supervision loss of the prediction results of the supervision branch and the student model and the true label, l sc is the consistency loss between the supervision branch and the prediction results of the student model, λ is the weight of the consistency loss, P g,T is the comprehensive representation of the prediction probability output results of the supervision branch and the teacher model, P g is the prediction result of the supervision branch, x i is the labeled data sample, y i is the true label corresponding to the labeled output sample, P T is the prediction result probability of the teacher model, L CE is the cross-entropy loss calculation, L Dice is the Dice loss calculation; The loss function of unlabeled data is: L2＝l c +l p l c = L CE (P g (u j ), P T (u j )) + L CE (P g (u j ), P S (u j )) + L CE (P S (u j ), P T (u j )) where, l c is the consistency loss between the segmentation results of the supervision branch, the student model, and the teacher model, l p is the boundary information loss, u j is the unlabeled data sample, P S is the probability of the prediction result of the student model.

9. A semi-supervised bladder tumor medical image segmentation method based on a supervised branch and uncertainty estimation according to claim 1, characterized in that Six indicators, including the Dice coefficient, mean intersection over union, sensitivity, specificity, mean absolute error, and accuracy, are used in S6 to evaluate the bladder tumor segmentation and recognition results.

Images