An uncertainty-aware semi-supervised pancreas segmentation method based on evidence learning

Abstract

The present invention discloses an uncertainty-aware semi-supervised pancreatic segmentation method based on evidence learning. First, deformable convolution is introduced to construct a deformable U-Net with a receptive field that can adapt to different scales and deformation sizes, so as to better extract the variable geometric features of the pancreas; secondly, based on the MT framework, evidence deep learning is introduced to model the prediction uncertainty of the model, and the output of the model is regarded as evidence and formalized as a Dirichlet distribution. The prediction category probability and uncertainty are calculated through subjective logic, and reliable pseudo-labels on unlabeled data are generated according to the uncertainty information to guide model learning; finally, a boundary loss that is more sensitive to the target boundary segmentation quality is introduced to make the model pay more attention to the segmentation effect of the target boundary. The present invention overcomes the problem that the existing semi-supervised learning methods are difficult to ensure the quality of pseudo-labels, balance the estimation accuracy and computational cost, and provides a more targeted solution for the pancreatic segmentation task.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to an uncertainty-aware semi-supervised pancreas segmentation method based on evidence learning. Background Art

[0002] In recent years, deep learning-based image segmentation methods have been widely used in the field of medical image analysis due to their excellent performance, and have achieved good performance in multiple medical image tasks. However, this type of fully supervised method relies on a large amount of labeled data, while in clinical practice, high-quality labeled data is often difficult to obtain. Manually labeling medical image data at the pixel and voxel levels is time-consuming and costly, and requires doctors to have rich professional knowledge and experience. In practical applications, due to the small size of available medical image segmentation datasets, insufficient model training will lead to overfitting. This limits the further development of deep learning methods in medical image analysis. In addition, for the pancreatic organ, due to its complex anatomical structure and large shape variability, the pancreas is located in a complex background in the CT image, and the contrast with the surrounding tissue is weak, resulting in blurred edges of the pancreas that are difficult to define.

[0003] In the face of the problem of scarce labeled data available in real clinical application scenarios, semi-supervised learning is an effective solution. It can train and optimize the model with a small amount of labeled data, and learn with information from a large amount of unlabeled data to capture the distribution of data, alleviating the need for labeled data. Currently, the more popular semi-supervised learning methods can be roughly divided into pseudo-label self-training methods and consistency regularization.

[0004] Among them, the pseudo-label self-training method uses the model trained on the labeled data to make predictions on the unlabeled data, and uses the predictions with confidence exceeding a certain set threshold as pseudo-labels, and then combines the pseudo-labels with the labeled data to retrain the model, and repeats this process, such as FixMatch and Cross Pseudo Supervision (CPS). The consistency regularization method forces the model to produce stable and consistent predictions on the same unlabeled data under different perturbations, such as input perturbations, feature perturbations, and network perturbations, such as Mean Teacher (MT) and Cross Consistency-Training (CCT).

[0005] However, these methods do not take into account the quality of pseudo-labels. Due to the poor calibration error of neural networks, the models are usually overconfident, for example, they still have high confidence in wrong predictions, which will produce confirmation bias and seriously affect model performance.

[0006] In order to improve the quality of pseudo-labels, some uncertainty-aware methods have been proposed, such as the Uncertainty Aware Mean Teacher (UA-MT) method based on Monte Carlo dropout, the information entropy-based method, and the prediction variance-based method. However, these methods still have some problems. For example, although the uncertainty assessment method based on MC-dropout can be mathematically guaranteed by Bayesian theory, it requires multiple sampling operations and has a high training cost. The information entropy-based method reduces uncertainty by forcing the network to make more confident predictions, and the prediction variance-based method estimates uncertainty by calculating multiple predictions. Although these two methods have low computational costs, due to the lack of theoretical support, the uncertainty estimation accuracy is low, resulting in unstable quality of the generated pseudo-labels.

[0007] Faced with the challenges of the pancreas organ being difficult to define due to its complex background, large shape variability, and blurred edges due to low contrast with other organs, most of the current deep learning-based methods achieve stronger feature expression by fusing multi-scale features, combining contextual information, or adding attention mechanisms. However, due to the fixed geometric structure of the building blocks in conventional convolutional neural networks, such as convolution units and pooling units, they can only be sampled at fixed positions, which limits their ability to adapt to highly deformable target features. In addition, the AP indicators based on Mask IoU, such as Dice loss and mIoU, which are commonly used in segmentation tasks, only focus on the overall segmentation effect of the target to be segmented, but are not sensitive to the accuracy of the target segmentation boundary area. As a result, the current segmentation network predictions are not accurate enough, the edges are rough, and the segmentation effect is still not ideal.

[0008] In summary, due to the selection of pseudo-labels, existing semi-supervised learning methods have difficulty balancing the quality of pseudo-labels, uncertainty estimation accuracy and computational cost. For the pancreas segmentation task, the commonly used network is limited by its fixed convolutional structure and the loss function that does not focus on the boundary segmentation effect, resulting in unsatisfactory segmentation results for objects such as the pancreas that have a high degree of scale and shape changes. Summary of the invention

[0009] In order to overcome the defects of the prior art, the present invention discloses an uncertainty-aware semi-supervised pancreatic segmentation method based on evidence learning. First, deformable convolution is introduced to construct a deformable U-Net with a receptive field that can adapt to different scales and deformation sizes to better extract the variable geometric features of the pancreas; secondly, based on the MT framework, evidence deep learning is introduced to model the prediction uncertainty of the model, and the output of the model is regarded as evidence and formalized as a Dirichlet distribution. The prediction category probability and uncertainty are calculated through subjective logic, and reliable pseudo-labels on unlabeled data are generated according to the uncertainty information to guide model learning; finally, a boundary loss that is more sensitive to the target boundary segmentation quality is introduced to make the model pay more attention to the segmentation effect of the pancreatic target boundary and improve the quality of the target segmentation boundary.

[0010] In order to achieve the above technical objectives, the specific technical solutions of the present invention are as follows:

[0011] An uncertainty-aware semi-supervised pancreas segmentation method based on evidence learning, the method comprising the following steps:

[0012] Step 1: Obtain and preprocess the pancreatic CT dataset.

[0013] Step 2: Divide the data set preprocessed in step 1 into a training set and a test set, where the training set is D = D l ∪D u ,in, represents a labeled dataset of N labeled CT slices, is an unlabeled dataset with M unlabeled slices, and Represent the labeled CT images and the unlabeled CT images respectively, y i Represents the corresponding true label.

[0014] Step 3: Based on the U-Net model, replace the conventional convolutions in all convolutional layers of the encoder with deformable convolutions to construct a deformable U-Net that can adapt to the features of targets with different deformations.

[0015] Step 4: Use the deformable U-Net obtained in step 3 as the backbone model. Based on the MT framework, introduce evidence learning and design an uncertainty-aware semi-supervised learning framework based on evidence learning. Use evidence deep learning to model the prediction uncertainty of the model so that the teacher model can give the predicted segmentation results and corresponding uncertainty estimates in a single forward reasoning. The teacher model's prediction of unlabeled data is used as the learning target of the student model. The student model learns from reliable pseudo-labels based on uncertainty information through uncertainty-aware consistency loss, and adds boundary loss for constraints, so that the model pays more attention to the segmentation effect of the target boundary area.

[0016] Step 5: Train on the training set D, where for the labeled data D l , the student model minimizes D l The supervised loss on the unlabeled data D is optimized. u , the student model is optimized by uncertainty-aware consistency loss. The teacher model does not participate in training, and the weights are updated by the exponential moving average (EMA) of the student model weights until convergence.

[0017] Step 6: Use the final trained teacher model for pancreatic organ segmentation.

[0018] Furthermore, the preprocessing in step 1 is specifically as follows: collecting three-dimensional pancreatic CT data and performing slice processing, removing irrelevant background images that do not contain the target area to be segmented, and obtaining a two-dimensional slice image; adjusting the window width to 340HU and the window level to 10HU according to the pancreatic tissue density, and clipping the original image CT value to [-100, 240] through HU constraint, setting the CT value below -100 to -100, and setting the CT value above 240 to 240. The image is grayscale normalized and center cropped to place the pancreatic target at the center of the image, reduce the interference of irrelevant background, and finally normalize the image to obtain a preprocessed image.

[0019] Furthermore, the specific process of step 3 is as follows: First, the U-Net model consists of three parts: an encoder, a decoder, and a jump connection. The encoder includes four downsampling modules, each of which includes two convolutional layers of size 3×3 and a maximum pooling layer of 2×2. The decoder includes four upsampling modules, each of which is upsampled by two convolutional layers of size 3×3 and subsequent deconvolution operations. While upsampling, the feature maps output by the corresponding downsampling modules transmitted by the jump connection are fused. The 3x3 convolutions in all convolutional layers in the U-Net encoder are replaced with 3x3 deformable convolutions. Unlike conventional convolutions, deformable convolutions add a learnable offset, which allows the convolution kernel to offset according to the input feature map and focus on the target area of ​​interest, thereby achieving adaptive positioning of objects of different shapes.

[0020] Specifically, conventional convolution consists of two steps. First, a regular grid R is used to sample the input feature map x, and then the sum of the weighted sampled values ​​w is calculated as the output of the current position. For each position p0 on the output feature map y, there is:

[0021]

[0022] In the above formula, pn is the enumeration of the convolution kernel sampling area, and w is the weight of the convolution kernel. In deformable convolution, an offset {Δp n |n=1,N}, where N=|R|, so the above formula becomes the following form:

[0023]

[0024] After the offset, the sampling area of ​​the convolution operation becomes an irregular shape. n Usually a decimal, resulting in p0+p n +Δp n The value position of is not an integer and there is no real point corresponding to it on the feature map, so it is necessary to use bilinear interpolation and calculate it as:

[0025]

[0026] Where p represents a real point in the feature map x, q represents a point at an integer position, and G(·,·) is a bilinear interpolation kernel function. Since the feature map is two-dimensional, the kernel function can be decomposed into the product of two one-dimensional kernel functions:

[0027] G(q,p)=g(q x , p x )·g(q y , p y )

[0028] where q x With p x are the coordinates of points q and p on the x-axis, q y With p y are the coordinates of points q and p on the y-axis.

[0029] By replacing the conventional convolution in the U-Net encoder part with a deformable convolution with a learnable offset, a deformable U-Net with a receptive field that can adapt to scale and shape changes is constructed.

[0030] Furthermore, in step 4, the uncertainty-aware semi-supervised learning framework process based on evidence learning is specifically as follows:

[0031] The framework is based on the MT architecture and contains a teacher model and a student model, where both the teacher model and the student model use the deformable U-Net obtained in step 3. First, the segmentation uncertainty of the model is modeled. The traditional convolutional neural network uses a softmax activation function in the last layer to produce a category probability distribution, but does not consider the uncertainty of the model. The output of the model may be overconfident, for example, there may be a high confidence in the wrongly predicted samples. Therefore, evidential deep learning is used to model segmentation uncertainty, and the output of the model is regarded as evidence. The evidence represents the amount of information collected by the model from the data to support the classification of a given sample as a certain category, and is parameterized as a Dirichlet distribution. The subjective logic theory is used to quantify the predicted probability and uncertainty of the segmentation category with the Dirichlet distribution in a single forward inference to generate trustworthy predictions. Specifically:

[0032] For a K classification task, given an input x i , replace the last softmax layer of the network with the exponential activation function g, and the evidence vector e = [e1, ...e k ] is obtained by the transformation function g.

[0033]

[0034] Among them, K is an integer representing the number of categories, f out (x i ) is the original network for input x i The output before the softmax layer, 0<τ<1 is the scaling parameter, set The belief quality, uncertainty and predicted probability of category k are calculated by subjective logic as follows:

[0035]

[0036] in can be viewed as the parameters of the Dirichlet distribution, which models the density of the split probability and uncertainty.

[0037] The Dirichlet density function is defined as:

[0038]

[0039] where p i is the pixel prediction value, B(a i ) is parameter a i K-dimensional polynomial Beta function, S K is a K-dimensional simplex.

[0040] After uncertainty modeling, the model can be optimized through the loss function so that the model learns the evidence of positive samples of different categories. Specifically, the framework loss function consists of the supervised loss on the labeled data and the uncertainty-aware unsupervised consistency loss on the unlabeled data, which is defined as:

[0041]

[0042] Among them, L s represents the supervision loss, which is used to evaluate the quality of network output on labeled data, L c represents the unsupervised consistency loss used to measure the consistency between the teacher model and the student model under the same input and different perturbations of unlabeled data. f(·) represents the segmentation neural network, θ and θ′ represent the weights of the student and teacher models, respectively, and η and η′ represent different perturbation operations (such as adding noise to the input and network dropout). λ1 is the weight that controls the proportion of unsupervised loss. is a Gaussian warm-up function, which is used to adjust the size of the unsupervised loss. It increases as the training progresses. t is the current number of iterations, and t max The total number of iterations

[0043] Supervision loss L s By the evidence-based cross entropy loss (evidence loss) L Evi 、Dice loss L Dice and the Boundary loss L based on the boundary segmentation quality metric Boundary Composition. Defined as:

[0044] L s =λ2L Evi +λ3L Dice +λ4L Boundary

[0045] Among them, λ2, λ3 and λ4 are the weights for controlling the proportion of evidence loss, Dice loss and Boundary loss. The definition of evidence loss is as follows:

[0046] L Evi =L θ +βL KL

[0047] Where β is the annealing coefficient, set to Use evidence-based cross entropy loss L θ Let pixel prediction value p i Close to label y i , p i The density of a i Parameterized Dirichlet distribution. L θ Defined as:

[0048]

[0049] where ψ(·) is the inverse gamma function. By optimizing L θ , generating evidence of positive samples of different categories. However, L θ There is no guarantee that negative samples will produce close to zero evidence. Therefore, the Kullback-Leibler (KL) divergence is added to the loss function to penalize the divergence with negative samples, defined as:

[0050]

[0051] where Γ(·) is the gamma function, D(p i ,1) is a uniform Dirichlet distribution,

[0052] The subjective logic model has two parts: belief quality, i and the uncertain part u i . Evidence loss generates evidence to reduce uncertainty. However, evidence loss evaluates each pixel equally, ignoring the relationship between pixels in the task. In the case of class imbalance such as pancreatic segmentation (pancreatic pixels account for <5%), it is difficult to train and fit the pancreatic target (foreground), causing the prediction results of the network model to be more inclined to the background. Therefore, this framework is trained together with Dice loss. By optimizing L Evi , the model generates evidence for positive samples and reduces the evidence for negative samples. Dice Constrain the relationship between different pixels. Dice loss is as follows:

[0053]

[0054] Where N is the total number of pixels, y i Represents the true label value of the pixel, p i Represents the pixel prediction value.

[0055] At the same time, in order to make the model pay more attention to the segmentation effect of the boundary of the target area, the Boundary loss based on the boundary segmentation quality metric is used for constraint. Unlike the Dice loss, the Boundary loss does not consider all pixels, but only evaluates the pixels directly on or near the mask boundary. Specifically, given two segmentation masks G and P, the Boundary loss first calculates the original pixel set within the distance d of each mask boundary, and then calculates the intersection of the two sets, as defined by the following formula:

[0056]

[0057] Among them, G dis the set of pixels within the boundary d of the true label, P d is the set of pixels within the prediction area boundary d, y i With p i is the pixel’s category label and prediction value.

[0058] Guided by the estimated uncertainty, the output of the teacher model is evaluated to filter out relatively unreliable (high uncertainty) predictions and only select reliable predictions as the target for the student model to learn. c Designed as the mean square error (MSE) of the teacher model and the student model predictions, it is only used for reliable predictions that have been filtered through uncertainty information and is defined as follows:

[0059]

[0060] Where N is the total number of pixels, f i ′ and f i are the predictions of the teacher model and the student model for the i-th pixel, u i is the estimated uncertainty of the i-th pixel, H is the threshold for selecting credible targets, is the indicator function. By using the uncertainty-aware consistency loss during training, both the student and the teacher can learn more reliable knowledge, thus reducing the overall uncertainty of the model.

[0061] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0062] 1. The present invention replaces conventional convolution with a deformable convolution with a learnable offset, so that the convolution kernel can be offset according to the input feature map and focus on the target area of ​​interest, which can adapt to the variable geometric shapes of the pancreatic organ.

[0063] 2. Based on the MT semi-supervised learning framework, the present invention uses evidence deep learning to model the prediction uncertainty of the model, so that the teacher model can give the prediction segmentation results and the corresponding uncertainty estimation in one forward reasoning, and guide the student model to learn from more reliable pseudo-labels based on the uncertainty information. Compared with the existing technology, the uncertainty-aware semi-supervised learning framework designed based on evidence learning can improve the quality of pseudo-labels, reduce confirmation bias, and ensure the accuracy of uncertainty estimation and computational cost.

[0064] 3. The present invention uses a boundary loss function that is more sensitive to boundary segmentation quality, so that the model pays more attention to the effect of the target segmentation boundary and fully learns the pancreatic boundary feature information. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1is a structural diagram of a deformable convolution module in an embodiment of the present invention;

[0066] Figure 2 Schematic diagram of conventional convolution and deformable convolution sampling positions in an embodiment of the present invention;

[0067] Figure 3 : is a structural diagram of a deformable U-Net model in an embodiment of the present invention;

[0068] Figure 4 This is a framework diagram of uncertainty-aware semi-supervised learning based on evidence learning in an embodiment of the present invention;

[0069] Figure 5 Schematic diagram of Dice loss and Boundary loss in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to more clearly describe the technical solution of the present invention, the specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings. The specific implementation modes described herein are only used to explain and illustrate the present invention and are not used to limit the present invention.

[0071] An uncertainty-aware semi-supervised pancreatic segmentation method based on evidence learning. This method first introduces deformable convolution to construct a deformable U-Net with a receptive field that can adapt to different scales and deformation sizes. Secondly, based on the MT framework, with deformable U-Net as the backbone model, the prediction uncertainty of the model is modeled using evidence deep learning. The output of the model is regarded as evidence and formalized as a Dirichlet distribution. The predicted category probability and uncertainty are calculated through subjective logic, and reliable pseudo-labels on unlabeled data are generated based on the uncertainty information to guide model learning. Finally, a boundary loss that is more sensitive to the quality of target boundary segmentation is introduced to make the model pay more attention to the segmentation effect of the pancreatic target boundary and improve the quality of the target segmentation boundary. Specifically, Figure 1-5 As shown, it includes the following steps, which are executed in sequence:

[0072] Step 1: Obtain and preprocess the medical image dataset, specifically:

[0073] The three-dimensional pancreatic CT data were collected and sliced, and the irrelevant background images that did not contain the target area to be segmented were eliminated to obtain a two-dimensional slice image; according to the pancreatic tissue density, the window width was adjusted to 340HU, and the window level was adjusted to 10HU. The original image CT value was truncated to [-100, 240] through HU constraint, and the CT value below -100 was set to -100, and the CT value above 240 was set to 240. Grayscale normalization and center cropping were performed to place the pancreatic target in the center of the image to reduce the interference of irrelevant background, and finally the image pixels were normalized to obtain the preprocessed image.

[0074] Step 2: Divide the data set preprocessed in step 1 into a training set and a test set, where the training set is D = D l ∪D u ,in, represents a labeled dataset of N labeled CT slices, is an unlabeled dataset with M unlabeled slices, and Represent the labeled CT images and the unlabeled CT images respectively, y i Represents the corresponding true label.

[0075] Step 3: Based on the U-Net model, deformable convolution is introduced to construct a deformable U-Net that can adapt to the features of different deformed targets. Specifically:

[0076] First, the U-Net model consists of three parts: encoder, decoder, and skip connection. The encoder contains four downsampling modules, each of which contains two 3×3 convolutional layers and a 2×2 maximum pooling layer. The decoder includes four upsampling modules, each of which is upsampled by two 3×3 convolutional layers and subsequent deconvolution operations. While upsampling, the feature maps output by the corresponding downsampling modules transmitted by the skip connection are fused.

[0077] Replace all 3x3 convolutions in all convolutional layers in the U-Net encoder with 3x3 deformable convolutions. Unlike conventional convolutions, deformable convolutions add a learnable offset, allowing the convolution kernel to offset according to the input feature map and focus on the target area of ​​interest, thereby achieving adaptive positioning of objects of different shapes.

[0078] Specifically, the deformable convolution is constructed as follows: the conventional convolution consists of two steps. First, a regular grid is used to sample the input feature map x, and then the sum of the weighted sampled values ​​w is calculated as the output of the current position. For each position p0 on the output feature map y, there is:

[0079]

[0080] In the above formula, p n is the enumeration of the convolution kernel sampling area, and w is the weight of the convolution kernel. In deformable convolution, an offset {Δp n |n=1,N}, where N=|R|, so the above formula becomes the following form:

[0081]

[0082] After the offset, the sampling area of ​​the convolution operation becomes an irregular shape. n Usually a decimal, resulting in p0+p n +Δp n The value position of is not an integer and there is no real point corresponding to it on the feature map, so it is necessary to use bilinear interpolation and calculate it as:

[0083]

[0084] Where p represents a real point in the feature map x, q represents a point at an integer position, and G(·,·) is a bilinear interpolation kernel function. Since the feature map is two-dimensional, the kernel function can be decomposed into the product of two one-dimensional kernel functions:

[0085] G(q,p)=g(q x , p x )·g(q y , p y )

[0086] where q x With p x are the coordinates of points q and p on the x-axis, q y With p y are the coordinates of points q and p on the y-axis.

[0087] Deformable convolution Figure 1 As shown, the schematic diagram of deformable convolution and conventional convolution sampling points is as follows Figure 2 By replacing the conventional convolution of the U-Net encoder part with a deformable convolution with a learnable offset, a deformable U-Net with a receptive field that can adapt to scale and shape changes is constructed, as shown in Figure 3 shown.

[0088] Step 4: Use the deformable U-Net obtained in step 3 as the backbone model. Based on the MT framework, introduce evidence learning and design an uncertainty-aware semi-supervised learning framework based on evidence learning, such as Figure 4 Specifically:

[0089] The framework is based on the MT architecture and contains a teacher model and a student model, where both the teacher model and the student model use the deformable U-Net obtained in step 3. First, the segmentation uncertainty of the model is modeled using evidence deep learning. The traditional convolutional neural network uses a softmax activation function in the last layer to produce a category probability distribution, but does not consider the uncertainty of the model. The output of the model may be overconfident, for example, there may be a high confidence in the wrongly predicted samples. Therefore, the segmentation uncertainty is modeled using evidence deep learning, and the output of the model is regarded as evidence. The evidence represents the amount of information collected by the model from the data to support the classification of a given sample into a specific category, and is parameterized as a Dirichlet distribution. The subjective logic theory is used to quantify the predicted probability and uncertainty of the segmentation category with the Dirichlet distribution in a single forward inference to generate trustworthy predictions. Specifically:

[0090] For a K classification task, given an input x i , replace the last softmax layer of the network with the exponential activation function g, and the evidence vector e = [e1, ...e k ] is obtained by the transformation function g.

[0091]

[0092] Among them, K is an integer representing the number of categories, f out (x i ) is the original network for input x i The output before the softmax layer, 0<τ<1 is the scaling parameter, set The belief quality, uncertainty and predicted probability of category k are calculated by subjective logic as follows:

[0093]

[0094] in It can be seen as the parameter of the Dirichlet distribution, which models the density of the segmentation probability and uncertainty. The Dirichlet density function is defined as:

[0095]

[0096] where p i is the pixel prediction value, B(a i ) is parameter a i K-dimensional polynomial Beta function, S K is a K-dimensional simplex.

[0097] After uncertainty modeling, the model can be optimized through the loss function so that the model learns the evidence of positive samples of different categories. Specifically, the framework loss function consists of the supervised loss on the labeled data and the uncertainty-aware unsupervised consistency loss on the unlabeled data, which is defined as:

[0098]

[0099] Among them, L s represents the supervision loss, which is used to evaluate the quality of network output on labeled data, L c represents the unsupervised consistency loss used to measure the consistency between the teacher model and the student model under the same input and different perturbations of unlabeled data. f(·) represents the segmentation neural network, θ and θ′ represent the weights of the student and teacher models, respectively, and η and η′ represent different perturbation operations (such as adding noise to the input and network dropout). λ1 is the weight that controls the proportion of unsupervised loss. is a Gaussian warm-up function, which is used to adjust the size of the unsupervised loss. It increases as the training progresses. t is the current number of iterations, and t mx The total number of iterations

[0100] Supervision loss L s By the evidence-based cross entropy loss (evidence loss) L Evi 、Dice loss L Dice and the Boundary loss L based on the boundary segmentation quality metric Boundary Composition. Defined as:

[0101] L s =λ2L Evi +λ3L Dice +λ4L Boundary

[0102] Among them, λ2, λ3 and λ4 are the weights for controlling the proportion of evidence loss, Dice loss and Boundary loss. The definition of evidence loss is as follows:

[0103] L Evi =L θ +βL KL

[0104] Where β is the annealing coefficient, set to Use evidence-based cross entropy loss L θ Let pixel prediction value p i Close to label y i , p i The density of a i Parameterized Dirichlet distribution. L θ Defined as:

[0105]

[0106] where ψ(·) is the inverse gamma function. By optimizing L θ , generating evidence of positive samples of different categories. However, L θ There is no guarantee that negative samples will produce close to zero evidence. Therefore, the Kullback-Leibler (KL) divergence is added to the loss function to penalize the divergence with negative samples, defined as:

[0107]

[0108] where Γ(·) is the gamma function, D(p i ,1) is a uniform Dirichlet distribution,

[0109] The subjective logic model has two parts: belief quality, i and the uncertain part u i . Evidence loss generates evidence to reduce uncertainty. However, evidence loss evaluates each pixel equally, ignoring the relationship between pixels in the task. In the case of class imbalance such as pancreatic segmentation (pancreatic pixels account for <5%), it is difficult to train and fit the pancreatic target (foreground), causing the prediction results of the network model to be more inclined to the background. Therefore, this framework is trained together with Dice loss. By optimizing L Evi , the model generates evidence for positive samples and reduces the evidence for negative samples. Dice Constrain the relationship between different pixels. Dice loss is as follows:

[0110]

[0111] Where N is the total number of pixels, y i Represents the true label value of the pixel, p i Represents the pixel prediction value.

[0112] At the same time, in order to make the model pay more attention to the segmentation effect of the boundary of the target area, the Boundary loss based on the boundary segmentation quality metric is used for constraint. Unlike the Dice loss, the Boundary loss does not consider all pixels, but only evaluates the pixels directly on or near the mask boundary, such as Figure 5 Specifically, given two segmentation masks G and P, the Boundary loss first calculates the set of original pixels within the boundary d of each mask, and then calculates the intersection and union ratio of these two sets. The Boundary loss is defined as follows:

[0113]

[0114] Among them, G d is the set of pixels within the boundary d of the true label, P d is the set of pixels within the prediction area boundary d, y i With p i is the pixel’s category label and prediction value.

[0115] Guided by the estimated uncertainty, the output of the teacher model is evaluated to filter out relatively unreliable (high uncertainty) predictions and only select reliable predictions as the target for the student model to learn. c Designed as the mean square error (MSE) of the teacher model and the student model predictions, it is only used for reliable predictions that have been filtered through uncertainty information and is defined as follows:

[0116]

[0117] Where N is the total number of pixels, f i ′ and f i are the predictions of the teacher model and the student model for the i-th pixel, u i is the estimated uncertainty of the i-th pixel, H is the threshold for selecting credible targets, is the indicator function. Through our uncertainty-aware unsupervised consistency loss during training, both students and teachers can learn more reliable knowledge, thus reducing the overall uncertainty of the model.

[0118] Step 5: Train on the training set D, where for the labeled data D l , the student model minimizes D l The supervised loss on the unlabeled data D is optimized. u , the student model is optimized by uncertainty-aware consistency loss. The teacher model does not participate in training, and the weights are updated by the exponential moving average (EMA) of the student model weights until convergence.

[0119] Step 6: Use the final trained teacher model for pancreatic organ segmentation.

[0120] The present invention is described in detail above in conjunction with the accompanying drawings, but the specific implementation of the present invention is not limited to the above-mentioned manner. For those of ordinary skill in the art, several variations, improvements and substitutions can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. As long as various non-substantial improvements are made using the inventive concept and technical solution of the present invention, or the inventive concept and technical solution are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. An uncertainty-aware semi-supervised pancreas segmentation method based on evidence learning, characterized in that: The method comprises the following steps: Step 1, obtain and preprocess the pancreatic CT dataset; Step 2: Divide the data set preprocessed in step 1 into a training set and a test set, where the training set is D = D l ∪D u ,in, represents a labeled dataset of N labeled CT slices, is an unlabeled dataset with M unlabeled slices, and Represent the labeled CT images and the unlabeled CT images respectively, y i Indicates the corresponding true label; Step 3: Based on the U-Net model, replace the conventional convolutions in all convolutional layers of the encoder with deformable convolutions to construct a deformable U-Net that can adapt to the features of different deformed targets. Step 4, using the deformable U-Net obtained in step 3 as the backbone model, based on the MT framework, introducing evidence learning, and designing an uncertainty-aware semi-supervised learning framework based on evidence learning; the framework first uses evidence deep learning to model the prediction uncertainty of the model, so that the teacher model gives the predicted segmentation result and the corresponding uncertainty estimate in a single forward reasoning, and the prediction of the teacher model on the unlabeled data is used as the learning target of the student model. The student model learns from reliable pseudo-labels according to the uncertainty information through the uncertainty-aware consistency loss; then, the boundary loss that is more sensitive to the boundary segmentation quality is added as a constraint, so that the model pays more attention to the segmentation effect of the target boundary area; Step 5: Train on the training set D, where for the labeled data D l , the student model minimizes D l The supervised loss on the unlabeled data D is optimized. u , the student model is optimized through uncertainty-aware consistency loss; the teacher model does not participate in gradient updates, and the parameters are updated through the exponential moving average of the student model parameters until convergence; Step 6: Use the final trained teacher model for pancreatic organ segmentation.

2. The uncertainty-aware semi-supervised pancreas segmentation method based on evidence learning as claimed in claim 1, characterized in that: The step 1 specifically comprises: acquiring three-dimensional pancreatic CT data and performing slice processing, removing irrelevant background images that do not contain the target area to be segmented, and obtaining a two-dimensional slice image; adjusting the window width to 340HU and the window level to 10HU according to the pancreatic tissue density, and clipping the original image CT value to [-100, 240] through HU constraint, setting the CT value below -100 to -100, and setting the CT value above 240 to 240; and performing grayscale standardization and center cropping on the image to place the pancreatic target at the center of the image and reduce the interference of irrelevant background, and finally performing normalization on the image to obtain a preprocessed image.

3. The uncertainty-aware semi-supervised pancreas segmentation method based on evidence learning as claimed in claim 1, characterized in that: The step 3 is specifically as follows: First, the U-Net model consists of three parts: encoder, decoder, and skip connection; The encoder consists of four downsampling modules, each of which consists of two 3×3 convolutional layers and a 2×2 maximum pooling layer. The decoder consists of four upsampling modules, each of which is upsampled by two 3×3 convolutional layers and subsequent deconvolution operations. While upsampling, the feature maps output by the corresponding downsampling modules transmitted by the jump connection are fused. Replace all 3x3 convolutions in all convolutional layers in the U-Net encoder with 3x3 deformable convolutions; Specifically, conventional convolution consists of two steps. First, a regular grid R is used to sample the input feature map x, and then the sum of the weighted sampled values ​​w is calculated as the output of the current position. For each position p0 on the output feature map y, there is: In the above formula, p n is the enumeration of the convolution kernel sampling area, and w is the weight of the convolution kernel; In the deformable convolution, an offset {Δp n |n=1,N}, where N=|R|, so the above formula becomes the following form: After the offset, the sampling area of ​​the convolution operation becomes an irregular shape; Due to Δp n Usually a decimal, resulting in p0+p n +Δp n The value position of is not an integer and there is no real point corresponding to it on the feature map, so it is necessary to use bilinear interpolation and calculate it as: Where p represents a real point in the feature map x, q represents a point at an integer position, and G(·,·) is a bilinear interpolation kernel function; Since the feature map is two-dimensional, the kernel function can be decomposed into the product of two one-dimensional kernel functions: G(q,p)=g(q x ,p x )·g(q y ,p y ), where q x With p x are the coordinates of points q and p on the x-axis, q y With p y are the coordinates of points q and p on the y-axis.

4. The uncertainty-aware semi-supervised pancreas segmentation method based on evidence learning according to claim 1, characterized in that: The uncertainty-aware semi-supervised learning framework process based on evidence learning in step 4 is specifically as follows: The framework is based on the MT architecture and includes a teacher model and a student model, where both the teacher model and the student model use the deformable U-Net obtained in step 3; First, we use evidence deep learning to model the segmentation uncertainty of the model. Specifically: For a K classification task, given an input x i , replace the last softmax layer of the network with the exponential activation function g, and the evidence vector e = [e1, ...e k ] is obtained by transforming function g; Among them, K is an integer representing the number of categories, f out (x i ) is the original network for input x i The output before the softmax layer, 0<τ<1 is the scaling parameter, set The belief quality, uncertainty and predicted probability of category k are calculated by subjective logic as follows: in is the parameter of the Dirichlet distribution, which models the density of the partition probability and uncertainty; The Dirichlet density function is defined as: where p i is the pixel prediction value, B(a i ) is parameter a i K-dimensional polynomial Beta function, S K is a K-dimensional simplex; Then, the model is optimized through the loss function so that the model learns the evidence of positive samples of different categories. Specifically, the framework loss function consists of the supervised loss on the labeled data and the uncertainty-aware unsupervised consistency loss on the unlabeled data, which is defined as: Among them, L s represents the supervision loss, which is used to evaluate the quality of network output on labeled data, L c represents the unsupervised consistency loss used to measure the consistency between the teacher model and the student model under the same input and different perturbations of unlabeled data; f(·) represents the segmentation neural network, θ and θ′ represent the weights of the student and teacher models, η and η′ represent different perturbations; λ1 is the weight controlling the proportion of unsupervised loss; is a Gaussian warm-up function, which is used to adjust the size of the unsupervised loss. It increases as the training progresses. t is the current number of iterations, and t max is the total number of iterations; Supervision loss L s By the evidence-based cross entropy loss L Evi 、Dice loss L Dice and the Boundary loss L based on the boundary segmentation quality metric Boundary Composition; defined as: L s =λ2L Evi +λ3L Dice +λ4L Boundary Among them, λ2, λ3 and λ4 are the weights for controlling the proportion of evidence loss, Dice loss and Boundary loss; Loss of evidence Evi The definition is as follows: L Evi =L θ +βL KL Where β is the annealing coefficient, set to Among them, t is the current iteration number, t max is the total number of iterations; Use evidence-based cross entropy loss L θ Let pixel prediction value p i Close to label y i , p i The density of a i Parameterized Dirichlet distribution; L θ Defined as: where ψ(·) is the digamma function; By optimizing L θ , generate evidence of positive samples of different categories; The Kullback-Leibler divergence is added to the loss function to penalize the divergence with negative samples, defined as: where KL refers to the Kullback-Leibler divergence, Γ(·) is the gamma function, and D(p i ,1) is a uniform Dirichlet distribution, Combined with Dice loss training, by optimizing L Evi , the model generates evidence for positive samples and reduces the evidence for negative samples, using L Dice Constrain the relationship between different pixels; Dice loss is as follows: Where N is the total number of pixels, y i Represents the true label value of the pixel, p i Represents the pixel prediction value; At the same time, in order to make the model pay more attention to the segmentation effect of the boundary of the target area, the Boundary loss based on the boundary segmentation quality metric is used for constraint; specifically, given two segmentation masks G and P, the Boundary loss first calculates the original pixel set within the distance d from each mask boundary, and then calculates the intersection and union ratio of the two sets, which is defined as follows: Among them, G d is the set of pixels within the boundary d of the true label, P d is the set of pixels within the prediction area boundary d, y i With p i is the category label and predicted value of the pixel; Guided by the estimated uncertainty, the output of the teacher model is evaluated to filter out relatively unreliable predictions and only select reliable predictions as the learning targets of the student model; the uncertainty-aware unsupervised consistency loss L c Designed as the mean square error of the teacher model and the student model predictions, it is only used for reliable predictions that have been filtered through uncertainty information and is defined as follows: Where N is the total number of pixels, f i ′ and f i are the predictions of the teacher model and the student model for the i-th pixel, u i is the estimated uncertainty of the i-th pixel, H is the threshold for selecting credible targets, is the indicator function.

Images