A medical image segmentation model learning system based on voxel uncertainty

Abstract

This paper discloses a voxel-based uncertainty-based medical image segmentation model learning system, comprising five functional modules: an image acquisition module, a region of interest delineation module, an image preprocessing module, an uncertainty-guided image segmentation model training module, and a medical image segmentation model testing module. This system proposes an end-to-end voxel-based uncertainty-guided medical image segmentation model learning method, generating accurate segmentation results and reliable uncertainty estimates without excessively increasing computational burden and complexity, thereby simultaneously improving the performance, robustness, and interpretability of medical image segmentation models.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a medical image segmentation model learning system based on voxel uncertainty. Background Art

[0002] Accurate and robust medical image segmentation algorithms are powerful tools for AI-assisted clinical diagnosis. Since the advent of deep learning, the convolutional neural network-based U-Net and its various variants have become mainstream medical image segmentation models and are widely used for tumor analysis, lesion detection, and organ-at-risk contouring. However, the semantic segmentation of organ structures and lesions in medical images is inherently ambiguous. For example, segmentation boundaries may not be clearly visible, or annotations may differ between experts. However, most current state-of-the-art methods do not account for this ambiguity, instead learning a one-to-one mapping from images to segmentation masks. Even if these models achieve good segmentation performance, they are prone to unreliable predictions and are questioned by clinicians. Clinicians often require not only accurate segmentation results but also reliability and credibility. Furthermore, due to the frequent presence of out-of-distribution and noisy data in clinical settings, the performance of these methods can significantly degrade when deployed in real-world systems.

[0003] Existing uncertainty estimation methods have limited clinical application in medical image segmentation and can be categorized as Bayesian, model ensemble, and test-time augmentation. Bayesian methods explicitly model uncertainty by representing the parameters of medical image segmentation models as random variables, but the training process is difficult to optimize and requires more data. Model ensemble methods distribute the same output across different medical image segmentation models, but require simultaneous maintenance of all networks, resulting in high computational costs. Test-time augmentation methods augment the test set with different perturbations to evaluate the performance and uncertainty of the medical image segmentation model. This method requires strict control over the magnitude of the perturbation; excessively large or small perturbations can result in out-of-distribution data. These methods also suffer from poor complexity and efficiency, and struggle to balance model accuracy and reliability. Currently, research on voxel ambiguity estimation in medical image semantic segmentation is relatively scarce. Uncertainty estimation methods for medical image segmentation models with low computational complexity and high efficiency are increasingly favored. Using uncertainty to guide model learning will lead to more robust development and more reliable segmentation results in clinical applications.

[0004] In general, the uncertainty estimation techniques currently used in clinical practice perform well in specific medical image segmentation tasks, but they still have the following shortcomings:

[0005] 1. The Bayesian-based method introduces Dropout technology in the testing phase to estimate the uncertainty of the medical image segmentation model. Although it reduces the computational burden, it produces inconsistent outputs, making the model difficult to optimize and requiring a large amount of data.

[0006] 2. The model integration-based method uses multiple deep models to integrate predictions to model the uncertainty of medical image segmentation models. It requires maintaining multiple different networks at the same time, resulting in huge computational burden and complexity.

[0007] 3. Test-time augmentation-based methods use a test set to perform data augmentation of varying magnitudes to evaluate medical image segmentation models and model uncertainty. However, the magnitude of the augmentation must be strictly controlled, which complicates uncertainty assessment.

[0008] 4. None of the above methods can estimate uncertainty through a single forward propagation of the medical image segmentation model. They have the disadvantages of low efficiency and high complexity, and it is difficult to balance the uncertainty and accuracy of the medical image segmentation model.

[0009] In summary, in medical image segmentation models, how to reduce the computational complexity of uncertainty assessment and improve efficiency, guide model learning in a voxel-wise manner, balance the accuracy and reliability of the model, and provide clinicians with accurate, reliable and robust segmentation results are key issues that need to be solved urgently. Summary of the Invention

[0010] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a medical image segmentation model learning system based on voxel uncertainty. The uncertainty is modeled according to subjective logic theory. The feature class probabilities at different scales of the decoder are parameterized as Dirichlet distributions and the Dirichlet intensity is further calculated to quantify the uncertainty of the model. At the same time, an uncertainty adaptive learner is introduced to fine-tune the segmentation results at the voxel level according to the threshold to guide the model to learn more discriminatively.

[0011] To achieve the above objectives, the present invention provides a technical solution: a medical image segmentation model learning system based on voxel uncertainty, comprising:

[0012] The image acquisition module is used to collect medical image data sets, including CT image data of the same organ part in multiple samples and multi-sequence MRI image data acquired at the same time; the medical image data sets are randomly divided into training set, validation set and test set;

[0013] The region of interest delineation module is used to delineate the region of interest in the acquired CT image data and multi-sequence MRI image data as the true value label;

[0014] Image preprocessing module, used to preprocess the collected CT image data, multi-sequence MRI image data and the delineated true value labels;

[0015] The uncertainty guided image segmentation model training module is used to train the medical image segmentation model, which includes a feature extractor S, an uncertainty fusion device UF, an uncertainty measurer QU and an uncertainty adaptive learner USAT, and is used for iterative training on the training set; the Dirichlet distribution value obtained by the uncertainty measurer QU is used to train the medical image segmentation model. Input into the uncertainty adaptive learner USAT to obtain the modified Dirichlet distribution value , determined by the following formula:

[0016] ;

[0017] Combined with the modified Dirichlet distribution value , using subjective logic theory to model the revised uncertainty map And the segmentation result map , determined by the following formula:

[0018] ;

[0019] The uncertainty adaptive learner USAT combines the idea of ​​human course learning and adaptively adjusts the threshold based on the learning state of the model to optimize the segmentation result; the uncertainty adaptive learner USAT includes an adaptive global threshold and adaptive local threshold ;

[0020] The adaptive local threshold It is based on the difference between categories and is set by the learning difficulty of the category. It is adjusted in real time as the model learns. The category accuracy The low threshold is reduced, which is beneficial for the accuracy For high categories, keep a high threshold to ensure the final accuracy and combine the validation set segmentation results. ,by Represents the total number of validation sets, Representative The number of iterations, Representative The adaptive global threshold at the iteration Adaptive local thresholding of categories Determined by the following formula:

[0021] ;

[0022] The adaptive global threshold Related to the uncertainty of the model on the training data to reflect the overall learning state; combined with momentum decay Steadily increase the adaptive global threshold during training , to ensure high segmentation accuracy in the later stages of training, Represents the total number of training set samples, the adaptive global threshold Determined by the following formula:

[0023] ;

[0024] Dirichlet distribution value obtained by combining uncertainty measure QU and adaptive global threshold Calculate the value of the modified Dirichlet distribution , determined by the following formula:

[0025] ;

[0026] The corrected uncertainty map obtained by combining the uncertainty adaptive learner USAT Calculating uncertainty loss function ;

[0027] The modified Dirichlet distribution value obtained by combining the uncertainty adaptive learner USAT , No. The corresponding individual pixel Category segmentation result diagram , calculate the first The corresponding individual pixel Modified Dirichlet distribution post-processing value of the category ;

[0028] Combined with true value labels Calculate the divergence loss function ;

[0029] The medical image segmentation model testing module is used to input all samples in the test set into the medical image segmentation models of each iterative training round obtained in the uncertainty-guided image segmentation model training module to generate corresponding segmentation results; and combine the true value labels obtained by the region of interest delineation module to evaluate the segmentation quality of the region of interest in the segmentation results, and finally select the model with the best segmentation result quality among the models obtained in each round of iterative training as the optimal medical image segmentation model.

[0030] Furthermore, the region of interest delineation module determines and delineates the region of interest of each sample according to the tissue structure characteristics in the reference image, and uses it as the region of focus of the segmentation model.

[0031] Furthermore, the image preprocessing module pre-processes the image according to the characteristics of the medical image segmentation model, thereby improving the performance and generalization ability of the medical image segmentation model;

[0032] For the medical image segmentation model, the image preprocessing module adopts cropping, format conversion, normalization and data enhancement. The operations are as follows: each slice of the image is resampled to an isotropic voxel spacing of 1.0mm, and the value range is normalized from -1000 to 1000 Hounsfield units to 0 to 1 Hounsfield unit, and then randomly divided into training set, validation set and test set. Finally, the training set is randomly rotated by 90°, 180° and 270°, randomly flipped in the axial, sagittal and coronal positions, and randomly scaled. Data enhancement strategy.

[0033] Furthermore, the uncertainty guides the image segmentation model training module to divide the training set obtained in the image preprocessing module into Small batches are collected and the medical image segmentation model is trained batch by batch. During the training process, the gradient is returned and the model parameters are adjusted according to the loss function value, so that it gradually converges to the optimal value. At the same time, the validation set is used for verification calculation to assist in adjusting the model parameters and prevent the model from overfitting. Finally, the trained medical image segmentation model is obtained. Represents the training set of the current batch, corresponding to the true value label of the region of interest. The medical image segmentation model training process includes the following steps:

[0034] 1) Input the feature extractor S to get the feature map output by each layer of decoder; use represents the feature map, Indicates the Hierarchical feature map, feature map Determined by the following formula:

[0035] ;

[0036] 2) All feature maps obtained in step 1) Input into the uncertainty fuser UF to obtain the fusion graph , determined by the following formula:

[0037] ;

[0038] The uncertainty fuser UF aims to enhance the domain spatial response and summarize the nonlinear information across channels at the voxel level, so as to emphasize the semantic difference between the segmentation target and the background; Perform convolution with a kernel size of 1×1×1 , perform trilinear interpolation on all low-resolution feature maps after convolution , and finally with the first level feature map Through the Concatenate operation, feature superposition is performed on the channel dimension to obtain a fusion graph , determined by the following formula:

[0039] ;

[0040] 3) The fusion image obtained in step 2) Input into the uncertainty metric QU and obtain the uncertainty graph And the segmentation result map , determined by the following formula:

[0041] ;

[0042] The uncertainty measure QU is based on the Dempster-Shafer idea and is a generalization of Bayesian theory to subjective logic. By introducing a Softplus layer after the uncertainty fuser UF, the fusion graph is input. , to obtain non-negative evidence voxels , Indicates the Voxel value, Represents the total number of voxels, Category Evidence voxels Determined by the following formula:

[0043] ;

[0044] No. Category Evidence voxels Parameterize the Dirichlet distribution through subjective logic to obtain the Category Dirichlet distribution value , determined by the following formula:

[0045] ;

[0046] exist The Dirichlet distribution intensity can be obtained by accumulating the Dirichlet distribution value on the category channel of the segmentation category , determined by the following formula:

[0047] ;

[0048] Combined with Dirichlet distribution value and Dirichlet distribution intensity , using subjective logic theory to model and calculate uncertainty maps And the segmentation result map , determined by the following formula

[0049] ;

[0050] This means that Category The more evidence a voxel obtains, the higher the prediction confidence value, the smaller the uncertainty, and the more reliable the prediction result.

[0051] 4) Real value labels obtained based on the region of interest delineation module Computing uncertainty label maps , determined by the following formula:

[0052] ;

[0053] Where, Representative The corresponding individual pixel The true value label of the category, Representative The corresponding individual pixel Uncertainty label map for the categories;

[0054] Uncertainty loss function Determined by the following formula:

[0055] ;

[0056] Where, Representative The corresponding individual pixel Corrected uncertainty maps for categories;

[0057] No. The corresponding individual pixel Modified Dirichlet distribution post-processing value of the category , determined by the following formula:

[0058] ;

[0059] Divergence loss function Determined by the following formula:

[0060] ;

[0061] Where, is the gamma function, is the digamma function, is the post-processed value of the modified Dirichlet distribution;

[0062] Combined with the fusion image obtained in step 2) and the true value label Calculate the cross entropy-Dice coefficient combined loss function , determined by the following formula:

[0063] ;

[0064] Where, Representative The corresponding individual pixel Fusion map of categories;

[0065] The total loss function Determined by the following formula:

[0066] ;

[0067] According to the gradient of the loss function, the back propagation algorithm is used to update the model parameters. During the back propagation process, the medical image segmentation model automatically calculates the gradient of the loss function for each parameter;

[0068] 5) According to the Adam optimization algorithm, the calculated gradient is used to update the model parameters. Each parameter update will make the model loss function Reduce, every preset number of iterations, use the validation set to test the medical image segmentation model and calculate the accuracy index of the model. If the test results of the model do not meet the requirements, it is necessary to adjust the model structure, loss function and hyperparameters and retrain the model;

[0069] 6) Repeat steps 2)-5) until the model's loss function becomes stable or reaches the preset number of iterations.

[0070] Furthermore, the medical image segmentation model testing module tests the medical image segmentation model obtained by the uncertainty-guided image segmentation model training module, inputs the test set into the medical image segmentation model obtained by each iterative training round, obtains the segmentation results, and calculates the accuracy index and the reliability index of the uncertainty estimation according to the segmentation results;

[0071] The accuracy index is calculated by measuring the difference between the model segmentation results and the true value labels. It is divided into region-based calculation methods and boundary-based calculation methods. Among them, the region-based metric compares the similarity between the segmentation results and the true value labels, and the indicator used is the Dice coefficient.

[0072] The Dice coefficient is an indicator that compares the degree of overlap between the segmentation result and the true value label. It is calculated as follows: the segmentation result and the true value label are converted into binary images respectively, and then the two are multiplied to obtain the intersection. The number of voxels in the intersection is divided by the sum of the total number of voxels in the two binary images. The result of the division is the Dice coefficient. The Dice coefficient ranges from 0 to 1. The closer the value is to 1, the better the quality of the segmentation result.

[0073] The Dice coefficient is determined by the following expression:

[0074] ;

[0075] Where, Representative The true value label under each voxel, Representative Predicted value under each voxel;

[0076] The boundary-based metric compares the boundary differences between the segmentation results and the true value labels. The indicator used is the average symmetric surface distance, which is calculated as follows: the cumulative value of the Euclidean distance from each voxel point in the segmentation result to the set of midpoints in the true value label and the cumulative value of the Euclidean distance from each voxel point in the true value label to the set of midpoints in the segmentation result are calculated respectively. Then, the sum of the two is taken and divided by the sum of the midpoint set of the segmentation result and the midpoint set of the true value label as the average symmetric surface distance. The smaller the average symmetric surface distance value, the smaller the boundary difference between the segmentation result and the true value label.

[0077] and Represent the true value label and the point set in the segmentation result respectively, and Represent the points in the point set, represents the average symmetric surface distance, which is determined by the following expression:

[0078] ;

[0079] The reliability index of medical image segmentation model can be obtained by direct or indirect calculation;

[0080] The direct method is to obtain the confidence index. The confidence index is used to measure the reliability of the prediction results of the medical image segmentation model, that is, the degree of certainty of the model's classification results for each voxel point. The calculation method is to take the maximum value of the predicted logical value of the model output layer for each voxel, specifically the value after the Sigmoid or Softmax activation function, and then calculate the average value of all pixels in each channel of each test sample as the confidence value of each organ, and use represents the confidence value, represents the total number of samples in the test set, Indicates the The probability value of the sample prediction and the confidence value are determined by the following expression:

[0081] ;

[0082] The indirect method is calculated by combining the accuracy index and using the prediction calibration error as the indicator to measure the reliability of uncertainty estimation in the medical image segmentation model. The calculation method is the absolute difference between the statistical accuracy index and the confidence level. The absolute difference is averaged to obtain the prediction calibration error.

[0083] use represents the prediction calibration error, Indicates the The accuracy index of the samples, Indicates the The confidence index of the sample is given by the following expression:

[0084] ;

[0085] The model with the best segmentation result quality among the models obtained in each round of iterative training is selected as the optimal medical image segmentation model, which means that for the models obtained in each round of iterative training, the model with the largest regional overlap, the smallest boundary difference and the most reliable uncertainty estimation is selected as the optimal image segmentation model.

[0086] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0087] 1. The system of this invention uses subjective logic theory to predict medical image segmentation models, eliminating the need for excessive model modification and increased computational burden. It quantifies voxel-level uncertainty through the lens of evidence-based deep learning to address model reliability issues in clinical deployment. It also utilizes an uncertainty fuser and uncertainty adaptor to enhance model learning at a more detailed level, thereby improving the performance of medical image segmentation models in clinical applications.

[0088] 2. Compared with the Bayesian-based method, the system of the present invention has consistent output, which makes the medical image segmentation model easy to optimize and reduces the dependence on the amount of data.

[0089] 3. Compared with the method based on model integration, the system of the present invention is always composed of one main model, and there is no need to maintain multiple models at the same time, which reduces the computing cost.

[0090] 4. Compared with the method based on test time enhancement, the system of the present invention does not need to worry about the problem of disturbing data amplitude, which reduces the complexity of calculation.

[0091] 5. Compared with the existing uncertainty assessment methods of medical image segmentation models, the system of the present invention is based on an end-to-end deep learning method and does not require complex pre-processing and post-processing of the data. The uncertainty fuser, uncertainty measurer and uncertainty adaptive learner in the system can be efficiently used to guide the learning of different categories of medical image segmentation models. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 It is an architectural diagram of the system of the present invention.

[0093] Figure 2 This is a schematic diagram of the uncertainty-guided image segmentation model training module.

[0094] Figure 3 This is the structural diagram of the uncertainty fuser UF.

[0095] Figure 4 It is the structural diagram of the uncertainty measurer QU.

[0096] Figure 5 This is the structural diagram of the uncertainty adaptive learner USAT. DETAILED DESCRIPTION

[0097] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0098] like Figure 1 As shown, this embodiment discloses a medical image segmentation model learning system based on voxel uncertainty, including: an image acquisition module, a region of interest delineation module, an image preprocessing module, an uncertainty-guided image segmentation model training module and a medical image segmentation model testing module.

[0099] Specifically, the image acquisition module is used to collect an image dataset, which should include multiple samples of organ images obtained simultaneously at the same location. In this embodiment, the sample sites selected are all abdominal organs, including the stomach, liver, kidneys, gallbladder, left and right adrenal glands, spleen, esophagus, pancreas, and portal vein. Three public datasets of different sizes, BTCV, LiTs2017, and Flare2022, were used for implementation. Patients were randomly divided into training sets, validation sets, and test sets with a ratio of 8:1:1.

[0100] Specifically, the region of interest delineation module delineates the region of interest in the reference image, and the radiologist delineates various regions of interest on the human abdomen in the dataset, and uses the delineation results as the true value labels in this embodiment;

[0101] Specifically, the image preprocessing module pre-processes the image according to the characteristics of the medical image segmentation model, so that the performance and generalization ability of the medical image segmentation model can be improved, thereby better solving practical problems;

[0102] For the medical image segmentation model, the image preprocessing module performs the following operations: resampling each image slice to an isotropic voxel spacing of 1.0 mm, and normalizing the value range from -1000 (Hounsfield unit) to 1000 (Hounsfield unit) to 0 (Hounsfield unit) to 1 (Hounsfield unit), and then randomly dividing the training set, validation set and test set. Finally, the training set is randomly rotated by 90°, 180° and 270°, randomly flipped in the axial, sagittal and coronal positions, and randomly scaled. Data augmentation strategies.

[0103] Specifically, the uncertainty guidance image segmentation model training module divides the training set obtained in the image preprocessing module into The medical image segmentation model is trained batch by batch, wherein in this embodiment, the minibatch size is set to 1, and the specific value can be adjusted according to the size of the GPU available video memory; in the training phase, the minibatch size is set to 1. Represents the training set of the current batch, corresponding to the true value label of the region of interest. Indicates; Figure 2 As shown in FIG, the medical image segmentation model includes a feature extractor S, an uncertainty fuser UF, an uncertainty measurer QU, and an uncertainty adaptive learner USAT. The models used by the feature extractor S in this embodiment are Attention U-Net and nnU-Net. The training process includes the following steps:

[0104] 1) If Figure 2 As shown, the training set of the current batch Input the feature extractor S to get the feature map output by each layer of decoder; use represents the feature map, Indicates the Hierarchical feature map, feature map Determined by the following formula

[0105] ;

[0106] 2) All feature maps obtained in step 1) Input into the uncertainty fuser UF to obtain the fusion graph , determined by the following formula:

[0107] ;

[0108] like Figure 3As shown, the uncertainty fusion UF aims to enhance the domain spatial response and summarize the nonlinear information across channels at the voxel level, so as to emphasize the semantic difference between the segmentation target and the background; Perform convolution with a kernel size of 1×1×1 , perform trilinear interpolation on all low-resolution feature maps after convolution , and finally with the first level feature map Through the Concatenate operation, feature superposition is performed on the channel dimension to obtain a fusion graph , determined by the following formula:

[0109] ;

[0110] 3) The fusion image obtained in step 2) Input into the uncertainty metric QU and obtain the uncertainty graph And the segmentation result map , determined by the following formula:

[0111] ;

[0112] like Figure 4 As shown in the figure, the uncertainty measure QU is based on the Dempster-Shafer idea, which is a generalization of Bayesian theory to subjective logic. By introducing a Softplus layer after the uncertainty fusion UF, the fusion graph is input. , to obtain non-negative evidence voxels , Indicates the Voxel value, Represents the total number of voxels, Category Evidence voxels Determined by the following formula:

[0113] ;

[0114] No. Category Evidence voxels Parameterize the Dirichlet distribution through subjective logic to obtain the Category Dirichlet distribution value , can be determined by the following formula:

[0115] ;

[0116] exist The Dirichlet distribution intensity can be obtained by accumulating the Dirichlet distribution value on the category channel of the segmentation category , can be determined by the following formula:

[0117] ;

[0118] Combined with Dirichlet distribution value and Dirichlet distribution intensity , using subjective logic theory to model and calculate uncertainty maps And the segmentation result map , can be determined by the following formula

[0119] ;

[0120] This means that Category The more evidence a voxel obtains, the higher the prediction confidence value, the smaller the uncertainty, and the more reliable the prediction result;

[0121] 4) The Dirichlet distribution value obtained in step 3) Input into the uncertainty adaptive learner USAT to obtain the modified Dirichlet distribution value , determined by the following formula:

[0122] ;

[0123] Combined with the modified Dirichlet distribution value , using subjective logic theory to model the revised uncertainty map And the segmentation result map , can be determined by the following formula:

[0124] ;

[0125] like Figure 5 As shown, the uncertainty adaptive learner USAT combines the idea of ​​human course learning and adaptively adjusts the threshold based on the learning state of the model to reduce mis-segmentation and optimize the segmentation results; the uncertainty adaptive learner USAT includes an adaptive global threshold and an adaptive local threshold; the adaptive local threshold It is based on the difference between categories and is set by the learning difficulty of the category. It is adjusted in real time as the model learns. The category accuracy The low threshold is reduced, which is beneficial for the accuracy For high categories, keep a high threshold to ensure the final accuracy and combine the validation set segmentation results. ,by Represents the total number of validation sets, Representative The number of iterations, Representative The adaptive global threshold at the iteration Adaptive local thresholding of categories It can be determined by the following formula:

[0126] ;

[0127] The adaptive global threshold Related to the uncertainty of the model on the training data to reflect the overall learning state; combined with momentum decay Steadily increase the adaptive global threshold during training , to ensure high segmentation accuracy in the later stages of training, Represents the total number of training set samples, adaptive global threshold It can be determined by the following formula:

[0128] ;

[0129] Combined with the Dirichlet distribution value obtained in step 3) and adaptive global threshold Calculate the value of the modified Dirichlet distribution , can be determined by the following formula:

[0130] ;

[0131] 5) If Figure 2 As shown, the true value label obtained based on the region of interest delineation module Computing uncertainty label maps , can be determined by the following formula:

[0132] ;

[0133] Where, Representative The corresponding individual pixel The true value label of the category, Representative The corresponding individual pixel Uncertainty label map for the categories;

[0134] Combined with the corrected uncertainty map obtained in step 4) Calculating uncertainty loss function , can be determined by the following formula:

[0135] ;

[0136] Where, Representative The corresponding individual pixel Corrected uncertainty maps for categories;

[0137] Combined with the modified Dirichlet distribution value obtained in step 4) , No. The corresponding individual pixel Category segmentation result diagram , can be further calculated to obtain The corresponding individual pixel Modified Dirichlet distribution post-processing value of the category , can be determined by the following formula:

[0138] ;

[0139] Further combined with the true value label Calculate the divergence loss function , determined by the following formula:

[0140] ;

[0141] Where, is the gamma function, is the digamma function, is the post-processed value of the modified Dirichlet distribution;

[0142] Combined with the fusion image obtained in step 2) and the true value label Calculate the cross entropy-Dice coefficient combined loss function , can be determined by the following formula:

[0143] ;

[0144] Where, Representative The corresponding individual pixel Fusion map of categories;

[0145] The total loss function It can be determined by the following formula:

[0146] ;

[0147] According to the gradient of the loss function, the back propagation algorithm is used to update the model parameters. During the back propagation process, the medical image segmentation model automatically calculates the gradient of the loss function for each parameter;

[0148] 6) According to the Adam optimization algorithm, the gradient calculated in step 4) is used to update the model parameters. Each parameter update will make the model loss function Reduce, every preset number of iterations, use the validation set to test the medical image segmentation model and calculate the accuracy index of the model. If the test results of the model do not meet the requirements, it is necessary to adjust the model structure, loss function and hyperparameters and retrain the model;

[0149] 7) Repeat steps 2)-6) until the model's loss function Become stable or reach the preset number of iterations;

[0150] Specifically, the medical image segmentation model testing module tests the medical image segmentation model obtained by the uncertainty-guided image segmentation model training module, inputs the test set into the medical image segmentation model obtained by each iterative training round, obtains the segmentation results, and calculates the accuracy index and the reliability index of the uncertainty estimation according to the segmentation results;

[0151] The accuracy index calculation is performed by measuring the difference between the model segmentation results and the true value labels. It is divided into region-based calculation methods and boundary-based calculation methods. Among them, the region-based metric compares the similarity between the segmentation results and the true value labels. In this embodiment, the metric used is the Dice coefficient.

[0152] The Dice coefficient is an indicator that compares the degree of overlap between the segmentation result and the true value label. It is calculated as follows: the segmentation result and the true value label are converted into binary images respectively, and then the two are multiplied to obtain the intersection. The number of voxels in the intersection is divided by the sum of the total number of voxels in the two binary images. The result of the division is the Dice coefficient. The Dice coefficient ranges from 0 to 1. The closer the value is to 1, the better the quality of the segmentation result.

[0153] The Dice coefficient is determined by the following expression:

[0154] ;

[0155] The boundary-based metric compares the boundary differences between the segmentation result and the true value label. In this embodiment, the metric used is the average symmetric surface distance, which is calculated as follows: the cumulative Euclidean distance from each voxel point in the segmentation result to the set of midpoints in the true value label and the cumulative Euclidean distance from each voxel point in the true value label to the set of midpoints in the segmentation result are calculated. The sum of the two is then divided by the sum of the midpoints in the segmentation result and the set of midpoints in the true value label to obtain the average symmetric surface distance. The smaller the average symmetric surface distance value, the smaller the boundary difference between the segmentation result and the true value label.

[0156] and Represent the true value label and the point set in the segmentation result respectively, and Represent the points in the point set, represents the average symmetric surface distance, which is determined by the following expression:

[0157] ;

[0158] The reliability index of medical image segmentation model can be obtained by direct or indirect calculation;

[0159] The direct method is to obtain the confidence index. Confidence is a measure of the reliability of the prediction results of the medical image segmentation model, that is, the degree of certainty of the model's classification results for each voxel point. It is calculated by taking the maximum value of the predicted logits for each voxel, specifically the value after the Sigmoid or Softmax activation function, and then calculating the average value of all pixels in each channel of each test sample as the confidence value of each organ. represents the confidence value, represents the total number of samples in the test set, Indicates the The probability value of the sample prediction and the confidence value are determined by the following expression:

[0160] ;

[0161] The indirect method is calculated by combining the accuracy index. In this embodiment, the prediction calibration error is mainly used to balance the reliability of the uncertainty estimation in the medical image segmentation model. The calculation method is the absolute difference between the statistical accuracy index and the confidence level. The absolute differences are averaged to obtain the prediction calibration error.

[0162] use represents the prediction calibration error, Indicates the The accuracy index of the samples, Indicates the The confidence index of the sample is given by the following expression:

[0163] ;

[0164] In this example, we distribute the normal state and Gaussian noise Under the conditions of , different medical image segmentation models in the two public datasets LiTs2017 and Flare2022 were tested and the Dice coefficient, average symmetric surface distance and prediction calibration error were calculated as follows:

[0165] Table 1 Segmentation performance evaluation results in the LiTS2017 dataset, N and OOD represent normal state and Gaussian noise respectively The conditions, bold and bold black indicate the optimal and suboptimal results respectively

[0166]

[0167] Table 2 Segmentation performance evaluation results in the Flare2022 dataset, N and OOD represent normal state and Gaussian noise respectively The conditions, bold and bold black indicate the optimal and suboptimal results respectively

[0168]

[0169] As shown in Tables 1 and 2, under normal conditions, this embodiment significantly improves both the Dice coefficient and ASSD on the LiTS2017 and Flare2022 datasets compared to the baseline model. Specifically, on the LiTS2017 dataset, the Attention U-Net and nnU-Net powered by this embodiment improve the Dice coefficient by 2.4% and 3.7%, respectively, and reduce the ASSD by 1.39 and 2.34, respectively. On the Flare2022 dataset, the Attention U-Net and nnU-Net powered by this embodiment improve the Dice coefficient by 3.6% and 4.9%, respectively, and reduce the ASSD by 0.97 and 1.22, respectively. Furthermore, this embodiment achieves optimal performance in the ECE metric on both datasets, demonstrating the effectiveness of its uncertainty estimation scheme. Compared to other uncertainty estimation methods: Dropout U-Net, U-Net Ensemble, Probabilistic U-Net, and TBraTS, this embodiment achieves superior performance on both datasets. Specifically, on the LiTS2017 dataset, the nnU-Net powered by this embodiment achieved a Dice coefficient 1.7% higher than TBrats, 0.09 lower than Probabilistic U-Net in ASSD, and 0.003 lower than both TBrats and Probabilistic U-Net in ECE. On the Flare2022 dataset, the Attention U-Net and nnU-Net powered by this embodiment achieved top-two results in all three metrics. These results demonstrate the superiority of this embodiment.

[0170] In order to further verify the reliability of this embodiment, Gaussian noise is added to the image data. The same metrics were tested under the same conditions, as shown in Tables 1 and 2. It is observed that the segmentation accuracy of most models decreases significantly after the addition of Gaussian noise, especially for the uncertainty estimation methods Dropout U-Net, U-Net Ensemble, and Probabilistic U-Net. Despite some improvements made by TBraTS, it still struggles to maintain high performance under high noise conditions. However, this embodiment demonstrates impressive results. On the LiTS2017 dataset, the fluctuations of the Dice coefficient, ASSD, and ECE for the Attention U-Net and nnU-Net enhanced by this embodiment are only 11.2% and 5.8%, 3.1 and 2.2, and 0.034 and 0.022, respectively. These are significantly lower than those of other uncertainty estimation methods, such as 26.7% (Dropout U-Net), 7.53% (Probabilistic U-Net), and 0.069% (Probabilistic U-Net). In the Flare2022 dataset, the Attention U-Net and nnU-Net implemented in this embodiment achieved floating values ​​of only 8.5% and 5.1% in Dice, 1.79 and 0.53, and 0.053 and 0.032 in ASSD, respectively, while the Dropout U-Net achieved floating values ​​as high as 35.5%, 8.97, and 0.239. These results demonstrate that this embodiment has the best noise resistance and can maintain optimal performance even in high-noise conditions.

[0171] In the uncertainty-guided image segmentation model training module of this embodiment, the feature extractor S, uncertainty fuser UF, uncertainty metric QU, and uncertainty adaptive learner USAT all play important roles. The effectiveness of each is experimentally tested on the BTCV dataset, and the results are as follows:

[0172] Table 3 Ablation experiment results on the BTCV dataset

[0173]

[0174] As shown in Table 3 above, the proposed feature extractor S, uncertainty fuser UF, uncertainty metric QU, and uncertainty adaptive learner USAT play an important role in guiding image segmentation model training. Using nnU-Net as the feature extractor, the addition of the uncertainty metric QU alone resulted in a 2% increase in the Dice coefficient and a 1.61% reduction in the ASSD. This demonstrates that segmentation performance is significantly improved when voxel-level uncertainty is introduced to guide image segmentation model training. Furthermore, the introduction of the uncertainty fuser UF and the uncertainty adaptive learner USAT further enhances model accuracy. Their introduction increases the Dice coefficient by 3.4% and 3.2%, respectively, and reduces the ASSD by 2.88 and 2.35, respectively. Ultimately, the synergistic combination of the feature extractor S, uncertainty fuser UF, uncertainty metric QU, and uncertainty adaptive learner USAT results in a 4.5% increase in the Dice coefficient and a 3.11% reduction in the ASSD, demonstrating a significant enhancement in the learning capability of the image segmentation model. Furthermore, the reliability of this embodiment is also reflected in the reduction in the ECE metric.

[0175] The model with the best segmentation result quality among the models obtained in each round of iterative training is selected as the optimal medical image segmentation model, which means that for the models obtained in each round of iterative training, the model with the largest regional overlap, the smallest boundary difference and the most reliable uncertainty estimation is selected as the optimal image segmentation model.

[0176] The above embodiments are preferred implementations of the present invention, but the implementations of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A medical image segmentation model learning system based on voxel uncertainty, characterized by: include: Image acquisition module, used to collect medical image data sets, including CT image data of the same organ part in multiple samples and multi-sequence MRI image data obtained at the same time; The medical imaging dataset is randomly divided into training set, validation set and test set; The region of interest delineation module is used to delineate the region of interest in the acquired CT image data and multi-sequence MRI image data as the true value label; Image preprocessing module, used to preprocess the collected CT image data, multi-sequence MRI image data and the outlined true value labels; The uncertainty guided image segmentation model training module is used to train the medical image segmentation model, which includes a feature extractor S, an uncertainty fusion device UF, an uncertainty measurer QU and an uncertainty adaptive learner USAT, and is used for iterative training on the training set; wherein the Dirichlet distribution value obtained by the uncertainty measurer QU is Input into the uncertainty adaptive learner USAT to obtain the modified Dirichlet distribution value , determined by the following formula: ； Combined with the modified Dirichlet distribution value , using subjective logic theory to model the revised uncertainty map And the segmentation result map , determined by the following formula: ； The uncertainty adaptive learner USAT combines the idea of ​​human course learning and adaptively adjusts the threshold based on the learning state of the model to optimize the segmentation result; the uncertainty adaptive learner USAT includes an adaptive global threshold and adaptive local threshold ; The adaptive local threshold It is based on the difference between categories and is set by the learning difficulty of the category. It is adjusted in real time as the model learns. The category accuracy The low threshold is reduced, which is beneficial for the accuracy For high categories, keep a high threshold to ensure the final accuracy and combine the validation set segmentation results. ,by Represents the total number of validation sets, Representative The number of iterations, Representative The adaptive global threshold at the iteration Adaptive local thresholding of categories Determined by the following formula: ； The adaptive global threshold Related to the uncertainty of the model on the training data to reflect the overall learning state; combined with momentum decay Steadily increase the adaptive global threshold during training , to ensure high segmentation accuracy in the later stages of training, Represents the total number of training set samples, the adaptive global threshold Determined by the following formula: ； Dirichlet distribution value obtained by combining uncertainty measure QU and adaptive global threshold Calculate the value of the modified Dirichlet distribution , determined by the following formula: ； The corrected uncertainty map obtained by combining the uncertainty adaptive learner USAT Calculate the uncertainty loss function with the uncertainty label map ; The modified Dirichlet distribution value obtained by combining the uncertainty adaptive learner USAT , No. The corresponding individual pixel Category segmentation result map , calculate the first The corresponding individual pixel Modified Dirichlet distribution post-processing value of the category ; Combined with the modified Dirichlet distribution post-processing value and the true value label Calculate the divergence loss function ; The medical image segmentation model testing module is used to select the model with the best segmentation result quality among the models obtained in each round of iterative training as the optimal medical image segmentation model.

2. The voxel-based uncertainty-based medical image segmentation model learning system according to claim 1, characterized in that: The region of interest delineation module determines and delineates the region of interest of each sample according to the tissue structure characteristics in the reference image, and uses it as the focus area of ​​the segmentation model.

3. The medical image segmentation model learning system based on voxel uncertainty according to claim 2, characterized in that: The image preprocessing module pre-processes the image according to the characteristics of the medical image segmentation model, thereby improving the performance and generalization ability of the medical image segmentation model; For the medical image segmentation model, the image preprocessing module adopts cropping, format conversion, normalization and data enhancement. The operations are as follows: each slice of the image is resampled to an isotropic voxel spacing of 1.0mm, and the value range is normalized from -1000 to 1000 Hounsfield units to 0 to 1 Hounsfield unit, and then randomly divided into training set, validation set and test set. Finally, the training set is randomly rotated by 90°, 180° and 270°, randomly flipped in the axial, sagittal and coronal positions, and randomly scaled. Data enhancement strategy.

4. The voxel-based uncertainty-based medical image segmentation model learning system according to claim 3, characterized in that: The uncertainty guides the image segmentation model training module to divide the training set obtained by the image preprocessing module into Small batches are collected and the medical image segmentation model is trained batch by batch. During the training process, the gradient is returned and the model parameters are adjusted according to the loss function value, so that it gradually converges to the optimal value. At the same time, the validation set is used for verification calculation to assist in adjusting the model parameters and prevent the model from overfitting. Finally, the trained medical image segmentation model is obtained. Represents the training set of the current batch, corresponding to the true value label of the region of interest. The medical image segmentation model training process includes the following steps: 1) Input the feature extractor S to obtain the feature map output by each layer of decoder; use represents the feature map, Indicates the Hierarchical feature map, feature map Determined by the following formula: ； 2) All feature maps obtained in step 1) Input into the uncertainty fuser UF to obtain the fusion graph , determined by the following formula: ； The uncertainty fuser UF aims to enhance the domain spatial response and summarize the nonlinear information across channels at the voxel level, thereby emphasizing the semantic difference between the segmented target and the background; Feature maps at different levels Perform convolution with a kernel size of 1×1×1 , perform trilinear interpolation on all low-resolution feature maps after convolution , and finally with the first level feature map Through the Concatenate operation, feature superposition is performed on the channel dimension to obtain a fusion graph , determined by the following formula: ； 3) The fusion image obtained in step 2) Input into the uncertainty metric QU and obtain the uncertainty graph And the segmentation result map , determined by the following formula: ； The uncertainty measure QU is based on the Dempster-Shafer idea and is a generalization of Bayesian theory to subjective logic; By introducing a Softplus layer after the uncertainty fuser UF, the fusion map is input , to obtain non-negative evidence voxels , Indicates the Voxel value, Represents the total number of voxels, Category Evidence voxels Determined by the following formula: ； No. Category Evidence voxels Parameterize the Dirichlet distribution through subjective logic to obtain the Category Dirichlet distribution value , determined by the following formula: ； exist The Dirichlet distribution intensity can be obtained by accumulating the Dirichlet distribution value on the category channel of the segmentation category , determined by the following formula: ； Combined with Dirichlet distribution value and Dirichlet distribution intensity , using subjective logic theory to model and calculate uncertainty maps And the segmentation result map , determined by the following formula ； This means that Category The more evidence a voxel obtains, the higher the prediction confidence value, the smaller the uncertainty, and the more reliable the prediction result. 4) The Dirichlet distribution value obtained in step 3) Input into the uncertainty adaptive learner USAT to obtain the modified Dirichlet distribution value ; 5) Real value labels obtained based on the region of interest delineation module Computing uncertainty label maps , determined by the following formula: ； Where, Representative The corresponding individual pixel The true value label of the category, Representative The corresponding individual pixel Uncertainty label map for the categories; Uncertainty loss function Determined by the following formula: ； Where, Representative The corresponding individual pixel Corrected uncertainty maps for categories; No. The corresponding individual pixel Modified Dirichlet distribution post-processing value of the category , determined by the following formula: ； Divergence loss function Determined by the following formula: ； Where, is the gamma function, is the digamma function, is the post-processed value of the modified Dirichlet distribution; Combined with the fusion image obtained in step 2) and the true value label Calculate the cross entropy-Dice coefficient combined loss function , determined by the following formula: ； Where, Representative The corresponding individual pixel Fusion map of categories; The total loss function Determined by the following formula: ； According to the gradient of the loss function, the back propagation algorithm is used to update the model parameters. During the back propagation process, the medical image segmentation model automatically calculates the gradient of the loss function for each parameter; 6) According to the Adam optimization algorithm, the calculated gradient is used to update the model parameters. Each parameter update will make the model loss function Reduce, every preset number of iterations, use the validation set to test the medical image segmentation model and calculate the accuracy index of the model. If the test results of the model do not meet the requirements, it is necessary to adjust the model structure, loss function and hyperparameters and retrain the model; 7) Repeat steps 2)-6) until the model's loss function becomes stable or reaches the preset number of iterations.

5. The voxel-based uncertainty-based medical image segmentation model learning system according to claim 4, characterized in that: The medical image segmentation model testing module tests the medical image segmentation model obtained by the uncertainty-guided image segmentation model training module, inputs the test set into the medical image segmentation model obtained by each iterative training round, obtains the segmentation results, and calculates the accuracy index and the reliability index of the uncertainty estimation according to the segmentation results; The accuracy index is calculated by measuring the difference between the model segmentation results and the true value labels. It is divided into region-based calculation methods and boundary-based calculation methods. Among them, the region-based metric compares the similarity between the segmentation results and the true value labels, and the indicator used is the Dice coefficient. The Dice coefficient is an indicator that compares the degree of overlap between the segmentation result and the true value label. It is calculated as follows: the segmentation result and the true value label are converted into binary images respectively, and then the two are multiplied to obtain the intersection. The number of voxels in the intersection is divided by the sum of the total number of voxels in the two binary images. The result of the division is the Dice coefficient. The Dice coefficient ranges from 0 to 1. The closer the value is to 1, the better the quality of the segmentation result. The Dice coefficient is determined by the following expression: ； Where, Representative The true value label under each voxel, Representative Predicted value under each voxel; The boundary-based metric compares the boundary differences between the segmentation results and the true value labels. The indicator used is the average symmetric surface distance, which is calculated as follows: the cumulative value of the Euclidean distance from each voxel point in the segmentation result to the set of midpoints in the true value label and the cumulative value of the Euclidean distance from each voxel point in the true value label to the set of midpoints in the segmentation result are calculated respectively. Then, the sum of the two is taken and divided by the sum of the midpoint set of the segmentation result and the midpoint set of the true value label as the average symmetric surface distance. The smaller the average symmetric surface distance value, the smaller the boundary difference between the segmentation result and the true value label. and Represent the true value label and the point set in the segmentation result respectively, and Represent the points in the point set, represents the average symmetric surface distance, which is determined by the following expression: ； The reliability index of medical image segmentation model can be obtained by direct or indirect calculation; The direct method is to obtain the confidence index. The confidence index is used to measure the reliability of the prediction results of the medical image segmentation model, that is, the degree of certainty of the model's classification results for each voxel point. The calculation method is to take the maximum value of the predicted logical value of the model output layer for each voxel, specifically the value after the Sigmoid or Softmax activation function, and then calculate the average value of all pixels in each channel of each test sample as the confidence value of each organ, and use represents the confidence value, represents the total number of samples in the test set, Indicates the The probability value of the sample prediction and the confidence value are determined by the following expression: ； The indirect method is calculated by combining the accuracy index and using the prediction calibration error as the indicator to measure the reliability of uncertainty estimation in the medical image segmentation model. The calculation method is the absolute difference between the statistical accuracy index and the confidence level. The absolute difference is averaged to obtain the prediction calibration error. use represents the prediction calibration error, Indicates the The accuracy index of the samples, Indicates the The confidence index of the sample is given by the following expression: ； The model with the best segmentation result quality among the models obtained in each round of iterative training is selected as the optimal medical image segmentation model, which means that for the models obtained in each round of iterative training, the model with the largest regional overlap, the smallest boundary difference and the most reliable uncertainty estimation is selected as the optimal image segmentation model.

Images