Medical image segmentation model learning system based on voxel uncertainty

Abstract

The invention discloses a medical image segmentation model learning system based on voxel uncertainty. The medical image segmentation model learning system comprises five functional modules: an image acquisition module, a region-of-interest sketching module, an image preprocessing module, an uncertainty guidance image segmentation model training module and a medical image segmentation model testing module. According to the system, an end-to-end voxel uncertainty guided medical image segmentation model learning method is provided, an accurate segmentation result and reliable uncertainty estimation are generated under the condition that calculation burden and complexity are not increased too much, and the performance, robustness and interpretability of a medical image segmentation model are improved at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a medical image segmentation model learning system based on voxel uncertainty. Background Art

[0002] Accurate and robust medical image segmentation algorithms are powerful tools for AI-assisted clinical diagnosis. Since the emergence of deep learning, U-Net and its various variants based on convolutional neural networks have become the mainstream medical image segmentation models and are widely used in tumor analysis, lesion detection, and contouring of organs at risk. However, the semantic segmentation of organ structures and lesions in medical images is inherently fuzzy. For example, the segmentation boundaries may not be clearly visible, or there may be differences in annotations between different experts. However, most current advanced methods do not consider this ambiguity, but instead learn a one-to-one mapping from images to segmentation masks. Even though these models have good segmentation performance, they are prone to unreliable predictions and are questioned by clinicians. Clinicians often need not only accurate segmentation results, but also the reliability and credibility of the segmentation results. At the same time, due to the frequent presence of out-of-distribution data and noisy data in the training set in clinical settings, the performance of these methods will drop significantly after being deployed in actual systems.

[0003] Existing uncertainty estimation methods are rarely used in clinical applications of medical image segmentation and can be divided into Bayesian-based, model ensemble-based, and test-time enhancement-based methods. Bayesian-based methods explicitly model uncertainty by expressing the parameters of the medical image segmentation model as random variables, but the training process is difficult to optimize and requires more data; model ensemble-based methods obtain distributed outputs from different medical image segmentation models for the same output, but all networks need to be maintained at the same time, which is computationally expensive; test-time enhancement-based methods enhance the test set data by using different perturbations to evaluate the performance and uncertainty of the medical image segmentation model. This method requires strict control of the amplitude of the perturbation, as too large or too small perturbations will result in data outside the distribution; the above methods perform poorly in terms of complexity and efficiency, and it is difficult to balance model accuracy and reliability. At present, research on voxel ambiguity estimation in medical image semantic segmentation is still relatively lacking. Medical image segmentation model uncertainty estimation methods with low computational complexity and high efficiency will be more popular. Using uncertainty to guide the learning of medical image segmentation models will enable them to achieve more robust development and more reliable segmentation results in clinical applications.

[0004] In general, the uncertainty estimation techniques commonly used in clinical practice perform well in specific medical image segmentation tasks, but still have the following shortcomings: 1. The Bayesian-based method introduces Dropout technology in the testing phase to estimate the uncertainty of the medical image segmentation model. Although it reduces the computational burden, it will produce inconsistent outputs, making the model difficult to optimize and requiring a large amount of data.

[0005] 2. The model integration-based method uses multiple deep models to integrate predictions to model the uncertainty of medical image segmentation models. It requires maintaining multiple different networks at the same time, resulting in huge computational burden and complexity.

[0006] 3. The test time enhancement method uses the test set to perform data enhancement of different magnitudes to evaluate the medical image segmentation model and model uncertainty. However, the magnitude of the enhancement needs to be strictly controlled, which increases the complexity of uncertainty assessment.

[0007] 4. None of the above methods can estimate uncertainty through a single forward propagation of the medical image segmentation model. They have the disadvantages of low efficiency and high complexity, and it is difficult to balance the uncertainty and accuracy of the medical image segmentation model.

[0008] In summary, in medical image segmentation models, how to reduce the computational complexity of uncertainty assessment and improve efficiency, guide model learning in a voxel-based uncertainty manner, balance the accuracy and reliability of the model, and provide clinicians with accurate, reliable and robust segmentation results are key issues that need to be solved urgently. Summary of the invention

[0009] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a medical image segmentation model learning system based on voxel uncertainty. The uncertainty is modeled according to the subjective logic theory, and the uncertainty of the model is quantified by parameterizing the feature class probabilities of different scales of the decoder into Dirichlet distribution and further calculating the Dirichlet intensity. At the same time, an uncertainty adaptive learner is introduced to fine-tune the segmentation results at the voxel level according to the threshold to guide the model to learn more discriminatively.

[0010] To achieve the above object, the technical solution provided by the present invention is: a medical image segmentation model learning system based on voxel uncertainty, comprising: An image acquisition module is used to acquire medical image data sets, including CT image data of the same organ part in multiple samples and multi-sequence MRI image data acquired at the same time; the medical image data sets are randomly divided into a training set, a validation set, and a test set; The region of interest delineation module is used to delineate the region of interest in the acquired CT image data and multi-sequence MRI image data as the true value label; The image preprocessing module is used to preprocess the collected CT image data, multi-sequence MRI image data and the outlined true value labels. It uses cropping, format conversion, normalization and data enhancement to make the training data more reasonable and diverse to improve the robustness, generalization performance and training effect of the model, thereby obtaining new data samples; The uncertainty-guided image segmentation model training module is used to train the medical image segmentation model, which includes a feature extractor S, an uncertainty fuser UF, an uncertainty measurer QU, and an uncertainty adaptive learner USAT, all of which are used for iterative training on the training set. During the training process, gradient backpropagation and model parameter adjustment are performed according to the loss function value, so that the model gradually converges to the optimal value. At the same time, the validation set is used for validation calculation to assist in adjusting the model parameters and prevent overfitting of the model training, so as to finally obtain a trained medical image segmentation model. The medical image segmentation model testing module is used to input all samples in the test set into the medical image segmentation models of each iterative training round obtained in the uncertainty-guided image segmentation model training module to generate corresponding segmentation results; and evaluate the segmentation quality of the region of interest in the segmentation results in combination with the true value labels obtained by the region of interest delineation module, and finally select the model with the best segmentation result quality among the models obtained in each round of iterative training as the optimal medical image segmentation model.

[0011] Furthermore, the region of interest delineation module determines and delineates the region of interest of each sample according to the tissue structure characteristics in the reference image, and uses it as the region that the segmentation model focuses on.

[0012] Furthermore, the image preprocessing module preprocesses the image according to the characteristics of the medical image segmentation model, so that the performance and generalization ability of the medical image segmentation model can be improved; For the medical image segmentation model, the image preprocessing module performs the following operations: resample each image slice to an isotropic voxel spacing of 1.0 mm, and normalize the value range from -1000 (Henrique unit) to 1000 (Henrique unit) to 0 (Henrique unit) to 1 (Henrique unit), and then randomly divide the training set, validation set and test set, and finally adopt a data enhancement strategy of random rotation of 90°, 180° and 270°, random axial, sagittal and coronal flipping and random scaling for the training set.

[0013] Furthermore, the uncertainty guides the image segmentation model training module to divide the training set obtained in the image preprocessing module into Small batches are used to train the medical image segmentation model batch by batch; during the training phase, Represents the training set of the current batch, corresponding to the true value label of the region of interest. Indicates that the medical image segmentation model consists of a feature extractor S, an uncertainty fusion device UF, an uncertainty measurer QU and an uncertainty adaptive learner USAT, and the training process includes the following steps: 1) Input the feature extractor S to obtain the feature map output by each layer of decoder; use represents the feature map, Indicates feature map, also called the Hierarchical feature map, feature map Determined by the following formula: ; 2) All feature maps obtained in step 1) Input into the uncertainty fuser UF to obtain the fusion graph , determined by the following formula: ; The uncertainty fusion UF aims to enhance the domain spatial response and summarize the nonlinear information across channels at the voxel level, so as to emphasize the semantic difference between the segmented target and the background; Different levels of feature maps Perform convolution with a kernel size of 1×1×1 , perform trilinear interpolation on all low-resolution feature maps after convolution , and finally with the first level feature map Through the Concatenate operation, the features are superimposed on the channel dimension to obtain a fusion graph , determined by the following formula: ; 3) The fusion image obtained in step 2) Input into the uncertainty metric QU and obtain the uncertainty graphs And the segmentation result map , determined by the following formula: ; The uncertainty measure QU is based on the Dempster-Shafer idea and is a generalization of Bayesian theory to subjective logic. By introducing a Softplus layer after the uncertainty fuser UF, the fusion graph is input. , to obtain non-negative evidence voxels , Indicates Voxel value, represents the total number of voxels, Category Evidence Voxels Determined by the following formula: ; No. Category Evidence Voxels The Dirichlet distribution is parameterized by subjective logic to obtain the Category Dirichlet distribution value , determined by the following formula: ; exist The Dirichlet distribution intensity can be obtained by accumulating the Dirichlet distribution value on the category channel of the segmentation category , determined by the following formula: ; Combined with Dirichlet distribution value and the Dirichlet distribution intensity , using subjective logic theory to model and calculate uncertainty maps And the segmentation result map , determined by the following formula ; This means that Category The more evidence a voxel obtains, the higher the prediction confidence value, the smaller the uncertainty, and the more reliable the prediction result; 4) The Dirichlet distribution value obtained in step 3) Input into the uncertainty adaptive learner USAT to obtain the modified Dirichlet distribution value , determined by the following formula: ; Combined with the modified Dirichlet distribution value , using subjective logic theory to model the corrected uncertainty map And the segmentation result map , determined by the following formula: ; The uncertainty adaptive learner USAT combines the idea of ​​human course learning and adaptively adjusts the threshold based on the learning state of the model to reduce mis-segmentation and optimize the segmentation results; the uncertainty adaptive learner USAT includes an adaptive global threshold and adaptive local threshold ; The adaptive local threshold It is based on the difference between categories and is set by the learning difficulty of the category. It is adjusted in real time as the model learns. The category accuracy The low threshold is reduced, and the accuracy is high categories, keep a high threshold to ensure the final accuracy, and combine the segmentation results of the validation set ,by Represents the total number of validation sets, Representative The number of iterations, Representative The adaptive global threshold at the iteration Adaptive local thresholding of classes Determined by the following formula: ; The adaptive global threshold Related to the uncertainty of the model on the training data to reflect the overall learning state; combined with momentum decay Steadily increase the adaptive global threshold during training , to ensure high segmentation accuracy in the later stages of training. Represents the total number of training set samples, the adaptive global threshold Determined by the following formula: ; Combined with the Dirichlet distribution value obtained in step 3) and adaptive global threshold Computes the value of the modified Dirichlet distribution , determined by the following formula: ; 5) The true value label obtained based on the region of interest delineation module Compute uncertainty label map , determined by the following formula: ; In the formula, Representative The corresponding individual pixels The true value label of the category, Representative The corresponding individual pixels Uncertainty label map of the categories; Combined with the corrected uncertainty map obtained in step 4) Calculating uncertainty loss function , determined by the following formula: ; In the formula, Representative The corresponding individual pixels Corrected uncertainty map for the categories; Combined with the modified Dirichlet distribution value obtained in step 4) , No. The corresponding individual pixels Category segmentation results , we can further calculate the The corresponding individual pixels Modified Dirichlet distribution post-processing value of the category , determined by the following formula: ; Further combined with the true value label Calculate the divergence loss function , determined by the following formula: ; In the formula, is the gamma function, is the digamma function, is the post-processed value of the modified Dirichlet distribution; Combined with the fusion image obtained in step 2) and the true value label Calculate the cross entropy-Dice coefficient combined loss function , determined by the following formula: ; In the formula, Representative The corresponding individual pixels Fusion graph of categories; The total loss function It can be determined by the following formula: ; According to the gradient of the loss function, the back propagation algorithm is used to update the parameters of the model. During the back propagation process, the medical image segmentation model automatically calculates the gradient of the loss function for each parameter; 6) According to the Adamw optimization algorithm, the gradient calculated in step 4) is used to update the model parameters. Each parameter update will make the model's loss function Reduce, every preset number of iterations, use the validation set to test the medical image segmentation model and calculate the accuracy index of the model. If the test results of the model do not meet the requirements, it is necessary to adjust the model structure, loss function and hyperparameters and retrain the model; 7) Repeat steps 2)-6) until the model's loss function becomes stable or reaches the preset number of iterations.

[0014] Furthermore, the medical image segmentation model testing module tests the medical image segmentation model obtained by the uncertainty-guided image segmentation model training module, inputs the test set into the medical image segmentation model obtained by each iterative training round, obtains the segmentation result, and calculates the accuracy index and the reliability index of the uncertainty estimation according to the segmentation result; The accuracy index calculation is carried out by measuring the difference between the model segmentation result and the true value label, which is divided into a region-based calculation method and a boundary-based calculation method. Among them, the region-based measurement compares the similarity between the segmentation result and the true value label, and the indicator used is the Dice coefficient. The Dice coefficient is an indicator of the degree of overlap between the segmentation result and the true value label. It is calculated as follows: the segmentation result and the true value label are converted into binary images respectively, and then the two are multiplied to obtain the intersection. The number of voxels in the intersection is divided by the sum of the total number of voxels in the two binary images. The result of the division is the Dice coefficient. The value range of the Dice coefficient is 0 to 1. The closer the value is to 1, the better the quality of the segmentation result. The Dice coefficient is determined by the following expression: ; In the formula, Representative The true value label under each voxel, Representative Predicted value at each voxel; The boundary-based metric compares the boundary difference between the segmentation result and the true value label. The indicator used is the average symmetric surface distance, which is calculated as follows: the cumulative value of the Euclidean distance from each voxel point in the segmentation result to the midpoint set of the true value label and the cumulative value of the Euclidean distance from each voxel point in the true value label to the midpoint set of the segmentation result are calculated respectively, and then the sum of the two is taken and divided by the sum of the midpoint set of the segmentation result and the midpoint set of the true value label as the average symmetric surface distance; the smaller the average symmetric surface distance value, the smaller the boundary difference between the segmentation result and the true value label; and Represent the point sets in the true value label and the segmentation result respectively, and Represent the points in the point set respectively. represents the average symmetric surface distance, which is determined by the following expression: ; Medical image segmentation model The reliability index of the medical image segmentation model can be obtained by direct or indirect calculation. The direct method is to obtain the confidence index. The confidence is a measure of the reliability of the prediction results of the medical image segmentation model, that is, the degree of certainty of the model for the classification results of each voxel point. The calculation method is to take the maximum value of the predicted logical value of the model output layer for each voxel, specifically the value after the Sigmoid or Softmax activation function, and then calculate the average value of all pixels in each channel of each test sample as the confidence value of each organ, and use represents the confidence value, represents the total number of samples in the test set, Indicates The probability value of the sample prediction and the confidence value are determined by the following expression: ; The indirect method is calculated by combining the accuracy index, and the index used is the prediction calibration error, which is used to balance the reliability of uncertainty estimation in the medical image segmentation model. The calculation method is the absolute difference between the statistical accuracy index and the confidence level, and the absolute difference is averaged to obtain the prediction calibration error; use represents the prediction calibration error, Indicates The accuracy index of the samples is Indicates The confidence index of the sample is given by the following expression: ; The model with the best segmentation result quality among the models obtained in each round of iterative training is selected as the optimal medical image segmentation model, which means that for the models obtained in each round of iterative training, the model with the largest regional overlap, the smallest boundary difference and the most reliable uncertainty estimation is selected as the optimal image segmentation model.

[0015] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. The system of the present invention adopts subjective logic theory to predict and model the medical image segmentation model, without excessive model modification and increase in computational burden. It quantifies the uncertainty at the voxel level from the perspective of evidence deep learning to solve the problem of model reliability in clinical deployment. At the same time, uncertainty fuser and uncertainty adaptor are used to strengthen model learning at a more detailed level to improve the performance of medical image segmentation models in clinical applications.

[0016] 2. Compared with the Bayesian-based method, the system of the present invention has consistent output, so that the medical image segmentation model is easy to optimize and reduces the dependence on the amount of data.

[0017] 3. Compared with the method based on model integration, the system of the present invention is always composed of one main model, and there is no need to maintain multiple models at the same time, which reduces the computing cost.

[0018] 4. Compared with the method based on test time enhancement, the system of the present invention does not need to worry about the problem of disturbance data amplitude, thus reducing the complexity of calculation.

[0019] 5. Compared with the existing uncertainty assessment methods of medical image segmentation models, the system of the present invention is based on an end-to-end deep learning method and does not require complex pre-processing and post-processing of the data. The uncertainty aggregator, uncertainty measurer and uncertainty adaptive learner in the system can be efficiently used to guide the learning of different categories of medical image segmentation models. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is an architecture diagram of the system of the present invention.

[0021] Figure 2 It is a schematic diagram of the uncertainty-guided image segmentation model training module.

[0022] Figure 3 It is the structural diagram of the uncertainty fuser UF.

[0023] Figure 4 It is the structural diagram of the uncertainty measurer QU.

[0024] Figure 5 It is the structural diagram of the uncertainty adaptive learner USAT. DETAILED DESCRIPTION

[0025] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0026] like Figure 1 As shown, this embodiment discloses a medical image segmentation model learning system based on voxel uncertainty, including: an image acquisition module, a region of interest delineation module, an image preprocessing module, an uncertainty-guided image segmentation model training module and a medical image segmentation model testing module.

[0027] Specifically, the image acquisition module is used to collect image data sets, which should include organ images obtained from the same part of multiple samples at the same time. In this embodiment, the parts of the sample selection are all abdominal organs, including the stomach, liver, kidneys, gallbladder, left and right adrenal glands, spleen, esophagus, pancreas and portal vein, and three public data sets of different sizes, BTCV, LiTs2017 and Flare2022, are used for implementation; patients are randomly divided, and the ratio of training set, validation set and test set is 8:1:1.

[0028] Specifically, the region of interest delineation module delineates the region of interest in the reference image, and the radiologist delineates various regions of interest on the abdomen of the human body in the data set, and uses the delineation results as the true value labels in this embodiment; Specifically, the image preprocessing module preprocesses the image according to the characteristics of the medical image segmentation model, so that the performance and generalization ability of the medical image segmentation model can be improved, thereby better solving practical problems; For the medical image segmentation model, the image preprocessing module performs the following operations: resample each image slice to an isotropic voxel spacing of 1.0 mm, and normalize the value range from -1000 (Henrique unit) to 1000 (Henrique unit) to 0 (Henrique unit) to 1 (Henrique unit), and then randomly divide the training set, validation set and test set, and finally adopt a data enhancement strategy of random rotation of 90°, 180° and 270°, random axial, sagittal and coronal flipping and random scaling for the training set.

[0029] Specifically, the uncertainty guidance image segmentation model training module divides the training set obtained in the image preprocessing module into The medical image segmentation model is trained batch by batch. In this embodiment, the minibatch size is set to 1, and the specific value can be adjusted according to the size of the GPU available video memory. Represents the training set of the current batch, corresponding to the true value label of the region of interest. Indicates; Figure 2 As shown, the medical image segmentation model includes a feature extractor S, an uncertainty fuser UF, an uncertainty measurer QU and an uncertainty adaptive learner USAT, wherein the model used by the feature extractor S in this embodiment is Attention u-net and nnU-Net, and the training process includes the following steps: 1) If Figure 2 As shown, the training set of the current batch Input the feature extractor S to obtain the feature map output by each layer of decoder; use represents the feature map, Indicates feature map, also called the Hierarchical feature map, feature map Determined by the following formula ; 2) All feature maps obtained in step 1) Input into the uncertainty fuser UF to obtain the fusion graph , determined by the following formula: ; like Figure 3 As shown, the uncertainty fusion UF aims to enhance the domain spatial response and summarize the nonlinear information across channels at the voxel level, so as to emphasize the semantic difference between the segmented target and the background; Different levels of feature maps Perform convolution with a kernel size of 1×1×1 , perform trilinear interpolation on all low-resolution feature maps after convolution , and finally with the first level feature map Through the Concatenate operation, the features are superimposed on the channel dimension to obtain a fusion graph , determined by the following formula: ; 3) The fusion image obtained in step 2) Input into the uncertainty metric QU and obtain the uncertainty graphs And the segmentation result map , determined by the following formula: ; like Figure 4 As shown in the figure, the uncertainty measure QU is based on the Dempster-Shafer idea, which is a generalization of Bayesian theory to subjective logic. By introducing a Softplus layer after the uncertainty fuser UF, the fusion graph is input , to obtain non-negative evidence voxels , Indicates Voxel value, represents the total number of voxels, Category Evidence Voxels Determined by the following formula: ; No. Category Evidence Voxels The Dirichlet distribution is parameterized by subjective logic to obtain the Category Dirichlet distribution value , can be determined by the following formula: ; exist The Dirichlet distribution intensity can be obtained by accumulating the Dirichlet distribution value on the category channel of the segmentation category , can be determined by the following formula: ; Combined with Dirichlet distribution value and the Dirichlet distribution intensity , using subjective logic theory to model and calculate uncertainty maps And the segmentation result map , can be determined by the following formula ; This means that Category The more evidence a voxel obtains, the higher the prediction confidence value, the smaller the uncertainty, and the more reliable the prediction result; 4) The Dirichlet distribution value obtained in step 3) Input into the uncertainty adaptive learner USAT to obtain the modified Dirichlet distribution value , determined by the following formula: ; Combined with the modified Dirichlet distribution value , using subjective logic theory to model the corrected uncertainty map And the segmentation result map , can be determined by the following formula: ; like Figure 5 As shown, the uncertainty adaptive learner USAT combines the idea of ​​human course learning and adaptively adjusts the threshold based on the learning state of the model to reduce mis-segmentation and optimize the segmentation result; the uncertainty adaptive learner USAT includes an adaptive global threshold and an adaptive local threshold; the adaptive local threshold It is based on the difference between categories and is set by the learning difficulty of the category. It is adjusted in real time as the model learns. The category accuracy The low threshold is reduced, and the accuracy is high categories, keep a high threshold to ensure the final accuracy, and combine the segmentation results of the validation set ,by Represents the total number of validation sets, Representative The number of iterations, Representative The adaptive global threshold at the iteration Adaptive local thresholding of classes It can be determined by the following formula: ; The adaptive global threshold Related to the uncertainty of the model on the training data to reflect the overall learning state; combined with momentum decay Steadily increase the adaptive global threshold during training , to ensure high segmentation accuracy in the later stages of training. Represents the total number of training set samples, adaptive global threshold It can be determined by the following formula: ; Combined with the Dirichlet distribution value obtained in step 3) and adaptive global threshold Computes the value of the modified Dirichlet distribution , can be determined by the following formula: ; 5) If Figure 2 As shown, the true value label obtained based on the region of interest delineation module Compute uncertainty label map , can be determined by the following formula: ; In the formula, Representative The corresponding individual pixels The true value label of the category, Representative The corresponding individual pixels Uncertainty label map of the categories; Combined with the corrected uncertainty map obtained in step 4) Calculating uncertainty loss function , can be determined by the following formula: ; In the formula, Representative The corresponding individual pixels Corrected uncertainty map for the categories; Combined with the modified Dirichlet distribution value obtained in step 4) , No. The corresponding individual pixels Category segmentation results , we can further calculate the The corresponding individual pixels Modified Dirichlet distribution post-processing value of the category , can be determined by the following formula: ; Further combined with the true value label Calculate the divergence loss function , determined by the following formula: ; In the formula, is the gamma function, is the digamma function, is the post-processed value of the modified Dirichlet distribution; Combined with the fusion image obtained in step 2) and the true value label Calculate the cross entropy-Dice coefficient combined loss function , can be determined by the following formula: ; In the formula, Representative The corresponding individual pixels Fusion graph of categories; The total loss function It can be determined by the following formula: ; According to the gradient of the loss function, the back propagation algorithm is used to update the parameters of the model. During the back propagation process, the medical image segmentation model automatically calculates the gradient of the loss function for each parameter; 6) According to the Adamw optimization algorithm, use the gradient calculated in step 4) to update the model parameters. Each parameter update will make the model's loss function Reduce, every preset number of iterations, use the validation set to test the medical image segmentation model and calculate the accuracy index of the model. If the test results of the model do not meet the requirements, it is necessary to adjust the model structure, loss function and hyperparameters and retrain the model; 7) Repeat steps 2)-6) until the model's loss function Become stable or reach the preset number of iterations; Specifically, the medical image segmentation model testing module tests the medical image segmentation model obtained by the uncertainty-guided image segmentation model training module, inputs the test set into the medical image segmentation model obtained by each iterative training round, obtains the segmentation result, and calculates the accuracy index and the reliability index of the uncertainty estimation according to the segmentation result; The accuracy index calculation is performed by measuring the difference between the model segmentation result and the true value label, which is divided into a region-based calculation method and a boundary-based calculation method; among which, the region-based metric is to compare the similarity between the segmentation result and the true value label. In this embodiment, the indicator used is the Dice coefficient; The Dice coefficient is an indicator of the degree of overlap between the segmentation result and the true value label. It is calculated as follows: the segmentation result and the true value label are converted into binary images respectively, and then the two are multiplied to obtain the intersection. The number of voxels in the intersection is divided by the sum of the total number of voxels in the two binary images. The result of the division is the Dice coefficient. The value range of the Dice coefficient is 0 to 1. The closer the value is to 1, the better the quality of the segmentation result. The Dice coefficient is determined by the following expression: ; The boundary-based metric is to compare the boundary differences between the segmentation result and the true value label. In this embodiment, the indicator used is the average symmetric surface distance, which is calculated as follows: the cumulative value of the Euclidean distance from each voxel point in the segmentation result to the midpoint set of the true value label and the cumulative value of the Euclidean distance from each voxel point in the true value label to the midpoint set of the segmentation result are calculated respectively, and then the sum of the two is taken and divided by the sum of the midpoint set of the segmentation result and the midpoint set of the true value label as the average symmetric surface distance. The smaller the average symmetric surface distance value, the smaller the boundary difference between the segmentation result and the true value label; and Represent the point sets in the true value label and the segmentation result respectively, and Represent the points in the point set respectively. represents the average symmetric surface distance, which is determined by the following expression: ; The reliability index of the medical image segmentation model can be obtained by direct or indirect calculation; The direct method is to obtain the confidence index. Confidence is a measure of the reliability of the prediction results of the medical image segmentation model, that is, the degree of certainty of the model's classification results for each voxel point. The calculation method is to take the maximum value of the predicted logits for each voxel, specifically the value after the Sigmoid or Softmax activation function, and then calculate the average value of all pixels in each channel of each test sample as the confidence value of each organ, and use represents the confidence value, represents the total number of samples in the test set, Indicates The probability value of the sample prediction and the confidence value are determined by the following expression: ; The calculation method of the indirect method is to calculate in combination with the accuracy index. In this embodiment, the prediction calibration error is mainly used to balance the reliability of uncertainty estimation in the medical image segmentation model. The calculation method is the absolute difference between the statistical accuracy index and the confidence level, and the absolute difference is averaged to obtain the prediction calibration error; use represents the prediction calibration error, Indicates The accuracy index of the samples is Indicates The confidence index of the sample is given by the following expression: ; In this example, we distribute the normal state and Gaussian noise Under the condition of , different medical image segmentation models in the two public datasets LiTs2017 and Flare2022 were tested and the results of calculating the Dice coefficient, average symmetric surface distance and prediction calibration error are as follows: Table 1 Segmentation performance evaluation results in the LiTS2017 dataset, N and OOD represent normal state and Gaussian noise respectively The conditions are shown in bold and bold black, respectively, indicating the optimal and suboptimal results. Table 2 Segmentation performance evaluation results in the Flare2022 dataset, N and OOD represent normal state and Gaussian noise respectively The conditions are shown in bold and bold black, respectively, indicating the optimal and suboptimal results. As shown in Tables 1 and 2, under normal conditions, compared with the baseline model, this embodiment has significantly improved the Dice coefficient and ASSD on the LiTS2017 and Flare2022 datasets. Specifically, on the LiTS2017 dataset, the Attention u-ne and nnU-Net supported by this embodiment increased the Dice coefficient by 2.4% and 3.7%, respectively, and reduced the ASSD by 1.39 and 2.34, respectively. On the Flare2022 dataset, the Attention u-ne and nnU-Net supported by this embodiment increased the Dice coefficient by 3.6% and 4.9%, respectively, and reduced the ASSD by 0.97 and 1.22, respectively. In addition, this embodiment achieved the best performance in the ECE indicators of both datasets, proving the effectiveness of the uncertainty estimation scheme in this embodiment. Compared with other uncertainty estimation methods Dropout U-Net, U-Net Ensemble, Probabilistic U-Net and TBraTS, this embodiment achieved excellent performance on both datasets. Specifically, in the LiTS2017 dataset, the nnU-Net powered by this embodiment performs 1.7% higher than TBraTS in the Dice coefficient, 0.09 lower than ProbabilisticU-Net in ASSD, and 0.003 lower than TBraTS and ProbabilisticU-Net in ECE. In the Flare2022 dataset, the AttentionU-Net and nnU-Net powered by this embodiment achieve the top two results in all three indicators. These results demonstrate the superiority of this embodiment.

[0030] In order to further verify the reliability of this embodiment, Gaussian noise is added to the image data. The same index tests were performed under the conditions of , as shown in Tables 1 and 2. It is observed that after the Gaussian noise is added, the segmentation accuracy of most models is significantly reduced, especially the Dropout U-Net, U-Net Ensemble and Probabilistic U-Net in the uncertainty estimation method. Although TBraTS has made some improvements, it is still difficult to maintain high performance under high noise conditions. However, this embodiment shows impressive results. In the LiTS2017 dataset, the volatility values ​​of the Attention u-ne and nnU-Net in the Dice coefficient, ASSD and ECE of this embodiment are only 11.2% and 5.8%, 3.1 and 2.2, and 0.034 and 0.022, which are significantly lower than other uncertainty estimation methods, such as 26.7% (Dropout U-Net), 7.53 (Probabilistic U-Net) and 0.069 (Probabilistic U-Net). In the Flare2022 dataset, the floating values ​​of Attention u-ne and nnU-Net in Dice, ASSD and ECE of this embodiment are only 8.5% and 5.1%, 1.79 and 0.53, and 0.053 and 0.032, while the floating values ​​of Dropout U-Net can be as high as 35.5%, 8.97 and 0.239. These results show that this embodiment has the best noise resistance and can maintain the best performance under high noise conditions.

[0031] In the uncertainty-guided image segmentation model training module of this embodiment, the feature extractor S, the uncertainty fuser UF, the uncertainty measurer QU and the uncertainty adaptive learner USAT all play an important role. The effectiveness of each is experimentally tested on the BTCV dataset, and the results are as follows: Table 3 Ablation experiment results on BTCV dataset As shown in Table 3 above, the proposed feature extractor S, uncertainty fusion device UF, uncertainty metric QU and uncertainty adaptive learner USAT play an important role in guiding the training of the image segmentation model. Selecting nnU-Net as the feature extractor, adding only the uncertainty metric QU increases the Dice coefficient by 2% and reduces the ASSD by 1.61, which shows that when the uncertainty at the voxel level is introduced to guide the training of the image segmentation model, the performance of the segmentation is significantly improved. In addition, the introduction of the uncertainty fusion device UF and the uncertainty adaptive learner USAT further improves the accuracy of the model. The introduction of the two increases the Dice coefficient by 3.4% and 3.2%, respectively, and reduces the ASSD by 2.88 and 2.35, respectively. Finally, under the synergistic combination of the feature extractor S, the uncertainty fusion device UF, the uncertainty metric QU and the uncertainty adaptive learner USAT, this embodiment creates a Dice coefficient increase of 4.5% and a reduction of ASSD by 3.11, which shows that the learning ability of the image segmentation model is significantly enhanced. In addition, the reliability of this embodiment is also reflected in the decline of the ECE index.

[0032] The model with the best segmentation result quality among the models obtained in each round of iterative training is selected as the optimal medical image segmentation model, which means that for the models obtained in each round of iterative training, the model with the largest regional overlap, the smallest boundary difference and the most reliable uncertainty estimation is selected as the optimal image segmentation model.

[0033] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A medical image segmentation model learning system based on voxel uncertainty, characterized in that: include: An image acquisition module is used to acquire medical image data sets, including CT image data of the same organ in multiple samples and multi-sequence MRI image data acquired at the same time; The medical imaging dataset is randomly divided into training set, validation set and test set; The region of interest delineation module is used to delineate the region of interest in the acquired CT image data and multi-sequence MRI image data as the true value label; The image preprocessing module is used to preprocess the collected CT image data, multi-sequence MRI image data and the outlined true value labels. It uses cropping, format conversion, normalization and data enhancement to make the training data more reasonable and diverse to improve the robustness, generalization performance and training effect of the model, thereby obtaining new data samples; The uncertainty-guided image segmentation model training module is used to train the medical image segmentation model, which includes a feature extractor S, an uncertainty fuser UF, an uncertainty measurer QU, and an uncertainty adaptive learner USAT, all of which are used for iterative training on the training set. During the training process, gradient backpropagation and model parameter adjustment are performed according to the loss function value, so that the model gradually converges to the optimal value. At the same time, the validation set is used for validation calculation to assist in adjusting the model parameters and prevent overfitting of the model training, so as to finally obtain a trained medical image segmentation model. The medical image segmentation model testing module is used to input all samples in the test set into the medical image segmentation models of each iterative training round obtained in the uncertainty-guided image segmentation model training module to generate corresponding segmentation results; and evaluate the segmentation quality of the region of interest in the segmentation results in combination with the true value labels obtained by the region of interest delineation module, and finally select the model with the best segmentation result quality among the models obtained in each round of iterative training as the optimal medical image segmentation model.

2. The medical image segmentation model learning system based on voxel uncertainty according to claim 1, characterized in that: The region of interest delineation module determines and delineates the region of interest of each sample according to the tissue structure characteristics in the reference image, and uses it as the region that the segmentation model focuses on.

3. The medical image segmentation model learning system based on voxel uncertainty according to claim 2, characterized in that: The image preprocessing module preprocesses the image according to the characteristics of the medical image segmentation model, so that the performance and generalization ability of the medical image segmentation model can be improved; For the medical image segmentation model, the image preprocessing module performs the following operations: resampling each image slice to an isotropic voxel spacing of 1.0 mm, and normalizing the value range from -1000 to 1000 Henry units to 0 to 1 Henry unit, and then randomly dividing the training set, validation set, and test set. Finally, the training set is randomly rotated by 90°, 180°, and 270°, and randomly axial, sagittal, and coronal flips, as well as random scaling data enhancement strategies are adopted.

4. The medical image segmentation model learning system based on voxel uncertainty according to claim 3, characterized in that: The uncertainty guides the image segmentation model training module to divide the training set obtained by the image preprocessing module into Small batches are used to train the medical image segmentation model batch by batch; during the training phase, Represents the training set of the current batch, corresponding to the true value label of the region of interest. Indicates that the medical image segmentation model consists of a feature extractor S, an uncertainty fusion device UF, an uncertainty measurer QU and an uncertainty adaptive learner USAT, and the training process includes the following steps: 1) Input into the feature extractor S to obtain the feature map output by each layer of decoder; use represents the feature map, Indicates feature map, also called the Hierarchical feature map, feature map Determined by the following formula: ； 2) All feature maps obtained in step 1) Input into the uncertainty fuser UF to obtain the fusion graph , determined by the following formula: ； The uncertainty fuser UF aims to enhance the domain spatial response and summarize the nonlinear information across channels at the voxel level, so as to emphasize the semantic difference between the segmented target and the background; Respectively Different levels of feature maps Perform convolution with a kernel size of 1×1×1 , perform trilinear interpolation on all low-resolution feature maps after convolution , and finally with the first level feature map Through the Concatenate operation, the features are superimposed on the channel dimension to obtain a fusion graph , determined by the following formula: ； 3) The fusion image obtained in step 2) Input into the uncertainty metric QU and obtain the uncertainty graphs And the segmentation result map , determined by the following formula: ； The uncertainty measure QU is based on the Dempster-Shafer idea and is a generalization of Bayesian theory to subjective logic; By introducing a Softplus layer after the uncertainty fuser UF, the fusion graph is input , to obtain non-negative evidence voxels , Indicates Voxel value, represents the total number of voxels, Category Evidence Voxels Determined by the following formula: ； No. Category Evidence Voxels The Dirichlet distribution is parameterized by subjective logic to obtain the Category Dirichlet distribution value , determined by the following formula: ； exist The Dirichlet distribution intensity can be obtained by accumulating the Dirichlet distribution value on the category channel of the segmentation category , determined by the following formula: ； Combined with Dirichlet distribution value and the Dirichlet distribution intensity , using subjective logic theory to model and calculate uncertainty maps And the segmentation result map , determined by the following formula ； This means that Category The more evidence a voxel obtains, the higher the prediction confidence value, the smaller the uncertainty, and the more reliable the prediction result; 4) The Dirichlet distribution value obtained in step 3) Input into the uncertainty adaptive learner USAT to obtain the modified Dirichlet distribution value , determined by the following formula: ； Combined with the modified Dirichlet distribution value , using subjective logic theory to model the corrected uncertainty map And the segmentation result map , determined by the following formula: ； The uncertainty adaptive learner USAT combines the idea of ​​human course learning and adaptively adjusts the threshold based on the learning state of the model to reduce mis-segmentation and optimize the segmentation results; the uncertainty adaptive learner USAT includes an adaptive global threshold and adaptive local threshold ; The adaptive local threshold It is based on the difference between categories and is set by the learning difficulty of the category. It is adjusted in real time as the model learns. The category accuracy The low threshold is reduced, and the accuracy is high categories, keep a high threshold to ensure the final accuracy, and combine the segmentation results of the validation set ,by Represents the total number of validation sets, Representative The number of iterations, Representative The adaptive global threshold at the iteration Adaptive local thresholding of classes Determined by the following formula: ； The adaptive global threshold Related to the uncertainty of the model on the training data to reflect the overall learning state; combined with momentum decay Steadily increase the adaptive global threshold during training , to ensure high segmentation accuracy in the later stages of training. Represents the total number of training set samples, the adaptive global threshold Determined by the following formula: ； Combined with the Dirichlet distribution value obtained in step 3) and adaptive global threshold Computes the value of the modified Dirichlet distribution , determined by the following formula: ； 5) The true value label obtained based on the region of interest delineation module Compute uncertainty label map , determined by the following formula: ； In the formula, Representative The corresponding individual pixels The true value label of the category, Representative The corresponding individual pixels Uncertainty label map of the categories; Combined with the corrected uncertainty map obtained in step 4) Calculating uncertainty loss function , determined by the following formula: ； In the formula, Representative The corresponding individual pixels Corrected uncertainty map for the categories; Combined with the modified Dirichlet distribution value obtained in step 4) , No. The corresponding individual pixels Category segmentation results , further calculations yield the The corresponding individual pixels Modified Dirichlet distribution post-processing value of the category , determined by the following formula: ； Further combined with the true value label Calculate the divergence loss function , determined by the following formula: ； In the formula, is the gamma function, is the digamma function, is the post-processed value of the modified Dirichlet distribution; Combined with the fusion image obtained in step 2) and the true value label Calculate the cross entropy-Dice coefficient combined loss function , determined by the following formula: ； In the formula, Representative The corresponding individual pixels Fusion graph of categories; The total loss function It can be determined by the following formula: ； According to the gradient of the loss function, the back propagation algorithm is used to update the parameters of the model. During the back propagation process, the medical image segmentation model automatically calculates the gradient of the loss function for each parameter; 6) According to the Adamw optimization algorithm, the gradient calculated in step 4) is used to update the model parameters. Each parameter update will make the model's loss function Reduce, every preset number of iterations, use the validation set to test the medical image segmentation model and calculate the accuracy index of the model. If the test results of the model do not meet the requirements, it is necessary to adjust the model structure, loss function and hyperparameters and retrain the model; 7) Repeat steps 2)-6) until the model's loss function becomes stable or reaches the preset number of iterations.

5. The medical image segmentation model learning system based on voxel uncertainty according to claim 4, characterized in that: The medical image segmentation model testing module tests the medical image segmentation model obtained by the uncertainty-guided image segmentation model training module, inputs the test set into the medical image segmentation model obtained by each iterative training round, obtains the segmentation result, and calculates the accuracy index and the reliability index of the uncertainty estimation according to the segmentation result; The accuracy index calculation is carried out by measuring the difference between the model segmentation result and the true value label, which is divided into a region-based calculation method and a boundary-based calculation method. Among them, the region-based measurement compares the similarity between the segmentation result and the true value label, and the indicator used is the Dice coefficient. The Dice coefficient is an indicator of the degree of overlap between the segmentation result and the true value label. It is calculated as follows: the segmentation result and the true value label are converted into binary images respectively, and then the two are multiplied to obtain the intersection. The number of voxels in the intersection is divided by the sum of the total number of voxels in the two binary images. The result of the division is the Dice coefficient. The value range of the Dice coefficient is 0 to 1. The closer the value is to 1, the better the quality of the segmentation result. The Dice coefficient is determined by the following expression: ； In the formula, No. The true value label under each voxel, Representative Predicted value at each voxel; The boundary-based metric compares the boundary difference between the segmentation result and the true value label. The indicator used is the average symmetric surface distance, which is calculated as follows: the cumulative value of the Euclidean distance from each voxel point in the segmentation result to the midpoint set of the true value label and the cumulative value of the Euclidean distance from each voxel point in the true value label to the midpoint set of the segmentation result are calculated respectively, and then the sum of the two is taken and divided by the sum of the midpoint set of the segmentation result and the midpoint set of the true value label as the average symmetric surface distance; the smaller the average symmetric surface distance value, the smaller the boundary difference between the segmentation result and the true value label; and Represent the point sets in the true value label and the segmentation result respectively, and Represent the points in the point set respectively. represents the average symmetric surface distance, which is determined by the following expression: ； The reliability index of the medical image segmentation model can be obtained by direct or indirect calculation; The direct method is to obtain the confidence index. The confidence index is used to measure the reliability of the prediction results of the medical image segmentation model, that is, the degree of certainty of the model's classification results for each voxel point. The calculation method is to take the maximum value of the predicted logical value of the model output layer for each voxel, specifically the value after the Sigmoid or Softmax activation function, and then calculate the average value of all pixels in each channel of each test sample as the confidence value of each organ, and use represents the confidence value, represents the total number of samples in the test set, Indicates The probability value of the sample prediction and the confidence value are determined by the following expression: ； The indirect method is calculated by combining the accuracy index, and the index used is the prediction calibration error, which is used to balance the reliability of uncertainty estimation in the medical image segmentation model. The calculation method is the absolute difference between the statistical accuracy index and the confidence level, and the absolute difference is averaged to obtain the prediction calibration error; use represents the prediction calibration error, Indicates The accuracy index of the samples is Indicates The confidence index of the sample is given by the following expression: ； The model with the best segmentation result quality among the models obtained in each round of iterative training is selected as the optimal medical image segmentation model, which means that for the models obtained in each round of iterative training, the model with the largest regional overlap, the smallest boundary difference and the most reliable uncertainty estimation is selected as the optimal image segmentation model.

Images