Uncertain medical image out-of-distribution data detection method based on evidence reconciliation

Abstract

The invention relates to the field of computer medical image processing, and provides an uncertain medical image distribution external data detection method based on evidence reconciliation. The method comprises the steps of performing feature extraction on an input medical image, generating specific evidence representation, exporting a corresponding prediction result, performing uncertainty measurement, and performing identification on an out-of-distribution sample. The method is easy to realize, the evidence obtained in the traditional method is corrected, the detection capability of the out-of-distribution sample is improved, and the credibility of medical auxiliary diagnosis is enhanced. According to the method, the misdiagnosis risk caused by incomplete training data in medical auxiliary diagnosis can be effectively reduced, and the calculation complexity of an original network is not influenced. In future application, the problem of excessive confidence in medical auxiliary diagnosis is relieved, and a credible basis is provided for diagnosis of doctors.

Description

Technical Field

[0001] The present invention relates to the field of computer medical image processing. Background Art

[0002] In traditional classification tasks, the model assumes that the training data and the test data are independent and identically distributed (iid), that is, the training data and the test data are independent of each other and have the same feature distribution. Under this assumption, the model that fits the features of the training set samples can achieve similar performance on the test set, but this assumption is very idealistic and greatly simplifies the algorithm's robustness requirements. The mismatch between the performance of the model in the training set and the test set in various practical applications has shown that this assumption is unreliable in most cases. Especially in medical auxiliary diagnosis, the deviation of the feature distribution of training data and test data is very likely to lead to misdiagnosis of diseases and bring great risks to patients. In recent years, deep neural networks have been widely used in various medical applications, such as classification and segmentation tasks, achieving state-of-the-art performance. However, given the potential risks caused by the overconfidence of traditional neural networks in prediction results, deep learning methods with uncertainty estimation have become increasingly important because they can ensure the stability of the model in the presence of out-of-distribution test samples.

[0003] Out-of-distribution detection methods based on uncertainty estimation, such as evidence deep learning and its variants, have shown their advantages in computational performance, efficiency, and scalability. In this type of method, higher uncertainty means that the test sample is more different from the training data and is more likely to be an out-of-distribution sample. However, most of these studies consider identifying abnormal samples that are significantly different from the training samples, such as collecting natural images as out-of-distribution samples of medical imaging data. These methods ignore the near out-of-distribution problem inherent in medical imaging. In this case, due to the incompleteness of the training data, there may be near out-of-distribution samples in the test data that do not belong to any training category. Misdiagnosis of these near out-of-distribution samples can lead to incorrect treatment plans in clinical applications and pose a very high risk. Recently, some near out-of-distribution sample detection methods based on density models, preprocessing, and outlier exposure have been proposed. However, these methods are easily affected by the quality of the training set and have problems such as high computational complexity and difficulty in convergence. Their reliability in practical applications is often not guaranteed. Summary of the invention

[0004] The purpose of the present invention is to overcome the problems in the prior art methods, such as the strong dependence of auxiliary diagnosis on the completeness of training data, the overconfidence of deep learning models in diagnosis results, and the inaccurate uncertainty measurement. A method for detecting out-of-distribution data of uncertain medical images based on evidence reconciliation is proposed under the framework of Evidential Neural Network. The auxiliary diagnosis method is mainly divided into two steps, namely: reconciling evidence generation, classification prediction and out-of-distribution detection.

[0005] Technical solution of the present invention:

[0006] The method for detecting out-of-distribution data of uncertain medical images based on evidence reconciliation comprises the following steps:

[0007] Step 1: Dataset and model preparation:

[0008] 1.1 Collect images and corresponding category labels from hospitals and online public data sets, and build classified image data sets after preprocessing, data cleaning and organization, including training data and test data.

[0009] 1.2 Collect pre-trained neural networks trained on natural image datasets for image feature extraction.

[0010] 1.3 Construct an evidence reconciliation neural network, which includes a pre-trained neural network, an evidence generation module, and an evidence reconciliation module, which are connected in sequence.

[0011] Step 2: Training and Optimization:

[0012] 2.1 Use the pre-trained neural network to extract features from the training image data obtained in step 1.1.

[0013] 2.2 The features obtained in step 2.1 are passed through the evidence generation module to generate original evidence.

[0014] 2.3 The original evidence obtained in step 2.2 is reconciled and corrected through the evidence reconciliation module to obtain a reconciled evidence representation.

[0015] 2.4 Generate the Dirichlet distribution using the harmonic evidence obtained in step 2.3.

[0016] 2.5 Combine the category labels in the image dataset in step 1 to calculate the harmonic dual gamma evidence loss as the evaluation indicator.

[0017] 2.6 Freeze the pre-trained neural network parameters and train the evidence generation module and the evidence reconciliation module.

[0018] Step 3: Image category prediction and out-of-distribution detection for test data:

[0019] 3.1 Input the test data into the trained evidence reconciliation neural network to obtain the reconciliation evidence and the corresponding Dirichlet distribution.

[0020] 3.2 Derive the classification prediction and uncertainty measure of the test sample from the Dirichlet distribution in step 3.1.

[0021] 3.3 Compare the preset threshold with the uncertainty measurement value in step 3.2 to determine whether the sample is out-of-distribution.

[0022] 3.3.1 If the uncertainty measure value is greater than the threshold, it is judged as an out-of-distribution sample and medical experts are requested to intervene.

[0023] 3.3.2 If the uncertainty measure value is less than the threshold, it is judged as a sample within the distribution and the classification prediction is output.

[0024] The beneficial effects of the present invention are:

[0025] The uncertainty medical image out-of-distribution data detection method based on evidence reconciliation proposed in the present invention corrects the evidence obtained in the traditional uncertainty evidence method, improves the detection ability of out-of-distribution samples, and enhances the credibility of medical-assisted diagnosis. The uncertainty measurement in the present invention can be used as a basis for decision-making, effectively reducing the risk of misdiagnosis caused by incomplete training data in medical-assisted diagnosis. The calibration of evidence can effectively improve the model's recognition effect on out-of-distribution samples. As a medical-assisted diagnosis solution, it can provide a reliable auxiliary basis for doctors' subsequent treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is the overall flow chart of the method of the present invention.

[0027] Figure 2 It is a flow chart of implementing training optimization of the present invention.

[0028] Figure 3 It is a flow chart of implementing image category prediction and out-of-distribution detection in the present invention.

[0029] Figure 4 It is a schematic diagram comparing the network workflow of the present invention with the traditional method.

[0030] FIG5( a ) is a schematic diagram of network training according to an embodiment of the present invention.

[0031] FIG5( b ) is a schematic diagram of output category prediction for samples within a true distribution according to an embodiment of the present invention.

[0032] FIG5(c) is a schematic diagram of requesting expert intervention for samples outside the true distribution according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The Evidential Neural Network is based on an intuitive fact: the prediction of classification problems should obey the categorical distribution, so the conjugate prior of the categorical distribution, namely the Dirichlet distribution, can be used to model the categorical distribution. This method uses the network output evidence to generate the Dirichlet distribution, and derives the classification prediction and the corresponding uncertainty based on the Bayesian theory.

[0034] According to subjective logic, out-of-distribution detection should be regarded as a non-frequentist situation, in which the evidence supporting different categories in the output of the evidence neural network should offset each other to correct the uncertainty and achieve more effective detection of out-of-distribution samples.

[0035] The present invention proposes an uncertain medical image out-of-distribution data detection method based on evidence reconciliation under the framework of an evidence neural network. The method extracts features from the input medical images and generates specific evidence representations, derives corresponding prediction results and uncertainty metrics, and identifies out-of-distribution samples.

[0036] The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0037] like Figure 1 As shown in the figure, the uncertainty medical image out-of-distribution data detection method based on evidence reconciliation has the following workflow:

[0038] Step 1: Dataset preparation. The specific implementation process is as follows:

[0039] 1.1 Collect image data X and corresponding category labels Y from hospitals and public datasets to create a dataset, denoted as D = {X, Y}.

[0040] 1.2 Collect pre-trained neural networks trained on natural image datasets.

[0041] 1.3 Construct an evidence reconciliation neural network, which is composed of a pre-trained neural network, an evidence generation module, and an evidence reconciliation module in sequence.

[0042] Step 2: Training optimization. The specific process is as follows: Figure 2 The specific implementation process is as follows:

[0043] 2.1 Freeze the pre-trained neural network parameters, take a sample (x, y)∈D from the data set D, input the image data x into the pre-trained neural network to extract features, and obtain feature z.

[0044] 2.2 Input feature z into the evidence generation module. The original evidence is obtained in the module The non-negativity of evidence is limited by the softplus activation function. The softplus function is:

[0045]

[0046] 2.3 According to the non-frequency hypothesis in subjective logic, evidence supporting different categories should offset each other to eliminate the impact on uncertainty. Based on this, the present invention reconciles and corrects the original evidence. The specific operation of the evidence reconciliation module is as follows:

[0047] 2.3.1 Based on the non-frequency hypothesis, firstly, according to the original evidence Calculate the maximum reconcilable evidence value

[0048]

[0049] 2.3.2 Calculate the corrected reconciliation evidence representation e:

[0050]

[0051] 2.4 Based on the reconciliation evidence in 2.3, the Dirichlet distribution Dir(μ|α) with parameter α is generated. Assuming that there are K classes in the training set, its probability density can be expressed as:

[0052]

[0053] in μ is a random variable of Dirichlet distribution, e=(e1,e2,…,e K ) is the reconciliation evidence in vector form.

[0054] 2.5 Combine the Dirichlet distribution parameter α with the category label y in the data set in step 1, and calculate the harmonic gamma evidence loss as an evaluation index for training the evidence harmonic neural network. Harmonic gamma evidence loss L rece The calculation formula is:

[0055]

[0056] where ψ(·) is the digamma function, α=(α1,α2,…,α K ) are the parameters of the Dirichlet distribution in vector form.

[0057] 2.6 Freeze the pre-trained neural network parameters and train the evidence generation module and the evidence reconciliation module.

[0058] Step 3: predict the image category and detect out-of-distribution detection for the test data. The process is as follows: Figure 3 The specific implementation process is as follows:

[0059] 3.1 Input the test image into the evidence reconciliation neural network trained in step 2 to obtain the reconciled evidence and the corresponding Dirichlet distribution Dir(μ|α).

[0060] 3.2 Derive the classification prediction p(μ) and uncertainty measure ent from the Dirichlet distribution.

[0061]

[0062] ent=-p(μ)logp(μ)

[0063] 3.3 Given a threshold τ, compare the uncertainty measure ent with τ and make a decision.

[0064] 3.3.1 If ent>τ, the sample is judged to be out-of-distribution and does not belong to any category in the training set. The doctor should be reminded to intervene and make further decisions.

[0065] 3.3.2 If ent ≤ τ, the sample is judged to be an in-distribution sample, and the classification prediction p is used as the sample prediction result output.

[0066] The comparison between the present invention and the traditional method is as follows Figure 4 As shown:

[0067] The blue arrows indicate the data flow of samples in the traditional evidence neural network. After the sample is input, it passes through the pre-trained neural network and the evidence generation module in turn to obtain the feature vector and the original evidence respectively. Finally, the Dirichlet distribution is generated for subsequent processing.

[0068] The yellow arrows indicate the data flow of the sample in the evidence reconciliation neural network of the present invention. The sample is passed through the pre-trained neural network to obtain the feature vector, and then passes through the evidence generation module to obtain the original evidence. In the evidence reconciliation module, the present invention converts the original subjective opinion generated by the original evidence into a reconciled opinion based on the subjective logic theory, and derives its corresponding reconciled evidence representation. Finally, the Dirichlet distribution is generated for subsequent processing.

[0069] Among them, the blue dotted arrows represent the data flow of out-of-distribution samples in the traditional evidential neural network, and the yellow dotted arrows represent the data flow of out-of-distribution samples in the harmonic evidential neural network of the present invention, both of which generate Dirichlet distributions that are close to uniform distributions.

[0070] Among them, the blue solid arrows represent the data flow of samples within the distribution in the traditional evidence neural network, and the yellow solid arrows represent the data flow of samples within the distribution in the harmonized evidence neural network of the present invention. The Dirichlet distribution generated by the harmonized evidence neural network of the present invention can derive more accurate prediction results than the traditional evidence neural network, increase the output difference between samples within the distribution and samples outside the distribution, thereby achieving better out-of-distribution data detection capabilities.

[0071] Example

[0072] During training, as shown in Figure 5(a), the medical imaging data in the collected dataset needs to be input into the evidence reconciliation network to model and constrain its feature generation process. The evidence reconciliation neural network consists of three parts: pre-trained neural network, evidence generation module, and evidence reconciliation module.

[0073] First, the image data is input into the pre-trained neural network to obtain its corresponding feature vector. This feature can reflect the grayscale, texture and other feature information of the image to a certain extent, providing effective support for classification diagnosis. Subsequently, the feature vector is input into the evidence generation module to generate the original evidence value corresponding to each category. Then, this value is input into the reconciliation module to obtain the reconciled evidence. Finally, the Dirichlet distribution corresponding to the reconciled evidence is calculated, and the dual gamma evidence loss is calculated in combination with the data label to update the network parameters. Repeat the above operation several times until the network training is completed.

[0074] In the testing phase, as shown in Figure 5(b) and Figure 5(c), the test sample is input into the evidence reconciliation network and its corresponding reconciliation evidence representation is output. Then the Dirichlet distribution corresponding to the reconciliation evidence is calculated, and the corresponding prediction value and uncertainty measure are derived. In Figure 5(b), the certainty of the test sample is lower than the given threshold, so the classification result is directly output. In Figure 5(c), the certainty of the test sample is higher than the given threshold, so a doctor is requested to intervene.

[0075] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. An uncertain out-of-distribution data detection method for medical images based on evidence reconciliation, characterized in that: Includes steps: Step 1: Dataset and model preparation; Step 2: Training optimization; Step 3: Image category prediction and out-of-distribution detection for test data.

2. The method according to claim 1, characterized in that The step 1 specifically includes: Step 1.1: Collect images and corresponding category labels, and build a classified image dataset after preprocessing, data cleaning, and organization, including training data and test data; Step 1.2: Collect pre-trained neural networks trained on natural image datasets for image feature extraction; Step 1.3: Construct an evidence reconciliation neural network, which includes a pre-trained neural network, an evidence generation module, and an evidence reconciliation module, which are connected in sequence.

3. The method according to claim 1, characterized in that The step 2 specifically includes: Step 2.1: Use the pre-trained neural network to extract features from the training image data obtained in step 1.1; Step 2.2: The features obtained in step 2.1 are passed through the evidence generation module to generate original evidence; Step 2.3: The original evidence obtained in step 2.2 is reconciled and corrected by the evidence reconciliation module to obtain a reconciled evidence representation; Step 2.4: Generate the Dirichlet distribution using the harmonic evidence obtained in step 2.3; Step 2.5: Calculate the harmonic dual gamma evidence loss as the evaluation index based on the category labels in the image dataset in step 1; Step 2.6: Freeze the pre-trained neural network parameters and train the evidence generation module and evidence reconciliation module.

4. The method according to claim 1, characterized in that The step 3 specifically includes: Step 3.1: Input the test data into the trained evidence reconciliation neural network to obtain the reconciliation evidence and the corresponding Dirichlet distribution; Step 3.2: Derive the classification prediction and uncertainty measure of the test sample from the Dirichlet distribution in step 3.1; Step 3.3: Compare the preset threshold with the uncertainty measure value in step 3.2 to determine whether the sample is out-of-distribution.

5. The method according to claim 4, characterized in that The step 3.3 specifically includes: Step 3.3.1: If the uncertainty measure value is greater than the threshold, it is determined to be an out-of-distribution sample and medical experts are requested to intervene; Step 3.3.2: If the uncertainty measure value is less than the threshold, it is judged as an in-distribution sample and the classification prediction is output.

Images