Evidence harmonization-based uncertainty medical image distribution out-of-distribution detection method

Abstract

The present application relates to the field of computer medical image processing, and proposes an out-of-distribution data detection method for uncertain medical images based on evidence reconciliation. The method comprises the following steps: feature extraction is performed on the input medical image to generate a specific evidence representation, a corresponding prediction result and an uncertainty measure are derived, and out-of-distribution samples are identified. The present application is easy to implement, corrects the evidence obtained in the traditional method, improves the detection capability of out-of-distribution samples, and enhances the credibility of medical auxiliary diagnosis. Using the present application can effectively reduce the misdiagnosis risk caused by incomplete training data in medical auxiliary diagnosis, and does not affect the computational complexity of the original network. Future application: alleviate the overconfidence problem in medical auxiliary diagnosis, and provide reliable basis for doctor diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer medical image processing. BACKGROUND

[0002] In traditional classification tasks, models are assumed to be i.i.d. (Independent Identically Distributed) between training and testing data, i.e., training and testing data are independent and have the same feature distribution. Under this assumption, the model fitted to the training set sample features can achieve similar performance on the test set, but this assumption is very idealized and greatly simplifies the algorithm's requirements for robustness. In various practical applications, the mismatch between the model's performance on the training set and the test set has shown that this assumption is unreliable in most cases. In particular, in medical auxiliary diagnosis, the feature distribution shift between training data and testing data is likely to lead to misdiagnosis of diseases and pose a great risk to patients. In recent years, deep neural networks have been widely used in various medical applications, such as classification and segmentation tasks, achieving the most advanced performance. However, given the potential risks caused by the overconfidence of traditional neural networks in predicting results, deep learning methods with uncertainty estimation have become increasingly important because they can guarantee the stability of the model in the presence of out-of-distribution (OOD) test samples.

[0003] Out-of-distribution detection methods based on uncertainty estimation, such as evidence deep learning and its variants, have shown their advantages in computational performance, efficiency, and scalability. In this type of method, higher uncertainty represents a greater difference between the test sample and the training data, and it is more likely to belong to the out-of-distribution sample. However, most of these studies consider identifying abnormal samples that have significant differences with training samples, such as collecting natural images as out-of-distribution samples of medical image data. These methods ignore the inherent near-out-of-distribution (Near OOD) problem in medical images. In this case, due to the imperfection of training data, there may be near-out-of-distribution samples in the test data that do not belong to any training class. Misdiagnosis of these near-out-of-distribution samples will lead to incorrect treatment plans in clinical applications and pose a high risk. Recently, some near-out-of-distribution sample detection methods based on density models, preprocessing, and outlier exposure have been proposed. However, these methods are easily affected by the quality of the training set and have problems such as high computational complexity, convergence difficulty, and unreliable reliability in practical applications. SUMMARY

[0004] The purpose of the present application is to overcome the problems of strong dependence on training data completeness in prior art methods, overconfidence of deep learning models on diagnostic results, and inaccurate uncertainty measurement, and to propose an uncertainty medical image out-of-distribution data detection method based on evidence reconciliation under the framework of evidence neural network (Evidential Neural Network). The auxiliary diagnosis method mainly includes two steps: reconciliation evidence generation, classification prediction and out-of-distribution detection.

[0005] The technical scheme of the present application is as follows:

[0006] The uncertainty medical image out-of-distribution data detection method based on evidence reconciliation comprises the following steps:

[0007] Step 1: data set and model preparation:

[0008] 1.1 Collect images and corresponding class labels from hospitals and online public data sets, and construct a classification image data set after preprocessing, data cleaning and sorting, including training data and test data.

[0009] 1.2 Collect pre-trained neural networks trained on natural image data sets for image feature extraction.

[0010] 1.3 Construct an evidence reconciliation neural network, which includes a pre-trained neural network, an evidence generation module and an evidence reconciliation module, which are connected in sequence.

[0011] Step 2: training and optimization:

[0012] 2.1 Extract features from the training image data obtained in step 1.1 using the pre-trained neural network.

[0013] 2.2 Generate original evidence from the features obtained in step 2.1 using the evidence generation module.

[0014] 2.3 Correct the original evidence obtained in step 2.2 by the evidence reconciliation module to obtain the reconciled evidence representation.

[0015] 2.4 Generate Dirichlet distribution using the reconciled evidence obtained in step 2.3.

[0016] 2.5 Calculate the reconciled double gamma evidence loss as an evaluation index by combining the class labels in the image data set in step 1.

[0017] 2.6 Freeze the pre-trained neural network parameters and train the evidence generation module and the evidence reconciliation module.

[0018] Step 3: image class prediction and out-of-distribution detection for test data:

[0019] 3.1 Input the test data into the trained evidence harmonization neural network to obtain the harmonized evidence and the corresponding Dirichlet distribution.

[0020] 3.2 Derive the classification prediction and uncertainty measure of the test sample from the Dirichlet distribution in step 3.1.

[0021] 3.3 Compare the preset threshold value with the size relationship of the uncertainty measure value in step 3.2 to determine whether the sample belongs to the out-of-distribution sample.

[0022] 3.3.1 If the uncertainty measure value is greater than the threshold value, the out-of-distribution sample is determined, and the medical expert is requested to intervene.

[0023] 3.3.2 If the uncertainty measure value is less than the threshold value, the in-distribution sample is determined, and the classification prediction is output.

[0024] The beneficial effects of the present application are:

[0025] The uncertainty medical image out-of-distribution data detection method based on evidence harmonization provided by the present application corrects the evidence obtained in the traditional uncertainty evidence method, improves the detection capability of out-of-distribution samples, and enhances the credibility of medical auxiliary diagnosis. The uncertainty measure in the present application can be used as a decision basis to effectively reduce the misdiagnosis risk caused by incomplete training data in medical auxiliary diagnosis. The calibration of the evidence can effectively improve the recognition effect of the model on the out-of-distribution samples, and as a medical auxiliary diagnosis scheme, it can provide reliable auxiliary basis for the subsequent treatment of doctors. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is the general flowchart of the method of the present application.

[0027] Figure 2 is the flowchart of the training optimization implementation of the present application.

[0028] Figure 3 is the flowchart of the image category prediction and out-of-distribution detection implementation of the present application.

[0029] Figure 4 is the comparison schematic diagram of the network workflow of the present application and the traditional method.

[0030] Fig. 5(a) is a schematic diagram of the network training of the embodiment of the present application.

[0031] Fig. 5(b) is a schematic diagram of the embodiment of the present application outputting the category prediction for the real in-distribution sample.

[0032] Fig. 5(c) is a schematic diagram of the embodiment of the present application requesting expert intervention for the real out-of-distribution sample. DETAILED DESCRIPTION

[0033] Evidential Neural Network is based on an intuitive fact that the prediction of a classification problem should obey a Categorical Distribution, so the Conjugate Prior of the Categorical Distribution, Dirichlet Distribution, can be used to model the Categorical Distribution. This method uses the network output evidence to generate Dirichlet Distribution, and derives the classification prediction and the corresponding uncertainty based on Bayesian theory.

[0034] According to Subjective Logic, out-of-distribution detection should be regarded as a Non-frequentist Situation, in which the evidence supporting different categories in the evidence neural network output should be offset to correct the uncertainty, so as to achieve more effective detection of out-of-distribution samples.

[0035] The present application proposes an out-of-distribution data detection method for uncertain medical images based on evidence reconciliation under the framework of an evidence neural network. The method extracts features from input medical images and generates specific evidence representations, derives corresponding prediction results and uncertainty measures, and identifies out-of-distribution samples.

[0036] The technical solutions provided by the present application will be further described below with reference to specific embodiments and their accompanying drawings. The advantages and features of the present application will become clearer in combination with the following description.

[0037] As shown in Figure 1 , the out-of-distribution data detection method for uncertain medical images based on evidence reconciliation has the following workflow:

[0038] Step 1, data set preparation, the specific implementation process is as follows:

[0039] 1.1 Collect image data X and corresponding category label Y from hospitals and public data sets to make a data set, denoted as D = {X, Y}.

[0040] 1.2 Collect a pre-trained neural network trained on a natural image data set.

[0041] 1.3 Construct an evidence reconciliation neural network, which is composed of a pre-trained neural network, an evidence generation module, and an evidence reconciliation module in sequence.

[0042] Step 2, training and optimization, the specific process is as shown in Figure 2 , the specific implementation process is as follows:

[0043] 2.1 Freeze the pre-trained neural network parameters, sample (x, y) e D from the dataset D, input the image data x into the pre-trained neural network to extract features, and obtain the features z.

[0044] 2.2 Input the features z into the evidence generation module. The original evidence The non-negativity of the evidence is limited by the softplus activation function. The softplus function is as follows:

[0045]

[0046] 2.3 According to the non-frequency assumption in subjective logic, the evidences of different categories should cancel each other out to eliminate the influence on uncertainty. The present application is based on this to correct the original evidence by harmonization. The specific operation of the evidence harmonization module is as follows:

[0047] 2.3.1 Based on the non-frequency assumption, first calculate the maximum harmonizable evidence value

[0048]

[0049] 2.3.2 Calculate the corrected harmonized evidence representation e:

[0050]

[0051] 2.4 Generate Dir(μ|α) according to the harmonized evidence in 2.3, assuming that there are K categories in the training set, and the probability density can be represented as:

[0052]

[0053] where μ is a random variable of Dirichlet distribution, and e = (e1, e2, …, e K ) is the harmonized evidence in vector form.

[0054] 2.5 Combine the Dirichlet distribution parameter α with the category label y in step 1 dataset, calculate the harmonized double gamma evidence loss as an evaluation index, and use it to train the evidence harmonization neural network. The calculation formula of the harmonized double gamma evidence loss L rece is as follows:

[0055]

[0056] where ψ(·) is the digamma function, and α = (α1, α2, …, α K ) is the Dirichlet distribution parameter in vector form. ​

[0057] 2.6 Freeze the pre-trained neural network parameters, train the evidence generation module and the evidence reconciliation module.

[0058] Step 3, image category prediction and out-of-distribution detection for test data, the process is as shown in Figure 3 , and the specific implementation process is as follows:

[0059] 3.1 Input the test image into the evidence reconciliation neural network trained in step 2 to obtain the reconciled evidence and the corresponding Dirichlet distribution Dir(mu|a).

[0060] 3.2 Derive the classification prediction p(mu) and the uncertainty measure ent from the Dirichlet distribution.

[0061]

[0062] ent = -p(mu)logp(mu)

[0063] 3.3 Compare the uncertainty measure ent with the threshold τ and make a decision.

[0064] 3.3.1 If ent>τ, determine that the sample is an out-of-distribution sample and does not belong to any class in the training set, and the doctor should be prompted to intervene and make further decisions.

[0065] 3.3.2 If ent≤τ, determine that the sample is an in-distribution sample, and use the classification prediction p as the sample prediction result output.

[0066] The comparison between the present application and the conventional method is as shown in Figure 4 :

[0067] The blue arrow represents the data flow of the sample in the conventional evidence neural network. After the sample is input, it passes through the pre-trained neural network and the evidence generation module in turn to obtain the feature vector and the original evidence. Finally, the Dirichlet distribution is generated for subsequent processing.

[0068] The yellow arrow represents the data flow of the sample in the evidence reconciliation neural network of the present application. The sample passes through the pre-trained neural network to obtain the feature vector, and then passes through the evidence generation module to obtain the original evidence. In the evidence reconciliation module, the present application is based on the subjective logic theory, and converts the original evidence into reconciled opinions and derives the corresponding reconciled evidence representation. Finally, the Dirichlet distribution is generated for subsequent processing.

[0069] Among them, the blue dashed arrow represents the data flow of the out-of-distribution sample in the conventional evidence neural network, and the yellow dashed arrow represents the data flow of the out-of-distribution sample in the reconciled evidence neural network of the present application, both of which generate Dirichlet distribution close to uniform distribution.

[0070] Wherein, the blue solid line arrow represents the data flow of the in-distribution sample in the traditional evidence neural network, and the yellow solid line arrow represents the data flow of the in-distribution sample in the evidence harmonization neural network of the application. The Dirichlet distribution generated by the evidence harmonization neural network of the application can derive more accurate prediction results than the traditional evidence neural network, increase the output difference between the in-distribution sample and the out-of-distribution sample, and thus achieve better out-of-distribution data detection capability.

[0071] Embodiment

[0072] During training, as shown in FIG. 5(a), the medical image data in the collected data set needs to be input into the evidence harmonization network to model and constrain the feature generation process thereof. The evidence harmonization neural network is composed of a pre-trained neural network, an evidence generation module, and an evidence harmonization module.

[0073] First, the image data is input into the pre-trained neural network to obtain the corresponding feature vector thereof. The feature can reflect the gray scale, texture and other characteristic information of the image to some extent, and provide effective support for classification and diagnosis. Subsequently, the feature vector is input into the evidence generation module to generate the original evidence value corresponding to each class. Then, the value is input into the harmonization module to obtain the harmonized evidence. Finally, the Dirichlet distribution corresponding to the harmonized evidence is calculated, and the double gamma evidence loss is calculated in combination with the data label to update the network parameters. The above operations are repeated for several times until the network training is completed.

[0074] In the test stage, as shown in FIG. 5(b) and FIG. 5(c), the test sample is input into the evidence harmonization network to output the corresponding harmonized evidence representation. Subsequently, the Dirichlet distribution corresponding to the harmonized evidence is calculated to derive the corresponding prediction value and uncertainty measure. In FIG. 5(b), the test sample has a certainty lower than a given threshold, so the classification result is directly output, and in FIG. 5(c), the test sample has a certainty higher than the given threshold, so the doctor is requested to intervene.

[0075] The above description is only a description of the preferred embodiments of the application, and does not limit the scope of the application in any way. Any modification or modification made by any person skilled in the art according to the above disclosed technical content shall be regarded as an equivalent effective embodiment, and shall fall within the scope of the technical scheme protected by the application.

Claims

1. A method for detection of out-of-distribution data in uncertain medical images based on evidence reconciliation, characterized in that, The method comprises the steps of: Step 1: data set, model preparation; Step 2: training optimization; Step 3: image category prediction and out-of-distribution detection for test data; The step 1, specifically includes: Step 1.1: collect images and corresponding category labels, and construct a classification image data set after preprocessing, data cleaning and sorting, including training data and test data; Step 1.2: collect a pre-trained neural network trained on a natural image data set for image feature extraction; Step 1.3: build an evidence reconciliation neural network, which includes a pre-trained neural network, an evidence generation module and an evidence reconciliation module, which are connected in sequence; The step 2, specifically includes: Step 2.1: using the pre-trained neural network to extract features from the training image data obtained in step 1.1; Step 2.2: generating original evidence by the evidence generation module from the features obtained in step 2.1; Step 2.3: correcting the original evidence obtained in step 2.2 by the evidence reconciliation module to obtain reconciled evidence representation; Step 2.4: generating Dirichlet distribution by the reconciled evidence obtained in step 2.3; Step 2.5: calculating the reconciled double gamma evidence loss as an evaluation index by combining the category labels in the image data set in step 1; Step 2.6: freezing the pre-trained neural network parameters, and training the evidence generation module and the evidence reconciliation module; The step 3, specifically includes: Step 3.1: inputting the test data into the trained evidence reconciliation neural network to obtain reconciled evidence and corresponding Dirichlet distribution; Step 3.2: deriving the classification prediction and uncertainty measure of the test sample from the Dirichlet distribution in step 3.1; Step 3.3: comparing the size relationship between the preset threshold and the uncertainty measure value in step 3.2 to determine whether the sample belongs to out-of-distribution sample.

2. The method of claim 1, wherein, The step 3.3, specifically includes: Step 3.3.1: if the uncertainty measure value is greater than the threshold, it is determined as out-of-distribution sample, and medical experts are requested to intervene; Step 3.3.2: if the uncertainty measure value is less than the threshold, it is determined as in-distribution sample, and the classification prediction is output.

Images