Out-of-distribution data detection method for image classification based on uncertainty in deep learning based on super-opinion evidence

Abstract

The present invention relates to the field of computer image processing, and more specifically to a method for detecting out-of-distribution data in image classification based on uncertainty in super-opinion evidence deep learning. The method comprises the following steps: Step 1: Dataset preparation; Step 2: Super-opinion network construction and training optimization; Step 3: Image category prediction and out-of-distribution detection. The present invention innovatively expands and improves the uncertainty method framework of traditional evidence theory, capable of extracting common evidence from images that is ignored by traditional evidence deep learning, extracting evidence more comprehensively, and making full use of the rich similar features in complex images, thereby significantly improving the ability to detect out-of-distribution data, enhancing the reliability of machine learning-assisted applications, and significantly improving the model's recognition effect on out-of-distribution data.

Description

Technical Field

[0001] The present invention relates to the field of computer image processing, and in particular to a method for detecting out-of-distribution data in image classification based on super-opinion evidence deep learning uncertainty. Background Art

[0002] Image classification, a core research area in computer vision, aims to accurately classify a given image into different categories. This technology plays a key role in numerous practical applications. However, current image classification methods rely primarily on limited training data to learn feature representations of object categories, which limits their performance when faced with categories not previously seen in the training data—the so-called out-of-distribution detection problem.

[0003] This problem is particularly prevalent and challenging in real-world applications, as the number of object categories is both large and diverse, and collecting sufficient data for all possible categories is difficult. Therefore, improving the generalization capabilities of object detection models to effectively handle out-of-distribution data that deviates from the training distribution has become a challenging and practical research topic. This problem is particularly crucial in high-risk fields such as medical image classification.

[0004] Some researchers have proposed out-of-distribution detection methods based on density models, preprocessing techniques, and outlier exposure strategies. However, the performance of these methods is easily affected by the quality of the training data and may require additional data for correction. Furthermore, these methods face challenges such as high computational complexity, poor convergence, and weak transferability, which limit their robustness in practical applications. Therefore, despite their theoretical appeal, their effectiveness and reliability in practical applications still require further verification and improvement.

[0005] Uncertainty estimation methods based on deep learning, such as evidence-based deep learning and its derivative models, have demonstrated significant advantages in computational performance, efficiency, and scalability. However, existing out-of-distribution detection methods based on evidence-based deep learning all construct evidence within a polynomial opinion framework, which can only extract sharp evidence for each category while failing to fully consider the ambiguous evidence shared across multiple categories. This limitation makes it difficult for these methods to cope with the challenges posed by high-similarity images in complex images. In complex images with high similarity, a large amount of shared ambiguous evidence is ignored, significantly reducing the model's confidence in the in-distribution data.

[0006] In image category prediction and out-of-distribution detection, misjudgment of in-distribution samples may lead to incorrect decisions in high-risk applications (such as auxiliary diagnosis in the medical field), bringing extremely high risks. Summary of the Invention

[0007] Given the limitations of existing technologies, this paper discloses a method for detecting out-of-distribution data in image classification based on the uncertainty of deep learning with super-opinion evidence. This method aims to address several key issues in the field of auxiliary applications: a strong reliance on large amounts of training data and additional data, overconfidence in the results of auxiliary applications by deep learning models, and the inability of deep learning with evidence to fully utilize the abundant similar information in images. To this end, a method for detecting out-of-distribution data in image classification based on the uncertainty of deep learning with super-opinion evidence is proposed.

[0008] Technical solution of the present invention:

[0009] A method for out-of-distribution data detection in image classification based on uncertainty in deep learning with super-opinion evidence includes the following steps:

[0010] Step 1: Dataset preparation:

[0011] A large amount of image data and corresponding category labels are collected from online public datasets. After data cleaning and data preprocessing, an image classification dataset D is constructed with K sample categories. The sample in the dataset is (x, y)∈D, where image data x and category label y.

[0012] Step 2: Super opinion network construction and training optimization;

[0013] The super opinion network includes: a feature extractor, a super opinion evidence generation module and an opinion projection module;

[0014] The feature extractor is pre-trained on a large general dataset. The image data is fed into the feature extractor to obtain a feature vector reflecting the image's grayscale, texture, and other feature information.

[0015] The feature vector is input into the super-opinion evidence generation module, which generates a super-opinion evidence value corresponding to each composite set and generates the corresponding super-opinion belief;

[0016] Then, these super-opinion beliefs and the corresponding aggregate information are fed into the opinion projection module and converted into projected polynomial opinion beliefs for each category;

[0017] The expected classification probability obtained from the Dirichlet distribution is used as the network prediction, and the bi-gamma evidence loss ψ(·) is calculated with the classification label y in the dataset as the evaluation metric. The entire super-opinion network is trained until the network training reaches the expected stable state.

[0018] Step 3: Image category prediction and out-of-distribution detection;

[0019] The images to be predicted are sequentially input into the optimal super-opinion network trained in step 2 to obtain the Dirichlet distribution parameters of the polynomial opinions. The test sample classification prediction p and uncertainty measure u are obtained through the Dirichlet distribution. Given an uncertainty threshold τ, the uncertainty measure u is compared with τ as follows:

[0020] If u>τ, it is determined that the sample does not belong to any category in the training set and is out-of-distribution data, and experts should be reminded to intervene for further decision-making;

[0021] If u≤τ, the classification prediction p is used as the sample prediction result output.

[0022] Beneficial effects:

[0023] The out-of-distribution data detection method for image classification based on the uncertainty of super-opinion evidence deep learning proposed in this paper innovatively expands and improves the uncertainty method framework of traditional evidence theory. This method can extract common evidence of images that is ignored by traditional evidence deep learning, extract evidence more comprehensively, and make full use of the rich similar features in complex images, thereby significantly improving the detection ability of out-of-distribution data and enhancing the reliability of machine learning-assisted applications.

[0024] The uncertainty measurement technique proposed in this paper can effectively reduce the risk of misclassifying out-of-distribution data as known categories due to incomplete training data in machine learning-assisted applications. By extracting more comprehensive evidence, this method can significantly improve the model's recognition of out-of-distribution data. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the overall flow chart of the present invention;

[0026] Figure 2 It is a detailed flow chart of step 2 of the present invention;

[0027] Figure 3 It is a detailed flow chart of step 3 of the present invention;

[0028] Figure 4 It is an overview of the module structure and workflow of the present invention;

[0029] Figure 5 This is a schematic diagram of the training process of an embodiment of the present invention on real data in the field of medical skin disease classification and detection;

[0030] Figure 6 This is a diagram showing the predicted classification results of an embodiment of the present invention based on real data in the field of medical skin disease classification detection;

[0031] Figure 72. It is a schematic diagram of detecting out-of-distribution data using uncertainty measurement on real data in the field of medical skin disease classification detection according to an embodiment of the present invention;

[0032] Figure 8 This is a schematic diagram comparing the present invention with traditional evidence deep learning methods in actual classification detection examples;

[0033] Figure 9 This is a comparison chart of the effects of the embodiment of the present invention and other methods. DETAILED DESCRIPTION

[0034] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Products that can achieve the same functions are equivalent replacements and improvements and are all included in the scope of protection of the present invention.

[0035] Figure 1 This is the overall flow chart (steps 1, 2, and 3) of the present invention for classifying medical images and measuring their uncertainty to determine whether they are out-of-distribution samples.

[0036] Figure 2 It is a detailed flow chart of step 2 (implementing the training optimization process) of the present invention.

[0037] Figure 3 This is a flowchart for implementing step 3 (in terms of image category prediction and out-of-distribution detection) of the present invention.

[0038] A method for detecting out-of-distribution data in image classification based on uncertainty in deep learning of super-opinion evidence includes the following steps: (e.g. Figure 1 )

[0039] Step 1: Dataset preparation:

[0040] A large amount of image data and corresponding category labels are collected from online public datasets. After data cleaning and data preprocessing, an image classification dataset D is constructed with K sample categories. The sample in the dataset is (x, y)∈D, where image data x and classification label y.

[0041] Step 2: Super opinion network construction and training optimization (such as Figure 2 )

[0042] The super-opinion network includes: a feature extractor, a super-opinion evidence generation module, and an opinion projection module; wherein the feature extractor is a prior art technology obtained by pre-training on a large general data set, and image data is fed into the feature extractor to obtain feature vectors reflecting feature information such as image grayscale and texture; these feature vectors are input into the super-opinion evidence generation module to generate super-opinion evidence values ​​corresponding to each composite set and generate corresponding super-opinion beliefs; then, these super-opinion beliefs and corresponding set information are fed into the opinion projection module, converted into projected polynomial opinion beliefs for each category, and bijected into Dirichlet distribution parameters corresponding to the polynomial opinions.

[0043] It should be noted that the concepts of polynomial opinion and super-opinion originate from the theory of subjective logic. Traditional evidence-based deep learning uses polynomial opinion to model evidence, but the resulting evidence is incomplete and incomplete, leading to inaccurate uncertainty estimates. This invention uses super-opinion to model more comprehensive evidence, resulting in more accurate uncertainty estimates and, in turn, better out-of-distribution data detection.

[0044] The super opinion network training and optimization process includes the following steps:

[0045] 2.1 Use the feature extractor to extract the features of the image data obtained in step 1.

[0046] Take a sample (x, y)∈D from the dataset, input the image data x into the pre-trained feature extractor for feature extraction, and obtain the feature f.

[0047] 2.2 The feature f is input into the super-opinion evidence generation module to obtain super-evidence e H :

[0048] e H =ReLU(f)

[0049] The non-negativity of evidence is ensured by the ReLU activation function, where the ReLU function is:

[0050]

[0051] 2.3 Super Opinion Evidence Generation Module further models the non-negative super opinion evidence in 2.2 to obtain super opinion, where super opinion consists of a triple: prior probability a H , super opinion belief b H and uncertainty u; the modeling method is as follows:

[0052]

[0053] The calculation method of S is as follows:

[0054]

[0055] Among them, K is the number of sample categories, is the dataset category domain, The superdomain corresponding to the dataset category domain (i.e., including the domain all non-empty proper subsets of );

[0056] 2.4 Opinion Projection Module H Perform opinion projection and output the parameter α of the Dirichlet distribution Dir of the multinomial opinion.

[0057] The opinion projection module includes a projection coefficient calculation module and a polynomial opinion processing module, wherein the projection coefficient calculation module includes a fully connected neural network layer for projecting super opinions into polynomial opinions (as in steps 2.4.1 and 2.4.2); the polynomial opinion processing module is used to biject the polynomial opinions into corresponding Dirichlet distribution parameters α (as in step 2.4.3);

[0058] The specific operations are as follows:

[0059] 2.4.1 Super-opinion belief b H Input the projection coefficient calculation module, W is the fully connected layer parameter of the projection coefficient calculation module, and calculate the matrix W corresponding to the composite set category of each super opinion H :

[0060]

[0061] 2.4.2 Based on the lack of prior knowledge of relative basic probability, the super opinion b H Perform the average distribution to obtain the polynomial opinion belief b:

[0062]

[0063] The polynomial opinion thus obtained is composed of a triplet of prior probability a, super-opinion belief b and uncertainty u, where a=1.

[0064] It should be noted that according to the projection of super opinions to polynomial opinions in subjective logic, each super opinion should be assigned to its corresponding category polynomial opinion according to the relative basic probability. In the absence of prior knowledge of the relative basic probability, it is represented by an average distribution.

[0065] 2.4.3 The polynomial opinion processing module bijects the polynomial opinion into the Dirichlet distribution parameter α corresponding to the polynomial opinion according to the following formula:

[0066]

[0067] αk =b k S+a k

[0068] Where B(α) is the K-dimensional polynomial beta function, K is the number of sample categories, p i is the probability of the category corresponding to the i-th parameter derived from the Dirichlet distribution;

[0069] 2.5 Use the expected predicted classification probability obtained by Dirichlet distribution as the network prediction, and calculate the double gamma evidence loss ψ(·) as the evaluation indicator with the classification label y in the dataset in step 1 to train the entire super opinion network mentioned above.

[0070] The loss function is calculated as:

[0071]

[0072] Repeat the above steps until the network training reaches the expected stable state.

[0073] Step 3 Image category prediction and out-of-distribution detection: (e.g. Figure 3 )

[0074] 3.1 Input the images to be predicted into the optimal super-opinion network trained in step 2 in sequence to obtain the Dirichlet distribution parameters of the polynomial opinions.

[0075] 3.2 Obtain the test sample classification prediction p and uncertainty measure u through Dirichlet distribution.

[0076]

[0077] 3.3 Given an uncertainty threshold τ, where τ ranges from 0 to 1 and is manually selected based on the task and data, compare the uncertainty measure u with τ as follows:

[0078] 3.3.1 If u > τ, the sample is determined to not belong to any class in the training set and is considered out-of-distribution data. An expert (for example, a doctor) should be alerted to intervene for further decision-making.

[0079] 3.3.2 If u≤τ, use the classification prediction p as the sample prediction result output.

[0080] Figure 4 This is the overall model framework diagram. The upper part is the main model framework, and the lower two parts are the specific module details. In step 2.1, the data is processed through the feature extractor to obtain features. Then, in step 2.2, it is modeled as hyper-evidence. In step 2.3, hyper-opinions are generated. In step 2.4, it is projected into polynomial opinions and corresponding Dirichlet distribution parameters. In step 2.5, the predicted probability and uncertainty are obtained, and they are optimized using the loss function.

[0081] Among them, steps 2.2 and 2.3 are the super-opinion evidence generation module (in the super-opinion evidence generation module, the features are first activated as super-opinion evidence, and then super-opinion beliefs are generated based on the super-opinion evidence), step 2.4 is the opinion projection module (super-opinion beliefs are projected into polynomial opinions and corresponding Dirichlet distribution parameters), and step 2.5 is the model optimization (calculating the loss and then backpropagating to optimize the entire super-opinion network).

[0082] Training phase:

[0083] like Figure 5 As shown in Figure 3, the training process begins by inputting the collected image dataset into the Super Opinion Network. The Super Opinion Network consists of three core modules: feature extractor, super opinion evidence generation module, and opinion projection module.

[0084] First, the image data is fed into a feature extractor to obtain feature vectors reflecting image grayscale, texture, and other characteristic information. This feature information provides strong support for subsequent classification and diagnosis. These feature vectors are then fed into a super-opinion evidence generation module to generate super-opinion evidence values ​​corresponding to each composite set and generate corresponding super-opinion beliefs. These super-opinion beliefs and the corresponding set information are then fed into an opinion projection module, where they are converted into projected polynomial opinion beliefs for each class. Finally, based on the Dirichlet distribution corresponding to the polynomial opinions and combined with the true labels of the data, a bi-gamma evidence loss function is calculated, and the network parameters are updated accordingly. This process is repeated until the network training reaches the desired stable state.

[0085] Testing phase:

[0086] like Figure 6 and Figure 7 As shown in , the test sample is input into the super opinion network and its corresponding super opinion evidence representation is output. Then, the Dirichlet distribution corresponding to the multinomial opinion evidence after opinion projection is calculated, and the corresponding prediction classification results and uncertainty measures are derived. Figure 6 In , the uncertainty of the test sample is lower than the set threshold, so the classification result is directly output. Figure 7 In the case of a test sample, if the uncertainty of the test sample exceeds the set threshold, it indicates that the model's classification result for the sample is not certain, and a doctor is recommended to intervene for further diagnosis. This process ensures that the model can take appropriate measures when faced with high uncertainty, thereby improving the safety and reliability of deep learning models in practical applications.

[0087] Figure 8 This is a schematic diagram comparing the present invention with traditional evidence deep learning methods in actual classification detection samples.

[0088] Figure 9This is a table comparing the effects of the present invention with other methods in actual classification and out-of-distribution detection data set census. The actual classification and out-of-distribution detection data set census are compared with other methods. ↓ indicates that the lower the value, the better, and ↑ indicates that the higher the value, the better.

[0089] The comparison methods include: classic methods MSP (maximum class probability), ODIN (out-of-distribution detector), MCDropout (Monte Carlo Dropout); some current mainstream methods, such as openGAN, GradNorm, VIM, KNN, DICE, RankFeat, ASH, SHE, GEN, G-ODIN, CSI, MOS, VOS, LogitNorm, as well as the traditional evidence deep learning method EDL and its variant RED.

[0090] Three indicators are used as out-of-distribution data detection indicators, namely:

[0091] 1) FPR95: The false positive rate (FPR) when the true positive rate (TPR) is equal to 95%. The lower the score, the better the performance.

[0092] 2) AUROC (area under the Receiver Operating Characteristic (ROC) curve): The area under the ROC curve can be interpreted as the probability that a positive ID sample has a higher detection score than a negative OOD sample. The larger the AUROC value, the better the performance.

[0093] 3) AUPR (area under the Precision-Recall (PR) curve): The PR curve is drawn by plotting the relationship between precision and recall. The larger the AUPR value, the better the performance.

[0094] AUROC is the most common metric used as the primary measure of OOD detection performance, while accuracy measures the performance of detecting ID samples. In-distribution classification accuracy is also used to measure the model's classification accuracy. The goal is to detect more OOD samples while maintaining ID classification performance. Compared with various methods, experimental results show that the proposed method improves the detection capability of out-of-distribution data while maintaining classification accuracy, surpassing current mainstream methods.

[0095] The above description is only a description of the preferred embodiments of the present application and does not limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical content should be regarded as equivalent valid embodiments and fall within the scope of protection of the technical solution of the present application.

Claims

1. A method for detecting out-of-distribution data in image classification based on uncertainty in deep learning with super-opinion evidence, characterized in that: The following steps are involved: Step 1: Dataset preparation: Collect a large amount of image data and corresponding category labels from online public datasets, and build an image classification dataset after data cleaning and data preprocessing. , the number of sample categories is ; Samples in the dataset , where image data Category Tags ; Step 2: Super opinion network construction and training optimization; The super opinion network includes: a feature extractor, a super opinion evidence generation module and an opinion projection module; The feature extractor is pre-trained on a large general dataset. The image data is fed into the feature extractor to obtain a feature vector reflecting the image's grayscale, texture, and other feature information. The feature vector is input into the super-opinion evidence generation module, which generates a super-opinion evidence value corresponding to each composite set and generates the corresponding super-opinion belief; Then, these super-opinion beliefs and the corresponding set information are fed into the opinion projection module, converted into the projected polynomial opinion beliefs of each category, and bijected into the Dirichlet distribution parameters corresponding to the polynomial opinions; The expected classification probability obtained using the Dirichlet distribution is used as the network prediction, and the classification label in the dataset is Computing dual-gamma evidence loss As an evaluation indicator, the above super opinion network is trained until the network training reaches the expected stable state; Step 3: Image category prediction and out-of-distribution detection; The images to be predicted are sequentially input into the optimal super-opinion network trained in step 2 to obtain the Dirichlet distribution parameters of the polynomial opinions; the classification prediction of the test samples is obtained through the Dirichlet distribution. , uncertainty ; Given an uncertainty threshold , comparative uncertainty and ,as follows: like , then it is determined that the sample does not belong to any category in the training set and is out-of-distribution data, and experts should be reminded to intervene for further decision-making; like , then use classification prediction Output as sample prediction result; In step 2, the super opinion network training optimization process includes the following steps: 2.1 Use the feature extractor to extract the features of the image data obtained in step 1; Take samples from the dataset , the image data Input the feature extractor in the pre-trained feature extractor to extract the features. ; 2.2 Characteristics Input the super-opinion evidence generation module to obtain super-evidence ; 2.3 Super Opinion Evidence Generation Module further models the super opinion evidence in 2.2 to obtain super opinion, where super opinion is composed of triples: prior probability , super opinion belief and uncertainty Composition; the modeling method is as follows: The calculation method is as follows: in, is the number of sample categories, is the dataset category domain, It is the superdomain corresponding to the category domain of the dataset; 2.4 Opinion Projection Module for the Super Opinion Beliefs Obtained in 2.3 Perform opinion projection and output the Dirichlet distribution of multinomial opinions Parameters ; The opinion projection module includes a projection coefficient calculation module and a polynomial opinion processing module, wherein the projection coefficient calculation module includes a layer of fully connected neural network for projecting super opinions into polynomial opinions; the polynomial opinion processing module is used to biject the polynomial opinions into corresponding Dirichlet distribution parameters. ; The specific operations are as follows: 2.4.1 Super-opinion beliefs Input projection coefficient calculation module, It is the fully connected layer parameter of the projection coefficient calculation module, which calculates the matrix corresponding to the composite set category of each super opinion : 2.4.2 Based on the lack of prior knowledge of relative basic probability, the super-opinion belief Perform average distribution to obtain polynomial opinion beliefs : Thus, a polynomial opinion is obtained; the polynomial opinion is composed of the triple prior probability , super opinion belief and uncertainty Composition, of which ; 2.4.3 The polynomial opinion processing module bijects the polynomial opinion into the Dirichlet distribution parameters corresponding to the polynomial opinion according to the following formula : in yes dimensional polynomial beta function, is the number of sample categories, is the probability of the category corresponding to the i-th parameter derived from the Dirichlet distribution; 2.5 Use the expected predicted classification probability obtained by Dirichlet distribution as the network prediction, and compare it with the classification label in the dataset in step 1 Computing dual-gamma evidence loss As an evaluation metric, the above super opinion network is trained; The loss function is calculated as: Repeat the above steps until the network training reaches the expected stable state.

2. The method according to claim 1, wherein: The super evidence The calculation method is as follows: in is characterized by; The non-negativity of evidence is ensured by the ReLU activation function, where The function is: 。 3. The method according to claim 1, wherein: Step 3: Image category prediction and out-of-distribution detection includes the following steps: 3.1 Input the image to be predicted into the optimal super-opinion network trained in step 2 to obtain the Dirichlet distribution parameters of the polynomial opinion; 3.2 Obtaining test sample classification predictions through Dirichlet distribution , uncertainty measurement ; 3.3 Given uncertainty threshold ,in The value range is 0-1, which is manually selected according to the task and data, and the uncertainty measure is compared. and ,as follows: 3.3.1 If , then it is determined that the sample does not belong to any category in the training set and is out-of-distribution data, and experts should be reminded to intervene for further decision-making; 3.3.2 If , then use classification prediction Output as sample prediction result.

Images