Image classification distribution external data detection method based on super-opinion evidence deep learning uncertainty

Abstract

The invention relates to the field of computer image processing, in particular to an image classification distribution external data detection method based on hyper-opinion evidence deep learning uncertainty. Comprising the following steps: step 1, preparing a data set; step 2, building a hyper-opinion network and carrying out training optimization on the hyper-opinion network; step 3, image category prediction and distribution outside detection; according to the method, an uncertainty method framework of a traditional evidence theory is innovatively expanded and perfected, the common evidence of the image ignored by traditional evidence deep learning can be extracted, the evidence can be extracted more comprehensively, and rich similar features in a complex image are fully utilized, so that the detection capability of out-of-distribution data is remarkably improved, and the detection efficiency of the out-of-distribution data is improved. And the reliability of machine learning auxiliary application is enhanced, and the recognition effect of the model on out-of-distribution data is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer image processing, and in particular to a method for detecting out-of-distribution data in image classification based on super-opinion evidence deep learning uncertainty. Background Art

[0002] Image classification is one of the core research directions in the field of computer vision. Its core task is to accurately classify a given image. This technology plays a key role in many practical applications. However, current image classification methods mainly rely on limited training data to learn the feature representation of object categories, which limits their performance when facing categories that have not appeared in the training data, which is the so-called out-of-distribution data detection problem.

[0003] This problem is particularly common and challenging in real-world application scenarios, because the number of object categories is not only large but also varied, and it is difficult to collect sufficient data for all possible categories. Therefore, how to improve the generalization ability of object detection models so that they can effectively handle out-of-distribution data that deviates from the training distribution has become a challenging and practical research topic. This problem is even more critical in high-risk fields such as medical image classification.

[0004] Some researchers have proposed out-of-distribution detection methods based on density models, preprocessing techniques, and outlier exposure strategies. However, the performance of these methods is easily affected by the quality of training data and may require additional data for correction. In addition, these methods also face problems such as high computational complexity, difficult convergence, and weak migration ability, which limit their robustness in practical applications. Therefore, although these methods are attractive in theory, their effectiveness and reliability in practical applications still need further verification and improvement.

[0005] Uncertainty estimation methods based on deep learning, such as evidence deep learning and its derivative models, have shown significant advantages in computational performance, efficiency, and scalability. However, existing out-of-distribution detection methods based on evidence deep learning all construct evidence within a polynomial opinion framework, which can only extract sharp evidence for each category, but fail to fully consider the fuzzy evidence shared between multiple categories. This limitation makes it difficult for these methods to cope with the challenges posed by high-similarity images in complex images. In complex images with high similarity, a large amount of shared fuzzy evidence is ignored, which significantly reduces the model's confidence in the in-distribution data.

[0006] In image category prediction and out-of-distribution detection, misjudgment of in-distribution samples may lead to wrong decisions in high-risk applications (such as auxiliary diagnosis in the medical field), bringing extremely high risks. Summary of the invention

[0007] In view of the limitations of the prior art, the present invention discloses an out-of-distribution data detection method for image classification based on super-opinion evidence deep learning uncertainty, aiming to solve several key problems in the field of auxiliary applications: strong dependence on a large amount of training data and additional data, overconfidence of deep learning models in auxiliary application results, and inability of evidence deep learning to fully utilize a large amount of similar information in images. To this end, a method for out-of-distribution data detection for image classification based on super-opinion evidence deep learning uncertainty is proposed.

[0008] Technical solution of the present invention:

[0009] A method for out-of-distribution data detection in image classification based on super-opinion evidence deep learning uncertainty includes the following steps:

[0010] Step 1: Dataset preparation:

[0011] A large amount of image data and corresponding category labels are collected from online public data sets. After data cleaning and data preprocessing, an image classification data set D is constructed, and the number of sample categories is K. The sample (x, y)∈D in the data set is image data x and classification label y.

[0012] Step 2: Super opinion network construction and training optimization;

[0013] The super opinion network includes: a feature extractor, a super opinion evidence generation module and an opinion projection module;

[0014] Among them, the feature extractor is obtained by pre-training on a large general data set, and the image data is sent to the feature extractor to obtain a feature vector reflecting the feature information such as image grayscale and texture;

[0015] The feature vector is input into the super-opinion evidence generation module to generate a super-opinion evidence value corresponding to each composite set and generate a corresponding super-opinion belief;

[0016] Then, these super-opinion beliefs and the corresponding set information are fed into the opinion projection module and converted into the projected polynomial opinion beliefs of each category;

[0017] The expected classification probability obtained by Dirichlet distribution is used as the network prediction, and the bi-gamma evidence loss ψ(·) is calculated with the classification label y in the dataset as the evaluation indicator. The entire super-opinion network is trained until the network training reaches the expected stable state.

[0018] Step 3: Image category prediction and out-of-distribution detection;

[0019] The images to be predicted are sequentially input into the optimal super opinion network trained in step 2 to obtain the Dirichlet distribution parameters of the polynomial opinions; the test sample classification prediction p and uncertainty measure u are obtained through the Dirichlet distribution; given the uncertainty threshold τ, the uncertainty measure u is compared with τ as follows:

[0020] If u>τ, it is determined that the sample does not belong to any category in the training set and is out-of-distribution data, and experts should be reminded to intervene for further decision-making;

[0021] If u≤τ, the classification prediction p is used as the sample prediction result output.

[0022] Beneficial effects:

[0023] The out-of-distribution data detection method for image classification based on super-opinion evidence deep learning uncertainty proposed in the present invention innovatively expands and improves the uncertainty method framework of traditional evidence theory. This method can extract common evidence of images ignored by traditional evidence deep learning, extract evidence more comprehensively, and make full use of the rich similar features in complex images, thereby significantly improving the ability to detect out-of-distribution data and enhancing the reliability of machine learning-assisted applications.

[0024] In this invention, the proposed uncertainty measurement technology can effectively reduce the risk of misclassifying out-of-distribution data as known categories due to incomplete training data in the process of machine learning-assisted applications. By extracting evidence more comprehensively, this method can significantly improve the model's recognition effect on out-of-distribution data. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the overall flow chart of the present invention;

[0026] Figure 2 It is a detailed flow chart of step 2 of the present invention;

[0027] Figure 3 It is a detailed flow chart of step 3 of the present invention;

[0028] Figure 4 It is an overview diagram of the module structure and workflow of the present invention;

[0029] Figure 5 It is a schematic diagram of the training process of an embodiment of the present invention on real data in the field of medical skin disease classification detection;

[0030] Figure 6 This is a diagram of the predicted classification results of an embodiment of the present invention on real data in the field of medical skin disease classification detection;

[0031] Figure 7It is a schematic diagram of detecting out-of-distribution data by uncertainty measurement on real data in the field of medical skin disease classification detection according to an embodiment of the present invention;

[0032] Figure 8 It is a schematic diagram comparing the present invention with the traditional evidence deep learning method in actual classification detection examples;

[0033] Fig. 9 It is a comparison diagram of the effects of the embodiment of the present invention and other methods. DETAILED DESCRIPTION

[0034] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is described in detail below in conjunction with the accompanying drawings and embodiments. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Products that can achieve the same functions are equivalent replacements and improvements, and are all included in the protection scope of the present invention.

[0035] Figure 1 This is the overall flow chart (steps 1, 2, and 3) of the present invention for classifying medical images and measuring their uncertainty to determine whether they are out-of-distribution samples.

[0036] Figure 2 It is a detailed flow chart of step 2 (implementing the training optimization process) of the present invention.

[0037] Figure 3 It is an implementation flow chart of step 3 (in terms of image category prediction and out-of-distribution detection) of the present invention.

[0038] A method for detecting out-of-distribution data in image classification based on uncertainty in deep learning based on super-opinion evidence includes the following steps: (e.g. Figure 1 )

[0039] Step 1: Dataset preparation:

[0040] A large amount of image data and corresponding category labels are collected from online public data sets. After data cleaning and data preprocessing, an image classification data set D is constructed with K sample categories. The sample (x, y)∈D in the data set is image data x and classification label y.

[0041] Step 2: Super opinion network construction and training optimization (such as Figure 2 )

[0042] The super-opinion network includes: a feature extractor, a super-opinion evidence generation module and an opinion projection module; wherein the feature extractor is a prior art obtained by pre-training on a large general data set, and the image data is fed into the feature extractor to obtain a feature vector reflecting feature information such as image grayscale and texture; these feature vectors are input into the super-opinion evidence generation module to generate super-opinion evidence values ​​corresponding to each composite set and generate corresponding super-opinion beliefs; then, these super-opinion beliefs and corresponding set information are fed into the opinion projection module, converted into polynomial opinion beliefs of each category after projection, and bijected into Dirichlet distribution parameters corresponding to the polynomial opinions.

[0043] It should be noted that the concepts of polynomial opinions and super opinions come from the theory of subjective logic. Traditional evidence deep learning uses polynomial opinions to model evidence, and the modeled evidence is incomplete and incomplete, resulting in inaccurate uncertainty estimation. The present invention uses super opinions to model more comprehensive evidence, thereby obtaining more accurate uncertainty estimation, and thus obtaining better out-of-distribution data detection results.

[0044] The super opinion network training optimization process includes the following steps:

[0045] 2.1 Use the feature extractor to extract the features of the image data obtained in step 1.

[0046] Take a sample (x, y)∈D from the data set, input the image data x into the pre-trained feature extractor for feature extraction, and obtain the feature f.

[0047] 2.2 The feature f is input into the super-opinion evidence generation module to obtain super-evidence e H :

[0048] e H =ReLU(f)

[0049] The non-negativity of evidence is ensured by the ReLU activation function, where the ReLU function is:

[0050]

[0051] 2.3 Super-opinion evidence generation module further models the non-negative super-opinion evidence in 2.2 to obtain super-opinion, where the super-opinion consists of a triple: prior probability a H , Super Opinion Belief b H and uncertainty u; the modeling method is as follows:

[0052]

[0053] The calculation method of S is as follows:

[0054]

[0055] Among them, K is the number of sample categories, is the dataset category domain, The superdomain corresponding to the dataset category domain (i.e., including the domain all non-empty proper subsets of );

[0056] 2.4 Opinion Projection Module The super opinion belief b obtained in 2.3 H Opinion projection is performed and the output is the parameter α of the Dirichlet distribution Dir of the multinomial opinion.

[0057] The opinion projection module includes a projection coefficient calculation module and a polynomial opinion processing module, wherein the projection coefficient calculation module includes a layer of fully connected neural network for projecting super opinions to polynomial opinions (such as steps 2.4.1 and 2.4.2); the polynomial opinion processing module is used to biject the polynomial opinions into corresponding Dirichlet distribution parameters α (such as step 2.4.3);

[0058] The specific operations are as follows:

[0059] 2.4.1 Super-opinion belief b H Input projection coefficient calculation module, W is the fully connected layer parameter of projection coefficient calculation module, calculate the matrix W corresponding to the composite set category of each super opinion H :

[0060]

[0061] 2.4.2 Based on the lack of prior knowledge of relative basic probability, the super opinion b H Perform average distribution to obtain the polynomial opinion belief b:

[0062]

[0063] The polynomial opinion thus obtained is composed of a triplet of prior probability a, super-opinion belief b and uncertainty u, where a=1.

[0064] It should be noted that according to the projection of super opinions to polynomial opinions in subjective logic, each super opinion should be assigned to its corresponding category polynomial opinion according to the relative basic probability. In the absence of prior knowledge of the relative basic probability, it is represented by an average distribution.

[0065] 2.4.3 The polynomial opinion processing module bijects the polynomial opinion into the Dirichlet distribution parameter α corresponding to the polynomial opinion according to the following formula:

[0066]

[0067] αk =b k S+a k

[0068] Where B(α) is a K-dimensional polynomial beta function, K is the number of sample categories, and p i is the probability of the category corresponding to the i-th parameter derived from the Dirichlet distribution;

[0069] 2.5 Use the expected predicted classification probability obtained by Dirichlet distribution as the network prediction, and calculate the double gamma evidence loss ψ(·) as the evaluation indicator with the classification label y in the dataset in step 1 to train the entire super opinion network mentioned above.

[0070] The loss function is calculated as:

[0071]

[0072] Repeat the above steps until the network training reaches the expected stable state.

[0073] Step 3 Image category prediction and out-of-distribution detection: (e.g. Figure 3 )

[0074] 3.1 Input the images to be predicted into the optimal super opinion network trained in step 2 in sequence to obtain the Dirichlet distribution parameters of the polynomial opinions.

[0075] 3.2 Obtain the test sample classification prediction p and uncertainty measure u through Dirichlet distribution.

[0076]

[0077] 3.3 Given an uncertainty threshold τ, where τ ranges from 0 to 1, manually selected according to the task and data, compare the uncertainty measure u with τ as follows:

[0078] 3.3.1 If u>τ, it is determined that the sample does not belong to any category in the training set and is out-of-distribution data. Experts (such as doctors) should be reminded to intervene for further decision-making.

[0079] 3.3.2 If u≤τ, the classification prediction p is used as the sample prediction result output.

[0080] Figure 4 The overall model framework diagram is shown in the figure. The upper part is the main framework of the model, and the lower two parts are the specific module details. Step 2.1 The data is extracted by the feature extractor to obtain features, and then modeled as hyper-evidence in step 2.2, hyper-opinions are generated in step 2.3, and projected into polynomial opinions and corresponding Dirichlet distribution parameters in step 2.4. Step 2.5 obtains the predicted probability and uncertainty, which are optimized by the loss function.

[0081] Among them, steps 2.2 and 2.3 are the super-opinion evidence generation module (in the super-opinion evidence generation module, the features are first activated as super-opinion evidence, and then super-opinion beliefs are generated based on the super-opinion evidence), step 2.4 is the opinion projection module (the super-opinion beliefs are projected into polynomial opinions and corresponding Dirichlet distribution parameters), and step 2.5 is the model optimization (calculating the loss and then back-propagating to optimize the entire super-opinion network).

[0082] Training phase:

[0083] like Figure 5 As shown, the training process begins by inputting the collected image dataset into the Super Opinion Network. The Super Opinion Network consists of three core modules: feature extractor, super opinion evidence generation module and opinion projection module.

[0084] First, the image data is fed into the feature extractor to obtain feature vectors that reflect the image grayscale, texture and other feature information. These feature information provides strong support for subsequent classification diagnosis. Subsequently, these feature vectors are further input into the super-opinion evidence generation module to generate super-opinion evidence values ​​corresponding to each composite set and generate corresponding super-opinion beliefs. Then, these super-opinion beliefs and the corresponding set information are fed into the opinion projection module and converted into polynomial opinion beliefs for each category after projection. Finally, based on the Dirichlet distribution corresponding to the polynomial opinions and combined with the true labels of the data, the double gamma evidence loss function is calculated, and the network parameters are updated accordingly. The above operations are repeated until the network training reaches the expected stable state.

[0085] Testing phase:

[0086] like Figure 6 and Figure 7 As shown in , the test sample is input into the super-opinion network and its corresponding super-opinion evidence representation is output. Then, the Dirichlet distribution corresponding to the multinomial opinion evidence after opinion projection is calculated, and the corresponding prediction classification results and uncertainty measures are derived. Figure 6 In , the uncertainty of the test sample is lower than the set threshold, so the classification result is directly output. Figure 7 In the test, if the uncertainty of the test sample is higher than the set threshold, it means that the model is not sure about the classification result of the sample, and the doctor is recommended to intervene for further diagnosis. This process ensures that the model can take appropriate measures when faced with high uncertainty, thereby improving the safety and reliability of the practical application of deep learning models.

[0087] Figure 8 It is a schematic diagram comparing the present invention with the traditional evidence deep learning method in actual classification detection samples.

[0088] Fig. 9This is a comparison table of the effects of the present invention with other methods in actual classification and out-of-distribution detection data set census, where ↓ indicates that the lower the value, the better, and ↑ indicates that the higher the value, the better.

[0089] The comparison methods include: classic methods MSP (maximum class probability), ODIN (out-of-distribution detector), MCDropout (Monte Carlo Dropout); some current mainstream methods, such as openGAN, GradNorm, VIM, KNN, DICE, RankFeat, ASH, SHE, GEN, G-ODIN, CSI, MOS, VOS, LogitNorm, and traditional evidence deep learning method EDL and its variant RED.

[0090] Three indicators are used as out-of-distribution data detection indicators, namely:

[0091] 1) FPR95: The value of the false positive rate (FPR) when the true positive rate (TPR) is equal to 95%. The lower the score, the better the performance.

[0092] 2) AUROC (area under the Receiver Operating Characteristic (ROC) curve): The area under the ROC curve can be interpreted as the probability that a positive ID sample has a higher detection score than a negative OOD sample. The larger the AUROC value, the better the performance.

[0093] 3) AUPR (area under the Precision-Recall (PR) curve): The PR curve is drawn by plotting the relationship between precision and recall. The larger the AUPR value, the better the performance.

[0094] AUROC is the most common metric, and AUROC is used as the main metric for OOD detection performance, while accuracy measures the performance of detecting ID samples. At the same time, the in-distribution classification accuracy is used to measure the accuracy of the model for classification. The goal is to detect more OOD samples while maintaining the ID classification performance. Compared with a variety of methods, the experimental results show that the method of the present invention improves the detection ability of out-of-distribution data while ensuring the classification accuracy, and exceeds the current mainstream methods.

[0095] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A method for out-of-distribution data detection in image classification based on super-opinion evidence deep learning uncertainty, characterized in that: The following steps are involved: Step 1: Dataset preparation: A large amount of image data and corresponding category labels are collected from online public data sets. After data cleaning and data preprocessing, an image classification data set D is constructed, and the number of sample categories is K. The sample (x, y)∈D in the data set is image data x and classification label y. Step 2: Super opinion network construction and training optimization; The super opinion network includes: a feature extractor, a super opinion evidence generation module and an opinion projection module; Among them, the feature extractor is obtained by pre-training on a large general data set, and the image data is sent to the feature extractor to obtain a feature vector reflecting the feature information such as image grayscale and texture; The feature vector is input into the super-opinion evidence generation module to generate a super-opinion evidence value corresponding to each composite set and generate a corresponding super-opinion belief; Then, these super-opinion beliefs and the corresponding set information are fed into the opinion projection module, converted into the projected polynomial opinion beliefs of each category, and bijected into the Dirichlet distribution parameters corresponding to the polynomial opinions; The expected classification probability obtained by Dirichlet distribution is used as the network prediction, and the bi-gamma evidence loss ψ(·) is calculated with the classification label y in the dataset as the evaluation indicator. The entire super-opinion network is trained until the network training reaches the expected stable state. Step 3: Image category prediction and out-of-distribution detection; The images to be predicted are sequentially input into the optimal super opinion network trained in step 2 to obtain the Dirichlet distribution parameters of the polynomial opinions; the test sample classification prediction p and uncertainty measure u are obtained through the Dirichlet distribution; given the uncertainty threshold τ, the uncertainty measure u is compared with τ as follows: If u>τ, it is determined that the sample does not belong to any category in the training set and is out-of-distribution data, and experts should be reminded to intervene for further decision-making; If u≤τ, the classification prediction p is used as the sample prediction result output.

2. The method according to claim 1, characterized in that in step 2, the super opinion network training optimization process comprises the following steps: 2.1 Using a feature extractor to extract features of the image data obtained in step 1; Take a sample (x, y)∈D from the data set, input the image data x into the pre-trained feature extractor for feature extraction, and obtain the feature f; 2.2 The feature f is input into the super-opinion evidence generation module to obtain super-evidence e H ; 2.3 Super-opinion evidence generation module further models the super-opinion evidence in 2.2 to obtain super-opinion, where the super-opinion consists of a triple: prior probability a H , Super opinion belief b H and uncertainty u; the modeling method is as follows: The calculation method of S is as follows: Among them, K is the number of sample categories, is the dataset category domain, It is the superdomain corresponding to the category domain of the dataset; 2.4 Opinion Projection Module The super opinion belief b obtained in 2.3 H Perform opinion projection, and output the parameter α of the Dirichlet distribution Dir of the multinomial opinion; The opinion projection module includes a projection coefficient calculation module and a polynomial opinion processing module, wherein the projection coefficient calculation module includes a layer of fully connected neural network for projecting super opinions to polynomial opinions; the polynomial opinion processing module is used to biject the polynomial opinions into corresponding Dirichlet distribution parameters α; The specific operations are as follows: 2.4.1 Super-opinion belief b H Input projection coefficient calculation module, W is the fully connected layer parameter of projection coefficient calculation module, calculate the matrix W corresponding to the composite set category of each super opinion H : 2.4.2 Based on the lack of prior knowledge of relative basic probability, the super opinion b H Perform an average distribution to obtain the polynomial opinion belief b: The polynomial opinion thus obtained is composed of a triplet of prior probability a, super-opinion belief b and uncertainty u, where a=1; 2.4.3 The polynomial opinion processing module bijects the polynomial opinion into the Dirichlet distribution parameter α corresponding to the polynomial opinion according to the following formula: a k =b k S+a k Where B(α) is a K-dimensional polynomial beta function, K is the number of sample categories, and p i is the probability of the category corresponding to the i-th parameter derived from the Dirichlet distribution; 2.5 Use the expected predicted classification probability obtained by Dirichlet distribution as the network prediction, and calculate the double gamma evidence loss ψ(·) with the classification label y in the data set in step 1 as the evaluation indicator to train the entire super opinion network mentioned above; The loss function is calculated as: Repeat the above steps until the network training reaches the expected stable state.

3. The method according to claim 2, characterized in that: The super evidence H The calculation method is as follows: have been H =ReLU(f) The non-negativity of evidence is ensured by the ReLU activation function, where the ReLU function is:

4. The method according to claim 1, characterized in that: Step 3 Image category prediction and out-of-distribution detection includes the following steps: 3.1 Input the image to be predicted into the optimal super opinion network trained in step 2 to obtain the Dirichlet distribution parameters of the polynomial opinions; 3.2 Obtain the test sample classification prediction p and uncertainty measure u through Dirichlet distribution; 3.3 Given an uncertainty threshold τ, where τ ranges from 0 to 1, manually selected according to the task and data, compare the uncertainty measure u with τ as follows: 3.3.1 If u>τ, it is determined that the sample does not belong to any category in the training set and is out-of-distribution data, and experts should be reminded to intervene for further decision-making; 3.3.2 If u≤τ, use the classification prediction p as the sample prediction result output.

Images