Out-of-distribution detection method and system based on abnormal activation and multi-source information fusion
By combining the image classification probability output by the classifier and the depth characteristics output by the feature extractor, the out-of-distribution score is calculated, and the limitations of relying on single source information in the prior art are solved, and the accuracy of out-of-distribution detection and system security are improved.
Patent Information
- Application Number
- CN202411609795.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-03-11
AI Technical Summary
The existing off-distribution detection methods rely on single source information and cannot effectively handle the abnormal activation of deep learning models when facing off-distribution samples, resulting in limited detection accuracy.
The class correlation terms are calculated based on the image classification probability output by the classifier, the class independent terms and exception factors are calculated based on the depth features output by the feature extractor, and the external distribution score is calculated based on the information to distinguish the external distribution sample from the internal distribution sample.
The calculation of the out-of-distribution scores is achieved by fusing multiple information from the feature space and probability space, overcoming the limitations of single source information, improving the accuracy of out-of-distribution detection, avoiding the model's wrong decisions on out-of-distribution samples, and enhancing the security and robustness of the system.
Smart Images
Figure CN119600332A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to an out-of-distribution detection method, system and computer-readable storage medium based on abnormal activation and multi-source information fusion. Background Art
[0002] Out-of-distribution detection is an important research direction in the field of machine learning and deep learning. It focuses on how to identify samples that the model has not seen during training, that is, out-of-distribution samples. Image classification models are usually trained based on a specific distribution or dataset during training. However, in practical applications, the model may encounter unseen images that may come from different distributions. Out-of-distribution detection can help identify these abnormal samples, thereby preventing the model from making unreliable predictions. In some key application areas, such as autonomous driving and medical diagnosis, incorrect image classification may have serious consequences. By detecting out-of-distribution samples, the model can be prevented from making wrong decisions on these unfamiliar images, thereby improving the safety of the system.
[0003] Researchers have proposed many out-of-distribution (OOD) scoring functions to improve the model's ability to detect out-of-distribution data in image classification. Most of these methods construct scoring functions based on single-source information (such as probability-based and feature-based information). For example, out-of-distribution detection is achieved based on output probability information, including: (1) establishing an out-of-distribution detection evaluation benchmark based on output probability; (2) applying generalized entropy to probability space to deepen the exploration of out-of-distribution detection. Out-of-distribution detection is achieved based on feature information, including: (1) locating the best out-of-distribution-in-distribution separation layer by comparing the feature norm ratio of in-distribution and out-of-distribution data in each network layer; (2) using existing class representation patterns to achieve quantitative evaluation of out-of-distribution data. Out-of-distribution detection is achieved based on confidence (i.e., logit) information, including: (1) calculating the energy score on the confidence as a scoring function to distinguish out-of-distribution data; (2) using the maximum confidence as the score of out-of-distribution data. Out-of-distribution detection is achieved based on gradient information, including: (1) exploring the differences in gradient space; (2) improving the model's ability to detect abnormal samples by orthogonal projection of gradients.
[0004] However, existing out-of-distribution detection methods face a flaw, namely information limitation. Most post-hoc out-of-distribution detection methods tend to rely on information from a single source. Existing methods usually use the confidence (i.e., logit) and prediction probability output by the model, which contain rich category-related information and can reflect the confidence that the input sample belongs to a specific category. In contrast, the features of the intermediate feature layer are usually regarded as a source of information that is not related to the category because the feature extraction process does not directly depend on the specific category label.
[0005] Researchers have also noticed this shortcoming, so they proposed the virtual-logit matching method (ViM): embedding features into the confidence space as virtual confidence branches. However, when faced with out-of-distribution samples, deep learning models often produce abnormal activations, and these samples are mistakenly given extremely high confidences. This is the so-called "over-confidence" problem. The virtual confidence matching method is affected by the quality of the model output and cannot effectively handle these abnormal activations, so its detection accuracy is limited. Existing technologies also attempt to alleviate the "over-confidence" problem through activation clipping, but this method is too direct and may undermine the integrity of the information, thereby reducing the model's ability to identify data within the distribution. Summary of the invention
[0006] In order to solve the above technical problems, the present invention provides an out-of-distribution detection method based on abnormal activation and multi-source information fusion, comprising:
[0007] Calculate class-related terms based on the image classification probability output by the classifier in order to obtain class-related information in the probability space;
[0008] Class-irrelevant items and anomaly factors are calculated based on the deep features of the input image samples extracted by the feature extractor, including:
[0009] The main space is constructed with the deep features of the network training data, and the class-independent items are calculated by quantifying the degree to which the deep features deviate from the main space, so as to obtain class-independent information in the feature space;
[0010] After the deep features are arranged in descending order according to the L1 norm, the first several items are combined as abnormal factors;
[0011] The out-of-distribution score is calculated according to the class-related items, the class-irrelevant items and the abnormal factors; the image samples with the out-of-distribution score greater than or equal to the preset value are output as in-distribution samples, and the image samples with the out-of-distribution score less than the preset value are output as out-of-distribution samples.
[0012] Preferably, the class-related terms are calculated based on the image classification probability output by the classifier, and the formula is:
[0013]
[0014] Among them, CDT is the class-related term, k is the total number of categories in the classification task, and p i is the predicted classification probability of the i-th category, γ is the power parameter; when selecting the optimal value of γ, the value of γ is set to [0.1, 0.2, 0.3, 0.5, 0.7, 0.9].
[0015] Preferably, the class-independent term is calculated by quantifying the degree to which the depth feature deviates from the main space, and the formula is:
[0016]
[0017] Among them, CAT is the class-independent item, Z is the deep feature, and P is the main space constructed with the deep features of the network training data.
[0018] Preferably, the main space P is a D-dimensional linear subspace. When the dimension of the depth feature is greater than or equal to 2048, the main space dimension D is 1000, otherwise the main space dimension D is 512.
[0019] Preferably, after the deep features are arranged in descending order according to the L1 norm, the first m items are taken and merged into the abnormal factors, and the formula is:
[0020]
[0021] Among them, α is the abnormal factor, z j is the jth value of the deep feature Z arranged in descending order according to the L1 norm; when selecting the optimal value of m, the value range of m is set to an odd number between 0 and 20.
[0022] Preferably, the out-of-distribution score is calculated based on the class-related items, the class-irrelevant items and the abnormal factors, and the formula is:
[0023]
[0024] Among them, Score is the out-of-distribution score, α is the abnormal factor, CDT is the class-related item, and CAT is the class-irrelevant item.
[0025] The present invention also provides an out-of-distribution detection system based on abnormal activation and multi-source information fusion, comprising:
[0026] A class-related information extraction module is used to calculate class-related items based on the image classification probability output by the classifier, so as to obtain class-related information in the probability space;
[0027] The class-irrelevant information and abnormal factor extraction module is used to calculate the class-irrelevant items and abnormal factors based on the deep features of the input image samples extracted by the feature extractor, including:
[0028] The main space is constructed with the deep features of the network training data, and the class-independent items are calculated by quantifying the degree to which the deep features deviate from the main space, so as to obtain class-independent information in the feature space;
[0029] After the deep features are arranged in descending order according to the L1 norm, the first several items are combined as abnormal factors;
[0030] The out-of-distribution sample detection module is used to calculate the out-of-distribution score based on class-related items, class-irrelevant items and abnormal factors; output image samples with out-of-distribution scores greater than or equal to a preset value as in-distribution samples, and output image samples with out-of-distribution scores less than a preset value as out-of-distribution samples.
[0031] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned out-of-distribution detection method based on abnormal activation and multi-source information fusion are implemented.
[0032] The above technical solution of the present invention has the following beneficial effects compared with the prior art:
[0033] The present invention discloses an out-of-distribution detection method based on abnormal activation and multi-source information fusion, which uses the image classification probability output by the classifier to obtain class-related information, uses the deep features output by the feature extractor to obtain class-irrelevant information and abnormal factors, and then calculates the out-of-distribution score to distinguish out-of-distribution samples from in-distribution samples. The present invention fuses multiple information in feature space and probability space to calculate the out-of-distribution score, so as to overcome the limitations of single-source information, give full play to the complementary advantages of various types of information, avoid the image classification model from making wrong decisions on out-of-distribution samples, and improve the security and robustness of the classification system. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:
[0035] Figure 1 It is a flow chart of an out-of-distribution detection method based on abnormal activation and multi-source information fusion of the present invention. DETAILED DESCRIPTION
[0036] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0037] Embodiment 1
[0038] Reference Figure 1 As shown, the present invention proposes an out-of-distribution detection method based on abnormal activation and multi-source information fusion, including:
[0039] Calculate class-related terms based on the image classification probability output by the classifier in order to obtain class-related information in the probability space;
[0040] Class-irrelevant items and anomaly factors are calculated based on the deep features of the input image samples extracted by the feature extractor, including:
[0041] The main space is constructed with the deep features of the network training data, and the class-independent items are calculated by quantifying the degree to which the deep features deviate from the main space, so as to obtain class-independent information in the feature space;
[0042] After the deep features are arranged in descending order according to the L1 norm, the first several items are combined as abnormal factors;
[0043] The out-of-distribution score is calculated according to the class-related items, the class-irrelevant items and the abnormal factors; the image samples with the out-of-distribution score greater than or equal to the preset value are output as in-distribution samples, and the image samples with the out-of-distribution score less than the preset value are output as out-of-distribution samples.
[0044] In the inference process of image prediction and classification, the input image passes through the feature extractor to obtain deep features, the deep features pass through the classifier to output the image classification probability, and finally the category of the input image is inferred based on the image classification probability.
[0045] The preset value of the out-of-distribution fraction is usually set to ensure that 95% of the in-distribution samples are correctly classified as the in-distribution class.
[0046] Combined with the reasoning process, the computational related items in out-of-distribution detection include:
[0047] In k classification tasks, the neural network will output a k-dimensional classification probability vector p, where each category i is assigned a predicted classification probability p i , and meet For in-distribution samples, the probability vector p should be close to a one-hot distribution, that is, most of the probabilities are concentrated in one category, and the probabilities of the remaining categories are close to 0.
[0048] Taking this background into consideration, the present invention defines the ratio R of the categories i for For samples within the distribution, when the predicted classification probability p of a certain category i When it is close to 1 (i.e. the sample is highly confident that it belongs to the i-th category), the ratio R of this category i tends to 0; while the predicted classification probability p of the remaining k-1 categories i is far away from 1, resulting in its ratio R i Significant increase.
[0049] The ratio R of all categories i Add them together to get the total ratio R, the formula is:
[0050]
[0051] Among them, k is the total number of categories of the classification task, p i is the predicted classification probability of the i-th category, and R is the total ratio.
[0052] The cumulative total ratio is usually a large positive value, especially for samples within the distribution, which has a strong one-hot characteristic, resulting in a significant increase in the total ratio of samples within the distribution. However, for samples outside the distribution, their probability distribution is smoother than that of samples within the distribution, which will directly lead to the total ratio of samples outside the distribution being lower than that of samples within the distribution.
[0053] By calculating the total ratio, the difference between samples within the distribution and samples outside the distribution in the probability space can be magnified, thus providing a powerful means to distinguish samples within the distribution from samples outside the distribution.
[0054] The present invention applies a power γ to the ratio of each category as a class-related term, and the calculation formula is:
[0055]
[0056] Among them, CDT is the class-related term, k is the total number of categories in the classification task, and p i is the predicted classification probability of the i-th category, and γ is the power parameter.
[0057] Preferably, γ is a hyperparameter, and its value range is [0.1, 0.2, 0.3, 0.5, 0.7, 0.9]. The specific value is set according to the detection effect after deployment.
[0058] The computationally irrelevant items in out-of-distribution detection include:
[0059] Deep features are usually regarded as typical class-independent information sources, and the class-independent information contained in them cannot be recovered from confidence and classification probability. Therefore, the present invention focuses on deep features to explore class-independent items.
[0060] In order to simplify the subsequent geometric processing, a new origin of the feature space coordinate system can be set: o = -(W T )+b, where ()+ denotes Moore-Penrose inverse. Construct a linear subspace through the origin o in the new feature space coordinate system.
[0061] Construct the main space P, which is a D-dimensional linear subspace composed of the covariance matrix Z of the deep feature Z T The eigenvectors corresponding to the largest D eigenvalues of Z are obtained on the training data. Any feature that deviates significantly from the main space P may belong to out-of-distribution data.
[0062] Preferably, the value of D in this embodiment follows the setting in the virtual confidence matching method: when the dimension of the depth feature Z is greater than or equal to 2048, D=1000, otherwise D=512.
[0063] For any deep feature Z, it can be decomposed Where Z P is the projection of the depth feature Z to the main space P, It is the orthogonal projection of the depth feature Z to the main space P. For samples outside the distribution, their depth feature Z will deviate further from the main space P relative to samples within the distribution, that is, there will be a larger orthogonal projection
[0064] Therefore, the present invention uses orthogonal projection to calculate the class-independent term to quantify the degree to which the deep feature deviates from the main space. By capturing this difference, the class-independent information represented by the deep feature can be used to distinguish between in-distribution samples and out-of-distribution samples.
[0065] The calculation formula for class-independent terms is:
[0066]
[0067] Among them, CAT is the class-independent item, Z is the deep feature, and P is the main space.
[0068] The calculation of anomaly factors in out-of-distribution detection includes:
[0069] Many studies believe that when the network is facing out-of-distribution samples, there will be a part of the activation performance that exceeds the higher activation (i.e., deep features) when facing in-distribution samples. This part of the activation is considered as abnormal activation in the present invention. Therefore, the present invention further increases the difference between in-distribution and out-of-distribution samples in the out-of-distribution score by capturing abnormal activation as an abnormal factor α, so as to effectively detect out-of-distribution samples.
[0070] The deep features output by the feature extractor are arranged in descending order according to the L1 norm. The ones at the front are usually abnormally highly activated. Therefore, the present invention combines the first m items into abnormal factors, and the formula is:
[0071]
[0072] Among them, α is the abnormal factor, z j is the jth value of the depth feature Z after being arranged in descending order according to the L1 norm. Preferably, m is a hyperparameter, and its value range is an odd number between 0 and 20, and its specific value is set according to the detection effect after deployment.
[0073] By calculating the class-related items and class-independent items as described above, the class-independent information and class-related information of the sample are distinguished and extracted. On this basis, the present invention combines these two types of information with the anomaly factor to construct a comprehensive out-of-distribution score, aiming to accurately determine whether the test sample belongs to a known distribution sample or an unknown distribution sample. The class-independent items and anomaly factors of the out-of-distribution samples will be greater than those of the in-distribution samples, while the class-related items will be smaller than those of the in-distribution samples, so direct multiplication and division can distinguish the in-distribution samples from the out-of-distribution samples.
[0074] The calculation formula of the out-of-distribution score Score is:
[0075]
[0076] Among them, α is the abnormal factor, CDT is the class-related term, and CAT is the class-irrelevant term.
[0077] The present invention effectively fuses class-independent information in features and class-dependent information in network prediction probability to detect out-of-distribution samples, and directly captures abnormal activations as abnormal factors to further increase the separability of in-distribution and out-of-distribution samples. By integrating three types of information, namely, class-dependent items, class-independent items, and abnormal factors, into the out-of-distribution score, the present invention can improve the capture of class-dependent information, class-independent information, and sensitivity to anomalies, thereby improving the accuracy of out-of-distribution detection.
[0078] Embodiment 2
[0079] To illustrate the beneficial effects of the present invention, this example compares the method of the present invention with the most advanced out-of-distribution detection methods on six large-scale image classification networks based on convolutional neural networks (CNNs) or transformers and six out-of-distribution datasets.
[0080] For the inference model, this embodiment adopts several popular convolution-based and transformer-based models in large-scale image classification. These models include BiT-S-R101×1, ViT-B / 16, RepVGG, ResNet-50-D, Data-efficient image Transformers (DeiT), and Shifted WINdows (Swin) Transformer.
[0081] This example uses ImageNet-1K as the in-distribution dataset for large-scale out-of-distribution detection benchmarks, and uses six commonly used out-of-distribution datasets for evaluation, including OpenImage-O, Texture, iNaturalist, ImageNet-O, SUN, and Places. These datasets cover different fields, including fine-grained images, scene images, texture images, complex images, etc. For all out-of-distribution datasets, this example deletes the same categories as the in-distribution dataset ImageNet-1K.
[0082] Table 1 Out-of-distribution detection results on classifiers RepVGG and BiT-S-R101x1
[0083]
[0084]
[0085] Table 1 shows the detailed experimental results of the out-of-distribution detection task on the classifiers RepVGG and BiT-S-R101x1. The in-distribution dataset is ImageNet-1K, and the out-of-distribution datasets are OpenImage-O, Texture, iNaturalist, ImageNet-O, SUN, and Places. Maha, KL, and Shannon represent the three benchmark methods of Mahalanobis, KL Matching, and Shannon entropy, respectively. A represents AUROC, the larger the value, the better. F represents FPR95, the smaller the value, the better. The best results are in bold, and the second best results are underlined.
[0086] The upper part of Table 1 lists the specific data of RepVGG. First, compared with other out-of-distribution detection methods, the present invention achieved the best results on five OOD datasets (except Places, which was slightly inferior to pNML in AUROC, but achieved the best results in FPR95). This shows that the method of the present invention performs well in distinguishing OOD samples from ID samples. Secondly, compared with the best baseline method ViM, the average FPR95 indicator of the six datasets has improved, down 8.32% (from 58.23% to 49.91%). This improvement not only proves the effectiveness of the present invention in detecting OOD samples, but also verifies the versatility and robustness of the present invention on various datasets.
[0087] As for the results of BiT-S-R101x1, it can be seen from the lower part of Table 1 that, although the present invention does not perform best in all scoring indicators like RepVGG, the present invention has achieved the best results in 7 of the 12 evaluation indicators of 6 data sets. In addition, 3 of the remaining 5 scoring indicators also achieved the second best results. It is particularly worth mentioning that the present invention also achieved the best results in the two key indicators of average AUROC and FPR95. In terms of average FPR95, the improvement of the present invention over BiT-S-R101x1 is also very significant, down about 6%.
[0088] In order to comprehensively evaluate the effectiveness and robustness of the present invention on different models, the remaining four models (ResNet-50-D, ViT-B / 16, Swin and DeiT) were further tested for OOD. The average results of these models on the six datasets are shown in Table 2.
[0089] Table 2 shows the average performance of OOD detection methods on four models. Using ImageNet-1K as ID data, the results of ResNet-50-D, Swin, DeiT and ViT-B / 16 are shown. The average results of six OOD datasets: OpenImage-O, Texture, iNaturalist, ImageNet-O, SUN and Places are reported. Maha, KL and Shannon represent the three benchmark methods of Mahalanobis, KL Matching and Shannon entropy respectively. A represents AUROC, the larger the value, the better. F represents FPR95, the smaller the value, the better. The best result is in bold and the second best result is underlined.
[0090] Table 2 Average performance of OOD detection methods on four models
[0091]
[0092] The results in Table 2 show that, firstly, in the two models ResNet-50-D and Swin, the present invention achieves the best results in AUROC and FPR95. Secondly, the AUROC and FPR95 scores of the other two models are also among the best. This fully demonstrates the versatility of the present invention and its performance advantages on various visual models.
[0093] In summary, the out-of-distribution detection method based on abnormal activation and multi-source information fusion described in the present invention uses the image classification probability output by the classifier to obtain class-related information, uses the deep features output by the feature extractor to obtain class-irrelevant information and abnormal factors, and then calculates the out-of-distribution score to distinguish out-of-distribution samples from in-distribution samples. The present invention fuses multiple information in the feature space and probability space to calculate the out-of-distribution score to overcome the limitations of single-source information, give full play to the complementary advantages of various types of information, avoid the image classification model from making wrong decisions on out-of-distribution samples, and improve the security and robustness of the classification system.
[0094] Embodiment 3
[0095] Based on the out-of-distribution detection method based on abnormal activation and multi-source information fusion described in Example 1, this embodiment provides an out-of-distribution detection system based on abnormal activation and multi-source information fusion, including:
[0096] A class-related information extraction module is used to calculate class-related items based on the image classification probability output by the classifier, so as to obtain class-related information in the probability space;
[0097] The class-irrelevant information and abnormal factor extraction module is used to calculate the class-irrelevant items and abnormal factors based on the deep features of the input image samples extracted by the feature extractor, including:
[0098] The main space is constructed with the deep features of the network training data, and the class-independent items are calculated by quantifying the degree to which the deep features deviate from the main space, so as to obtain class-independent information in the feature space;
[0099] After the deep features are arranged in descending order according to the L1 norm, the first several items are combined as abnormal factors;
[0100] The out-of-distribution sample detection module is used to calculate the out-of-distribution score based on class-related items, class-irrelevant items and abnormal factors; output image samples with out-of-distribution scores greater than or equal to a preset value as in-distribution samples, and output image samples with out-of-distribution scores less than a preset value as out-of-distribution samples.
[0101] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of an out-of-distribution detection method based on abnormal activation and multi-source information fusion as described in Embodiment 1 are implemented.
[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.
[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0106] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.
Claims
1. An out-of-distribution detection method based on abnormal activation and multi-source information fusion, characterized in that: include: Calculate class-related terms based on the image classification probability output by the classifier in order to obtain class-related information in the probability space; Class-irrelevant items and anomaly factors are calculated based on the deep features of the input image samples extracted by the feature extractor, including: The main space is constructed with the deep features of the network training data, and the class-independent items are calculated by quantifying the degree to which the deep features deviate from the main space, so as to obtain class-independent information in the feature space; After the deep features are arranged in descending order according to the L1 norm, the first several items are combined as abnormal factors; The out-of-distribution score is calculated according to the class-related items, the class-irrelevant items and the abnormal factors; the image samples with the out-of-distribution score greater than or equal to the preset value are output as in-distribution samples, and the image samples with the out-of-distribution score less than the preset value are output as out-of-distribution samples.
2. According to claim 1, the out-of-distribution detection method based on abnormal activation and multi-source information fusion is characterized in that: The image classification probability based on the classifier output calculates the class-related terms, and the formula is: Among them, CDT is the class-related term, k is the total number of categories in the classification task, and p i is the predicted classification probability of the i-th category, γ is the power parameter; when selecting the optimal value of γ, the value of γ is set to [0.1, 0.2, 0.3, 0.5, 0.7, 0.9].
3. The out-of-distribution detection method based on abnormal activation and multi-source information fusion according to claim 1 is characterized in that: The class-independent term is calculated by quantifying the degree to which the deep features deviate from the main space. The formula is: Among them, CAT is the class-independent item, Z is the deep feature, and P is the main space constructed with the deep features of the network training data.
4. The out-of-distribution detection method based on abnormal activation and multi-source information fusion according to claim 3 is characterized in that: The main space P is a D-dimensional linear subspace. When the dimension of the depth feature is greater than or equal to 2048, the main space dimension D is 1000, otherwise the main space dimension D is 512.
5. The out-of-distribution detection method based on abnormal activation and multi-source information fusion according to claim 1 is characterized in that: After arranging the deep features in descending order according to the L1 norm, the first m items are combined into the abnormal factors. The formula is: Among them, α is the abnormal factor, z j is the jth value of the deep feature Z arranged in descending order according to the L1 norm; when selecting the optimal value of m, the value range of m is set to an odd number between 0 and 20.
6. The out-of-distribution detection method based on abnormal activation and multi-source information fusion according to claim 1 is characterized in that: The out-of-distribution score is calculated based on the class-related items, class-irrelevant items, and abnormal factors. The formula is: Among them, Score is the out-of-distribution score, α is the abnormal factor, CDT is the class-related item, and CAT is the class-irrelevant item.
7. An out-of-distribution detection system based on abnormal activation and multi-source information fusion, characterized in that: include: A class-related information extraction module is used to calculate class-related items based on the image classification probability output by the classifier, so as to obtain class-related information in the probability space; The class-irrelevant information and abnormal factor extraction module is used to calculate the class-irrelevant items and abnormal factors based on the deep features of the input image samples extracted by the feature extractor, including: The main space is constructed with the deep features of the network training data, and the class-independent items are calculated by quantifying the degree to which the deep features deviate from the main space, so as to obtain class-independent information in the feature space; After the deep features are arranged in descending order according to the L1 norm, the first several items are combined as abnormal factors; The out-of-distribution sample detection module is used to calculate the out-of-distribution score based on class-related items, class-irrelevant items and abnormal factors; output image samples with out-of-distribution scores greater than or equal to a preset value as in-distribution samples, and output image samples with out-of-distribution scores less than a preset value as out-of-distribution samples.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the out-of-distribution detection method based on abnormal activation and multi-source information fusion as described in any one of claims 1 to 6 are implemented.