Out-of-distribution detection method and system based on intra-class difference

Abstract

The invention provides an out-of-distribution detection method and system based on intra-class differences, and relates to the technical field of machine learning, and the method comprises the steps: obtaining a test sample; inputting the test samples into the trained distribution external detection model to obtain a classification result of the test samples; wherein the out-of-distribution detection model comprises a feature extractor, an abnormal activation shielding module, a channel selection module and a classification detection module; the abnormal activation shielding module shields abnormal activation of the feature extractor; the channel selection module is used for selecting a plurality of channels with high discrimination for out-of-distribution detection of a test sample on the basis of discrimination of each channel of the feature extractor to in-distribution data, and the discrimination adopts inter-class difference and in-class difference measurement; and the classification detection module obtains a classification result of the test sample according to the output of the feature extractor. According to the invention, the accuracy of out-of-distribution detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to an out-of-distribution detection method and system based on intra-class differences. Background Art

[0002] Machine learning models have high performance and high robustness in closed and static application scenarios, but face many challenges in open and dynamic environments. One of the core issues is how to effectively deal with the inconsistency between the label spaces of training data and test data, that is, the dynamic changes in data categories.

[0003] As an emerging branch of machine learning, out-of-distribution detection is specifically used to deal with the problem of dynamic changes in data categories. Out-of-distribution detection mainly includes density-based methods and classification-based methods. Among them, the classification-based method models the conditional distribution of the in-distribution training data and designs a scoring function to determine whether the test sample belongs to out-of-distribution data. For the classification-based method, additional auxiliary out-of-distribution data sets are used to calibrate the model in the training phase, and in the test phase, it can be divided into confidence, feature, distance, gradient, pruning and activation-based methods. Among them, the activation-based method maximizes the difference between in-distribution and out-of-distribution data by truncating abnormally low or high activations. However, the activation-based method assumes that each channel contributes equally to out-of-distribution detection, but in fact, the contribution of different channels to out-of-distribution detection may not be the same. In this regard, the related art proposes to increase the separability between in-distribution and out-of-distribution data by discarding some channels, and select channels according to the class discriminability of in-distribution data, but it only considers the inter-class differences between classes. For the channel selection of out-of-distribution detection, it is not enough to rely solely on the inter-class differences between known classes, and it is difficult to fully guarantee the accuracy of out-of-distribution detection. Summary of the invention

[0004] In order to address the deficiencies of the prior art, an object of the present invention is to provide an out-of-distribution detection method and system based on intra-class differences to improve the accuracy of out-of-distribution detection.

[0005] To achieve the above object, according to some embodiments, a first aspect of the present invention provides an out-of-distribution detection method based on intra-class differences, comprising:

[0006] Obtain test samples;

[0007] Input the test sample into the trained out-of-distribution detection model to obtain the classification result of the test sample;

[0008] Among them, the out-of-distribution detection model includes a feature extractor, an abnormal activation shielding module, a channel selection module, and a classification detection module; the abnormal activation shielding module shields the abnormal activation of the feature extractor; the channel selection module selects multiple channels with high discrimination for out-of-distribution detection of test samples based on the discrimination of each channel of the feature extractor for in-distribution data, wherein the discrimination is measured by inter-class differences and intra-class differences; the classification detection module obtains the classification result of the test sample according to the output of the feature extractor.

[0009] A second aspect of the present invention provides an out-of-distribution detection system based on intra-class differences, comprising:

[0010] A sample acquisition module is configured to acquire a test sample;

[0011] The out-of-distribution detection module is configured to input the test sample into the trained out-of-distribution detection model to obtain the classification result of the test sample; wherein the out-of-distribution detection model includes a feature extractor, an abnormal activation shielding module, a channel selection module, and a classification detection module; the abnormal activation shielding module shields the abnormal activation of the feature extractor; the channel selection module selects multiple channels with high discrimination for out-of-distribution detection of the test sample based on the discrimination of each channel of the feature extractor for in-distribution data, wherein the discrimination is measured by inter-class differences and intra-class differences; the classification detection module obtains the classification result of the test sample according to the output of the feature extractor.

[0012] According to a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to complete the steps of the above-mentioned out-of-distribution detection method based on intra-class differences.

[0013] According to a fourth aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, complete the steps of the above-mentioned out-of-distribution detection method based on intra-class differences.

[0014] According to a fifth aspect of the present invention, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned out-of-distribution detection method based on intra-class differences.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] The present invention provides an out-of-distribution detection method and system based on intra-class differences. Considering the correlation between the intra-class differences of in-distribution data and the out-of-distribution detection performance, channels with high cohesion and low coupling on known classes are more conducive to out-of-distribution detection. Channels with small intra-class differences of in-distribution data are selected to realize out-of-distribution detection, thereby improving the performance of out-of-distribution detection. The intra-class differences of in-distribution data are measured by intra-class variance, and combined with inter-class differences, the discrimination of each channel on the in-distribution data is comprehensively measured, and channels with high discrimination are selected for out-of-distribution detection. At the same time, considering that it is easy to be affected by abnormal activation when estimating intra-class differences, in order to better eliminate abnormal activation, a dedicated bandpass filter is configured for each channel to shield abnormal activation, and abnormally high activation and abnormally low activation are eliminated at the same time, thereby further improving the accuracy of out-of-distribution detection.

[0017] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0019] Figure 1 Schematic diagram of the out-of-distribution detection model in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0021] Embodiment 1

[0022] Considering that the intra-class difference of in-distribution data is correlated with the out-of-distribution detection performance, the first embodiment of the present invention provides an out-of-distribution detection method based on intra-class difference, and introduces the intra-class difference as one of the metrics of channel quality, thereby improving the accuracy of out-of-distribution detection. Figure 1 As shown, an out-of-distribution detection method based on intra-class differences includes:

[0023] Obtain test samples;

[0024] Input the test sample into the trained out-of-distribution detection model to obtain the classification result of the test sample;

[0025] Among them, the out-of-distribution detection model includes a feature extractor, an abnormal activation shielding module, a channel selection module, and a classification detection module; the abnormal activation shielding module shields the abnormal activation of the feature extractor; the channel selection module selects multiple channels with high discrimination for out-of-distribution detection of test samples based on the discrimination of each channel of the feature extractor for in-distribution data, wherein the discrimination is measured by inter-class differences and intra-class differences; the classification detection module obtains the classification result of the test sample according to the output of the feature extractor.

[0026] In this embodiment, the test sample is an image, that is, the out-of-distribution detection method provided in this embodiment can realize classification detection of the image, correctly classify the known classes, and detect the unknown classes that have never been seen in the training set.

[0027] Traditional out-of-distribution detection based on channel activation assumes that each channel is essential for out-of-distribution detection, and it is necessary to correct abnormal activation on each channel. However, relevant studies have shown that different channels contribute differently to out-of-distribution detection, so discarding some channels is more conducive to out-of-distribution detection. The "channel" here refers to the neurons on the penultimate layer of the model. In this embodiment, the model uses MobileNetV2, which consists of two parts: a feature extractor and a classifier. The penultimate layer of the model actually refers to the last layer of the feature extractor, and the number of channels represents the dimension of the feature.

[0028] In the related art, channel selection is based solely on inter-class differences, that is, considering the positive correlation between the separability of the data categories within the distribution and the detection performance outside the distribution, channels with strong separability of the data categories within the distribution are selected to improve the detection performance outside the distribution. However, it is not enough to rely solely on the inter-class differences between known classes when selecting channels. Therefore, this embodiment additionally introduces intra-class differences as one of the metrics for the quality of channels. Channels with high cohesion and low coupling on known classes are more conducive to out-of-distribution detection. The smaller the intra-class difference, the greater the contribution of the corresponding channel to out-of-distribution detection.

[0029] Specifically, this embodiment selects channels with small inter-class differences in the data within the distribution to improve the out-of-distribution detection performance, and measures the intra-class differences in the data within the distribution by the intra-class variance; at the same time, considering that the estimation of intra-class differences is easily affected by abnormal activation, in order to better eliminate abnormal activation, this embodiment uses a bandpass filter to shield both abnormally high and abnormally low activations. The activation here refers to the output of the last layer of the feature extractor.

[0030] Based on this, this embodiment designs an abnormal activation shielding module and a channel selection module after the feature extractor to implement channel-level abnormal activation shielding and channel selection.

[0031] Traditional methods do not take channel specificity into account when denoising raw activations. However, considering channel specificity, the thresholds for defining abnormal activations on each channel should be different. Abnormally high or low activation values ​​are both abnormal phenomena. To this end, we configure a dedicated bandpass filter for each channel. Each bandpass filter BP is composed of a low-pass filter LP and a high-pass filter HP in series. The purpose is to filter out abnormally low activations after filtering out abnormally high activations.

[0032] The processing of the bandpass filter can be expressed as:

[0033] BP k (z k )=LP k (z k )+HP k (z k );

[0034]

[0035] Among them, z k =(F(x)) k represents the original activation (feature) on the kth channel, x is the test sample, and F is the feature extractor. and They represent the thresholds for under-activation and over-activation on the k-th channel, respectively, which are determined by the quantile of activation on the k-th channel. N is the order of the filter. ∈ is a constant greater than 0 to avoid zero division errors in high-pass filters.

[0036] For channel selection, a channel selection module is used to first evaluate the class distinguishability (inter-class difference) of the data within the distribution of each channel based on the inter-class similarity and variance, then evaluate the intra-class difference of the data within the distribution on each channel based on the intra-class variance, and finally measure the discriminability of the channel based on the inter-class difference and intra-class difference. In general, the greater the class distinguishability (inter-class difference), the better, and the smaller the intra-class difference, the better.

[0037] For each class from the training set, the average of the features is calculated to obtain the class prototype. Then, the inter-class similarity and inter-class variance between all class prototypes are calculated as the metric of inter-class difference. Next, the intra-class variance of each class prototype is calculated and summed as the intra-class difference. Finally, K channels are selected by weighing the inter-class difference and the intra-class difference.

[0038] Obtaining in-distribution class prototypes: The inter-class differences between in-distribution class prototypes are used to evaluate the class discriminability of the channel. Therefore, the class prototypes of the training set H = {h 1 ,h 2 ,...,h C}, where C represents the number of categories in the training set. Assume represents the i-th training sample from the c-th class, n c represents the number of training samples of the cth class, then the class prototype h of this class can be calculated c :

[0039]

[0040] Where BP={BP1,BP2,…,BP M} represents a set of bandpass filters, and M represents the set size, that is, the dimension of the output features of the feature extractor.

[0041] Inter-class difference score: The inter-class difference score consists of two parts: inter-class similarity and inter-class variance. Channels with low inter-class similarity and high inter-class variance are considered discriminative channels. Given the kth channel, where k∈{1,2,...,M}, M represents the number of channels on the second-to-last layer of the model, that is, the dimension of the feature extractor output feature. First, calculate the inter-class similarity S of the channel k :

[0042]

[0043] Among them, δ(·,·) represents cosine similarity, i and j represent two different categories, Represent the class prototypes of categories i and j respectively.

[0044] Then, calculate the between-class variance V k :

[0045]

[0046] in, Represents the mean of all class prototypes on the kth channel. The greater the difference between classes, the more obvious the separation between the class prototypes in the training set, thereby enhancing the class discriminability.

[0047] Next, the hyperparameter a is used as a balancing factor to weigh the inter-class similarity and inter-class variance to obtain the final inter-class difference score J k :

[0048] J k =aS k -(1-a)V k .

[0049] Intra-class difference score: Given the kth channel, calculate and sum the intra-class variance of each known class as a measure of intra-class difference:

[0050]

[0051] in, represents the activation of the jth sample in the ith class on the kth channel after filtering by the bandpass filter, n i represents the number of activations in the i-th known class, represents the intra-class difference score.

[0052] Finally, using the discrimination score A k , weighing the inter-class differences and intra-class differences to select K channels, making these channels have as large inter-class differences and as small intra-class differences as possible, that is, the A of the channels used for out-of-distribution detection k The bigger the better.

[0053]

[0054] In the formula, A k represents the discrimination score, and ω represents the trade-off coefficient, which is used to weigh the inter-class differences and intra-class differences.

[0055] Based on abnormal activation masking and channel selection, for sample x i The overall processing process can be expressed by the following formula:

[0056]

[0057] Wherein, TD represents a channel selection operation.

[0058] The above channel selection operation is compatible with any downstream out-of-distribution detection score. In this embodiment, the energy score is used as the out-of-distribution score. Given a test sample x test , first perform abnormal activation masking on each channel, and then select channels based on the discriminability score. Next, calculate the energy score:

[0059]

[0060] Here, G represents the classifier.

[0061] Finally, the energy score is used to determine whether the test sample belongs to in-distribution data or out-of-distribution data.

[0062]

[0063] Here, λ is the value at which 95% of the in-distribution data are predicted correctly.

[0064] This embodiment explores and finds that the intra-class difference of the data within the distribution is negatively correlated with the out-of-distribution detection performance. Based on this, by selecting channels with small intra-class differences in the data within the distribution for out-of-distribution detection, in order to achieve out-of-distribution detection based on intra-class differences, the intra-class difference degree of each channel is estimated according to the intra-class variance of the class prototype within the distribution, thereby improving the accuracy of out-of-distribution detection. In addition, considering that abnormally high and abnormally low activation values ​​will cause large errors in estimating the discriminant score, a bandpass filter is used to eliminate abnormally high and abnormally low activations at the same time.

[0065] Embodiment 2

[0066] This embodiment provides an out-of-distribution detection system based on intra-class differences, including:

[0067] A sample acquisition module is configured to acquire a test sample;

[0068] The out-of-distribution detection module is configured to input the test sample into the trained out-of-distribution detection model to obtain the classification result of the test sample; wherein the out-of-distribution detection model includes a feature extractor, an abnormal activation shielding module, a channel selection module, and a classification detection module; the abnormal activation shielding module shields the abnormal activation of the feature extractor; the channel selection module selects multiple channels with high discrimination for out-of-distribution detection of the test sample based on the discrimination of each channel of the feature extractor for in-distribution data, wherein the discrimination is measured by inter-class differences and intra-class differences; the classification detection module obtains the classification result of the test sample according to the output of the feature extractor.

[0069] It should be noted here that the various modules in this embodiment correspond one-to-one to the steps of the method in Example 1, and the specific implementation process is the same, which will not be repeated here.

[0070] Embodiment 3

[0071] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to complete the steps of the method in the first embodiment.

[0072] Embodiment 4

[0073] This embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the method in the first embodiment are completed.

[0074] Embodiment 5

[0075] This embodiment provides a computer program product, including a computer program / instruction, characterized in that the computer program / instruction implements the steps of the method in the first embodiment when executed by a processor.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An out-of-distribution detection method based on intra-class differences, characterized in that: include: Obtain test samples; Input the test sample into the trained out-of-distribution detection model to obtain the classification result of the test sample; Among them, the out-of-distribution detection model includes a feature extractor, an abnormal activation shielding module, a channel selection module, and a classification detection module; the abnormal activation shielding module shields the abnormal activation of the feature extractor; the channel selection module selects multiple channels with high discrimination for out-of-distribution detection of test samples based on the discrimination of each channel of the feature extractor for in-distribution data, wherein the discrimination is measured by inter-class differences and intra-class differences; the classification detection module obtains the classification result of the test sample according to the output of the feature extractor.

2. The out-of-distribution detection method based on intra-class differences as claimed in claim 1, characterized in that: The abnormal activation shielding module uses a bandpass filter to shield abnormal activation; the threshold of the bandpass filter of each channel is determined according to the quantile of activation on the channel.

3. The out-of-distribution detection method based on intra-class differences as claimed in claim 1, characterized in that: The intra-class difference is measured by the intra-class difference score, which is calculated by summing the intra-class variance of each known class.

4. The out-of-distribution detection method based on intra-class differences as claimed in claim 3, characterized in that: The inter-class difference is measured by the inter-class difference score, which is calculated based on the inter-class similarity and inter-class variance.

5. The out-of-distribution detection method based on intra-class differences as claimed in claim 4, characterized in that: The discrimination power is reflected by a discrimination power score, and the discrimination power score is calculated by weighting the intra-class difference score and the inter-class difference score.

6. The out-of-distribution detection method based on intra-class differences as claimed in claim 1, characterized in that: The energy score is used as the out-of-distribution score, and the test sample is detected out-of-distribution using multiple selected channels to obtain the energy score of the test sample. The energy score of the test sample is used to determine whether the test sample is out-of-distribution data.

7. An out-of-distribution detection system based on intra-class differences, characterized in that: include: A sample acquisition module is configured to acquire a test sample; The out-of-distribution detection module is configured to input the test sample into the trained out-of-distribution detection model to obtain the classification result of the test sample; wherein the out-of-distribution detection model includes a feature extractor, an abnormal activation shielding module, a channel selection module, and a classification detection module; the abnormal activation shielding module shields the abnormal activation of the feature extractor; the channel selection module selects multiple channels with high discriminability for out-of-distribution detection of the test sample based on the discriminability of each channel of the feature extractor for in-distribution data, wherein the discriminability is measured by inter-class differences and intra-class differences; the classification detection module obtains the classification result of the test sample according to the output of the feature extractor.

8. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to complete the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images