Robust distribution external detection method based on adversarial virtual outliers

Abstract

The invention discloses a robust distribution outside detection method based on adversarial virtual outliers, and the method comprises the steps: collecting and preprocessing in-distribution data, and obtaining a reliable in-distribution data set; adopting adversarial training to obtain a robust feature extractor, performing adversarial propagation on all the in-distribution data sets, and inputting the data sets into the robust feature extractor to obtain an in-distribution robust feature set; acquiring boundary points of the data in the distribution, and sampling by using the boundary points in a low-likelihood region around the boundary points to acquire data points outside the virtual distribution; carrying out logit normalization on the neural ODE module and carrying out training; the method comprises the following steps: enabling input data to pass through a robust feature extractor, a neural ODE module and an out-of-distribution detection module, and taking an obtained maximum softmax confidence coefficient as an out-of-distribution detection score so as to distinguish an in-distribution sample from an out-of-distribution sample; according to the method, the problems that the robustness factor is not considered and the strong hypothesis is depended when the virtual distribution external data is generated in a scene without extra data in the existing scheme can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning systems, and in particular to a robust out-of-distribution detection method based on counteracting virtual outliers. Background Art

[0002] When using deep learning systems in open environments, models often encounter inputs that differ from the distribution of their training data. This type of data is called out-of-distribution samples. Traditional deep learning models have significant difficulty handling out-of-distribution samples and can exhibit overconfidence, posing a significant security risk. Out-of-distribution detectors can identify out-of-distribution samples, improving the model's security to a certain extent. However, in open scenarios, out-of-distribution detectors are exposed to the risk of malicious attacks. Adversarial examples crafted by attackers can bypass the detectors and compromise the security of the entire system. Therefore, developing robust out-of-distribution detectors is crucial to ensuring the smooth operation of deep learning systems in open environments.

[0003] In the research on robust out-of-distribution detection in deep learning, most existing methods rely on additional out-of-distribution datasets to improve the robustness of out-of-distribution detectors. These methods combine the ideas of outlier exposure and adversarial training, and use additional out-of-distribution data for regularized training to build tighter robust decision boundaries. However, this type of method has stringent requirements for additional out-of-distribution datasets. They mainly include: 1. In terms of diversity, the out-of-distribution data needs to widely cover the distribution of potential abnormal types to prevent the model from overfitting to specific patterns; 2. In terms of relevance, the out-of-distribution data and the in-distribution data should have certain semantic or structural similarities in order to obtain a tighter decision boundary; 3. Scalability requires that the auxiliary out-of-distribution dataset have a large number of samples, usually twice or more than the in-distribution dataset; 4. Semantic non-overlapping stipulates that the semantic information of the out-of-distribution data cannot belong to the in-distribution category to avoid semantic confusion in the detector.

[0004] However, in practical applications, obtaining additional auxiliary out-of-distribution datasets that meet the aforementioned requirements is challenging. To achieve diversity and scalability, collecting large amounts of semantically rich out-of-distribution data is costly. To ensure semantic non-overlap, post-collection data cleaning is required, further increasing costs. In specialized fields, such as rare disease detection, out-of-distribution data represents rare categories within a long-tail distribution, making collection extremely expensive and difficult to scale. Furthermore, if the out-of-distribution data differs significantly from the actual application scenario, training will be ineffective. Currently, robust out-of-distribution detection methods for scenarios without additional data, such as the AROS method, assume a class-conditional Gaussian distribution in the feature space and sample virtual out-of-distribution data in low-likelihood regions. However, this approach has significant drawbacks: the distributional assumptions on the feature space are too strong, and the virtual out-of-distribution data do not account for adversarial factors, resulting in numerous limitations in practical use. Therefore, there is an urgent need to develop robust out-of-distribution detection methods that do not rely on strong distributional assumptions and fully account for adversarial factors in scenarios without additional data. Summary of the Invention

[0005] The purpose of the present invention is to provide a robust out-of-distribution detection method based on counteracting virtual outliers. This method can solve the problem that existing solutions do not consider robustness factors and rely on strong assumptions when generating virtual out-of-distribution data in scenarios without additional data, significantly improve the model's robust out-of-distribution detection capability in scenarios without additional data, and enhance the security and stability of deep learning systems in complex open environments.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A robust out-of-distribution detection method based on counteracting virtual outliers, the method comprising:

[0008] Step 1: Collect and preprocess in-distribution data to obtain a reliable in-distribution data set;

[0009] Step 2: Use adversarial training to obtain a robust feature extractor. After all in-distribution data sets have been propagated adversarially, they are input into the robust feature extractor to obtain an in-distribution robust feature set.

[0010] Step 3: Get the boundary points of the data within the distribution, and use the boundary points to sample the low-likelihood area around them to obtain the data points outside the virtual distribution;

[0011] Step 4: Logit normalize the neural ODE module and train the neural ODE module; where ODE represents a differential equation;

[0012] Step 5: The input data is passed through the robust feature extractor, the neural ODE module and the out-of-distribution detection module in sequence. The output vector is normalized by logit to obtain the maximum softmax confidence. The maximum softmax confidence is used as the out-of-distribution detection score to distinguish in-distribution samples from out-of-distribution samples.

[0013] It can be seen from the technical solution provided by the above-mentioned present invention that the above-mentioned method can solve the problem that the existing solution does not consider robustness factors and relies on strong assumptions when generating virtual out-of-distribution data in scenarios without additional data, significantly improves the model's robust out-of-distribution detection capability in scenarios without additional data, and enhances the security and stability of deep learning systems in complex open environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 A flowchart of a robust out-of-distribution detection method based on counteracting virtual outliers provided by an embodiment of the present invention;

[0016] Figure 2 Schematic diagram of the overall framework of the robust out-of-distribution detection solution described in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments, and do not constitute a limitation of the present invention. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0018] like Figure 1 FIG. 1 is a flow chart of a robust out-of-distribution detection method based on counteracting virtual outliers provided by an embodiment of the present invention, wherein the method includes:

[0019] Step 1: Collect and preprocess in-distribution data to obtain a reliable in-distribution data set;

[0020] In this step, based on the in-distribution data categories required for system detection, in-distribution samples of the corresponding categories are collected;

[0021] The collected samples in the distribution are labeled, the sample resolution is unified through preprocessing, and the data set in the distribution is recorded as

[0022] Step 2: Use adversarial training to obtain a robust feature extractor. After all in-distribution data sets have been propagated adversarially, they are input into the robust feature extractor to obtain an in-distribution robust feature set.

[0023] In this step, the robust feature extractor uses a corresponding network structure based on the size of the training input, such as the commonly used Resnet model, and the output dimension is consistent with the number of categories of the data in the distribution;

[0024] The network structure is trained adversarially using an adversarial training strategy. Adversarial samples are generated using the Projected Gradient Descent (PGD) attack, and the classification loss is the cross-entropy loss. After training, the last layer of the network structure is removed to obtain a robust feature extractor.

[0025] In order to shift the in-distribution dataset to the out-of-distribution space, adversarial propagation is used to reduce the misjudgment of in-distribution samples under the adversarial setting. Specifically:

[0026] Adversarial propagation uses PGD attack to iterate the input in-distribution dataset multiple times:

[0027]

[0028] Where t represents the number of attack steps; is the adversarial sample obtained after attacking for t steps; α represents the iteration step size; Proj represents the projection operation; sign represents the sign function; S(x) represents the out-of-distribution detection score using the softmax confidence, and the formula is:

[0029]

[0030] Among them, f j (x) is the output of the classifier in dimension j; K is the number of categories, and the maximum output dimension of the classifier is recorded as k;

[0031] All the data sets in the distribution After adversarial propagation, the robust feature extractor is finally input to obtain the in-distribution robust feature set Z id .

[0032] Step 3: Get the boundary points of the data within the distribution, and use the boundary points to sample the low-likelihood area around them to obtain the data points outside the virtual distribution;

[0033] In this step, in order to avoid strong distribution in feature space, k-nearest neighbor distance is used to identify outliers. Specifically:

[0034] For the in-distribution robust feature set Z id Sample z belonging to category c in c , calculate its k-nearest neighbor distance d knn for:

[0035]

[0036] in For sample z c The robust feature set Z in the distribution id The kth adjacent point in d knn The feature with a large value indicates that it is a point that deviates from the cluster in category c, and this point is regarded as the boundary;

[0037] In order to prevent the selected boundary points from being concentrated in a certain area, a multi-round selection strategy is adopted. In each round of selection, for each category, first select the boundary points from Z id Randomly select N candidate points and calculate the d of all candidate points in this feature set knn Then sort them from largest to smallest and select the largest n points as boundary points. After multiple rounds of selection, all boundary points are collected to form a boundary point set B.

[0038] After obtaining the boundary point set B, we use the boundary points to sample the low-likelihood area around them to obtain data points outside the virtual distribution. Specifically:

[0039] For any boundary point z′∈B, take it as the center of a Gaussian kernel and use the Gaussian kernel to sample around it:

[0040] v~N(z′,σ 2 I)

[0041] Where v represents a virtual outlier, which conforms to the Gaussian distribution N, the mean vector of the Gaussian distribution N is z', σ is the standard deviation of the Gaussian distribution; I is the unit matrix; for each boundary point, a virtual outlier set V = (v1, v2, ..., v m );

[0042] In order to avoid confusion between virtual outliers and samples within the distribution, only the d knn The remaining points are removed and all the retained points are used as the synthetic virtual distribution data set.

[0043] Step 4: Logit normalizes the neural ODE (Ordinary Differential Equation) module and trains the neural ODE module;

[0044] In this step, the neural ODE module adopts a residual block or a fully connected structure. According to Lyapunov stability theory, for a neural ODE module, if the training strategy can satisfy the Jacobian matrix With a negative real part, it can produce an adversarial purification effect on the input. In order to make the neural ODE module have an adversarial purification effect, the following loss training is used:

[0045]

[0046] in Represents the input of the neural ODE module; represents the output of the neural ODE module; y k represents the data label; α1, α2, α3 represent hyperparameters; g1, g2 are exponential functions; f θ represents the differential function of the neural ODE;

[0047] The first term of this formula is the label mapping loss, which is used to map the final output of the module to the correct label. The second term makes the output of the neural ODE module approach the Lyapunov stable equilibrium point to prevent divergence. The third term makes the diagonal elements of the Jacobian matrix negative. The fourth term makes the Jacobian matrix have a main diagonal advantage. Among them, the third and fourth terms satisfy the Levy–Desplanques theorem, that is, the Jacobian matrix has a negative real part, which makes the neural ODE module have an adversarial purification effect.

[0048] In the absence of additional data, the input of the neural ODE module is For different input types, the loss In order to further improve the out-of-distribution detection capability and reduce the overconfidence problem of the model, a two-layer full connection is used for label mapping after the neural ODE module, and the output logit is normalized. Specifically:

[0049] For the output logit of the fully connected layer, it is decomposed into:

[0050]

[0051] Where z is the logit vector; ||h(z)|| is the size of the logit vector; is the direction vector of the logit vector; after logit normalization, the logit vector will be the direction vector replace;

[0052] loss After logit normalization, it has the following form:

[0053]

[0054] Among them U K Represents a uniform distribution in K dimensions; loss Different forms are used according to the input type. When the input y belongs to the distribution, the cross entropy loss of the output is calculated; when the input y belongs to the distribution, the cross entropy loss of the output is calculated. k KL divergence of L CE is the cross entropy loss;

[0055] In the specific implementation, the neural ODE module is combined with logit normalization processing to improve the model robustness and detection ability.

[0056] Step 5: Pass the input data through the robust feature extractor, neural ODE module and out-of-distribution detection module. The obtained output vector is logit normalized to obtain the maximum softmax confidence. The maximum softmax confidence is used as the out-of-distribution detection score to distinguish in-distribution samples from out-of-distribution samples.

[0057] In this step, if Figure 2 The figure shows the overall framework of the robust out-of-distribution detection scheme according to an embodiment of the present invention. During the use of the model, the input data passes through the robust feature extractor, the neural ODE module and the out-of-distribution detection module, and the output vector is (h1(z), h2(z), ..., h C (z), where the out-of-distribution detection module refers to the two fully connected layers before the logit normalization operation;

[0058] After logit normalization, the maximum softmax confidence S(x) is expressed as:

[0059]

[0060] Where · represents the true category corresponding to the input, and j in the denominator superimposes a total of C categories;

[0061] The maximum softmax confidence S(x) is used as the out-of-distribution detection score, and the threshold λ is set as the standard for distinguishing in-distribution samples from out-of-distribution samples. When S(x)>λ, the input is judged as an in-distribution sample; when S(x)<λ, the input is judged as an out-of-distribution sample.

[0062] It should be noted that the contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0063] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method.

[0064] An embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method.

[0065] In summary, the method described in the embodiment of the present invention has the following advantages:

[0066] 1. This invention generates virtual outliers through innovative adversarial propagation and non-parametric sampling, avoiding strong distribution assumptions in the feature space. This allows the model to focus on vulnerable areas, significantly improving the quality and effectiveness of virtual out-of-distribution data, and thus enhancing the model's ability to detect out-of-distribution samples.

[0067] 2. This paper integrates logit normalization into the neural ODE module, which effectively reduces the overconfidence problem of the model while ensuring the robustness of the model, and further improves the out-of-distribution detection performance;

[0068] 3. The method of the present invention provides an efficient and reliable solution for robust out-of-distribution detection without additional datasets, which greatly improves the security and stability of deep learning systems in scenarios without additional data.

[0069] In addition, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a program, and the corresponding program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0070] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or any form of implication that the information constitutes prior art already known to those skilled in the art.

Claims

1. A robust out-of-distribution detection method based on countering virtual outliers, characterized in that: The method comprises: Step 1: Collect and preprocess in-distribution data to obtain a reliable in-distribution data set; Step 2: Use adversarial training to obtain a robust feature extractor. After all in-distribution data sets have been propagated adversarially, they are input into the robust feature extractor to obtain an in-distribution robust feature set. Step 3: Get the boundary points of the data within the distribution, and use the boundary points to sample the low-likelihood area around them to obtain the data points outside the virtual distribution; Step 4: Logit normalize the neural ODE module and train the neural ODE module; where ODE represents a differential equation; Step 5: The input data is passed through the robust feature extractor, the neural ODE module and the out-of-distribution detection module in sequence. The output vector is normalized by logit to obtain the maximum softmax confidence. The maximum softmax confidence is used as the out-of-distribution detection score to distinguish in-distribution samples from out-of-distribution samples.

2. The robust out-of-distribution detection method based on counteracting virtual outliers according to claim 1, characterized in that: In step 1, based on the in-distribution data category required for system detection, the in-distribution samples of the corresponding category are collected; The collected samples in the distribution are labeled, the sample resolution is unified through preprocessing, and the data set in the distribution is recorded as 3. The robust out-of-distribution detection method based on counteracting virtual outliers according to claim 1, characterized in that: In step 2, the robust feature extractor uses a corresponding network structure based on the size of the training input, and the output dimension is consistent with the number of categories of the data in the distribution; Use adversarial training strategy to conduct adversarial training on the network structure, where adversarial samples are generated by projected gradient descent (PGD) attack, and the classification loss is cross entropy loss; After training is completed, the last layer of the network structure is removed to obtain a robust feature extractor; In order to shift the in-distribution dataset to the out-of-distribution space, adversarial propagation is used to reduce the misjudgment of in-distribution samples under the adversarial setting. Specifically: Adversarial propagation uses PGD attack to iterate the input in-distribution dataset multiple times: Where t represents the number of attack steps; is the adversarial sample obtained after attacking for t steps; α represents the iteration step size; Proj represents the projection operation; sign represents the sign function; S(x) represents the out-of-distribution detection score using the softmax confidence, and the formula is: Among them, f j (x) is the output of the classifier in dimension j; K is the number of categories, and the maximum output dimension of the classifier is recorded as k; All the data sets in the distribution After adversarial propagation, the robust feature extractor is finally input to obtain the in-distribution robust feature set Z id .

4. The robust out-of-distribution detection method based on counteracting virtual outliers according to claim 3, characterized in that: In step 3, in order to avoid strong distribution in feature space, k-nearest neighbor distance is used to identify outliers. Specifically: For the in-distribution robust feature set Z id Sample z belonging to category c in c , calculate its k-nearest neighbor distance d knn for: in For sample z c The robust feature set Z in the distribution id The kth adjacent point in d knn The feature with a large value indicates that it is a point that deviates from the cluster in category c, and this point is regarded as the boundary; In order to prevent the selected boundary points from being concentrated in a certain area, a multi-round selection strategy is adopted. In each round of selection, for each category, first select the boundary points from Z id Randomly select N candidate points and calculate the d of all candidate points in this feature set knn And sort them from large to small, and select the largest n as the boundary points; After multiple rounds of selection, all boundary points are collected to form a boundary point set B; After obtaining the boundary point set B, we use the boundary points to sample the low-likelihood area around them to obtain data points outside the virtual distribution. Specifically: For any boundary point z′∈B, take it as the center of a Gaussian kernel and use the Gaussian kernel to sample around it: v~N(z′,σ 2 AND) Where v represents a virtual outlier, which conforms to the Gaussian distribution N, the mean vector of the Gaussian distribution N is z', σ is the standard deviation of the Gaussian distribution; I is the unit matrix; for each boundary point, a virtual outlier set V = (v1, v2, ..., v m ); In order to avoid confusion between virtual outliers and samples within the distribution, only the d knn The remaining points are removed and all the retained points are used as the synthetic virtual distribution data set.

5. The robust out-of-distribution detection method based on counteracting virtual outliers according to claim 1, characterized in that: In step 4, the neural ODE module adopts a residual block or a fully connected structure. For a neural ODE module, the training strategy is used to make it satisfy the Jacobian matrix With a negative real part, it can produce an adversarial purification effect on the input. In order to make the neural ODE module have an adversarial purification effect, the following loss training is used: in Represents the input of the neural ODE module; represents the output of the neural ODE module; y k represents the data label; α1, α2, α3 represent hyperparameters; g1, g2 are exponential functions; f θ represents the differential function of the neural ODE; The first term of this formula is the label mapping loss, which is used to map the final output of the module to the correct label. The second term makes the output of the neural ODE module approach the Lyapunov stable equilibrium point. The third term makes the diagonal elements of the Jacobian matrix negative. The fourth term makes the Jacobian matrix have a main diagonal advantage. Among them, the third and fourth terms make the Jacobian matrix have a negative real part, which makes the neural ODE module have an adversarial purification effect. In the absence of additional data, the input of the neural ODE module is For different input types, the loss In order to further improve the out-of-distribution detection capability and reduce the overconfidence problem of the model, a two-layer full connection is used for label mapping after the neural ODE module, and the output logit is normalized. Specifically: For the output logit of the fully connected layer, it is decomposed into: Where z is the logit vector; ||h(z)|| is the size of the logit vector; is the direction vector of the logit vector; after logit normalization, the logit vector will be the direction vector replace; loss After logit normalization, it has the following form: Among them U K Represents a uniform distribution in K dimensions; loss Different forms are used according to the input type. When the input y belongs to the distribution, the cross entropy loss of the output is calculated; when the input y belongs to the distribution, the cross entropy loss of the output is calculated. k KL divergence of L CE is the cross entropy loss.

6. The robust out-of-distribution detection method based on counteracting virtual outliers according to claim 1, characterized in that: In step 5, during the model usage, the input data is passed through the robust feature extractor, the neural ODE module and the out-of-distribution detection module to obtain the output vector (h1(z), h2(z), ..., h C (z)), where the out-of-distribution detection module refers to the two fully connected layers before the logit normalization operation; After logit normalization, the maximum softmax confidence S(x) is expressed as: Where C represents the true category corresponding to the input, and j is used in the denominator to superimpose a total of C categories; The maximum softmax confidence S(x) is used as the out-of-distribution detection score, and the threshold λ is set as the standard for distinguishing in-distribution samples from out-of-distribution samples. When S(x)>λ, the input is judged as an in-distribution sample; when S(x)<λ, the input is judged as an out-of-distribution sample.

7. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.

8. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 6.

Images