Method and system for generating virtual out-of-distribution samples to suppress overconfidence of neural networks
By generating virtual out-of-distribution samples and providing low-confidence supervision signals, the problem of overconfidence in out-of-distribution samples in deep neural networks is solved, achieving higher classification accuracy and training efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU UNIVERSITY
- Filing Date
- 2023-06-19
- Publication Date
- 2026-07-03
AI Technical Summary
Existing technologies struggle to effectively curb the overconfidence of deep neural networks in out-of-distribution samples, particularly due to limitations in dataset selection and computational complexity, leading to increased risks in critical security areas.
By generating virtual out-of-distribution samples, the encoder maps training image samples from a high-dimensional feature space to a low-dimensional space. The KNN algorithm is used to find the farthest in-distribution sample pairs, generating virtual out-of-distribution samples and providing low-confidence supervision signals during training to suppress the overconfidence of the neural network.
The generated virtual out-of-distribution samples are closer to the in-distribution samples, which can better constrain the decision boundary, reduce the number of reference out-of-distribution samples, reduce the training burden of the neural network, and improve classification accuracy.
Smart Images
Figure CN116721310B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of neural network technology, and in particular to a method and system for generating virtual distribution out-of-samples to suppress neural network overconfidence. Background Technology
[0002] With the development of technology, modern deep neural networks are widely used in various fields, including autonomous driving, intelligent medical imaging diagnosis, facial recognition, and intelligent robotics. However, these neural networks have a fatal flaw: they often generate high confidence in out-of-distribution image inputs. These out-of-distribution sample inputs come from unknown classes and should not be predicted by the model. This poses a fatal risk to deployments in critical safety areas, necessitating the design of algorithms to mitigate the overconfidence of neural networks in out-of-distribution samples.
[0003] There are two main existing technical solutions. One is a post-processing algorithm, which processes the feature information extracted by the deep neural network to extract useful information at the feature level to distinguish between in-distribution and out-of-distribution samples, thereby suppressing the overconfidence of the neural network. The other is to fine-tune the deep neural network using information, updating the network's parameters to make low-confidence predictions for out-of-distribution images. These existing methods can identify out-of-distribution images by assigning them low confidence, thus avoiding processing by the deep neural network, but they have many unavoidable limitations. Existing methods use auxiliary out-of-distribution image datasets to simulate a subset of out-of-distribution samples deployed in the open world. This method has limitations; the optimal choice of auxiliary datasets remains an open question, and the challenges of data imbalance and computational complexity reduce its efficiency and practicality. Another approach uses generative adversarial networks to synthesize the required images, but synthesizing images in high-dimensional pixel images can be difficult to optimize, and the synthesized images may not be very representative. Summary of the Invention
[0004] The purpose of this invention is to provide a method and system for generating virtual distribution out-of-samples to suppress neural network overconfidence, aiming to solve the above-mentioned problems in the prior art.
[0005] This invention provides a method for generating virtual out-of-distribution samples to suppress neural network overconfidence, comprising:
[0006] S1. Sample the training images and input the training image samples into the encoder module. The encoder module maps the training image samples from the high-dimensional feature space to the low-dimensional feature space.
[0007] S2. Obtain training image samples within the distribution located at the edge, and use the KNN algorithm to find the K pairs of samples within the distribution that are farthest apart;
[0008] S3. Generate virtual out-of-distribution sample candidates based on the K most distant in-distribution sample pairs;
[0009] S4. Filter the virtual out-of-distribution sample candidates to obtain virtual out-of-distribution samples;
[0010] S5. By using the generated virtual out-of-distribution samples, during the training of the neural network, these virtual out-of-distribution samples are given a confidence level below a preset threshold to provide the neural network with supervision signals for unknown data, thereby suppressing the overconfidence of the neural network.
[0011] This invention provides a system for generating virtual out-of-sample distributions to suppress neural network overconfidence, comprising:
[0012] The dimensionality reduction module is used to sample the training images and input the training image samples into the encoder module. The encoder module maps the training image samples from the high-dimensional feature space to the low-dimensional feature space.
[0013] The module for finding samples within the boundary distribution is used to obtain training image samples within the distribution located at the edge. The KNN algorithm is used to find the K pairs of samples within the distribution that are farthest apart.
[0014] The out-of-distribution sample candidate module is used to generate virtual out-of-distribution sample candidates based on the K pairs of in-distribution samples that are furthest apart.
[0015] The virtual out-of-distribution sample module is used to filter virtual out-of-distribution sample candidates to obtain virtual out-of-distribution samples;
[0016] The overconfidence suppression module is used to utilize generated virtual out-of-distribution samples during the training of the neural network. These virtual out-of-distribution samples are given a confidence level below a preset threshold, providing the neural network with supervision signals of unknown data, thereby suppressing the overconfidence of the neural network.
[0017] By employing the embodiments of the present invention, virtual out-of-distribution samples are generated around the in-distribution samples in the feature space, supplementing the supervision signals of unknown data lacking in the neural network. The virtual out-of-distribution samples generated by the embodiments of the present invention can be closer to the in-distribution samples and can better constrain the decision boundary of the in-distribution samples. Because the generated virtual out-of-distribution samples are closer to the in-distribution samples, these out-of-distribution samples are often more valuable for reference, thereby reducing the number of reference out-of-distribution samples and alleviating the training burden of deep neural networks.
[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0020] Figure 1 This is a flowchart of a method for generating virtual distribution out-of-samples to suppress neural network overconfidence according to an embodiment of the present invention;
[0021] Figure 2 This is a schematic diagram of a system for generating virtual distribution out-of-samples to suppress neural network overconfidence, according to an embodiment of the present invention.
[0022] Figure 3 This is a flowchart of a specific embodiment of the present invention. Detailed Implementation
[0023] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] Method Implementation Examples
[0025] According to embodiments of the present invention, a method for generating virtual distribution out-of-samples to suppress neural network overconfidence is provided. Figure 1 This is a flowchart of a method for generating virtual distribution out-of-samples to suppress neural network overconfidence according to an embodiment of the present invention, such as... Figure 1 As shown, the method for generating virtual distribution out-of-samples to suppress neural network overconfidence according to an embodiment of the present invention specifically includes:
[0026] S1. Sample the training images and input the training image samples into the encoder module. The encoder module maps the training image samples from the high-dimensional feature space to the low-dimensional feature space.
[0027] S2. Obtain training image samples within the distribution located at the edge, and use the KNN algorithm to find the K pairs of samples within the distribution that are farthest apart;
[0028] S3. Generate virtual out-of-distribution sample candidates based on the K most distant in-distribution sample pairs;
[0029] S4. Filter the virtual out-of-distribution sample candidates to obtain virtual out-of-distribution samples;
[0030] S5. By using the generated virtual out-of-distribution samples, during the training of the neural network, these virtual out-of-distribution samples are given a confidence level below a preset threshold to provide the neural network with supervision signals for unknown data, thereby suppressing the overconfidence of the neural network.
[0031] like Figure 3 This is a flowchart of a specific embodiment of the present invention. The following is a description of a specific embodiment, and the specific steps are as follows:
[0032] 1. First, the training images are sampled, and the training image samples (x) are input into the encoder module. The encoder module maps the normal flow from the high-dimensional sample space to the low-dimensional feature space, i.e., the feature vector (z). The encoder is built using convolutional kernels and activation functions, and can extract local and global features of the image samples. The mathematical expression of the encoder is as follows:
[0033] z = Encoder(x) Formula 1;
[0034] 2. Locate the in-distribution image samples located at the edge. Use the KNN algorithm to find the K pairs of in-distribution samples that are furthest apart. Because of the large distance, these in-distribution sample points are considered more likely to be in-distribution images located at the edge.
[0035] 3. Next, virtual out-of-distribution samples are randomly generated from the in-distribution samples of the marginal distribution using a Gaussian distribution.
[0036] 4. Next, select the virtual out-of-distribution sample that is farthest from the current class center within a certain range in the opposite direction between the class centers of each in-distribution class. In this way, the selected out-of-distribution sample is relatively close to the in-distribution sample. Selecting the farthest in the opposite direction is only to make the selected out-of-distribution sample more in line with the definition of an out-of-distribution sample, rather than selecting a possible marginal in-distribution sample.
[0037] 5. By using the generated virtual out-of-distribution samples, during the training of the neural network, these virtual out-of-distribution samples are given low confidence to provide the neural network with supervision signals of unknown data, thereby further suppressing the overconfidence of the neural network.
[0038] In practical applications, we input training image samples, which are then processed by an encoder to extract low-dimensional feature vectors. Next, we search for boundary samples within the in-distribution set. Based on these boundary samples, we randomly generate out-of-distribution samples using a multivariate Gaussian distribution. Finally, we select the virtual out-of-distribution sample furthest from the current class center within a certain range in the opposite direction of the class centers of each in-distribution class. By designing a novel loss function, we assign high confidence to the training samples and low confidence to the generated virtual out-of-distribution samples, thus further guiding the neural network.
[0039] This invention helps the neural network to better learn the decision boundary between training samples and out-of-distribution samples by selecting samples within the edge distribution of image samples in the feature space, generating out-of-distribution samples based on the in-distribution samples, and selecting appropriate out-of-distribution samples more efficiently. This ensures that the deep neural network can accurately identify out-of-distribution samples while maintaining a high classification accuracy.
[0040] The embodiments of the present invention have the following beneficial effects:
[0041] This method proposes a technique to suppress overconfidence in neural networks by generating virtual out-of-distribution samples around the in-distribution samples in the feature space, thus supplementing the supervision signals of unknown data lacking in the neural network.
[0042] Compared to other methods that generate out-of-distribution samples, the virtual out-of-distribution samples generated by this method are closer to the in-distribution samples and can better constrain the decision boundaries of the in-distribution samples. Because the generated virtual out-of-distribution samples are closer to the in-distribution samples, these out-of-distribution samples are often more valuable for reference, thereby reducing the number of reference out-of-distribution samples and alleviating the training burden of deep neural networks.
[0043] System Implementation Examples
[0044] According to embodiments of the present invention, a system for generating virtual distribution out-of-sample data to suppress neural network overconfidence is provided. Figure 2 This is a schematic diagram of a system for generating virtual distribution out-of-sample samples to suppress neural network overconfidence, as described in an embodiment of the present invention. Figure 2 As shown, a schematic diagram of a system for generating virtual out-of-sample data to suppress neural network overconfidence according to an embodiment of the present invention specifically includes:
[0045] Dimensionality reduction module 20 is used to sample the training images, inputting the training image samples into the encoder module, which then maps the training image samples from a high-dimensional feature space to a low-dimensional feature space. Specifically, dimensionality reduction module 20 is used for:
[0046] An encoder is constructed using convolutional kernels and activation functions to extract local and global features from image samples. The mathematical expression for the encoder is as follows:
[0047] z = Encoder(x) Formula 1.
[0048] The module 22 for finding samples within the boundary distribution is used to obtain training image samples within the distribution located at the edge. The KNN algorithm is used to find the K pairs of samples within the distribution that are farthest apart.
[0049] The out-of-distribution sample candidate module 24 is used to generate virtual out-of-distribution sample candidates based on the K pairs of in-distribution samples that are furthest apart; the out-of-distribution sample candidate module 24 is specifically used for:
[0050] Virtual out-of-distribution candidate samples are randomly generated using a Gaussian distribution.
[0051] The virtual out-of-distribution sample module 26 is used to filter virtual out-of-distribution sample candidates to obtain virtual out-of-distribution samples; specifically, the virtual out-of-distribution sample module 26 is used for:
[0052] Set a preset range threshold, and select the virtual off-distribution sample that is farthest from the current class center within the preset range threshold in the opposite direction between the centers of each distribution.
[0053] The overconfidence suppression module 28 is used to provide supervision signals of unknown data to the neural network by giving these virtual out-of-distribution samples a confidence level below a preset threshold during the training process of the neural network, thereby suppressing the overconfidence of the neural network.
[0054] Module 28 for suppressing overconfidence is specifically used for:
[0055] A new loss function is designed, with a first confidence level and a second confidence level. The training images are given a first confidence level, and the generated virtual out-of-distribution samples are given a second confidence level to further guide the neural network. The first confidence level is greater than the second confidence level.
[0056] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0057] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.
Claims
1. A method for generating a virtual out-of-distribution sample to suppress overconfidence of a neural network, characterized in that, include: S1. Sample the training images and input the training image samples into the encoder module. The encoder module maps the training image samples from the high-dimensional feature space to the low-dimensional feature space. S2. Obtain training image samples within the distribution located at the edge, and use the KNN algorithm to find the K pairs of samples within the distribution that are farthest apart; S3. Generate virtual off-distribution sample candidates based on the K farthest in-distribution samples; S4. Filter the virtual out-of-distribution sample candidates to obtain virtual out-of-distribution samples; S5. By using the generated virtual out-of-distribution samples, during the training of the neural network, these virtual out-of-distribution samples are given a confidence level below a preset threshold to provide the neural network with a supervisory signal for unknown data, thereby suppressing the overconfidence of the neural network. S4 specifically includes: Set a preset range threshold, and select the virtual off-distribution sample that is farthest from the current class center within the preset range threshold in the opposite direction between each class center in the distribution. S5 specifically includes: A new loss function is designed, with a first confidence level and a second confidence level. The training images are given a first confidence level, and the generated virtual out-of-distribution samples are given a second confidence level to further guide the neural network. The first confidence level is greater than the second confidence level.
2. The method of claim 1, wherein, S1 specifically includes: An encoder is constructed using convolutional kernels and activation functions to extract local and global features from image samples. The mathematical expression for the encoder is as follows: Official 1.
3. The method according to claim 1, characterized in that, S3 specifically includes: Virtual out-of-distribution candidate samples are randomly generated using a Gaussian distribution.
4. A system for generating virtual distribution out-of-samples to suppress neural network overconfidence, characterized in that, include: The dimensionality reduction module is used to sample the training images and input the training image samples into the encoder module. The encoder module maps the training image samples from the high-dimensional feature space to the low-dimensional feature space. The module for finding samples within the boundary distribution is used to obtain training image samples within the distribution located at the edge. The KNN algorithm is used to find the K pairs of samples within the distribution that are farthest apart. The out-of-distribution sample candidate module is used to generate virtual out-of-distribution sample candidates based on the K pairs of in-distribution samples that are furthest apart. The virtual out-of-distribution sample module is used to filter the virtual out-of-distribution sample candidates to obtain virtual out-of-distribution samples; The overconfidence suppression module is used to utilize generated virtual out-of-distribution samples during the training of the neural network. These virtual out-of-distribution samples are given a confidence level below a preset threshold, providing the neural network with supervision signals of unknown data, thereby suppressing the overconfidence of the neural network. The virtual distribution out-of-sample module is specifically used for: Set a preset range threshold, and select the virtual off-distribution sample that is farthest from the current class center within the preset range threshold in the opposite direction between each class center in the distribution. The module for suppressing overconfidence is specifically used for: A new loss function is designed, with a first confidence level and a second confidence level. The training images are given a first confidence level, and the generated virtual out-of-distribution samples are given a second confidence level to further guide the neural network. The first confidence level is greater than the second confidence level.
5. The system according to claim 4, characterized in that, The dimensionality reduction module is specifically used for: An encoder is constructed using convolutional kernels and activation functions to extract local and global features from image samples. The mathematical expression for the encoder is as follows: Official 1.
6. The system according to claim 4, characterized in that, The off-distribution sample candidate module is specifically used for: Virtual out-of-distribution candidate samples are randomly generated using a Gaussian distribution.