Out-of-distribution sample identification method based on robust outlier sampling

Abstract

The invention discloses an out-of-distribution sample identification method based on robust outlier sampling. The method comprises the following steps: acquiring an in-distribution sample data set, and training the in-distribution sample data set based on a target convolutional neural network to obtain a trained target convolutional neural network; quantifying each sample in the sample data set in the distribution by using a mixed measurement index to obtain the mixed measurement index of each sample; constructing a distribution area of each type of samples based on the mixed measurement index of each sample in each type of samples in the in-distribution sample data set; and sorting the mixed measurement index of each sample in the distribution area of each type of samples to obtain a plurality of samples with the maximum sorting of each type of samples. According to the invention, a technical problem that a convolutional neural network for target detection cannot accurately identify an out-of-distribution sample when the out-of-distribution sample exists in target detection in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an out-of-distribution sample recognition method based on robust outlier sampling. Background Art

[0002] As the research of deep learning technology advances, object detection provides solutions for a variety of industries. However, deep neural networks trained with samples from closed domains usually show poor generalization ability when exposed to out-of-distribution samples in the open world. Out-of-distribution samples are samples whose label set has no intersection with the training category, and therefore should not be given predictions of existing categories. In order to improve the generalization ability of the model, the problem of out-of-distribution detection, which determines whether the input is in-distribution or out-of-distribution, has received increasing attention from the research community. Existing solutions mainly focus on image-level out-of-distribution detection, with the goal of determining whether the input is an in-distribution or out-of-distribution sample. However, object-level out-of-distribution detection can better understand the scene by finding unknown objects from a given image. It can more accurately distinguish abnormal areas and instances within the scene, thereby providing a more fine-grained way for scene understanding, and providing solutions for autonomous driving, medical image detection, key safety behaviors, etc.

[0003] In general, existing out-of-distribution detection solutions can be divided into three categories, including classification-based, density-based, and distance-based solutions. Previous methods are based on an assumption that they will cluster ID data together to form high-density areas, while OOD data will appear in low-density areas. For example, the VOS method assumes that the distribution of object features follows a Gaussian-like distribution, while POEM uses Thompson sampling to accurately depict the decision boundary. However, due to the diversity of data samples, these assumptions may not hold in the real world. In addition, most existing solutions rely on additional out-of-distribution datasets to regularize the model boundaries. But the auxiliary datasets cannot cover all unknown data distributions well, resulting in biased model decision boundaries. Moreover, they mainly focus on out-of-distribution detection at the image level, and rarely study the detection of out-of-distribution objects. Summary of the invention

[0004] An embodiment of the present invention provides an out-of-distribution sample recognition method based on robust outlier sampling, so as to at least solve the technical problem that in the prior art target detection, when there are out-of-distribution samples, the convolutional neural network for target detection cannot accurately recognize the out-of-distribution samples.

[0005] According to one aspect of an embodiment of the present invention, a method for identifying out-of-distribution samples based on robust outlier sampling is provided. The method may include: obtaining an in-distribution sample data set, training the in-distribution sample data set based on a target convolutional neural network, and obtaining a trained target convolutional neural network; quantifying each sample in the in-distribution sample data set with a mixed metric index, and obtaining a mixed metric index for each sample; constructing a distribution area for each class of samples based on the mixed metric index of each sample in each class of samples in the in-distribution sample data set; sorting the mixed metric index of each sample in the distribution area of ​​each class of samples, and obtaining multiple samples with the largest sorting for each class of samples; perturbing the features of each sample of the multiple samples with the largest sorting for each class of samples, and obtaining multiple new samples for each class of samples; identifying the new samples based on the trained target convolutional neural network, and obtaining identification results.

[0006] Optionally, after quantizing each sample in the sample data set within the distribution with a target coordinate, a mixed metric index for each sample is obtained, including: obtaining an energy metric and a distance metric for each sample; marking the energy metric and the distance metric for each sample in a target coordinate system to obtain a coordinate point; introducing a projection line in the target coordinate system to calculate the distance from the coordinate origin to the projection point, wherein the distance from the projection point of each sample to the coordinate origin is defined as the mixed metric index for each sample.

[0007] Optionally, the energy measurement expression of each sample is:

[0008] in, represents the energy measure of each sample, Represents each sample, represents the bounding box of each sample, represents the parameters of the trained target convolutional neural network, represents the target convolutional neural network, is the category corresponding to each sample.

[0009] Optionally, the distance metric of each sample is expressed as:

[0010] in, is the distance metric for each sample, and The corresponding category mean of the jth category, Representation and The corresponding nearest neighbor category mean of non-j category, express and with The corresponding category mean of the jth category, express and with The corresponding nearest neighbor category mean of non-j category, is a non-zero parameter.

[0011] Optionally, the expression of the mixed metric indicator of each sample is:

[0012] in, is the mixed metric for each sample, is the contribution weight of balancing the energy metric and the distance metric, Represents the slope of the projection line.

[0013] Optionally, the expression for obtaining multiple new samples of each class of samples by perturbing the features of each of the multiple samples with the largest ranking of each class of samples is:

[0014] in, For each new sample corresponding to each class of samples, For each sample feature, For disturbance, represents uniform distribution, represents a uniform distribution range.

[0015] Optionally, the expression of the loss function of the trained target convolutional neural network in the process of identifying new samples based on the trained target convolutional neural network to obtain the recognition result is:

[0016] in, is the loss of bounding box regression, is the cross entropy loss, Uncertain loss.

[0017] Optionally, the expression of the uncertain loss is:

[0018] in, is the number of sample data sets in the distribution, is the number of new samples corresponding to the sample data set in the distribution, is the target convolutional neural network after training, is the expectation of the sample data set within the distribution, is the expectation of the new sample corresponding to the sample data set within the distribution, is the mixed metric for each sample in the sample dataset within the distribution, The mixing metric for each new sample corresponding to the in-distribution sample dataset.

[0019] Beneficial effects of the present invention: (1) In view of the fact that previous methods are overly dependent on auxiliary datasets and there is a label domain offset between the auxiliary datasets and the target domain, the new boundary sample metric provided by this invention takes into account the uncertainty of the sample, the inter-class and intra-class distance indicators, and can find difficult samples near the object decision boundary, that is, data at the distribution boundary; (2) The negative sample set constructed by the adversarial sample idea in the present invention effectively ensures the separability between the in-distribution sample data set and the out-of-distribution samples, and can form a globally compact decision boundary. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of an out-of-distribution sample identification method based on robust outlier sampling according to an embodiment of the present invention; Figure 2 is a schematic diagram of an out-of-distribution sample identification method based on robust outlier sampling according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] Example 1 According to an embodiment of the present invention, a method for identifying out-of-distribution samples based on robust outlier sampling is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] Figure 1 is a flow chart of a method for identifying out-of-distribution samples based on robust outlier sampling according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps: Step S101, obtaining a sample data set within the distribution, and training the sample data set within the distribution based on a target convolutional neural network to obtain a trained target convolutional neural network.

[0025] In the technical solution provided in the above step S101 of the present invention, Figure 2 is a schematic diagram of an out-of-distribution sample identification method based on robust outlier sampling according to an embodiment of the present invention, such as Figure 2 As shown, a sample data set within the distribution is obtained, for example, the sample data set within the distribution is a traffic data set, the traffic data set is 100,000 traffic images, each traffic image contains multiple vehicles and traffic participants, and target detection is performed on the targets in the sample data set within the distribution according to the target convolutional neural network to obtain a trained target convolutional neural network and several samples. The several samples can be vehicles and traffic participants of multiple different categories. This is only an example and is not specifically limited.

[0026] Step S102, quantify each sample in the in-distribution sample data set using a mixed metric to obtain a mixed metric for each sample.

[0027] In the technical solution provided in step S102 of the present invention, if Figure 2 As shown, each sample in the sample data set within the distribution is marked in the target coordinate system, that is, the x-axis in the target coordinate system is used as the energy measurement corresponding to each sample, and the y-axis in the target coordinate system is used as the distance measurement corresponding to each sample. The energy measurement and distance measurement of each sample are marked in the target coordinate system to obtain the coordinate point, and a projection line is introduced in the target coordinate to calculate the distance from the coordinate origin to the projection point, where the distance from the projection point of each sample to the coordinate origin is defined as a mixed measurement indicator for each sample, and the projection point is a point in the vertical direction from the coordinate point to the projection line.

[0028] Step S103, constructing the distribution area of ​​each type of samples based on the mixed metric index of each sample in each type of samples in the in-distribution sample data set.

[0029] In the technical solution provided in step S103 of the present invention, if Figure 2 As shown, the distribution area of ​​each type of samples is constructed according to the mixed metric indicators of each sample in each type of samples in the in-distribution sample data set.

[0030] Step S104, sorting the mixed metric index of each sample in the distribution area of ​​each type of samples, and obtaining multiple samples with the largest sorting of each type of samples.

[0031] In the technical solution provided in step S104 of the present invention, if Figure 2 As shown, the mixed metric index of each sample in the distribution area of ​​each type of samples is sorted from large to small, and multiple samples with the largest sorting of each type of samples are obtained.

[0032] Step S105 , perturb the features of each of the multiple samples with the largest ranking of each type of samples to obtain multiple new samples of each type of samples.

[0033] In the technical solution provided in the above step S105 of the present invention, after adding disturbance to the features of each sample of the plurality of samples with the largest ranking of each category of samples, a plurality of new samples corresponding to each category of samples are obtained, wherein the new samples are out-of-distribution samples.

[0034] Step S106, identifying the new sample based on the trained target convolutional neural network to obtain a recognition result.

[0035] In the technical solution provided in the above step S106 of the present invention, the new sample is identified according to the trained target convolutional neural network to obtain the identification result of the new sample.

[0036] For example, the categories of samples corresponding to the sample data set within the distribution are first type of vehicles and second type of vehicles, respectively. Therefore, the target convolutional neural network only detects first type of vehicles and second type of vehicles. When there are third type of vehicles, it cannot be detected. Therefore, according to the first type of vehicles and the second type of vehicles, a compact decision boundary can be formed according to the above algorithm to ensure that the target convolutional neural network can detect the third type of vehicles.

[0037] The above method of this embodiment is further introduced below.

[0038] As an optional implementation method, step S102, the method of quantifying each sample in the sample data set within the distribution using a mixed metric indicator to obtain a mixed metric indicator for each sample, includes: obtaining an energy metric and a distance metric for each sample; marking the energy metric and the distance metric for each sample in a target coordinate system to obtain a coordinate point; introducing a projection line in the target coordinate system to calculate the distance from the coordinate origin to the projection point, wherein the distance from the projection point of each sample to the coordinate origin is defined as the mixed metric indicator for each sample.

[0039] In this embodiment, if Figure 2 As shown, the energy metric and distance metric of each sample are obtained, and the energy metric and distance metric of each sample are marked in an xoy target coordinate system to obtain the coordinate point, that is, the position of the energy metric and distance metric of each sample in the xoy target coordinate system. A projection line is introduced in the target coordinate to calculate the distance from the coordinate origin to the projection point. The projection point is the projection point of the coordinate point to the projection line. The distance from the projection point of each sample to the coordinate origin is defined as the mixed metric indicator of each sample.

[0040] As an optional implementation manner, the expression of the energy metric of each sample is:

[0041] in, represents the energy measure of each sample, Represents each sample, represents the bounding box of each sample, represents the parameters of the trained target convolutional neural network, represents the target convolutional neural network, is the category corresponding to each sample.

[0042] As an optional implementation manner, the distance metric expression of each sample is:

[0043] in, is the distance metric for each sample, and The corresponding category mean of the jth category, Representation and The corresponding nearest neighbor category mean of non-j category, express and with The corresponding category mean of the jth category, express and with The corresponding nearest neighbor category mean of non-j category, is a non-zero parameter.

[0044] As an optional implementation manner, the expression of the mixed metric index of each sample is:

[0045] in, is the mixed metric for each sample, is the contribution weight of balancing the energy metric and the distance metric, Represents the slope of the projection line.

[0046] As an optional implementation manner, the expression for obtaining multiple new samples of each class of samples by perturbing the features of each of the multiple samples with the largest ranking of each class of samples is:

[0047] in, For each new sample corresponding to each class of samples, For each sample feature, For disturbance, represents uniform distribution, represents a uniform distribution range.

[0048] As an optional implementation method, the expression of the loss function of the trained target convolutional neural network in the process of identifying new samples based on the trained target convolutional neural network to obtain the recognition result is:

[0049] in, is the loss of bounding box regression, is the cross entropy loss, Uncertain loss.

[0050] As an optional implementation manner, the expression of the uncertain loss is:

[0051] in, is the number of sample data sets in the distribution, is the number of new samples corresponding to the sample data set in the distribution, is the target convolutional neural network after training, is the expectation of the sample data set within the distribution, is the expectation of the new sample corresponding to the sample data set within the distribution, is the mixed metric for each sample in the sample dataset within the distribution, The mixing metric for each new sample corresponding to the in-distribution sample dataset.

[0052] In this embodiment, the trained target convolutional neural network is a nonlinear multilayer perceptron, and the expression for determining the parameters of the neural network when the loss function of the trained target convolutional neural network is minimized in the process of identifying new samples is:

[0053] in, The label used to represent each sample, including the in-distribution sample dataset and new samples.

[0054] In an embodiment of the present invention, a sample data set within a distribution is obtained, and the sample data set within the distribution is trained based on a target convolutional neural network to obtain a trained target convolutional neural network; each sample in the sample data set within the distribution is quantified using a mixed metric to obtain a mixed metric for each sample; a distribution area for each type of sample is constructed based on the mixed metric of each sample in each type of sample in the sample data set within the distribution; the mixed metric of each sample in the distribution area for each type of sample is sorted to obtain multiple samples with the largest sorting for each type of sample; the features of each sample with the largest sorting for each type of sample are perturbed to obtain multiple new samples for each type of sample; the new samples are identified based on the trained target convolutional neural network to obtain an identification result, thereby solving the technical problem in the prior art that when there are out-of-distribution samples, the convolutional neural network for target detection cannot accurately identify the out-of-distribution samples, and achieving the technical effect that the convolutional neural network for target detection that processes the sample data set within the distribution to obtain new samples corresponding to the sample data set within the distribution, can accurately identify the new samples.

[0055] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0056] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0057] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0058] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0059] In addition, each functional unit in each embodiment of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0060] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for identifying out-of-distribution samples based on robust outlier sampling, characterized in that: include: Obtaining a sample data set within the distribution, training the sample data set within the distribution based on the target convolutional neural network, and obtaining a trained target convolutional neural network; Quantify each sample in the sample data set within the distribution with a mixed metric to obtain a mixed metric for each sample; Based on the mixed metric of each sample in each class of samples in the in-distribution sample dataset, the distribution area of ​​each class of samples is constructed; Sort the mixed metric indicators of each sample in the distribution area of ​​each type of sample, and obtain multiple samples with the largest ranking of each type of sample; Perturb the features of each of the multiple samples with the largest ranking of each class of samples to obtain multiple new samples of each class of samples; The new samples are identified based on the trained target convolutional neural network to obtain the recognition results.

2. The method according to claim 1, characterized in that The method of quantifying each sample in the in-distribution sample data set using a mixed metric indicator to obtain a mixed metric indicator for each sample includes: Get the energy metric and distance metric for each sample; Mark the energy metric and distance metric of each sample in a target coordinate system to obtain the coordinate point; A projection line is introduced into the target coordinate system to calculate the distance from the coordinate origin to the projection point, where the distance from the projection point of each sample to the coordinate origin is defined as the mixed metric indicator of each sample.

3. The method according to claim 2, characterized in that The expression of the energy measurement of each sample is: in, represents the energy measure of each sample, Represents each sample, represents the bounding box of each sample, represents the parameters of the trained target convolutional neural network, represents the target convolutional neural network, is the category corresponding to each sample.

4. The method according to claim 3, characterized in that The distance metric expression of each sample is: in, is the distance metric for each sample, and The corresponding category mean of the jth category, Representation and The corresponding nearest neighbor category mean of non-j category, express and with The corresponding category mean of the jth category, express and with The corresponding nearest neighbor category mean of non-j category, is a non-zero parameter.

5. The method according to claim 4, characterized in that The expression of the mixed metric index of each sample is: in, is the mixed metric for each sample, is the contribution weight of balancing the energy metric and the distance metric, Represents the slope of the projection line.

6. The method according to claim 1, characterized in that The expression for obtaining multiple new samples of each class of samples by perturbing the features of each of the multiple samples with the largest ranking of each class of samples is: in, For each new sample corresponding to each class of samples, For each sample feature, For disturbance, represents uniform distribution, represents a uniform distribution range.

7. The method according to claim 1, characterized in that The expression of the loss function of the trained target convolutional neural network in the process of identifying new samples based on the trained target convolutional neural network to obtain the recognition result is: in, is the loss of bounding box regression, is the cross entropy loss, Uncertain loss.

8. The method according to claim 7, characterized in that The expression of the uncertain loss is: in, is the number of samples in the distribution, is the number of new samples corresponding to the sample data set in the distribution, is the target convolutional neural network after training, is the expectation of the sample data set within the distribution, is the expectation of the new sample corresponding to the sample data set within the distribution, is the mixed metric for each sample in the in-distribution sample dataset, The mixing metric for each new sample corresponding to the in-distribution sample dataset.

9. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

10. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

Images