A Neural Network Open Set Recognition Method Based on Minimum Directed Bounding Box

Abstract

The present invention discloses a neural network open set recognition method based on a minimum directed bounding box, comprising: S1, collecting multiple categories of communication data, and converting the communication data into multiple categories of image data through short-time Fourier transform; S2, taking the multiple categories of image data as training sets, using the training sets to train a neural network, and obtaining the trained neural network and the minimum directed bounding box corresponding to each category of image data in the training set; S3, inputting the communication image data to be classified into the trained neural network, performing dimensionality reduction processing on the output data of the trained neural network, and then comparing the data with the minimum directed bounding box corresponding to the category to complete the neural network open set recognition. The present invention can better represent the density distribution of the neural network output feature space, and on this basis compress the feature space representation range, thereby improving the accuracy of classifying untrained samples, and the classification accuracy in the original space with a sample coverage rate of 100% reaches 98%, which is better than the traditional algorithm.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and in particular to a neural network open set recognition method based on a minimum directed bounding box. Background Art

[0002] Neural networks are a powerful machine learning algorithm that aims to simulate the human brain nervous system. Since the 1950s and 1960s, neural networks have been widely used and developed, and have made great progress in the 21st century, becoming an indispensable tool in various machine learning, artificial intelligence, and computer vision tasks. It has been widely used in image recognition, speech recognition, natural language processing, game AI and other fields.

[0003] However, neural networks also have flaws and limitations. One of them is that when the training set contains only a limited number of categories, the classification model cannot handle data that does not belong to any known category in the test set. This happens because current neural network classification models usually use fully connected layers, softmax activation functions, and crossentropy loss functions, resulting in the output being a probability distribution that represents the confidence probability that the input sample belongs to each category. Even when faced with unknown classification data, the model will still give a probability distribution rather than a judgment of "uncertain" or "unknown".

[0004] There are currently several solutions to this problem:

[0005] 1. Add an additional category and classify the unknown classification data into this category for training and testing. The disadvantage of this method is that it requires additional data annotation and data augmentation, and may not cover all unknown classification data.

[0006] 2. Use the binary cross entropy loss function to convert the multi-classification problem into multiple binary classification problems. The disadvantage of this method is that it may reduce the accuracy and efficiency of the model.

[0007] 3. Use the distance measurement method to convert each sample into a high-dimensional feature vector, and use a specific loss function to optimize the distance or similarity between samples. The disadvantage of this method is that it requires additional sample mining and model structure design. Summary of the invention

[0008] In view of the above-mentioned deficiencies in the prior art, the present invention provides a neural network open set recognition method based on a minimum directed bounding box, which solves the problem that after the neural network training is completed, when unknown classification data that is not in the training set is input into the trained neural network, the network will forcibly classify the unknown classification data into known categories instead of classifying it as unknown categories.

[0009] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a neural network open set recognition method based on a minimum directed bounding box, characterized in that it includes:

[0010] S1, collect multiple types of communication data, and convert the communication data into multiple types of image data through short-time Fourier transform;

[0011] S2, using multiple types of image data as training sets, using the training sets to train a neural network, and obtaining the trained neural network and the minimum directed bounding box corresponding to each type of image data in the training set;

[0012] S3. Input the communication image data to be classified into the trained neural network, perform dimension reduction processing on the output data of the trained neural network, and compare it with the minimum directed bounding box corresponding to the category to complete the neural network open set recognition.

[0013] Further: S2 includes:

[0014] S21, using the multi-class image data as a training set, using the training set to train the neural network, and obtaining the trained neural network and high-dimensional data corresponding to each class of image data;

[0015] S22, performing PCA dimensionality reduction on the high-dimensional data corresponding to each type of image data to obtain training data after dimensionality reduction;

[0016] S23, performing OBB calculation on the training data after dimensionality reduction to obtain the minimum directional bounding box corresponding to each type of image data.

[0017] Further: the S22 includes:

[0018] S221, performing zero-mean conversion on the high-dimensional data corresponding to each type of image data to obtain zero-mean converted high-dimensional data;

[0019] S222, calculating the eigenvalues ​​and eigenvectors of the covariance matrix of the zero-mean high-dimensional data;

[0020] S223. According to the eigenvalues ​​and eigenvectors of the covariance matrix, the high-dimensional data with zero mean is reduced in dimension to obtain training data after the dimension reduction.

[0021] Further: S23 includes:

[0022] S231, calculating the expectation of the training data after dimensionality reduction;

[0023] S232, calculating the covariance matrix of the training data after dimensionality reduction according to the expectation of the training data after dimensionality reduction;

[0024] S233, calculating the direction basis vectors of the bounding box according to the covariance matrix of the training data after dimensionality reduction;

[0025] S234. Calculate the center coordinates and half length of the bounding box according to the direction basis vectors of the bounding box, and obtain the minimum directional bounding box corresponding to each type of image data.

[0026] Further: S3 includes:

[0027] S31, inputting the data to be classified into the trained neural network to obtain labels and high-dimensional data corresponding to the data to be classified;

[0028] S32, according to the label of the neural network classification, select the eigenvalues ​​and eigenvectors of the covariance matrix of the high-dimensional data with zero mean under the label to reduce the dimension of the high-dimensional data corresponding to the data to be classified, and obtain the data to be classified after the dimension reduction;

[0029] S33, comparing the data to be classified after dimension reduction with the minimum directed bounding box corresponding to the image data of this type, and completing the neural network open set recognition.

[0030] Further: in S33, if the data to be classified after dimensionality reduction is within the range of the minimum directional bounding box corresponding to the image data of this type, the data to be classified after dimensionality reduction is identified as a known classification; otherwise, the data to be classified after dimensionality reduction is identified as an unknown classification.

[0031] The beneficial effects of the present invention are:

[0032] 1. When the present application inputs unknown classified communication image data that is not in the training set into the trained neural network, the network will classify it as unknown classification, rather than forcibly classifying the unknown classified data into known classifications;

[0033] 2. A neural network unknown label classification algorithm combining dimensionality reduction algorithm and minimum directed bounding box algorithm (OBB) is proposed, which can better represent the density distribution of the neural network output feature space, and on this basis compress the feature space representation range and improve the accuracy of classifying untrained samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of the neural network open set recognition method based on the minimum directed bounding box.

[0035] Figure 2 This is the structure diagram of the trained neural network.

[0036] Figure 3 The relationship between the unknown signal identification accuracy and sample coverage. DETAILED DESCRIPTION

[0037] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0038] like Figure 1 As shown, in one embodiment of the present invention, a neural network open set recognition method based on a minimum directed bounding box is provided, comprising:

[0039] S1, collect multiple types of communication data, and convert the communication data into multiple types of image data through short-time Fourier transform;

[0040] The Short Time Fourier Transform (STFT) is implemented by taking the Fourier transform over different time windows of the signal:

[0041] First, define a window function , by translating the window function to the center , and with the original communication data x ( t ) can be multiplied to obtain the windowed signal y ( t ):

[0042]

[0043] Perform Fourier transform to obtain the spectrum distribution of the first segment sequence :

[0044]

[0045] in, F (.) represents Fourier transform, is the center position of the window function, is a complex exponential signal, j is an imaginary number, w is the frequency;

[0046] In practical application scenarios, the signal appears in the form of discrete points, so what we get is a series of spectrum sequences :

[0047]

[0048] in is the input signal time domain sample, is the window function, is a complex exponential signal in the discrete case;

[0049] In this embodiment, the sizes of multiple types of image data are all 128*240.

[0050] S2, using multiple types of image data as training sets, using the training sets to train a neural network, and obtaining the trained neural network and the minimum directed bounding box corresponding to each type of image data in the training set;

[0051] In this embodiment, the trained neural network structure diagram is as follows: Figure 2 As shown;

[0052] S3. Input the communication image data to be classified into the trained neural network, perform dimension reduction processing on the output data of the trained neural network, and compare it with the minimum directed bounding box corresponding to the category to complete the neural network open set recognition.

[0053] The S2 includes:

[0054] S21, using the multi-class image data as a training set, using the training set to train the neural network, and obtaining the trained neural network and high-dimensional data corresponding to each class of image data;

[0055] S22, performing PCA dimensionality reduction on the high-dimensional data corresponding to each type of image data to obtain training data after dimensionality reduction;

[0056] S23, performing OBB calculation on the training data after dimensionality reduction to obtain the minimum directional bounding box corresponding to each type of image data.

[0057] The S22 includes:

[0058] S221, performing zero-mean conversion on the high-dimensional data corresponding to each type of image data to obtain zero-mean converted high-dimensional data;

[0059] In this embodiment, there are m strip n dimensional image data, the original data is organized into columns n OK m Column Matrix X , the variance of the projection of all data on this basis for:

[0060]

[0061] in, m is the sample size, is an orthogonal basis, is the data point, for The projection distance on the base is As the central data point, the data is processed before data operation. x Initialize the mean to 0, that is ,thereby:

[0062]

[0063] S222. Calculate the eigenvalues ​​and eigenvectors of the covariance matrix of the zero-mean high-dimensional data, and the expression is:

[0064]

[0065] in, For the matrix X The covariance matrix of , according to which its eigenvalues ​​and corresponding eigenvectors can be obtained;

[0066] S223, according to the eigenvalues ​​and eigenvectors of the covariance matrix, reducing the dimension of the zero-mean high-dimensional data to obtain training data after dimensionality reduction;

[0067] The method of obtaining it is: arrange the eigenvectors into a matrix by row from top to bottom according to the corresponding eigenvalue size, and take the first k Rows make up the matrix p Therefore, the dimension is reduced to k The data after dimensioning is:

[0068]

[0069] in, To reduce the dimension to k Dimensioned data;

[0070] The S23 includes:

[0071] S231, calculating the expectation of the training data after dimensionality reduction;

[0072] In this embodiment, the method for calculating the expectation of the training data after dimensionality reduction is specifically:

[0073] Given a n The 3D convex hull of a triangle , then the area of ​​the entire bounding box is A M for:

[0074]

[0075] in, They are the three vertices of the triangle. A k is the area of ​​any triangle;

[0076] According to the triangle centroid formula, the average value of the three vertices can be obtained , then the weighted average of the centroid of the entire convex polyhedron M M for:

[0077]

[0078] Among them, the superscript k Indicates k triangle, superscript M represents the entire convex polyhedron, the subscript i Indicates i components, in three dimensions i =0,1,2, the parametric equation of the triangle is:

[0079]

[0080] in, , ;

[0081] According to the definition of expectation, we have:

[0082]

[0083]

[0084] in, and are the coordinates of each vector element in the dimension-reduced data;

[0085] S232, according to the expectation of the training data after dimensionality reduction, calculate the covariance matrix of the training data after dimensionality reduction C i,j ;

[0086] Its expression is: ;

[0087] S233, calculating the direction basis vectors of the bounding box according to the covariance matrix of the training data after dimensionality reduction;

[0088] For the integral of a convex polyhedron, it can be regarded as the sum of the integrals of each triangle, then:

[0089]

[0090]

[0091]

[0092]

[0093] The eigenvalues ​​and eigenvectors are given by the characteristic equation of the matrix:

[0094] Given a matrix A , its eigenvalue λ and the corresponding eigenvectorv The following relations are satisfied: A * v = λ * v ;

[0095] By solving the characteristic equation det ( A - λI ) = 0, solve for the eigenvalue λ Then, substitute it into the equation A * v = λ * v In the solution, the eigenvector v ;

[0096] in, A is a n × n The square array, v is a n dimensional column vector, λ is a real number; I is the identity matrix, det Represents the determinant.

[0097] Substituting the s233 covariance matrix into the above eigenvalue eigenvector calculation formula, a 3×3 covariance matrix can be calculated, which represents the distribution relationship of the point set in three-dimensional space. The eigenvector of the matrix can be used to characterize this matrix. Its eigenvector is the direction vector of the three axes of the directed bounding box. Calculate its eigenvector and normalize it. These vectors are the direction basis vectors of the bounding box. ;

[0098] in, v 0 , v 1 , v 2 Representing three different directions respectively;

[0099] S234, calculating the center coordinates and half length of the bounding box according to the direction basis vectors of the bounding box, and obtaining the minimum directional bounding box corresponding to each type of image data;

[0100] Project the points on the convex hull onto the direction vectors and then find the maximum and minimum values ​​in each direction , so the center of the bounding box C and half length The calculation formula is:

[0101]

[0102]

[0103] The S3 includes:

[0104] S31, inputting the data to be classified into the trained neural network to obtain labels and high-dimensional data corresponding to the data to be classified;

[0105] S32, according to the label of the neural network classification, select the eigenvalues ​​and eigenvectors of the covariance matrix of the high-dimensional data with zero mean under the label to reduce the dimension of the high-dimensional data corresponding to the data to be classified, and obtain the data to be classified after the dimension reduction;

[0106] S33, comparing the dimension-reduced data to be classified with the minimum directed bounding box corresponding to the image data of this type, to complete the neural network open set recognition.

[0107] In S33, if the data to be classified after dimensionality reduction is within the range of the minimum directional bounding box corresponding to the image data of this type, the data to be classified after dimensionality reduction is identified as a known classification; otherwise, the data to be classified after dimensionality reduction is identified as an unknown classification.

[0108] like Figure 3 As shown in the figure, the algorithm can better represent the density distribution of the feature space output by the neural network, and on this basis compress the representation range of the feature space and improve the accuracy of classifying untrained samples. Experiments have shown that the classification accuracy of the new algorithm in the original space with a sample coverage of 100% reaches 98%, which is better than the traditional algorithm.

[0109] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A neural network open set recognition method based on minimum directed bounding box, characterized in that: include: S1, collect multiple types of communication data, and convert the communication data into multiple types of image data through short-time Fourier transform; S2, using multiple types of image data as training sets, using the training sets to train a neural network, and obtaining the trained neural network and the minimum directed bounding box corresponding to each type of image data in the training set; S2 includes: S21, using the multi-class image data as a training set, using the training set to train the neural network, and obtaining the trained neural network and high-dimensional data corresponding to each class of image data; S22, performing PCA dimensionality reduction on the high-dimensional data corresponding to each type of image data to obtain training data after dimensionality reduction; S22 includes: S221, performing zero-mean conversion on the high-dimensional data corresponding to each type of image data to obtain zero-mean converted high-dimensional data; S222, calculating the eigenvalues ​​and eigenvectors of the covariance matrix of the zero-mean high-dimensional data; S223, according to the eigenvalues ​​and eigenvectors of the covariance matrix, reducing the dimension of the zero-mean high-dimensional data to obtain training data after dimensionality reduction; S23, performing OBB calculation on the training data after dimensionality reduction to obtain the minimum directed bounding box corresponding to each type of image data; S23 includes: S231, calculating the expectation of the training data after dimensionality reduction; S232, calculating the covariance matrix of the training data after dimensionality reduction according to the expectation of the training data after dimensionality reduction; S233, calculating the direction basis vectors of the bounding box according to the covariance matrix of the training data after dimensionality reduction; S234, calculating the center coordinates and half length of the bounding box according to the direction basis vectors of the bounding box, and obtaining the minimum directional bounding box corresponding to each type of image data; S3, inputting the communication image data to be classified into the trained neural network, performing dimension reduction processing on the output data of the trained neural network and comparing it with the corresponding minimum directed bounding box, so as to complete the neural network open set recognition; S3 includes: S31, inputting the data to be classified into the trained neural network to obtain labels and high-dimensional data corresponding to the data to be classified; S32, according to the label of the neural network classification, select the eigenvalues ​​and eigenvectors of the covariance matrix of the high-dimensional data with zero mean under the label to reduce the dimension of the high-dimensional data corresponding to the data to be classified, and obtain the data to be classified after the dimension reduction; S33, comparing the data to be classified after dimension reduction with the minimum directed bounding box corresponding to the image data of this type, and completing the neural network open set recognition.

2. The neural network open set recognition method based on the minimum directed bounding box according to claim 1 is characterized in that: In S33, if the data to be classified after dimensionality reduction is within the range of the minimum directional bounding box corresponding to the image data of this type, the data to be classified after dimensionality reduction is identified as a known classification; otherwise, the data to be classified after dimensionality reduction is identified as an unknown classification.

Images