Target identification optimization method and system based on neural network

Abstract

The invention relates to the technical field of neural network optimization, and discloses a target identification method and system based on multiple consistency perception, and the method comprises the steps: constructing a training data set, determining a supervision signal, and carrying out the model initialization training through the training data set and the supervision signal; constructing a self-consistency loss function based on the consistency of the prediction label and the given label of the target sample; constructing a context consistency learning loss function based on the prediction label of the target sample and the prediction label of the neighbor sample; constructing an open set identification loss function based on a multi-branch structure based on the prediction labels of the target sample under different perspectives; performing joint optimization through a self-consistency loss function, a context consistency learning loss function and a multi-branch structure-based open set identification loss function to obtain an optimized reference model; target matching identification is carried out by using the optimized reference model; according to the invention, the problems of large data marking demand and serious label noise influence of the existing target identification method are solved.

Description

Technical Field

[0001] The present invention relates to the field of neural network optimization technology, and in particular to a target recognition optimization method and system based on a neural network. Background Art

[0002] With the rapid development of deep neural network related technologies, computer vision technology using deep neural networks has achieved remarkable results, especially in the fields of image recognition and image classification, which have been applied to many commercial scenarios, such as face recognition, automated robots, and the currently popular autonomous driving. However, since the reliability of deep learning models depends to a certain extent on the number and quality of sample labels in the application scenario, deep neural networks require a large amount of labeled data to train the model, thereby continuously improving the accuracy of the model in its target field, and ultimately obtaining an optimized model with a certain generalization ability and suitable for actual application scenarios. However, the labeled data extracted from real scenes often contains label noise, which poses new challenges to the training of effective deep neural networks. Although manual data cleaning and labeling is a reliable method to improve label quality, it is costly, time-consuming and labor-intensive, and cannot be applied to large-scale data sets. In order to improve the accuracy of the model, the deep neural network training relies on labeled data with label noise, and the amount of labeled data needs to be doubled, which will put a greater burden on the existing system hardware, especially the hardware devices used for face recognition at station gates, subway gates, shopping malls and community access control. Most of them use an integrated design with poor performance, and most of them are domestic hardware with weak computing power. If the labeled data is increased, it will seriously affect the recognition speed of these devices, thereby affecting the normal use of the equipment. It can be seen that the existing neural network-based target recognition methods have the problems of large demand for labeled data and serious influence of label noise. Summary of the invention

[0003] The present invention provides a neural network-based target recognition optimization method and system to solve the problems of large label data requirements and serious label noise impact in existing neural network-based target recognition methods.

[0004] In order to achieve the above object, the present invention is implemented by the following technical solutions: In a first aspect, the present invention provides a target recognition optimization method based on a neural network, comprising: Obtain target samples to construct a training data set, determine a given label as a supervisory signal, and use the training data set and the supervisory signal to initialize the benchmark model to obtain an initialized benchmark model; Obtain the predicted label of the target sample, and construct a self-consistency loss function based on the consistency between the predicted label of the target sample and the given label; Obtain the predicted labels of the target sample's neighboring samples, and construct a contextual consistency learning loss function based on the predicted labels of the target sample and the predicted labels of the target sample's neighboring samples; Construct an open set recognition loss function based on a multi-branch structure for classifying and identifying the predicted labels of target samples; The optimized benchmark model is obtained by jointly optimizing the initial benchmark model through the self-consistency loss function, the context consistency learning loss function and the multi-branch structure open set recognition loss function; Construct a target recognition test set containing target samples to be queried and candidate samples, input the images in the target samples to be queried and the images in the candidate samples into the optimized benchmark model, and perform target matching and recognition on the images in the target samples to be queried and the images in the candidate samples.

[0005] Optionally, determining a given label as a supervisory signal includes: Construct a feature extractor and a category predictor, and use the feature extractor to extract features of the target samples in the training data set to obtain sample features, and use the category predictor to predict the category of the target samples in the training data set to obtain the predicted category score; The given label of the target sample is determined based on the predicted category score, and the given label is used as a supervisory signal.

[0006] Optionally, the initializing the benchmark model by using the training data set and the supervisory signal to obtain the initialized benchmark model includes: The benchmark model is optimized by given labels, and the optimized loss function satisfies the following relationship: ; In the formula, is the softmax function, represents the cross entropy loss function, N represents the number of samples, For a given label, represents the predicted category score, represents the loss function for a given label, T represents the temperature parameter; The benchmark model optimized with a given label is optimized again using the predicted label with the highest class score as the supervisory signal. The optimized loss function satisfies the following relationship: ; In the formula, is the softmax function, represents the cross entropy loss function, N represents the number of samples, represents the predicted label, represents the predicted category score, represents the loss function for predicting labels, T represents the temperature parameter; The baseline model trained with given labels and predicted labels is used as the initialized baseline model obtained through initialization training.

[0007] Optionally, the self-consistency loss function is constructed based on the consistency between the predicted label of the target sample and the given label, including: Construct a time-defined function that satisfies the following relationship: ; In the formula, For a given label, represents the predicted label, represents the time-dependent function value; The given label and the predicted label are compared using the defined time function. When the target sample has the same given label and predicted label, The output is 1, otherwise it is 0; The hyperparameters are designed to dynamically update the training of the benchmark model, and their hyperparameters satisfy the following relationship: ; In the formula, represents the hyperparameter, represents the exponential function, Indicates the current number of training times. Indicates the total number of training times Based on the hyperparameters and the defined time function, a self-consistent loss function is constructed, and its self-consistent loss function satisfies the following relationship: ; In the formula, represents the loss function for predicting labels, represents the loss function for a given label, It means to define a time function. represents the hyperparameter, represents the self-consistency loss function.

[0008] Optionally, constructing a context consistency learning loss function based on the predicted label of the target sample and the predicted labels of neighboring samples of the target sample includes: The predicted label of the target sample is compared with the predicted label of its neighboring samples in the feature space, and the cosine similarity of their features is calculated. The calculation formula satisfies the following relationship: ; According to the calculation results, the neighboring samples of the target sample in the feature space are searched, and the context consistency learning loss function is determined based on the predicted label of the target sample and the predicted label of the neighboring sample. The loss function satisfies the following relationship: ; In the formula, represents the relative entropy used to evaluate the difference between two distributions, Represents samples in feature space The set of K nearest neighbor samples of Represents the similarity value, represents the context consistency learning loss function, N represents the number of samples, represents the predicted category score, is the softmax function, represents the cosine similarity, Representation sample The vector features of Representation sample The vector features of Representation sample The transpose of the vector, T Represents the temperature parameter.

[0009] Optionally, the loss function of the open set recognition based on the multi-branch structure is: ; In the formula, represents the loss function of open set recognition based on multi-branch structure, represents the number of labeled samples, represents the number of unlabeled samples, Representation sample The confidence level output by logistic regression is: Representation sample The confidence level output by logistic regression is: represents the true value of the sample confidence within the division, Represents the true value of the out-of-distribution sample confidence.

[0010] Optionally, the step of jointly optimizing the initialized benchmark model by using a self-consistency loss function, a context consistency learning loss function, and a multi-branch structure open set recognition loss function to obtain an optimized benchmark model includes: The initialization benchmark model is jointly optimized by combining the self-consistency loss function, the context consistency learning loss function, and the multi-branch structure open set recognition loss function to obtain the optimized benchmark model. The loss function of the optimized benchmark model satisfies the following relationship: ; In the formula, represents the loss function of the optimized baseline model, represents the self-consistency loss function, represents the context consistency learning loss function, represents the loss function of open set recognition based on multi-branch structure, , , Represents the loss weight corresponding to the loss function; The optimized baseline model is .

[0011] Optionally, performing target matching and recognition on the image in the query sample and the image in the candidate sample includes: The model feature extractor in the optimized benchmark model is used to extract the features of the query sample and the candidate sample, and the extracted features are calculated for similarity, and the results are sorted in descending order; A matching recognition threshold is determined, and candidate images whose ranking is less than or equal to the matching recognition threshold are selected from the candidate image subset as matching recognition results.

[0012] In a second aspect, an embodiment of the present application provides a target recognition optimization system based on a neural network, including a processor and a memory; Memory, used to store computer programs; The processor is used to implement any method step described in the first aspect when executing a program stored in the memory.

[0013] Beneficial effects: The target recognition optimization method based on neural network provided by the present invention determines the supervisory signal by constructing a training data set, and performs model initialization training using the training data set and the supervisory signal; constructs a self-consistency loss function based on the consistency between the predicted label of the target sample and the given label; constructs a context consistency learning loss function based on the predicted label of the target sample and the predicted label of the neighboring sample; constructs a multi-branch structure open set recognition loss function based on the predicted label of the target sample at different perspectives; obtains an optimized benchmark model by jointly optimizing the self-consistency loss function, the context consistency learning loss function and the multi-branch structure open set recognition loss function; and uses the optimized benchmark model to perform target matching and recognition; through the above method, the representation and extraction ability of the model is improved by using the context consistency learning method, and the problem of low robustness of the neural network model to noise labels can be effectively solved by using the self-consistency and open set recognition methods. At the same time, the optimized neural network model has high applicability and can be used in all-in-one equipment and most domestic hardware in public places, thereby improving the working efficiency of visual recognition software in all-in-one equipment and domestic hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Flow chart of a target recognition optimization method based on a neural network according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the usual meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate quantity restrictions, but indicate the existence of at least one. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0017] See also Figure 1 , the embodiment of the present application provides a target recognition optimization method based on a neural network, comprising: The target recognition optimization method based on neural network includes: Obtain target samples to construct a training data set, determine a given label as a supervisory signal, and use the training data set and the supervisory signal to initialize the benchmark model to obtain an initialized benchmark model; Obtain the predicted label of the target sample, and construct a self-consistency loss function based on the consistency between the predicted label of the target sample and the given label; Obtain the predicted labels of the target sample's neighboring samples, and construct a contextual consistency learning loss function based on the predicted labels of the target sample and the predicted labels of the target sample's neighboring samples; Construct an open set recognition loss function based on a multi-branch structure for classifying and identifying the predicted labels of target samples; The optimized benchmark model is obtained by jointly optimizing the initial benchmark model through the self-consistency loss function, the context consistency learning loss function and the multi-branch structure open set recognition loss function; Construct a target recognition test set containing target samples to be queried and candidate samples, input the images in the target samples to be queried and the images in the candidate samples into the optimized benchmark model, and perform target matching and recognition on the images in the target samples to be queried and the images in the candidate samples.

[0018] Optionally, in the step of obtaining a target sample to construct a training data set, determining a given label as a supervisory signal, and initializing a benchmark model using the training data set and the supervisory signal to obtain an initialized benchmark model, the following steps are included: The training dataset is defined as , where the sample size is N , the number of sample categories is C categories, Indicates training samples, and their corresponding labels ,sample As input, and use the feature extractor And the category predictor obtains the sample feature representation and predicted class scores .

[0019] Based on the sample prediction category, the baseline model is optimized using the given label, where the loss function is defined as: (1) in, is the softmax function, represents the cross entropy loss function, T Represents the temperature parameter.

[0020] In this embodiment, the temperature parameter is usually set to 2, and the temperature parameter 2 is only used as an example here and is not limited.

[0021] As the model trains, the reliability of its predicted labels gradually increases. To prevent the model from overfitting to potentially noisy labels, we continue to use the predicted label with the highest class score. As a supervisory signal optimization model, the loss function is defined as: (2) The model initialization loss function is defined as .

[0022] Optionally, in constructing a self-consistency loss function based on the consistency between the predicted label of the target sample and the given label, a self-consistency strategy is introduced to mine clean samples, that is, the time function is defined: (3) If the sample has the same given and predicted label, then output 1, otherwise output 0.

[0023] And design hyperparameters: (4) Dynamically update model training. When the model starts training, the reliability of the predicted labels is low. By Assign smaller weight values ​​to suppress the interference of predicted labels. As the model is trained, the predicted labels will become more and more reliable. The value will also increase adaptively, that is, giving The weight of will also become larger and larger. Introducing the self-consistency awareness strategy will reduce the interference of unreliable given labels in the early stage of model training. The self-consistency loss function is defined as: (5) Optionally, in constructing a context consistency learning loss function based on the predicted label of the target sample and the predicted labels of neighboring samples of the target sample, the steps of performing context consistency learning include: The self-consistency strategy takes into account the characteristics of the sample itself, but due to the interference of noise labels, only considering the sample's own information will aggravate the noise interference. Therefore, the context consistency learning strategy is introduced to penalize the difference between the predicted label of each sample and the weighted combination of the predicted labels of its nearest samples, so that each sample is consistent with its feature space. K Neighboring samples have similar predictions, thus correcting incorrect given labels of samples. The contextual consistency learning strategy utilizes the label consistency relationship between samples and their neighboring samples in the feature space to effectively correct noisy labels.

[0024] By comparing each sample with its feature space K The neighboring samples (rather than the sample itself) are compared to each other, encouraging each sample to have similar predicted labels as its neighboring samples.

[0025] calculate ,Right now With its neighbors The feature cosine similarity of , searching for samples in the feature space of K - Neighbors .

[0026] Calculated by searching K For neighbor samples, the predicted label of the expected sample is close to the predicted label of the neighbor sample, and the loss is calculated. The loss function of context consistency learning is defined as: (6) in, represents the relative entropy used to evaluate the difference between two distributions, and Represents samples in feature space of K A collection of nearest neighbor samples, similarity value It is normalized to keep a probability distribution.

[0027] Optionally, the loss function of the open set recognition based on the multi-branch structure is: (7) In the formula, represents the loss function of open set recognition based on multi-branch structure, represents the number of labeled samples, represents the number of unlabeled samples, Representation sample The confidence level output by logistic regression is: Representation sample The confidence level output by logistic regression is: represents the true value of the sample confidence within the division, Represents the true value of the out-of-distribution sample confidence.

[0028] Optionally, combine the self-consistency loss, context consistency loss and multi-branch structure based open set recognition loss to jointly optimize the model: (8) In the formula, represents the loss function of the optimized baseline model, represents the self-consistency loss function, represents the context consistency learning loss function, represents the loss function of open set recognition based on multi-branch structure, , , Represents the loss weight corresponding to the loss function.

[0029] Optionally, in the target matching recognition between the image in the query sample and the image in the candidate sample, by using the trained model feature extractor Extract the target sample to be searched With candidate sample features Features, calculate similarity , and sort them in descending order, and select the candidate images in the candidate image subset whose ranking is less than or equal to the third threshold as the matching recognition results.

[0030] The embodiment of the present application also provides a target recognition optimization system based on a neural network, including a processor and a memory; Memory, used to store computer programs; The processor is used to implement any method step in the target recognition optimization based on neural network when executing the program stored in the memory.

[0031] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A target recognition optimization method based on a neural network, characterized in that: include: Obtain target samples to construct a training data set, determine a given label as a supervisory signal, and use the training data set and the supervisory signal to initialize the benchmark model to obtain an initialized benchmark model; Obtain the predicted label of the target sample, and construct a self-consistency loss function based on the consistency between the predicted label of the target sample and the given label; Obtain the predicted labels of the target sample's neighboring samples, and construct a contextual consistency learning loss function based on the predicted labels of the target sample and the predicted labels of the target sample's neighboring samples; Construct an open set recognition loss function based on a multi-branch structure for classifying and identifying the predicted labels of target samples; The optimized benchmark model is obtained by jointly optimizing the initial benchmark model through the self-consistency loss function, the context consistency learning loss function and the multi-branch structure open set recognition loss function; Construct a target recognition test set containing target samples to be queried and candidate samples, input the images in the target samples to be queried and the images in the candidate samples into the optimized benchmark model, and perform target matching and recognition on the images in the target samples to be queried and the images in the candidate samples.

2. The target recognition optimization method based on neural network according to claim 1 is characterized in that: The step of determining a given label as a supervisory signal comprises: Construct a feature extractor and a category predictor, and use the feature extractor to extract features of the target samples in the training data set to obtain sample features, and use the category predictor to predict the category of the target samples in the training data set to obtain the predicted category score; The given label of the target sample is determined based on the predicted category score, and the given label is used as a supervisory signal.

3. The target recognition optimization method based on neural network according to claim 1 is characterized in that: The initializing and training the benchmark model using the training data set and the supervisory signal to obtain the initialized benchmark model includes: The benchmark model is optimized by given labels, and the optimized loss function satisfies the following relationship: ； In the formula, is the softmax function, represents the cross entropy loss function, N represents the number of samples, For a given label, represents the predicted category score, represents the loss function for a given label, T represents the temperature parameter; The benchmark model optimized with a given label is optimized again using the predicted label with the highest class score as the supervisory signal. The optimized loss function satisfies the following relationship: ； In the formula, is the softmax function, represents the cross entropy loss function, N represents the number of samples, represents the predicted label, represents the predicted category score, represents the loss function for predicting labels, T represents the temperature parameter; The baseline model trained with given labels and predicted labels is used as the initialized baseline model obtained through initialization training.

4. The target recognition optimization method based on neural network according to claim 1 is characterized in that: The self-consistency loss function is constructed based on the consistency of the predicted label of the target sample and the given label, including: Construct a time-defined function that satisfies the following relationship: ； In the formula, For a given label, represents the predicted label, represents the time-dependent function value; The given label and the predicted label are compared using the defined time function. When the target sample has the same given label and predicted label, The output is 1, otherwise it is 0; The hyperparameters are designed to dynamically update the training of the benchmark model, and their hyperparameters satisfy the following relationship: ； In the formula, represents the hyperparameter, represents the exponential function, Indicates the current number of training times. Indicates the total number of training times Based on the hyperparameters and the defined time function, a self-consistent loss function is constructed, and its self-consistent loss function satisfies the following relationship: ； In the formula, represents the loss function for predicting labels, represents the loss function for a given label, It means to define a time function. represents the hyperparameter, represents the self-consistency loss function.

5. The target recognition optimization method based on neural network according to claim 1 is characterized in that: The constructing of a context consistency learning loss function based on the predicted label of the target sample and the predicted labels of neighboring samples of the target sample includes: The predicted label of the target sample is compared with the predicted label of its neighboring samples in the feature space, and the cosine similarity of their features is calculated. The calculation formula satisfies the following relationship: ； According to the calculation results, the neighboring samples of the target sample in the feature space are searched, and the context consistency learning loss function is determined based on the predicted label of the target sample and the predicted label of the neighboring sample. The loss function satisfies the following relationship: ； In the formula, represents the relative entropy used to evaluate the difference between two distributions, Represents samples in feature space The set of K nearest neighbor samples of Represents the similarity value, represents the context consistency learning loss function, N represents the number of samples, represents the predicted category score, is the softmax function, represents the cosine similarity, Representation sample The vector features of Representation sample The vector features of Representation sample The transpose of the vector, T Represents the temperature parameter.

6. The target recognition optimization method based on neural network according to claim 1 is characterized in that: The loss function of the open set recognition based on the multi-branch structure is: ； In the formula, represents the loss function of open set recognition based on multi-branch structure, represents the number of labeled samples, represents the number of unlabeled samples, Representation sample The confidence level output by logistic regression is: Representation sample The confidence level output by logistic regression is: represents the true value of the sample confidence within the division, Represents the true value of the out-of-distribution sample confidence.

7. The target recognition optimization method based on neural network according to claim 1 is characterized in that: The optimized benchmark model is obtained by jointly optimizing the initialized benchmark model through the self-consistency loss function, the context consistency learning loss function and the multi-branch structure open set recognition loss function, including: The initialization benchmark model is jointly optimized by combining the self-consistency loss function, the context consistency learning loss function, and the multi-branch structure open set recognition loss function to obtain the optimized benchmark model. The loss function of the optimized benchmark model satisfies the following relationship: ； In the formula, represents the loss function of the optimized baseline model, represents the self-consistency loss function, represents the context consistency learning loss function, represents the loss function of open set recognition based on multi-branch structure, , , Represents the loss weight corresponding to the loss function; The optimized baseline model is .

8. The target recognition optimization method based on neural network according to claim 1 is characterized in that: The target matching and recognition of the image in the query sample and the image in the candidate sample includes: The model feature extractor in the optimized benchmark model is used to extract the features of the query sample and the candidate sample, and the extracted features are calculated for similarity, and the results are sorted in descending order; A matching recognition threshold is determined, and candidate images whose ranking is less than or equal to the matching recognition threshold are selected from the candidate image subset as matching recognition results.

9. A target recognition optimization system based on neural network, characterized in that: Including processor and memory; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 8 when executing a program stored in a memory.

Images