An object similarity recognition method based on a deep learning neural network

Abstract

The application provides an object similarity recognition method based on a deep learning neural network S10: collecting pre-training object pictures and storing them according to categories; S20: constructing an object similarity recognition model, and performing first-stage object similarity recognition model training and evaluation on a certain type of object sample; S30: performing model quantization optimization on the generated first-stage object similarity recognition model, and constructing a second-stage object similarity recognition model; S40: object similarity recognition model prediction; the application can support a small sample recognition algorithm, can be used in object classification similarity recognition and object individual similarity recognition scenes, is convenient and fast for docking into two object recognition scenes of detection and classification, realizes efficient transfer learning in multiple business scenes, reduces the prediction calculation amount and the model volume, facilitates model acceleration operation, and ensures that the model method generated by the application can be deployed for reasoning in multiple places.

Description

Technical Field

[0001] This invention relates to the field of object similarity recognition technology, and in particular to an object similarity recognition method based on deep learning neural networks. Background Technology

[0002] Currently, there are three main types of methods for object similarity recognition: geometric feature-based methods, template-based methods, and model-based methods.

[0003] Geometric feature-based methods are the earliest and most traditional, and usually need to be combined with other algorithms to achieve better results. Template-based methods can be divided into relevance matching methods, eigenface methods, linear discriminant analysis methods, singular value decomposition methods, neural network methods, and dynamic connection matching methods. Model-based methods include those based on hidden Markov models, active shape models, and active appearance models.

[0004] Existing object similarity recognition methods suffer from the following main problems: 1. They require a large amount of recognition and annotation data to support the accuracy training and prediction of the recognition algorithm, especially when new items are added, as the amount of data for these new items is very limited, making it difficult to support detailed parameter adjustments for algorithm learning. 2. Recognition methods are difficult to efficiently adapt and deploy on server and mobile devices, and large model algorithms are inefficient on mobile devices. 3. Recognition methods are difficult to efficiently transfer learn across multiple business scenarios; for example, methods for face recognition are difficult to quickly transfer to vehicle recognition scenarios. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing an object similarity recognition method based on deep learning neural networks.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for object similarity recognition based on deep learning neural networks includes the following steps:

[0008] S10: Collect images of pre-trained items and store them in a catalog according to their categories;

[0009] The collected pre-training item images are stored in two levels: item category similarity and item individual similarity.

[0010] S20: Construct an object similarity recognition model, and conduct the first stage of object similarity recognition model training and evaluation for a certain type of item sample;

[0011] Obtain the pre-trained PP-LCNetV2 model and pre-training parameters, use PP-LCNetV2 as the backbone model for object recognition, remove the last fully connected layer in the model network hierarchy, add multiple fully connected softmax layers, and conduct the first stage of object similarity recognition model training and evaluation for a certain type of item samples.

[0012] S30: Perform model quantization optimization on the generated first-stage object similarity recognition model, and construct the second-stage object similarity recognition model;

[0013] The first-stage object similarity recognition model is converted from float32 format to int8 format to reduce computation and model size. The opt tool is used to build the second-stage object similarity recognition model.

[0014] S40: Object similarity recognition model prediction;

[0015] The cosine distance of the included angle is used to measure the difference in the direction of the 128-dimensional vector output by the object similarity recognition model in the second stage, thereby identifying the similarity differences between different object samples in the same category.

[0016] Furthermore, step S1 specifically includes:

[0017] S101: General image preprocessing to generate training data;

[0018] S102: Data preprocessing at different levels;

[0019] (1) For training data for similarity recognition of item categories, the category should be used as the directory name, and the item images should be placed in the corresponding category directory according to the corresponding category;

[0020] (2) For training data for individual item similarity recognition, the individual name should be used as the directory name, and the item images should be placed in the corresponding individual directory with the corresponding individual name.

[0021] Furthermore, step S20 specifically includes:

[0022] S201: Obtain the pre-trained PP-LCNetV2 model and pre-training parameters;

[0023] S202: The model network with the last fully connected (FC) layer removed from the PP-LCNetV2 model network is used as the backbone network of the object similarity recognition model;

[0024] S203: Add multiple fully connected softmax layers as the head prediction network of the object similarity recognition model to form the object similarity recognition model;

[0025] S204: Using the training data preprocessed in S10, a deep neural network is trained and evaluated on the object similarity recognition model to generate the first-stage object similarity recognition model of the present invention.

[0026] Furthermore, step S30 specifically includes:

[0027] S301: Compress the first stage of the object similarity recognition model;

[0028] Within the PaddleSlim framework, the model path, distillation, quantization, and training parameters are configured in the config file. The first-stage object similarity recognition model calls the interface paddleslim.auto_compression.AutoCompression to perform quantization and distillation on the first-stage object similarity recognition model.

[0029] S302: Optimize the compressed model and output the second-stage object similarity recognition model;

[0030] The model optimization tool opt is used to complete the optimization steps and output a lightweight, optimal executable model.

[0031] Furthermore, the multi-layer fully connected softmax consists of three fully connected layers with hidden layers of 1024, 512, and 128 neurons, respectively. The output of the final object similarity recognition model is a 128-dimensional vector.

[0032] Furthermore, step S40 includes the following steps:

[0033] S401: Obtain the 128-dimensional vector output by the object similarity recognition model in the second stage;

[0034] S402: Calculate the cosine similarity of the obtained 128-dimensional vectors;

[0035] S403: Compare the calculated cosine similarity with the corresponding distance threshold to predict the object similarity recognition model.

[0036] Furthermore, the formula for calculating the cosine similarity of the obtained 128-dimensional vectors is as follows:

[0037] Where A and B are 128-dimensional vectors output by the second-stage object similarity recognition model, respectively, vector A(A1,A2,A3…A…). n ) and vector B(B1,B2,B3…B n ), n is 128.

[0038] Furthermore, general image preprocessing includes resizing the image to a uniform pixel size, random cropping the image, normalizing the image pigments, erasing random areas of the image, and filling the surrounding pixels of the image.

[0039] Furthermore, the optimization steps in step S302 include quantization, subgraph fusion, hybrid scheduling, and kernel optimization.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention uses the pre-trained PP-LCNetV2 model and pre-trained parameters in the Baidu PaddlePaddle AI model library as the basis for subsequent model customization and fine-tuning parameter fine-tuning. The input image is only fine-tuned for the fully connected layer neurons, without the need to readjust the parameters of the algorithm for recognition learning. During use, a large amount of recognition and labeling data is not required to support the accuracy training and prediction recognition of the recognition algorithm. Recognition can also be achieved when there is very little data on the new item scene when new items are added. (2) The paddleslim and opt tools are used to optimize the object similarity recognition model in the first stage, such as quantization, subgraph fusion, hybrid scheduling, and kernel optimization, reducing the prediction computation and model size. This facilitates the accelerated operation of the model and the deployment of inference on the mobile terminal and other end-users, ensuring that the model method generated by the present invention can be deployed for inference in multiple places, including servers and mobile cloud terminals, etc. (3) By collecting and preprocessing images, hierarchical storage is used to enable the model to solve problems at different levels. For hierarchical data preprocessing, it can be used in scenarios such as object classification similarity recognition and individual object similarity recognition, which can be conveniently and quickly connected to the two major object recognition scenarios of detection and classification. According to the input of preprocessed image data of different levels and targets, the model can be fine-tuned to achieve image classification and recognition of different targets. Through hierarchical classification and recognition, it is ensured that the method of the present invention can achieve efficient transfer learning in multiple business scenarios. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the steps of an object similarity recognition method based on a deep learning neural network according to the present invention.

[0042] Figure 2 This is a diagram showing the overall network structure of the PP-LCNetV2 model, which is a method for object similarity recognition based on deep learning neural networks according to the present invention.

[0043] Figure 3 This is a diagram of the multi-layer fully connected softmax network structure of an object similarity recognition method based on deep learning neural networks according to the present invention. Detailed Implementation

[0044] To provide a further understanding of the purpose, structure, features, and functions of the present invention, detailed descriptions are provided below with reference to specific embodiments.

[0045] A method for object similarity recognition based on deep learning neural networks includes the following steps:

[0046] S10: Collect images of pre-trained items and store them in a catalog according to their categories;

[0047] Similarity recognition is performed at two levels: similarity in item categories and similarity in individual items. Training for recognition at different levels requires collecting training data for the corresponding level and performing corresponding data preprocessing.

[0048] Specifically, it includes:

[0049] S101: General image preprocessing to generate training data;

[0050] General image preprocessing includes resizing the image to 224*224 pixels, random cropping, pigment normalization, random region erasure, and pixel filling around the image. Training data is generated after general image preprocessing.

[0051] S102: Data preprocessing at different levels;

[0052] (1) For training data for similarity recognition of item classification, the category should be used as the directory name, and the item images should be placed in the corresponding category directory according to the corresponding category.

[0053] (2) For training data for individual item similarity recognition, the individual name should be used as the directory name, and the item images should be placed in the corresponding individual directory with the corresponding individual name.

[0054] Hierarchical storage is designed to enable models to solve problems at different levels, allowing for comparisons between individuals within different or the same category. Hierarchical data preprocessing facilitates applications in object classification and individual object similarity recognition, providing convenient and quick integration with object detection and classification scenarios. For example, high-level similarity recognition can be used for different animal levels, while low-level similarity recognition can be used for recognizing different human faces.

[0055] S20: Construct an object similarity recognition model, and conduct the first stage of object similarity recognition model training and evaluation for a certain type of item sample;

[0056] Obtain the pre-trained PP-LCNetV2 model and pre-training parameters from the AI ​​model library, use PP-LCNetV2 as the backbone model for object recognition, remove the last fully connected layer in the model network hierarchy, add multiple fully connected softmax layers, and perform the first stage of object similarity recognition model training and evaluation for a certain type of item.

[0057] S201: Obtain the pre-trained PP-LCNetV2 model and pre-training parameters from the Baidu PaddlePaddle AI model library.

[0058] The pre-trained PP-LCNetV2 model and its corresponding pre-training parameters are derived from training on a large number of images and are stored in the Baidu PaddlePaddle AI model library. This invention uses the pre-trained PP-LCNetV2 model and its pre-training parameters as a basis for subsequent model customization and fine-tuning.

[0059] S202: The model network of the PP-LCNetV2 model network with the last FC fully connected layer removed from the hierarchy is used as the backbone network of the model of the present invention;

[0060] This invention uses the PP-LCNetV2 lightweight model from Baidu PaddlePaddle AI model library as the backbone model for object recognition. A specific backbone network PP-LCNet is designed for Intel CPU devices and its acceleration library MKLDNN. Compared with other lightweight state-of-the-art (SOTA) models, this backbone network can further improve the performance of the model without increasing inference time, and ultimately significantly surpass the existing SOTA models.

[0061] The overall network structure of the PP-LCNetV2 model is as follows: Figure 2 As shown, the PP-LCNetV2 model is an optimization based on PP-LCNetV1. It mainly uses a reparameterization strategy to combine depthwise convolutions with different kernel sizes, and optimizes pointwise convolutions, shortcuts, etc.

[0062] S203: Add multiple fully connected softmax layers as the head prediction network of the model of this invention, forming an object similarity recognition model (backbone + head).

[0063] A multi-layer fully connected softmax network structure is as follows: Figure 3 As shown, the multi-layer fully connected softmax consists of three fully connected layers with hidden layers of 1024, 512, and 128 neurons respectively. Thus, the output of the final object similarity recognition model is a 128-dimensional vector.

[0064] S204: Using the training data preprocessed in S10, a deep neural network is trained and evaluated on the object similarity recognition model (backbone+head) to generate the first-stage object similarity recognition model of the present invention.

[0065] Using training data from images collected by S10, the model parameters before the fully connected layers are frozen, and fine-tuning is performed only on the neurons in the fully connected layers.

[0066] This invention uses the pre-trained PP-LCNetV2 model and pre-trained parameters from the Baidu PaddlePaddle AI model library as a basis for subsequent model customization and fine-tuning. For the input image, fine-tuning training is only performed on the neurons of the fully connected layer, without the need to readjust the parameters of the algorithm for recognition learning. During use, a large amount of recognition and annotation data is not required to support the accuracy training and prediction recognition of the recognition algorithm. Recognition can also be achieved when there is very little data in the scene of the new item being added.

[0067] S30: Perform model quantization optimization on the generated first-stage object similarity recognition model, and construct the second-stage object similarity recognition model.

[0068] The first-stage object similarity recognition model is converted from float32 format to int8 format to reduce computation and model size. The opt tool is then used to build the second-stage object similarity recognition model.

[0069] S301: Compress the first stage of the object similarity recognition model;

[0070] Within the PaddleSlim framework of Baidu PaddlePaddle's classification component, the parameters for model path, distillation, quantization, and training are configured in the config file. In the first stage, the object similarity recognition model calls the interface paddleslim.auto_compression.AutoCompression to perform quantization and distillation on the first stage of the object similarity recognition model.

[0071] S302: Optimize the compressed model and output the second-stage object similarity recognition model;

[0072] The opt tool is used to automate the optimization process and output a lightweight, optimal executable model for efficient prediction and execution on both edge and cloud sides. The optimization process includes methods such as quantization, subgraph fusion, hybrid scheduling, and kernel optimization.

[0073] The second-stage object similarity recognition model is a lightweight model distilled and quantized from the first-stage object similarity recognition model. This facilitates faster model operation and deployment and inference on mobile devices and other endpoints, ensuring that the model method generated by this invention can be deployed and inferred in multiple locations, including servers, mobile devices, and cloud platforms.

[0074] S40: Object similarity recognition model prediction;

[0075] The cosine distance of the included angle is used to measure the difference in the direction of the 128-dimensional vector output by different objects for the second stage of the object similarity recognition model, thereby identifying the similarity differences of different object samples in the same category, and completing the server and edge cloud deployment of a certain type of object similarity recognition model method.

[0076] The specific steps of the object similarity recognition model prediction are as follows:

[0077] S401: Obtain the 128-dimensional vector output by the object similarity recognition model in the second stage;

[0078] The second-stage generated object similarity recognition model is used to predict the similarity of individual input objects. For example, two driver face images can be transformed into two 128-dimensional vectors, denoted as vector A(A1, A2, A3…A…). n ) and vector B(B1,B2,B3…B n This achieves the transformation of the data dimension space representation;

[0079] S402: Calculate the cosine similarity of the obtained 128-dimensional vectors;

[0080] The cosine similarity of the obtained 128-dimensional vector is calculated according to the following formula. The difference in the direction of the output 128-dimensional vector of different objects for the object recognition model is measured by the calculated cosine distance of the angle.

[0081] n is 128.

[0082] S403: Compare the cosine distance of the included angle for object similarity recognition with the corresponding distance threshold, thereby enabling the object similarity recognition model to predict.

[0083] Compare the cosine distance of the included angle calculated in S402 with the distance threshold of the corresponding object. If the distance is less than the distance threshold, it means that the two input images are highly similar and can be identified as the same object.

[0084] Distance thresholds are obtained through validation with a large amount of data. For example, the distance threshold between different human faces is a threshold value (e.g., 0.6) given after comparing a huge number of faces and segmenting them with a certain percentage success rate (e.g., 95%). If the input is two images of faces, and the calculated cosine distance between the two images is less than 0.6, it can be considered as an image of the same person's face.

[0085] This invention, by inputting preprocessed image data of different levels and targets and fine-tuning the model, can achieve image classification and recognition of different targets. The involved model method can create a foundational core algorithm for general commercial vehicle object similarity recognition. Subsequent mining of information from driver face recognition, commercial vehicle recognition, and vehicle load recognition will provide visual object recognition and detection capabilities for multiple business object recognition scenarios, accelerating the rapid deployment of object similarity recognition methods in the following business scenarios:

[0086] 1. Multi-object tracking and detection algorithms: fast object identification and generation of corresponding unique IDs in object detection methods.

[0087] 2. Supports scenarios such as driver facial recognition, driver check-in, and DMS active safety.

[0088] 3. Multi-attribute identification of various objects such as vehicles and people, quickly recognizing vehicle brand, color, and other scene requirements.

[0089] 4. Vehicle load recognition in the field of vision, efficiently filtering suspected overloaded vehicles.

[0090] 5. For algorithmic support of small sample recognition, a feature network is constructed through the first-stage feature backbone model.

[0091] The present invention has been described by the above-described embodiments; however, these embodiments are merely examples for implementing the present invention. It must be noted that the disclosed embodiments do not limit the scope of the present invention. Conversely, any modifications and refinements made without departing from the spirit and scope of the present invention are within the scope of patent protection of the present invention.

Claims

1. An object similarity recognition method based on deep learning neural networks, characterized in that: Includes the following steps: S10: Collect images of pre-trained items and store them in a catalog according to their categories; The collected pre-training item images are stored in two levels: item category similarity and item individual similarity. S20: Construct an object similarity recognition model, and conduct the first stage of object similarity recognition model training and evaluation for a certain type of item sample; Obtain the pre-trained PP-LCNetV2 model and pre-training parameters, use PP-LCNetV2 as the backbone model for object recognition, remove the last fully connected layer in the model network hierarchy, add multiple fully connected softmax layers, and conduct the first stage of object similarity recognition model training and evaluation for a certain type of item samples. S201: Obtain the pre-trained PP-LCNetV2 model and pre-training parameters; S202: The model network with the last fully connected (FC) layer removed from the PP-LCNetV2 model network is used as the backbone network of the object similarity recognition model; S203: Add multiple fully connected softmax layers as the head prediction network of the object similarity recognition model to form the object similarity recognition model; S204: Using the training data preprocessed in S10, a deep neural network is trained and evaluated for the object similarity recognition model to generate the first-stage object similarity recognition model. S30: Perform model quantization optimization on the generated first-stage object similarity recognition model, and construct the second-stage object similarity recognition model; The first-stage object similarity recognition model is converted from float32 format to int8 format to reduce computation and model size. The opt tool is used to build the second-stage object similarity recognition model. S301: Compress the first stage of the object similarity recognition model; Within the PaddleSlim framework, the model path, distillation, quantization, and training parameters are configured in the config file. The first-stage object similarity recognition model calls the interface paddleslim.auto_compression.AutoCompression to perform quantization and distillation on the first-stage object similarity recognition model. S302: Optimize the compressed model and output the second-stage object similarity recognition model; The model optimization tool opt is used to complete the optimization steps and output a lightweight, optimal executable model. S40: Object similarity recognition model prediction; The cosine distance of the included angle is used to measure the difference in the direction of the 128-dimensional vector output by the object similarity recognition model in the second stage, thereby identifying the similarity differences between different object samples in the same category.

2. The object similarity recognition method based on deep learning neural networks as described in claim 1, characterized in that: Step S1 specifically includes: S101: General image preprocessing to generate training data; S102: Data preprocessing at different levels; (1) For training data for similarity recognition of item categories, the category should be used as the directory name, and the item images should be placed in the corresponding category directory according to the corresponding category; (2) For training data for individual item similarity recognition, the individual name should be used as the directory name, and the item images should be placed in the corresponding individual directory with the corresponding individual name.

3. The object similarity recognition method based on deep learning neural networks as described in claim 1, characterized in that: The multi-layer fully connected softmax consists of three fully connected layers with hidden layers of 1024, 512, and 128 neurons respectively. The output of the final object similarity recognition model is a 128-dimensional vector.

4. The object similarity recognition method based on deep learning neural networks as described in claim 1, characterized in that: Step S40 includes the following steps: S401: Obtain the 128-dimensional vector output by the object similarity recognition model in the second stage; S402: Calculate the cosine similarity of the obtained 128-dimensional vectors; S403: Compare the calculated cosine similarity with the corresponding distance threshold to predict the object similarity recognition model.

5. The object similarity recognition method based on deep learning neural networks as described in claim 4, characterized in that: The formula for calculating the cosine similarity of the obtained 128-dimensional vectors is as follows: Where A and B are 128-dimensional vectors output by the second-stage object similarity recognition model, respectively, vector A(A1,A2,A3…A…). n ) and vector B(B1,B2,B3…B n ), n is 128.

6. The object similarity recognition method based on deep learning neural networks as described in claim 2, characterized in that: General image preprocessing includes resizing images to a uniform pixel size, random cropping, pigment normalization, random area erasure, and pixel filling around the image.

7. The object similarity recognition method based on deep learning neural networks as described in claim 1, characterized in that: The optimization steps in step S302 include quantization, subgraph fusion, hybrid scheduling, and kernel optimization.

Images