A target recognition method and apparatus

Abstract

This invention discloses a target recognition method and apparatus, relating to the field of computer technology. One specific implementation of the method includes: generating multi-pose images corresponding to targets contained in a basic image dataset; adding the multi-pose images to the basic image dataset to obtain an intermediate image dataset; performing target detection on multiple images in the intermediate image dataset to obtain target detection results; using the intermediate image dataset and the target detection results as a training set, training a target recognition model based on a pre-trained benchmark recognition model; and inputting the image to be recognized into the target recognition model for target recognition. This method generates multi-pose images for each target in the basic image dataset, adds these multi-pose images to the training set to train the target recognition model, and then uses the trained model to recognize targets in various poses. The model has strong generalization ability and does not require target alignment or target orientation processing.

Description

Technical Field

[0001] This invention relates to the field of computers, and more particularly to a target recognition method and apparatus. Background Technology

[0002] Object recognition refers to the identification and classification of specific objects in an image using computer vision and image processing techniques, and it forms the basis of many computer vision tasks. In recent years, driven by deep learning technology, object recognition technology has made great progress and is widely used in fields such as video surveillance, intelligent transportation, and smart homes.

[0003] The implementation process of target recognition technology in the prior art generally includes three steps: target detection, target alignment, and target straightening. Target detection is used to obtain the rectangular box containing the target from the input image. Target alignment requires key point detection on the input image with the drawn rectangular box. Target straightening requires straightening the tilted or large-angle deflected target based on the key points.

[0004] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0005] (1) The accuracy of target recognition depends on the accuracy of target detection and target alignment. If the key point detection is inaccurate, the image obtained when the target is aligned will be incomplete. If such an image is added to the training set, the target recognition model trained later will not be able to correctly recognize the target.

[0006] (2) In order to improve the accuracy of target detection and target alignment, a high-precision model is needed. However, the high-precision model takes a long time, and target alignment has high requirements for the quality of the input image.

[0007] (3) Facial reshaping can cause certain problems such as target deformation or filling, affecting the recognition of targets in various poses. Summary of the Invention

[0008] In view of this, embodiments of the present invention provide a target recognition method and apparatus, which generates multi-pose images of each target in a basic image dataset, adds the multi-pose images to a training set to train a target recognition model, and then uses the trained model to recognize targets in various poses. The model has strong generalization ability and does not require target alignment and target orientation processing.

[0009] To achieve the above objectives, according to one aspect of the present invention, a target recognition method is provided.

[0010] An embodiment of the present invention provides a target recognition method, comprising: generating multi-pose images corresponding to targets contained in a basic image dataset; adding the multi-pose images to the basic image dataset to obtain an intermediate image dataset; performing target detection on multiple images in the intermediate image dataset to obtain target detection results; using the intermediate image dataset and the target detection results as a training set, training a target recognition model based on a pre-trained benchmark recognition model; and inputting the image to be recognized into the target recognition model to perform target recognition.

[0011] Optionally, generating multi-pose images corresponding to targets contained in the basic image dataset includes: constructing a multi-pose image dataset; classifying the images in the multi-pose image dataset according to angle and pose information; labeling the image poses according to the classification results, and then encoding the labeling results into pose control parameters based on a set encoding rule; training a multi-pose image generation model based on a generative adversarial network model using the pose control parameters as conditions; and inputting multiple images from the basic image dataset into the multi-pose image generation model to output multi-pose images corresponding to targets contained in the basic image dataset.

[0012] Optionally, the multi-pose image includes an in-plane rotated image and / or an out-of-plane rotated image; generating the corresponding multi-pose image for the target contained in the basic image dataset includes: rotating each image in the basic image dataset according to a set first rotation angle to obtain an in-plane rotated image corresponding to the target; fitting the images containing the target in the basic image dataset using a three-dimensional gradient model, rotating the fitted images according to a set second rotation angle, and projecting them onto a two-dimensional plane to obtain an out-of-plane rotated image corresponding to the target.

[0013] Optionally, before adding the multi-pose image to the base image dataset, the method further includes: extracting a first feature from the image containing the current target in the base image dataset to determine the feature center of the image; extracting a second feature from the multi-pose image corresponding to the current target to determine a first similarity between the second feature and the feature center; if the first similarity is less than or equal to a set first threshold, deleting the corresponding multi-pose image to complete image cleaning; adding the multi-pose image to the base image dataset includes: adding the cleaned multi-pose image to the base image dataset.

[0014] Optionally, before adding the multi-pose image to the base image dataset, the method further includes: calculating a second similarity between the image containing the current target in the base image dataset and the corresponding out-of-plane rotated image; if the second similarity is less than a set second threshold or greater than a set third threshold, then deleting the corresponding out-of-plane rotated image to complete image cleaning; adding the multi-pose image to the base image dataset includes: adding the cleaned out-of-plane rotated image to the base image dataset.

[0015] Optionally, after the step of performing target detection on multiple images in the intermediate image dataset, the method further includes: keeping the center of the first bounding box obtained from the target detection unchanged, and adjusting the current image in the intermediate image dataset to a set size that includes the target; before the step of inputting the image to be recognized into the target recognition model, the method further includes: performing target detection on the image to be recognized, keeping the center of the second bounding box obtained from the target detection unchanged, and adjusting the image to be recognized to the size; the step of inputting the image to be recognized into the target recognition model includes: inputting the size-adjusted image to be recognized into the target recognition model.

[0016] Optionally, the method further includes: performing target detection on multiple images of the basic image dataset and drawing bounding boxes containing the targets; performing feature point detection on the targets within the bounding boxes to obtain multiple feature points of the targets; using similarity transformation or affine transformation to achieve target orientation based on the coordinates of the feature points; and using the orientation images as a training set to train based on a deep learning algorithm to obtain the benchmark recognition model.

[0017] To achieve the above objectives, according to another aspect of the present invention, a target recognition device is provided.

[0018] An embodiment of the present invention provides a target recognition device, comprising: a generation and addition module, configured to generate multi-pose images corresponding to targets contained in a basic image dataset, and add the multi-pose images to the basic image dataset to obtain an intermediate image dataset; a target detection module, configured to perform target detection on multiple images in the intermediate image dataset to obtain target detection results; a model training module, configured to use the intermediate image dataset and the target detection results as a training set to train a target recognition model based on a pre-trained benchmark recognition model; and a target recognition module, configured to input the image to be recognized into the target recognition model for target recognition.

[0019] Optionally, the generation and addition module is further configured to: construct a multi-pose image dataset; classify the images in the multi-pose image dataset according to angle and pose information; label the image poses according to the classification results, and then encode the labeling results into pose control parameters in combination with the set encoding rules; train a multi-pose image generation model based on a generative adversarial network model using the pose control parameters as conditions; and input multiple images from the basic image dataset into the multi-pose image generation model to output multi-pose images corresponding to the targets contained in the basic image dataset.

[0020] Optionally, the multi-pose image includes an in-plane rotated image and / or an out-of-plane rotated image; the generation and addition module is further configured to: rotate the base image dataset according to a set first rotation angle to obtain an in-plane rotated image corresponding to the target; and use a three-dimensional gradient model to fit the images containing the target in the base image dataset respectively, and rotate the fitted images according to a set second rotation angle and project them onto a two-dimensional plane to obtain an out-of-plane rotated image corresponding to the target.

[0021] Optionally, the apparatus further includes: a first cleaning module, configured to extract a first feature of an image containing the current target in the basic image dataset to determine the feature center of the image; extract a second feature of a multi-pose image corresponding to the current target to determine a first similarity between the second feature and the feature center; and delete the corresponding multi-pose image if the first similarity is less than or equal to a set first threshold to complete image cleaning; the generation and addition module is further configured to add the cleaned multi-pose image to the basic image dataset.

[0022] Optionally, the device further includes: a second cleaning module, configured to calculate a second similarity between the image containing the current target in the basic image dataset and the corresponding out-of-plane rotated image; if the second similarity is less than a set second threshold or greater than a set third threshold, then the corresponding out-of-plane rotated image is deleted to complete image cleaning; the generation and addition module is further configured to add the cleaned out-of-plane rotated image to the basic image dataset.

[0023] Optionally, the device further includes: a first size adjustment module, configured to keep the center of the first bounding box obtained from target detection unchanged, and adjust the current image of the intermediate image dataset to a set size containing the target; a second size adjustment module, configured to perform target detection on the image to be recognized, keep the center of the second bounding box obtained from target detection unchanged, and adjust the image to be recognized to the set size; the target recognition module is further configured to input the size-adjusted image to be recognized into the target recognition model.

[0024] Optionally, the apparatus further includes: a benchmark recognition model training module, configured to perform target detection on multiple images of the basic image dataset respectively, draw bounding boxes containing the targets; perform feature point detection on the targets within the bounding boxes to obtain multiple feature points of the targets; perform target orientation transformation by similarity transformation or affine transformation according to the coordinates of the feature points; and use the orientation-transformed images as a training set to train the benchmark recognition model based on a deep learning algorithm.

[0025] To achieve the above objectives, according to another aspect of the present invention, an electronic device is provided.

[0026] An electronic device according to an embodiment of the present invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a target recognition method according to an embodiment of the present invention.

[0027] To achieve the above objectives, according to another aspect of the present invention, a computer-readable medium is provided.

[0028] An embodiment of the present invention provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements a target recognition method according to an embodiment of the present invention.

[0029] One embodiment of the above invention has the following advantages or beneficial effects: By generating multi-pose images for each target in the basic image dataset and adding these multi-pose images to the training set for training the target recognition model, the trained model can then be used to recognize targets in various poses. The model has strong generalization ability and does not require target alignment or target orientation processing. By using generative adversarial networks or rotating images to generate multi-pose images for each target in the basic image dataset, the problem of insufficient sample data in target recognition can be improved, and the recognition rate of targets in multi-pose situations can be increased. Cleaning the multi-pose images ensures the accuracy of the trained target recognition model, while reducing the number of images in the training set and reducing the model convergence time. By unifying the image size, the amount of computation is reduced while ensuring the integrity of image information.

[0030] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0031] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0032] Figure 1 This is a schematic diagram of the main steps of the target recognition method according to an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the main process of the target recognition method according to Embodiment 1 of the present invention;

[0034] Figure 3 This is a schematic diagram of the result of generating multi-pose images using the in-plane rotation image method in Embodiment 1 of the present invention;

[0035] Figure 4 This is a schematic diagram of the result of generating a multi-pose image using an out-of-plane rotating image method in one embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of the main process of generating multi-pose images using a generative adversarial network in Embodiment 1 of the present invention;

[0037] Figure 6 This is a schematic diagram of the main process of the face recognition method according to Embodiment 2 of the present invention;

[0038] Figure 7 This is a schematic diagram of the main modules of the target recognition device according to an embodiment of the present invention;

[0039] Figure 8 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0040] Figure 9 This is a schematic diagram of the structure of a computer device suitable for implementing electronic devices according to embodiments of the present invention. Detailed Implementation

[0041] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0042] Figure 1 This is a schematic diagram of the main steps of the target recognition method according to an embodiment of the present invention.

[0043] like Figure 1 As shown, the target recognition method of this invention mainly includes the following steps:

[0044] Step S101: Generate corresponding multi-pose images for the targets contained in the basic image dataset, and add the multi-pose images to the basic image dataset to obtain an intermediate image dataset. Using a generative adversarial network or image rotation method, generate images of each target in the basic image dataset from multiple angles; the generated images are the multi-pose images of that target. Adding the multi-pose images to the basic image dataset facilitates subsequent target detection on the resulting image dataset containing multi-pose images.

[0045] Step S102: Perform object detection on multiple images in the intermediate image dataset to obtain object detection results. An object detection model is used to perform object detection on multiple images in the intermediate image dataset to obtain bounding boxes containing the objects. The position and size of these bounding boxes in the image represent the position and size of the object in the image, respectively.

[0046] Step S103: Using the intermediate image dataset and the target detection results as the training set, train the target recognition model based on the pre-trained benchmark recognition model. The process of training the benchmark recognition model is as follows: For multiple images in the basic image dataset, perform three steps sequentially: target detection, target alignment, and target straightening, to obtain straightened images; use the straightened images as the training set, and train the model based on a deep learning algorithm to obtain the benchmark recognition model. Using the intermediate image dataset containing multi-pose images as input and the target detection results as output, train the new model based on the benchmark recognition model until convergence; the trained new model is the target recognition model.

[0047] Step S104: Input the image to be recognized into the target recognition model for target recognition. After training the target recognition model, input the image to be recognized into the target recognition model, and output the target recognition result after model processing. In this embodiment, the size of the image to be recognized can also be compressed or expanded to a set size, and the compressed or expanded image must contain the target to be recognized, thereby reducing the amount of computation while ensuring the integrity of image information.

[0048] Figure 2 This is a schematic diagram of the main process of the target recognition method according to Embodiment 1 of the present invention.

[0049] like Figure 2 As shown, the target recognition method of Embodiment 1 of the present invention mainly includes the following steps:

[0050] Step S201: Construct a basic image dataset and preprocess multiple images from the basic image dataset to train a benchmark recognition model. The training process of the benchmark recognition model includes four steps: object detection, object alignment, object straightening, and model training. Specifically: Object detection is performed on multiple images from the basic image dataset, and bounding boxes containing the objects are drawn; feature point detection is performed on the objects within the bounding boxes to obtain multiple feature points of the object; based on the coordinates of the feature points, similarity transformation or affine transformation is used to straighten the object; the straightened images are used as the training set to train a CNN (Convolutional Neural Networks) classifier. The trained CNN classifier is the benchmark recognition model. The purpose of training the benchmark recognition model is to prepare for subsequent image cleaning and object recognition model training.

[0051] In an optional embodiment, after training the CNN classifier, metric learning can be used to fine-tune the CNN classifier, and the fine-tuned CNN classifier can be used as the baseline recognition model. Additionally, during the training of the baseline recognition model, the images in the base image dataset can be standardized to a uniform size, for example, 140×140.

[0052] Step S202: Generate corresponding multi-pose images for each target contained in the basic image dataset. The images in the basic image dataset contain various targets of interest to the user, such as people, animals, and vehicles. When taking images containing these targets using portable devices, it is difficult to control the pose and angle of the targets in the images. Large-angle tilting or deflection of the targets will result in poor target alignment, making target alignment impossible and thus preventing target recognition. Therefore, in this embodiment, a corresponding multi-pose image is generated for each target, and the model is trained using this multi-pose image to achieve recognition of various pose images captured naturally. This step not only obtains training set images from multiple angles but also greatly increases the number of images in the training set.

[0053] In this embodiment, 30 multi-pose images can be generated for each target. There are various ways to generate multi-pose images, such as using a generative adversarial network (GAN). For specific implementation details, please refer to the section on... Figure 5 The following explains how to generate multi-pose images using image rotation. For example, image rotation can be used, including in-plane and out-of-plane rotation images. The following section provides a detailed explanation of how to generate multi-pose images using image rotation.

[0054] (1) Generate multi-pose images using in-plane rotation:

[0055] Assume the input image is X, (x1, y1)∈X are points on the input image, the rotation angle is α, and the output image is Y, (x2, y2)∈Y are points on the output image.

[0056]

[0057] During rotation, the rotation angle α can be ±5°, ±10°, ... ±35°. The interval between rotation angles can also be adjusted, for example, every 3°, to generate a denser in-plane rotation image. Figure 3 This is a schematic diagram of the result of generating a multi-pose image using an in-plane rotating image method in Embodiment 1 of the present invention. Figure 3 The leftmost image is the original image, the middle image is the image obtained by rotating the original image 10° clockwise, and the rightmost image is the image obtained by rotating the original image -10° counterclockwise.

[0058] (2) Generate multi-pose images using out-of-plane rotation:

[0059] The images containing the target in the basic image dataset are fitted using a three-dimensional gradient model (3DMM). Then, the fitted images are rotated in three dimensions according to the rotation angle β and projected onto a two-dimensional plane to obtain the out-of-plane rotated image of the target. Figure 4 This is a schematic diagram of the result of generating a multi-pose image using an out-of-plane rotating image method in one embodiment of the present invention. Figure 4 The leftmost image is the original image, and the other images are two-dimensional images obtained by rotating the original images according to the corresponding rotation angles.

[0060] Step S203: Clean the generated multi-pose image. Since the generated multi-pose image may not achieve the same quality level as the real image, or may exhibit significant deformation, it is necessary to clean the generated multi-pose image. In this embodiment, the following two methods are used for cleaning.

[0061] (1) For multi-pose images generated using generative adversarial networks or in-plane rotation images, the cleaning process is as follows:

[0062] (11) Use the benchmark recognition model to extract the first feature of the image containing the current target in the basic image dataset to determine the feature center of the image. Assume that there are N images of the current target in the basic image dataset. Use the benchmark recognition model to extract the features of these N images, and then calculate the mean of the N features as the feature center.

[0063] (12) Use the benchmark recognition model to extract the second feature of the multi-pose image corresponding to the current target, and determine the first similarity between the second feature and the feature center. Use the benchmark recognition model to extract the feature of the multi-pose image corresponding to the current target, and calculate the similarity S between this feature and the feature center.

[0064] (13) If the first similarity is less than or equal to the set first threshold, delete the corresponding multi-pose image to complete image cleaning. Set the first threshold, and the threshold can be dynamically adjusted according to the performance of the benchmark recognition model in the future. In the embodiment, if the similarity S≤0.2, discard the multi-pose image.

[0065] (2) For the multi-pose images generated by the method of out-of-plane rotating images, the cleaning process is as follows:

[0066] (21) Calculate the second similarity between the image containing the current target in the basic image dataset and the corresponding out-of-plane rotating image. Use methods such as cosine similarity and histogram to calculate the similarity S between the image containing the current target and the corresponding out-of-plane rotating image.

[0067] (22) If the second similarity is less than the set second threshold or greater than the set third threshold, delete the corresponding out-of-plane rotating image to complete image cleaning. Set two thresholds, namely the second threshold S1 and the third threshold S2, S1 < S2. If S < S1, it means that the generated out-of-plane rotating image is basically dissimilar to the original image, and discard the out-of-plane rotating image; if S > S2, it means that the generated out-of-plane rotating image is too homogeneous with the original image, and discard the out-of-plane rotating image; if S1≤S≤S2, keep the out-of-plane rotating image. In the embodiment, S1 is set to 0.15 and S2 is set to 0.9, and the threshold can be dynamically adjusted according to the performance of the benchmark recognition model in the future.

[0068] Step S204: Add the cleaned multi-pose images to the basic image dataset to obtain an intermediate image dataset. After cleaning the generated multi-pose images, add them to the basic image dataset to expand the number of images in the dataset.

[0069] Step S205: Perform object detection on multiple images in the intermediate image dataset respectively to obtain object detection results. In the embodiment, use an object detection model to perform object detection on multiple images in the intermediate image dataset respectively to obtain a bounding box containing the target with a fixed size (128×128); then keep the center of the bounding box unchanged and expand it to obtain an image with a size of 140×140, and no target alignment and target straightening operations are performed on the obtained image subsequently.

[0070] Step S206: Using the intermediate image dataset and object detection results as the training set, train the object recognition model based on the benchmark recognition model. Using the intermediate image dataset as input and the object detection results as output, train the new model based on the benchmark recognition model until convergence. The resulting new model is the object recognition model.

[0071] Step S207: Deploy the target recognition model into the application. Input the image to be recognized into the target recognition model for target recognition. After deploying the trained target recognition model into the application, subsequent target recognition only requires providing the original image to be recognized for target detection, obtaining a bounding box containing the target, and then adjusting the image to a specified size before inputting it into the target recognition model to obtain the target recognition result. No target alignment or orientation operations are required. In this embodiment, after target detection, a 128×128 bounding box containing the target can be obtained. This bounding box is expanded to include the entire target before being input into the target recognition model to obtain the target recognition result.

[0072] Figure 5 This is a schematic diagram of the main process of generating multi-pose images using a generative adversarial network in Embodiment 1 of the present invention. Figure 5 As shown, the process of generating multi-pose images using a generative adversarial network in Embodiment 1 of the present invention includes:

[0073] Step S501: Construct a multi-pose image dataset and classify the images in the dataset according to their angle and pose information. Collect multi-pose images, perform preprocessing such as mean subtraction and normalization on the multi-pose images, and then classify the multi-pose images of the same target according to a set angle, for example, each 15° is considered as a category.

[0074] Step S502: Label the image poses according to the classification results, and then encode the labeled results into pose control parameters according to the set encoding rules. The labeling method is as follows: from -45° to 45°, every 15° represents a pose, labeled as poses 0 to 6. The pose control parameters are 7-dimensional one-hot vectors, and the encoding rule is: if pose i (i∈{0, 1, ..., 6}), then the (i+1)th element of the corresponding pose control parameter is 1, and the other elements are 0. One-hot encoding uses a code structure where the number of bits corresponds to the number of states, and only one bit is 1 while all others are 0.

[0075] Step S503: Using the pose control parameters as conditions, train the generator network G and the discriminator network D based on the generative adversarial network (GAN) model. The GAN consists of two deep neural networks, G and D. The generator network G simulates the original data to generate images with a distribution similar to the original data. The discriminator network D distinguishes whether the input image comes from the generator network G or the original image data. The two networks can be optimized using a minimax objective function. The trained generator network G can generate images with various poses based on the pose control parameters, while the discriminator network D extracts features from the input image.

[0076] Step S504: Input the random vector and attitude control parameters into the generator network G to obtain images of the same target in different poses. In this embodiment, a uniform distribution on [-1, 1] is randomly sampled to obtain a random vector z. Different vectors z represent different targets. Based on the vector z and the attitude control parameters, the generator network G trained in step S503 is used to obtain images of the same target in various poses.

[0077] Step S505: Use randomly sampled images in different poses as the training set to train an encoder for extracting image identity features. The trained encoder and the generator network G constitute a multi-pose image generation model. In this embodiment, the input vector z0 is randomly sampled under a uniform distribution in [-1, 1]. An element at a random position within the range of 1 to 7 is set to 1, and the rest are set to 0, resulting in the input pose control parameter y0. z0 and y0 are input into the generator network G to obtain the sampled image output Q. The encoder's network structure is the same as the discrimination network D, except that the last fully connected layer is changed to have the same dimension as the input vector z0. The output vector of the fully connected layer and the input pose control parameter y0 are input into the generator network G to obtain the corresponding image output Q0. By optimizing the loss function, the image output Q0 is made as close as possible to the sampled image output Q, and the trained encoder can obtain the identity features of the input image.

[0078] Step S506: Input multiple images from the basic image dataset into the multi-pose image generation model to output multi-pose images corresponding to the targets contained in the basic image dataset. In this embodiment, each image in the basic image dataset obtains identity features through the encoder trained in step S505, and adds pose control parameters. Then, through the generator network G trained in step S503, images of different targets in various poses are obtained.

[0079] In Example 2, the base image dataset is a face image dataset, including multiple face images. The aforementioned target recognition method is applicable to face recognition. This example addresses the problem in existing technologies where the success rate of face image recognition is low due to arbitrary shooting angles, resulting in large-angle deflections. To solve this problem, this example adds a large number of multi-pose images to the training set, making the trained face recognition model more generalizable and capable of handling naturally captured multi-pose images. The following section combines... Figure 6 Example 2 will be described in detail.

[0080] Figure 6 This is a schematic diagram of the main process of the face recognition method according to Embodiment 2 of the present invention.

[0081] like Figure 6 As shown, the face recognition method of Embodiment 1 of the present invention mainly includes the following steps:

[0082] Step S601: Construct a face image dataset. Preprocess all face images in the dataset and use the preprocessed face images as the training set to train the baseline recognition model. The training method for the baseline recognition model is consistent with the training method for traditional face recognition models. For the images in the face image dataset, preprocessing is performed using three steps: face detection, face alignment, and face straightening, resulting in a 140×140 straightened face image. This straightened face image is then used as the training set to train a CNN classifier; the trained CNN classifier is the baseline recognition model.

[0083] Step S602: Generate corresponding multi-pose face images for each person in the face image dataset. One of the main reasons existing face recognition models require face image alignment and repositioning is that the training data is insufficient to encompass images in various poses, resulting in models trained using these data being unable to recognize naturally captured images in various poses. In this embodiment, training images from various angles are added to the training set, enabling the trained model to recognize face images from various angles without requiring face alignment and repositioning.

[0084] Each image in a face image dataset can include one or more faces. In the case of multiple faces, the goal is to detect the ground truth faces. This requires extracting the ground truth faces from the image. Then, generative adversarial networks (GANs) or image rotation can be used to generate corresponding multi-pose face images for the ground truth faces. Here, "ground truth" refers to the labels. For example, if image A contains the faces of Zhang San, Li Si, and Wang Wu, and the label is Zhang San, then we need to extract Zhang San's face from image A. During classifier training, Li Si and Wang Wu will be treated as noise. The following describes the implementation process of generating multi-pose face images from an image containing only one face.

[0085] (1) Construct a multi-pose face image dataset and classify the face images in the multi-pose face image dataset according to the angle and pose information;

[0086] (2) Label the image pose according to the classification results, and then encode the labeling results into pose control parameters in combination with the set encoding rules;

[0087] (3) Using the attitude control parameters as conditions, train the generative network G and the discriminative network D based on the conditional generative adversarial network (CGAN) model;

[0088] (4) Input the random vector and attitude control parameters into the generator network G to obtain images of the same person in different poses.

[0089] (5) Use randomly sampled images in different poses as the training set to train an encoder for extracting image identity features. The network structure of the encoder is the same as that of the discriminator network D, except that the last fully connected layer is changed to have the same dimension as the random sampling.

[0090] (6) Input each face image from the face image dataset into the multi-pose image generation model to obtain the multi-pose face image corresponding to each face in the face image dataset. The trained encoder and the generator network G constitute the multi-pose image generation model. The face image is processed by the trained encoder to obtain identity features, and then the pose control parameters are added. The trained generator network G is then used to obtain images of different faces in various poses.

[0091] Step S603: Clean the generated multi-pose face images. The specific implementation process for this step is detailed in step S203.

[0092] Step S604: Add the cleaned multi-pose face images to the face image dataset to obtain an intermediate image dataset. After cleaning the generated multi-pose face images, add them to the face image dataset to expand the number of images in the face dataset.

[0093] Step S605: Perform face detection on multiple face images in the intermediate image dataset to obtain face detection results. In this embodiment, a face detection model is used to perform face detection on multiple face images in the intermediate image dataset to obtain a 128×128 face bounding box; then, keeping the center of the face bounding box fixed, it is expanded to obtain a 140×140 face image. After obtaining this face image, no further face alignment or face straightening operations are performed.

[0094] Step S606: Using the intermediate image dataset and face detection results as the training set, train the face recognition model based on the baseline recognition model. Using the intermediate image dataset as input and the face detection results as output, train the new model based on the baseline recognition model until convergence. The resulting new model is the face recognition model. After training the face recognition model, if retraining is required subsequently, face alignment and face straightening operations are unnecessary; training can be performed directly based on this face recognition model.

[0095] Step S607: Deploy the face recognition model into the application. Input the face image to be recognized into the face recognition model to perform face recognition. After deploying the trained face recognition model into the application, when performing face recognition subsequently, only the original face image to be recognized needs to be provided for face detection to obtain a 128×128 face bounding box. Then, the image to be recognized is expanded to include the entire head. The expanded image is then input into the face recognition model to output the face recognition result.

[0096] As can be seen from the target recognition method of this invention, by generating multi-pose images of each target in the basic image dataset and adding these multi-pose images to the training set for training the target recognition model, the trained model can then be used to recognize targets in various poses. The model has strong generalization ability and does not require target alignment or orientation processing. By using generative adversarial networks or rotating images to generate multi-pose images of each target in the basic image dataset, the problem of insufficient sample data in target recognition can be improved, and the recognition rate of targets in multi-pose situations can be increased. Cleaning the multi-pose images ensures the accuracy of the trained target recognition model, while reducing the number of images in the training set and reducing the model convergence time. By unifying the image size, the amount of computation is reduced while ensuring the integrity of image information.

[0097] Figure 7 This is a schematic diagram of the main modules of a target recognition device according to an embodiment of the present invention.

[0098] like Figure 7 As shown, the target recognition device 700 of this embodiment mainly includes:

[0099] The generation and addition module 701 is used to generate corresponding multi-pose images for targets contained in the basic image dataset, and add the multi-pose images to the basic image dataset to obtain an intermediate image dataset. Using a generative adversarial network or image rotation method, images of each target in the basic image dataset are generated from multiple angles; the generated images are the multi-pose images of that target. Adding the multi-pose images to the basic image dataset facilitates subsequent target detection on the resulting image dataset containing multi-pose images.

[0100] The object detection module 702 is used to perform object detection on multiple images in the intermediate image dataset to obtain object detection results. An object detection model is used to perform object detection on multiple images in the intermediate image dataset to obtain bounding boxes containing the objects. The position and size of these bounding boxes in the image represent the position and size of the object in the image, respectively.

[0101] The model training module 703 is used to train the target recognition model based on the pre-trained benchmark recognition model, using the intermediate image dataset and the target detection results as the training set. The process of training the benchmark recognition model is as follows: for multiple images in the basic image dataset, target detection, target alignment, and target straightening are processed sequentially to obtain straightened images; these straightened images are then used as the training set to train the benchmark recognition model using a deep learning algorithm. Alternatively, the intermediate image dataset containing multi-pose images can be used as input, and the target detection results as output. A new model is trained based on the benchmark recognition model until convergence; the trained new model is the target recognition model.

[0102] The target recognition module 704 is used to input the image to be recognized into the target recognition model for target recognition. After the target recognition model is trained, the image to be recognized is input into the target recognition model, and the target recognition result can be output after the model processes it. In the embodiment, the size of the image to be recognized can also be compressed or expanded to a set size, and the compressed or expanded image must contain the target to be recognized, thereby reducing the amount of computation while ensuring the integrity of the image information.

[0103] In addition, the target recognition device 700 of this embodiment may further include: a first cleaning module, a second cleaning module, a first size adjustment module, a second size adjustment module, and a reference recognition model training module. Figure 7(Not shown in the image). The system includes: a first cleaning module for cleaning multi-pose images generated using generative adversarial networks or in-plane rotation; a second cleaning module for cleaning multi-pose images generated using out-of-plane rotation; a first size adjustment module for adjusting the image size of the intermediate image dataset; a second size adjustment module for adjusting the size of the image to be recognized; and a benchmark recognition model training module for training the benchmark recognition model. The specific implementations of these modules are as described above.

[0104] As can be seen from the above description, by generating multi-pose images of each target in the basic image dataset and adding these multi-pose images to the training set for training the target recognition model, the trained model can then be used to recognize targets in various poses. The model has strong generalization ability and does not require target alignment or orientation processing. Using generative adversarial networks or rotating images to generate multi-pose images of each target in the basic image dataset can improve the problem of insufficient sample data in target recognition and improve the recognition rate of targets in multi-pose situations. Cleaning the multi-pose images ensures the accuracy of the trained target recognition model while reducing the number of images in the training set and reducing the model convergence time. By unifying the image size, the amount of computation is reduced while ensuring the integrity of image information.

[0105] Figure 8 An exemplary system architecture 800 is shown that can be applied to the target recognition method or target recognition device of the present invention.

[0106] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, and 803, a network 804, and a server 805. Network 804 serves as the medium for providing communication links between terminal devices 801, 802, and 803 and server 805. Network 804 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0107] Users can use terminal devices 801, 802, and 803 to interact with server 805 via network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0108] Terminal devices 801, 802, and 803 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0109] Server 805 can be a server that provides various services, such as a backend management server that supports the basic image dataset and images to be recognized provided by the administrator (for example only). The backend management server can generate multi-pose images of the targets in the basic image dataset, perform target detection, model training, target recognition, and other processing, and feed back the processing results (such as target recognition results) to the terminal device.

[0110] It should be noted that the target recognition method provided in this application embodiment is generally executed by server 805, and correspondingly, the target recognition device is generally set in server 805.

[0111] It should be understood that Figure 8 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0112] According to embodiments of the present invention, the present invention also provides an electronic device and a computer-readable medium.

[0113] The electronic device of the present invention includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the target recognition method of the embodiments of the present invention.

[0114] The present invention provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the target recognition method of the present invention.

[0115] The following is for reference. Figure 9 It shows a schematic diagram of the structure of a computer system 900 suitable for implementing an electronic device according to embodiments of the present invention. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0116] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 902 or programs loaded from storage section 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the system 900. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0117] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed.

[0118] In particular, according to the embodiments disclosed in this invention, the processes described in the above main step diagrams can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit (CPU) 901, it performs the functions defined in the system of this invention.

[0119] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0120] The main step diagrams and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0121] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including a generation and addition module, an object detection module, a model training module, and an object recognition module. The names of these modules do not necessarily limit the module itself; for example, the generation and addition module can also be described as "a module that generates corresponding multi-pose images of objects contained in a basic image dataset, adds the multi-pose images to the basic image dataset, and obtains an intermediate image dataset."

[0122] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to include: generating corresponding multi-pose images of targets contained in a basic image dataset; adding the multi-pose images to the basic image dataset to obtain an intermediate image dataset; performing target detection on multiple images in the intermediate image dataset to obtain target detection results; using the intermediate image dataset and the target detection results as a training set, training a target recognition model based on a pre-trained benchmark recognition model; and inputting the image to be recognized into the target recognition model to perform target recognition.

[0123] As can be seen from the above description, by generating multi-pose images of each target in the basic image dataset and adding these multi-pose images to the training set for training the target recognition model, the trained model can then be used to recognize targets in various poses. The model has strong generalization ability and does not require target alignment or target orientation processing.

[0124] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0125] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0126] All actions involving the acquisition of information, signals, or data in this application are carried out in accordance with the relevant data protection regulations of the country where the application is located and with the authorization granted by the owner of the relevant device.

Claims

1. A target recognition method, characterized in that, include: Generative adversarial networks or rotated images are used to generate corresponding multi-pose images of the targets contained in the basic image dataset; The multi-pose image includes an out-of-plane rotated image; Extract the first feature of the image containing the current target in the basic image dataset to determine the feature center of the image; extract the second feature of the multi-pose image corresponding to the current target to determine the first similarity between the second feature and the feature center; if the first similarity is less than or equal to a set first threshold, delete the corresponding multi-pose image. Calculate the second similarity between the image containing the current target and the corresponding out-of-plane rotated image in the base image dataset; If the second similarity is less than the set second threshold or greater than the set third threshold, the corresponding out-of-plane rotated image is deleted to complete the image cleaning. The cleaned out-of-plane rotated image is added to the base image dataset to obtain the intermediate image dataset; Target detection is performed on multiple images in the intermediate image dataset to obtain target detection results; Keeping the center of the first bounding box obtained from the target detection unchanged, adjust the current image of the intermediate image dataset to the set size that includes the target; Using the intermediate image dataset and the target detection results as the training set, a target recognition model is trained based on the pre-trained benchmark recognition model. Perform target detection on the image to be recognized, keep the center of the second bounding box obtained from the target detection unchanged, and adjust the image to be recognized to the specified size; The resized image to be recognized is input into the target recognition model for target recognition.

2. The method according to claim 1, characterized in that, The base image dataset contains target-generated multi-pose images, including: Construct a multi-pose image dataset and classify the images in the multi-pose image dataset according to angle and pose information; The image pose is labeled based on the classification results, and then the labeled results are encoded into pose control parameters in accordance with the set encoding rules. Using the aforementioned attitude control parameters as conditions, a multi-pose image generation model is trained based on a generative adversarial network model. Multiple images from the base image dataset are input into the multi-pose image generation model to output multi-pose images corresponding to the target contained in the base image dataset.

3. The method according to claim 1, characterized in that, The multi-pose image also includes an in-plane rotated image; The base image dataset contains target-generated multi-pose images, including: The basic image dataset is rotated separately according to the set first rotation angle to obtain the in-plane rotated image corresponding to the target; The images containing the target in the basic image dataset are fitted using a three-dimensional gradient model. The fitted images are then rotated according to a set second rotation angle and projected onto a two-dimensional plane to obtain the out-of-plane rotated image of the target.

4. The method according to claim 1, characterized in that, The method further includes: Target detection is performed on multiple images in the basic image dataset, and bounding boxes containing the targets are drawn. Feature point detection is performed on the target within the bounding box to obtain multiple feature points of the target; Based on the coordinates of the feature points, the target can be oriented using similarity transformation or affine transformation. The corrected images are used as the training set, and the model is trained based on a deep learning algorithm to obtain the benchmark recognition model.

5. A target recognition device, characterized in that, include: The generation and addition module is used to generate corresponding multi-pose images of targets contained in the base image dataset by using generative adversarial networks or rotating images. The multi-pose image includes an out-of-plane rotated image; a first feature of the image containing the current target in the basic image dataset is extracted to determine the feature center of the image; a second feature of the multi-pose image corresponding to the current target is extracted to determine the first similarity between the second feature and the feature center; if the first similarity is less than or equal to a set first threshold, the corresponding multi-pose image is deleted. Calculate the second similarity between the image containing the current target and the corresponding out-of-plane rotated image in the base image dataset; If the second similarity is less than the set second threshold or greater than the set third threshold, the corresponding out-of-plane rotated image is deleted to complete the image cleaning; the cleaned out-of-plane rotated image is added to the basic image dataset to obtain the intermediate image dataset; The target detection module is used to perform target detection on multiple images in the intermediate image dataset to obtain target detection results; Keeping the center of the first bounding box obtained from the target detection unchanged, adjust the current image of the intermediate image dataset to the set size that includes the target; The model training module is used to train the target recognition model based on the pre-trained benchmark recognition model, using the intermediate image dataset and the target detection results as the training set. The target recognition module is used to perform target detection on the image to be recognized, keep the center of the second bounding box obtained by the target detection unchanged, adjust the image to be recognized to the size, and input the size-adjusted image to be recognized into the target recognition model for target recognition.

6. The apparatus according to claim 5, characterized in that, The generation and addition module is also used for: Construct a multi-pose image dataset and classify the images in the multi-pose image dataset according to angle and pose information; The image pose is labeled based on the classification results, and then the labeled results are encoded into pose control parameters in accordance with the set encoding rules. Using the aforementioned attitude control parameters as conditions, a multi-pose image generation model is trained based on a generative adversarial network model. as well as Multiple images from the base image dataset are input into the multi-pose image generation model to output multi-pose images corresponding to the target contained in the base image dataset.

7. The apparatus according to claim 5, characterized in that, The multi-pose image also includes an in-plane rotated image; the generation and addition module is further used for: The basic image dataset is rotated separately according to the set first rotation angle to obtain the in-plane rotated image corresponding to the target; as well as The images containing the target in the basic image dataset are fitted using a three-dimensional gradient model. The fitted images are then rotated according to a set second rotation angle and projected onto a two-dimensional plane to obtain the out-of-plane rotated image of the target.

8. The apparatus according to claim 5, characterized in that, The device further includes: a benchmark recognition model training module, used for... Target detection is performed on multiple images in the basic image dataset, and bounding boxes containing the targets are drawn. Feature point detection is performed on the target within the bounding box to obtain multiple feature points of the target; Based on the coordinates of the feature points, similarity transformation or affine transformation is used to orient the target; and The corrected images are used as the training set, and the model is trained based on a deep learning algorithm to obtain the benchmark recognition model.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

Images