Multi-pose image data generation method and device for face recognition, equipment and storage medium

Abstract

The invention discloses a multi-pose image data generation method and device for face recognition, equipment and a storage medium, and relates to the technical field of face image generation. The method comprises the following steps: obtaining a face data set of a specific scene and adjusting a face image generation model to obtain a new generation model; generating a new face data set by using the new model, and extracting feature vectors through a potential vector generator to form a potential vector set; performing face detection and screening on the new data set, classifying potential vectors according to a posture estimation angle, and labeling an angle value as a label; training the potential vector with the label by using a support vector machine to obtain decision boundaries of different attitude angles; and performing linear interpolation on the potential vector based on the decision boundary, and generating face images at different attitude angles by using the generation model. According to the method, more accurate and robust face image generation is realized, and the method has important significance in improving the accuracy and efficiency of multi-pose face recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of face image generation, and in particular relates to a method, device, equipment and storage medium for generating multi-posture image data for face recognition. Background Art

[0002] With the development of computer vision and deep learning technology, face image generation has become a research hotspot. However, existing face image generation methods often only focus on the generation of random data. For face recognition training, a single image does not have the ability to improve the recognition rate of face recognition in practical applications, ignoring the requirement of face recognition for generating images with single targets and multiple poses. Therefore, it is of great significance to develop a method that can generate multiple face images with different face poses based on different pose boundaries for a single face image to improve the recognition rate of face recognition in practical applications. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for generating multi-posture image data for face recognition, which realize active early warning braking of the side blind spot of a semi-trailer car train, so as to improve the active safety of the semi-trailer car train.

[0004] In a first aspect, an embodiment of the present invention provides a method for generating multi-pose image data for face recognition, which is applied to face image generation, and the method flow is as follows:

[0005] S1. Obtain face dataset information for a specific scene, use the face image generation model to perform fine-tuning, map the model space to the new face data sample space, and obtain a new generation model;

[0006] S2. Generate a new face dataset using the new generative model. Capture the details and global structure of the input image through the set latent vector generator, extract the directional features to form a feature vector that represents the face image, and form a latent vector set.

[0007] S3, perform face detection on the face image data in the generated new face data set, screen according to the face detection result score, further perform threshold screening and classification on the faces that meet the requirements in combination with the posture estimation angle in the face detection result, divide the latent vectors of the face images that meet the requirements into multiple groups of face image latent vectors according to different posture angle direction threshold ranges, and use the corresponding positive and negative values ​​of the face angle as the label of each group of latent vectors;

[0008] S4, using the latent vector label and the latent vector of the face image as training data, using SVM to perform a linear classification operation with the largest interval in the feature space to obtain the decision boundaries of different face posture angles;

[0009] S5. Use the decision boundary to perform linear interpolation on the face image latent vector generated by the generative model to form a new face image latent vector, and then use the generative model in S1 to generate multiple photos of a person at different postures and angles.

[0010] Preferably, in step S1, the face data set of a specific scene includes multiple face posture images of visible light and multiple face posture images of near infrared collected from various cameras;

[0011] The face image generation model includes an image generator and an image discriminator;

[0012] The model is adjusted based on the pre-trained model on a large dataset, and the face dataset of a specific scene is used to further adjust the model parameters, mapping the model space to the new face data sample space to obtain a new generative model.

[0013] Preferably, in step S2, the latent vector set is a latent vector set Z of the face image formed by capturing the details and global structure of the input image by the generative model in S1 through the set latent vector generator in the generated image to form a feature vector representing the face image.

[0014] Preferably, the specific steps of step S3 are as follows:

[0015] S31. Perform face detection on the face image data in the generated new face dataset, and the obtained face score should meet the following conditions:

[0016]

[0017] Among them, I c is a set of face images that meet the requirements; I is the face dataset to be detected; score is the face score in the face detection result; threshold is the set score threshold, and the corresponding potential vector set of face images is

[0018] S32, further screening and classifying the face pose estimation angles for the face image set that meets the requirements, in order to obtain the expected number of faces with different poses, the face pose angles include three directions of pitch, yaw, and roll for face pose estimation, and the range of the angles in each direction meets the following conditions:

[0019] pitch angle ,|angle|∈{0,5,10,15,20,25};

[0020] yaw angle ,|angle|∈{0,5,10,15,20,25};

[0021] roll angle ,|angle|∈{0,5,10,15,20,25};

[0022] Among them, pitch angle , yaw angle 、roll angle are the angle ranges included in the three directions of pitch, yaw, and roll respectively; the multiple sets of face potential vectors corresponding to the three directions of pitch, yaw, and roll are:

[0023]

[0024] Among them, the angle value is the set angle range; dir represents the three directions of pitch, yaw, and roll;

[0025] S33, using the corresponding positive and negative values ​​of the face angle as the label of each group of potential vectors, that is, in the process of performing face posture estimation angle screening and classification in S32, using the positive and negative values ​​of the current face posture angle as the classification label basis, the following conditions should be met:

[0026]

[0027] in, Labels for filtering and classifying face pose estimation angles in three directions: pitch, yaw, and roll. The angle value is the set angle range. dir represents the three directions of pitch, yaw, and roll.

[0028] Preferably, the latent vector label and the latent vector of the face image in step S4 are used as training data, which are composed as follows:

[0029]

[0030] Among them, T dir The training set for the three posture estimation directions needs to be carried out for the next step of training; the angle value is the set angle range; dir represents the three directions of pitch, yaw, and roll;

[0031] It involves processing the potential vector labels and the potential vectors of the face image. Specifically, step S4 uses a support vector machine to perform a linear classification operation with the largest interval in the feature space to obtain the decision boundaries of the face at different posture angles. These decision boundaries include the directional features of the current face in the three directions of pitch, yaw, and roll at the posture estimation angle, as shown below:

[0032]

[0033] Among them, the angle value is the set angle range; dir represents the three directions of pitch, yaw, and roll.

[0034] Preferably, step S5 uses the decision boundary to linearly interpolate the latent vector of the face image generated by the generative model to form a new latent vector of the face image, and then uses the generative model in S1 to generate multiple photos of a person at different postures and angles. In this process, the latent vector of the face image is first generated by the generative model, and then the decision boundary learned by the SVM algorithm is used to linearly interpolate these latent vectors, wherein the process of interpolating the new latent vector of the face image is as follows:

[0035]

[0036] Where; L is an arithmetic progression from start_d to end_d, with a total of steps elements; is the transpose of the decision boundary;

[0037] Through interpolation, new latent vectors are generated in the latent vector space, which correspond to face images with different posture angles. Finally, these new latent vectors are input into the generative model in S1 to generate the corresponding face images.

[0038] In a second aspect, an embodiment of the present invention provides a multi-pose image data generation device for face recognition, which is used to store a multi-pose image data generation method for face recognition; the device includes a data collection module, a model adjustment module, an image generation module, a detection and screening module, a classification module and an interpolation image generation module;

[0039] The data collection module is used to collect and prepare face datasets for specific scenarios;

[0040] The model adjustment module is used to adjust and train the generated model;

[0041] The image generation module is used to generate new face images;

[0042] The detection and screening module is used to realize face detection and posture estimation functions;

[0043] The classification module is used to classify features using SVM;

[0044] The interpolation image generation module is used to complete linear interpolation and generate new images.

[0045] In a third aspect, an embodiment of the present invention provides a multi-pose image data generation device for face recognition, which is applied to execute a multi-pose image data generation method for face recognition.

[0046] In a fourth aspect, an embodiment of the present invention further provides a storage medium for generating multi-pose image data for face recognition, which is used to store a method for generating multi-pose image data for face recognition.

[0047] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0048] 1. The present invention can meet the needs of face recognition in specific scenarios. By pre-training on a large dataset and then fine-tuning the dataset for a specific scenario, the model's adaptability and generalization ability to specific scenario data are improved. By using generative adversarial networks (GANs) to generate high-quality face images, the latent vector generator ensures the consistency of the details and global structure of the generated images.

[0049] 2. The present invention ensures that the generated face images cover different posture angles through posture estimation and classification, thereby improving the multi-posture adaptability of the face recognition system; through threshold screening of face detection and posture estimation, the quality and diversity of the generated images are ensured, while reducing unnecessary consumption of computing resources;

[0050] 3. The present invention reduces a lot of unfriendly face images to a great extent by using the scores and posture angle estimates in the face detection results, thereby ensuring the quality of face images. At the same time, using the positivity and negativity of the angle as the label of the SVM training sample also greatly reduces the difficulty of manual intervention in the labeling. Due to the use of linear division of the SVM decision boundary, the convergence time of SVM training can be greatly reduced, while ensuring that the obtained decision boundary has better robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A flow chart of a method for generating multi-pose facial image data for face recognition according to the present invention is shown. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Example 1

[0055] With the development of computer vision and deep learning technology, face image generation has become a research hotspot. However, existing face image generation methods often only focus on the generation of random data. For face recognition training, a single image does not have the ability to improve the recognition rate of face recognition in practical applications, and ignores the requirement of face recognition for generating single-target multi-pose images. Therefore, this embodiment develops a multi-pose face image data generation method for face recognition, thereby achieving more accurate and robust face image generation, which is of great significance to improving the accuracy and efficiency of multi-pose face recognition.

[0056] See also Figure 1 As shown, a method for generating multi-pose face image data for face recognition in this embodiment comprises the following steps:

[0057] S1. Obtain face dataset information for a specific scene, use the face image generation model to perform fine-tuning, map the model space to the new face data sample space, and obtain a new generation model.

[0058] S2. Generate a new face dataset using the new generative model, capture the details and global structure of the input image through the set latent vector generator, extract directional features to form a feature vector that represents the face image, and form a latent vector set.

[0059] S3. Perform face detection on the face image data in the generated new face data set, and screen them according to the result scores of the face detection. The faces that meet the requirements are further threshold screened and classified based on the posture estimation angle in the face detection results. The latent vectors of the face images that meet the requirements are divided into multiple groups of face image latent vectors according to different posture angle direction threshold ranges, and the corresponding positive and negative values ​​of the face angles are used as labels for each group of latent vectors.

[0060] S4. The latent vector label and the latent vector of the face image are used as training data, and SVM is used to perform a linear classification operation with the largest interval in the feature space to obtain the decision boundaries of different face posture angles.

[0061] S5. Use the decision boundary to perform linear interpolation on the latent vector of the face image generated by the generative model to obtain multiple photos of a person at different postures and angles.

[0062] In step S1, this example collects multiple facial pose images of visible light and multiple facial pose images of near-infrared from various cameras, which are set as subsets of the target face database. The face image generation model of stylegan3, which includes an image generator and an image discriminator, is adopted. According to the requirements, the collected face data set is preprocessed to the specified size of 1024*124, and the preprocessed face data set is fine-tuned based on the stylegan3 model pre-trained on a large data set, and the model space is mapped to the new face data sample space to obtain a new generation model stylegan3_s. This approach can reduce the training time of the model while obtaining a model with the same features as the custom data.

[0063] In step S2, the new image generator model stylegan3_s obtained in step S1 is used to generate a new face image dataset. In this example, in order to ensure that the subsequent steps can have a good effect, the generated face image data is set to 50k pictures, forming a new face image dataset I. At the same time, the latent vector generator set by stylegan3_s is used to capture the details and global structure of the input image, and form a high-dimensional vector that represents the face image, forming a latent vector set Z of the face image.

[0064] The new face image dataset I and the corresponding face image latent vector set Z obtained in step S1 and step S2 need to be further screened to filter out unqualified faces and corresponding latent vectors. In step S3 of this example, the following steps are performed:

[0065] S31 performs face detection on the face image data in the generated new face data set, and the obtained face score should meet the following conditions:

[0066]

[0067] Among them, I c is a set of face images that meet the requirements; I is the face dataset to be detected; score is the face score in the face detection result; threshold is the set score threshold, and the corresponding potential vector set of face images is

[0068] In this example, it is recommended to set the threshold value to 0.9. c A collection of face images that meet the requirements.

[0069] Further, in step S32, the face image set I that meets the requirements is cFurther screening and classification of face pose estimation angles are performed to obtain more faces with different poses. The face pose angles should include the three directions of pitch, yaw, and roll of face pose estimation. The range of angles in each direction should meet the following conditions:

[0070] pitch angle ,|angle|∈{0,5,10,15,20,25};

[0071] yaw angle ,|angle|∈{0,5,10,15,20,25};

[0072] roll angle ,|angle|∈{0,5,10,15,20,25};

[0073] Where pitch angle , yaw angle 、roll angle The angle range included in the three directions of pitch, yaw, and roll of posture estimation. Since the true value of the angle has the attribute of positive and negative values, the angle range mentioned in the formula of the above condition refers to the angle range after the absolute value. Furthermore, while obtaining the angle range included in the three directions of pitch, yaw, and roll of posture estimation, multiple groups of face latent vectors corresponding to each angle range are also screened and extracted and saved. The resulting multiple groups of face latent space sets are shown below:

[0074]

[0075] Among them, the angle value is the set angle range, and dir represents the three directions of pitch, yaw, and roll.

[0076] Then further, in step S33, according to the positive and negative value attributes of the true value of the face angle, this example uses the positive and negative values ​​of each face angle as the classification label basis for each group of potential vectors, and the labels should meet the following conditions:

[0077]

[0078] in, Labels for filtering and classifying face pose estimation angles in three directions: pitch, yaw, and roll. The angle value is the set angle range. dir represents the three directions of pitch, yaw, and roll.

[0079] Furthermore, for the face latent space set obtained in step S3: And the corresponding classification label set: The training data set is composed as follows:

[0080]

[0081] Among them, T dir The training set for the three posture estimation directions needs to be trained in the next step. The angle value is the set angle range, and dir represents the three directions of pitch, yaw, and roll. Further, this step uses the support vector machine (SVM) to perform the linear classification operation with the largest interval in the feature space. Through this operation process, the decision boundaries of the face at different posture angles are obtained. These decision boundaries contain the directional features of the current face in the three directions of pitch, yaw, and roll at the posture estimation angle.

[0082]

[0083] Among them, the angle value is the set angle range, and dir represents the three directions of pitch, yaw, and roll. This enables more accurate and robust face image generation, which is of great significance for improving the accuracy and efficiency of multi-pose face recognition.

[0084] Finally, the decision boundary is used to linearly interpolate the face image latent vector generated by the generative model to form a new face image latent vector. It is worth noting that the process of interpolating the new face image latent vector is as follows:

[0085]

[0086] Among them, L is an arithmetic progression from start_d to end_d, with a total of steps elements. is the transpose of the decision boundary. In this embodiment, the values ​​of start_d and end_d in the arithmetic sequence L are -3 and 3 respectively, and the value of steps is 10. According to the formula, the face potential vector is first calculated. Decision boundaries of faces at different poses and angles The purpose of this step is to adjust the arithmetic sequence L so that it is relative to and Finally, the face potential vector And the adjusted arithmetic sequence L and Perform interpolation. Here, each value in the adjusted arithmetic sequence L will be interpolated with Multiply the corresponding columns in and add the result to In this way, each sample in the adjusted arithmetic sequence L will follow Linear interpolation is performed in the defined direction, and the interpolation distance or step size is determined by the value in the adjusted arithmetic sequence L. Through interpolation, a new latent vector can be generated in the latent vector space This vector contains the face images corresponding to these latent vectors at different posture angles. Finally, these new latent vectors are input into the stylegan3_s model described in S1 to generate the corresponding face images. In this embodiment, since the value of steps is 10, 10 face photos in this direction will be generated, and these photos have different direction angles. This method can generate a series of face images with continuously changing posture angles, providing rich training data for applications such as face recognition.

[0087] A multi-posture image data generation device for face recognition, comprising a data collection module, a model adjustment module, an image generation module, a detection and screening module, a classification module and an interpolation image generation module;

[0088] The data collection module is used to collect and prepare face datasets for specific scenarios;

[0089] The model adjustment module is used to adjust and train the generated model;

[0090] The image generation module is used to generate new face images;

[0091] The detection and screening module is used to realize face detection and posture estimation functions;

[0092] The classification module is used to classify features using SVM;

[0093] The interpolation image generation module is used to complete linear interpolation and generate new images.

[0094] The beneficial effects of this embodiment are as follows: by pre-training on a large data set and then fine-tuning the data set for a specific scene, the adaptability and generalization ability of the model to data of a specific scene are improved; high-quality face images are generated using a generative adversarial network, and the latent vector generator ensures the consistency of the details and global structure of the generated image; through posture estimation and classification, it is ensured that the generated face images cover different posture angles, and the multi-posture adaptability of the face recognition system is improved; through threshold screening of face detection and posture estimation, the quality and diversity of the generated images are ensured, while unnecessary consumption of computing resources is reduced; using a support vector machine for classification, an accurate decision boundary can be obtained; by generating new images in the latent vector space through linear interpolation, the image data set is effectively expanded, so that the model can also have good performance at unseen posture angles.

[0095] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0096] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for generating multi-pose image data for face recognition, characterized in that: The process is as follows: S1. Obtain face dataset information for a specific scene, use the face image generation model to perform fine-tuning, map the model space to the new face data sample space, and obtain a new generation model; S2, using the new generative model to generate a new face data set, capturing the details and global structure of the input image through the set latent vector generator, extracting directional features to form feature vectors that characterize the face image, and forming a latent vector set; S3, performing face detection on the face image data in the generated new face data set, screening according to the result score of the face detection, further screening and classifying the faces that meet the requirements based on the threshold value of the posture estimation angle in the face detection result, dividing the latent vectors of the face images that meet the requirements into multiple groups of face image latent vectors according to different posture angle direction threshold ranges, and taking the corresponding positive and negative values ​​of the face angle as the label of each group of latent vectors; S4, using the latent vector label and the latent vector of the face image as training data, using SVM to perform a linear classification operation with the largest interval in the feature space to obtain decision boundaries at different face posture angles; S5. Use the decision boundary to perform linear interpolation on the facial image latent vector generated by the generative model to form a new facial image latent vector, and then use the generative model described in S1 to generate multiple photos of a person at different postures and angles.

2. The method for generating multi-pose image data for face recognition according to claim 1, characterized in that: In step S1, the face data set of the specific scene includes multiple face posture images of visible light and multiple face posture images of near infrared collected from various cameras; The face image generation model includes an image generator and an image discriminator; The model adjustment is based on a model pre-trained on a large dataset, and a face dataset of a specific scene is used to further adjust the parameters of the model, mapping the model space to a new face data sample space to obtain a new generative model.

3. The method for generating multi-pose image data for face recognition according to claim 1, characterized in that: In step S2, the latent vector set is the feature vector that represents the face image, and forms the latent vector set Z of the face image. The generative model in S1 captures the details and global structure of the input image through the set latent vector generator in the generated image to form a latent vector set Z of the face image.

4. The method for generating multi-pose image data for face recognition according to claim 1, characterized in that: The specific steps of step S3 are as follows: S31, perform face detection on the face image data in the generated new face data set, and the obtained face score meets the following conditions: Among them, I c is a set of face images that meet the requirements; I is the face dataset to be detected; score is the face score in the face detection result; threshold is the set score threshold, and the corresponding potential vector set of face images is S32, further screening and classifying the face pose estimation angles for the face image set that meets the requirements, in order to obtain the expected number of faces with different poses, the face pose angles include three directions of pitch, yaw, and roll for face pose estimation, and the range of the angles in each direction meets the following conditions: pitch angle ,|angle|∈{0,5,10,15,20,25}; yaw angle ,|angle|∈{0,5,10,15,20,25}; roll angle ,|angle|∈{0,5,10,15,20,25}; Among them, pitch angle , yaw angle 、roll angle are the angle ranges included in the three directions of pitch, yaw, and roll respectively; the multiple sets of face potential vectors corresponding to the three directions of pitch, yaw, and roll are: Among them, the angle value is the set angle range; dir represents the three directions of pitch, yaw, and roll; S33, the positive and negative values ​​of the corresponding face angles are used as labels for each group of potential vectors, that is, in the process of face posture estimation angle screening and classification in S32, the positive and negative values ​​of the current face posture angles are used as the classification label basis, which should meet the following conditions: in, Labels for filtering and classifying face pose estimation angles in three directions: pitch, yaw, and roll. The angle value is the set angle range. dir represents the three directions of pitch, yaw, and roll.

5. The method for generating multi-pose image data for face recognition according to claim 1, characterized in that: The latent vector label and the latent vector of the face image described in step S4 are used as training data, which are composed as follows: Among them, T dir The training set for the three posture estimation directions needs to be carried out for the next step of training; the angle value is the set angle range; dir represents the three directions of pitch, yaw, and roll; It involves processing the potential vector labels and the potential vectors of the face image. Specifically, step S4 uses a support vector machine to perform a linear classification operation with the largest interval in the feature space to obtain the decision boundaries of the face at different posture angles. These decision boundaries include the directional features of the current face in the three directions of pitch, yaw, and roll at the posture estimation angle, as shown below: Among them, the angle value is the set angle range; dir represents the three directions of pitch, yaw, and roll.

6. The method for generating multi-pose image data for face recognition according to claim 1, characterized in that: Step S5 uses the decision boundary to perform linear interpolation on the face image latent vector generated by the generative model to form a new face image latent vector, and then uses the generative model described in S1 to generate multiple photos of a person at different postures and angles. In this process, the latent vector of the face image is first generated by the generative model, and then the decision boundary learned by the SVM algorithm is used to perform linear interpolation on these latent vectors, wherein the process of interpolating the new face image latent vector is as follows: Where; L is an arithmetic progression from start_d to end_d, with a total of steps elements; is the transpose of the decision boundary; Through interpolation, new latent vectors are generated in the latent vector space, and these latent vectors correspond to face images with different posture angles. Finally, these new latent vectors are input into the generation model described in S1 to generate the corresponding face images.

7. A multi-pose image data generation device for face recognition, characterized in that: A method for generating multi-pose image data for face recognition according to any one of claims 1 to 6 is used to store the data; the device comprises a data collection module, a model adjustment module, an image generation module, a detection and screening module, a classification module and an interpolation image generation module; The data collection module is used to collect and prepare face datasets for specific scenarios; The model adjustment module is used to adjust and train the generated model; The image generation module is used to generate new face images; The detection and screening module is used to realize face detection and posture estimation functions; The classification module is used to classify features using SVM; The interpolation image generation module is used to complete linear interpolation and generate new images.

8. A multi-pose image data generation device for face recognition, characterized in that: Applicable to executing a method for generating multi-pose image data for face recognition as described in any one of claims 1 to 6.

9. A storage medium for generating multi-pose image data for face recognition, characterized in that: The method for generating multi-pose image data for face recognition according to any one of claims 1 to 6 is applied to store the data.

Images