System and method for face pre-recognition and image reconstruction

Abstract

The present application relates to a kind of face pre-identification and image reconstruction system and method, for the face alignment link of face recognition system, the method includes the following steps: the quality of face image obtained by face alignment link is judged;Corresponding face image reconstruction model is established based on generative adversarial network;When the judgment result is unqualified, the face image is input into the established face image reconstruction model, and the reconstructed face image is obtained and output.The present application is applied after face alignment link, and before face recognition, first, the quality of face image is judged, when quality is judged as unqualified, face image is reconstructed by generative adversarial network, compared with the practice of filling constant, can obtain the image result of higher robustness of feature vector, improve the recognition accuracy of face recognition system, avoid affecting product performance and experience.

Description

Technical Field

[0001] This invention relates to the field of information and communication engineering, and specifically to a system and method for face pre-recognition and image reconstruction. Background Technology

[0002] In the face alignment process of a face recognition system, several key points are typically predicted by a key point detection model, then matched with a template to calculate an affine matrix, and finally resampled from the entire image to obtain an aligned fixed-size face image, which serves as the input for subsequent face recognition models.

[0003] If the face region is located at the edge of the entire image or the face pose angle is large, the face image obtained after affine transformation is prone to the problem of "coordinate out-of-bounds" pixel sampling points. Generally, this problem is solved by "zero padding" or "mean padding," that is, replacing the pixel values ​​that are outside the image range (out-of-bounds) with zero or the image mean. This results in the aligned face image having obvious jagged edges with artificial marks, which can easily lead to a decrease in face recognition score and affect product performance and user experience. Summary of the Invention

[0004] The purpose of this invention is to overcome the defects of the prior art and provide a system and method for face pre-recognition and image reconstruction, which solves the problem that the existing practice of directly padding the aligned face image with zeros or averages when it crosses the boundary causes obvious jagged artificial marks on the edges, which in turn leads to a decrease in face recognition score and affects product performance and user experience.

[0005] The technical solution to achieve the above objectives is:

[0006] This invention provides a method for face pre-recognition and image reconstruction, used in the face alignment stage of a face recognition system. The method includes the following steps:

[0007] The quality of the face images obtained during the face alignment process is assessed.

[0008] A corresponding face image reconstruction model is established based on generative adversarial networks;

[0009] If the judgment result is unqualified, the face image is input into the established face image reconstruction model to obtain the reconstructed face image and output it.

[0010] This invention is applied after the face alignment stage and before face recognition. It first judges the quality of the face image. When the quality is deemed unqualified, the face image is reconstructed through a generative adversarial network. Compared with the method of filling constants, it can obtain image results with higher feature vector robustness, improve the recognition accuracy of the face recognition system, and avoid affecting product performance and user experience.

[0011] A further improvement of the face pre-recognition and image reconstruction method of the present invention is that the step of judging the quality of the face image includes:

[0012] The face image is input into the face recognition system to extract the corresponding face feature vector;

[0013] Set an index box, and use the index box to randomly and repeatedly set the corresponding feature values ​​in the face image to zero, thereby obtaining multiple face-similar images;

[0014] The obtained facial similarity images are input into the facial recognition system to extract the corresponding facial similarity feature vectors;

[0015] Calculate the similarity index between face similarity feature vectors;

[0016] Determine whether the similarity index is higher than a preset value. If it is higher than the preset value, the determination result is qualified; if it is not higher than the preset value, the determination result is unqualified.

[0017] A further improvement of the face pre-recognition and image reconstruction method of the present invention lies in calculating the similarity index between face similarity feature vectors and face feature vectors using the following formula:

[0018]

[0019] Where Val represents the similarity index, N represents the number of similar face images, X represents the face feature vector, and Xi represents the face similarity feature vector, with values ​​ranging from 1 to N.

[0020] A further improvement of the face pre-recognition and image reconstruction method of the present invention lies in the step of establishing a corresponding face image reconstruction model based on a generative adversarial network, which includes:

[0021] Provide a normal face alignment image, and use a mask to crop a random region from the provided normal face alignment image to obtain the cropped image;

[0022] The cropped image and its corresponding mask are used as training data.

[0023] The training data is fed into a generative network to obtain a generated image;

[0024] The face recognition system is used as a discrimination network to discriminate the generated images.

[0025] The generative network and the discriminative network are trained alternately until convergence, thus obtaining a generative adversarial network.

[0026] A further improvement of the face pre-recognition and image reconstruction method of the present invention is that, before judging the quality of the face image, it further includes:

[0027] The number of out-of-bounds points in the face alignment process was counted.

[0028] The total number of out-of-bounds points obtained from the statistics is compared with the set value. If it is lower than the set value, the quality of the face image is not judged. If it is higher than the set value, the quality of the face image is judged.

[0029] This invention also provides a system for face pre-recognition and image reconstruction, used in the face alignment stage of a face recognition system, the system comprising:

[0030] The image quality judgment unit is used to judge the quality of the face image obtained in the face alignment process and obtain the judgment result;

[0031] A face image reconstruction model based on generative adversarial networks; and

[0032] An image processing unit, connected to the image quality judgment unit and the face image reconstruction model, is used to input the face image into the face image reconstruction model when the judgment result is unqualified, so as to obtain and output the reconstructed face image.

[0033] A further improvement of the face pre-recognition and image reconstruction system of the present invention is that the image quality judgment unit includes a feature vector acquisition module, a similar image establishment module, a similarity index calculation module, and a quality judgment module;

[0034] The feature vector acquisition module is used to input the face image into the face recognition system to obtain the corresponding face feature vector;

[0035] The similar image creation module is connected to the feature vector acquisition module and is used to randomly and repeatedly set the corresponding feature values ​​in the face image to zero using a set index box to obtain multiple face similar images.

[0036] The feature vector acquisition module is also used to input the face similarity image into the face recognition system to obtain the corresponding face similarity feature vector;

[0037] The similarity index calculation module is connected to the feature vector acquisition module and is used to calculate the similarity index based on the face similarity feature vector and the face feature vector.

[0038] The quality judgment module is connected to the similarity index calculation module and is used to determine whether the similarity index is higher than a preset value. If it is higher than the preset value, the judgment result is qualified; if it is not higher than the preset value, the judgment result is unqualified.

[0039] A further improvement of the face pre-recognition and image reconstruction system of the present invention is that the similarity index calculation module calculates the similarity index using the following formula:

[0040]

[0041] Where Val represents the similarity index, N represents the number of similar face images, X represents the face feature vector, and Xi represents the face similarity feature vector, with values ​​ranging from 1 to N.

[0042] A further improvement of the face pre-recognition and image reconstruction system of the present invention is that the face image reconstruction model is obtained by training a generative adversarial network. The training data for training the generative adversarial network includes a cropped image obtained by cropping a normal face-aligned image using a mask to randomly crop a region and the corresponding mask. The discriminant network in the generative adversarial network is the face recognition system.

[0043] A further improvement of the face pre-recognition and image reconstruction system of the present invention is that it further includes a pre-judgment unit connected to the image quality judgment unit;

[0044] The prediction unit is used to count the number of out-of-bounds points in the face alignment process and compare the total number of out-of-bounds points with a set value. If the total number is lower than the set value, the face image quality is not judged; if the total number is higher than the set value, the face image quality is judged. Attached Figure Description

[0045] Figure 1 This is a flowchart of the face pre-recognition and image reconstruction method of the present invention.

[0046] Figure 2 This is a system diagram of the face pre-recognition and image reconstruction system of the present invention. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0048] See Figure 1This invention provides a system and method for face pre-recognition and image reconstruction, addressing the problem of jagged, artificial marks on face image edges caused by constant filling of boundary points during face alignment, leading to reduced face recognition scores and impacting product performance and user experience. The system and method are applied between face alignment and face recognition. After face alignment, the face image undergoes quality assessment. If the quality is satisfactory, no further processing is performed, and the satisfactory face image is directly input into the face recognition system for recognition. If the quality is unsatisfactory, the face image is reconstructed. Reconstruction using a generative adversarial network (GAN) improves the quality of the face image. Inputting the reconstructed face image into the face recognition system enhances its accuracy. The system and method for face pre-recognition and image reconstruction of this invention will be described below with reference to the accompanying drawings.

[0049] See Figure 2 The diagram below shows a system diagram of the face pre-recognition and image reconstruction system of the present invention. Figure 2 The present invention describes the face pre-recognition and image reconstruction system.

[0050] like Figure 2 As shown, the face pre-recognition and image reconstruction system of the present invention is used in the face alignment stage of a face recognition system. The system includes an image quality judgment unit 21, a face image reconstruction model 22, and an image processing unit 23. The image quality judgment unit 21 and the face image reconstruction model 22 are both connected to the image processing unit 23. The image quality judgment unit 21 is used to judge the quality of the face image obtained in the face alignment stage and obtain the judgment result. The face image reconstruction model 22 is established based on a generative adversarial network. When the judgment result is unqualified, the image processing unit 23 is used to input the face image into the face image reconstruction model to obtain the reconstructed face image and output it.

[0051] Preferably, the face recognition system includes a face alignment module and a face recognition module. The system of the present invention is connected between the face alignment module and the face recognition module, receives the face image output by the face alignment module, and judges the quality of the face image. When the judgment result is qualified, the image processing unit 23 directly inputs the face image to the face recognition module for face recognition. When the judgment result is unqualified, the image processing unit 23 reconstructs the face image and inputs the reconstructed face image to the face recognition module for face recognition.

[0052] In one specific embodiment of the present invention, the image quality judgment unit of the present invention includes a feature vector acquisition module, a similar image establishment module, a similarity index calculation module, and a quality judgment module;

[0053] The feature vector acquisition module is used to input face images into the face recognition system to obtain the corresponding face feature vectors;

[0054] The similar image creation module is connected to the feature vector acquisition module. It is used to randomly and repeatedly set the corresponding feature values ​​in the face image to zero using a set index box in order to obtain multiple face similar images.

[0055] The feature vector acquisition module is also used to input similar face images into the face recognition system to obtain the corresponding similar face feature vectors;

[0056] The similarity index calculation module is connected to the feature vector acquisition module and is used to calculate the similarity index based on the face similarity feature vector and the face feature vector.

[0057] The quality judgment module is connected to the similarity index calculation module to determine whether the similarity index is higher than the preset value. If it is higher than the preset value, the judgment result is qualified; if it is not higher than the preset value, the judgment result is unqualified.

[0058] Preferably, the preset value is 0.6. When the similarity index is higher than 0.6, the image quality is considered acceptable, and when it is lower than 0.6, the image quality is considered unacceptable.

[0059] Furthermore, the similarity index calculation module calculates the similarity index using the following formula:

[0060]

[0061] Where Val represents the similarity index, N represents the number of similar face images, X represents the face feature vector, and Xi represents the face similarity feature vector, with values ​​ranging from 1 to N.

[0062] This invention assesses image quality by calculating the robustness index Val of random vector similarity, comprehensively considering both timeliness and accuracy. The principle behind using Val to assess image quality is as follows: A face recognition module in a face recognition system pre-identifies a face image to obtain corresponding face feature vectors. Then, an index box is randomly placed on the face image, and the feature value within the index box is set to zero. This process is repeated N times to obtain N similar face images. These images are then input into the face recognition module to obtain corresponding similar feature vectors, X1, X2, ..., XN. The cosine similarity between these N similar feature vectors and the face feature vectors is then compared using a preset value to determine whether the image quality is acceptable.

[0063] In one specific embodiment of the present invention, the face image reconstruction model is obtained by training a generative adversarial network. The training data for training the generative adversarial network includes a cropped image obtained by cropping a normal face-aligned image using a mask to randomly crop a region and the corresponding mask. The discriminant network in the generative adversarial network is a face recognition system.

[0064] The face image reconstruction model of this invention only needs to be trained when the system is established, and can be used directly thereafter without repeated training. The Generative Adversarial Network (GAN) includes a generator network and a discriminator network. The generator network generates face images, which are then used to reconstruct the face images. The discriminator network identifies the generated face images. The training data for the generator network of this invention is obtained through the following steps: providing a normal face-aligned image; randomly cropping the normal face-aligned image using a mask to obtain multiple cropped images and corresponding masks; using the cropped images and corresponding masks as training data and inputting them into the generator network for training; the images output by the generator network are input into the discriminator network for discrimination. Preferably, the discriminator network uses the face recognition module of a face recognition system for discrimination. The generator network and the discriminator network are trained alternately until convergence, thus obtaining the face image reconstruction model.

[0065] Furthermore, when the image processing unit 23 determines that the result is unqualified, it inputs the face image into the generative network in the face image reconstruction model to obtain the reconstructed face image and output it.

[0066] In one specific embodiment of the present invention, a prediction unit connected to the image quality judgment unit 21 is also included;

[0067] The prediction unit is used to count the number of out-of-bounds points in the face alignment process and compare the total number of out-of-bounds points with a set value. If the total number is lower than the set value, the face image quality is not judged; if the total number is higher than the set value, the face image quality is judged.

[0068] Preferably, the setting is 15% of the total number of pixels in the face image. When the total number of out-of-bounds pixels exceeds 15%, the face image is considered to be of unacceptable quality. If it does not exceed 15%, the face image is considered to be of acceptable quality, and the face image can be directly input into the face recognition module of the face recognition system for face recognition.

[0069] Thus, the system of the present invention performs a second judgment on the quality of face images. First, it judges the number of out-of-bounds points. When the number of out-of-bounds points is small, the image quality is considered to be qualified. When the number of out-of-bounds points is large, the random vector similarity robustness index Val is judged. In this way, the quality of face images can be accurately determined.

[0070] The present invention also provides a method for face pre-recognition and image reconstruction, which is described below.

[0071] like Figure 1 As shown, the method of the present invention is used in the face alignment stage of a face recognition system, and the method includes the following steps:

[0072] Execute step S11 to judge the quality of the face image obtained in the face alignment process; then execute step S12.

[0073] Execute step S12 to establish a corresponding face image reconstruction model based on the generative adversarial network; then execute step S13.

[0074] In step S13, if the judgment result is unqualified, the face image is input into the established face image reconstruction model to obtain the reconstructed face image and output it.

[0075] If the judgment result is qualified, image reconstruction is not performed, and the face image is directly input into the face recognition system for face recognition.

[0076] This invention is applied after the face alignment stage and before face recognition. It first judges the quality of the face image. When the quality is deemed unqualified, the face image is reconstructed through a generative adversarial network. Compared with the method of filling constants, it can obtain image results with higher feature vector robustness, improve the recognition accuracy of the face recognition system, and avoid affecting product performance and user experience.

[0077] Preferably, the face recognition system includes a face alignment module and a face recognition module. The method of the present invention is used to judge the quality of the face image output by the face alignment module. If the face image quality is judged to be unqualified, the face image is reconstructed and the reconstructed face image is input into the face recognition module for face recognition. If the face image quality is judged to be qualified, the face image is input into the face recognition module for face recognition.

[0078] In one specific embodiment of the present invention, the step of judging the quality of a face image includes:

[0079] A face image is input into a face recognition system to extract the corresponding face feature vector;

[0080] Set an index box, and use the index box to randomly and repeatedly set the corresponding feature values ​​in the face image to zero, thereby obtaining multiple face-similar images;

[0081] The obtained facial similarity images are input into the facial recognition system to extract the corresponding facial similarity feature vectors;

[0082] Calculate the similarity index between face similarity feature vectors;

[0083] Determine whether the similarity index is higher than the preset value. If it is higher than the preset value, the result is qualified; if it is not higher than the preset value, the result is unqualified.

[0084] In one specific embodiment of the present invention, the similarity index between face similarity feature vectors and face feature vectors is calculated using the following formula:

[0085]

[0086] Where Val represents the similarity index, N represents the number of similar face images, X represents the face feature vector, and Xi represents the face similarity feature vector, with values ​​ranging from 1 to N.

[0087] In one specific embodiment of the present invention, the step of establishing a corresponding face image reconstruction model based on a generative adversarial network includes:

[0088] Provide a normal face alignment image, and use a mask to crop a random region from the provided normal face alignment image to obtain the cropped image;

[0089] The cropped image and its corresponding mask are used as training data.

[0090] The training data is fed into the generative network to obtain the generated image;

[0091] The generated images are judged using a face recognition system as a discriminant network.

[0092] The generative network and the discriminative network are trained alternately until convergence, thus obtaining a generative adversarial network.

[0093] In one specific embodiment of the present invention, before judging the quality of the face image, the method further includes:

[0094] The number of out-of-bounds points in the face alignment process was counted.

[0095] The total number of out-of-bounds points obtained from the statistics is compared with the set value. If it is lower than the set value, the quality of the face image is not judged. If it is higher than the set value, the quality of the face image is judged.

[0096] The face image quality assessment method of this invention first filters out images with potentially severe quality degradation by counting the number of out-of-bounds points, and then performs a second assessment by calculating the robustness index of random vector similarity, thus comprehensively considering the timeliness and accuracy of the assessment.

[0097] Compared to directly filling in the image with constants, image reconstruction using generative adversarial networks can yield image results with higher robustness to random feature vectors, thus improving the recognition accuracy of face recognition systems.

[0098] The present invention has been described in detail above with reference to the accompanying drawings and embodiments. Those skilled in the art can make various modifications to the present invention based on the above description. Therefore, certain details in the embodiments should not be construed as limiting the present invention, and the scope of protection of the present invention shall be defined by the appended claims.

Claims

1. A method for face pre-recognition and image reconstruction, used in the face alignment stage of a face recognition system, characterized in that, The method includes the following steps: The quality of the face images obtained during the face alignment process is assessed. A corresponding face image reconstruction model is established based on generative adversarial networks; If the judgment result is unqualified, the face image is input into the established face image reconstruction model to obtain the reconstructed face image and output it. The steps for determining the quality of a face image include: The face image is input into the face recognition system to extract the corresponding face feature vector; Set an index box, and use the index box to randomly and repeatedly set the corresponding feature values ​​in the face image to zero, thereby obtaining multiple face-similar images; The obtained facial similarity images are input into the facial recognition system to extract the corresponding facial similarity feature vectors; Calculate the similarity index between face similarity feature vectors; Determine whether the similarity index is higher than a preset value. If it is higher than the preset value, the determination result is qualified; if it is not higher than the preset value, the determination result is unqualified. The similarity index between two face feature vectors is calculated using the following formula: ； in, Indicators of similarity This indicates the number of images with similar faces. Represents a facial feature vector. This represents a facial similarity feature vector, with values ​​ranging from 1 to... ; The steps for building a face image reconstruction model based on generative adversarial networks include: Provide a normal face alignment image, and use a mask to crop a random region from the provided normal face alignment image to obtain the cropped image; The cropped image and its corresponding mask are used as training data. The training data is fed into a generative network to obtain a generated image; The face recognition system is used as a discrimination network to discriminate the generated images. The generative and discriminative networks are trained alternately until convergence, thus obtaining the generative adversarial network. The face image reconstruction model only needs to be trained when the system is built, and can be used directly thereafter without repeated training. Before judging the quality of facial images, the following steps are also included: The number of out-of-bounds points in the face alignment process was counted. The total number of out-of-bounds points obtained from the statistics is compared with the set value. If it is lower than the set value, the quality of the face image is not judged. If it is higher than the set value, the quality of the face image is judged.

2. A system for face pre-recognition and image reconstruction, used in the face alignment stage of a face recognition system, characterized in that, The system includes: The image quality judgment unit is used to judge the quality of the face image obtained in the face alignment process and obtain the judgment result; A face image reconstruction model based on generative adversarial networks; and An image processing unit, connected to the image quality judgment unit and the face image reconstruction model, is used to input the face image into the face image reconstruction model when the judgment result is unqualified, so as to obtain and output the reconstructed face image; The image quality judgment unit includes a feature vector acquisition module, a similar image establishment module, a similarity index calculation module, and a quality judgment module. The feature vector acquisition module is used to input the face image into the face recognition system to obtain the corresponding face feature vector; The similar image creation module is connected to the feature vector acquisition module and is used to randomly and repeatedly set the corresponding feature values ​​in the face image to zero using a set index box to obtain multiple face similar images. The feature vector acquisition module is also used to input the face-similar image into the face recognition system to obtain the corresponding face-similar feature vector; The similarity index calculation module is connected to the feature vector acquisition module and is used to calculate the similarity index based on the face similarity feature vector and the face feature vector. The quality judgment module is connected to the similarity index calculation module and is used to judge whether the similarity index is higher than a preset value. If it is higher than the preset value, the judgment result is qualified; if it is not higher than the preset value, the judgment result is unqualified. The similarity index calculation module calculates the similarity index using the following formula: ； in, Indicators of similarity This indicates the number of images with similar faces. Represents a facial feature vector. This represents a facial similarity feature vector, with values ​​ranging from 1 to... ; The face image reconstruction model is obtained by training a generative adversarial network. The training data for training the generative adversarial network includes cropped images obtained by randomly cropping normal face-aligned images using masks, and the corresponding masks. The discriminant network in the generative adversarial network is the face recognition system. The face image reconstruction model only needs to be trained when the system is established, and can be used directly thereafter without repeated training. It also includes a prediction unit connected to the image quality judgment unit; The prediction unit is used to count the number of out-of-bounds points in the face alignment process and compare the total number of out-of-bounds points with a set value. If the total number is lower than the set value, the face image quality is not judged; if the total number is higher than the set value, the face image quality is judged.

Images