A face recognition optimization method and device

Abstract

The present invention provides a face recognition optimization method and device. Among them, the method includes: determining the accuracy of the face recognition result; when the accuracy of the face recognition result is lower than a preset accuracy threshold, performing face detection to obtain the face frame and the coordinate information of the target key points, and determining the deflection angle of the face according to the coordinate information; when the deflection angle reaches or exceeds a preset deflection angle threshold, then performing a mapping from the side face feature to the front face feature based on the front and side face residual mapping model to obtain the remapped front face feature, and performing face recognition according to the remapped front face feature. By using the face recognition optimization method disclosed by the present invention, even when the face deflection angle is relatively large, it is still possible to effectively judge the face features, quickly implement face recognition, and improve the accuracy and robustness of face recognition.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to a face recognition optimization method and device. In addition, it also relates to an electronic device and a non-transitory computer-readable storage medium. Background Art

[0002] Face recognition is a biometric technology that automatically identifies a person's identity based on the person's facial features, also known as face image recognition, facial appearance recognition, face recognition, facial recognition, etc. Generally speaking, face recognition is an abbreviation for identity recognition and verification based on optical face images. With the rapid development of computer technology, face recognition technology has been widely studied and applied, and face recognition has become one of the most popular research topics in pattern recognition and image processing in the past 30 years. The purpose of face recognition is to extract personalized features of a person from a face image and use them to identify the person's identity. In the trend of the continuous expansion of the commercial application field of face recognition technology, "swiping face" to handle business is becoming more and more popular and convenient. In addition to the two major fields of security and finance, face recognition has also been widely applied in many scenarios such as business acceptance and auditing of telecom operators, and shows significant application value.

[0003] Currently, the mainstream face recognition technology is based on the geometric structure features of the face, and extracts and analyzes the structure features of the facial silhouette curve. Geometric features include the local shape features of the facial features such as eyes, nose, mouth, etc. The geometric features used for recognition are feature vectors based on the shape and geometric relationship of facial organs. Essentially, it is the matching between feature vectors, and its components usually include the Euclidean distance, curvature, angle, etc. between two specified points on the face.

[0004] However, the existing face recognition technology is prone to errors in side face scenarios and cannot accurately recognize. Especially when the deflection angle of the face in the collected image is too large, the face frame information and face key point information obtained by performing face detection on the image are very likely to deviate. Generally speaking, the larger the deflection angle, the greater the error. At this time, inputting the wrong face information into the feature extraction network again makes the extracted face features deviate greatly from the frontal face features stored in the database features, resulting in a low similarity calculated finally compared with the set threshold. If the deflection angle of the currently collected face is too large, it may affect the recognition result, resulting in the verification not passing, and the face verification cannot be quickly realized, affecting business acceptance. Therefore, how to optimize the face recognition process when the face deflection angle is too large has become an important research topic in this field. Summary of the Invention

[0005] Therefore, the present invention provides a face recognition optimization method and device to solve the problem that the accuracy and stability of face recognition are poor when the face deflection angle is too large in the prior art.

[0006] The present invention provides a face recognition optimization method, including:

[0007] Determine the accuracy of the face recognition result; when the accuracy of the face recognition result is lower than a preset accuracy threshold, perform face detection to obtain the face frame and the coordinate information of the target key points, and determine the deflection angle of the face according to the coordinate information;

[0008] When the deflection angle reaches or exceeds a preset deflection angle threshold, then perform mapping from the side face feature to the front face feature based on the front and side face residual mapping model to obtain the remapped front face feature; perform face recognition according to the remapped front face feature.

[0009] Further, the front and side face residual mapping model includes a residual mapping branch layer for generating the residual value and a head pose estimation layer for determining the deflection angle of the face;

[0010] The mapping from the side face feature to the front face feature based on the front and side face residual mapping model to obtain the remapped front face feature specifically includes:

[0011] Input the original face feature vector extracted by the original feature extraction model into the front and side face residual mapping model, control the residual variable of the front and side face through the deviation value, and output the remapped front face feature.

[0012] Further, the front and side face residual mapping model is trained with face feature samples, the corresponding residual mapping results of the face feature samples, and real labels; wherein, the face feature samples include side face feature data;

[0013] The front and side face residual mapping model is used to map the original face features extracted by the original feature extraction model based on the representation features obtained by feature extraction of the face feature samples.

[0014] Further, the determination of the accuracy of the face recognition result specifically includes:

[0015] Obtain a face image for face detection and extract the face features of the face image;

[0016] Compare the face features with the features in the database to determine the similarity, and determine the face recognition result according to the similarity;

[0017] Determine the accuracy of the face recognition result corresponding to the face recognition result according to the face recognition accuracy detection model.

[0018] Further, the face recognition according to the remapped front face feature specifically includes: comparing the remapped front face feature with the features in the database to determine the similarity, and determining the face recognition result according to the similarity.

[0019] Further, determining the deflection angle of the human face according to the coordinate information specifically includes: inputting the coordinate information into the head pose estimation layer in the front and side face residual mapping model to obtain the deflection angle of the human face.

[0020] The present invention also provides a face recognition optimization device, including:

[0021] A face recognition result accuracy detection unit for determining the accuracy of the face recognition result;

[0022] A deflection angle estimation unit for performing face detection, obtaining the coordinate information of the face frame and target key points, and determining the deflection angle of the human face according to the coordinate information when the accuracy of the face recognition result is lower than a preset accuracy threshold;

[0023] A feature remapping unit for mapping the side face feature to the front face feature based on the front and side face residual mapping model when the deflection angle reaches or exceeds a preset deflection angle threshold to obtain the remapped front face feature;

[0024] A face recognition unit for performing face recognition according to the remapped front face feature.

[0025] Further, the front and side face residual mapping model includes a residual mapping branch layer for generating the residual value and a head pose estimation layer for determining the deflection angle of the human face; the feature remapping unit is specifically used for: inputting the original face feature vector extracted by the original feature extraction model into the front and side face residual mapping model, controlling the residual variable of the front and side face through the deviation value, and outputting the remapped front face feature.

[0026] Further, the front and side face residual mapping model is trained with a face feature sample, the corresponding residual mapping result of the face feature sample, and a true label; wherein, the face feature sample includes side face feature data;

[0027] The front and side face residual mapping model is used to map the original face feature extracted by the original feature extraction model based on the representation feature obtained by feature extraction of the face feature sample.

[0028] Further, the face recognition result accuracy detection unit is specifically used for:

[0029] Obtaining a face image for face detection and extracting the face feature of the face image;

[0030] Comparing the face feature with the database feature to determine the similarity, and determining the face recognition result according to the similarity;

[0031] Determine the face recognition result accuracy corresponding to the face recognition result according to the face recognition accuracy detection model.

[0032] Further, the face recognition unit is specifically configured to: compare the remapped frontal face features with the gallery features to determine the similarity, and determine the face recognition result according to the similarity.

[0033] Further, the face recognition based on the remapped frontal face features specifically includes: comparing the remapped frontal face features with the gallery features to determine the similarity, and determining the face recognition result according to the similarity.

[0034] Correspondingly, the present invention also provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the program, the steps of the face recognition optimization method described in any one of the above are implemented.

[0035] Correspondingly, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the face recognition optimization method described in any one of the above are implemented.

[0036] By using the face recognition optimization method of the present invention, the deficiencies of the existing face recognition ability are made up. After obtaining the face features through algorithm model training and optimization, for a face with an excessively large deflection angle, the corrected frontal face features are generated according to the feature mapping, and the similarity between the remapped frontal face features and the gallery features is calculated to achieve fast face verification, reducing the error in finally calculating the face similarity, and being able to effectively judge the face features and quickly achieve face recognition under the condition of a large face deflection angle, improving the accuracy and robustness of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0038] Figure 1 It is a flowchart of a face recognition optimization method provided by an embodiment of the present invention;

[0039] Figure 2 It is an overall flowchart of the face recognition optimization method provided by an embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of the network structure design provided by an embodiment of the present invention;

[0041] Figure 4 Schematic structural diagram of a face recognition optimization device provided by an embodiment of the present invention;

[0042] Figure 5 Schematic physical structure diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0044] Based on the face recognition optimization method described in the present invention, the embodiments thereof will be described in detail below. As Figure 1 shown, it is a flowchart of the face recognition optimization method provided by an embodiment of the present invention, and the specific implementation process includes the following steps:

[0045] Step 101: Determine the accuracy of the face recognition result.

[0046] In the embodiment of the present invention, a face image of a person is collected from terminals such as a camera and a smart mobile device, face detection and feature extraction are performed to obtain corresponding face features, and the similarity between the face features and the target face features in the database is calculated. When the similarity reaches a preset similarity threshold, it is determined that the target person corresponding to the target face features in the database is recognized.

[0047] Among them, judging whether the face recognition result of the algorithm is accurate includes multiple situations: if it is actually the same person and the algorithm also recognizes the same person, then the probability of correct recognition; if it is actually the same person but the algorithm recognizes as different persons, then the probability of incorrect recognition; if it cannot be recognized, then the probability of recognition failure. Specifically, the face recognition result accuracy corresponding to the face recognition result can be determined according to the face recognition accuracy detection model. The algorithm formula corresponding to this face recognition accuracy detection model is as follows:

[0048] rate = MR / (MR + M1 + M2) (1)

[0049] In the formula, MR is the probability of correct recognition; M1 is the probability of incorrect recognition; M2 is the probability of not being recognized, that is, the maximum similarity is lower than the preset similarity threshold; rate is the face recognition result accuracy.

[0050] Step 102: When the accuracy of the face recognition result is lower than the preset accuracy threshold, perform face detection to obtain the coordinates of the face frame and the target key points, and determine the deflection angle of the face according to the coordinate information.

[0051] As Figure 2 shown, in the specific implementation process, when the recognition accuracy is low and lower than the preset accuracy threshold, the face recognition solution is adjusted. First, perform face detection to obtain the coordinates of the face frame and the target key points; and according to the coordinate information, calculate the deflection angle of the face through the head pose estimation layer in the front and side face residual mapping model.

[0052] Step 103: When the deflection angle reaches or exceeds the preset deflection angle threshold, map the side face feature to the front face feature based on the front and side face residual mapping model to obtain the remapped front face feature.

[0053] As Figure 3 shown, the front and side face residual mapping model consists of two parts: a residual mapping branch layer for generating the residual value and a head pose estimation layer for determining the deflection angle of the face. Among them, the residual mapping branch layer is used to generate R in the above formula (2). The residual mapping branch layer is composed of two FC layers (fully connected layers). The purpose of using this residual mapping branch layer is to reduce the distance between the side face and the front face. The specific formula is as follows:

[0054] Dmin|f'(Xs)-(f(Xp)+H(Xp)*R(Xp))| (2)

[0055] In the formula, D is a distance calculation function used to calculate the absolute value of the distance between the side face input and the front face. By training this residual mapping branch layer, the distance value between the front and side faces is continuously reduced.

[0056] The head pose estimation layer is used to calculate the deflection angle of the face, and then map the calculated deflection angle to between 0 and 1 as the coefficient for residual calculation. When the deflection angle of the face is too large, this coefficient quickly reaches 1, and then the residual part of the side face quickly becomes larger.

[0057] In an embodiment of the present invention, when the deflection angle reaches or exceeds a preset deflection angle threshold, an adjustment is made for side face recognition, and a front and side face residual mapping model is added on the basis of the original feature extraction model; the original face feature vector extracted by the original feature extraction model (i.e., the convolutional neural network model) is input into the front and side face residual mapping model; the residual variable of the front and side face is controlled by a deviation value to analyze the original face feature vector, realize the mapping of the side face feature and the front face feature, and output the corrected remapped front face feature. The front and side face residual mapping model is trained with face feature samples, the corresponding residual mapping results of the face feature samples, and true labels, wherein the face feature samples include side face feature data. The front and side face residual mapping model is used to map the original face features extracted by the original feature extraction model based on the representation features obtained by feature extraction of the face feature samples.

[0058] In a specific implementation process, when training the front and side face residual mapping model, the method of deep learning can be used to complete the mapping of the side face feature to the front face feature. In order to adapt to facial inputs in any pose, a control mechanism is introduced to adaptively control the residual variables of the front and side face by a deviation value to adapt to face inputs at various angles. The residual formula corresponding to the front and side face residual mapping model is:

[0059] f'(Xp) = M(f(Xp)) = f(Xp) + H(Xp) * R(f(Xp)) = f'(Xs) (3)

[0060] In the formula, Xp represents the front face, Xs represents the side face, M represents the mapping function from the side face to the front face, f represents the original convolutional neural network, and f’ is the optimized convolutional neural network. H(Xp) represents the deviation value in the newly added module, and R is the residual value. That is, the front face feature is equal to the original face feature input (i.e., the original input side face feature) plus the product of the deviation and the residual in the residual mapping branch layer. In practice, if the original face feature input is the front face, it can be seen from the above formula (3) that the residual part is 0.

[0061] Step 104: Perform face recognition according to the remapped front face feature.

[0062] Specifically, compare the remapped front face feature with the gallery feature, calculate the similarity between the remapped front face feature and the gallery feature, and determine the face recognition result according to the similarity, that is, if it is greater than a preset similarity threshold, the algorithm considers the recognition successful, otherwise the recognition fails.

[0063] By adopting the face recognition optimization method described in the embodiments of the present invention, the deficiencies of the existing face recognition capabilities are made up for. After obtaining face features through algorithm model training and optimization, for a face with an excessively large deflection angle, a corrected frontal face feature is generated according to the feature mapping, and the similarity between the remapped frontal face feature and the gallery feature is calculated to achieve rapid verification of the face, reducing the error in finally calculating the face similarity. It can effectively judge face features and quickly achieve face recognition under the condition of a large face deflection angle, improving the accuracy and robustness of face recognition.

[0064] Corresponding to the above-provided face recognition optimization method, the present invention also provides a face recognition optimization device. Since the embodiments of this device are similar to the above method embodiments, the description is relatively simple. For related parts, please refer to the description in the above method embodiment section. The following description of the embodiments of the face recognition optimization device is only illustrative. Please refer to Figure 4 As shown, it is a schematic structural diagram of a face recognition optimization device provided by an embodiment of the present invention. The face recognition optimization device described in the present invention specifically includes the following parts:

[0065] The face recognition result accuracy detection unit 401 is used to determine the accuracy of the face recognition result.

[0066] The deflection angle estimation unit 402 is used to perform face detection, obtain the face frame and the coordinate information of the target key points when the accuracy of the face recognition result is lower than the preset accuracy threshold, and determine the deflection angle of the face according to the coordinate information;

[0067] The feature remapping unit 403 is used to, when the deflection angle reaches or exceeds the preset deflection angle threshold, perform mapping from the side face feature to the frontal face feature based on the frontal and side face residual mapping model to obtain the remapped frontal face feature;

[0068] The face recognition unit 404 is used to perform face recognition according to the remapped frontal face feature.

[0069] By adopting the face recognition optimization device described in the embodiments of the present invention, the deficiencies of the existing face recognition capabilities are made up for. After obtaining face features through algorithm model training and optimization, for a face with an excessively large deflection angle, a corrected frontal face feature is generated according to the feature mapping, and the similarity between the remapped frontal face feature and the gallery feature is calculated to achieve rapid verification of the face, reducing the error in finally calculating the face similarity. It can effectively judge face features and quickly achieve face recognition under the condition of a large face deflection angle, improving the accuracy and robustness of face recognition.

[0070] Corresponding to the face recognition optimization method provided above, the present invention also provides an electronic device. Since the embodiments of this electronic device are similar to the above method embodiments, the description is relatively simple. For related parts, please refer to the description in the above method embodiment section. The electronic device described below is only illustrative. As Figure 5 shown, it is a schematic diagram of the physical structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may include: a processor 501, a memory 502, and a communication bus 503. Among them, the processor 501 and the memory 502 communicate with each other through the communication bus 503. The processor 501 can call the logical instructions in the memory 502 to execute the face recognition optimization method, which includes: determining the accuracy of the face recognition result. When the accuracy of the face recognition result is lower than a preset accuracy threshold, face detection is performed to obtain the face frame and the coordinate information of the target key points, and the deflection angle of the face is determined according to the coordinate information; when the deflection angle reaches or exceeds a preset deflection angle threshold, then based on the front and side face residual mapping model, the mapping from the side face feature to the front face feature is performed to obtain the remapped front face feature, and face recognition is performed according to the remapped front face feature.

[0071] In addition, when the logical instructions in the above-mentioned memory 502 can be implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., which can store program codes.

[0072] On the other hand, an embodiment of the present invention further provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the face recognition optimization method provided by each of the above method embodiments. The method includes: determining the accuracy of the face recognition result; when the accuracy of the face recognition result is lower than a preset accuracy threshold, performing face detection to obtain the coordinates of a face frame and target key points, and determining the deflection angle of the face according to the coordinate information; when the deflection angle reaches or exceeds a preset deflection angle threshold, performing a mapping from the side face feature to the front face feature based on a front and side face residual mapping model to obtain a remapped front face feature, and performing face recognition according to the remapped front face feature.

[0073] In another aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the face recognition optimization method provided by each of the above embodiments. The method includes: determining the accuracy of the face recognition result; when the accuracy of the face recognition result is lower than a preset accuracy threshold, performing face detection to obtain the coordinates of a face frame and target key points, and determining the deflection angle of the face according to the coordinate information; when the deflection angle reaches or exceeds a preset deflection angle threshold, performing a mapping from the side face feature to the front face feature based on a front and side face residual mapping model to obtain a remapped front face feature, and performing face recognition according to the remapped front face feature.

[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0075] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, also by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A face recognition optimization method, characterized in that, Including: Determine the accuracy of the face recognition result; When the accuracy of the face recognition result is lower than the preset accuracy threshold, perform face detection to obtain the face frame and the coordinate information of the target key points, and determine the deflection angle of the face according to the coordinate information; When the deflection angle reaches or exceeds the preset deflection angle threshold, perform mapping from the side face feature to the front face feature based on the front and side face residual mapping model to obtain the remapped front face feature; Perform face recognition according to the remapped front face feature; The front and side face residual mapping model includes a residual mapping branch layer for generating residual values and a head pose estimation layer for determining the deflection angle of the face; The performing mapping from the side face feature to the front face feature based on the front and side face residual mapping model to obtain the remapped front face feature specifically includes: Input the original face feature vector extracted by the original feature extraction model into the front and side face residual mapping model, control the residual variable of the front and side face through the deviation value, and output the remapped front face feature.

2. The face recognition optimization method according to claim 1, characterized in that The front and side face residual mapping model is trained with face feature samples, the corresponding residual mapping results of the face feature samples, and real labels; wherein, the face feature samples include side face feature data; The front and side face residual mapping model is used to map the original face features extracted by the original feature extraction model based on the representation features obtained by feature extraction of the face feature samples.

3. The face recognition optimization method according to claim 1, wherein The determining the deflection angle of the face according to the coordinate information specifically includes: Input the coordinate information into the head pose estimation layer in the front and side face residual mapping model to obtain the deflection angle of the face.

4. The face recognition optimization method according to claim 1, wherein The determining the accuracy of the face recognition result specifically includes: Obtain a face image for face detection and extract the face features of the face image; Compare the face features with the features in the database to determine the similarity, and determine the face recognition result according to the similarity; 5. The face recognition optimization method according to claim 1, wherein Determine the accuracy of the face recognition result corresponding to the face recognition result according to the face recognition accuracy detection model. The performing face recognition according to the remapped front face feature specifically includes:

6. A face recognition optimization device, characterized in that, Compare the remapped front face feature with the features in the database to determine the similarity, and determine the face recognition result according to the similarity. Including: A face recognition result accuracy detection unit for determining the accuracy of the face recognition result; A deflection angle estimation unit for performing face detection when the accuracy of the face recognition result is lower than the preset accuracy threshold, obtaining the face frame and the coordinate information of the target key points, and determining the deflection angle of the face according to the coordinate information; A feature remapping unit for performing mapping from the side face feature to the front face feature based on the front and side face residual mapping model when the deflection angle reaches or exceeds the preset deflection angle threshold to obtain the remapped front face feature; A face recognition unit for performing face recognition according to the remapped front face feature; The front and side face residual mapping model includes a residual mapping branch layer for generating residual values and a head pose estimation layer for determining the deflection angle of the human face; the feature remapping unit is specifically configured to: input the original human face feature vector extracted by the original feature extraction model into the front and side face residual mapping model, control the residual variables of the front and side faces through the deviation value, and output the remapped front face features.

7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the steps of the face recognition optimization method described in any one of claims 1-5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the face recognition optimization method described in any one of claims 1-5 are implemented.

Images