Face image evaluation method, device and equipment and readable storage medium

Abstract

The present application provides a kind of face image evaluation method, device, equipment and readable storage medium, face image evaluation method includes: using face detection model to detect the image to be detected, obtain the face frame and attitude angle of all faces to be detected in the image to be detected;For each face to be detected, determine the face image of the face to be detected according to the face frame of the face to be detected;Using face quality evaluation model to evaluate face image, obtain the score of face image;According to the score of face image and the corresponding attitude angle, the comprehensive score of face image is calculated.The present application detects the face frame and attitude angle of all faces in face detection model, and the face quality evaluation model no longer detects the attitude angle, so that the efficiency of the face quality evaluation model can be greatly improved in the case of multiple faces in the snapshot image needing to be evaluated separately, thereby meeting the application requirements of multiple face real-time snapshot even on embedded devices.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and in particular to a facial image evaluation method, apparatus, device, and readable storage medium. Background Technology

[0002] The camera tracks faces passing through the capture area over a period of time, capturing multiple images. High-quality face images are then selected from these captures and used in downstream applications such as face recognition. Current face capture processes consist of two steps: face detection and face quality assessment. Face detection identifies faces in the captured images, obtaining target face bounding boxes and facial landmarks. Face quality assessment evaluates the quality of the detected faces, considering factors such as blurriness, pose angle, and completeness, and assigns a face quality score. Finally, high-scoring face images are selected for downstream applications such as face recognition.

[0003] Currently, face detection models and face quality assessment models are used for face detection and face quality assessment, respectively. The face detection model only needs to perform one detection for a captured image to obtain the location and key points of all faces in the captured image. However, the face quality assessment model can only input one face at a time for face quality assessment. When there are many faces in the captured image, the face quality assessment model must run many times to assess the face quality of each face. Since the face quality assessment model needs to evaluate the blurriness, pose angle, and completeness of each face in turn, the face quality assessment has an excessively long latency, resulting in low overall efficiency of face detection and face quality assessment. This is especially true on embedded devices with limited computing power, which cannot filter high-quality face images from multiple real-time captured face images for subsequent face recognition and other applications. Summary of the Invention

[0004] The main objective of this invention is to provide a face image evaluation method, apparatus, device, and readable storage medium, aiming to solve the technical problem that the efficiency of face detection and face quality evaluation in current real-time capture of multiple faces is low, especially on embedded devices with limited computing power, which cannot meet the application requirements of real-time capture of multiple faces.

[0005] In a first aspect, the present invention provides a face image evaluation method, the face image evaluation method comprising:

[0006] Acquire the image to be detected;

[0007] The face detection model is used to detect the image to be detected, and the face bounding boxes and pose angles of all faces to be detected in the image are obtained.

[0008] For each face to be detected, if the pitch angle of the face to be detected is within the first preset angle range and the yaw angle of the face to be detected is within the second preset angle range, then the face image of the face to be detected is determined according to the face bounding box of the face to be detected.

[0009] A face quality assessment model is used to evaluate face images and obtain a score for each face image.

[0010] A comprehensive score for the face image is calculated based on the face image score and the corresponding pose angle.

[0011] Optionally, before using the face detection model to detect the image to be detected and obtaining the face bounding boxes and pose angles of all faces to be detected in the image to be detected, the following steps are included:

[0012] Obtain a first training dataset, which includes multiple images, each containing multiple human faces;

[0013] For each face in each image, label the face confidence score, face bounding box, key points and pose angles to obtain the first labeled training dataset;

[0014] The multi-task neural network was trained using the first labeled training dataset, where face confidence, face bounding box, key points and pose angles each used a separate loss function;

[0015] When the first loss function converges, the trained face detection model is obtained. The first loss function is:

[0016] Ltotal1=α1*Lreg+β1*Lcls+γ1*Llmk+δ1*Lpose;

[0017] Where Ltotal1 is the first loss function, Lreg is the loss function for the face bounding box, Lcls is the loss function for face confidence, Llmk is the loss function for key points, Lpose is the loss function for pose angles, and α1, β1, γ1 and δ1 are the weights corresponding to each loss function.

[0018] Optionally, the step of using a face detection model to detect the image to be detected and obtaining the face bounding boxes and pose angles of all faces to be detected in the image to be detected includes:

[0019] The face detection model is used to detect the image to be detected, and the face confidence, face bounding box and pose angle of all the faces to be detected in the image are obtained.

[0020] Optionally, for each face to be detected, if the pitch angle of the face to be detected is within a first preset angle range, and the yaw angle of the face to be detected is within a second preset angle range, then determining the face image of the face to be detected based on the face bounding box of the face to be detected includes:

[0021] For each face to be detected, if the pitch angle of the face to be detected is within the first preset angle range, and the yaw angle of the face to be detected is within the second preset angle range, and the face confidence of the face to be detected is greater than the preset confidence, then the face image of the face to be detected is determined based on the face bounding box of the face to be detected.

[0022] Optionally, before evaluating the face image using the face quality assessment model to obtain a score for the face image, the following steps are included:

[0023] Obtain a second training dataset, which includes multiple images, each containing a human face;

[0024] For each image, data labels are assigned to the face probability score, completeness score, and blur score to obtain the labeled second training dataset;

[0025] The multi-task neural network was trained using the labeled second training dataset, where face probability scoring, completeness scoring, and blurriness scoring each used a separate loss function;

[0026] When the second loss function converges, the trained face quality assessment model is obtained. The second loss function is:

[0027] Ltotal2=α2*Lface+β2*Lcomp+γ2*Lblur;

[0028] Where Ltotal2 is the second loss function, Lface is the loss function for face probability, Lcomp is the loss function for completeness, Lblur is the loss function for blur, and α2, β2 and γ2 are the weights corresponding to each loss function.

[0029] Optionally, the scoring of the face image includes a face probability score, a completeness score, and a blur score. The calculation of the comprehensive score of the face image based on the score and the corresponding pose angle includes:

[0030] If the face probability score of the face image is greater than the preset score, then the pose angle score of the face image is calculated using Formula 1 based on the pose angle corresponding to the face image score. Formula 1 is:

[0031]

[0032] Based on the pose angle score, integrity score, and blur score of the face image, a comprehensive score for the face image is calculated using Formula 2, which is:

[0033] FQ = α4S1 + β4S2 + γ4S3;

[0034] Wherein, S3 is the attitude angle score, pitch is the pitch angle of the attitude angle, yaw is the yaw angle of the attitude angle, roll is the roll angle of the attitude angle, α3 is the weight of the pitch angle of the attitude angle, β3 is the weight of the yaw angle of the attitude angle, γ3 is the weight of the roll angle of the attitude angle, R is the angle range of the attitude angle, FQ is the comprehensive score, S1 is the ambiguity score, α4 is the weight of the ambiguity score, S2 is the completeness score, β4 is the weight of the completeness score, and γ4 is the weight of the attitude angle score.

[0035] Optionally, after calculating the comprehensive score of the face image based on the face image score and the corresponding pose angle, the process includes:

[0036] Select the face image with the highest overall score from the face images belonging to the same face to be detected;

[0037] Face recognition is performed on the face image with the highest comprehensive score.

[0038] Secondly, the present invention also provides a face image evaluation device, the face image evaluation device comprising:

[0039] The acquisition module is used to acquire the image to be detected;

[0040] The detection module is used to detect the image to be detected using a face detection model, and obtain the face bounding boxes and pose angles of all faces to be detected in the image to be detected;

[0041] The determination module is used to determine the face image of the face to be detected based on the face bounding box of the face to be detected for each face to be detected if the pitch angle of the face to be detected is within a first preset angle range and the yaw angle of the face to be detected is within a second preset angle range.

[0042] The evaluation module is used to evaluate face images using a face quality evaluation model and obtain a score for the face image;

[0043] The calculation module is used to calculate the comprehensive score of a face image based on its score and the corresponding pose angle.

[0044] Thirdly, the present invention also provides a face image evaluation device, the face image evaluation device including a processor, a memory, and a face image evaluation program stored in the memory and executable by the processor, wherein when the face image evaluation program is executed by the processor, it implements the steps of the face image evaluation method as described above.

[0045] Fourthly, the present invention also provides a readable storage medium storing a face image evaluation program, wherein when the face image evaluation program is executed by a processor, it implements the steps of the face image evaluation method as described above.

[0046] In this invention, an image to be detected is acquired; a face detection model is used to detect the image to obtain the bounding boxes and pose angles of all faces to be detected in the image; for each face to be detected, if the pitch angle of the pose angle of the face to be detected is within a first preset angle range, and the yaw angle of the pose angle of the face to be detected is within a second preset angle range, then the face image of the face to be detected is determined based on the bounding boxes of the face to be detected; the face image is evaluated using a face quality assessment model to obtain a score for the face image; and a comprehensive score for the face image is calculated based on the score of the face image and the corresponding pose angle. This invention, in real-time capture of multiple faces, detects the bounding boxes and pose angles of all faces in the image to be detected within the face detection model. By placing pose angle detection within the face detection model, only a small increase in data dimensionality is added. The pose angle is no longer detected in the face quality assessment model. For cases where multiple faces in a captured image require separate face quality assessments, this significantly improves the efficiency of the face quality assessment model, thereby enhancing the overall efficiency of face detection and face quality assessment. This allows the invention to meet the application requirements of real-time capture of multiple faces even on embedded devices. Furthermore, by judging the range of pose angles, face images that do not conform to the capture angle are filtered out, improving the accuracy and efficiency of face quality assessment. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating an embodiment of the face image evaluation method of the present invention;

[0048] Figure 2 for Figure 1 A detailed flowchart of step S50;

[0049] Figure 3 This is a schematic diagram of the evaluation and recognition process of an embodiment of the face image evaluation method of the present invention;

[0050] Figure 4 This is a schematic diagram of the functional modules of an embodiment of the face image evaluation device of the present invention;

[0051] Figure 5 This is a schematic diagram of the hardware structure of an embodiment of the face image evaluation device of the present invention.

[0052] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0054] In a first aspect, embodiments of the present invention provide a method for evaluating human face images.

[0055] To more clearly demonstrate the face image evaluation method provided in the embodiments of this application, we will first introduce the application scenarios of the face image evaluation method provided in the embodiments of this application.

[0056] The face image evaluation method provided in this application is applied to a camera tracking faces passing through the shooting area over a period of time, capturing multiple images, then filtering out high-quality face images from the captured images, and then using the high-quality face images for downstream applications such as face recognition. Face capture consists of two steps: face detection and face quality assessment, using face detection models and face quality assessment models respectively. The face detection model only needs one detection pass for a captured image to obtain the location and key points of all faces in the image. However, the face quality assessment model can only input one face at a time for assessment. When there are many faces in the captured image, the face quality assessment model must run multiple times to assess the face quality of each face. Because the face quality assessment model needs to evaluate the blurriness, pose angle, and completeness of each face, the assessment latency becomes too long, resulting in low overall efficiency for both face detection and face quality assessment. Therefore, it is essential to improve the efficiency of face detection and face quality assessment to efficiently filter high-quality face images from multiple face images captured by the camera, meeting the application requirements of real-time multi-face capture.

[0057] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the face image evaluation method of the present invention, as shown below. Figure 1 As shown, the face image evaluation method includes:

[0058] Step S10: Obtain the image to be detected.

[0059] In this embodiment, in the application scenario of real-time capture of multiple faces, the image to be detected is acquired through a camera, and the image to be detected includes multiple faces captured by the camera.

[0060] Step S20: Use a face detection model to detect the image to be detected, and obtain the face bounding boxes and pose angles of all faces to be detected in the image to be detected.

[0061] In this embodiment, the face detection model is a trained multi-task neural network model. The image to be detected is input into the face detection model, and the face detection model runs to detect the image to be detected. After detection, the face bounding box and pose angle of each face in the image to be detected are output one by one.

[0062] Step S30: For each face to be detected, if the pitch angle of the face to be detected is within the first preset angle range and the yaw angle of the face to be detected is within the second preset angle range, then the face image of the face to be detected is determined according to the face bounding box of the face to be detected.

[0063] In this embodiment, the face pose angles include pitch angle, yaw angle, and roll angle. Special attention is paid to pitch angle and yaw angle, which have a significant impact on face image quality. For each face, the pitch and yaw angle ranges are detected and judged. Faces that do not conform to the capture angle are filtered out and not included in subsequent face quality assessment, thereby improving the accuracy and efficiency of face quality assessment. Specifically, the pitch angle of the face to be detected needs to be within a first preset angle range, such as 15-20 degrees, and the yaw angle needs to be within a second preset angle range, such as 30-45 degrees. The first and second preset angle ranges can be set according to the specific capture scenario requirements. For faces that conform to the capture angle, the face image is determined based on the coordinates of the face bounding box, and the determined face image is used for subsequent face quality assessment.

[0064] Step S40: Use a face quality assessment model to evaluate the face image and obtain a score for the face image.

[0065] In this embodiment, the face quality assessment model is a trained multi-task neural network model. The face image determined in step S30 is input into the face quality assessment model for evaluation, and the score of the face image is output after evaluation.

[0066] Step S50: Calculate the comprehensive score of the face image based on the score of the face image and the corresponding pose angle.

[0067] In this embodiment, all face bounding boxes and pose angles are uniformly detected in the face detection model. When performing face quality assessment on each face separately, pose angles are no longer detected, which can greatly improve the efficiency of the face quality assessment model. Since pose angles also affect the quality of face images, the pose angles detected by the face detection model can be normalized to calculate the pose angle score. Combined with the score of the face image assessed by the face quality assessment model, the comprehensive score of the face image is calculated.

[0068] In this embodiment, during real-time capture of multiple faces, an image to be detected is acquired through a camera. This image includes multiple faces captured by the camera. The face detection model detects the bounding boxes and pose angles of all faces. Since pose angle detection is performed within the face detection model, it only slightly increases the data dimension of the model. The pose angle is not detected in the face quality assessment model. For cases where multiple faces in the captured image require separate face quality assessments, this significantly improves the efficiency of the face quality assessment model, thereby enhancing the overall efficiency of face detection and face quality assessment. Even on embedded devices, it can meet the application requirements of real-time capture of multiple faces. Furthermore, by judging the pose angle range, face images that do not conform to the capture angle are filtered out and not subsequently assessed for face quality, thus improving the accuracy and efficiency of face quality assessment. The pose angles detected by the face detection model are normalized, and a pose angle score is calculated. Combined with the score of the face image assessed by the face quality assessment model, a comprehensive score for the face image is calculated, ensuring the accuracy of face quality assessment.

[0069] Further, in one embodiment, before step S20, the following steps are included:

[0070] Obtain a first training dataset, which includes multiple images, each containing multiple human faces;

[0071] For each face in each image, label the face confidence score, face bounding box, key points and pose angles to obtain the first labeled training dataset;

[0072] The multi-task neural network was trained using the first labeled training dataset, where face confidence, face bounding box, key points and pose angles each used a separate loss function;

[0073] When the first loss function converges, the trained face detection model is obtained. The first loss function is:

[0074] Ltotal1=α1*Lreg+β1*Lcls+γ1*Llmk+δ1*Lpose;

[0075] Where Ltotal1 is the first loss function, Lreg is the loss function for the face bounding box, Lcls is the loss function for face confidence, Llmk is the loss function for key points, Lpose is the loss function for pose angles, and α1, β1, γ1 and δ1 are the weights corresponding to each loss function.

[0076] In this embodiment, after obtaining the first training dataset, existing face detection models, such as Retinaface and Yoloface, can be used to label the face bounding boxes and key points in the images of the dataset. Then, a pose estimation model (such as Hopenet) is used to label the pose angles. The face bounding boxes, key points, and pose angles are stored in the following format: face_x face_y face_width face_height landmark1.xlandmark1.y landmark2.x landmark2.y landmark3.xlandmark3.y landmark4.xlandmark4.y landmark5.x landmark5.y pitch yaw The data labels for each face's bounding box, key points, and pose angle are stored on a separate line, with multiple faces stored across multiple lines. Face confidence is used to distinguish faces from the background; face regions in the image are labeled with 1, and background regions with 0, resulting in face confidence data labels. The face images and their corresponding data labels are input together into a multi-task neural network for training. Key points and pose angles share features to further improve the detection accuracy of the face detection model. The output dimension of the face detection model is expanded from N*15 to N*18, where N is the number of anchor points. The number of anchor points for the bounding box, face confidence, key points, and pose angles are 4, 1, and 1, respectively. 10 and 3, totaling 18; the first loss function is the overall loss function, which is calculated by combining the individual loss functions for face confidence, face bounding box, key points, and pose angle with different weight coefficients. Specifically, the loss function for face bounding box can use GIOU, the loss function for face confidence can use cross-entropy, the loss function for key points can use wingloss, and the loss function for pose angle can also use wingloss. The weight coefficients for face bounding box, face confidence, key points, and pose angle can be set to 0.05, 0.5, 0.05, and 0.02 respectively.

[0077] Further, in one embodiment, step S20 includes:

[0078] The face detection model is used to detect the image to be detected, and the face confidence, face bounding box and pose angle of all the faces to be detected in the image are obtained.

[0079] In this embodiment, after the trained face detection model is used to detect the image to be detected, the face confidence score, face bounding box, key points and pose angle of each face are output one by one. The face confidence score is used for subsequent detection and judgment of whether it is a face, and the key points are used for subsequent face recognition.

[0080] Further, in one embodiment, step S30 includes:

[0081] For each face to be detected, if the pitch angle of the face to be detected is within the first preset angle range, and the yaw angle of the face to be detected is within the second preset angle range, and the face confidence of the face to be detected is greater than the preset confidence, then the face image of the face to be detected is determined based on the face bounding box of the face to be detected.

[0082] In this embodiment, in addition to detecting and judging the pitch and yaw angle ranges to filter out faces that do not meet the capture angle, thereby improving the accuracy and efficiency of face quality assessment, the face confidence is further detected and judged. The face confidence output by the face detection model is a value in the range of [0,1]. If the face confidence of the face to be detected is greater than the preset confidence, it is judged as a face; otherwise, it is judged as a non-face and filtered out, without further face quality assessment. This can further improve the accuracy and efficiency of face quality assessment.

[0083] Further, in one embodiment, before step S40, the following steps are included:

[0084] Obtain a second training dataset, which includes multiple images, each containing a human face;

[0085] For each image, data labels are assigned to the face probability score, completeness score, and blur score to obtain the labeled second training dataset;

[0086] The multi-task neural network was trained using the labeled second training dataset, where face probability scoring, completeness scoring, and blurriness scoring each used a separate loss function;

[0087] When the second loss function converges, the trained face quality assessment model is obtained. The second loss function is:

[0088] Ltotal2=α2*Lface+β2*Lcomp+γ2*Lblur;

[0089] Where Ltotal2 is the second loss function, Lface is the loss function for face probability, Lcomp is the loss function for completeness, Lblur is the loss function for blur, and α2, β2 and γ2 are the weights corresponding to each loss function.

[0090] In this embodiment, after acquiring the second training dataset, for each face image, the blurriness can be obtained using methods such as OpenCV Laplacian. After normalization, a blurriness score is obtained. Labeling software can be used to annotate the face images with data labels. Specifically, in the face probability score, faces are labeled as 1, and non-faces as 0. The numerical range of the face blurriness score is [0,1], where 1 is the clearest and 0 is the most blurry. In the face completeness score, completeness is labeled as 1, and incompleteness as 0. Separate loss functions are used for the face probability score, completeness score, and blurriness score. After feature extraction, the backbone network connects three heads, each head including a convolutional layer and a fully connected layer, used to process the extracted face probability score, completeness score, and blurriness score, respectively. The features of the fuzziness score are classified, and the output dimension of each fully connected layer is (64, 1), where 64 represents the feature dimensions of the face probability score, completeness score, and fuzziness score, respectively, and 1 represents the face probability score, completeness score, and fuzziness score, respectively. The second loss function is also the overall loss function, which is obtained by combining the individual loss functions of the face probability score, completeness score, and fuzziness score with different weight coefficients. Specifically, the loss function for the face probability score can use cross-entropy, the loss function for the completeness score can use cross-entropy, and the loss function for the fuzziness score can use wingloss. The weight coefficient for the face probability score can be set to 0.5, the weight coefficient for the completeness score can be set to 0.5, and the weight coefficient for the fuzziness score can be set to 1. By removing the pose angle detection from the face quality assessment model and placing the pose angle detection in the face detection model, it has been verified that the size of the face quality assessment model can be reduced from 7MB (megabytes) to 300KB (kilobytes), thereby greatly improving the efficiency of the face quality assessment model.

[0091] Furthermore, in one embodiment, reference is made to Figure 2 , Figure 2 for Figure 1 A detailed flowchart of step S50 is shown below. Figure 2 As shown, step S50 includes:

[0092] Step S501: If the face probability score of the face image is greater than the preset score, then the pose angle score of the face image is calculated according to the pose angle corresponding to the score of the face image using Formula 1, where Formula 1 is:

[0093]

[0094] Step S502: Based on the pose angle score, integrity score, and blur score of the face image, calculate the comprehensive score of the face image using Formula 2. Formula 2 is:

[0095] FQ = α4S1 + β4S2 + γ4S3;

[0096] Wherein, S3 is the attitude angle score, pitch is the pitch angle of the attitude angle, yaw is the yaw angle of the attitude angle, roll is the roll angle of the attitude angle, α3 is the weight of the pitch angle of the attitude angle, β3 is the weight of the yaw angle of the attitude angle, γ3 is the weight of the roll angle of the attitude angle, R is the angle range of the attitude angle, FQ is the comprehensive score, S1 is the ambiguity score, α4 is the weight of the ambiguity score, S2 is the completeness score, β4 is the weight of the completeness score, and γ4 is the weight of the attitude angle score.

[0097] In this embodiment, the face probability score of the face image output by the face quality assessment model is detected and judged, and face images with too low face probability scores are filtered out. Subsequent calculation of the comprehensive face image score is not performed. The aforementioned filtering of faces that do not conform to the capture angle through pose angle range detection, and filtering of non-faces through face confidence detection, can be used in combination or separately to improve the accuracy and efficiency of the face quality assessment model. The pose angle range R is usually 90 degrees. Using Formula 1, the pose angle can be normalized and converted to the range of [0,1] to facilitate the calculation of the comprehensive face image score by combining the integrity score and the ambiguity score. Specifically, the weight of the pitch angle can be set to 0.3, the weight of the yaw angle can be set to 0.5, the weight of the roll angle can be set to 0.2, the weight of the ambiguity score can be set to 0.4, the weight of the integrity score can be set to 0.3, and the weight of the pose angle score can be set to 0.3.

[0098] Furthermore, in one embodiment, reference is made to Figure 3 , Figure 3 This is a schematic diagram of the evaluation and recognition process of an embodiment of the face image evaluation method of the present invention, as shown below. Figure 3 As shown, after step S50, the following steps are included:

[0099] Step S60: Select the face image with the highest comprehensive score from the face images belonging to the same face to be detected;

[0100] Step S70: Perform face recognition on the face image with the highest comprehensive score.

[0101] In this embodiment, in the application scenario of real-time capture of multiple faces, the camera tracks multiple faces appearing in the shooting area, and each face forms a tracking trajectory. Over a period of time, the camera captures multiple images, each of which includes multiple faces. Face detection and face quality assessment models are used to perform face detection and face quality assessment on each captured image, respectively. After obtaining the comprehensive score of all face images, for the same face, the face image with the highest comprehensive score is selected as the final face image of that face. That is, the highest quality face image is selected from the multiple face images of that face for subsequent face recognition and other applications.

[0102] Secondly, embodiments of the present invention also provide a face image evaluation device.

[0103] Reference Figure 4 , Figure 4 This is a schematic diagram of the functional modules of an embodiment of the face image evaluation device of the present invention.

[0104] In this embodiment, the face image evaluation device includes:

[0105] Acquisition module 10 is used to acquire the image to be detected;

[0106] Detection module 20 is used to detect the image to be detected using a face detection model, and obtain the face bounding boxes and pose angles of all faces to be detected in the image to be detected;

[0107] The determination module 30 is used to determine the face image of the face to be detected based on the face bounding box of the face to be detected for each face to be detected if the pitch angle of the face to be detected is within a first preset angle range and the yaw angle of the face to be detected is within a second preset angle range.

[0108] Evaluation module 40 is used to evaluate face images using a face quality evaluation model and obtain a score for the face images;

[0109] The calculation module 50 is used to calculate the comprehensive score of the face image based on the score of the face image and the corresponding pose angle.

[0110] Furthermore, in one embodiment, the face image evaluation device further includes a first training module, used for:

[0111] Obtain a first training dataset, which includes multiple images, each containing multiple human faces;

[0112] For each face in each image, label the face confidence score, face bounding box, key points and pose angles to obtain the first labeled training dataset;

[0113] The multi-task neural network was trained using the first labeled training dataset, where face confidence, face bounding box, key points and pose angles each used a separate loss function;

[0114] When the first loss function converges, the trained face detection model is obtained. The first loss function is:

[0115] Ltotal1=α1*Lreg+β1*Lcls+γ1*Llmk+δ1*Lpose;

[0116] Where Ltotal1 is the first loss function, Lreg is the loss function for the face bounding box, Lcls is the loss function for face confidence, Llmk is the loss function for key points, Lpose is the loss function for pose angles, and α1, β1, γ1 and δ1 are the weights corresponding to each loss function.

[0117] Furthermore, in one embodiment, the detection module 20 is used for:

[0118] The face detection model is used to detect the image to be detected, and the face confidence, face bounding box and pose angle of all the faces to be detected in the image are obtained.

[0119] Furthermore, in one embodiment, the determining module 30 is used for:

[0120] For each face to be detected, if the pitch angle of the face to be detected is within the first preset angle range, and the yaw angle of the face to be detected is within the second preset angle range, and the face confidence of the face to be detected is greater than the preset confidence, then the face image of the face to be detected is determined based on the face bounding box of the face to be detected.

[0121] Furthermore, in one embodiment, the face image evaluation device further includes a second training module, used for:

[0122] Obtain a second training dataset, which includes multiple images, each containing a human face;

[0123] For each image, data labels are assigned to the face probability score, completeness score, and blur score to obtain the labeled second training dataset;

[0124] The multi-task neural network was trained using the labeled second training dataset, where face probability scoring, completeness scoring, and blurriness scoring each used a separate loss function;

[0125] When the second loss function converges, the trained face quality assessment model is obtained. The second loss function is:

[0126] Ltotal2=α2*Lface+β2*Lcomp+γ2*Lblur;

[0127] Where Ltotal2 is the second loss function, Lface is the loss function for face probability, Lcomp is the loss function for completeness, Lblur is the loss function for blur, and α2, β2 and γ2 are the weights corresponding to each loss function.

[0128] Furthermore, in one embodiment, the scoring of the face image includes a face probability score, a completeness score, and a blur score. The calculation module 50 is used for:

[0129] If the face probability score of the face image is greater than the preset score, then the pose angle score of the face image is calculated using Formula 1 based on the pose angle corresponding to the face image score. Formula 1 is:

[0130]

[0131] Based on the pose angle score, integrity score, and blur score of the face image, a comprehensive score for the face image is calculated using Formula 2, which is:

[0132] FQ = α4S1 + β4S2 + γ4S3;

[0133] Wherein, S3 is the attitude angle score, pitch is the pitch angle of the attitude angle, yaw is the yaw angle of the attitude angle, roll is the roll angle of the attitude angle, α3 is the weight of the pitch angle of the attitude angle, β3 is the weight of the yaw angle of the attitude angle, γ3 is the weight of the roll angle of the attitude angle, R is the angle range of the attitude angle, FQ is the comprehensive score, S1 is the ambiguity score, α4 is the weight of the ambiguity score, S2 is the completeness score, β4 is the weight of the completeness score, and γ4 is the weight of the attitude angle score.

[0134] Furthermore, in one embodiment, the face image evaluation device further includes a selection module, used for:

[0135] Select the face image with the highest overall score from the face images belonging to the same face to be detected;

[0136] Face recognition is performed on the face image with the highest comprehensive score.

[0137] The functions of each module in the aforementioned face image evaluation device correspond to the steps in the aforementioned face image evaluation method embodiment, and their functions and implementation processes will not be described in detail here.

[0138] Thirdly, embodiments of the present invention provide a face image evaluation device.

[0139] Reference Figure 5 , Figure 5 This is a schematic diagram of the hardware structure of an embodiment of the face image evaluation device of the present invention. In this embodiment, the face image evaluation device may include a processor 1001 (e.g., a Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize communication between these components; the user interface 1003 may include a display screen or an input unit such as a keyboard; the network interface 1004 may optionally include a standard wired interface or a wireless interface (e.g., Wireless Fidelity, Wi-Fi interface); the memory 1005 may be high-speed random access memory (RAM) or stable memory (non-volatile memory), such as a disk storage device; the memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art will understand that… Figure 5 The hardware structure shown does not constitute a limitation of the invention and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0140] Continue to refer to Figure 5 , Figure 5 The memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a face image evaluation program. The processor 1001 can call the face image evaluation program stored in the memory 1005 and execute the face image evaluation method provided in this embodiment of the invention.

[0141] Fourthly, embodiments of the present invention also provide a readable storage medium.

[0142] The present invention stores a face image evaluation program on a readable storage medium, wherein when the face image evaluation program is executed by a processor, it implements the steps of the face image evaluation method described above.

[0143] The method implemented when the face image evaluation procedure is executed can be referred to in various embodiments of the face image evaluation method of the present invention, and will not be repeated here.

[0144] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0145] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of the present invention.

[0147] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A face image evaluation method, characterized by, The face image evaluation method comprises: obtaining a to-be-detected image, the to-be-detected image comprising a plurality of faces; detecting the to-be-detected image using a face detection model to obtain face frames and pose angles of all to-be-detected faces in the to-be-detected image; for each to-be-detected face, if a pitch angle of the pose angle of the to-be-detected face is within a first preset angle range and a yaw angle of the pose angle of the to-be-detected face is within a second preset angle range, determining a face image of the to-be-detected face according to the face frame of the to-be-detected face; evaluating the face image using a face quality evaluation model to obtain a score of the face image, the evaluation of the face image comprising evaluation of face probability, completeness and blurriness of the face image, and the score of the face image comprising a face probability score, a completeness score and a blurriness score; calculating a comprehensive score of the face image according to the score of the face image and the corresponding pose angle; before the evaluation of the face image using the face quality evaluation model to obtain the score of the face image, comprising: obtaining a second training data set, the second training data set comprising a plurality of images, each image comprising a face; labeling data tags of the face probability score, the completeness score and the blurriness score for each image to obtain a labeled second training data set; training a multi-task neural network using the labeled second training data set, wherein the face probability score, the completeness score and the blurriness score are respectively used with a separate loss function; when the second loss function converges, obtaining a trained face quality evaluation model, the second loss function being: Ltotal2 = α2*Lface +β2*Lcomp +γ2*Lblur; wherein Ltotal2 is the second loss function, Lface is a loss function of the face probability, Lcomp is a loss function of the completeness, Lblur is a loss function of the blurriness, and α2, β2 and γ2 are weights corresponding to the respective loss functions.

2. The face image evaluation method of claim 1, wherein before the detection of the to-be-detected image using the face detection model to obtain the face frames and the pose angles of all to-be-detected faces in the to-be-detected image, comprising: obtaining a first training data set, the first training data set comprising a plurality of images, each image comprising a plurality of faces; labeling data tags of face confidence, face frame, key point and pose angle for each face in each image to obtain a labeled first training data set; training a multi-task neural network using the labeled first training data set, wherein the face confidence, the face frame, the key point and the pose angle are respectively used with a separate loss function; when the first loss function converges, obtaining a trained face detection model, the first loss function being: Ltotal1 = α1*Lreg +β1*Lcls +γ1*Llmk +δ1*Lpose ; wherein Ltotal1 is the first loss function, Lreg is a loss function of the face frame, Lcls is a loss function of the face confidence, Llmk is a loss function of the key point, Lpose is a loss function of the pose angle, and α1, β1, γ1 and δ1 are weights corresponding to the respective loss functions.

3. The face image evaluation method of claim 2, wherein The face detection model is used to detect the image to be detected, and face frames and attitude angles of all faces to be detected in the image to be detected are obtained. The face detection model is used to detect the image to be detected, and face confidence, face frames and attitude angles of all faces to be detected in the image to be detected are obtained.

4. The face image evaluation method of claim 3, wherein For each face to be detected, if the pitch angle of the attitude angle of the face to be detected is within the first preset angle range, and the yaw angle of the attitude angle of the face to be detected is within the second preset angle range, then a face image of the face to be detected is determined according to the face frame of the face to be detected. For each face to be detected, if the pitch angle of the attitude angle of the face to be detected is within the first preset angle range, and the yaw angle of the attitude angle of the face to be detected is within the second preset angle range, and the face confidence of the face to be detected is greater than a preset confidence, then a face image of the face to be detected is determined according to the face frame of the face to be detected.

5. The face image evaluation method of claim 1, wherein The score of the face image includes a face probability score, a completeness score and a blur score, and the comprehensive score of the face image is calculated according to the score of the face image and the corresponding attitude angle. If the face probability score of the face image is greater than a preset score, then the attitude angle score of the face image is calculated according to the attitude angle corresponding to the score of the face image by using Formula 1, wherein the Formula 1 is: ； The comprehensive score of the face image is calculated according to the attitude angle score, the completeness score and the blur score of the face image by using Formula 2, wherein the Formula 2 is: ； Wherein, S3 is the attitude angle score, pitch is the pitch angle of the attitude angle, yaw is the yaw angle of the attitude angle, roll is the roll angle of the attitude angle, α3 is the weight of the pitch angle of the attitude angle, β3 is the weight of the yaw angle of the attitude angle, γ3 is the weight of the roll angle of the attitude angle, R is the angle range of the attitude angle, FQ is the comprehensive score, S1 is the blur score, α4 is the weight of the blur score, S2 is the completeness score, β4 is the weight of the completeness score, and γ4 is the weight of the attitude angle score.

6. The face image evaluation method of claim 1, wherein After the comprehensive score of the face image is calculated according to the score of the face image and the corresponding attitude angle, the following steps are included: Selecting a face image with the highest comprehensive score from face images belonging to the same face to be detected; Performing face recognition on the face image with the highest comprehensive score.

7. A face image evaluation apparatus characterized by comprising: The face image evaluation device includes: An acquisition module is configured to acquire an image to be detected, wherein the image to be detected includes multiple faces; A detection module is configured to use a face detection model to detect the image to be detected, and obtain face frames and attitude angles of all faces to be detected in the image to be detected; A determination module is configured to, for each face to be detected, if the pitch angle of the attitude angle of the face to be detected is within the first preset angle range, and the yaw angle of the attitude angle of the face to be detected is within the second preset angle range, then determine a face image of the face to be detected according to the face frame of the face to be detected; An evaluation module is configured to evaluate the face image using a face quality evaluation model to obtain a score of the face image, wherein the evaluation of the face image includes evaluation of face probability, completeness and blur of the face image, and the score of the face image includes a face probability score, a completeness score and a blur score. The computing module is configured to calculate a comprehensive score of the face image according to the score of the face image and the corresponding pose angle; The face image evaluation device further comprises a second training module configured to: obtain a second training data set, the second training data set comprising a plurality of images, each image comprising a face; label each image with a data label of a face probability score, a completeness score and a blur score to obtain a labeled second training data set; train the multi-task neural network using the labeled second training data set, wherein the face probability score, the completeness score and the blur score are respectively trained using a separate loss function; when the second loss function converges, a trained face quality evaluation model is obtained, the second loss function being: Ltotal2 = α2*Lface +β2*Lcomp +γ2*Lblur; wherein Ltotal2 is the second loss function, Lface is the loss function of the face probability, Lcomp is the loss function of the completeness, Lblur is the loss function of the blur, and α2, β2 and γ2 are weights corresponding to each loss function.

8. A face image evaluation apparatus characterized by, The face image evaluation device comprises a processor, a memory, and a face image evaluation program stored on the memory and executable by the processor, wherein when the face image evaluation program is executed by the processor, the steps of the face image evaluation method according to any one of claims 1 to 6 are implemented.

9. A readable storage medium, characterized by, The readable storage medium stores a face image evaluation program, wherein when the face image evaluation program is executed by the processor, the steps of the face image evaluation method according to any one of claims 1 to 6 are implemented.

Images