Face quality evaluation model training method and device, equipment and storage medium

Abstract

The application discloses a face quality evaluation model training method and device, equipment and a storage medium, relates to the technical field of face quality evaluation model training, and the method comprises the following steps: constructing an initial face quality evaluation model and a joint loss function by fusing a multi-target regression network and a binary classification network; and training the initial face quality evaluation model based on a training sample set comprising multi-dimensional sample quality scores and comprehensive sample quality evaluation results to obtain a face quality evaluation model. Therefore, based on the face quality evaluation model of the application, the quality evaluation result of a face image can be obtained by automatically fusing multi-dimensional quality scores through a binary classification network, without setting the influence weight of each dimension score, so that the technical problem that the influence weight of each dimension quality on face quality is difficult to determine in the existing face quality evaluation method and the face quality evaluation accuracy is low is solved, and the face quality evaluation accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of face quality assessment model training technology, and in particular to a face quality assessment model training method, apparatus, device and storage medium. Background Technology

[0002] In related technologies, before the face recognition stage, multi-dimensional quality scores of face images are calculated, and these scores are combined using a weighted approach to obtain a face image quality assessment result, thus achieving face quality assessment. However, in existing face quality assessment methods, the weights of each quality dimension on the face quality are difficult to determine, resulting in low accuracy in face quality assessment. Summary of the Invention

[0003] The main objective of this invention is to provide a method, apparatus, device, and storage medium for training a face quality assessment model, aiming to solve the technical problem that the influence weights of various dimensions of quality on face quality are difficult to determine in existing face quality assessment methods, resulting in low accuracy of face quality assessment.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] In a first aspect, the present invention provides a method for training a face quality assessment model, the method comprising:

[0006] Obtain the initial face quality assessment model and training sample set; the initial face quality assessment model includes a multi-objective regression network and a binary classification network; the training sample set includes multiple face image samples and the sample face occlusion score, sample face pose score, sample face blur score, sample face illumination score, and sample quality assessment results of the face image samples;

[0007] The face image samples are input into a multi-objective regression network to obtain the predicted face occlusion score, predicted face pose score, predicted face blur score, and predicted face illumination score of the face image samples.

[0008] The predicted face occlusion score, predicted face pose score, predicted face blur score, and predicted face illumination score are input into a binary classification network to obtain the prediction quality evaluation results.

[0009] A joint loss function is constructed based on the predicted face occlusion score, predicted face pose score, predicted face blur score, predicted face illumination score, and prediction quality assessment results.

[0010] The model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model are updated according to the joint loss function to obtain the face quality assessment model.

[0011] Optionally, a joint loss function is constructed based on the predicted face occlusion score, predicted face pose score, predicted face blur score, predicted face illumination score, and prediction quality assessment results, including:

[0012] The first loss function is constructed based on the mean squared error between the predicted face occlusion score, predicted face pose score, predicted face blur score, predicted face illumination score and the sample face occlusion score, sample face pose score, sample face blur score and sample face illumination score.

[0013] A second loss function is constructed based on the cross-entropy of the prediction quality assessment results and the sample quality assessment results;

[0014] Based on the first loss function and the second loss function, the joint loss function is obtained.

[0015] Optionally, before obtaining the training sample set, the method further includes:

[0016] Based on a preset number of occlusion level scores, the area of ​​the complete face and the area of ​​the unoccluded face in the face image sample, the sample face occlusion score is obtained;

[0017] The sample face pose score is obtained based on the sample face pitch angle, sample face yaw angle and sample face roll angle of the face image sample.

[0018] Based on the image gradient of the face image samples, obtain the blur score and illumination score of the sample face.

[0019] Optionally, based on a preset number of occlusion level scores, the area of ​​the complete face and the area of ​​the unoccluded face in the face image sample, a sample face occlusion score is obtained, including:

[0020] Using Formula 1, the occlusion score of the sample face is obtained based on the occlusion level score, the area of ​​the complete face, and the area of ​​the unoccluded face; Formula 1 is:

[0021]

[0022] Among them, Q occ δ is used to score the face occlusion of the sample. i For the i-th occlusion level, Area sub_face To avoid obscuring the face area, Area whole_face This represents the total area of ​​a human face.

[0023] Optionally, before obtaining the sample face pose score based on the sample face pitch angle, sample face yaw angle, and sample face roll angle of the face image sample, the method further includes:

[0024] Using Formula 2, the pitch angle of the face in the sample is obtained based on the three-dimensional coordinate matrix and coordinate weight matrix of the facial key points in the face image sample; Formula 2 is:

[0025] Q roll =atan2(2(wx+yx),1-2(x 2 +y 2 Using Formula 3, the yaw angle of the sample face is obtained based on the three-dimensional coordinate matrix and the coordinate weight matrix; Formula 3 is:

[0026] Q yaw =arcsin(2(wy-zx))

[0027] Using Formula 4, the roll angle of the sample face is obtained based on the three-dimensional coordinate matrix and the coordinate weight matrix; Formula 4 is:

[0028] Q pitch =atan2(2(wx+yz),1-2(x 2 +z 2 ))

[0029] Among them, Q roll For the sample face pitch angle, Q yaw For the sample face yaw angle, Q pitch Let x be the rolling angle of the sample face, where x, y, and z are three-dimensional coordinate matrices, and w is the coordinate weight matrix.

[0030] Optionally, the model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model are updated according to the joint loss function to obtain the face quality assessment model, including:

[0031] Update the model parameters of the multi-objective regression network and the binary classification network until the joint loss function is minimized to obtain the face quality assessment model.

[0032] Optionally, after updating the model parameters of the multi-objective regression network and the binary classification network until the joint loss function is minimized and the face quality assessment model is obtained, the method further includes:

[0033] Obtain the face image of the person to be evaluated;

[0034] The face image to be evaluated is input into the face quality assessment model to obtain the face occlusion score, face pose score, face blur score, face illumination score, and quality assessment result of the face image to be evaluated.

[0035] Secondly, the present invention also provides a face quality assessment model training device, the device comprising:

[0036] The acquisition module is used to acquire the initial face quality assessment model and the training sample set. The initial face quality assessment model includes a multi-objective regression network and a binary classification network. The training sample set includes multiple face image samples and the sample face occlusion score, sample face pose score, sample face blur score, sample face illumination score, and sample quality assessment results of the face image samples.

[0037] The first network module is used to input face image samples into a multi-objective regression network to obtain the predicted face occlusion score, predicted face pose score, predicted face blur score, and predicted face illumination score of the face image samples.

[0038] The second network module is used to input the predicted face occlusion score, predicted face pose score, predicted face blur score, and predicted face illumination score into the binary classification network to obtain the prediction quality evaluation results.

[0039] The loss construction module is used to construct a joint loss function based on the predicted face occlusion score, predicted face pose score, predicted face blur score, predicted face illumination score, and prediction quality assessment results.

[0040] The model optimization module is used to update the model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model according to the joint loss function, so as to obtain the face quality assessment model.

[0041] Thirdly, the present invention also provides a face quality assessment model training device, the device comprising: a memory, a processor, and a face quality assessment model training program stored in the memory and executable on the processor, wherein the face quality assessment model training program is configured to implement the steps of any of the face quality assessment model training methods described above.

[0042] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the face quality assessment model training method as described above.

[0043] This invention provides a method, apparatus, device, and storage medium for training a face quality assessment model. The method involves obtaining predicted face occlusion scores, predicted face pose scores, predicted face blur scores, and predicted face illumination scores for face image samples through a multi-objective regression network in the initial face quality assessment model. Then, the method uses a binary search network in the initial face quality assessment model to obtain the predicted quality assessment results for the face image samples based on these scores. Finally, a joint loss function is constructed by combining the predicted face occlusion scores, predicted face pose scores, predicted face blur scores, predicted face illumination scores, and the predicted quality assessment results. When training the initial face quality assessment model using a training sample set, the model parameters of the initial face quality assessment model are updated according to the joint loss function to obtain the face quality assessment model.

[0044] Therefore, this invention constructs an initial face quality assessment model and joint loss function by fusing a multi-objective regression network and a binary classification network. The initial face quality assessment model is then trained using a training sample set that includes multi-dimensional sample quality scores and comprehensive sample quality assessment results, resulting in a new face quality assessment model. Based on this face quality assessment model, a multi-dimensional quality score of the face image can be obtained through a multi-objective regression network, and the multi-dimensional quality scores can be automatically fused through a binary classification network to obtain the face image quality assessment result. This eliminates the need to set the influence weights of each dimension's score, solving the technical problem in existing face quality assessment methods where the influence weights of each dimension's quality on face quality are difficult to determine, leading to low accuracy in face quality assessment. This invention improves the accuracy of face quality assessment. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the structure of the face quality assessment model training device of the present invention;

[0047] Figure 2 This is a flowchart illustrating the first embodiment of the face quality assessment model training method of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of a face quality assessment model according to one embodiment of the present invention;

[0049] Figure 4This is a flowchart illustrating the second embodiment of the face quality assessment model training method of the present invention;

[0050] Figure 5 This is a schematic diagram of the modules of the first embodiment of the face quality assessment model training device of the present invention.

[0051] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0053] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0054] In this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that an apparatus or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an apparatus or system. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the apparatus or system that includes that element.

[0055] The quality of the captured face determines the effectiveness of the face recognition model. Due to factors such as camera shake, pose, occlusion, and shooting distance, it is difficult to guarantee the consistency of the quality of the captured face images. This can lead to issues such as occluded faces, excessively large face poses (in other words, the face's orientation angle exceeds the range that the face model can recognize), blurred faces, excessively bright or dark lighting, and insufficient face resolution. All of these can exceed the capabilities of the human eye or machine for face recognition, causing the face recognition model to fail and affecting its performance. To address these issues and ensure that face photos meet the quality range required for model recognition, current practices typically include a face quality assessment before face recognition.

[0056] Existing technologies can be divided into two types of face quality assessment methods: one is to assess the quality of an image by combining its inherent feature attributes with prior knowledge. This method has the advantages of simple algorithm construction and fast operation speed, but the error is relatively large; the other is to obtain a face quality assessment model through machine learning methods to assess the quality of face images.

[0057] The first evaluation method is a single-dimensional independent evaluation of multiple factors. The evaluation methods are mostly traditional image processing and graphic geometry methods, which are not very accurate. At the same time, they lack comprehensive quality evaluation results that combine multiple factors, making them difficult to apply in practice.

[0058] The second evaluation method proposes to combine multi-dimensional quality assessment scores using a weighted approach to provide a comprehensive quality assessment result. However, the weight of each dimension's quality assessment score on the quality of the face image is difficult to determine in this method. Furthermore, it primarily relies on conventional factors affecting image quality, such as image brightness and sharpness, for joint quality assessment of face images, failing to comprehensively consider all dimensions affecting face recognition, such as face occlusion and face pose.

[0059] Furthermore, when training face quality assessment models built using machine learning or deep neural networks, learning methods without reference evaluation require a large amount of manually labeled data. For learning methods with reference evaluation, the main approach is to calculate the distance between the face image to be evaluated and a pre-prepared reference image to obtain enhanced facial brightness, contrast, blur, occlusion, pose variation, and face matching scores to train the face quality assessment model. For example, a cosine distance can be calculated between a reference image and the face image to be evaluated as the quality assessment result. The problem with this method is that it cannot accurately quantify the quality assessment result for each dimension, and the lack of selection criteria for the reference image leads to inaccurate quality assessment results.

[0060] Given the difficulty in determining the weights of each dimension's influence on face quality in existing face quality assessment methods, leading to low accuracy, this invention provides a face quality assessment model training method. The overall approach is as follows:

[0061] The method includes: obtaining an initial face quality assessment model and a training sample set; the initial face quality assessment model includes a multi-objective regression network and a binary classification network; the training sample set includes multiple face image samples and sample face occlusion scores, sample face pose scores, sample face blur scores, sample face illumination scores, and sample quality assessment results; inputting the face image samples into the multi-objective regression network to obtain predicted face occlusion scores, predicted face pose scores, predicted face blur scores, and predicted face illumination scores; inputting the predicted face occlusion scores, predicted face pose scores, predicted face blur scores, and predicted face illumination scores into the binary classification network to obtain predicted quality assessment results; constructing a joint loss function based on the predicted face occlusion scores, predicted face pose scores, predicted face blur scores, predicted face illumination scores, and predicted quality assessment results; updating the model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model based on the joint loss function to obtain the face quality assessment model.

[0062] This invention provides a method for training a face quality assessment model. An initial face quality assessment model and joint loss function are constructed by fusing a multi-objective regression network and a binary classification network. The initial face quality assessment model is then trained on a training sample set including multi-dimensional sample quality scores and comprehensive sample quality assessment results to obtain the final face quality assessment model. Therefore, based on this invention's face quality assessment model, multi-dimensional quality scores of face images can be obtained through a multi-objective regression network, and the multi-dimensional quality scores can be automatically fused through a binary classification network to obtain the face image quality assessment result. This eliminates the need to set the influence weights of each dimension's score, solving the technical problem in existing face quality assessment methods where the influence weights of each dimension's quality on face quality are difficult to determine, leading to low accuracy in face quality assessment. This method improves the accuracy of face quality assessment.

[0063] The following provides a detailed description of the face quality assessment model training method, apparatus, equipment, and storage medium used in the technical implementation of this invention:

[0064] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of the face quality assessment model training device in the hardware operating environment involved in the embodiments of this application.

[0065] like Figure 1As shown, the device may include: a processor 1001, such as a CPU; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include electronic devices such as tablets and PDAs (Personal Digital Assistants). Optionally, the user interface 1003 may also be a display screen or an input unit such as a keyboard. The memory 1005 may be high-speed RAM or non-volatile memory, such as disk storage. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0066] It is understood that the device may also include a network interface 1004, which may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). Optionally, the device may also include RF (Radio Frequency) circuitry, sensors, audio circuitry, a Wi-Fi module, etc.

[0067] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0068] The face quality assessment model training method, apparatus, device, and storage medium of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] Based on, but not limited to, the above hardware structure, refer to Figures 2 to 4 , Figure 2 This is a flowchart illustrating the first embodiment of the face quality assessment model training method of the present invention. Figure 3 This is a schematic diagram of the structure of a face quality assessment model according to one embodiment of the present invention. Figure 4 This is a flowchart illustrating the second embodiment of the face quality assessment model training method of the present invention.

[0070] This embodiment provides a method for training a face quality assessment model, which may include:

[0071] Step S400: Obtain the initial face quality assessment model and training sample set.

[0072] The initial face quality assessment model includes a multi-objective regression network and a binary classification network. The training sample set includes multiple face image samples and the sample face occlusion score, sample face pose score, sample face blur score, sample face illumination score, and sample quality assessment results.

[0073] In this embodiment, the executing entity can be, for example, Figure 1 The face quality assessment model training device shown can be a physical server including an independent host, or a virtual server hosted by a host cluster.

[0074] like Figure 3 As shown, the multi-objective regression network can include four residual convolutional modules (ResNet) layers connected in series, an Averagepool pooling layer, a fully connected layer, and an objective regression layer. Each residual convolutional module layer can consist of three convolutional layers: 1×1 convolution, 3×3 convolution, and 1×1 convolution. The four residual convolutional module layers have 3, 4, 6, and 3 residual convolutional blocks respectively, used for feature extraction from the face image, ultimately outputting 2048 7×7 two-dimensional feature maps. The fully connected layer can be a 6-dimensional fully connected layer, i.e., 6-dFC, which can convert the two-dimensional feature map into a one-dimensional vector, representing a quality score vector quantized in 6 dimensions of face quality. The objective regression layer can include a sigmoid calculation layer, enabling multi-objective regression calculation on a one-dimensional vector. The binary classification network can include a softmax binary classification layer, enabling binary classification calculation.

[0075] It should be noted that the training sample set can include multiple face image samples and sample quality labels for each face image sample. The sample quality labels can include sample quality scores and sample quality evaluation results across six dimensions. The sample quality scores can include face occlusion scores, face pose scores (including face pitch angle, face yaw angle, and face roll angle), face blur scores, and face illumination scores, which can be pre-calculated based on the face image samples. The sample quality evaluation results can include whether the face image quality is acceptable or unacceptable, and can be pre-labeled manually based on the face image samples.

[0076] Furthermore, as one implementation, before step S400, the method may further include:

[0077] Step S100: Obtain the sample face occlusion score based on a preset number of occlusion level scores, the area of ​​the complete face and the area of ​​the unoccluded face in the face image sample.

[0078] In this embodiment, face occlusion can include mask occlusion, glasses occlusion, and occlusion by other objects. In practical use, a preset number of occlusion levels can be set according to the actual situation. The sample face occlusion score is obtained by combining the complete face area and the unoccluded face area in the face image sample.

[0079] In one specific implementation, step S100 may include: obtaining a sample face occlusion score based on Formula 1, the occlusion level score, the complete face area, and the unoccluded face area. Formula 1 is:

[0080]

[0081] Among them, Q occ δ is used to score the face occlusion of the sample. i For the i-th occlusion level, Area sub_face To avoid obscuring the face area, Area whole_face This represents the total area of ​​a human face.

[0082] In this embodiment, M occlusion level scores can be set, i = 0, 1, ..., M. For example, 6 occlusion level scores can be set, including [0.0, 0.2, 0.4, 0.6, 0.8, 1.0]. The unoccluded face area in the face image sample... sub_face and complete face area whole_face Given the M occlusion levels, the score δ i The occlusion score that minimizes the occlusion level on the right side of Formula 1 is the occlusion score of the sample face.

[0083] Step S200: Obtain the sample face pose score based on the sample face pitch angle, sample face yaw angle and sample face roll angle of the face image sample.

[0084] In this embodiment, the sample face pose score can be represented by calculating the sample face pitch angle, sample face yaw angle, and sample face roll angle in the face image sample.

[0085] As a specific implementation, before step S200, the method may include: using Formula 2, obtaining the pitch angle of the sample face based on the three-dimensional coordinate matrix and coordinate weight matrix of the facial key points in the face image sample; Formula 2 is:

[0086] Q roll =atan2(2(wx+yz),1-2(x 2 +y 2 ))

[0087] Using Formula 3, the yaw angle of the sample face is obtained based on the three-dimensional coordinate matrix and the coordinate weight matrix; Formula 3 is:

[0088] Q yaw =arcsin(2(wy-zx))

[0089] Using Formula 4, the roll angle of the sample face is obtained based on the three-dimensional coordinate matrix and the coordinate weight matrix; Formula 4 is:

[0090] Q pitch =atan2(2(wx+yz),1-2(x 2 +z 2 ))

[0091] Among them, Q roll For the sample face pitch angle, Q yaw For the sample face yaw angle, Q pitch Let x be the rolling angle of the sample face, where x, y, and z are three-dimensional coordinate matrices, and w is the coordinate weight matrix.

[0092] In this embodiment, head orientation in a face image sample can be estimated through head pose estimation. First, 2D key points of the face are detected. Then, the transformation relationship between the 3D points of the face key points and the corresponding 2D points is solved to obtain the face rotation matrix in the face image sample (which may include the three-dimensional coordinate matrix of the face key points). Euler angles are solved to obtain the pitch angle, yaw angle, and roll angle of the sample face.

[0093] Step S300: Based on the image gradient of the face image sample, obtain the sample face blur score and the sample face illumination score.

[0094] In this embodiment, image gradients can be calculated using different gradient operators to serve as sample face blur scores and sample face illumination scores. Preferably, the sample face blur score can be calculated using the Laplacian operator, and the sample face illumination score can be calculated using the tenengrad gradient function.

[0095] Specifically, based on the image gradient of the face image samples, the blur score and illumination score of the sample face are obtained, which may include:

[0096] Using Formula 5, the Laplacian convolution result of each pixel in the face sample image is calculated to obtain the face blur score Q of the face image sample. blur Formula five is:

[0097]

[0098] Where T(a,b) is the result of the Laplacian convolution of pixel (a,b).

[0099] Using Formula 6, the grayscale value of each pixel in the face sample image is calculated to obtain the face blur score Q of the face image sample. illu Formula six is:

[0100]

[0101] Where G(a,b) is the image grayscale value of pixel (a,b).

[0102] Therefore, the sample quality score set Y k，1 It can be represented as:

[0103] Y k，1 ={Q occ (k), Q pitch (k), Q yaw (k), Q roll (k), Q blur (k), Q illu (k)};

[0104] Where k is the kth training sample in the training sample set.

[0105] Furthermore, since the sample quality assessment results can include both acceptable and unacceptable facial image quality, the sample quality assessment result set Y... k，2 It can be represented as:

[0106] Y k，2 ={1, 0};

[0107] Here, 1 indicates that the face image quality is acceptable, and 0 indicates that the face image quality is unacceptable.

[0108] Therefore, the training sample set Sample k It can be represented as:

[0109] Sample k

[0110] ={Picture(k), Q} occ (k), Q pitch (k), Q yaw (k), Q roll (k), Q blur (k), Q illu (k), 1, 0} Step S500: Input the face image samples into the multi-objective regression network to obtain the predicted face occlusion score, predicted face pose score, predicted face blur score and predicted face illumination score of the face image samples.

[0111] Step S600: Input the predicted face occlusion score, predicted face pose score, predicted face blur score, and predicted face illumination score into the binary classification network to obtain the prediction quality evaluation results.

[0112] In this embodiment, when training the face quality assessment model, face image samples from the training sample set are input into the initial face quality assessment model. The face image samples are first predicted by a multi-objective regression network to obtain predicted face occlusion scores, predicted face pose scores, predicted face blur scores, and predicted face illumination scores. Then, a binary classification network obtains the predicted quality assessment results based on the multi-dimensional quality scores output by the multi-objective regression network.

[0113] Step S700: Construct a joint loss function based on the predicted face occlusion score, predicted face pose score, predicted face blur score, predicted face illumination score, and prediction quality assessment results.

[0114] In this embodiment, the joint loss function is constructed by combining the outputs of the multi-objective regression network and the binary classification network with the sample quality labels. This can integrate and optimize the model parameters of the multi-objective regression network and the binary classification network, thereby improving the accuracy of single-dimensional quality scores and quality assessment results.

[0115] As a specific implementation, step S700 may include: constructing a first loss function based on the mean square error between the predicted face occlusion score, predicted face pose score, predicted face blur score, predicted face illumination score and the sample face occlusion score, sample face pose score, sample face blur score and sample face illumination score; constructing a second loss function based on the cross-entropy of the prediction quality assessment result and the sample quality assessment result; and obtaining a joint loss function based on the first loss function and the second loss function.

[0116] In this embodiment, the loss function of the multi-objective regression network is the first loss function, which can be constructed based on the prediction quality score and sample quality score of the multi-objective regression network prediction output. The loss function of the binary classification network is the second loss function, which can be constructed based on the prediction quality assessment result and sample quality assessment result of the binary classification network prediction output.

[0117] Specifically, the first loss function and the second loss function can be obtained through Formulas 7 and 8, thereby constructing the joint loss function. Formula 7 is as follows:

[0118]

[0119] L1 is the first loss function, n is the number of face image samples in the training sample set, k is the k-th face image sample, and f(x) k ) represents the prediction quality score of the multi-objective regression network prediction output, yk Score the quality of the sample.

[0120] Formula 8 is:

[0121] L2=-[ylog(p)+(1-y)log(1-p)];

[0122] L2 is the second loss function, y is the sample quality assessment result, and p is the probability that the predicted quality assessment result is the same as y.

[0123] Therefore, the joint loss function L is:

[0124] L = L1 + L2.

[0125] Step S800: Update the model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model according to the joint loss function to obtain the face quality assessment model.

[0126] In this embodiment, when training the initial face quality assessment model using face image samples from the training sample set, a multi-objective regression network is first used to predict the face image samples, obtaining predicted face occlusion scores, predicted face pose scores, predicted face blur scores, and predicted face illumination scores—multi-dimensional quality scores. Then, a binary classification network is used to obtain the predicted quality assessment results of the face image samples based on the multi-dimensional quality scores. Finally, based on the predicted face occlusion scores, predicted face pose scores, predicted face blur scores, predicted face illumination scores, and predicted quality assessment results, a joint loss function is constructed. The multi-objective regression network and the binary classification network in the initial face quality assessment model are updated with the aim of optimizing the joint loss function, resulting in a well-trained face quality assessment model.

[0127] As a specific implementation, step S800 may include: updating the model parameters of the multi-objective regression network and the binary classification network until the joint loss function is minimized, thereby obtaining the face quality assessment model.

[0128] In this embodiment, when training the initial face quality assessment model, the goal is to optimize the joint loss function to the minimum. Based on the prediction results of the initial face quality assessment model on the face image samples in the training sample set, the model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model are updated to obtain the face quality assessment model.

[0129] Furthermore, as an implementation, after step S800, the method may include: acquiring a face image to be evaluated; inputting the face image to be evaluated into a face quality assessment model to obtain a face occlusion score, face pose score, face blur score, face illumination score, and quality assessment result of the face image to be evaluated.

[0130] In this embodiment, after the face quality assessment model is constructed and trained, the face image to be assessed is input into the face quality assessment model to obtain the face occlusion score, face pose score, face blur score, face illumination score and quality assessment result of the face image to be assessed.

[0131] It is understandable that before inputting the face image to be evaluated into the face quality assessment model, a face detection model can be used to detect the face image and crop out the face region to obtain the face image to be evaluated. Furthermore, after cropping the face image to be evaluated, a preset resolution threshold can be set to remove lower-resolution face images to obtain higher-resolution face images, which are then input into the face quality assessment model for face quality assessment. The preset resolution threshold can be set according to actual needs.

[0132] Firstly, this embodiment provides a method for training a face quality assessment model. An initial face quality assessment model and joint loss function are constructed by fusing a multi-objective regression network and a binary classification network. The initial face quality assessment model is then trained on a training sample set including multi-dimensional sample quality scores and comprehensive sample quality assessment results to obtain the face quality assessment model. Therefore, based on the face quality assessment model of this embodiment, multi-dimensional quality scores of face images can be obtained through a multi-objective regression network, and the multi-dimensional quality scores can be automatically fused through a binary classification network to obtain the face image quality assessment result. This eliminates the need to set the influence weights of each dimension score, solving the technical problem in existing face quality assessment methods where the influence weights of each dimension quality on face quality are difficult to determine, leading to low accuracy in face quality assessment. This improves the accuracy of face quality assessment. Furthermore, this embodiment fuses multi-dimensional quality scores through a binary classification network, requiring only a simple binary classification function to achieve the fusion of multi-dimensional quality scores, simplifying the fusion process.

[0133] Secondly, in this embodiment, the sample quality scores of each dimension of the face image samples in the training sample set are automatically calculated by different algorithms, eliminating the need for manual calculation and labeling of the sample quality scores of each dimension, thus reducing the cost of manual labeling.

[0134] Thirdly, the joint loss function in this embodiment is constructed by fusing the first loss function of the multi-objective regression network and the second loss function of the binary classification network. When training the face quality assessment model, the model parameters of the face quality assessment model are updated based on the joint loss function, so that the multi-objective regression network and the binary classification network can improve each other's learning accuracy. The multi-dimensional fusion optimizes the model parameters, thereby improving the accuracy of single-dimensional quality scoring and comprehensive quality assessment.

[0135] Based on the same inventive concept, referring to Figure 5 , Figure 5This is a schematic diagram of the modules of a first embodiment of the face quality assessment model training device of the present invention. This embodiment provides a face quality assessment model training device, which may include:

[0136] The acquisition module 10 is used to acquire the initial face quality assessment model and the training sample set. The initial face quality assessment model includes a multi-objective regression network and a binary classification network. The training sample set includes multiple face image samples and the sample face occlusion score, sample face pose score, sample face blur score, sample face illumination score, and sample quality assessment results of the face image samples.

[0137] The first network module 20 is used to input face image samples into a multi-objective regression network to obtain the predicted face occlusion score, predicted face pose score, predicted face blur score, and predicted face illumination score of the face image samples.

[0138] The second network module 30 is used to input the predicted face occlusion score, predicted face pose score, predicted face blur score, and predicted face illumination score into the binary classification network to obtain the prediction quality evaluation result.

[0139] The loss construction module 40 is used to construct a joint loss function based on the predicted face occlusion score, predicted face pose score, predicted face blur score, predicted face illumination score, and prediction quality assessment results.

[0140] The model optimization module 50 is used to update the model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model according to the joint loss function, so as to obtain the face quality assessment model.

[0141] For more details on the specific implementation of the above-mentioned face quality assessment model training device, please refer to the description of the specific implementation of the face quality assessment model training method in any of the above embodiments. For the sake of brevity, these details will not be repeated here.

[0142] Furthermore, embodiments of the present invention also propose a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the face quality assessment model training method described above. Therefore, further details will not be repeated here. Additionally, the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments of this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0143] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for training a face quality assessment model, characterized in that, The method includes: Obtain an initial face quality assessment model and a training sample set; the initial face quality assessment model includes a multi-objective regression network and a binary classification network; the training sample set includes multiple face image samples and the sample face occlusion score, sample face pose score, sample face blur score, sample face illumination score, and sample quality assessment results of the face image samples; The face image sample is input into the multi-objective regression network to obtain the predicted face occlusion score, predicted face pose score, predicted face blur score, and predicted face illumination score of the face image sample. The predicted face occlusion score, the predicted face pose score, the predicted face blur score, and the predicted face illumination score are input into the binary classification network to obtain the prediction quality evaluation result. A joint loss function is constructed based on the predicted face occlusion score, the predicted face pose score, the predicted face blur score, the predicted face illumination score, and the prediction quality assessment result. The model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model are updated according to the joint loss function to obtain the face quality assessment model. The step of constructing a joint loss function based on the predicted face occlusion score, the predicted face pose score, the predicted face blur score, the predicted face illumination score, and the prediction quality assessment result includes: A first loss function is constructed based on the mean square error between the predicted face occlusion score, the predicted face pose score, the predicted face blur score, the predicted face illumination score and the sample face occlusion score, the sample face pose score, the sample face blur score and the sample face illumination score. A second loss function is constructed based on the cross-entropy of the predicted quality assessment results and the sample quality assessment results; The joint loss function is obtained by summing the first loss function and the second loss function.

2. The method as described in claim 1, characterized in that, Before obtaining the training sample set, the method further includes: The occlusion score of the sample face is obtained based on a preset number of occlusion level scores, the area of ​​the complete face and the area of ​​the unoccluded face in the face image sample; The sample face pose score is obtained based on the sample face pitch angle, sample face yaw angle and sample face roll angle of the sample face image. Based on the image gradient of the face image sample, obtain the face blur score and the face illumination score of the sample.

3. The method as described in claim 2, characterized in that, The step of obtaining the sample face occlusion score based on a preset number of occlusion level scores, the area of ​​the complete face and the area of ​​the unoccluded face in the face image sample, includes: Using Formula 1, the occlusion score of the sample face is obtained based on the occlusion level score, the complete face area, and the unoccluded face area; Formula 1 is: Wherein, Q occ The δ is used to score the face occlusion of the sample. i For the i-th occlusion level score, Area sub_face The area refers to the unobstructed face area. whole_face The area of ​​the complete human face is given.

4. The method as described in claim 2, characterized in that, Before obtaining the sample face pose score based on the sample face pitch angle, sample face yaw angle, and sample face roll angle of the sample face image, the method further includes: Using Formula 2, the pitch angle of the face in the face image sample is obtained based on the three-dimensional coordinate matrix and coordinate weight matrix of the facial key points; Formula 2 is: Using Formula 3, the yaw angle of the sample face is obtained based on the three-dimensional coordinate matrix and the coordinate weight matrix; Formula 3 is: Using Formula 4, the rolling angle of the sample face is obtained based on the three-dimensional coordinate matrix and the coordinate weight matrix; Formula 4 is: Wherein, Q roll The Q is the pitch angle of the sample face. yaw The Q is the yaw angle of the sample face. pitch Let x be the rolling angle of the sample face, y be the three-dimensional coordinate matrix, and w be the coordinate weight matrix.

5. The method according to any one of claims 1 to 4, characterized in that, The step of updating the model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model according to the joint loss function to obtain the face quality assessment model includes: Update the model parameters of the multi-objective regression network and the binary classification network until the joint loss function is minimized to obtain the face quality assessment model.

6. The method as described in claim 5, characterized in that, After updating the model parameters of the multi-objective regression network and the binary classification network until the joint loss function is minimized to obtain the face quality assessment model, the method further includes: Obtain the face image of the person to be evaluated; The face image to be evaluated is input into the face quality assessment model to obtain the face occlusion score, face pose score, face blur score, face illumination score, and quality assessment result of the face image to be evaluated.

7. A face quality assessment model training device, characterized in that, The device includes: The acquisition module is used to acquire an initial face quality assessment model and a training sample set; the initial face quality assessment model includes a multi-objective regression network and a binary classification network; the training sample set includes multiple face image samples and the sample face occlusion score, sample face pose score, sample face blur score, sample face illumination score, and sample quality assessment results of the face image samples; The first network module is used to input the face image sample into the multi-objective regression network to obtain the predicted face occlusion score, predicted face pose score, predicted face blur score and predicted face illumination score of the face image sample. The second network module is used to input the predicted face occlusion score, the predicted face pose score, the predicted face blur score, and the predicted face illumination score into the binary classification network to obtain the prediction quality evaluation result. The loss construction module is used to construct a joint loss function based on the predicted face occlusion score, the predicted face pose score, the predicted face blur score, the predicted face illumination score, and the prediction quality assessment result. The model optimization module is used to update the model parameters of the multi-objective regression network and the binary classification network in the initial face quality assessment model according to the joint loss function, so as to obtain the face quality assessment model. The loss construction module is further configured to: construct a first loss function based on the mean square error between the predicted face occlusion score, the predicted face pose score, the predicted face blur score, the predicted face illumination score and the sample face occlusion score, the sample face pose score, the sample face blur score, and the sample face illumination score; construct a second loss function based on the cross-entropy of the predicted quality assessment result and the sample quality assessment result; and obtain the joint loss function based on the sum of the first loss function and the second loss function.

8. A face quality assessment model training device, characterized in that, The device includes: a memory, a processor, and a face quality assessment model training program stored in the memory and executable on the processor, wherein the face quality assessment model training program is configured to implement the steps of the face quality assessment model training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the face quality assessment model training method as described in any one of claims 1 to 6.

Images