A face quality evaluation method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a face quality evaluation method, device, electronic equipment and storage medium, wherein the method comprises: obtaining a first face image; using a face recognition network to perform feature extraction on the first face image to obtain a Gaussian distribution feature for representing uncertainty of the first face image; and determining a quality evaluation result of the first face image based on the Gaussian distribution feature obtained from the first face image. The Gaussian distribution feature in the present disclosure integrates the relevant uncertainty features of the image. Compared with the traditional method of determining the image quality by relying on human logical rules, the present disclosure can more accurately evaluate the sample quality based on the predicted uncertainty of each image, is less affected and constrained by factors such as scene environment, has higher robustness, and thus can adapt to the requirements of various application scenarios.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image data processing, and in particular, to a face quality evaluation method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence (AI), face recognition technology is widely used in various fields. In order to ensure the real-time and accuracy of face recognition, it is often necessary to select a high-quality face image from the collected video for recognition task.

[0003] In the traditional face image quality screening process, face quality evaluation is involved, which mainly scores the face image through several fixed dimensions such as illumination, blurriness, face pose, etc., and then integrates the scores to determine the overall score of the image. However, in order to cope with different application scenarios, different scoring standards are usually needed. For example, for the two scenarios of day and night, the scoring standard corresponding to the illumination dimension is different, so that the robustness of the existing quality evaluation is not high, and it is difficult to adapt to the needs of various application scenarios. SUMMARY

[0004] The embodiments of the present disclosure at least provide a face quality evaluation method, device, electronic equipment and storage medium.

[0005] In a first aspect, the embodiments of the present disclosure provide a face quality evaluation method, which comprises:

[0006] obtaining a first face image;

[0007] performing feature extraction on the first face image using a face recognition network to obtain a Gaussian distribution feature for representing uncertainty of the first face image;

[0008] determining a quality evaluation result of the first face image based on the Gaussian distribution feature obtained from the first face image.

[0009] By using the face quality evaluation method, in the case of obtaining the first face image, the face recognition network can be used to extract features from the face image, so that the quality evaluation result of the sample can be determined based on the obtained Gaussian distribution features. This is mainly because the Gaussian distribution features can represent the uncertainty of the sample. The higher the uncertainty of the sample, the worse the quality of the sample. Conversely, the higher the certainty of the sample, the higher the quality of the sample. The Gaussian distribution features in the present disclosure integrate the relevant uncertainty features of the image. Compared with the traditional method of determining the image quality by relying on human logic, the sample quality can be more accurately evaluated based on the predicted uncertainty of each image, which is not affected by the scene environment and other factors, and has high robustness, thereby adapting to the needs of various application scenarios.

[0010] In a possible implementation, the quality evaluation result includes a quality score, and the number of the first face images is multiple; and the method further includes:

[0011] determining an initial quality score of each of the first face images based on the Gaussian distribution features obtained from the multiple first face images respectively;

[0012] determining an average quality score corresponding to each of the first face images, and a maximum quality score and a minimum quality score corresponding to each face class label pre-labeled for each of the first face images based on the initial quality score of each of the first face images;

[0013] adjusting the initial quality score of each of the first face images under each face class label based on the maximum quality score and the minimum quality score corresponding to each face class label and the average quality score, to obtain an adjusted quality score of each of the first face images.

[0014] Here, based on the distribution characteristics of the Gaussian distribution features, the initial quality score of the first face image can be adjusted based on the maximum quality score and the minimum quality score corresponding to the face class label, so that the distribution of the adjusted quality score is more uniform, which is more conducive to overall quality evaluation.

[0015] In a possible implementation, before adjusting the initial quality score of each of the first face images under each face class label based on the maximum quality score and the minimum quality score corresponding to each face class label and the average quality score, the method further includes:

[0016] determining, for each of the face class labels, whether a first difference between the maximum quality score corresponding to the face class label and the average quality score is greater than a second difference between the average quality score and the minimum quality score corresponding to the face class label;

[0017] in response to the first difference being greater than the second difference, adjusting the minimum quality score corresponding to the face class label based on the maximum quality score corresponding to the face class label and the average quality score, to obtain an adjusted minimum quality score corresponding to the face class label;

[0018] in response to the first difference being less than or equal to the second difference, adjusting the maximum quality score corresponding to the face class label based on the minimum quality score corresponding to the face class label and the average quality score, to obtain an adjusted maximum quality score corresponding to the face class label.

[0019] In a possible implementation, the adjusting the initial quality scores of the first face images under each of the face class labels based on the maximum quality score and the minimum quality score corresponding to each of the face class labels, and the average quality score, comprises:

[0020] determining, for each of the face class labels, a third difference between the initial quality score of each of the first face images under the face class label and the minimum quality score corresponding to the face class label, and a fourth difference between the maximum quality score and the minimum quality score corresponding to the face class label;

[0021] performing a ratio operation on the third difference and the fourth difference to determine an adjusted quality score of each of the first face images under the face class label.

[0022] Here, the quality score can be adjusted based on the closeness between the third difference and the fourth difference, further making the adjusted quality score more uniform, thereby facilitating subsequent network training.

[0023] In a possible implementation, after the quality evaluation result of the first face image is determined, the method further comprises:

[0024] taking the first face image as input data of a to-be-trained face quality evaluation network, taking the quality evaluation result of the first face image as contrast supervision data of output result of the to-be-trained face quality evaluation network, training the to-be-trained face quality evaluation network to obtain a trained face quality evaluation network.

[0025] In one possible implementation, before using the first face image as input data for the face quality assessment network to be trained, and using the quality assessment result of the first face image as comparative supervision data for the output result of the face quality assessment network to be trained, the method further includes:

[0026] If the network training speed is determined to be less than a preset threshold, the first face image is processed into grayscale to obtain the processed first face image.

[0027] Here, the first face image can be processed into grayscale first, so as to obtain a processed first face image with less computational cost, thereby achieving a higher network training speed.

[0028] In one possible implementation, after obtaining the trained face quality assessment network, the method further includes:

[0029] Acquire the image to be evaluated;

[0030] The image to be evaluated is input into a trained face quality assessment network to obtain a quality score for the image to be evaluated.

[0031] In response to the quality score of the image to be evaluated being greater than a preset threshold, face comparison is performed on the image to be evaluated to obtain the face comparison result.

[0032] Here, face comparison is only performed when the quality score of the image to be evaluated is determined to be relatively high, thereby achieving faster and more secure face recognition.

[0033] In one possible implementation, the face recognition network includes an extraction layer and a classification layer; the face recognition network is trained according to the following steps:

[0034] Obtain each second face image;

[0035] For each of the plurality of second face images, the extraction layer of the face recognition network is used to extract features from the second face image to obtain an image feature vector and a Gaussian distribution feature vector used to characterize the uncertainty of the second face image.

[0036] The image feature vector and the Gaussian distribution feature vector are sampled in the same dimension to obtain the sampled image features;

[0037] The sampled image features are classified through the classification layer of the face recognition network to obtain a classification result, and the classification result is compared with the pre-labeled result of the second face image.

[0038] In response to the comparison result indicating that the classification result is inconsistent with the pre-labeled result, network parameter values of the face recognition network are adjusted, and the next round of network training is performed until the comparison result indicates that the classification result is consistent with the pre-labeled result.

[0039] In a possible implementation, the Gaussian distribution feature includes a Gaussian variance vector consistent with the dimension of the image feature vector of the first face image; and determining the quality evaluation result of the first face image based on the Gaussian distribution feature obtained from the first face image includes:

[0040] averaging each vector value included in the Gaussian variance vector obtained from the first face image to obtain an average vector value;

[0041] taking the average vector value as the quality evaluation result of the first face image.

[0042] The Gaussian distribution feature here can be a Gaussian variance vector corresponding to the image feature vector, each dimension of the vector can correspond to a bias condition of a corresponding dimension of the image feature vector, and a more accurate quality evaluation result can be obtained by averaging vector values.

[0043] In a second aspect, the embodiments of the present disclosure further provide a device for face quality evaluation, and the device includes:

[0044] a obtaining module configured to obtain a first face image;

[0045] an extracting module configured to perform feature extraction on the first face image by using a face recognition network to obtain a Gaussian distribution feature for characterizing uncertainty of the first face image;

[0046] an evaluating module configured to determine a quality evaluation result of the first face image based on the Gaussian distribution feature obtained from the first face image.

[0047] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which includes a processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the face quality evaluation method according to any one of the first aspect and various embodiments thereof.

[0048] In a fourth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps of the face quality evaluation method according to any one of the first aspect and various embodiments thereof.

[0049] For effects of the device, the electronic device, and the computer readable storage medium for face quality assessment described above, refer to the description of the method for face quality assessment described above, which will not be repeated here.

[0050] In order to make the above objectives, characteristics and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings herein are incorporated into the description and form a part of the description. The drawings show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the description. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be considered as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0052] Figure 1 A flow chart of a method for face quality assessment provided by an embodiment of the present disclosure is shown;

[0053] Figure 2 A schematic diagram of a device for face quality assessment provided by an embodiment of the present disclosure is shown;

[0054] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure more clear, the following will combine the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, but not all the embodiments. The components of the embodiments of the present disclosure described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.

[0056] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0057] The term "and / or", merely describes an associated relationship, which means that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of the plurality or any combination of at least two of the plurality, for example, including at least one of A, B, and C, which can mean including any one or more elements selected from the set consisting of A, B, and C.

[0058] It is found through research that face recognition is one of the most widely used biometric technologies at present, and is also a common algorithm in the field of security and the like. It is widely used in many scenes, such as person-identity comparison, access control, and person search.

[0059] However, the overall process of a perfect face recognition system is not simple. After obtaining a face image, face detection is first performed, quality judgment is performed after obtaining a face frame, low-quality face images are filtered out, face alignment is performed, and finally face recognition is performed in the recognition system. The quality judgment is very important for the whole system, and directly affects the recognition effect of the subsequent recognition system.

[0060] Among them, the quality of the face image detected by the image detection is uneven. From the perspective of evaluation efficiency, if the quality model is not sensitive enough to low-quality images, the low-quality face images fed to the recognition model are easy to cause unnecessary misrecognition or recognition rejection. If the high-quality face image is too strict, the face image is likely to be filtered out, and the recognition efficiency is also low.

[0061] The existing face quality evaluation method mainly scores the face image through several fixed dimensions, such as illumination, blurriness, face pose, etc. Then the overall score of the image is determined by the respective scoring results. However, this method may need to set different scoring standards for different application scenarios. For example, for the two scenarios of day and night, the scoring standard corresponding to the illumination dimension is different, so the robustness of the existing quality evaluation is not high, and it cannot adapt to the needs of various application scenarios.

[0062] Based on the above research, the present disclosure provides a face quality evaluation scheme based on Gaussian distribution feature extraction to determine the quality evaluation result, to improve the robustness of quality evaluation while ensuring high accuracy evaluation.

[0063] To facilitate the understanding of the present embodiment, first of all, a method for face quality evaluation disclosed by the present embodiment is introduced in detail, the execution subject of the method for face quality evaluation provided by the present embodiment is generally an electronic device with certain computing power, which for example includes: a terminal device or a server or other processing device, the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the method for face quality evaluation can be realized by a processor calling computer readable instructions stored in a memory.

[0064] Referring to Figure 1 The flowchart of the method for face quality evaluation provided by the present embodiment is shown, the method includes steps S101-S103, wherein:

[0065] S101: obtaining a first face image;

[0066] S102: performing feature extraction on the first face image by using a face recognition network to obtain a Gaussian distribution feature for representing uncertainty of the first face image;

[0067] S103: determining a quality evaluation result of the first face image based on the Gaussian distribution feature obtained from the first face image.

[0068] To facilitate the understanding of the method for face quality evaluation provided by the present embodiment, the application scenario of the method is first simply explained. The method for face quality evaluation in the present embodiment can be mainly applied in the field of face recognition, for example, can be applied in the application scenarios of person-certificate comparison, access control and criminal search, etc., and the face with better quality evaluation result can be used for high-quality face recognition in each of the above application scenarios. In addition, it can also be applied in related fields that need face quality evaluation, which is not specifically limited here.

[0069] It is considered that the evaluation robustness of the face quality evaluation method realized by the scoring method based on a plurality of fixed dimensions in the related art is not high, and cannot adapt to the needs of various application scenarios. Based on this, the present embodiment provides a method for face quality evaluation based on Gaussian distribution feature extraction to improve the robustness of face quality evaluation.

[0070] The first face image can be any image that needs to be evaluated for face quality, for example, a face image captured by a camera included in an access control system. The first face image can be one or more, for example, a plurality of face images captured in succession for any target person.

[0071] For any first face image, a face recognition network can be used to extract features from the first face image, and the extracted features are Gaussian distribution features representing uncertainty of the first face image, that is, the face recognition network in the embodiment of the present disclosure can map the first face image to a Gaussian distribution in a high-dimensional space.

[0072] Since the Gaussian distribution features in the embodiment of the present disclosure can represent the quality of the image to some extent, the Gaussian distribution features obtained based on the first face image can be used to determine the quality evaluation result of the first face image.

[0073] In a specific application, the Gaussian distribution features can be a Gaussian variance vector consistent with the dimension of the image feature vector of the first face image. Each dimension of the image feature vector can represent the image feature of the corresponding dimension, and the vector value of the Gaussian variance vector of the corresponding dimension can be used to represent the deviation value of the image feature of the corresponding dimension. The smaller the deviation value, the higher the image quality, and vice versa. Here, by averaging the vector values included in the Gaussian variance vector obtained based on the first face image, an average vector value can be obtained, and the obtained average vector value can be used as the quality evaluation result. The determined quality evaluation result is closer to the actual situation.

[0074] Considering the key role of the training of the face recognition network in extracting the Gaussian distribution features, the training process of the face recognition network will be described in detail as follows.

[0075] Step one, obtaining each second face image;

[0076] Step two, for each second face image in the plurality of second face images, extracting features from the second face image by the extraction layer of the face recognition network to obtain an image feature vector and a Gaussian distribution feature vector representing uncertainty of the second face image;

[0077] Step three, sampling the image feature vector and the Gaussian distribution feature vector in the same dimension to obtain a sampled image feature;

[0078] Step four, classifying the sampled image feature by the classification layer of the face recognition network to obtain a classification result, and comparing the classification result with a pre-labeled result of the second face image.

[0079] Step five, in response to the comparison result indicating that the classification result is inconsistent with the pre-labeled result, adjusting the network parameter value of the face recognition network, and performing the next round of network training until the comparison result obtained indicates that the classification result is consistent with the pre-labeled result.

[0080] The face recognition network here includes two output branches of the extraction layer, which are respectively used to predict the d-dimensional image feature vector μ and the Gaussian distribution feature vector σ. In the training stage, since the image feature vector and the Gaussian distribution feature vector adopt the same dimension feature vector, the image feature vector and the Gaussian distribution feature vector can be sampled in the same dimension to obtain the sampled image feature, for example, the image feature value and the Gaussian distribution feature value in the second dimension can be sampled at the same time, so as to analyze the image quality from a single feature dimension.

[0081] In the case of the classification layer included in the face recognition network after sampling the image feature, the iterative training of the network can be realized based on the comparison result between the obtained classification result and the pre-labeled result of the second face image, until the training is stopped, and the trained face recognition network is obtained.

[0082] In the embodiment of the present disclosure, in order to facilitate the determination of the quality evaluation result of each first face image in the same reference level, the initial quality score of the first face image can be determined first, and then the overall quality score is adjusted to obtain a more accurate adjusted quality score. Specifically, the following steps can be used to achieve this:

[0083] Step one, based on the Gaussian distribution features obtained by the plurality of first face images respectively, determining the initial quality score of each first face image;

[0084] Step two, based on the initial quality score of each first face image, determining the average quality score corresponding to each first face image, and the maximum quality score and the minimum quality score corresponding to each face class label pre-labeled for each first face image;

[0085] Step three, based on the maximum quality score and the minimum quality score corresponding to each face class label, and the average quality score, adjusting the initial quality score of each first face image under each face class label to obtain the adjusted quality score of the first face image.

[0086] Here, the average quality score corresponding to each first face image can be determined based on the initial quality score of each first face image, and the maximum quality score and the minimum quality score corresponding to each face class label pre-labeled for each first face image, and then the initial quality score of each first face image under each face class label is adjusted based on the above several quality scores.

[0087] Here, considering that the average quality score reflects the score of the overall sample, the average of the initial quality scores of each first face image can be determined. The maximum quality score and the minimum quality score corresponding to each face class label correspond to the score of the corresponding class label, which can be selected from the initial quality scores of each first face image belonging to the face class label, i.e., the maximum and minimum quality scores in a class label. It should be noted that the average quality score can also be obtained by methods such as taking the average after removing the maximum and minimum values, which is not limited here.

[0088] In the adjustment process for each face class label, a third difference between the initial quality score of each first face image under the face class label and the minimum quality score corresponding to the face class label can be determined, and a fourth difference between the maximum quality score and the minimum quality score corresponding to the face class label can be determined, and then the third difference and the fourth difference are ratio operated to determine the adjusted quality score of each first face image under the face class label.

[0089] Here, the quality score of each first face image can be adjusted to the quality distribution range within the minimum quality score and the maximum quality score, so that the quality evaluation is at the same reference level, and the subsequent network training can also be reduced. Underfitting caused by the relatively concentrated quality score corresponding to a class label.

[0090] Before the quality score adjustment, the deviation of the overall score under each face class label can be determined, and then the overall distribution adjustment is realized based on the deviation. Here, it can be determined whether the first difference between the maximum quality score corresponding to the face class label and the average quality score is greater than the second difference between the average quality score and the minimum quality score corresponding to the face class label, and based on the determination result, it can be determined whether the overall deviation is to the minimum quality score or to the maximum quality score.

[0091] In a case where it is determined that the overall deviation is towards the maximum quality score, i.e., the first difference is greater than the second difference, the minimum quality score corresponding to the face class label can be adjusted based on the maximum quality score corresponding to the face class label and the average quality score, and then the adjustment of the quality scores of the first face images within the face class label is realized through the adjusted minimum quality score.

[0092] Similarly, in a case where it is determined that the deviation is towards the minimum quality score, i.e., the first difference is less than the second difference, the maximum quality score corresponding to the face class label can be adjusted based on the minimum quality score corresponding to the face class label and the average quality score, and then the adjustment of the quality scores of the first face images within the face class label is realized through the adjusted maximum quality score.

[0093] In order to facilitate further understanding of the above adjustment process of the quality score, the following can be combined with the formula for specific description. Here, the quality score corresponding to the i-th face class label can be set as s i , the determined average quality score is s m , and the maximum quality score and the minimum quality score in the i-th face class label are s i max and s i min , respectively. Here, the following formula can be used for deviation:

[0094] If s i max -s m >s m -s i min , then s i min =2*s m -s i max .

[0095] If s i max -s m ≤s m -s i min , then s i max =2*s m -s i min .

[0096] Wherein,

[0097] It can be known that in the embodiment of the disclosure, the feature of each first face image is no longer a predicted vector, but a Gaussian distribution, and the variance σ of the distribution of each image can be considered as the uncertainty of the sample. By calculating the harmonic mean of σ, the basic initial quality score A can be obtained. Then, the initial quality score A is adjusted in some distribution, including normalization and stretching to an artificial intuitive distribution, to obtain the adjusted quality score B.

[0098] In the embodiments of the present disclosure, in a case where the quality evaluation result of each first face image is determined, the training of the face quality evaluation network can be performed. Here, the first face image can be used as input data of the face quality evaluation network to be trained, the quality evaluation result of the first face image can be used as contrast supervision data of output result of the face quality evaluation network to be trained, the face quality evaluation network to be trained is trained, and the trained face quality evaluation network is obtained.

[0099] In a specific application, the quality evaluation result used can be the initial quality score A based on the above, or the quality score B after distribution adjustment.

[0100] The face quality evaluation network training described above can be the corresponding relationship between the first face image and the quality score, so that in a case where the face quality evaluation network is trained, the quality evaluation for any to-be-evaluated image can be realized.

[0101] In the embodiments of the present disclosure, after the image score pair of each first face image is generated, a convolutional neural network can be used as the face quality evaluation network to train a regression task, and the network input is an image of h*w, and the number of channels can be determined according to the network size.

[0102] In a specific application, in some application scenarios such as automatic driving and the like with relatively high real-time requirements, network construction and updating need to be performed more quickly, and in this case, there is a clear requirement for network training speed. In the embodiments of the present disclosure, in a case where the network training speed is less than a preset threshold, the first face image can be subjected to grayscale processing, and the processed first face image is used as input data of the network.

[0103] In addition, in the embodiments of the present disclosure, RGB can also be used as input, the backbone layer in the network can use a small-scale network such as mobilenetv2, and the output is a single numerical value. After the numerical value is subjected to a sigmoid function, a prediction score can be obtained. By comparing the prediction score with the corresponding quality score in the image score pair, a loss function value can be determined, and the training of the network can be supported.

[0104] In an actual application scenario, for some special cases of specific scenarios, the training network and data can be fine-tuned. For example, the security check person certificate comparison pays more attention to motion blur and decoration occlusion. By adding such data or enriching the data augmentation scheme, the network can be more sensitive to the above low-quality conditions.

[0105] In a case where the face quality evaluation network is trained in the embodiments of the present disclosure, the fast quality evaluation of the to-be-evaluated image can be realized based on the evaluation network, and then the subsequent face comparison can be facilitated. The face comparison can be realized through the following steps:

[0106] Step one, obtaining an image to be evaluated;

[0107] Step two, inputting the image to be evaluated into the trained face quality evaluation network to obtain a quality score of the image to be evaluated;

[0108] Step three, in response to the quality score of the image to be evaluated being greater than a preset threshold, performing face comparison on the image to be evaluated to obtain a face comparison result.

[0109] Here, in the case of obtaining the quality score of the image to be evaluated based on the face quality evaluation network, it can be determined whether the quality score of the image to be evaluated is greater than the preset threshold, and in the case of determining that the quality score is high, face comparison can be performed to verify the related information of the face.

[0110] In actual application, the above-mentioned face quality evaluation network can be directly embedded into the face recognition process, used for screening out low-quality images in a sequence of images, or directly selecting the highest-quality image to be delivered to the subsequent recognition model; it can also be used for business scenarios such as ensuring high quality when storing faces.

[0111] Those skilled in the art can understand that in the above-mentioned method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined by its function and possible internal logic.

[0112] Based on the same inventive concept, the disclosure embodiments also provide a face quality evaluation device corresponding to the face quality evaluation method. Since the principle of solving problems in the device of the disclosure embodiments is similar to the above-mentioned face quality evaluation method of the disclosure embodiments, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described.

[0113] Referring to Figure 2 Fig. 1 shows a schematic diagram of a face quality evaluation device provided by the disclosure embodiments, which includes an acquisition module 201, an extraction module 202, and an evaluation module 203; wherein,

[0114] The acquisition module 201 is configured to acquire a first face image.

[0115] The extraction module 202 is configured to perform feature extraction on the first face image by using a face recognition network to obtain a Gaussian distribution feature for representing uncertainty of the first face image.

[0116] The evaluation module 203 is configured to determine a quality evaluation result of the first face image based on the Gaussian distribution feature obtained from the first face image.

[0117] The device for face quality evaluation can be used to extract features of the first face image by using the face recognition network, so as to determine the quality evaluation result of the sample based on the obtained Gaussian distribution features. This is mainly because the Gaussian distribution features can represent the uncertainty of the sample. The higher the uncertainty of the sample, the worse the quality of the sample. Conversely, the higher the certainty of the sample, the higher the quality of the sample. The Gaussian distribution features in the present disclosure fuse the relevant uncertainty features of the image. Compared with the traditional method of determining the image quality by relying on human logical rules, the sample quality can be more accurately evaluated based on the predicted uncertainty of each image, which is not disturbed and constrained by the scene environment and other factors, has high robustness, and thus meets the needs of various application scenarios.

[0118] In a possible implementation, the quality evaluation result includes a quality score, and the number of the first face images is multiple; the evaluation module 203 is configured to determine the quality evaluation result of the first face images based on the Gaussian distribution features obtained from the first face images according to the following steps:

[0119] determine the initial quality score of each first face image based on the Gaussian distribution features obtained from the multiple first face images respectively;

[0120] determine the average quality score corresponding to each first face image, and the maximum quality score and the minimum quality score corresponding to each face class label pre-labeled for each first face image based on the initial quality score of each first face image;

[0121] adjust the initial quality score of each first face image under each face class label based on the maximum quality score and the minimum quality score corresponding to each face class label, and the average quality score, to obtain the adjusted quality score of each first face image.

[0122] In a possible implementation, the evaluation module 203 is further configured to:

[0123] Before adjusting the initial quality score of each first face image under each face class label based on the maximum quality score, the minimum quality score and the average quality score corresponding to each face class label, for each face class label, it is determined whether a first difference between the maximum quality score and the average quality score corresponding to the face class label is greater than a second difference between the average quality score and the minimum quality score corresponding to the face class label; in response to the first difference being greater than the second difference, the minimum quality score corresponding to the face class label is adjusted based on the maximum quality score and the average quality score corresponding to the face class label to obtain an adjusted minimum quality score corresponding to the face class label; in response to the first difference being less than or equal to the second difference, the maximum quality score corresponding to the face class label is adjusted based on the minimum quality score and the average quality score corresponding to the face class label to obtain an adjusted maximum quality score corresponding to the face class label.

[0124] In a possible implementation, the evaluation module 203 is configured to adjust the initial quality score of each first face image under each face class label based on the maximum quality score, the minimum quality score and the average quality score corresponding to each face class label according to the following steps:

[0125] For each face class label, a third difference between the initial quality score of each first face image under the face class label and the minimum quality score corresponding to the face class label is determined, and a fourth difference between the maximum quality score and the minimum quality score corresponding to the face class label is determined.

[0126] The third difference and the fourth difference are subjected to ratio operation to determine an adjusted quality score of each first face image under the face class label.

[0127] In a possible implementation, the apparatus further includes:

[0128] The training module 204 is configured to, after determining the quality evaluation result of the first face image, take the first face image as input data of a to-be-trained face quality evaluation network, take the quality evaluation result of the first face image as comparison supervision data of an output result of the to-be-trained face quality evaluation network, train the to-be-trained face quality evaluation network, and obtain a trained face quality evaluation network.

[0129] In a possible implementation, the training module 204 is further configured to, before taking the first face image as input data of the to-be-trained face quality evaluation network and taking the quality evaluation result of the first face image as comparison supervision data of the output result of the to-be-trained face quality evaluation network, perform grayscale processing on the first face image to obtain a processed first face image in a case where a network training speed is less than a preset threshold.

[0130] In a possible implementation, the apparatus further includes:

[0131] The identification module 205 is configured to, after the trained face quality assessment network is obtained, acquire a to-be-evaluated image; input the to-be-evaluated image into the trained face quality assessment network to obtain a quality score of the to-be-evaluated image; and in response to the quality score of the to-be-evaluated image being greater than a preset threshold, perform face comparison on the to-be-evaluated image to obtain a face comparison result.

[0132] In a possible implementation, the face recognition network includes an extraction layer and a classification layer; and the extraction module 202 is configured to train the face recognition network according to the following steps:

[0133] Obtain each second face image;

[0134] For each second face image in the plurality of second face images, perform feature extraction on the second face image through the extraction layer of the face recognition network to obtain an image feature vector and a Gaussian distribution feature vector for representing uncertainty of the second face image;

[0135] Sample the image feature vector and the Gaussian distribution feature vector in the same dimension to obtain a sampled image feature;

[0136] Classify the sampled image feature through the classification layer of the face recognition network to obtain a classification result, and compare the classification result with a pre-labeled result of the second face image;

[0137] In response to the comparison result indicating that the classification result is inconsistent with the pre-labeled result, adjust a network parameter value of the face recognition network, and perform a next round of network training until the obtained comparison result indicates that the classification result is consistent with the pre-labeled result.

[0138] In a possible implementation, the Gaussian distribution feature includes a Gaussian variance vector consistent with a dimension of the image feature vector of the first face image; and the evaluation module 203 is configured to determine a quality evaluation result of the first face image based on the obtained Gaussian distribution feature of the first face image according to the following steps:

[0139] Average each vector value included in the Gaussian variance vector obtained for the first face image to obtain an average vector value;

[0140] Take the average vector value as the quality evaluation result of the first face image.

[0141] The description of the processing procedure of each module in the apparatus and the interaction procedure between the modules can refer to the related description in the method embodiments, which will not be repeated here.

[0142] This disclosure also provides an electronic device, such as... Figure 3 The diagram shown is a schematic representation of an electronic device structure provided in this embodiment of the present disclosure, including: a processor 301, a memory 302, and a bus 303. The memory 302 stores machine-readable instructions executable by the processor 301 (e.g., ...). Figure 2 The device includes modules 201 for acquisition, 202 for extraction, and 203 for evaluation (and corresponding execution instructions). When the electronic device is running, the processor 301 communicates with the memory 302 via the bus 303. When a machine-readable instruction is executed by the processor 301, the following processing is performed:

[0143] Obtain the first face image;

[0144] A face recognition network is used to extract features from the first face image to obtain a Gaussian distribution feature that characterizes the uncertainty of the first face image;

[0145] The quality assessment result of the first face image is determined based on the Gaussian distribution features obtained from the first face image.

[0146] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the face quality assessment method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0147] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the face quality assessment method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0148] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0149] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here. In several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and another division can be made in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0150] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0151] In addition, the functional units in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0152] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part of the prior art or the part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make an electronic device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present disclosure. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, and various program code storage media.

[0153] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not intended to limit the present disclosure. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easy changes to the technical solutions described in the foregoing embodiments, or easily think of changes or equivalent replacements for some of the technical features; and these modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for facial quality assessment, characterized in that, The method includes: Obtain the first face image; The first face image is used to extract features using a face recognition network to obtain Gaussian distribution features that characterize the uncertainty of the first face image; Based on the Gaussian distribution features obtained from the first face image, the quality assessment result of the first face image is determined; The quality assessment result includes a quality score, and the number of the first face images is multiple; determining the quality assessment result of the first face image based on the Gaussian distribution features obtained from the first face image includes: Based on the Gaussian distribution features obtained from multiple first face images, an initial quality score is determined for each first face image. Based on the initial quality score of each first face image, the average quality score corresponding to each first face image is determined, as well as the maximum and minimum quality scores corresponding to each face category label pre-annotated for each first face image. Based on the maximum and minimum quality scores corresponding to each face category label, and the average quality score, the initial quality scores of each first face image under each face category label are adjusted to obtain the adjusted quality scores of each first face image.

2. The method according to claim 1, characterized in that, Before adjusting the initial quality score of each first face image under each face category label based on the maximum and minimum quality scores corresponding to each face category label, and the average quality score, the method further includes: For each face category label, determine whether the first difference between the maximum quality score corresponding to the face category label and the average quality score is greater than the second difference between the average quality score and the minimum quality score corresponding to the face category label; In response to the first difference being greater than the second difference, the minimum quality score corresponding to the face category label is adjusted based on the maximum quality score and the average quality score corresponding to the face category label to obtain the adjusted minimum quality score corresponding to the face category label. In response to the first difference being less than or equal to the second difference, the maximum quality score corresponding to the face category label is adjusted based on the minimum quality score and the average quality score corresponding to the face category label to obtain the adjusted maximum quality score corresponding to the face category label.

3. The method according to claim 1 or 2, characterized in that, The adjustment of the initial quality score of each first face image under each face category label based on the maximum and minimum quality scores corresponding to each face category label, and the average quality score, includes: For each face category label, a third difference is determined between the initial quality score of each first face image under that face category label and the minimum quality score corresponding to that face category label, and a fourth difference is determined between the maximum quality score and the minimum quality score corresponding to that face category label. The third difference is compared with the fourth difference to determine the adjusted quality score of each first face image under the face category label.

4. The method according to claim 1, characterized in that, After determining the quality assessment result of the first face image, the method further includes: The first face image is used as input data for the face quality assessment network to be trained, and the quality assessment result of the first face image is used as comparative supervision data for the output result of the face quality assessment network to be trained. The face quality assessment network to be trained is then trained to obtain a trained face quality assessment network.

5. The method according to claim 4, characterized in that, Before using the first face image as input data for the face quality assessment network to be trained, and using the quality assessment result of the first face image as comparative supervision data for the output result of the face quality assessment network to be trained, the method further includes: If the network training speed is determined to be less than a preset threshold, the first face image is processed into grayscale to obtain the processed first face image.

6. The method according to claim 4 or 5, characterized in that, After obtaining the trained face quality assessment network, the method further includes: Acquire the image to be evaluated; The image to be evaluated is input into a trained face quality assessment network to obtain a quality score for the image to be evaluated. In response to the quality score of the image to be evaluated being greater than a preset threshold, face comparison is performed on the image to be evaluated to obtain the face comparison result.

7. The method according to claim 1, characterized in that, The face recognition network includes an extraction layer and a classification layer; the face recognition network is trained according to the following steps: Obtain each second face image; For each of the plurality of second face images, the extraction layer of the face recognition network is used to extract features from the second face image to obtain an image feature vector and a Gaussian distribution feature vector used to characterize the uncertainty of the second face image. The image feature vector and the Gaussian distribution feature vector are sampled in the same dimension to obtain the sampled image features; The sampled image features are classified through the classification layer of the face recognition network to obtain a classification result, and the classification result is compared with the pre-labeled result of the second face image. In response to the comparison result indicating that the classification result is inconsistent with the pre-labeled result, the network parameter values ​​of the face recognition network are adjusted, and the next round of network training is performed until the obtained comparison result indicates that the classification result is consistent with the pre-labeled result.

8. The method according to claim 1, characterized in that, The Gaussian distribution feature includes a Gaussian variance vector with the same dimension as the image feature vector of the first face image; determining the quality assessment result of the first face image based on the Gaussian distribution feature obtained from the first face image includes: The average vector value is obtained by averaging the individual vector values ​​included in the Gaussian variance vector obtained from the first face image. The average vector value is used as the quality assessment result of the first face image.

9. A device for facial quality assessment, characterized in that, The device includes: The acquisition module is used to acquire the first face image; The extraction module is used to extract features from the first face image using a face recognition network to obtain Gaussian distribution features that characterize the uncertainty of the first face image. An evaluation module is used to determine the quality evaluation result of the first face image based on the Gaussian distribution features obtained from the first face image. The quality assessment results include a quality score, and there are multiple first face images; the assessment module 203 is used to determine the quality assessment result of the first face image based on the Gaussian distribution features obtained from the first face image according to the following steps: Based on the Gaussian distribution features obtained from multiple first face images, the initial quality score of each first face image is determined. Based on the initial quality score of each first face image, determine the average quality score corresponding to each first face image, as well as the maximum and minimum quality scores corresponding to each face category label pre-annotated for each first face image; Based on the maximum, minimum, and average quality scores corresponding to each face category label, the initial quality scores of each first face image under each face category label are adjusted to obtain the adjusted quality score of each first face image.

10. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the face quality assessment method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the face quality assessment method as described in any one of claims 1 to 8.

Images