Method, device and computer equipment for judging face image quality

Abstract

The present application provides a method, device and computer equipment for determining the quality of facial images. After obtaining the initial sorting of the facial images to be determined by comparing the features of the quality determination image with the facial images to be determined of the same user, a second number of facial image sample pairs are screened out from the first number of facial images to be determined, and the quality annotation results of the facial image sample pairs by the third number of users are integrated to obtain a more accurate target quality score for each facial image to be determined. It can be seen that the present application targets various landing scenarios, combines the quality annotation of the facial images to be determined by users with different preferences, that is, takes into account various factors affecting the image quality in the image acquisition scenario, improves the accuracy of the quality scores of the facial images to be determined, and thus improves the output accuracy of the facial image quality prediction model trained thereby.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and computer equipment for determining the quality of a face image. Background Art

[0002] In recent years, face recognition technology has been widely used in fields such as security monitoring. Face capture is an important part of face recognition. The quality of the captured face image directly affects the efficiency and accuracy of face recognition.

[0003] In practical applications, in order to obtain high-resolution facial images, the size of the face detection box is usually used to screen multiple faces in the same scene. However, this method cannot determine whether there are problems such as occlusion, light, motion blur, etc. on the face, which reduces the quality of the screened face images and thus affects the face recognition effect.

[0004] In this regard, the art proposes to detect blurry, overexposed and dark facial images at a low cost based on manually designed visual features, so as to improve the quality of facial images used for face recognition. However, this method is often only applicable to scenes with simple environments and has great limitations. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method, apparatus and computer equipment for determining the quality of facial images, which improve the accuracy of determining the quality of facial images.

[0006] In order to solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0007] The present application provides a method for determining the quality of a face image, the method comprising:

[0008] Acquire a first number of pending facial images and quality determination images corresponding to the first number of pending facial images, wherein the quality determination images refer to high-quality facial images that belong to the same user as the corresponding pending facial images;

[0009] Obtaining an initial quality ranking of the first number of the to-be-determined face images by using a feature comparison result of the to-be-determined face images and the corresponding quality determination images;

[0010] Selecting a second number of face image sample pairs from the first number of pending face images;

[0011] Obtaining a third number of quality labeling results corresponding to each of the second number of face image sample pairs;

[0012] The initial quality ranking is updated using the third number of quality annotation results to obtain target quality scores for each of the first number of pending facial images.

[0013] The present application also provides a facial image quality determination device, the device comprising:

[0014] A pending facial image acquisition module is used to acquire a first number of pending facial images and quality determination images corresponding to the first number of pending facial images, wherein the quality determination images refer to high-quality facial images belonging to the same user as the corresponding pending facial images;

[0015] A feature comparison module, configured to obtain an initial quality ranking of the first number of the to-be-determined face images by using a feature comparison result of the to-be-determined face images and the corresponding quality determination images;

[0016] A sample pair screening module, used for screening a second number of human face image sample pairs from the first number of pending human face images;

[0017] A quality labeling result acquisition module, used to acquire a third number of quality labeling results corresponding to each of the second number of face image sample pairs;

[0018] An updating module is used to update the initial quality ranking by using the third number of quality labeling results to obtain a target quality score for each of the first number of pending face images.

[0019] The present application also provides a computer device, the computer device comprising:

[0020] Communication interface;

[0021] A memory, used for storing a program for implementing the above-mentioned method for determining the quality of facial images;

[0022] The processor is used to load and execute the program to implement the various steps of the above-mentioned method for determining the quality of facial images.

[0023] Based on the above technical solution, the present application provides a method, device and computer equipment for determining the quality of facial images. The initial ranking of each pending facial image is obtained by comparing the feature of the quality determination image with the pending facial image of the same user. After that, a second number of facial image sample pairs are screened out from the first number of pending facial images, and the quality annotation results of the facial image sample pairs by the third number of users are integrated to obtain a more accurate quality score for each pending facial image, that is, the initial quality ranking is updated using the third number of quality annotation results, and then the target quality score of each of the first number of pending facial images is obtained. It can be seen that the present application targets various landing scenarios, combines the quality annotation of pending facial images by users with different preferences, that is, takes into account various factors affecting image quality in the image acquisition scenario, improves the accuracy of the quality score of each pending facial image obtained thereby, and then improves the output accuracy of the facial image quality prediction model trained thereby. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0025] Figure 1 A hardware structure diagram of a computer device provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of an application scenario of a computer device provided in an embodiment of the present application;

[0027] Figure 3 A flowchart of an embodiment of a method for determining the quality of a facial image provided by the present application;

[0028] Figure 4 A flowchart of a detailed embodiment of the method for determining the quality of a facial image provided by the present application;

[0029] Figure 5 A schematic diagram of an application scenario flow of a method for determining the quality of a facial image provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of the structure of a facial image quality determination device provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of the structure of another facial image quality determination device provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of the structure of another facial image quality determination device provided in an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of the structure of another facial image quality determination device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] It should be noted that the facial image quality determination method provided in the embodiment of the present application can be applied to a computer device with data processing capabilities, which may be a server set up on the network side, or a terminal device such as a PC (personal computer) set up on the user side. The present application does not limit the product type of the computer device. The computing device can implement the facial image quality determination method provided in the embodiment of the present application by loading a program corresponding to the facial image quality determination method provided in the embodiment of the present application. The program can be stored in the memory of the computing device and called by the processor to implement the program function.

[0036] Reference Figure 1 , shows a hardware structure block diagram of a computer device provided in an embodiment of the present application, the computer device may include but is not limited to: a communication interface 11, a memory 12 and a processor 13, wherein:

[0037] In the embodiment of the present application, the number of the communication interface 11, the memory 12, and the processor 13 can be at least one, and the communication interface 11, the memory 12, and the processor 13 can communicate with each other through a communication bus.

[0038] Optionally, the communication interface 11 can be an interface of a communication module, such as an interface of a GSM module, which is used to realize data interaction with other devices, such as receiving facial images, such as pending facial images for model training, and test facial images that need to be tested for quality judgment. It can also be used to obtain intermediate data generated in the process of facial image quality judgment.

[0039] The processor 13 may be a central processing unit CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the facial expression synthesis method provided in the embodiment of the present application.

[0040] The memory 12 may include a high-speed RAM memory, or may include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0041] In the present application, the memory 12 can store a program consisting of multiple instructions for implementing the facial image quality determination method provided in the embodiment of the present application. The processor 13 calls and loads the program stored in the memory 13, thereby implementing the facial image quality determination method provided in the embodiment of the present application. The specific implementation process can refer to the description of the corresponding embodiment below.

[0042] It should be noted that Figure 1 The hardware structure of the computer device shown is only optional. According to the needs of use, the computing device can also be provided with output devices such as display screens, voice announcers, information input devices (keyboards, mice, etc.), image collectors, etc., which will not be described in detail in this application.

[0043] The following is an introduction to the facial image quality determination method provided in the embodiment of the present application from the perspective of a computer device. The method steps described below can be implemented by executing corresponding programs by a computer device.

[0044] In the actual application of this application, in various landing scenarios, such as Figure 2 After capturing a face image, the various cameras, terminal devices and other image acquisition devices shown can send the face image to a computer device wirelessly, and the computer device obtains the quality score of the face image according to the face image quality determination method described in the method embodiment below. The specific implementation process can refer to the description of the corresponding part below. It should be noted that the application scenario of the face image quality determination method in the landing scenario is not limited to Figure 2 The application scenarios shown in the figure, but the implementation methods for determining the quality of facial images in each implementation scenario are similar, and this application will not describe them in detail one by one.

[0045] Reference Figure 3 , shows a flow chart of an embodiment of a method for determining the quality of a face image provided by the present application, which method can be applied to a computer device such as Figure 3 As shown, the method may include but is not limited to the following steps:

[0046] Step S11, obtaining a first number of pending facial images and quality determination images corresponding to the pending facial images;

[0047] Among them, the pending facial images may be facial images that require quality annotation, and specifically may be facial images of one or more users captured by an image acquisition device at different times in the same scene or in different scenes. Therefore, the data on shooting angle, light, occlusion, blur, etc. of the first number of pending facial images obtained in this embodiment are often different, and the specific value of the first number is not limited.

[0048] The quality judgment image can be a high-quality facial image of the user corresponding to the pending facial image, which can be used as a standard image for judging the quality of the corresponding pending facial image collected, and can specifically be an ID photo indicating the identity of the user. In this embodiment, the ID photo corresponding to the pending facial image can be obtained from the ID library of the household registration department and other departments as the quality judgment image of the pending facial image, but is not limited to this acquisition method. The present application can also use other methods to obtain high-quality quality judgment images, such as using a face recognition model to select the face image with the highest score from the face image set of the corresponding user, that is, the face image with a high face recognition degree as the quality judgment image of the user, etc., which will not be described in detail in this application.

[0049] The first number of pending facial images obtained by the computer device can be directly obtained from the video monitoring system in each scene, or can be obtained by an image acquisition device after collecting facial images of at least one user in multiple scenes and uploading them, etc. This embodiment does not limit the method for obtaining multiple pending facial images.

[0050] Step S12, performing feature comparison on the first number of pending face images and the corresponding quality determination images to obtain feature comparison results;

[0051] Step S13, using the feature comparison results of the first number of to-be-determined facial images, determining the initial quality ranking of the first number of to-be-determined facial images;

[0052] This embodiment does not limit the method of how to implement the feature comparison of the two images in step S12. The pre-trained face recognition model can be used to obtain the similarity between each pending face image and the corresponding quality judgment image. Thereafter, the similarity can be used to determine the initial quality score of the corresponding pending face image, such as directly using the similarity as the initial quality score of the corresponding pending face image, or determining the quality score corresponding to each similarity according to the set conversion relationship between the quality score and the similarity, and using it as the initial quality score of the corresponding pending face image, etc.

[0053] Among them, the face recognition model can be obtained by training the pending face images of multiple users based on a neural network. The specific training process of the face recognition model is not described in detail in this application. The face recognition model can input the pending face image and the corresponding high-quality quality judgment image, and output the similarity between the pending face image and the quality judgment image, that is, the probability that the pending face image is a high-quality image. This embodiment can use the similarity with the quality judgment image to measure the richness of the information retained by the face capture image. Generally, the clearer the pending face image and the smaller the face angle, the higher the similarity of the information read of the face image with the corresponding quality judgment image; conversely, the lower the similarity.

[0054] In this case, the higher the initial quality score of the pending facial image, the higher the recognition similarity between the image and the user himself. The pending facial image has features that are friendly to the facial recognition algorithm. When identity recognition is performed on such pending facial images subsequently, the user identity of the pending facial image can be identified more quickly and accurately.

[0055] Optionally, for the feature comparison between the pending face image and the corresponding quality determination image, feature extraction can be performed separately, and then the extracted similar features can be compared to obtain a feature comparison result, and the feature comparison result characterizes the similarity between the pending face image and the quality determination image; of course, the present application can also use other similarity algorithms to achieve feature comparison between each pending face image and the corresponding quality determination image, which will not be described in detail in the present application. In the actual application of this embodiment, there is often a certain amount of noise in the feature comparison process of step S12, which reduces the accuracy of the obtained feature comparison result. In order to improve the accuracy of the quality determination result of the pending face image, the present application can further denoise the initial quality score obtained above. The specific implementation process can refer to the description of each step below.

[0056] Step S14, screening a second number of face image sample pairs from the first number of pending face images;

[0057] In this embodiment, the method described in the above steps is referred to. When the quality of the first number of pending face images is determined to be basically ordered, that is, after obtaining an ordered initial quality ranking according to the initial quality score of each pending face image, the quality can be annotated by a multi-person manual annotation method, and the initial quality score of each pending face image obtained in the above method can be corrected to ensure that the best possible quality judgment result is obtained under limited manual annotation. In this embodiment, the face image sample pair can be composed of any two randomly selected pending face images. The second number of face image sample pairs usually includes the first number of pending face images, but the specific value of the second number is not limited. In order to facilitate the description of the technical solution provided by the present application, the first number can be recorded as m, and the second number can be recorded as C. C can be the total number of face image sample pairs composed of m pending face images. Of course, the value of C can also be less than the total number. The present application mainly uses C as the total number as an example for explanation below. The implementation scheme of C taking other data is similar, and the present application does not describe them one by one.

[0058] It can be seen that C face image sample pairs contain m undetermined face images. If the undetermined face images are denoted as O i , i = 1, 2, ..., m, then the screened C face image sample pairs may include (O1, O2), (O1, O3), ..., (O1, O m )、…、(O m-1 ,O m ), combined with the above description of obtaining the corresponding part of the pending facial image, the facial image sample pairs obtained in this embodiment may include facial image sample pairs composed of different pending facial images of the same user, and may also include facial image sample pairs composed of pending facial images of different users.

[0059] Optionally, the present application may also sort the first number of pending face images according to their initial quality, and select two pending face images whose sorting position difference is less than a second threshold value (the specific value is not limited, and a relatively small value can be selected) from the first number of pending face images as a face image sample pair, to obtain a second number of face image sample pairs. That is, two pending face images with a small initial quality score difference are selected to form a face image sample pair, but this is not limited to this implementation method. Usually, in order to obtain positive and negative training samples for model training, pending face image samples with higher initial quality are usually selected to form a face image sample pair, and pending face image samples with lower initial quality can also be selected to form a face image sample pair. The specific screening process will not be described in detail one by one in this embodiment.

[0060] Step S15, obtaining a third number of quality labeling results corresponding to each of the second number of facial image sample pairs;

[0061] In this embodiment, the method of manual labeling by multiple people can be used to perform denoising on the pending face images. Therefore, after obtaining C face sample pairs, n (i.e., the third number, the specific value is not limited) users can manually compare the quality of the two pending face images in each face image sample pair to determine which pending face image in the face sample pair has higher quality and which pending face image has lower quality. That is, the n users manually label each face image sample pair respectively, so that each face image sample pair obtains n quality labeling results.

[0062] Specifically, the present application can output the obtained C facial image sample pairs for n selected users to manually annotate the quality, and obtain the quality annotation results of each facial image sample pair, that is, the quality comparison results of the two pending facial images in the facial image sample pair, and then based on this, the initial sorting of the pending facial images (that is, the sorting obtained by step S14) is corrected to obtain new quality sorting results of the m pending facial images, but it is not limited to the manual annotation method described in this embodiment.

[0063] The selected n users may be ordinary users of different age groups, professional users in the field of image acquisition and processing, or randomly selected users. The present application does not limit the identities of the n users who need to manually annotate the quality of face images. It should be understood that in actual applications, the quality of the annotations of the same pending face image by different users is often not consistent, so the quality annotation results of the same face image sample pair by different users may be different.

[0064] If the user who performs manual annotation is denoted as u j , j = 1, 2, ..., n, the present application can use the quality labeling results of n users on C facial image sample pairs to obtain a quality comparison matrix of n*C facial image sample pairs, the rows of the matrix represent the quality labeling results of one user among the n users on the C facial image sample pairs, and the columns of the matrix represent the quality labeling results of n users on one facial image sample pair among the C facial image sample pairs, referring to the n*C quality comparison matrix shown below, but not limited to this:

[0065]

[0066] In the above n*C quality comparison matrix, O i >O i+1 It can be represented by the pending face image O i The quality of the face image O i+1The element 1 in the matrix can indicate that the user in the row agrees with the quality comparison result of the face image sample pair in the column, -1 can indicate that the user in the row disagrees with the quality comparison result of the face image sample pair in the column, and ? can indicate that the user in the row has not marked the quality comparison result of the face image sample pair in the column.

[0067] Taking element 1 in the first row and first column of the above-mentioned quality comparison matrix as an example, the element 1 can indicate that for the face image sample pair consisting of the pending face image O1 and the pending face image O2, the user u1 determines that the quality of the pending face image O1 is higher than that of the pending face image O2, that is, the user agrees that O1>O2; the element -1 in the second row and first column of the matrix can indicate that the user u2 determines that the quality of the pending face image O1 is lower than that of the pending face image O2, that is, the user disagrees that O1>O2.

[0068] Step S16, using the third number of quality labeling results, updating the initial quality ranking of the first number of pending facial images, and obtaining the target quality score of each of the first number of pending facial images.

[0069] This embodiment can use the second number of facial image samples to correspond to the third number of quality annotation results, generate a quality comparison matrix, and correct it by matrix filling method to obtain a quality score matrix, and then optimize and solve it by loss function to obtain the target quality score of each of the first number of pending facial images contained therein.

[0070] In summary, by using the feature comparison results of the quality judgment image and the pending facial images of the same user, the initial ranking of the pending facial images is obtained. After that, a second number of facial image sample pairs are screened out from the first number of pending facial images, and the quality annotation results of the facial image sample pairs of a third number of users are integrated to obtain a more accurate quality score for each pending facial image. That is, the third number of quality annotation results are used to update the initial quality ranking, and then the target quality score of each of the first number of pending facial images is obtained.

[0071] It can be seen that this application targets various landing scenarios and combines the quality annotation of pending facial images by users with different preferences, that is, taking into account various factors affecting image quality in the image acquisition scenario, thereby improving the accuracy of the quality scores of each pending facial image obtained thereby, and thereby improving the output accuracy of the facial image quality prediction model trained thereby.

[0072] Reference Figure 4 , is a flowchart of a detailed embodiment of the face image quality determination method of the present application, which can be applied to computer devices such as Figure 4As shown, the method may include but is not limited to the following steps:

[0073] Step S21, obtaining a first number of pending facial images and quality determination images corresponding to the pending facial images;

[0074] Step S22, performing feature comparison on the first number of pending face images and the corresponding quality determination images to obtain feature comparison results;

[0075] Step S23, using the feature comparison results of the first number of undetermined facial images, determining the initial quality scores of the corresponding undetermined facial images;

[0076] Step S24, sorting the first number of pending face images according to the initial quality scores;

[0077] Step S25, selecting a second number of human face image sample pairs from the sorted first number of pending human face images;

[0078] Step S26, obtaining a third number of quality labeling results corresponding to each of the second number of facial image sample pairs;

[0079] Regarding the implementation process of step S21 to step S26, reference may be made to the description of step S11 to step S15 and corresponding parts of the above embodiment, which will not be repeated in this embodiment.

[0080] Step S27, using the second number of facial image samples to respectively correspond to the third number of quality annotation results, to obtain a quality score matrix with a second number of rows and a first number of columns;

[0081] In this embodiment, as described above, the third number of quality labeling results corresponding to the second number of face image samples will constitute the above quality comparison matrix. On this basis, this embodiment can process the quality comparison matrix in a matrix completion manner to obtain an n*m quality score matrix. The specific implementation process is not limited in this application.

[0082] Optionally, for each of the n users who perform manual annotation, the quality of the C facial image sample pairs can be separately labeled, that is, the quality of the two pending facial images in each facial image sample pair can be manually labeled. At this time, the quality labeling result obtained can be determined by the user's preference for judging the quality of facial images, and based on the quality labeling results of the C facial image sample pairs labeled by the same user, a ranking result of the quality heights of the m pending facial images can be obtained.

[0083] After that, the basic quality score of the above sorting results can be determined, that is, a sample quality score is determined for the lowest quality pending face image, and then the sample quality scores of other pending face images are determined by increasing the predetermined step size in sequence according to the sorting order (i.e., from the quality of the pending face images from low to high). In this way, the sample quality scores corresponding to n groups of m pending face images can be obtained, and the sample quality scores corresponding to these n groups of m pending face images can form an n*m quality score matrix.

[0084] For example: take the quality labeling results of C facial image sample pairs by user u1 as an example. Since the quality labeling results of these C facial image sample pairs show the comparison results of the qualities of any two of the m pending facial images, the quality ranking results of the m pending facial images can be obtained from these C quality comparison results. Therefore, based on the quality labeling results of the C facial image sample pairs by each user, a ranking result of the qualities of the m pending facial images can be obtained. Based on the quality labeling results of the C facial image sample pairs by n users, n ranking results of the qualities of the m pending facial images can be obtained.

[0085] Afterwards, for any ranking result of the quality of the m pending face images, the lowest quality sample quality score can be defined as 0, and the quality order from low to high is increased by 0.1, to obtain the sample quality scores corresponding to the m pending face images under this ranking, but it is not limited to the method of determining the sample quality scores described in this embodiment. In this way, after obtaining the sample quality scores of n groups of m pending face images, the obtained n*m quality score matrix can be as follows:

[0086]

[0087] Step S28, using a preset loss function to perform rank reduction processing on the quality score matrix to obtain a target quality score matrix with a fourth number of rows and a first number of columns;

[0088] As analyzed above, n users often have different preferences for judging the quality of facial images, that is, different users often have different standards for determining high-quality facial images. Specifically, they may have different requirements for various parameters such as shooting angle, blur, light, occlusion, etc. of high-quality pending facial images. The present application may regard a requirement for these parameters as a user's preference for high-quality facial images.

[0089] Therefore, assuming that there are k (i.e., the fourth number) preferences among n users, in this way, in the process of optimizing and solving the above-mentioned n*m quality score matrix, the rank of the quality score matrix can be made close to k, and an optimized matrix of k rows and m columns, i.e., the target quality score matrix, can be obtained.

[0090] Among them, this embodiment does not limit the specific value of the fourth number k. When n users are selected, the preference types k of these n users for facial image quality judgment can be determined. The quality labeling results of m pending facial images by n users can be classified to obtain the number of preference types of these n users for facial image quality judgment, that is, the number of types obtained by classification. Of course, other methods can also be used to determine the value of k, which will not be described in detail in this application.

[0091] It can be seen that this embodiment can obtain a fourth number of quality labeling preferences represented by the third number of quality labeling results corresponding to the second number of facial image samples, and use a preset loss function to reduce the rank of the quality score matrix so that the difference between the rank of the quality score matrix and the fourth number is less than a first threshold (the value of which is not limited and is usually relatively small), and obtain a target quality score matrix with a fourth number of rows and a first number of columns.

[0092] Optionally, the preset loss function may be a monotonically decreasing convex function l(Z)=max(0,1-z), which can be used to measure the consistency between the sorting matrix F (i.e., the quality score matrix) and the observed comparison pairs (i.e., the face image sample pairs) in the set Ω (i.e., the set of face images to be determined). i The quality annotation result O of a face image sample pair j >O k , the consistency between F and set Ω can be expressed by l(F i,j -F i,k ) is measured. Generally speaking, the smaller the loss is, the greater the consistency between F and the set Ω is. Based on this, in the process of optimizing the matrix F, we can find F∈R n*m The low-rank matrix, from which the initial loss function can be:

[0093]

[0094] The ranking function rank(F) is a non-convex function. This embodiment can use the trace norm of the matrix tracenorm|F tr , to replace the ranking function rank(F) in the above formula (3), and transform the optimization problem of the above formula (3) into a convex function optimization problem, thus obtaining the following formula:

[0095]

[0096] In this embodiment, the above formula (4) can be used as a preset loss function to optimize the quality score matrix obtained above. The specific processing process is not described in detail in this embodiment.

[0097] Step S29, performing mean normalization processing on the sample quality scores in each column of the target quality score matrix to obtain target quality scores of the first number of pending face images;

[0098] As analyzed above, after matrix optimization processing, a k*m target quality score matrix will be obtained, which contains the annotation results of the quality of each pending face image by users with different preferences. In this way, for each pending face image, the target quality score of the pending face image can be obtained by taking the average and normalizing the annotation results of users who do not like it. In this way, a 1*m matrix can be obtained, which is a one-dimensional vector composed of the target quality scores of m pending face images.

[0099] It can be seen that the quality scores of each pending face image obtained by the above method in this embodiment integrate the judgment preferences of multiple people on the face image quality, and make the face image quality judgment result conform to the user's senses on the basis of recognition friendliness, thereby improving the accuracy and reliability of the face image quality judgment result. Step S210, using a deep neural network to train a first number of pending face images marked with target quality scores to obtain a face image quality prediction model;

[0100] In this embodiment, the target quality scores of the first number of pending facial images are obtained in the above manner, and the pending facial images with the target quality scores are used as training data. A deep neural network is used to implement the training of the facial image quality prediction model. The specific training process is not limited in this embodiment.

[0101] Optionally, the present application may use the SGD (stochastic gradient descent) algorithm to continuously train the training data obtained in the above manner to obtain a face image prediction model, but is not limited to this training method.

[0102] Step S211, obtaining a face image to be tested;

[0103] Step S212: input the face image to be tested into the face image quality prediction model to obtain the predicted quality score of the face image to be tested.

[0104] In the actual application of this embodiment, after the facial image quality prediction model is obtained in the above manner, it can be stored. In this way, when it is necessary to record a certain facial image as a test facial image for quality judgment, the test facial image can be used as a model input, and the facial image quality prediction model is input to obtain the predicted quality score of the test facial image online accurately and quickly.

[0105] In summary, by using the feature comparison results of the quality judgment image and the pending facial images of the same user, the initial ranking of the pending facial images is obtained. After that, a second number of facial image sample pairs are screened out from the first number of pending facial images, and the quality annotation results of the facial image sample pairs of a third number of users are integrated to obtain a more accurate quality score for each pending facial image. That is, the third number of quality annotation results are used to update the initial quality ranking, and then the target quality score of each of the first number of pending facial images is obtained.

[0106] It can be seen that this application targets various implementation scenarios and combines the quality annotation of pending facial images by users with different preferences, that is, taking into account various factors affecting image quality in the image acquisition scenario, thereby improving the accuracy of the quality scores of each pending facial image obtained thereby, and thereby improving the output accuracy of the facial image quality prediction model trained thereby.

[0107] Based on the face image quality determination method described above, the following will be explained in combination with specific application scenarios. Figure 5 The flowchart of an application scenario of the facial image quality determination method shown in the figure, the computer device can obtain facial images captured by different users in various scenarios, use them as pending facial images, and the ID photos corresponding to each user, calculate the similarity between each pending facial image and the corresponding ID photo, and sort the m pending facial images according to the similarity, and determine the initial quality score of each of the m pending facial images.

[0108] Afterwards, a multi-person manual annotation and denoising method will be used to obtain a more objective quality ranking of the m pending face images, and the different subjective preferences of different users in the definition of face image quality in multi-person annotation will be integrated. After C face image sample pairs are screened out from the m pending face images, n users will perform quality annotation on them, that is, judge the quality of the two pending face images in each face image sample pair. Afterwards, based on the quality annotation results of the C face image sample pairs by the same user, the sample quality scores of the m pending face images are obtained, and an n*m quality score matrix is ​​formed, in which each row represents the annotation result of a user, and each column represents the annotation score of a face image sample pair.

[0109] Furthermore, the number k of quality preference types existing in n users is determined, and the n*m ​​quality score matrix is ​​optimized by rank reduction so that the rank of the quality score matrix is ​​close to k. The k rows of compressed results are averaged and normalized to obtain the target quality scores of the m pending face images. Afterwards, the m pending face images and their corresponding target quality scores are used as training data, and a deep neural network is used to train the model to obtain and store the face image quality prediction model.

[0110] In this case, when it is necessary to make a quality judgment on any face image to be tested, the face image to be tested can be directly input into the face image quality prediction model to quickly and accurately obtain the predicted quality score of the face image to be tested. The quality score obtained at this time is based on recognition friendliness and takes into account the preferences of different users for face image quality judgment, so that the predicted quality score is more in line with human perception.

[0111] Reference Figure 6 , is a schematic diagram of the structure of a facial image quality judgment device provided in an embodiment of the present application, which can be applied to a computer device, such as Figure 6 As shown, the device may include:

[0112] The pending facial image acquisition module 21 is used to acquire a first number of pending facial images and quality determination images corresponding to the first number of pending facial images;

[0113] The quality determination image may refer to a high-quality face image that belongs to the same user as the corresponding pending face image. The acquisition method of the pending face image and the quality determination image may refer to the description of the corresponding part of the above method embodiment, which will not be repeated in this embodiment.

[0114] A feature comparison module 22, configured to obtain an initial quality ranking of the first number of the to-be-determined face images using a feature comparison result between the to-be-determined face images and the corresponding quality determination images;

[0115] Optionally, the feature comparison module 22 may include:

[0116] A similarity determination unit, used to determine the similarity between the to-be-determined face image and the corresponding quality determination image;

[0117] An initial quality score determination unit, used to determine the initial quality score of the corresponding to-be-determined face image according to the similarity;

[0118] The quality ranking unit is used to sort the first number of pending face images according to the size of the initial quality scores to obtain an initial quality ranking.

[0119] A sample pair screening module 23, configured to screen a second number of human face image sample pairs from the first number of pending human face images;

[0120] Continuing the description of the optional embodiments above, the sample pair screening module 23 can be specifically used to sort according to the initial quality, and screen two pending face images whose sorting position difference is less than a second threshold from the first number of pending face images as a face image sample pair, to obtain a second number of face image sample pairs, but is not limited to this sample pair screening method.

[0121] A quality labeling result acquisition module 24 is used to acquire a third number of quality labeling results corresponding to each of the second number of face image sample pairs;

[0122] The updating module 25 is used to update the initial quality ranking by using the third number of quality labeling results to obtain the target quality score of each of the first number of pending face images.

[0123] As an optional embodiment of the present application, Figure 7 As shown, based on the above embodiment, the updating module 25 may specifically include:

[0124] The quality score matrix acquisition unit 251 is used to obtain a quality score matrix with a second number of rows and a first number of columns by using the third number of quality annotation results corresponding to the second number of face image samples respectively.

[0125] Optionally, in the practical application of the present application, the quality score matrix acquisition unit 251 may include:

[0126] A correction subunit is used to obtain the sample quality scores corresponding to the first number of undetermined face images in the second number group by using the third number of quality annotation results corresponding to the second number of face image samples respectively according to the matrix filling method;

[0127] The matrix generating subunit is used to form a quality score matrix with a second number of rows and a first number of columns based on the sample quality scores corresponding to the first number of pending facial images in the second number groups.

[0128] A rank reduction processing unit 252 is used to perform rank reduction processing on the quality score matrix by using a preset loss function to obtain a target quality score matrix with a fourth number of rows and a first number of columns, wherein the fourth number is less than the third number;

[0129] Optional, such as Figure 8 As shown, the rank reduction processing unit 252 may specifically include:

[0130] Instruct the labeling preference acquisition subunit 2521 to acquire a fourth number of quality labeling preferences represented by the third number of quality labeling results corresponding to the second number of facial image samples;

[0131] The rank reduction processing subunit 2522 is used to use a preset loss function to perform rank reduction processing on the quality score matrix so that the difference between the rank of the quality score matrix and the fourth number is less than a first threshold, and obtain a target quality score matrix with a fourth number of rows and a first number of columns.

[0132] The normalization processing unit 253 is used to perform mean normalization processing on the sample quality scores in each column of the target quality score matrix to obtain target quality scores of the first number of pending face images.

[0133] Optionally, based on the above embodiments, Fig. 9 As shown, the device may also include:

[0134] A model training module 26, configured to use a pending face image with a target quality score as training data, perform model training on the training data according to a deep neural network, and obtain a face image quality prediction module;

[0135] The face image acquisition module 27 is used to acquire the face image to be tested;

[0136] The quality prediction module 28 is used to input the face image to be tested into the face image quality prediction module to obtain the predicted quality score of the face image to be tested.

[0137] It should be noted that each module or unit described in the above-mentioned device embodiments belongs to a program function module, and the process of realizing the corresponding function can refer to the description of the corresponding part of the above-mentioned method embodiment, which will not be described in detail one by one in this embodiment.

[0138] An embodiment of the present application also provides a storage medium on which a computer program is stored. The computer program is executed by a processor to implement the steps of the above-mentioned facial image quality determination method. The specific implementation process can refer to the description of the above-mentioned method embodiment.

[0139] The embodiment of the present application further provides a processor for loading and executing a program for implementing the above-mentioned facial image quality determination method, wherein the program is mainly used for:

[0140] Acquire a first number of pending facial images and quality determination images corresponding to the first number of pending facial images, wherein the quality determination images refer to high-quality facial images that belong to the same user as the corresponding pending facial images;

[0141] Obtaining an initial quality ranking of the first number of the to-be-determined face images by using a feature comparison result of the to-be-determined face images and the corresponding quality determination images;

[0142] Selecting a second number of face image sample pairs from the first number of pending face images;

[0143] Obtaining a third number of quality labeling results corresponding to each of the second number of face image sample pairs;

[0144] The initial quality ranking is updated using the third number of quality annotation results to obtain target quality scores for each of the first number of pending facial images.

[0145] It should be understood that the processor can also implement other steps of the method for determining the quality of facial images by executing the program. The specific implementation process can refer to the description of the corresponding part of the above method embodiment.

[0146] The present application also provides a computer device. Figure 1 and 2 The computer device may include a communication interface 11, a memory 12 and a processor 13, wherein the memory 12 is used to store a program for implementing the facial image quality determination method as described above; the processor 13 is used to load and execute the program stored in the memory 12 to implement the various steps of the facial image quality determination method as described above, but the composition structure of the computer device is not limited to, and can also refer to the description of the corresponding part of the above embodiment, which can be determined according to the product type of the computer device, and this application will not describe them one by one.

[0147] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the devices and computer equipment disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0148] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0149] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0150] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the core idea or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining the quality of a face image, characterized in that: The method comprises: Acquire a first number of pending facial images and quality determination images corresponding to the first number of pending facial images, wherein the quality determination images refer to high-quality facial images that belong to the same user as the corresponding pending facial images; Obtaining an initial quality ranking of the first number of the to-be-determined face images by using a feature comparison result of the to-be-determined face images and the corresponding quality determination images; According to the initial quality sorting, two to-be-determined face images whose sorting position difference is less than a second threshold are selected from the first number of to-be-determined face images as a face image sample pair, to obtain a second number of face image sample pairs; Obtaining a third number of quality labeling results corresponding to each of the second number of face image sample pairs; Using the third number of quality annotation results corresponding to the second number of facial image samples, respectively, to obtain, in a matrix filling manner, sample quality scores corresponding to the first number of undetermined facial images in the second number group; The sample quality scores corresponding to the first number of to-be-determined face images in the second number of groups respectively form a quality score matrix having a second number of rows and a first number of columns; Performing rank reduction processing on the quality score matrix using a preset loss function to obtain a target quality score matrix with a fourth number of rows and a first number of columns, wherein the fourth number is less than the third number; Perform mean normalization processing on the sample quality scores in each column of the target quality score matrix to obtain the target quality scores of the first number of pending face images.

2. The method according to claim 1, characterized in that: The using a preset loss function to perform rank reduction processing on the quality score matrix to obtain a target quality score matrix with a fourth number of rows and a first number of columns includes: Obtaining a fourth number of quality labeling preferences represented by the third number of quality labeling results corresponding to each of the second number of facial image samples; The quality score matrix is ​​rank reduced by using a preset loss function so that the difference between the rank of the quality score matrix and the fourth number is less than a first threshold, thereby obtaining a target quality score matrix with a fourth number of rows and a first number of columns.

3. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Using the undetermined face image with the target quality score as training data, performing model training on the training data according to the deep neural network, and obtaining a face image quality prediction module; Obtain the face image to be tested; The face image to be tested is input into the face image quality prediction module to obtain the predicted quality score of the face image to be tested.

4. The method according to claim 3, characterized in that The step of obtaining the initial quality ranking of the first number of the to-be-determined face images by using the feature comparison result of the to-be-determined face image and the corresponding quality determination image comprises: Determining the similarity between the pending face image and the corresponding quality determination image; Determining an initial quality score of a corresponding pending face image based on the similarity; The first number of pending face images are sorted according to the initial quality scores to obtain an initial quality sorting.

5. A facial image quality determination device, characterized in that: The device comprises: A pending facial image acquisition module is used to acquire a first number of pending facial images and quality determination images corresponding to the first number of pending facial images, wherein the quality determination images refer to high-quality facial images belonging to the same user as the corresponding pending facial images; A feature comparison module, configured to obtain an initial quality ranking of the first number of the to-be-determined face images by using a feature comparison result of the to-be-determined face images and the corresponding quality determination images; a sample pair screening module, configured to screen, according to the initial quality sorting, two to-be-determined face images whose sorting position difference is less than a second threshold from the first number of to-be-determined face images as a face image sample pair, to obtain a second number of face image sample pairs; A quality labeling result acquisition module, used to acquire a third number of quality labeling results corresponding to each of the second number of face image sample pairs; A correction subunit is used to obtain the sample quality scores corresponding to the first number of undetermined face images in the second number group by using the third number of quality annotation results corresponding to the second number of face image samples respectively according to the matrix filling method; A matrix generating subunit, configured to form a quality score matrix having a second number of rows and a first number of columns, using the sample quality scores corresponding to each of the first number of undetermined facial images in the second number of groups; a rank reduction processing unit, configured to perform rank reduction processing on the quality score matrix using a preset loss function to obtain a target quality score matrix with a fourth number of rows and a first number of columns, wherein the fourth number is less than the third number; A normalization processing unit is used to perform mean normalization processing on the sample quality scores in each column in the target quality score matrix to obtain the target quality scores of the first number of pending face images.

6. The device according to claim 5, characterized in that The rank reduction processing unit comprises: The instruction labeling preference acquisition subunit is used to acquire a fourth number of quality labeling preferences represented by the third number of quality labeling results corresponding to each of the second number of facial image samples; The rank reduction processing subunit is used to use a preset loss function to perform rank reduction processing on the quality score matrix so that the difference between the rank of the quality score matrix and the fourth number is less than a first threshold, and obtain a target quality score matrix with a fourth number of rows and a first number of columns.

7. The device according to any one of claims 5 to 6, characterized in that: The device also includes: A model training module, used to use a pending face image with a target quality score as training data, perform model training on the training data according to a deep neural network, and obtain a face image quality prediction module; A face image acquisition module for obtaining a face image to be tested, used for obtaining a face image to be tested; The quality prediction module is used to input the face image to be tested into the face image quality prediction module to obtain the predicted quality score of the face image to be tested.

8. The device according to claim 7, characterized in that The feature comparison module comprises: A similarity determination unit, used to determine the similarity between the to-be-determined face image and the corresponding quality determination image; An initial quality score determination unit, used to determine the initial quality score of the corresponding to-be-determined face image according to the similarity; The quality ranking unit is used to sort the first number of pending face images according to the size of the initial quality scores to obtain an initial quality ranking.

9. A computer device, characterized in that: The computer device comprises: Communication interface; A memory, used to store a program for implementing the method for determining the quality of a facial image according to any one of claims 1 to 4; A processor is used to load and execute the program to implement each step of the facial image quality determination method as described in any one of claims 1 to 4.

10. A storage medium, characterized in that: The storage medium stores a computer program, and the computer program is executed by a processor to implement each step of the facial image quality determination method as described in any one of claims 1 to 4.

Images