Face quality judgment method based on local contribution variance

Abstract

The present invention discloses a method for judging face quality based on local contribution variance, comprising the steps of: performing a mask operation on the aligned face, extracting face features from the face image after the mask operation, calculating the similarity between the face features after the mask operation and the features of the original face image, and performing statistical changes on a series of similarities to obtain the quality of the face image. The method for judging face quality based on local contribution variance proposed by the present invention can quickly judge whether the face image is suitable for face recognition, and can effectively eliminate blurry, occluded and other poor quality photos captured in the face recognition system actually deployed, thereby improving the performance of the face recognition system and reducing the amount of calculation of the face recognition system. The present invention has been tested in a large number of actual scenarios, and it is shown that compared with the current existing technology, the method has the advantages of being both accurate and less time-consuming, and is very beneficial for deployment in embedded terminals with limited resources.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for judging face quality based on local contribution variance. Background Art

[0002] As a popular research direction in computer vision, face recognition technology has achieved many impressive results under the vigorous development of neural networks and is widely used in our daily lives. However, in addition to active face recognition systems, there are also a large number of passive face recognition systems, and the face data collected by them is heavily polluted, which is a severe challenge to the current face recognition model. Therefore, it is very important to develop face quality judgment and select face images that are not suitable for face recognition. Based on this, the present invention proposes a face quality judgment method based on local contribution variance.

[0003] In the past, the early methods of face quality judgment required the quality score of the face dataset to be annotated, which not only consumed a lot of manpower, but also the subjective identification of humans may not be consistent with the degree of influence of the actual running recognizer. Therefore, this method has many disadvantages. Among the methods developed recently, some methods use the uncertainty of the model as a measure of face image quality judgment, which lacks instability; some methods use the similarity distribution between the same category and the similarity distribution between different categories to measure face quality. The quality score obtained by this method is highly correlated with the selected dataset and is not universal. Summary of the invention

[0004] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is to provide a face quality judgment method based on local contribution variance, which gets rid of the dependence on manual labeling and can be easily deployed in any face recognition system to achieve the purpose of improving the performance of the face recognition system.

[0005] To achieve the above object, the present invention provides a method for judging face quality based on local contribution variance, comprising the following steps:

[0006] Step 1: Face detection and face key point detection;

[0007] Step 2: Face alignment;

[0008] Step 3: Perform mask operation on the aligned faces to obtain a series of masked images;

[0009] Step 4: Mapping the masked image into facial features through a face recognition network;

[0010] Step 5: Obtain a series of local contributions by calculating the feature similarity between the original image and the mask image;

[0011] Step 6: Calculate the variance of local contribution to obtain the face image quality;

[0012] Step 7: Use a lightweight neural network for fitting to obtain a face quality score predictor.

[0013] Furthermore, the masking operation is performed on the aligned faces to obtain a series of masked images, specifically: the masking operation is performed on the aligned face images, from top to bottom, every k rows are performed in sequence, the resolution of the aligned face images is usually 112*112, and (112 / k)+1 images are obtained

[0014] Furthermore, the masked image is mapped to a face feature through a face recognition network, specifically: the aligned face image is sent to the face recognition network to obtain the face feature F of the original face image 0 , the obtained (112 / k)+1 face pictures are sent to the face recognition network to obtain (112 / k)+1 face features F i , where i∈[1,(112 / K)+1].

[0015] Furthermore, the variance of the local contribution is calculated to obtain the face image quality, specifically: the obtained F 0 and F i The cosine similarities are calculated in sequence to obtain (112 / k)+1 similarities, and the variance of these similarities is calculated to obtain the face quality score.

[0016] Furthermore, the lightweight neural network uses MobileFaceNet as the backbone network.

[0017] The beneficial effects of the present invention are:

[0018] The face quality judgment method based on local contribution variance proposed in the present invention is an unsupervised method, which saves a lot of loss of manually labeled data compared with previous methods. At the same time, the new face quality score calculation method proposed has good versatility, can be applied to most of the current face recognition neural networks, and can be simply and efficiently deployed in the existing face recognition system to achieve the purpose of quickly improving the performance of the recognition system. At the same time, the present invention can be deployed on devices with scarce computing resources such as embedded devices. Through the method proposed in the invention, the quality score pseudo-label of the face data set can be calculated, and then a lightweight network is used for fitting. After that, the lightweight network can be used to predict the face quality score, further achieving the purpose of rapid deployment and improving the performance of the face recognition system.

[0019] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the face quality judgment method based on local contribution variance of the present invention.

[0021] Figure 2 This is a model diagram of the face quality judgment method based on local contribution variance of the present invention. DETAILED DESCRIPTION

[0022] like Figure 1 As shown, the present invention provides a method for judging face quality based on local contribution variance, comprising the following steps:

[0023] Step 1: Face detection and face key point detection;

[0024] Step 2: Face alignment;

[0025] Step 3: Perform mask operation on the aligned faces to obtain a series of masked images;

[0026] Step 4: Mapping the masked image into facial features through a face recognition network;

[0027] Step 5: Obtain a series of local contributions by calculating the feature similarity between the original image and the mask image;

[0028] Step 6: Calculate the variance of local contribution to obtain the face image quality;

[0029] Step 7: Use a lightweight neural network for fitting to obtain a face quality score predictor.

[0030] This method calculates the variance of the local contribution of the image to determine whether the face image is suitable for face recognition. For traditional face quality judgment methods, it is necessary to manually annotate the quality score of the face data set, and then use neural networks to fit the data, which requires a lot of manpower. At the same time, manual data annotation is often subjective and cannot be used to judge whether the image is suitable for face recognition. It is impossible to determine what factors are beneficial for the neural network to extract facial features. Therefore, the annotated data is often not very accurate, resulting in the inefficiency and inaccuracy of traditional methods. In order to improve the effectiveness and efficiency of the face quality judgment method, the face quality judgment method based on local contribution variance adopts an unsupervised approach and does not rely on manually annotated data. The process of the method is as follows:

[0031] (1) Perform face detection and facial key point detection on the photo to obtain the location of the face and facial feature points in the photo.

[0032] (2) Perform face alignment on the photos based on the detection results of (1) to obtain an aligned face image.

[0033] (3) Perform masking operation on the aligned face images obtained in (2), from top to bottom, every k rows. The resolution of the aligned face images is usually 112*112, so (112 / k)+1 images are obtained.

[0034] (4) The aligned face image obtained in (2) is sent to the face recognition network to obtain the face feature F_0 of the original face image.

[0035] (5) The (112 / k)+1 face images obtained in (3) are sent to the face recognition network to obtain (112 / k)+1 face features F_i, where i∈[1,(112 / k)+1].

[0036] (6) Calculate the cosine similarity of F_0 obtained in (4) and F_i obtained in (5) in sequence to obtain (112 / k)+1 similarities. Calculate the variance of these similarities to obtain the face quality score.

[0037] This method extracts facial features by masking the aligned face image, and calculates the similarity with the facial features obtained from the aligned face image to indicate how much facial identity information is contained in the mask part. By analyzing the amount of facial information in different areas of the image, it characterizes whether the face image is suitable for face recognition. In this method, k is a hyperparameter that can be adjusted. The larger k is, the more times the neural network needs to be run, and the greater the time complexity. The smaller k is, the fewer times the neural network needs to be run, and the smaller the time complexity. At the same time, this method can be used to annotate the quality scores of large face datasets, and then use a lightweight network for fitting, which can be deployed in embedded devices and the like.

[0038] 1. Face quality score network based on lightweight CNN

[0039] The present invention uses MobileFaceNet as the backbone network, and connects the final output facial features to the fully connected layer to output the face quality score. The network uses the commonly used mean square error (MSE) for regression training, and uses the face quality score labels obtained in 1 above for fitting, so as to reduce the computational complexity and quickly deploy it to various face recognition systems including embedded systems, and quickly improve the performance and robustness of the face recognition system.

[0040] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0041] In addition, each step in the embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0042] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), disk or optical disk and other media that can store program codes.

[0043] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A face quality judgment method based on local contribution variance, characterized in that: The following steps are involved: Step 1: Face detection and face key point detection; Step 2: Face alignment; Step 3: Perform mask operation on the aligned faces to obtain a series of masked images; Step 4: Mapping the masked image into facial features through a face recognition network; Step 5: Obtain a series of local contributions by calculating the feature similarity between the original image and the mask image; Step 6: Calculate the variance of local contribution to obtain the face image quality; Step 7: Use a lightweight neural network for fitting to obtain a face quality score predictor; The masking operation is performed on the aligned faces to obtain a series of masked images, specifically: the masking operation is performed on the aligned face images, from top to bottom, every k rows are performed in sequence, the resolution of the aligned face images is usually 112*112, and (112 / k)+1 images are obtained.

2. A method for judging face quality based on local contribution variance as claimed in claim 1, characterized in that: The masked image is mapped to a face feature through a face recognition network, specifically: the aligned face image is sent to the face recognition network to obtain the face feature F0 of the original face image, and the (112 / k)+1 face image is sent to the face recognition network to obtain the (112 / k)+1 face feature F i , where i∈[1,(112 / k)+1].

3. A method for judging face quality based on local contribution variance as claimed in claim 1, characterized in that: The variance of the local contribution is calculated to obtain the face image quality, specifically: the obtained F0 and F i The cosine similarities are calculated in sequence to obtain (112 / k)+1 similarities, and the variance of these similarities is calculated to obtain the face quality score.

4. The method for judging face quality based on local contribution variance according to claim 1, characterized in that: The lightweight neural network uses MobileFaceNet as the backbone network.

Images