Face image quality evaluation method, device and electronic equipment

Abstract

The application provides a face image quality evaluation method and device and electronic equipment. The method comprises the following steps: obtaining a pre-trained first face recognition network, determining the number of layers of a residual network contained in the first face recognition network; performing random discarding on parameters in each layer of the residual network according to a preset random discarding probability, and obtaining at least two discarded second face recognition networks after at least two times of random discarding; processing the same face image by using each second face recognition network respectively to obtain a feature vector corresponding to each second face recognition network; calculating the distance between the feature vectors corresponding to the second face recognition networks, and evaluating the quality of the face image based on the distance and a threshold. The application improves the efficiency of face image quality evaluation, the credibility of the evaluation result is high, and the evaluation process is fast and convenient.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a face image quality evaluation method and device and electronic equipment. BACKGROUND

[0002] In actual industrial production, it is very important to evaluate the quality of a face image. Generally, the input of a face system is random, and there are various noises (such as blur, occlusion, illumination, etc.), and there are also various face attacks. If a face quality control link is not added, it will cause false recognition and false acceptance in subsequent face recognition, comparison, retrieval and other tasks.

[0003] The current face image quality evaluation method is divided into a manual method and a deep learning method. The manual method manually designs the features of a face image, such as calculating the blur and contrast of a face image, and calculating the interocular distance of a face image. This method often only considers the influence of a single factor, cannot comprehensively consider all influences, and needs manual participation, thereby reducing the efficiency of face image quality evaluation. The deep learning method is to build a neural network to regress the quality score of a face image. However, the quality score of the face image used to train the neural network is generally manually scored by a labeler according to the experience and feeling of the labeler, which reduces the credibility of the evaluation result of the neural network. SUMMARY

[0004] Therefore, the embodiments of the present application provide a face image quality evaluation method, device and electronic equipment to solve the problems that the prior art cannot comprehensively consider all influences, reduces the efficiency of face image quality evaluation, and the credibility of the evaluation result of the neural network is low.

[0005] In a first aspect, the embodiments of the present application provide a face image quality evaluation method, comprising: obtaining a pre-trained first face recognition network, and determining the number of layers of a residual network contained in the first face recognition network; performing random dropout on parameters in each layer of the residual network according to a preset random dropout probability, and obtaining at least two second face recognition networks after at least two times of random dropout; processing the same face image by using each second face recognition network respectively to obtain a feature vector corresponding to each second face recognition network; calculating the distance between the feature vectors corresponding to the second face recognition networks, and evaluating the quality of the face image based on the distance and a threshold.

[0006] In a second aspect, the embodiment of the present application provides a face image quality evaluation device, which comprises: an acquisition module configured to acquire a pre-trained first face recognition network and determine the number of layers of a residual network contained in the first face recognition network; a discarding module configured to discard parameters in each layer of the residual network according to a preset random discarding probability, and obtain at least two second face recognition networks after at least two times of random discarding; a processing module configured to process the same face image by using each second face recognition network respectively, and obtain a feature vector corresponding to each second face recognition network; and an evaluation module configured to calculate distances between the feature vectors corresponding to the second face recognition networks, and evaluate the quality of the face image based on the distances and a threshold.

[0007] In a third aspect, the embodiment of the present application provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the above method when executing the program.

[0008] The above at least one technical scheme adopted by the embodiment of the present application can achieve the following beneficial effects:

[0009] By acquiring a pre-trained first face recognition network, determining the number of layers of a residual network contained in the first face recognition network, discarding parameters in each layer of the residual network according to a preset random discarding probability, obtaining at least two second face recognition networks after at least two times of random discarding, processing the same face image by using each second face recognition network respectively, obtaining a feature vector corresponding to each second face recognition network, calculating distances between the feature vectors corresponding to the second face recognition networks, and evaluating the quality of the face image based on the distances and a threshold, the face image quality can be evaluated by using the second face recognition networks obtained after random discarding to process the same face image, and the face image quality can be evaluated based on the distances between the feature vectors of the face image, which can comprehensively consider all influences, improve the efficiency of face image quality evaluation, and has high credibility of evaluation results and fast and convenient evaluation process. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0011] Figure 1 is a flowchart of the face image quality evaluation method provided by the embodiment of the present application;

[0012] Figure 2 is a training process schematic diagram of a robust regression network provided by an embodiment of the present application;

[0013] Figure 3 is a structure schematic diagram of a face image quality evaluation device provided by an embodiment of the present application;

[0014] Figure 4 is a structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0015] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0016] In actual industrial production, it is very important to evaluate the quality of a face image. Generally, the input of a face system is random, there are various noises (such as blur, occlusion, illumination, etc.), and there are also various face attacks. If the face quality control link is not added, it will cause false recognition and false acceptance in subsequent face recognition, comparison, retrieval and other tasks.

[0017] The current face image quality evaluation method is divided into a manual method and a deep learning method. The manual method manually designs the features of a face image, such as calculating the blur and contrast of a face image, calculating the interocular distance of a face image, etc. This method often only considers the influence of a single factor, cannot comprehensively consider all influences, and needs manual participation, thereby reducing the efficiency of face image quality evaluation. The deep learning method is to build a neural network to regress the quality score of a face image, but the quality score of the face image used to train the neural network is generally scored by manual scoring, relying on the experience and feeling of the annotators, resulting in low credibility of the evaluation result of the neural network.

[0018] Therefore, to solve the problem of face image quality evaluation, an embodiment of the present application randomly discards the parameters in each residual network in a first face recognition network to obtain a plurality of discarded second face recognition networks, processes the same face image by using each second face recognition network to obtain a feature vector corresponding to the face image, and finally calculates the distance between each pair of feature vectors, and according to the distance and a preset threshold, the quality of the face image is evaluated. The content of the technical solution of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0019] Figure 1is a flowchart of a face image quality evaluation method provided by the embodiment of the present application. Figure 1 The face image quality evaluation method can be executed by a server. As shown in Figure 1 The face image quality evaluation method can specifically include:

[0020] S101, obtaining a pre-trained first face recognition network, and determining the number of layers of residual networks contained in the first face recognition network;

[0021] S102, performing random dropout on parameters in each layer of residual networks according to a preset random dropout probability, and obtaining at least two second face recognition networks after at least two times of random dropout;

[0022] S103, processing the same face image by using each second face recognition network respectively, and obtaining a feature vector corresponding to each second face recognition network;

[0023] S104, calculating the distance between the feature vectors corresponding to the second face recognition networks, and evaluating the quality of the face image based on the distance and a threshold.

[0024] Specifically, the first face recognition network of the embodiment of the present application can adopt any conventional face recognition neural network, and the structure of the face recognition neural network contains multiple layers of residual networks, for example, the structure of the face recognition neural network can be designed as a classic 50-layer residual network (IR50), the input of the face recognition neural network is a cropped face image, for example, the original face image is cropped into a face picture with a width and height of (112, 112), and the output of the face recognition neural network can be a 512-dimensional feature vector.

[0025] Further, the random dropout of the embodiment of the present application refers to performing random dropout on the parameters contained in each layer of residual networks in the face recognition neural network (here, the first face recognition network) according to a preset random dropout probability. It should be noted that the random dropout probability corresponding to each layer of residual networks can be different. When all the residual networks perform one time of random dropout, a face recognition neural network after random dropout (i.e., a second face recognition network) is obtained. By repeating the operation several times, several second face recognition networks can be obtained.

[0026] In some embodiments, the random dropout is performed on the parameters in each residual network according to a preset random dropout probability, including: setting a random dropout probability interval for each residual network in the first face recognition network, randomly selecting a random dropout probability from the random dropout probability interval as the probability of performing random dropout on the parameters in the residual network; and performing random dropout on the parameters in each residual network according to the random dropout probability of each residual network to obtain a second face recognition network after the dropout.

[0027] Specifically, in an optional example, the first face recognition network is denoted as N1, and the structure of the network is a classical 50-layer residual network (IR50). That is, the first face recognition network N1 includes 50 layers of residual networks, and a random dropout probability interval is set for each residual network. When performing the random dropout operation, a probability value is randomly selected from the random dropout probability interval as the probability of random dropout according to the random dropout probability interval corresponding to each residual network, and the parameters of each residual network are sequentially subjected to random dropout, and the parameters subjected to random dropout are set to 0. In this way, a new second face recognition network can be obtained.

[0028] For example, 0-0.5 is equally divided into 50 parts as the lower limit of the random dropout of each layer, and 0.1-0.6 is equally divided into 50 parts as the upper limit of the random dropout of each layer. That is, for the first layer of the network, the random dropout probability is randomly selected from (0, 0.1), for the second layer of the network, the random dropout probability is randomly selected from (0.01, 0.11), and so on. For the 49th layer of the network, the random dropout probability is randomly selected from (0.49, 0.59), and for the 50th layer of the network, the random dropout probability is randomly selected from (0.50, 0.60).

[0029] The reason for using the above equal division setting is that the shallower the face recognition network, the more detailed and miscellaneous the information, and if the dropout probability is large, the influence on the quality of the face image will be large. The deeper the face recognition network, the richer and more concise the semantics, and even if a small amount of features are retained, the discrimination can be strong, so the random dropout probability can be set to be larger.

[0030] In a specific example, according to the above random dropout manner, each layer of the residual network from 1 to 50 is provided with a random dropout probability, and the probability increases with the depth of the layer. By randomly dropping the first face recognition network N1 for 100 times, 100 variant networks (i.e., second face recognition networks) are obtained. Then, the same face image is input into the 100 variant networks to obtain 100 feature vectors: v1, v2,..., v100. 100 .

[0031] It should be noted that the first face recognition network N1 is randomly discarded at least twice, for example, the first face recognition network N1 is randomly discarded once to obtain the second face recognition network N2, and the first face recognition network N1 is randomly discarded again to obtain another second face recognition network N3. It should be understood that, in order to reduce the complexity of calculation, the embodiments of the present application can only be randomly discarded twice to obtain two different second face recognition networks. Based on the two feature vectors obtained by processing the same face image by using the two second face recognition networks, the distance between the two feature vectors is calculated, and the distance is used to evaluate the quality of the face image. However, it should be understood that the embodiments of the present application are not limited to two random discards, and more than two random discards are still valid, for example, the first face recognition network N1 is randomly discarded 100 times, so that 100 different second face recognition networks are obtained, and each second face recognition network can output a feature vector. At this time, the distance between the two feature vectors is calculated to evaluate the quality of the face image.

[0032] In actual application, if a face image is a high-quality image in the first face recognition network N1, it is considered that the face image is robust, so the first face recognition network N1 is randomly discarded once to obtain the second face recognition network N2, and the first face recognition network N1 is randomly discarded again to obtain the second face recognition network N3. The two feature vectors obtained by processing the same face image by using the second face recognition network N2 and the second face recognition network N3 should be the same, that is, the distance between the two feature vectors should be close to 0.

[0033] In some embodiments, the same face image is processed by using each second face recognition network respectively to obtain a feature vector corresponding to each second face recognition network, including: cropping the face image according to a preset width and height, inputting the cropped face image into each second face recognition network respectively, and outputting a feature vector corresponding to the cropped face image by using each second face recognition network respectively.

[0034] Specifically, in an optional example, the face image is cropped into a face picture with a width and a height of (112, 112), the cropped face image is denoted as img, and the feature vectors output by the face recognition network N2 and the face recognition network N3 are denoted as v1 and v2. The following formulas exist:

[0035] N2 = dropout(N1)

[0036] v1 = N2(img)

[0037] N3 = dropout(N1)

[0038] v2 = N3(img)

[0039] wherein N2 is a face recognition network obtained by randomly discarding N1, and N3 is a face recognition network obtained by randomly discarding N1 again, v1 is a feature vector output by the face recognition network N2 after inputting the cropped face image into the face recognition network N2, and v2 is a feature vector output by the face recognition network N3 after inputting the cropped face image into the face recognition network N3.

[0040] It should be noted that the embodiments of the present application are described by taking two random discards as an example. The face recognition network N1 obtains the face recognition network N2 and the face recognition network N3 after two random discards. Therefore, each face recognition network after random discard outputs a feature vector. However, when the number of random discards is larger, for example, the face recognition network N1 is randomly discarded 100 times, the number of the second face recognition networks obtained is 100, and the second face recognition networks output 100 feature vectors in total.

[0041] In some embodiments, the distance between the feature vectors corresponding to the second face recognition networks is calculated, and the quality of the face image is evaluated based on the distance and a threshold, including: calculating the inner product between the feature vectors output by the second face recognition networks, taking the inner product as the distance between the feature vectors, and calculating the mean and standard deviation corresponding to all distances; based on the mean and the standard deviation, calculating the network robustness value of the face image, comparing the network robustness value with a robustness threshold, and evaluating the quality of the face image according to the comparison result.

[0042] Specifically, after obtaining the feature vectors output by each second face recognition network, the distance between each two feature vectors is calculated using the inner product. The distance calculation formula of the feature vectors is as follows:

[0043] dist=dot(norm(v1),norm(v2))

[0044] wherein dist represents the distance between the feature vectors, d0t represents the inner product, and norm represents normalization of the feature vectors. If the quality of a face image is higher, the dist is smaller, and vice versa. If the dist is larger, it means that the quality of the face image is lower, that is, the robustness between the face image and the first face recognition network is poor.

[0045] In a specific example, it is assumed that there are 100 feature vectors, and the distance between each two feature vectors is calculated using the inner product, obtaining 100*99=9900 distance values. These distance values form a distribution, and the mean and standard deviation corresponding to the distribution are calculated in the following manner.

[0046] The mean of the "image-network robustness" is obtained by adding and averaging 100 feature vectors through the following formula:

[0047]

[0048] The standard deviation of the "image-network robustness" is obtained through the following formula:

[0049]

[0050] Finally, the formula for calculating the network robust value of the face image is:

[0051] n rob = 1-0.5*n_mean+0.5*n_std

[0052] Wherein, n rob is greater, that is, the network robust value of the face image is higher, the mean and the standard deviation are lower, which indicates that the network robustness of the face image is better, and it indicates that the quality of the face image is higher. The network robust value of the face image is used to measure the robustness of the face image in the face recognition network in the embodiment of the application. Not only the efficiency of the face image quality evaluation is improved, but also the credibility of the evaluation result is high, and the evaluation process is fast and convenient.

[0053] In some other embodiments, the method further comprises: taking the mean, the standard deviation and the network robust value corresponding to each face image as the label of the face image, training a pre-configured regression network using a training set composed of the face image and the label to obtain a regression network after training fitting; wherein the structure of the regression network comprises four stages, and each stage is used to output a feature map. A robust value regression module is arranged behind the last stage.

[0054] Specifically, in the foregoing embodiment, a face image needs to be calculated for 100 times (calculated by 100 second face recognition networks) to finally calculate the network robust value, which results in too long calculation time. Therefore, in the embodiment of the application, the mean, the standard deviation and the network robust value of the face image calculated above are taken as the label, a training set composed of the face image and the label is used to train a regression network, so that the regression network after training fitting can obtain the network robust value of the face image through one calculation, thereby reducing the calculation time and accelerating the calculation speed.

[0055] Further, in a specific example, assuming there is a batch of 500,000 face training data sets composed of face images. First, the technical solutions provided by the foregoing embodiments are used to calculate the mean and standard deviation of the network robustness of the 500,000 face images, for example: each face image is calculated to obtain a feature vector after random dropout of the face recognition network, and then 100 feature vectors can be obtained after 100 random dropouts of the face recognition network. Calculate the inner product of the 100 feature vectors to obtain the mean, standard deviation and network robustness value, and use the mean, standard deviation and network robustness value as the label of the face image.

[0056] Further, in a specific example, a regression network N r The backbone architecture uses a classic IR18 structure, which includes four stages (stages) and an 18-layer network. The input face image of the regression network is (112, 112) in width and height, and the four stages output four feature maps F1, F2, F3 and F4, respectively, with dimensions of (56, 56, 64), (28, 28, 128), (14, 14, 256) and (7, 7, 512).

[0057] Among them, after the above-mentioned fourth feature map F4, a robust value regression module R is added to regress the robust value. The network variance contains three 3x3 kernels, and each kernel is followed by a batch normalization (BN) and an activation function (ReLU). Followed by a global average pooling, followed by a matrix calculation with a dimension of (512, 256) and a dimension of (256, 128), and then a sigmoid function (activation function). 128 values can be obtained, and from the 128 values, the mean and standard deviation can be calculated, and then the network robustness value can be obtained.

[0058] It should be noted that when training the regression network using the face training data set, the following two network loss functions are used, and the two network loss functions specifically include the following contents:

[0059] First, the N1 recognition network needs to be used to constrain the N r The output feature maps of the first and second stages of the regression network. That is, the output feature maps F r The output feature maps F1 and F2 of the first and second stages of the regression network need to be consistent with the output feature maps F 1-1 , F 1-2 , which constitutes the first network loss function, which is the average of the absolute value of the difference between the two network feature maps. The formula is:

[0060] loss1=mean(|F1-F 1-1|)+mean(|F2-F 1-2 |)

[0061] Secondly, the mean value, the standard deviation and the robust value output by the regression module R of the regression network need to do a smooth_L1 loss (i.e. the second loss function) with the label of the picture. The formula is:

[0062] loss2=smoothl1(mean-mean′)+smoothl1(std-std′)+smoothl1(rob-rob′)

[0063] Wherein, the above formula with apostrophe is the output of the regression module, and the one without apostrophe is the label.

[0064] After the training and fitting through the above training process, the trained and fitted regression network can be obtained. When inferring a face picture, it is not necessary to perform 100 times of calculation through the 100 face recognition networks, but only one calculation through the trained and fitted regression network can obtain the network robust value.

[0065] In some embodiments, after obtaining the trained and fitted regression network, the method further comprises: inputting a face image to be evaluated into the trained and fitted regression network, outputting the network robust value corresponding to the face image to be evaluated by the trained and fitted regression network, comparing the network robust value with a robust threshold, and evaluating the quality of the face image to be evaluated according to the comparison result.

[0066] Specifically, the face image whose quality needs to be evaluated is input into the trained and fitted regression network (i.e. the robust value regression network), and the network robust value corresponding to the face image can be obtained through the calculation of the robust value regression network. The network robust value of the face image is compared with the robust threshold, so as to judge the quality of the face image.

[0067] Further, in the foregoing embodiments, two ways of calculating the robust value are respectively proposed, i.e. the first way is the robust value calculation based on the face recognition network, and the second way is to directly regress the robust value by using the robust regression network. In the following embodiments, the regression network itself is measured for robustness by the embodiments of the present application. Figure 2 is a training flow diagram of the robust regression network provided by the embodiments of the present application. As shown in Figure 2 , the training process of the robust regression network can specifically include:

[0068] S201, randomly discarding the parameters of each layer of the residual network in the regression network, and obtaining a plurality of discarded regression networks after a plurality of discarding;

[0069] S202, input the same face image into multiple discarded regression networks to obtain multiple network robust values corresponding to the multiple discarded regression networks;

[0070] S203, calculate a discarded robust mean value corresponding to the multiple network robust values, and calculate a quality robust value based on the discarded robust mean value and the original robust value, wherein the quality robust value is used as a label of the face image;

[0071] S204, train a pre-configured robust regression network by using a training set composed of the face image and the label to obtain a trained robust regression network;

[0072] The robust regression network is a neural network obtained by adding a quality regression module behind a last stage of the regression network, and during the training of the robust regression network, other parts except the quality regression module are frozen, and only the quality regression module is adjusted in parameters.

[0073] In a specific example, by performing 100 times of random dropout on the regression network, the robust values obtained by the same face image after 100 times of regression network with different dropouts are different. Because of information dropout, the 100 times of robust values are lower than the original robust value. If the decrease is not much, it indicates that the overall quality is high, and if the decrease is much, it indicates that the overall quality is poor.

[0074] After 100 times of network regression, 100 robust values are obtained, the robust values are averaged to obtain a discarded robust mean value rob2, the original robust value rob1 is subtracted from rob2, and then 1 is subtracted to obtain a quality robust value rob3:

[0075] rob3=1-(rob1-rob2)

[0076] The quality robust value is calculated for the face image in the training set to obtain a label of the quality robust value of each face image.

[0077] Further, the quality robust value is taken as the label of the face picture, and a pre-configured robust regression network is trained by using the training set composed of the face image and the label. Compared with the regression network in the foregoing embodiments, the robust regression network is obtained by adding a quality regression module QR after the fourth feature map F4 of the regression network. The structure of the quality regression module QR is consistent with that of the robust value regression module R, and finally a (128, 1) matrix is added, and a sigmoid function (activation function) is added, so that the robust regression network can regress the quality robust value. During the training of the robust regression network, other parts in the robust regression network are frozen, and only the QR part is not frozen, and the parameter adjustment can be performed on the QR part. The loss function used during the training is a smooth_L1 loss (i.e., the second loss function described above).

[0078] In some embodiments, after obtaining the trained robust regression network, the method further includes: inputting the face image to be evaluated into the trained robust regression network, and outputting the network robust value and the quality robust value corresponding to the face image to be evaluated by using the trained robust regression network; calculating a quality score based on the network robust value and the quality robust value, comparing the quality score with a quality score threshold, and evaluating the quality of the face image to be evaluated according to the comparison result.

[0079] Specifically, after the robust value regression network is trained, the main part of the robust value regression network is composed of the IR18 and has two regression modules R and QR. The R outputs 128 values, and the network robust value q1 of the face image can be further calculated. The QR outputs one value, that is, the quality robust value q2 of the face image. Finally, the quality score q of the face image can be calculated by the following formula:

[0080] q = q1*(q2*0.2+0.8)

[0081] The calculated quality score q of the face image is compared with the preset quality score threshold, so as to determine whether the quality of the face image is qualified.

[0082] It should be noted that in the foregoing embodiments, the random discarding of the face recognition network or the regression network can also be changed to random pixel erasing of the face image, or some or several layers of residual networks can be directly discarded. The specific discarding manner can be determined according to actual requirements, and the foregoing random discarding manner of the face recognition network or the regression network does not constitute a limitation on the technical solutions of the present application.

[0083] The following is a device embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0084] Figure 3 is a structural schematic diagram of a face image quality evaluation device provided by an embodiment of the present application. As shown in the figure, the face image quality evaluation device comprises: Figure 3

[0085] The obtaining module 301 is configured to obtain a pre-trained first face recognition network, and determine the number of layers of a residual network contained in the first face recognition network.

[0086] The discarding module 302 is configured to discard parameters in each layer of the residual network according to a preset random discarding probability, and obtain at least two discarded second face recognition networks after at least two times of random discarding.

[0087] The processing module 303 is configured to process the same face image by using each second face recognition network respectively, and obtain a feature vector corresponding to each second face recognition network.

[0088] The evaluation module 304 is configured to calculate distances between the feature vectors corresponding to the second face recognition networks, and evaluate the quality of the face image based on the distances and a threshold value.

[0089] In some embodiments, Figure 3 The discarding module 302 of the face image quality evaluation device sets a random discarding probability interval for each layer of the residual network in the first face recognition network, randomly selects a random discarding probability from the random discarding probability interval as the probability of random discarding the parameters of the residual network, and discards the parameters in each layer of the residual network according to the random discarding probability of each layer of the residual network to obtain the discarded second face recognition network.

[0090] In some embodiments, Figure 3 The processing module 303 of the face image quality evaluation device crops the face image according to a preset width and height, inputs the cropped face image into each second face recognition network respectively, and outputs the feature vector corresponding to the cropped face image by using each second face recognition network.

[0091] In some embodiments, Figure 3 The evaluation module 304 of the face image quality evaluation device calculates the inner product between the feature vectors output by the second face recognition networks, takes the inner product as the distance between the feature vectors, calculates the mean and standard deviation corresponding to all distances, calculates the network robust value of the face image based on the mean and standard deviation, compares the network robust value with a robust threshold value, and evaluates the quality of the face image according to the comparison result.

[0092] In some embodiments, Figure 3 ​The training module 305 of the quality evaluation method corresponds the mean value, the standard deviation and the network robust value of each face image to the label of the face image, trains the pre-configured regression network by using the training set composed of the face image and the label, and obtains the regression network after training fitting; wherein the structure of the regression network comprises four stages, and each stage is used for outputting a feature map, and a robust value regression module is arranged behind the last stage.

[0093] In some embodiments, Figure 3 The evaluation module 304 of the quality evaluation method inputs the face image to be evaluated into the regression network after training fitting after obtaining the regression network after training fitting, outputs the network robust value corresponding to the face image to be evaluated by using the regression network after training fitting, compares the network robust value with the robust threshold, and evaluates the quality of the face image to be evaluated according to the comparison result.

[0094] In some embodiments, Figure 3 The training module 305 of the quality evaluation method randomly discards the parameters of each layer of the residual network in the regression network, obtains a plurality of discarded regression networks after a plurality of times of discarding, inputs the same face image into the plurality of discarded regression networks, obtains a plurality of network robust values corresponding to the plurality of discarded regression networks, calculates the discarded robust mean value corresponding to the plurality of network robust values, calculates the quality robust value based on the discarded robust mean value and the original robust value, takes the quality robust value as the label of the face image, trains the pre-configured robust regression network by using the training set composed of the face image and the label, and obtains the robust regression network after training fitting; wherein the robust regression network is a neural network obtained by adding a quality regression module behind the last stage of the regression network, and when the robust regression network is trained, the other parts except the quality regression module are frozen, and only the parameters of the quality regression module are adjusted.

[0095] In some embodiments, Figure 3 The evaluation module 304 of the quality evaluation method inputs the face image to be evaluated into the robust regression network after training fitting after obtaining the robust regression network after training fitting, outputs the network robust value and the quality robust value corresponding to the face image to be evaluated by using the robust regression network after training fitting, calculates the quality score based on the network robust value and the quality robust value, compares the quality score with the quality score threshold, and evaluates the quality of the face image to be evaluated according to the comparison result.

[0096] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0097] Figure 4FIG. 4 is a structural schematic diagram of an electronic device 4 provided by an embodiment of the present application. As shown in FIG. 4, the electronic device 4 of this embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and capable of running on the processor 401. The processor 401 implements the steps in each of the method embodiments described above when executing the computer program 403. Alternatively, the processor 401 implements the functions of each module / unit in each of the device embodiments described above when executing the computer program 403. Figure 4

[0098] By way of example, the computer program 403 can be divided into one or more modules / units, which are stored in the memory 402 and executed by the processor 401 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 403 in the electronic device 4.

[0099] The electronic device 4 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The electronic device 4 can include but is not limited to the processor 401 and the memory 402. Those skilled in the art can understand that the electronic device 4 can include more or fewer components, or combine certain components, or include different components, for example, the electronic device can also include an input / output device, a network access device, a bus, and the like. Figure 4 The electronic device 4 shown in FIG. 4 is merely an example and does not constitute a limitation on the electronic device 4, which can include more or fewer components than those shown, or combine certain components, or include different components, for example, the electronic device can also include an input / output device, a network access device, a bus, and the like.

[0100] The processor 401 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0101] ​The memory 402 can be an internal storage unit of the electronic device 4, for example, a hard disk or a memory of the electronic device 4. The memory 402 can also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. Further, the memory 402 can include both the internal storage unit and the external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device. The memory 402 can also be used to temporarily store data that has been output or will be output.

[0102] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software function unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0103] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0104] Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0105] In the embodiments of the present application, it should be understood that the disclosed apparatus / computer device and method can be implemented in other manners. For example, the described apparatus / computer device embodiments are merely schematic. For example, the division of the modules or units is merely logical function division. There can be another division manner for the actual implementation, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0106] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0107] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically as separate units, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware, or in the form of software functional units.

[0108] If the integrated module / unit is implemented in the form of software functional units and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the above-mentioned embodiment methods can be completed by computer programs, and the computer programs can be stored in a computer readable storage medium. When the processor executes the computer programs, the steps of the above-mentioned various method embodiments can be implemented. The computer program can include computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier wave signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to the legislation and patent practice, the computer readable medium does not include electric carrier wave signal and telecommunication signal.

[0109] The above examples are only used to illustrate the technical solutions of the present application, but not limit the same; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for assessing the quality of facial images, characterized in that, include: Obtain the pre-trained first face recognition network and determine the number of layers in the residual network contained in the first face recognition network; According to a preset random discard probability, the parameters in each layer of the residual network are randomly discarded. After at least two random discards, at least two discarded second face recognition networks are obtained. The same face image is processed using each of the second face recognition networks to obtain the feature vector corresponding to each of the second face recognition networks; Calculate the distance between the feature vectors corresponding to the second face recognition network, and evaluate the quality of the face image based on the distance and a threshold; The method further includes: The mean, standard deviation, and network robustness value corresponding to each face image are used as the label of the face image. The pre-configured regression network is trained using the training set composed of the face images and the labels to obtain the trained and fitted regression network. The structure of the regression network includes four stages, each stage is used to output a feature map, and a robust value regression module is set after the last stage. After obtaining the trained and fitted regression network, the method further includes: The face image to be evaluated is input into the trained and fitted regression network. The trained and fitted regression network outputs the network robustness value corresponding to the face image to be evaluated. The network robustness value is compared with the robustness threshold. The quality of the face image to be evaluated is evaluated based on the comparison result. The method further includes: The parameters of each residual network layer in the regression network are randomly discarded. After multiple discarding processes, multiple discarded regression networks are obtained. Using the same face image as input to the multiple discarded regression networks, we obtain multiple network robustness values ​​corresponding to the multiple discarded regression networks; Calculate the dropout robust mean corresponding to the multiple network robust values, and calculate the quality robust value based on the dropout robust mean and the original robust value. Use the quality robust value as the label of the face image. Using the training set composed of the face images and labels, a pre-configured robust regression network is trained to obtain the trained and fitted robust regression network. The robust regression network is a neural network obtained by adding a quality regression module after the last stage of the regression network. When training the robust regression network, all parts except the quality regression module are frozen, and only the parameters of the quality regression module are adjusted.

2. The method according to claim 1, characterized in that, The step of randomly discarding parameters in each layer of the residual network according to a preset random discard probability includes: A random drop probability interval is set for each layer of the residual network in the first face recognition network, and a random drop probability is randomly selected from the random drop probability interval as the probability of randomly dropping the parameters of the residual network. Based on the random discard probability of the residual network in each layer, the parameters in the residual network in each layer are randomly discarded to obtain the second face recognition network after discarding.

3. The method according to claim 1, characterized in that, The step of processing the same face image using each of the second face recognition networks to obtain the feature vector corresponding to each of the second face recognition networks includes: The face image is cropped according to a preset width and height, and the cropped face image is input into each of the second face recognition networks. Each of the second face recognition networks outputs the feature vector corresponding to the cropped face image.

4. The method according to claim 3, characterized in that, The step of calculating the distance between the feature vectors corresponding to the second face recognition network, and evaluating the quality of the face image based on the distance and a threshold, includes: Calculate the inner product between the feature vectors output by the second face recognition network, use the inner product as the distance between the feature vectors, and calculate the mean and standard deviation of all distances. Based on the mean and standard deviation, the network robustness value of the face image is calculated, the network robustness value is compared with a robustness threshold, and the quality of the face image is evaluated based on the comparison result.

5. The method according to claim 1, characterized in that, After obtaining the trained and fitted robust regression network, the method further includes: The face image to be evaluated is input into the trained and fitted robust regression network, and the trained and fitted robust regression network is used to output the network robustness value and quality robustness value corresponding to the face image to be evaluated. A quality score is calculated based on the network robustness value and the quality robustness value. The quality score is then compared with a quality score threshold, and the quality of the face image to be evaluated is assessed based on the comparison result.

6. A facial image quality assessment device, characterized in that, include: The acquisition module is configured to acquire the pre-trained first face recognition network and determine the number of layers of the residual network contained in the first face recognition network. The discarding module is configured to randomly discard parameters in each layer of the residual network according to a preset random discarding probability, and after at least two random discards, at least two discarded second face recognition networks are obtained. The processing module is configured to process the same face image using each of the second face recognition networks respectively, and obtain the feature vector corresponding to each of the second face recognition networks; The evaluation module is configured to calculate the distance between the feature vectors corresponding to the second face recognition network, and evaluate the quality of the face image based on the distance and a threshold. The training module uses the mean, standard deviation, and network robustness value corresponding to each face image as the label of the face image. Using the training set composed of the face images and the labels, the pre-configured regression network is trained to obtain the trained and fitted regression network. The regression network structure includes four stages, each stage is used to output a feature map, and a robustness value regression module is set after the last stage. After obtaining the trained and fitted regression network, the evaluation module inputs the face image to be evaluated into the trained and fitted regression network, uses the trained and fitted regression network to output the network robustness value corresponding to the face image to be evaluated, compares the network robustness value with the robustness threshold, and evaluates the quality of the face image to be evaluated based on the comparison result. The training module randomly discards parameters from each residual network layer in the regression network. After multiple discardings, multiple discarded regression networks are obtained. The same face image is used as input to the multiple discarded regression networks to obtain multiple network robustness values ​​corresponding to the multiple discarded regression networks. The discard robustness mean corresponding to the multiple network robustness values ​​is calculated. Based on the discard robustness mean and the original robustness value, a quality robustness value is calculated and used as the label of the face image. The pre-configured robust regression network is trained using the training set composed of the face image and the label to obtain a trained and fitted robust regression network. The robust regression network is a neural network obtained by adding a quality regression module after the last stage of the regression network. When training the robust regression network, all parts except the quality regression module are frozen, and only the parameters of the quality regression module are adjusted.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1 to 5.

Images