Face unlocking method and device and storage medium

Abstract

The invention relates to a face unlocking method and device and a storage medium. The method comprises the steps that a face image is collected; inputting the face image into a face image quality evaluation model to obtain a first image quality evaluation value of the face image; the human face image quality evaluation model is obtained based on training of a human face image quality evaluation label data set, and the human face image quality evaluation label data set comprises human face image quality evaluation values of which the evaluation accuracy is greater than an accuracy threshold; and in response to the fact that the first image quality evaluation value is greater than a quality evaluation threshold value, taking the face image as a face image for face unlocking. Through the method and the device, the face image quality evaluation model with a good face image evaluation effect can be constructed, a clear face image can be obtained in a face unlocking process, time is saved, unlocking efficiency is improved, and user experience in the face unlocking process is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of display technology, and in particular to a face unlocking method, device and storage medium. Background Art

[0002] During the face unlocking process, the image quality obtained by the face image recognition algorithm will directly affect the recognition effect. Blurred face images will make image analysis and processing difficult or even fail to unlock. When entering face images, image quality detection is required to ensure that high-quality template images are entered; in the unlocking process, image quality detection can eliminate blurry images with poor quality to save unlocking time. Therefore, the evaluation of face image quality is very meaningful.

[0003] In the related art, the quality evaluation methods based on some functions can only judge the clarity of some images to a certain extent, and cannot deal with complex and changeable images well. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a face unlocking method, device and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a face unlocking method is provided, including:

[0006] Capture a facial image; input the facial image into a facial image quality assessment model to obtain a first image quality assessment value of the facial image; the facial image quality assessment model is trained based on a facial image quality assessment label data set, and the facial image quality assessment label data set includes facial image quality assessment values ​​whose assessment accuracy of the facial image quality assessment value is greater than an accuracy threshold; in response to the first image quality assessment value being greater than the quality assessment threshold, use the facial image as a facial image for face unlocking.

[0007] In one embodiment, the label data set is obtained in the following manner: multiple facial images are collected, and facial image quality evaluation is performed on each of the multiple facial images to obtain a first facial image quality evaluation label data set, wherein the first facial image quality evaluation label data set includes facial image quality evaluation values ​​of the multiple facial images; each facial image quality evaluation value in the first facial image quality evaluation label data set is normalized to obtain a second facial image quality evaluation label data set; wherein the evaluation accuracy of the normalized facial image quality evaluation value is higher than the accuracy threshold.

[0008] In one embodiment, the facial image quality evaluation includes: performing image grayscale processing on the facial area in a target facial image, wherein the target facial image is a facial image among the multiple facial images; determining facial image quality evaluation values ​​in multiple angular directions based on pixel gradient values ​​in multiple gradient directions for the image after the image grayscale processing; and selecting the minimum value among the facial image quality evaluation values ​​in the multiple angular directions as the facial image evaluation value of the target facial image.

[0009] In one embodiment, determining the facial image quality evaluation values ​​in multiple angular directions based on pixel gradient values ​​in multiple gradient directions includes: determining the cumulative value of the pixel gradient values ​​of the target pixel point in each gradient direction of a preset number of gradient directions, and obtaining the facial image quality evaluation values ​​in the preset number of angular directions, wherein one gradient direction corresponds to one angular direction, and the target pixel point is each pixel point in the length direction and the width direction of the image after grayscale processing; wherein each gradient direction includes multiple sub-gradient directions corresponding to the target pixel point, and the multiple sub-gradient directions include at least two of the horizontal direction, the vertical direction and the diagonal direction of the target pixel block, and the target pixel block has the target pixel point as a vertex, and is spaced K pixels in the length direction and M pixels in the width direction from other vertices, and K and M are positive integers greater than or equal to 0.

[0010] In one embodiment, the facial image quality assessment model is trained based on a facial image quality assessment label data set in the following manner: a training facial image is input into a prediction network to obtain a predicted evaluation value of the facial image quality of the training facial image; among multiple evaluation value intervals, the evaluation value interval to which the predicted evaluation value belongs is determined, and the multiple evaluation value intervals are pre-divided based on a first facial image quality evaluation label value of the training facial image; based on the evaluation value interval to which the predicted evaluation value belongs, a second facial image quality evaluation label value is determined; based on the loss between the first facial image quality evaluation label value and the second facial image quality evaluation label value, the prediction network is trained to obtain a facial image quality assessment model.

[0011] In one embodiment, the prediction network includes a capsule network of soft stage regression.

[0012] In one embodiment, the training face images are determined in the following manner: a first face image set with different clarity is obtained based on multiple scene attributes; face detection is performed on the images in the first face image set to obtain face detection results, as well as face bounding boxes and face key points of each face image; based on the face detection results, the face bounding boxes and the face key points, face images that meet the conditions are screened to obtain a second face image set; the second face image set is used as training face images; wherein the face images that meet the conditions include at least one of the following: the face detection result is that a face is detected; the pixel area in the face bounding box is within a preset range; the pixel coordinates corresponding to the face key points are located in an image having a preset length direction and a width direction.

[0013] In one implementation, in response to the first image quality evaluation value being less than the quality evaluation threshold, the face image is discarded.

[0014] In one embodiment, using the facial image as the facial image for face unlocking includes: using the facial image as the facial authentication image for face unlocking authentication, and the quality evaluation threshold is a first quality evaluation threshold; and / or using the facial image as a facial reference image for face unlocking entry, and the quality evaluation threshold is a second quality evaluation threshold; the first quality evaluation threshold is higher than the second quality evaluation threshold.

[0015] According to a second aspect of an embodiment of the present disclosure, a face unlocking device is provided, including:

[0016] A collection unit, used for collecting face images;

[0017] An execution unit, configured to input the face image into a face image quality evaluation model to obtain a first image quality evaluation value of the face image; the face image quality evaluation model is trained based on a face image quality evaluation label data set, and the face image quality evaluation label data set includes face image quality evaluation values ​​whose evaluation accuracy of face image quality evaluation values ​​is greater than an accuracy threshold;

[0018] A judgment unit is used to use the facial image as a facial image for face unlocking in response to the first image quality evaluation value being greater than a quality evaluation threshold.

[0019] In one embodiment, the execution unit obtains a label data set in the following manner: collecting multiple facial images, performing facial image quality evaluation on each of the multiple facial images, and obtaining a first facial image quality evaluation label data set, wherein the first facial image quality evaluation label data set includes facial image quality evaluation values ​​of the multiple facial images; normalizing each facial image quality evaluation value in the first facial image quality evaluation label data set to obtain a second facial image quality evaluation label data set; wherein the evaluation accuracy of the normalized facial image quality evaluation value is higher than the accuracy threshold.

[0020] In one embodiment, the execution unit performs facial image quality evaluation, including: performing image grayscale processing on the facial area in a target facial image, wherein the target facial image is a facial image among the multiple facial images; determining facial image quality evaluation values ​​in multiple angular directions based on pixel gradient values ​​in multiple gradient directions for the image after the image grayscale processing; and selecting the minimum value among the facial image quality evaluation values ​​in the multiple angular directions as the facial image evaluation value of the target facial image.

[0021] In one embodiment, the execution unit determines the facial image quality evaluation values ​​in multiple angular directions based on the pixel gradient values ​​in multiple gradient directions in the following manner: for each of a preset number of gradient directions, the pixel gradient value accumulation value of the target pixel point is determined respectively to obtain the facial image quality evaluation values ​​in the preset number of angular directions, wherein one gradient direction corresponds to one angular direction, and the target pixel point is each pixel point in the length direction and the width direction of the image after grayscale processing; wherein each gradient direction includes multiple sub-gradient directions corresponding to the target pixel point, and the multiple sub-gradient directions include at least two of the horizontal direction, the vertical direction and the diagonal direction of the target pixel block, and the target pixel block has the target pixel point as a vertex, and is spaced apart from other vertices by K pixels in the length direction and by M pixels in the width direction, and K and M are positive integers greater than or equal to 0.

[0022] In one embodiment, the facial image quality assessment model is trained based on a facial image quality assessment label data set in the following manner: a training facial image is input into a prediction network to obtain a predicted evaluation value of the facial image quality of the training facial image; among multiple evaluation value intervals, the evaluation value interval to which the predicted evaluation value belongs is determined, and the multiple evaluation value intervals are pre-divided based on a first facial image quality evaluation label value of the training facial image; based on the evaluation value interval to which the predicted evaluation value belongs, a second facial image quality evaluation label value is determined; based on the loss between the first facial image quality evaluation label value and the second facial image quality evaluation label value, the prediction network is trained to obtain a facial image quality assessment model.

[0023] In one embodiment, the prediction network includes a capsule network of soft stage regression.

[0024] In one embodiment, the execution unit determines the training face images in the following manner: obtaining a first face image set with different clarity based on multi-scene attributes; performing face detection on the images in the first face image set to obtain face detection results, as well as face bounding boxes and face key points of each face image; based on the face detection results, the face bounding boxes and the face key points, screening face images that meet the conditions to obtain a second face image set; using the second face image set as training face images; wherein the face images that meet the conditions include at least one of the following: the face detection result is that a face is detected; the pixel area in the face bounding box is within a preset range; the pixel coordinates corresponding to the face key points are located in an image with preset length and width directions.

[0025] In one implementation, the judgment unit is further configured to discard the facial image in response to the first image quality evaluation value being less than the quality evaluation threshold.

[0026] In one embodiment, the judgment unit uses the facial image as the facial image for face unlocking in the following manner: using the facial image as the facial authentication image for face unlocking authentication, the quality evaluation threshold is a first quality evaluation threshold; and / or using the facial image as a facial reference image entered for face unlocking, the quality evaluation threshold is a second quality evaluation threshold; the first quality evaluation threshold is higher than the second quality evaluation threshold.

[0027] According to a third aspect of an embodiment of the present disclosure, a face unlocking device is provided, characterized in that it includes:

[0028] processor;

[0029] a memory for storing processor-executable instructions;

[0030] Among them, the processor is configured to: execute the face unlocking method described in the first aspect or any one of the embodiments of the first aspect.

[0031] According to the fourth aspect of an embodiment of the present disclosure, a storage medium is provided, characterized in that instructions are stored in the storage medium, and when the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the face unlocking method described in the first aspect or any one of the embodiments of the first aspect.

[0032] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: the face image quality evaluation model obtained by training based on the label data set is used to evaluate the image quality of the face image collected during the face unlocking process, and when it is judged that the image quality evaluation value is greater than the quality evaluation threshold of the unlocking task, the collected face image is used in the face unlocking process. Through the present disclosure, a face image quality evaluation model with good evaluation effect of face images can be constructed, and the clarity and blur of the image can be quantitatively evaluated for complex face images. It can ensure that the face entry link of the face unlocking system obtains a clear face image, save time for the unlocking link, improve the unlocking rate, and enhance the user experience during the face unlocking process.

[0033] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0035] Figure 1 The present invention is a flowchart of a face unlocking method according to an exemplary embodiment.

[0036] Figure 2 The figure is a flowchart of a method for obtaining a label data set according to an exemplary embodiment.

[0037] Figure 3 The present invention is a flowchart of a method for evaluating the quality of a facial image according to an exemplary embodiment.

[0038] Figure 4 The present invention is a flowchart of a method for obtaining a face image quality evaluation model according to an exemplary embodiment.

[0039] Figure 5 The present invention is a flowchart of a method for determining a training face image according to an exemplary embodiment.

[0040] Figure 6The present invention is a block diagram of a face unlocking device according to an exemplary embodiment.

[0041] Figure 7 The present invention is a block diagram of a face unlocking device according to an exemplary embodiment.

[0042] Figure 8 The present invention is a block diagram of a face unlocking device according to an exemplary embodiment. DETAILED DESCRIPTION

[0043] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0044] In the face recognition unlocking process, the blur of the acquired face image will lead to difficulties in image analysis and processing, or even unlocking failure. In the face recording stage, image quality detection can ensure the recording of high-quality face images, which is convenient for subsequent recognition and unlocking. In the unlocking stage, image quality detection can filter out images with poor quality due to blur, etc., which can save unlocking time. Therefore, the evaluation of face image quality is very meaningful. Among them, there are various factors that cause image blur, such as defocus blur, motion blur, light and shadow blur, and image compression blur, etc., accompanied by the interference of image noise, which makes the task of face image quality evaluation difficult. At present, there is no public data set for face quality evaluation tasks, and the use of manually annotated data sets may have different standards for face quality scoring due to the manual annotation method. Therefore, it is necessary to design an effective method for face quality evaluation. By measuring the degree of blur, the quality of face image transmission or processing can be evaluated, and measures can be taken to improve the performance of the face unlocking system.

[0045] In the related technology, the commonly used evaluation functions for image quality evaluation include gradient-based evaluation functions, frequency-domain-based evaluation functions, information entropy-based evaluation functions, and statistical evaluation functions. These evaluation methods can determine the blur and clarity of an image to a certain extent. These evaluation methods have good recognition effects on images with bright colors and multiple colors, but they are not effective for facial images and cannot cope with complex and changeable facial images. The method of building a quality evaluation model based on deep learning requires a large amount of data with correct and evenly distributed scores from blur to clarity. At present, there are very few public data sets for quality evaluation tasks, and there is no data set specifically for facial image quality evaluation, making model training difficult.

[0046] In view of this, the present disclosure provides a face unlocking method, which trains a face image quality evaluation model based on a label data set including face image quality evaluation values ​​whose evaluation accuracy is greater than an accuracy threshold, and obtains a face image quality evaluation value through the face image quality evaluation model. Through the present disclosure, a clear and high-quality face image template can be entered in the face image entry stage of the face unlocking system, which can improve the correct unlocking rate of the face unlocking system and enhance the user experience.

[0047] In the disclosed embodiment, the specific implementation process of the face unlocking method is described.

[0048] Figure 1 is a flow chart of a face unlocking method according to an exemplary embodiment. Figure 1 As shown, the following steps are included:

[0049] In step S11, a face image is collected.

[0050] In the embodiment of the present disclosure, the camera of the face unlocking system may be used to recognize the current face and capture the current face image.

[0051] In step S12, the face image is input into the face image quality evaluation model to obtain a first image quality evaluation value of the face image. The face image quality evaluation model is trained based on a face image quality evaluation label data set, and the face image quality evaluation label data set includes face image quality evaluation values ​​whose evaluation accuracy of the face image quality evaluation value is greater than an accuracy threshold.

[0052] In the disclosed embodiment, the accuracy threshold may be preset, wherein the accuracy threshold represents the accuracy of the image quality reflected by the image quality evaluation value and the actual image quality of a person, and may be an experimental value measured based on relevant experiments. Wherein, if the evaluation accuracy of the face image quality evaluation value is greater than the accuracy threshold, it means that the obtained face image quality evaluation value is consistent with the actual image quality, and the evaluation accuracy is high, and it can be used to train a face image quality evaluation model.

[0053] In step S13, in response to the first image quality evaluation value being greater than the quality evaluation threshold, the face image is used as the face image for face unlocking.

[0054] In the embodiment of the present disclosure, during the face unlocking process, a quality evaluation threshold can be set in advance, where the quality evaluation threshold indicates that the first quality evaluation value obtained meets the unlocking task requirements, that is, the first image quality evaluation value is greater than the quality evaluation threshold, indicating that the currently collected face image can be used for face unlocking.

[0055] In the disclosed embodiment, a facial image quality evaluation model is trained based on a label data set containing facial image quality evaluation values ​​whose evaluation accuracy is greater than an accuracy threshold. The obtained facial image quality evaluation model is accurate in evaluating the image quality of facial images and can obtain an accurate first image quality evaluation value.

[0056] In the disclosed embodiment, the specific implementation process of the method for obtaining a label data set is described.

[0057] Figure 2 is a flowchart of a method for obtaining a label data set according to an exemplary embodiment. Figure 2 As shown, the following steps are included:

[0058] In step S21, multiple facial images are collected, and facial image quality evaluation is performed on each of the multiple facial images to obtain a first facial image quality evaluation label data set, which includes facial image quality evaluation values ​​of the multiple facial images.

[0059] In the disclosed embodiments, when the camera captures multiple facial images, it can capture facial images of a variety of different environmental scenes, lighting conditions, and various motion patterns of different user individuals, and it can also capture a variety of blurred facial images, thereby obtaining multiple real facial images of different clarity and quality.

[0060] In step S22, each facial image quality evaluation value in the first facial image quality evaluation label data set is normalized to obtain a second facial image quality evaluation label data set.

[0061] In the disclosed embodiment, there may be facial image quality evaluation values ​​with large errors in the first facial image quality evaluation label data set. The large errors may be, for example, when the image is actually blurred, and the facial image corresponding to the image is very large, which is inconsistent with the actual blurry state of the image. In this case, there is a situation where there is a large error.

[0062] In the embodiment of the present disclosure, the first face image quality evaluation label data set is normalized, and the normalization formula is as follows:

[0063]

[0064] Among them, s 0 is the face image quality value, s max and min are the maximum and minimum quality values ​​of all face images respectively, and s is the normalized face image quality score.

[0065] In step S23, the evaluation accuracy of the normalized face image quality evaluation value is higher than the accuracy threshold.

[0066] In the disclosed embodiment, the image quality evaluation value is normalized to convert the data into a decimal in the interval [0, 1], and the data is mapped to the range of 0-1, which facilitates data processing.

[0067] In the disclosed embodiment, the value with a larger error in the image quality evaluation value is adjusted, and the data of the data set can be further adjusted to make the data set more accurate, and the image quality evaluation value is normalized and converted to the same range, which is convenient for processing the image quality evaluation value and makes the processing result more accurate. The face image quality evaluation model is trained based on an accurate label data set, and the obtained face image quality evaluation model has an accurate effect of image quality evaluation of face images and can obtain an accurate first image quality evaluation value.

[0068] In the disclosed embodiment, the specific implementation process of the face image quality evaluation method is described.

[0069] Figure 3 is a flow chart of a method for evaluating the quality of a face image according to an exemplary embodiment. Figure 3 As shown, the following steps are included:

[0070] In step S31, image grayscale processing is performed on the face area in the target face image, where the target face image is a face image among multiple face images.

[0071] In the embodiment of the present disclosure, the face region is subjected to image grayscale processing to convert the color image into a grayscale image, wherein each pixel of the grayscale image only needs one byte to store the grayscale value, and the grayscale range is H 1 -H 2 , when the gray level is H 2 When the grayscale is H 1 When , it indicates the darkest. Compared with color images, grayscale images take up less memory and run faster. Grayscale images can increase visual contrast, highlight the target area, and facilitate processing of face areas.

[0072] In step S32, for the image after image grayscale processing, based on the pixel gradient values ​​in multiple gradient directions, the face image quality evaluation values ​​in multiple angular directions are determined.

[0073] In the disclosed embodiment, for the image after image grayscale processing, multiple angular directions corresponding to gradient directions are selected, for example: angle direction of A degree and angle direction of B degree, and the facial image quality evaluation values ​​in the angle directions corresponding to the multiple gradient directions are calculated respectively to obtain multiple facial image quality evaluation values ​​corresponding to multiple different angle directions.

[0074] In step S33, the minimum value among the facial image quality evaluation values ​​in multiple angular directions is selected as the facial image evaluation value of the target facial image.

[0075] In the disclosed embodiment, the obtained multiple facial image quality evaluation values ​​at multiple angles and directions are compared and the minimum value is selected as the facial image evaluation value of the target facial image.

[0076] In the disclosed embodiment, the face area of ​​the face image is converted to grayscale, and face image quality evaluation values ​​are calculated in multiple angular directions. The minimum value is selected as the face image evaluation value. Compared with selecting an angular direction to calculate the image quality evaluation value as the face image evaluation value, the obtained face image evaluation value is more accurate and more consistent with the actual image quality.

[0077] In the embodiment of the present disclosure, based on the pixel gradient values ​​in multiple gradient directions, the face image quality evaluation values ​​in multiple angle directions are determined, including:

[0078] Determine the pixel gradient value accumulation value of the target pixel point in each gradient direction of the preset number of gradient directions, and obtain the face image quality evaluation value in the preset number of angular directions, wherein one gradient direction corresponds to one angular direction, and the target pixel point is each pixel point in the length direction and the width direction of the image after grayscale processing;

[0079] Among them, each gradient direction includes multiple sub-gradient directions corresponding to the target pixel point, and the multiple sub-gradient directions include at least two of the horizontal direction, vertical direction and diagonal direction of the target pixel block. The target pixel block has the target pixel point as the vertex, and is spaced K pixels in the length direction and M pixels in the width direction from other vertices, and K and M are positive integers greater than or equal to 0.

[0080] In the embodiment of the present disclosure, considering the calculation complexity and the calculation accuracy comprehensively, the preset number of gradient directions may include a first gradient direction, a second gradient direction, a third gradient direction and a fourth gradient direction. The face image quality evaluation values ​​in the preset number of angle directions are the face image quality evaluation values ​​in the first angle direction, the second angle direction, the third angle direction and the fourth angle direction.

[0081] In the embodiment of the present disclosure, the face image quality evaluation values ​​in the first angle direction, the second angle direction, the third angle direction, and the fourth angle direction can be determined based on the pixel gradient value accumulation values ​​of the target pixel points in the first gradient direction, the second gradient direction, the third gradient direction, and the fourth gradient direction. For example, the following formula is used:

[0082]

[0083]

[0084]

[0085]

[0086] Where f represents the face image, D(f) represents the clarity value of the face image f, and D 1 (f) represents the face image quality evaluation value in the first angle direction, D 2 (f) represents the face image quality evaluation value in the second angle direction, D 3 (f) represents the face image quality evaluation value in the third angle direction, D 4 (f) represents the face image quality evaluation value in the fourth angle direction, h and w are the width and height of the face area image respectively, (x, y) represents the pixel coordinates, x represents the horizontal coordinate of the pixel coordinates, y represents the vertical coordinate of the pixel coordinates, and f(x, y) is the grayscale value of the face image at the pixel coordinates (x, y).

[0087] In the disclosed embodiment, based on the pixel gradient values ​​in multiple gradient directions, the facial image quality evaluation values ​​in multiple angular directions are determined, which can also be understood as an implementation process of quality scoring of facial images based on a multi-directional angular evaluation function.

[0088] Among them, the multi-directional quality evaluation function can preliminarily score the quality of the face image, which can be obtained by selecting the quality evaluation function of different angles to obtain the sub-function D 1 (f) D 2 (f) D 3 (f) and D 4 (f), for example: D 1 (f) can be the sub-function obtained by selecting direction A, and D2(f) can be the sub-function obtained by selecting direction B.

[0089] In the disclosed embodiment, let the width and height of the face image be h and w respectively, and the pixel coordinates of a certain pixel point of the face region image be (x, y). The grayscale value of the face image at the pixel point is f(x, y), and the clarity value of the face image f is D(f). Among them, four gradient directions are selected to determine the face quality evaluation values ​​in the first angle direction, the second angle direction, the third angle direction, and the fourth angle direction, respectively.

[0090] In the embodiment of the present disclosure, each gradient direction includes multiple sub-gradient directions corresponding to the target pixel point, and the multiple sub-gradient directions are, for example, a horizontal direction and a vertical direction, a diagonal direction and a diagonal direction. The multiple sub-gradient directions may be a first gradient direction, a second gradient direction, a third gradient direction, and a fourth gradient direction.

[0091] The sub-gradient directions included in the first gradient direction are determined based on the first target pixel block composed of pixel coordinates (x, y), pixel coordinates (x+1, y) and pixel coordinates (x, y+1). The pixel point with pixel coordinates (x, y) is the vertex coordinate of the first target pixel block, and (x+1, y) and (x, y+1) are another pixel point in the length direction and width direction respectively separated from (x, y) by 0 pixels. The first gradient direction includes the sub-gradient direction in the horizontal direction between (x+1, y) and (x, y) in the first target pixel block, and the sub-gradient direction in the vertical direction between (x, y+1) and (x, y). The pixel gradient value corresponding to the pixel point with pixel coordinates (x, y) can be expressed as |f(x+1, y)-f(x, y)| 2 -|f(x,y+1)-f(x,y)| 2 . Calculate the pixel gradient value accumulation value in the width h and height w directions of the face image as above The face image quality evaluation value in the first angle direction can be obtained.

[0092] The sub-gradient directions included in the second gradient direction are determined based on the pixel coordinates (x+1, y+1), the pixel coordinates (x, y), and the second target pixel block composed of the pixel coordinates (x+1, y) and the pixel coordinates (x, y+1). The pixel point with the pixel coordinates (x, y) is the vertex coordinate of the second target pixel block, (x+1, y) and (x, y+1) are another pixel point with a length direction and a width direction of 0 pixels from (x, y), and (x+1, y+1) is another pixel point with a length direction and a width direction of 1 pixel from (x, y). The second gradient direction includes the pixel point between (x+1, y+1) and (x, y) in the second target pixel block, which is θ with respect to the horizontal coordinate axis. 2 The diagonal sub-gradient direction in the degree direction, and the direction between (x+1,y) and (x,y+1) is (180-θ2 ) degree direction. The pixel gradient value corresponding to the pixel point at pixel coordinates (x, y) can be expressed as |f(x+1, y+1)-f(x, y)| 2 -|f(x+1,y)-f(x,y+1)| 2 . Calculate the pixel gradient value accumulation value in the width h and height w directions of the face image as above The face image quality evaluation value in the second angle direction can be obtained.

[0093] The sub-gradient directions included in the third gradient direction are determined based on the pixel coordinates (x+1, y+2), the pixel coordinates (x, y), and the third target pixel block composed of the pixel coordinates (x+2, y) and the pixel coordinates (x, y+1). The pixel point with the pixel coordinates (x, y) is the vertex coordinate of the third target pixel block, (x+1, y+2) is another pixel point that is 1 pixel away from (x, y) in the length direction and 2 pixels away from (x, y) in the width direction, (x+2, y) is another pixel point that is 2 pixels away from (x, y) in the length direction and 0 pixels away from (x, y) in the width direction, and (x, y+1) is another pixel point that is 0 pixels away from (x, y) in the length direction and 1 pixel away from (x, y) in the width direction. The third gradient direction includes the pixel points between (x+1, y+2) and (x, y) in the third target pixel block that are θ with respect to the horizontal coordinate axis. 3 The diagonal sub-gradient direction in the degree direction, and the direction between (x+2,y) and (x,y+1) that is (180-θ 3 ) degree direction. The pixel gradient value corresponding to the pixel point at pixel coordinates (x, y) can be expressed as |f(x+1, y+2)-f(x, y)| 2 -|f(x+2,y)-f(x,y+1)| 2 . Calculate the pixel gradient value accumulation value in the width h and height w directions of the face image as above The face image quality evaluation value in the third angle direction can be obtained.

[0094] The sub-gradient directions included in the fourth gradient direction are determined based on the pixel coordinates (x+1, y), the pixel coordinates (x, y+2), and the fourth target pixel block composed of the pixel coordinates (x+2, y+1) and the pixel coordinates (x, y). The pixel point with the pixel coordinates (x, y) is the vertex coordinate of the fourth target pixel block, (x+1, y) is another pixel point that is 1 pixel away from (x, y) in the length direction and 0 pixel points in the width direction, (x, y+2) is another pixel point that is 0 pixel away from (x, y) in the length direction and 2 pixel points in the width direction, and (x+2, y+1) is another pixel point that is 2 pixel away from (x, y) in the length direction and 1 pixel point in the width direction. The fourth gradient direction includes the fourth target pixel block between (x+1, y) and (x, y+2) that is θ with respect to the horizontal coordinate axis. 4 The diagonal sub-gradient direction in the degree direction, and the direction between (x+2,y+1) and (x,y) that is (180-θ 4 ) degree direction. The pixel gradient value corresponding to the pixel point at pixel coordinates (x, y) can be expressed as |f(x+1, y)-f(x, y+2)| 2 -|f(x+2,y+1)-f(x,y)| 2 . Calculate the pixel gradient value accumulation value in the width h and height w directions of the face image as above The face image quality evaluation value in the fourth angle direction can be obtained.

[0095] In the disclosed embodiment, multiple sub-gradient directions correspond to multiple angular directions. Among them, the first gradient direction corresponds to the first angular direction, the second gradient direction corresponds to the second angular direction, the third gradient direction corresponds to the third angular direction, and the fourth gradient direction corresponds to the fourth angular direction. Among them, in the first angular direction, the grayscale values ​​of the coordinate points with pixel coordinates (x+1, y) and the coordinate points with pixel coordinates (x, y+1) are selected to make the square difference of the absolute value of the difference, and the face image quality evaluation value is D1(f). In the second angular direction, the grayscale values ​​of the coordinate points with pixel coordinates (x+1, y+1) and the coordinate points with pixel coordinates (x, y) are selected to make the square of the absolute value of the difference, the grayscale values ​​of the coordinate points with pixel coordinates (x+1, y) and the coordinate points with pixel coordinates (x, y+1) are selected to make the square of the absolute value of the difference, and the difference between the two square values ​​is D2(f) to obtain the face image quality evaluation value. In the third angle direction, the square of the absolute value of the difference between the grayscale values ​​of the coordinate point with pixel coordinates (x+1, y+2) and the coordinate point with pixel coordinates (x, y) is selected, and the square of the absolute value of the difference between the grayscale values ​​of the coordinate point with pixel coordinates (x+2, y) and the coordinate point with pixel coordinates (x, y+1) is selected. The facial image quality evaluation value obtained by subtracting the two square values ​​is D3(f). In the fourth angle direction, the square of the absolute value of the difference between the grayscale values ​​of the coordinate point with pixel coordinates (x+1, y) and the coordinate point with pixel coordinates (x, y+2) is selected, and the square of the absolute value of the difference between the grayscale values ​​of the coordinate point with pixel coordinates (x+2, y+1) and the coordinate point with pixel coordinates (x, y) is selected. The square of the absolute value of the difference between the grayscale values ​​of the coordinate point with pixel coordinates (x+2, y+1) and the coordinate point with pixel coordinates (x, y) is selected. The facial image quality evaluation value obtained by subtracting the two square values ​​is D4(f). It is assumed that among D1(f), D2(f), D3(f) and D4(f), the facial image quality evaluation value represented by D3(f) is the smallest. Select D(f)=D3(f), and obtain the face image quality evaluation value D3(f) obtained by inputting the face image f into the multi-directional quality evaluation function.

[0096] In the disclosed embodiment, the multi-directional quality evaluation function can give appropriate image quality evaluation values ​​for most face images, but the individual numerical errors are large, and the unreasonable face image quality values ​​can be adjusted. Among them, in the case of large numerical errors, for example: during adjustment, a certain image looks blurry, and the image quality value should be low at this time, and the image quality value obtained by the multi-directional quality evaluation function is greater than the quality average threshold, wherein the image greater than the average threshold is clear, and the image less than the average threshold is blurry. At this time, the image quality value of the current image should correspond to a clear image, which is inconsistent with the actual quality of the image, and the image quality value of the image is adjusted at this time.

[0097] In the disclosed embodiment, image quality evaluation values ​​in multiple different angular directions are calculated based on a multi-directional quality evaluation function, and the minimum value is selected to obtain a facial image evaluation value of the target facial image. Compared with the facial image evaluation value calculated by selecting only one angular direction, the obtained facial image evaluation value is more accurate and more consistent with the actual image quality.

[0098] In the disclosed embodiment, the specific implementation process of the method for obtaining the face image quality evaluation model is described.

[0099] Figure 4 is a flowchart of a method for obtaining a face image quality evaluation model according to an exemplary embodiment. Figure 4 As shown, the following steps are included:

[0100] In step S41, the training face image is input into the prediction network to obtain the predicted evaluation value of the face image quality of the training face image.

[0101] In the disclosed embodiment, the prediction network can identify features of an input facial image based on information obtained through pre-learning and training, and output a prediction evaluation value reflecting good facial image quality based on the input facial image.

[0102] In step S42, the evaluation value interval to which the predicted evaluation value belongs is determined among a plurality of evaluation value intervals, wherein the plurality of evaluation value intervals are pre-divided based on the first face image quality evaluation label value of the training face image.

[0103] In the disclosed embodiment, the prediction network includes a soft stagewise regression network (SSR-Net), wherein the soft stagewise regression network includes: a multi-stage strategy, a dynamic interval, and a loss function.

[0104] In the embodiment of the present disclosure, the multiple evaluation value intervals can be obtained based on a multi-stage strategy of a soft stage regression network. The multi-stage strategy is to convert [D 1 ,D 2 The image quality score of the range is divided into K stages, each stage is divided into m non-overlapping intervals, and for the i-th interval, the interval representative score μ is used. The final score is obtained by using the weighted sum of the probability p of each stage and the interval representative score μ as shown in the following formula:

[0105]

[0106] Among them, m k is the number of intervals divided into the kth stage, m j is the number of intervals divided in the jth stage, i is the scoring interval index, and the interval representative score

[0107] In step S43, a second facial image quality evaluation label value is determined based on the evaluation value interval to which the predicted evaluation value belongs.

[0108] In the embodiment of the present disclosure, the offsets Δk and η can be adjusted based on the number of segments output by the image quality evaluation model through the dynamic range of the SSR network, and the offsets Δk and η can be adjusted based on the number of segments output by the image quality evaluation model. k Make adjustments as shown in the following formula:

[0109] m k '=m k (1+Δ k )

[0110]

[0111]

[0112] Among them, m k is the number of intervals divided into the kth stage, m j is the number of intervals divided into the jth stage, m k ' is the number of scoring intervals divided into the kth stage after adjustment, w k ' is the span of each scoring interval, i' is the index of the scoring interval after offset, η i (k) Adjust the offset for the number of segments in the i-th interval of the k-th stage.

[0113] In step S44, based on the loss between the first face image quality evaluation label value and the second face image quality evaluation label value, a prediction network is trained to obtain a face image quality evaluation model.

[0114] In the disclosed embodiment, the prediction network includes a soft stagewise regression capsule network (Soft Stagewise Regression Network, SSR-Net).

[0115] In the disclosed embodiment, the SSR network in the prediction network adopts fully supervised learning, and the loss of the first face image quality evaluation label value and the second face image quality evaluation label value can be calculated through the loss function of the SSR network, and the image quality evaluation model is trained until the loss converges.

[0116] In the disclosed embodiment, each vector neuron of the capsule network represents various attributes of a specific entity appearing in the image. In a face image, the specific entity may be a face, and the attributes of the specific entity may be the proportion of the facial features in the face, etc. These attributes may include many different types of instantiation parameters, such as the horizontal distance between the eyes of the face in the face image, the vertical distance between the eyebrows on the same side, etc. The modulus of the vector in the capsule network represents the probability of the feature existing, and the direction of the vector represents the posture information of the feature. The capsule network can fully utilize the spatial information in the image.

[0117] In the disclosed embodiment, by identifying features of a face in a face image based on a capsule network included in a prediction network, the face area can be accurately identified in the face image, thereby improving the efficiency and accuracy of the face unlocking process and enhancing the user experience.

[0118] Figure 5 is a flowchart of a method for determining a training face image according to an exemplary embodiment. Figure 5 As shown, the following steps are included:

[0119] In step S51, a first set of face images with different sharpness is acquired based on multiple scene attributes.

[0120] In the disclosed embodiment, the camera of the face unlocking system collects facial images, and can collect a variety of different environmental scenes, lighting conditions, and various motion patterns of different users. It can also collect blurred images due to appropriate shaking of the device to obtain facial image data of different clarity and image quality. All the acquired facial image data of different clarity and image quality constitute a first facial image set.

[0121] In step S52, face detection is performed on the images in the first face image set to obtain face detection results, as well as face bounding boxes and face key points of each face image.

[0122] In the disclosed embodiment, the face detection result of performing face detection on an image includes: a face is detected or not detected, the face boundary box of the face image is related to the location of the face area in the image currently undergoing face detection, and the face key points can be pre-set, for example: five face key points can be pre-set.

[0123] In step S53, based on the face detection result, the face bounding box and the face key points, the face images that meet the conditions are screened to obtain a second face image set.

[0124] The facial images that meet the conditions include at least one of the following:

[0125] The face detection result is that a face is detected.

[0126] The pixel area in the face bounding box is within the preset range.

[0127] The pixel coordinates corresponding to the facial key points are located in the image with a preset length direction and a preset width direction.

[0128] In the disclosed embodiment, the face boundary box is used to determine whether the pixel area is within a preset range, and the pixel coordinates corresponding to the face key points are used to determine whether they are located within an image having a preset length direction and a preset width direction.

[0129] In step S54, the second face image set is used as training face images.

[0130] In the embodiment of the present disclosure, the preset range can be determined based on the width and height of the preset image pixel size. Assume that the width of the image pixel size is w 0 , height is h 0 , the image pixel size corresponds to the pixel size width and height set corresponding to the acquisition characteristics of the current acquisition device. Then the preset length direction and width direction can be [0,w 0 ] and [0,h 0 ], the preset range can be a preset pixel area range. For example, the pixel area range can be [a, b]. If the pixel area S is less than a, it means that the pixel area of ​​the current face boundary box is too small. If the pixel area S is greater than b, it means that the pixel area of ​​the current face boundary box is too large. The preset range is used to control the face pixel area within a suitable range for image quality evaluation. The collected image is input into the face detection model, and the face image is screened out in combination with the screening conditions. An image with a detected face, predicted pixel coordinates within a preset range, and a moderate face pixel area can be obtained.

[0131] In the embodiment of the present disclosure, the collected face image can be used to perform face detection using a face detection model, and obtain a face boundary frame and N face key points. Assume that the pixel size width of the collected image is w 0 , height is h 0 , we can combine the face unlocking task to filter out the face images that meet at least one of the following conditions in the collected face image data: (1) No face is detected in the image. (2) The width and height coordinates of the predicted pixels of the N key points are not in [0,w 0 ] and [0,h 0 ] range. (3) Images whose pixel area of ​​the face bounding box is too large or too small. By filtering out face images, face images that meet the face unlocking task can be obtained.

[0132] In the disclosed embodiment, the collected facial images are processed in combination with screening conditions to screen out facial images that do not meet the quality evaluation standards for the face unlocking task, so as to obtain a highly accurate facial image quality score, which can improve the efficiency of face unlocking and enhance the user experience.

[0133] In the disclosed embodiment, in response to the first image quality evaluation value being less than a quality evaluation threshold, the face image is discarded.

[0134] In the disclosed embodiment, if the image quality evaluation value of the current face image is less than the quality evaluation threshold, indicating that the quality of the current face image does not meet the image quality standard of the face unlocking task, the current face image is discarded and a new face image is captured.

[0135] In the disclosed embodiment, if the face image quality evaluation value is less than the quality evaluation threshold, the current face image is discarded, and images whose face image quality does not meet the unlocking task quality standard can be screened out, so that face unlocking can be performed accurately, facilitating the completion of the face unlocking task.

[0136] In the embodiment of the present disclosure, using a face image as a face image for face unlocking includes:

[0137] The face image is used as a face authentication image for face unlocking authentication, and the quality evaluation threshold is a first quality evaluation threshold; and / or the face image is used as a face reference image for face unlocking entry, and the quality evaluation threshold is a second quality evaluation threshold; the first quality evaluation threshold is higher than the second quality evaluation threshold.

[0138] In the embodiment of the present disclosure, different thresholds can be set in different stages in combination with the face unlocking task. In the face entry stage, a quality threshold θ 1 The value of is greater than the average quality threshold value to record a clear face image, where the average quality threshold value is determined by experiments. In the face unlocking process, you can set the value of 1 Compared with the smaller quality threshold θ 2 , so that the unlockable image range of the image quality evaluation value collected in the face unlocking link is appropriately increased, and the face can be unlocked even in the case of poor image quality.

[0139] In the disclosed embodiment, different corresponding thresholds are set for different face unlocking task links. In the input link, the threshold is set higher than the average value, so that a high-definition face image can be input, which is convenient for identification and matching in the subsequent unlocking process. In the unlocking link, the threshold is set lower than the average value, so that the face unlocking task can be completed even if the quality of the recognized face image is poor, which is convenient for users to use and improves the user experience.

[0140] In the embodiment of the present disclosure, the face image quality assessment model can be trained based on the following process:

[0141] a) Use the camera of the face unlocking system to collect multiple face images of different clarity and image quality. The image acquisition module of the face unlocking system acquires multiple face images.

[0142] b) The image preprocessing module of the face unlocking system obtains the face area in each image. The face detection model is used to perform face detection on the collected face image data, and the face bounding box and N face key points of each image are obtained. Assume that the pixel size width of the collected image is w 0 , height is h 0 , combined with the face unlocking task, all collected face images that meet at least one of the following conditions are screened out: (1) Images where no face is detected. (2) Images where the width and height coordinates of the predicted pixels of the N key points are not in the range [0,w 0 ] and [0,h 0 ] range. (3) Images whose pixel areas of face bounding boxes are too large or too small, thus obtaining multiple face images after screening.

[0143] c) Based on the energy gradient function, four angle directions are selected to construct a multi-directional quality evaluation function. The face area of ​​each image is processed in grayscale, and the face image after grayscale processing is input into the multi-directional quality evaluation function. The image quality evaluation values ​​in the four angle directions are calculated respectively, and the minimum value is selected to obtain the preliminary quality evaluation value of each face image. The preliminary quality evaluation values ​​of all collected face images constitute the image quality evaluation data set.

[0144] d) Manually adjust the unreasonable image quality evaluation values ​​in the image quality evaluation data set obtained in step c), and use the normalization formula to normalize the image quality evaluation values ​​in all image quality evaluation data sets to obtain a label data set composed of the normalized image quality evaluation values.

[0145] e) Construct a face image quality evaluation model based on the SSR network combined with the capsule network.

[0146] The multiple face images screened in b) are respectively input into the face image quality assessment model, and the capsule network is used to identify the features of each face image respectively.

[0147] The SSR network includes: multi-stage strategy, dynamic interval and loss function.

[0148] The multi-stage strategy is used to pre-divide the image quality evaluation values ​​in the range [0,1] into K stages, and each stage is divided into m non-overlapping intervals. The stage and interval to which each image belongs are determined respectively, and the predicted image quality evaluation value s' of each image is obtained by using the weighted sum of the probability p of each stage and the representative score μ of the interval corresponding to each image.

[0149] The dynamic range is used to adjust the offset Δk and η for the stage m based on the number of segments output by the model. k Make adjustments.

[0150] The loss function is used to calculate the image quality loss L by subtracting the predicted image quality evaluation value s' of each image from the corresponding image quality evaluation value in the label data set. The model is trained based on the image quality loss of each image until the loss converges, and a face image quality evaluation model with good face image quality evaluation effect is obtained.

[0151] In the disclosed embodiment, a face image quality assessment model is constructed based on a prediction network, and a loss training model is performed based on a first face image quality assessment label value and a second face image quality assessment label value, so that a model with good image quality assessment effect can be obtained.

[0152] The disclosed embodiment proposes a face unlocking method, and can evaluate the clarity and blurriness of a face image based on an image quality evaluation value. The method proposed in the disclosed embodiment can ensure that a clear face image is obtained during the face entry phase of the face unlocking system, and save time for the unlocking phase, increase the unlocking rate, and improve the user experience.

[0153] Based on the same concept, the embodiment of the present disclosure also provides a face unlocking device.

[0154] It is understandable that in order to achieve the above functions, the face unlocking device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of the examples disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0155] Figure 6 FIG. 1 is a block diagram of a face unlocking device according to an exemplary embodiment. Figure 6 The device 100 includes a collecting unit 101, an executing unit 102 and a judging unit 103.

[0156] The acquisition unit 101 is used to acquire a face image.

[0157] The execution unit 102 is used to input the face image into the face image quality assessment model to obtain a first image quality assessment value of the face image; the face image quality assessment model is trained based on a face image quality assessment label data set, and the face image quality assessment label data set includes face image quality assessment values ​​whose assessment accuracy is greater than an accuracy threshold.

[0158] The judgment unit 103 is configured to use the facial image as a facial image for face unlocking in response to the first image quality evaluation value being greater than a quality evaluation threshold.

[0159] In one embodiment, the execution unit 102 obtains a label data set in the following manner: collects multiple facial images, and performs facial image quality evaluation on each of the multiple facial images to obtain a first facial image quality evaluation label data set, where the first facial image quality evaluation label data set includes facial image quality evaluation values ​​of the multiple facial images; normalizes each facial image quality evaluation value in the first facial image quality evaluation label data set to obtain a second facial image quality evaluation label data set; wherein the evaluation accuracy of the normalized facial image quality evaluation value is higher than the accuracy threshold.

[0160] In one embodiment, the execution unit 102 performs facial image quality evaluation, including: performing image grayscale processing on the facial area in the target facial image, where the target facial image is a facial image among multiple facial images; determining facial image quality evaluation values ​​in multiple angular directions based on pixel gradient values ​​in multiple gradient directions for the image after the image grayscale processing; and selecting the minimum value among the facial image quality evaluation values ​​in multiple angular directions as the facial image evaluation value of the target facial image.

[0161] In one embodiment, the execution unit 102 determines the facial image quality evaluation values ​​in multiple angular directions based on the pixel gradient values ​​in multiple gradient directions in the following manner, including: determining the pixel gradient value accumulation value of the target pixel point in each gradient direction of a preset number of gradient directions, and obtaining the facial image quality evaluation values ​​in the preset number of angular directions, wherein one gradient direction corresponds to one angular direction, and the target pixel point is each pixel point in the length direction and the width direction of the image after grayscale processing; wherein each gradient direction includes multiple sub-gradient directions corresponding to the target pixel point, and the multiple sub-gradient directions include at least two of the horizontal direction, the vertical direction and the diagonal direction of the target pixel block, and the target pixel block has the target pixel point as the vertex, and is spaced K pixels in the length direction and M pixels in the width direction from other vertices, and K and M are positive integers greater than or equal to 0.

[0162] In one embodiment, a facial image quality assessment model is trained based on a facial image quality assessment label data set in the following manner: a training facial image is input into a prediction network to obtain a predicted evaluation value of the facial image quality of the training facial image; among multiple evaluation value intervals, an evaluation value interval to which the predicted evaluation value belongs is determined, and the multiple evaluation value intervals are pre-divided based on a first facial image quality assessment label value of the training facial image; based on the evaluation value interval to which the predicted evaluation value belongs, a second facial image quality assessment label value is determined; based on the loss between the first facial image quality assessment label value and the second facial image quality evaluation label value, the prediction network is trained to obtain a facial image quality assessment model.

[0163] In one embodiment, the prediction network includes a capsule network for soft phase regression.

[0164] In one embodiment, the execution unit 102 determines the training face images in the following manner: based on multiple scene attributes, a first face image set with different clarity is obtained; face detection is performed on the images in the first face image set to obtain face detection results, as well as face bounding boxes and face key points of each face image; based on the face detection results, face bounding boxes and face key points, face images that meet the conditions are screened to obtain a second face image set; the second face image set is used as the training face images; wherein the face images that meet the conditions include at least one of the following: the face detection result is that a face is detected; the pixel area in the face bounding box is within a preset range; the pixel coordinates corresponding to the face key points are located in an image having a preset length direction and a width direction.

[0165] In one implementation, the judgment unit 103 is further configured to discard the face image in response to the first image quality evaluation value being less than a quality evaluation threshold.

[0166] In one embodiment, the judgment unit 103 uses the facial image as the facial image for face unlocking in the following manner: using the facial image as a facial authentication image for face unlocking authentication, and the quality evaluation threshold is a first quality evaluation threshold; and / or using the facial image as a facial reference image entered for face unlocking, and the quality evaluation threshold is a second quality evaluation threshold; the first quality evaluation threshold is higher than the second quality evaluation threshold.

[0167] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0168] Figure 71 is a block diagram of a device 200 for face unlocking according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0169] Reference Figure 7 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0170] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0171] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0172] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0173] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0174] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0175] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0176] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0177] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0178] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0179] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0180] Figure 8 is a block diagram of a device 300 for face unlocking according to an exemplary embodiment. For example, the device 300 may be provided as a server. Figure 8 , the apparatus 300 includes a processing component 322, which further includes one or more processors, and a memory resource represented by a memory 332 for storing instructions, such as an application, that can be executed by the processing component 322. The application stored in the memory 332 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 322 is configured to execute the instructions to perform the above method.

[0181] The device 300 may also include a power supply component 326 configured to perform power management of the device 300, a wired or wireless network interface 350 configured to connect the device 300 to a network, and an input / output (I / O) interface 358. The device 300 may operate based on an operating system stored in the memory 332, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0182] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0183] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0184] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0185] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0186] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A face unlocking method, It is characterized in that include: Collecting facial images; Inputting the face image into a face image quality evaluation model to obtain a first image quality evaluation value of the face image; The face image quality assessment model is trained based on a face image quality assessment label data set, wherein the face image quality assessment label data set includes face image quality assessment values ​​whose assessment accuracy is greater than an accuracy threshold; In response to the first image quality evaluation value being greater than a quality evaluation threshold, the facial image is used as a facial image for face unlocking.

2. The method according to claim 1, It is characterized in that The label dataset is obtained in the following way: Collecting a plurality of face images, and performing face image quality evaluation on each of the plurality of face images to obtain a first face image quality evaluation label data set, wherein the first face image quality evaluation label data set includes face image quality evaluation values ​​of the plurality of face images; Normalizing each face image quality evaluation value in the first face image quality evaluation label data set to obtain a second face image quality evaluation label data set; Among them, the evaluation accuracy of the normalized face image quality evaluation value is higher than the accuracy threshold.

3. The method according to claim 2, It is characterized in that The facial image quality evaluation comprises: Performing image grayscale processing on a face area in a target face image, wherein the target face image is a face image in the multiple face images; For the image after image grayscale processing, based on the pixel gradient values ​​in multiple gradient directions, determine the face image quality evaluation values ​​in multiple angle directions; The minimum value is selected from the facial image quality evaluation values ​​in the multiple angular directions as the facial image evaluation value of the target facial image.

4. The method according to claim 3, It is characterized in that The method of determining the face image quality evaluation values ​​in multiple angle directions based on the pixel gradient values ​​in multiple gradient directions includes: Determine the accumulated pixel gradient value of the target pixel point in each of the preset number of gradient directions, and obtain the face image quality evaluation values ​​in the preset number of angular directions, wherein one gradient direction corresponds to one angular direction, and the target pixel point is each pixel point in the length direction and the width direction of the grayscale processed image; Among them, each of the gradient directions includes multiple sub-gradient directions corresponding to the target pixel point, and the multiple sub-gradient directions include at least two of the horizontal direction, the vertical direction and the diagonal direction of the target pixel block. The target pixel block has the target pixel point as a vertex, and is spaced K pixels in the length direction and M pixels in the width direction from other vertices, and K and M are positive integers greater than or equal to 0.

5. The method according to any one of claims 1 to 4, It is characterized in that The face image quality assessment model is trained based on the face image quality assessment label dataset in the following manner: Inputting the training face image into the prediction network to obtain a predicted evaluation value of the face image quality of the training face image; Determine, among a plurality of evaluation value intervals, an evaluation value interval to which the predicted evaluation value belongs, wherein the plurality of evaluation value intervals are pre-divided based on the first face image quality evaluation label value of the training face image; Determining a second facial image quality evaluation label value based on the evaluation value interval to which the predicted evaluation value belongs; Based on the loss between the first face image quality evaluation label value and the second face image quality evaluation label value, the prediction network is trained to obtain a face image quality evaluation model.

6. The method according to claim 5, It is characterized in that The prediction network includes a capsule network for soft phase regression.

7. The method according to claim 5 or 6, It is characterized in that The training face image is determined in the following manner: Acquire a first face image set with different resolutions based on multiple scene attributes; Performing face detection on the images in the first face image set to obtain face detection results, as well as face bounding boxes and face key points of each face image; Based on the face detection result, the face bounding box and the face key points, screening face images that meet the conditions to obtain a second face image set; using the second face image set as training face images; The facial image that meets the conditions includes at least one of the following: The face detection result is that a face is detected; The pixel area in the face bounding box is within the preset range; The pixel coordinates corresponding to the facial key points are located in the image with a preset length direction and a preset width direction.

8. The method according to claim 1, It is characterized in that Also includes: In response to the first image quality evaluation value being less than the quality evaluation threshold, the face image is discarded.

9. The method according to claim 1, It is characterized in that The using the facial image as a facial image for face unlocking includes: Using the facial image as a facial authentication image for face unlocking authentication, the quality evaluation threshold being a first quality evaluation threshold; and / or Using the facial image as a facial reference image for face unlocking recording, and the quality evaluation threshold is a second quality evaluation threshold; The first quality evaluation threshold is higher than the second quality evaluation threshold.

10. A face unlocking device, It is characterized in that include: A collection unit, used for collecting face images; An execution unit, configured to input the face image into a face image quality evaluation model to obtain a first image quality evaluation value of the face image; the face image quality evaluation model is trained based on a face image quality evaluation label data set, and the face image quality evaluation label data set includes face image quality evaluation values ​​whose evaluation accuracy of face image quality evaluation values ​​is greater than an accuracy threshold; A judgment unit is used to use the facial image as a facial image for face unlocking in response to the first image quality evaluation value being greater than a quality evaluation threshold.

11. A face unlocking device, It is characterized in that include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Execute the face unlocking method described in any one of claims 1 to 9.

12. A storage medium, It is characterized in that The storage medium stores instructions. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to execute the face unlocking method described in any one of claims 1 to 9.

Images