Face image quality evaluation method and device
By identifying and optimizing key parts of face images, using predictive models to evaluate image quality and perform optimization processing, the problem of insufficient robustness of face image quality evaluation in the prior art is solved, and more accurate quality evaluation and optimization effects are achieved.
Patent Information
- Application Number
- CN202411948376.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-25
AI Technical Summary
The existing facial image quality evaluation method is not robust enough when dealing with complex scenes, and ignores the differences in the contribution of different areas of the facial image to quality, which is susceptible to changes in the external environment, resulting in identification errors.
By acquiring the face image to be detected, identifying and extracting multiple key part images, inputting them to the prediction model to obtain the image evaluation index, comparing and determining the quality level, and performing optimization processing for low-quality images to improve resolution.
Accurate evaluation of face image quality is achieved, robustness is improved in complex scenarios, and the quality of low-quality images is improved through optimization processing, reducing recognition errors.
Smart Images
Figure CN119887528A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method and device for evaluating the quality of facial images. Background Art
[0002] Face quality detection is a key technology in computer vision and provides preprocessing for face image processing. Traditional methods are based on computer vision algorithms, but they have poor generalization and are easily affected by environmental changes. Machine learning algorithms have improved accuracy, but are still not robust enough to situations such as lighting changes and motion blur. Deep learning methods, especially neural networks, are more robust than the previous two, but still need to be improved in complex scenarios.
[0003] Existing methods focus more on the overall face image, ignoring the differences in the contribution of different regions to the quality. At the same time, external environments such as weather and light changes can significantly affect the quality of face images and may lead to recognition errors. Summary of the invention
[0004] In view of this, the purpose of this application is to provide a facial image quality assessment method and device, aiming to achieve accurate assessment of facial image quality and effectively optimize low-quality facial images.
[0005] In a first aspect, the present application provides a method for assessing the quality of facial images, the method comprising: obtaining facial images to be detected, and identifying multiple key parts from each facial image to extract multiple key part images from the facial image; inputting the multiple key part images of each facial image into a prediction model to obtain an image evaluation index of the facial image; comparing the image evaluation index of each facial image with a preset evaluation index to determine the quality level of the facial image; performing optimization processing on facial images with low quality levels to obtain a high-resolution image corresponding to the facial image.
[0006] In a possible implementation, the prediction model is constructed in the following manner: acquiring face sample images, and identifying multiple key parts from each face sample image to extract multiple key part sample images from each face sample image; for each key part, obtaining a sample evaluation true value of the key part based on the key part sample image of the key part; performing weighted summation of the sample evaluation true values of all key parts to obtain a sample evaluation index of the face sample image; and expressing the relationship between the face sample image and the sample evaluation index of the face sample image using the prediction model.
[0007] In a possible implementation, the prediction model is trained in the following manner: for each key part, a key part sample image of the key part is input into the prediction model to obtain a sample evaluation prediction value of the key part; for each key part, the difference between the sample evaluation prediction value and the sample evaluation true value of the key part is calculated to obtain a loss value of the key part; the loss values corresponding to all key parts are weighted and summed according to preset weights to obtain a total loss function; and the parameters of the prediction model and the weight of the total loss function are updated through each iteration until the result of the total loss function is less than a preset standard.
[0008] In one possible implementation, the loss value of each key part is calculated in the following manner: for each key part, the average of the absolute values of the differences between the sample evaluation prediction values and the sample evaluation true values of all key part sample images of the key part is calculated to obtain the loss value of the key part.
[0009] In one possible implementation, the steps of performing optimization processing on a facial image with a low quality level to obtain a high-resolution image corresponding to the facial image include: extracting texture features of the facial image with a low quality level; inputting the texture features of the facial image into a reconstruction model to optimize the facial image to obtain a high-resolution image corresponding to the facial image.
[0010] In a possible implementation, texture features of a facial image with a low quality level are extracted in the following manner: extracting a local binary pattern (LBP) feature of the facial image; converting the LBP feature into a statistical histogram, and converting the statistical histogram into a one-dimensional vector; and normalizing the one-dimensional vector to obtain the texture feature.
[0011] In a second aspect, the present application provides a facial image quality assessment device, the device comprising: an acquisition module, used to acquire a facial image to be detected, and identify multiple key parts from each facial image to extract multiple key part images from the facial image; a prediction module, used to input multiple key part images of each facial image into a prediction model to obtain an image evaluation index of the facial image; an evaluation module, used to compare the image evaluation index of each facial image with a preset evaluation index to determine the quality level of the facial image; an optimization module, used to perform optimization processing on facial images with low quality levels to obtain a high-resolution image corresponding to the facial image.
[0012] In a possible implementation, the prediction module is also used to perform the following processing: obtaining face sample images, and identifying multiple key parts from each face sample image to extract multiple key part sample images from each face sample image; for each key part, obtaining the true sample evaluation value of the key part based on the key part sample image of the key part; performing weighted summation of the true sample evaluation values of all key parts to obtain the sample evaluation index of the face sample image; and expressing the relationship between the face sample image and the sample evaluation index of the face sample image using the prediction model.
[0013] In a third aspect, the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above method are performed.
[0014] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program executes the steps of the above method when executed by a processor.
[0015] The present application provides a method and device for assessing the quality of facial images, wherein the method comprises: obtaining facial images to be detected, and identifying multiple key parts from each facial image to extract multiple key part images from the facial sample image; inputting multiple key part images of each facial image into a prediction model to obtain an image evaluation index of the facial image; comparing the image evaluation index of each facial image with a preset evaluation index to determine the quality level of the facial image; performing optimization processing on facial images with low quality levels to obtain a high-resolution image corresponding to the facial image. Through the present application, accurate assessment of the quality of facial images is achieved, and low-quality facial images are effectively optimized.
[0016] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1A flowchart of a facial image quality assessment method provided in an embodiment of the present application;
[0019] Figure 2 A flowchart of constructing a prediction model provided in an embodiment of the present application;
[0020] Figure 3 A flowchart of a training prediction model provided in an embodiment of the present application;
[0021] Figure 4 A flowchart of optimizing the processing of facial images provided in an embodiment of the present application;
[0022] Figure 5 A schematic diagram of the structure of a facial image quality assessment device provided in an embodiment of the present application;
[0023] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0024] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.
[0025] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied to image processing technology.
[0026] Research has found that face quality detection is a key technology in the field of computer vision. It provides preprocessing support for face image processing, such as face recognition and attribute detection. By evaluating the quality of each frame, it can effectively screen out low-quality frame sequences, reduce manual burden, and improve processing efficiency. In the existing technology, although traditional computer vision algorithms can be used for quality detection, such as evaluating by calculating the variance of image pixel values, grayscale histograms, and images processed by the Laplace operator, their generalization is limited. Changes in the environment or equipment can easily lead to large fluctuations in results and insufficient accuracy.
[0027] In order to improve the accuracy, machine learning algorithms have been introduced. Although they have been improved compared with traditional methods, their robustness is still insufficient in complex scenarios such as lighting changes and motion blur. With the development of deep learning, neural networks are used to judge face quality and show good robustness, but they still need to be optimized in some complex environments. It is worth noting that existing methods focus more on the entire face image and ignore the different contributions of various facial regions (eyebrows, eyes, nose, mouth) to the overall quality. At the same time, external environments such as weather and light changes will also affect the quality of face images, thereby interfering with subsequent detection results and causing recognition errors. Technologies for optimizing low-quality face images, such as calculating Euclidean distance, similarity, image interpolation, etc., can improve image quality, but have poor robustness and high computational complexity, making it difficult to meet the needs of complex scenarios.
[0028] Based on this, the embodiments of the present application provide a facial image quality assessment method and device, aiming to achieve accurate assessment of facial image quality and effectively optimize low-quality facial images.
[0029] See also Figure 1 , Figure 1 The following is a flow chart of a method for evaluating the quality of a facial image provided by an embodiment of the present application. Figure 1 As shown in , the face image quality assessment method provided by the embodiment of the present application includes:
[0030] S101, obtaining a face image to be detected, and identifying a plurality of key parts from each face image to extract a plurality of key part images from the face sample image.
[0031] In this application, the multiple key parts are eyebrows, eyes, nose, and mouth.
[0032] S102: Input multiple key part images of each face image into a prediction model to obtain an image evaluation index of the face image.
[0033] Below through Figure 2 Introduce the specific process of building a prediction model.
[0034] Figure 2 A flowchart for constructing a prediction model provided in an embodiment of the present application.
[0035] S201, acquiring sample face images, and identifying a plurality of key parts from each sample face image, so as to extract a plurality of key part sample images from each sample face image.
[0036] S202 . For each key part, obtain a sample evaluation true value of the key part according to a key part sample image of the key part.
[0037] Here, according to the clarity, integrity and occlusion degree of each key part, the sample evaluation true value labels of the four key parts of each face sample are defined, and the sample evaluation true value is obtained by Y n ∈R 4 Indicates, where n = 1, 2, 3, ..., N, N is the total number of samples, Y n The sample evaluation true values of the four key parts of each sample, namely eyebrows, eyes, nose and mouth. The sample evaluation true value labels of each key part are divided into three levels: excellent (1.0-0.8), general (0.8-0.5), and poor (0.5-0.0).
[0038] S203: Perform a weighted summation on the true values of sample evaluations of all key parts to obtain a sample evaluation index of the sample face image.
[0039] S204: Using a prediction model to represent the relationship between the face sample image and the sample evaluation index of the face sample image.
[0040] Here, the quality scores of the four key parts are predicted. The prediction model is a regression prediction network based on Resnet50. Specifically, the key part sample image of each key part is input into the regression prediction network. The output of the regression prediction network is the predicted quality score y of the four key parts. n ∈R 4 The weighted sum of is the sample evaluation index.
[0041] Below through Figure 3 Introduce the specific process of training a prediction model.
[0042] Figure 3 A flowchart of the training prediction model provided in an embodiment of the present application.
[0043] S301 . For each key part, input a key part sample image of the key part into a prediction model to obtain a sample evaluation prediction value of the key part.
[0044] S302: For each key part, calculate the difference between the sample evaluation prediction value and the sample evaluation true value of the key part to obtain the loss value of the key part.
[0045] In a preferred example of the present application, the loss value of each key part is calculated in the following manner:
[0046] For each key part, the average of the absolute values of the differences between the sample evaluation prediction values and the sample evaluation true values of all key part sample images of the key part is calculated to obtain the loss value of the key part.
[0047] S303: Perform weighted summation on the loss values corresponding to all key parts according to preset weights to obtain a total loss function.
[0048] Specifically, the loss function for predicting the quality score of each key part is the L1 regression loss function. The total loss function L of the prediction network is the weighted sum of the loss functions of the four key parts, with the weight W∈R 4 Represents the corresponding weights of the four key parts. The weights are learnable parameters. The sum of the corresponding weights of the four key parts is 1, and each weight is initialized before training.
[0049] The total loss function can be obtained using the following formula:
[0050]
[0051] L=W1·L1+W2·L2+W3·L3+W4.L4
[0052] Among them, L i is the loss value corresponding to the i-th key part, i is the i-th key part, N is the total number of key part sample images, n is the n-th key part sample image, is the true value of the nth key part sample image corresponding to the ith key part, is the predicted value of the nth key part sample image at the ith key part, L is the loss function corresponding to the face sample image, W i is the weight corresponding to the i-th key part.
[0053] S304. Update the parameters of the prediction model and the weight of the total loss function through each iteration until the result of the total loss function is less than a preset standard.
[0054] return Figure 1 , S103, comparing the image evaluation index of each face image with a preset evaluation index to determine the quality level of the face image.
[0055] Here, a preset evaluation index can be set manually. If the image evaluation index is less than the preset evaluation index, the face image is considered to be a low-quality sample LR. i Otherwise, the face image is considered to be a high-quality sample HR i .
[0056] S104: Perform optimization processing on the face image with a low quality level to obtain a high-resolution image corresponding to the face image.
[0057] Below through Figure 4 This section describes the specific process of optimization.
[0058] Figure 4A flowchart for optimizing facial images provided in an embodiment of the present application.
[0059] S401. For a face image with a low quality level, extract texture features of the face image.
[0060] In a preferred example of the present application, the texture features of the face image with a low quality level are extracted in the following manner:
[0061] A local binary pattern (LBP) feature with a low quality level is extracted, the LBP feature is converted into a statistical histogram, the statistical histogram is converted into a one-dimensional vector, and the one-dimensional vector is normalized to obtain a texture feature.
[0062] S402: Inputting the texture features of the facial image into a reconstruction model to optimize the facial image and obtain a high-resolution image corresponding to the facial image.
[0063] Here, according to the LBP texture features of low-quality face images, a super-resolution reconstruction network based on SRResnet (Super-Resolution Residual Network) is designed. The network outputs the optimized high-resolution image SR i , where i represents the i-th low-quality face image, the loss function of the network is the MSE loss function, which calculates the reconstructed SR i Image and real HR i The sum of the squares of the differences in pixel values I between images is used to optimize the loss function through continuous iteration to obtain the best result.
[0064] The MSE loss function can be expressed using the following formula.
[0065]
[0066] Here, L MSE is the MSE loss function, N is the total number of low-quality face images, i represents the i-th low-quality face image, is the pixel value of the true high-resolution image of the i-th low-quality face image, is the pixel value of the high-resolution image obtained after the i-th sample is reconstructed by the network.
[0067] Based on the same inventive concept, an embodiment of the present application also provides a facial image quality assessment device corresponding to the facial image quality assessment method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned facial image quality assessment method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0068] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of a facial image quality assessment device provided in an embodiment of the present application. The facial image quality assessment device 500 includes:
[0069] The acquisition module 501 is used to acquire a face image to be detected, and identify a plurality of key parts from each face image to extract a plurality of key part images from the face sample image;
[0070] Prediction module 502, used for inputting multiple key part images of each face image into the prediction model to obtain the image evaluation index of the face image;
[0071] An evaluation module 503, for comparing the image evaluation index of each face image with a preset evaluation index to determine the quality level of the face image;
[0072] The optimization module 504 is used to perform optimization processing on the face image with a low quality level to obtain a high-resolution image corresponding to the face image.
[0073] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown in , the electronic device 600 includes a processor 610 , a memory 620 and a bus 630 .
[0074] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 communicates with the memory 620 via the bus 630. When the machine-readable instructions are executed by the processor 610, the above-mentioned Figure 1 as well as Figure 2 The steps of the face image quality assessment method in the method embodiment shown in the figure, and the specific implementation method thereof can be referred to the method embodiment, which will not be described in detail here.
[0075] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 to Figure 4 The steps of the face image quality assessment method in the method embodiment shown in the figure, and the specific implementation method thereof can be referred to the method embodiment, which will not be described in detail here.
[0076] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0077] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0078] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0079] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0080] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0081] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A method for assessing the quality of a face image, characterized in that: The method comprises: Acquire a face image to be detected, and identify multiple key parts from each face image to extract multiple key part images from the face image; Input multiple key part images of each face image into the prediction model to obtain the image evaluation index of the face image; Comparing the image evaluation index of each face image with a preset evaluation index to determine the quality level of the face image; An optimization process is performed on a face image with a low quality level to obtain a high-resolution image corresponding to the face image.
2. The method according to claim 1, characterized in that The prediction model is constructed in the following way: Acquire sample face images, and identify multiple key parts from each sample face image to extract multiple key part sample images from each sample face image; For each key part, according to the key part sample image of the key part, the sample evaluation true value of the key part is obtained; The true values of sample evaluations of all key parts are weighted and summed to obtain the sample evaluation index of the face sample image; The relationship between each face sample image and the sample evaluation index of the face sample image is represented by the prediction model.
3. The method according to claim 2, characterized in that The prediction model is trained in the following way: For each key part, a key part sample image of the key part is input into the prediction model to obtain a sample evaluation prediction value of the key part; For each key part, the difference between the sample evaluation prediction value and the sample evaluation true value of the key part is calculated to obtain the loss value of the key part; The loss values corresponding to all key parts are weighted and summed according to the preset weights to obtain the total loss function; The prediction model is iterated to update the parameters of the prediction model and the weight of the total loss function until the result of the total loss function is less than a preset standard.
4. The method according to claim 3, characterized in that The loss value of each key part is calculated as follows: For each key part, the average of the absolute values of the differences between the sample evaluation prediction values and the sample evaluation true values of all key part sample images of the key part is calculated to obtain the loss value of the key part.
5. The method according to claim 1, characterized in that The step of performing optimization processing on a face image with a low quality level to obtain a high-resolution image corresponding to the face image includes: For a face image with a low quality level, extracting texture features of the face image; The texture features of the face image are input into a reconstruction model to optimize the face image and obtain a high-resolution image corresponding to the face image.
6. The method according to claim 5, characterized in that The texture features of the face image with low quality level are extracted in the following way: Extracting the local binary pattern LBP feature of the face image; Convert the LBP feature into a statistical histogram, and convert the statistical histogram into a one-dimensional vector; The one-dimensional vector is normalized to obtain the texture feature.
7. A facial image quality assessment device, characterized in that: The device comprises: An acquisition module is used to acquire a face image to be detected, and identify a plurality of key parts from each face image to extract a plurality of key part images from the face image; A prediction module, used for inputting multiple key part images of each face image into a prediction model to obtain an image evaluation index of the face image; An evaluation module, used to compare the image evaluation index of each face image with a preset evaluation index to determine the quality level of the face image; The optimization module is used to perform optimization processing on a face image with a low quality level to obtain a high-resolution image corresponding to the face image.
8. The device according to claim 7, characterized in that The prediction module is also used to perform the following processing: Acquire sample face images, and identify multiple key parts from each sample face image to extract multiple key part sample images from each sample face image; For each key part, according to the key part sample image of the key part, the sample evaluation true value of the key part is obtained; The true values of sample evaluations of all key parts are weighted and summed to obtain the sample evaluation index of the face sample image; The relationship between the face sample image and the sample evaluation index of the face sample image is represented by the prediction model.
9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 6.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.