A method and apparatus for evaluating infrared image quality

Abstract

This application provides an infrared image quality assessment method and apparatus to improve the accuracy of infrared face image quality assessment. The method includes: acquiring a first infrared face image; inputting the first infrared face image into a trained face quality assessment model to obtain a quality score for the first infrared face image, wherein the face quality assessment model is obtained by training an initial assessment model based on a first sample set, the first sample set including at least one second infrared face image with a quality score label; the initial assessment model is obtained by training on visible light face images; and evaluating the quality of the first infrared face image based on its quality score. Using a face quality assessment model trained on infrared face images to assess the quality of the first infrared face image can improve the accuracy of infrared face image quality assessment.

Description

Technical Field

[0001] This application relates to the field of facial recognition technology, and in particular to an infrared image quality evaluation method and apparatus. Background Technology

[0002] With the development of deep learning, facial recognition technology has been widely used in scenarios such as mobile payment, video surveillance, and access control systems. Most existing facial recognition algorithms rely on high-quality facial images as input. However, in actual facial image acquisition, the quality of the obtained images varies greatly due to complex and variable ambient lighting, significantly impacting the performance of subsequent facial recognition algorithms. Therefore, facial quality assessment algorithms can be used to estimate the utility of the acquired facial images in achieving reliable and accurate recognition, thereby selecting higher-quality facial images for input into the facial recognition algorithm.

[0003] Current quality assessment algorithms are designed for visible light images. However, images captured in visible light are easily affected by lighting conditions. For example, facial details captured in low-light environments are not visible, resulting in low overall quality and a sharp drop in the recognition rate of face recognition algorithms. To solve this problem, face images captured under visible light are input into image processing algorithms to restore details in overexposed or underexposed areas of the face. Then, a quality assessment algorithm selects frontal face images with clear details and minimal occlusion for use in face recognition. However, this approach requires an image processing step before inputting the image into the face recognition system, which slows down the system's processing speed to some extent. Infrared imaging is robust to changes in lighting conditions, and can capture recognizable facial images even in low-light environments, which can improve recognition accuracy to some extent. Using infrared facial images as input is more conducive to improving the performance of facial recognition systems. However, due to the differences in the equipment used to acquire infrared and visible light facial images, existing quality assessment algorithms for evaluating visible light and infrared facial images acquired under the same conditions yield significantly different quality assessment results. In other words, existing quality assessment algorithms cannot accurately evaluate the quality of infrared facial images. Therefore, improving the accuracy of infrared facial image quality assessment is an urgent problem to be solved. Summary of the Invention

[0004] This application provides an infrared image quality evaluation method and apparatus to improve the accuracy of infrared face image quality evaluation.

[0005] In a first aspect, embodiments of this application provide an infrared image quality evaluation method, the method comprising: acquiring a first infrared face image; inputting the first infrared face image into a trained face quality evaluation model to obtain a quality score of the first infrared face image, wherein the face quality evaluation model is obtained by training an initial evaluation model based on a first sample set, the first sample set including at least one second infrared face image with a quality score label; the initial evaluation model is obtained by training based on visible light face images; and evaluating the quality of the first infrared face image based on the quality score of the first infrared face image.

[0006] In this embodiment, a face quality assessment model trained on infrared face images is used to assess the quality of the first infrared face image to be evaluated. This improves the accuracy of infrared face image quality assessment and allows for the selection of high-quality images that are beneficial for face recognition. Furthermore, infrared face images are minimally affected by changes in illumination, increasing the robustness of the subsequent face recognition system to illumination variations and thus enhancing the accuracy of infrared face image recognition.

[0007] Optionally, the initial evaluation model is obtained by inputting a second sample set into a neural network model, training the neural network model until convergence with the objective of minimizing the loss function between the predicted quality score and the quality score label output by the neural network model, thereby obtaining the initial evaluation model; wherein, the second sample set includes at least one visible light face image with a quality score label.

[0008] Optionally, the initial evaluation model includes a first convolutional network and a second convolutional network. The first convolutional network is used to extract facial detail features, and the second convolutional network is used to learn overall facial features based on the facial detail features and to analyze the overall facial features to obtain a quality score for the input image. The facial quality evaluation model is obtained through the following methods:

[0009] With the network parameters of the first convolutional network fixed, the second convolutional network is trained based on the first sample set with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the initial evaluation model, so as to obtain the network parameters of the second convolutional network.

[0010] The initial evaluation model trained on infrared face images is tested using a third sample set. If the test result does not meet the preset conditions, the second convolutional network is trained using the first sample set until the test result of the initial evaluation model trained on infrared face images based on the third sample set meets the preset conditions and converges, thus obtaining the face quality evaluation model. The third sample set includes at least one third infrared face image with a quality score label.

[0011] Optionally, the loss function includes any of the following: Smooth L1 loss function; L1 loss function; L2 loss function.

[0012] Optionally, the first sample set is obtained in the following way:

[0013] At least one second infrared face image is acquired using an infrared acquisition system;

[0014] Based on at least one preset factor, each of the at least one second infrared face images is scored and labeled with a quality score to obtain the first sample set;

[0015] The third sample set was obtained in the following way:

[0016] At least one third infrared face image is acquired using an infrared acquisition system;

[0017] Based on the at least one preset factor, each of the at least one third infrared face images is scored and labeled with a quality score to obtain the third sample set.

[0018] Optionally, the at least one preset factor includes at least one of the following:

[0019] The shooting angle of the target face corresponding to the infrared face image to be scored;

[0020] The degree of occlusion of the target face;

[0021] The makeup differences of the target face;

[0022] The distance between the target face and the infrared acquisition system.

[0023] Optionally, evaluating the quality of the first infrared face image based on its quality score includes: if the quality score of the first infrared face image is greater than a score threshold, then the first infrared face image is a high-quality image.

[0024] Secondly, embodiments of this application provide an infrared image quality evaluation device, which may include modules / units that execute any of the possible design methods described in the first aspect. These modules / units may be implemented in hardware or by hardware executing corresponding software.

[0025] For example, the apparatus may include an acquisition module and a processing module; wherein:

[0026] The acquisition module is used to acquire a first infrared face image;

[0027] The processing module is used to input the first infrared face image into a trained face quality assessment model to obtain a quality score for the first infrared face image. The face quality assessment model is obtained by training an initial assessment model based on a first sample set, the first sample set including at least one second infrared face image with a quality score label; the initial assessment model is obtained by training a visible light face image; and the quality of the first infrared face image is evaluated based on its quality score.

[0028] Optionally, the processing module is specifically used for:

[0029] The second sample set is input into the neural network model, and the neural network model is trained until convergence with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the neural network model, thus obtaining the initial evaluation model; wherein, the second sample set includes at least one visible light face image with a quality score label.

[0030] Optionally, the initial evaluation model includes a first convolutional network and a second convolutional network. The first convolutional network is used to extract facial detail features, and the second convolutional network is used to learn overall facial features based on the facial detail features and analyze the overall facial features to obtain a quality score of the input image.

[0031] The processing module is specifically used for:

[0032] With the network parameters of the first convolutional network fixed, the second convolutional network is trained based on the first sample set with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the initial evaluation model, so as to obtain the network parameters of the second convolutional network.

[0033] The initial evaluation model trained on infrared face images is tested using a third sample set. If the test result does not meet the preset conditions, the second convolutional network is trained using the first sample set until the test result of the initial evaluation model trained on infrared face images based on the third sample set meets the preset conditions and converges, thus obtaining the face quality evaluation model. The third sample set includes at least one third infrared face image with a quality score label.

[0034] Optionally, the loss function includes any of the following: Smooth L1 loss function; L1 loss function; L2 loss function.

[0035] Optionally, the acquisition module is specifically used for:

[0036] At least one second infrared face image is acquired using an infrared acquisition system;

[0037] The processing module is specifically used to: score each of the at least one second infrared face images according to at least one preset factor, and label each image with a quality score to obtain the first sample set.

[0038] Optionally, the acquisition module is specifically used for:

[0039] At least one third infrared face image is acquired using an infrared acquisition system;

[0040] The processing module is specifically used to: score each of the at least one third infrared face images in the at least one third infrared face image according to the at least one preset factor, and mark the quality score label to obtain the third sample set.

[0041] Optionally, the at least one preset factor includes at least one of the following:

[0042] The shooting angle of the target face corresponding to the infrared face image to be scored;

[0043] The degree of occlusion of the target face;

[0044] The makeup differences of the target face;

[0045] The distance between the target face and the infrared acquisition system.

[0046] Optionally, the processing module is specifically configured to: if the quality score of the first infrared face image is greater than the score threshold, then the first infrared face image is a high-quality image.

[0047] Thirdly, embodiments of this application also provide a computing device, including:

[0048] Memory, used to store program instructions;

[0049] A processor is configured to invoke program instructions stored in the memory and execute the method described in various possible designs of the first aspect, according to the obtained program instructions.

[0050] Fourthly, the application embodiments also provide a computer-readable storage medium storing computer-readable instructions that, when read and executed by a computer, cause the method described in any possible design of the first aspect to be implemented.

[0051] Fifthly, embodiments of this application also provide a computer program product including computer-readable instructions that, when executed by a processor, cause the method described in any possible design of the first aspect to be implemented. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1A and Figure 1B Infrared and visible light face images acquired under the same conditions are input into the same quality assessment model to evaluate the results.

[0054] Figure 2 A flowchart illustrating an infrared image quality evaluation method provided in this application embodiment;

[0055] Figure 3 This is a schematic diagram of the structure of an infrared image quality evaluation device provided in an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of a computing device structure provided in an embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] Compared to visible light images, using infrared face images as input is more conducive to improving the performance of face recognition systems. Current face quality assessment algorithms are mainly designed for visible light images. In one implementation, a face quality assessment algorithm designed for visible light images is used to evaluate the quality of infrared face images.

[0059] The following is combined Figure 1A and Figure 1B This demonstrates the difference in results when infrared and visible light face images, acquired under the same conditions, are input into the same quality assessment algorithm for evaluation. This quality assessment model is a face quality assessment algorithm designed for visible light face images.

[0060] like Figure 1A As shown, the evaluation results obtained from the quality assessment model designed for visible light images are input into visible light face images. The quality scores of visible light face images are mainly distributed between 60 and 83, and the distribution of scores is relatively concentrated.

[0061] like Figure 1B As shown, the evaluation results obtained from the quality assessment model designed for visible light images are infrared face images. The quality scores of infrared face images are concentrated in the range of 20 to 70, and the distribution of scores is relatively scattered.

[0062] visible, Figure 1A and Figure 1B The quality assessment results for infrared face images vary considerably. Directly applying a face quality assessment algorithm designed for visible light images to infrared face image quality assessment results in low accuracy. Therefore, this application provides an infrared image quality assessment method to improve the accuracy of infrared face image quality assessment.

[0063] Figure 2 An exemplary flowchart of an infrared image quality evaluation method provided in an embodiment of this application is shown. This method is applied to an infrared image quality evaluation device. Figure 2 As shown, the method includes:

[0064] Step 201: Obtain the first infrared face image.

[0065] The first infrared face image can be acquired through an infrared acquisition system.

[0066] Step 202: Input the first infrared face image into the trained face quality assessment model to obtain the quality score of the first infrared face image.

[0067] The face quality assessment model is obtained by training an initial assessment model based on a first sample set, which includes at least one second infrared face image with a quality score label. The initial assessment model is obtained by training a visible light face image.

[0068] The following provides an implementation method for obtaining the first sample set.

[0069] At least one second infrared face image is acquired using an infrared acquisition system. For example, the subject walks within a range of 0.3 to 1 meter from the infrared acquisition system, a certain number of infrared videos are captured, and each frame of the infrared video is extracted as the second infrared face image.

[0070] Then, based on at least one preset factor, each of the at least one second infrared face images is scored and labeled with a quality score to obtain the first sample set.

[0071] At least one of the preset factors includes at least one of the following:

[0072] The first item is the shooting angle of the target face corresponding to the infrared face image to be scored.

[0073] Taking the scoring based on the shooting angle of the target face as an example, the highest score is when the subject's face is captured at 0 degrees; the greater the angle at which the face is shifted to the left or right, the lower the score.

[0074] The second factor is the degree of occlusion of the target face.

[0075] Taking the scoring based on the degree of occlusion of the target face as an example, the highest score is given when the subject's face is unobstructed; the larger the occlusion area, the lower the score.

[0076] The third point is the difference in makeup on the target face.

[0077] Taking the scoring based on the differences in the target face's makeup as an example, the highest score is given to those whose faces are clean and free of oil or dirt, which is their true appearance; the larger the area of ​​the face covered by oil or dirt, the lower the score.

[0078] The fourth item is the distance between the target face and the infrared acquisition system.

[0079] Taking the scoring based on the distance between the target face and the infrared acquisition system as an example, the distance at the time of shooting will affect the image clarity; the farther the distance, the lower the score.

[0080] In this embodiment, each second infrared face image can be scored according to at least one of the above-mentioned methods. In specific implementations, when the infrared acquisition system's shooting distance is far, the acquired face images may exhibit blurred details, affecting the neural network feature extraction process. In a preferred embodiment of this application, when scoring each second infrared face image, in addition to scoring based on preset factors such as shooting angle, degree of occlusion, and makeup differences, the influence of distance on the scoring of the second infrared face image is also considered. That is, based on preset factors such as shooting angle, degree of occlusion, makeup differences, and distance, each second infrared face image in at least one second infrared face image is scored and labeled with a quality score, resulting in a first sample set.

[0081] For example, an infrared acquisition system acquires infrared facial videos of subjects within a certain distance range. After acquiring the infrared facial videos, factors such as occlusion, makeup, and other makeup effects are considered to score the quality of different frames of the infrared facial images in the video. Based on the impact of acquisition distance on facial clarity, different attenuation weights are applied to the scoring results to construct a first sample set. In addition to considering factors such as shooting angle, occlusion, and makeup to evaluate the quality of different frames in the infrared facial videos and obtaining scores, attenuating the scores based on shooting distance can effectively improve the sensitivity of the facial quality assessment model to distance.

[0082] In the above-described method of obtaining the first dataset, each of the at least one second infrared face images is scored. This can be done manually based on at least one preset factor, or by using an algorithm to score the second infrared face images.

[0083] The above initial evaluation model can be implemented in any of the following ways.

[0084] Method 1: The initial evaluation model can be a neural network model, such as a Convolutional Neural Network (CNN) model.

[0085] Alternatively, the initial evaluation model can also be obtained by training a neural network model using visible light images. The following section will provide a detailed introduction to the methods for obtaining the initial evaluation model.

[0086] First, a second sample set is obtained, which includes at least one visible light face image with a quality score label.

[0087] Then, the second sample set is input into the neural network model. The neural network model is trained until it converges, with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the neural network model, to obtain the initial evaluation model.

[0088] The loss function can be a Smooth L1 loss function, an L1 loss function, or an L2 loss function; this application does not impose any restrictions on this.

[0089] Based on the initial evaluation model obtained by the above-mentioned method two, the initial evaluation model may further include a first convolutional network and a second convolutional network. The first convolutional network is used to extract facial detail features, and the second convolutional network is used to learn the overall facial features based on the facial detail features and to analyze the quality score of the input image based on the overall facial features.

[0090] The first convolutional network is a shallow network in the initial evaluation model. It has a relatively small receptive field, which allows it to utilize more fine-grained feature information. The overlapping area of ​​the receptive field corresponding to each pixel in the extracted feature map is very small, which ensures that the network can capture more details.

[0091] The second convolutional network is the deep network of the initial evaluation model. As the number of downsampling or convolutions increases, the receptive field gradually increases, and the overlapping area between the receptive fields continues to increase. At this time, the information represented by the pixel is the information of a region, and the feature information between the region or adjacent regions is obtained.

[0092] Therefore, the initial evaluation model, which is pre-trained based on a large number of visible light face images, already has good feature extraction capabilities. When transferred to infrared face recognition, it can achieve quality evaluation of infrared face images captured by the infrared acquisition system. Furthermore, the amount of data required to train the initial evaluation model based on infrared face images is far less than the amount of data required to train the neural network model directly based on infrared face images.

[0093] The following provides a specific method for obtaining a face quality assessment model.

[0094] The first step is to fix the network parameters of the first convolutional network.

[0095] The second step involves training a second convolutional network on the first sample set, with the network parameters of the second convolutional network fixed, aiming to minimize the loss function between the predicted quality score and the quality score label of the initial evaluation model. The loss function can be a Smooth L1 loss function, an L1 loss function, or an L2 loss function; this application does not impose any restrictions on this.

[0096] It should be understood that the initial evaluation model obtained in the second step after training with infrared face images is not necessarily the final face evaluation model. Further testing and verification of the quality evaluation effect are needed. If the test results are good, then the initial evaluation model obtained in the second step after training with infrared face images is the final face quality evaluation model. If the test results are not good, the parameters of the second convolutional network need to be adjusted, and the first sample set needs to be used for training again.

[0097] The third step involves testing the initial evaluation model trained on infrared face images using a third sample set. If the test results do not meet preset conditions, the second convolutional network is trained using the first sample set until the test results of the initial evaluation model trained on infrared face images using the third sample set meet preset conditions, at which point the model converges, resulting in the face quality evaluation model. The third sample set includes at least one third infrared face image with a quality score label. The preset condition is that the average error between the predicted value of the image and the quality score label is less than an error threshold.

[0098] The testing process will be described in detail below.

[0099] After training the network parameters of the second convolutional network, all third infrared face images from the third sample set are input into the initial evaluation model trained with infrared face images. Each third infrared face image corresponds to a predicted value. The error between each predicted value and the quality score label is calculated. In other words, each third infrared face image corresponds to an error. The average error of all third infrared face images in the third sample set can be determined. If the average error is less than the error threshold, the model converges, and the face quality evaluation model is obtained. If the average error is greater than or equal to the error threshold, the parameters of the second convolutional network need to be adjusted. Then, the second and third steps are continued until the test results of the initial evaluation model after training meet the test conditions. Training is then stopped, and the face quality evaluation model is obtained.

[0100] In this embodiment of the application, the method for obtaining the third sample set can refer to the above-described method for obtaining the first sample set, and will not be repeated here.

[0101] Optionally, the proportion of the number of images in the first sample set to the total number of images in the first sample set and the second sample set can be 70% to 80%. For example, the ratio of the number of images in the first sample set to the second sample set can be 7:3, or for example, 8:2.

[0102] In this embodiment of the application, after the face quality assessment model has been trained, the face quality assessment model can be used to directly evaluate the infrared face image to obtain the quality score of the infrared face image.

[0103] Step 203: Evaluate the quality of the first infrared face image based on its quality score.

[0104] If the quality score of the first infrared face image is less than or equal to the score threshold, then the first infrared face image is a low-quality image; if the quality score of the first infrared face image is greater than the score threshold, then the first infrared face image is a high-quality image. The selected high-quality images can then be fed into the face recognition model for face recognition.

[0105] In this application, based on an initial evaluation model pre-trained on visible light images, transfer learning is used to generate a face quality evaluation model that can assess the quality of infrared face images. Since infrared face images are rarely affected by changes in illumination, this increases the robustness of the subsequent face recognition system to changes in illumination, thereby improving the accuracy of infrared face image recognition.

[0106] Furthermore, the infrared face quality assessment model can filter high-quality images that are conducive to face recognition, thus avoiding misidentification caused by factors such as occlusion, non-frontal faces, and long distances. Moreover, after acquiring many images to be recognized, they are first filtered by the face quality assessment model, and only the filtered high-quality images are input into the face recognition model, instead of performing face recognition on all images. This reduces the number of times the recognition model is called, effectively saving the resource overhead of edge devices and improving the performance of the face recognition algorithm to a certain extent.

[0107] Based on the same inventive concept, this application also provides an infrared image quality evaluation device, which is used to implement the method in the above method embodiments.

[0108] like Figure 3 As shown, the device 300 includes an acquisition module 310 and a processing module 320.

[0109] The acquisition module 310 is used to acquire the first infrared face image;

[0110] The processing module 320 is used to input the first infrared face image into a trained face quality assessment model to obtain a quality score of the first infrared face image. The face quality assessment model is obtained by training an initial assessment model based on a first sample set, the first sample set including at least one second infrared face image with a quality score label; the initial assessment model is obtained by training based on visible light face images; and the quality of the first infrared face image is evaluated based on the quality score of the first infrared face image.

[0111] Optionally, the processing module 320 is specifically used for:

[0112] The second sample set is input into the neural network model, and the neural network model is trained until convergence with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the neural network model, thus obtaining the initial evaluation model; wherein, the second sample set includes at least one visible light face image with a quality score label.

[0113] Optionally, the initial evaluation model includes a first convolutional network and a second convolutional network. The first convolutional network is used to extract facial detail features, and the second convolutional network is used to learn overall facial features based on the facial detail features and analyze the overall facial features to obtain a quality score of the input image.

[0114] The processing module 320 is specifically used for:

[0115] With the network parameters of the first convolutional network fixed, the second convolutional network is trained based on the first sample set with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the initial evaluation model, so as to obtain the network parameters of the second convolutional network.

[0116] The initial evaluation model trained on infrared face images is tested using a third sample set. If the test result does not meet the preset conditions, the second convolutional network is trained using the first sample set until the test result of the initial evaluation model trained on infrared face images based on the third sample set meets the preset conditions and converges, thus obtaining the face quality evaluation model. The third sample set includes at least one third infrared face image with a quality score label.

[0117] Optionally, the loss function includes any of the following: Smooth L1 loss function; L1 loss function; L2 loss function.

[0118] Optionally, module 310 is used specifically for:

[0119] At least one second infrared face image is acquired using an infrared acquisition system;

[0120] The processing module 320 is specifically used to: score each of the at least one second infrared face images in the first second infrared face images according to at least one preset factor, and mark the quality score label to obtain the first sample set.

[0121] Optionally, module 310 is used specifically for:

[0122] At least one third infrared face image is acquired using an infrared acquisition system;

[0123] The processing module 320 is specifically used to: score each of the at least one third infrared face images in the at least one third infrared face image according to the at least one preset factor, and mark the quality score label to obtain the third sample set.

[0124] Optionally, the at least one preset factor includes at least one of the following:

[0125] The shooting angle of the target face corresponding to the infrared face image to be scored;

[0126] The degree of occlusion of the target face;

[0127] The makeup differences of the target face;

[0128] The distance between the target face and the infrared acquisition system.

[0129] Optionally, the processing module 320 is specifically used to: if the quality score of the first infrared face image is greater than the score threshold, then the first infrared face image is a high-quality image.

[0130] Based on the same technical concept, embodiments of this application also provide a computing device, such as... Figure 4 As shown, it includes at least one processor 401 and a memory 402 connected to at least one processor. In this embodiment, the specific connection medium between the processor 401 and the memory 402 is not limited. Figure 4 Taking the connection between processor 401 and memory 402 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0131] In this embodiment of the application, the memory 402 stores instructions that can be executed by at least one processor 401. The at least one processor 401 can implement the steps of the above method by executing the instructions stored in the memory 402.

[0132] The processor 401 is the control center of the computer device, capable of connecting various parts of the computer device via various interfaces and lines. It performs resource configuration by running or executing instructions stored in the memory 402 and accessing data stored in the memory 402. Optionally, the processor 401 may include one or more processing units. The processor 401 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401. In some embodiments, the processor 401 and the memory 402 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0133] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0134] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 402 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 402 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0135] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing computer-readable instructions, which, when read and executed by a computer, enable the method in the above-described method embodiments to be implemented.

[0136] Based on the same technical concept, embodiments of this application also provide a computer program product, including computer-readable instructions, which, when executed by a processor, cause the method in the above method embodiments to be implemented.

[0137] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0140] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0141] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for evaluating the quality of infrared face images, characterized in that, The method includes: Acquire the first infrared face image; The first infrared face image is input into the trained face quality assessment model to obtain the quality score of the first infrared face image. The face quality assessment model is obtained by training an initial assessment model based on a first sample set, which includes at least one second infrared face image with a quality score label. The initial assessment model is obtained by training a visible light face image. The quality of the first infrared face image is evaluated based on its quality score. The initial evaluation model includes a first convolutional network and a second convolutional network. The first convolutional network is used to extract facial detail features, and the second convolutional network is used to learn the overall facial features based on the facial detail features and to analyze the overall facial features to obtain the quality score of the input image. The face quality assessment model was trained in the following way: The neural network model is trained based on the second sample set to obtain the initial evaluation model; the second sample set includes at least one visible light face image with a quality score label. Based on fixing the network parameters of the first convolutional network in the initial evaluation model, with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the initial evaluation model, the second convolutional network is trained on the first sample set to obtain the network parameters of the second convolutional network. The initial evaluation model trained on infrared face images is tested using a third sample set. If the test result does not meet the preset conditions, the second convolutional network is trained using the first sample set until the test result of the initial evaluation model trained on infrared face images based on the third sample set meets the preset conditions and converges, thus obtaining the face quality evaluation model. The third sample set includes at least one third infrared face image with a quality score label.

2. The method according to claim 1, characterized in that, The initial evaluation model was trained in the following manner: The second sample set is input into the neural network model, and the neural network model is trained until it converges with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the neural network model, thus obtaining the initial evaluation model.

3. The method according to claim 1 or 2, characterized in that, The loss function includes any of the following: Smooth L1 loss function; L1 loss function; L2 loss function.

4. The method according to claim 1, characterized in that, The first sample set was obtained in the following way: At least one second infrared face image is acquired using an infrared acquisition system; Based on at least one preset factor, each of the at least one second infrared face images is scored and labeled with a quality score to obtain the first sample set; The third sample set was obtained in the following way: At least one third infrared face image is acquired using an infrared acquisition system; Based on the at least one preset factor, each of the at least one third infrared face images is scored and labeled with a quality score to obtain the third sample set.

5. The method according to claim 4, characterized in that, The at least one preset factor includes at least one of the following: The shooting angle of the target face corresponding to the infrared face image to be scored; The degree of occlusion of the target face; The makeup differences of the target face; The distance between the target face and the infrared acquisition system.

6. The method according to claim 1, characterized in that, The step of evaluating the quality of the first infrared face image based on its quality score includes: If the quality score of the first infrared face image is greater than the score threshold, then the first infrared face image is a high-quality image.

7. An infrared face image quality evaluation device, characterized in that, The device includes: The acquisition module is used to acquire the first infrared face image; The processing module is used to input the first infrared face image into a trained face quality assessment model to obtain a quality score for the first infrared face image. The face quality assessment model is obtained by training an initial assessment model based on a first sample set, which includes at least one second infrared face image with a quality score label. The initial assessment model is trained based on visible light face images. The quality of the first infrared face image is evaluated based on its quality score. The initial evaluation model includes a first convolutional network and a second convolutional network. The first convolutional network is used to extract facial detail features, and the second convolutional network is used to learn overall facial features based on the facial detail features and to analyze the overall facial features to obtain a quality score for the input image. The processing module is specifically used for: The neural network model is trained based on the second sample set to obtain the initial evaluation model; the second sample set includes at least one visible light face image with a quality score label. Based on fixing the network parameters of the first convolutional network in the initial evaluation model, with the goal of minimizing the loss function between the predicted quality score and the quality score label output by the initial evaluation model, the second convolutional network is trained on the first sample set to obtain the network parameters of the second convolutional network. The initial evaluation model trained on infrared face images is tested using a third sample set. If the test result does not meet the preset conditions, the second convolutional network is trained using the first sample set until the test result of the initial evaluation model trained on infrared face images based on the third sample set meets the preset conditions and converges, thus obtaining the face quality evaluation model. The third sample set includes at least one third infrared face image with a quality score label.

8. A computing device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 6 according to the obtained program instructions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method according to any one of claims 1 to 6.

Images