A method and apparatus for evaluating the quality of infrared face images

Abstract

This invention discloses a method and apparatus for evaluating the quality of infrared face images, comprising: performing face detection on an input infrared face image to obtain facial key points of the infrared face image; aligning the facial key points of the infrared face image according to preset standard facial key points to obtain a perspective transformation matrix of the infrared face image and an aligned first face image; extracting pixels corresponding to the first face image in the infrared face image sequentially according to the perspective transformation matrix and performing Gaussian processing; replacing the corresponding pixels of the first face image with the pixels of the Gaussian-processed infrared face image to obtain a Gaussian-processed second face image; performing multi-dimensional quality evaluation on the infrared face image; and determining the quality evaluation result of the infrared face image based on the multi-dimensional quality evaluation result. This invention optimizes the face alignment and Gaussian denoising processing methods in the face image quality evaluation process, avoiding waste of computational resources.

Description

Technical Field

[0001] This invention relates to the field of digital image processing, and in particular to a method and apparatus for evaluating the quality of infrared face images. Background Technology

[0002] In existing access control and lock systems, facial recognition technology is frequently used. After acquiring a facial image, the image needs to be cropped to obtain the facial region for subsequent anti-spoofing and feature detection. The image quality of the acquired facial region image plays a crucial role in the accuracy of subsequent anti-spoofing and recognition. Before evaluating the facial image quality, Gaussian denoising and face alignment are required for the input facial image. Face alignment refers to aligning the facial image with a preset facial model so that the program can identify the location of key facial points in the facial image.

[0003] In existing technologies, face alignment and Gaussian denoising of infrared face images are performed in a step-by-step manner. First, the unaligned face image is subjected to full-image Gaussian denoising to obtain the Gaussian-denoised face image. Then, a perspective transformation matrix is ​​used to obtain the face-aligned image of the denoised face image. The drawback of this approach is that Gaussian denoising is performed on the entire unaligned face image, and the final face-aligned image does not contain all the pixels of the Gaussian-denoised face image, resulting in a waste of computational resources. Summary of the Invention

[0004] This invention provides a method and apparatus for evaluating the quality of infrared face images, which can effectively solve the problem in the prior art where Gaussian denoising is performed on the entire misaligned face image, but the resulting aligned face image does not contain all the pixels of the Gaussian denoised face image, resulting in a waste of computing resources.

[0005] An embodiment of the present invention provides a method for evaluating the quality of infrared face images, comprising:

[0006] Acquire infrared facial images;

[0007] Face detection is performed on the infrared face image to obtain the facial key points of the infrared face image;

[0008] Based on preset standard facial key points, the facial key points of the infrared face image are aligned to obtain the perspective transformation matrix of the infrared face image and the aligned first face image.

[0009] Based on the perspective transformation matrix, the pixels corresponding to the first face image in the infrared face image are extracted sequentially, and Gaussian processing is performed.

[0010] Replace the corresponding pixels of the first face image with the pixels of the infrared face image after Gaussian processing to obtain the second face image after Gaussian processing.

[0011] The infrared facial images are used to assess facial integrity and facial size.

[0012] Based on the face integrity assessment result, it is determined that the face is complete, and based on the face size assessment result, it is determined that the face size meets the preset size requirements. Then, image clarity assessment, image brightness assessment, and image contrast assessment are performed based on the second face image.

[0013] The quality assessment result of the infrared face image is determined based on any one or more of the following: face integrity assessment result, face size assessment result, image sharpness assessment result, image brightness assessment result, and image contrast assessment result.

[0014] Furthermore, a method for evaluating the quality of an infrared face image, wherein face detection is performed on the infrared face image to obtain facial key points of the infrared face image, includes:

[0015] Determine the coordinates, width, and height of the face region in the infrared face image to define the face region;

[0016] The positions of the eyes, bridge of the nose, and corners of the mouth are detected within the face area to generate facial key points of the infrared face image.

[0017] Furthermore, a method for evaluating the quality of infrared face images, wherein the assessment of face integrity based on the infrared face image includes:

[0018] The boundary of the face region is determined based on the coordinates, width, and height of the face region in the infrared face image.

[0019] If the distance between any boundary of the face region and the boundary of the infrared face image is less than a preset first threshold, the face integrity assessment result is determined to be unqualified.

[0020] If the distance between the boundary of the face region and the boundary of the infrared face image is not less than a preset first threshold, then a first preset score is generated, and the first preset score is used as the face integrity evaluation result.

[0021] Furthermore, a method for evaluating the quality of an infrared face image, wherein the face size assessment based on the infrared face image includes:

[0022] Based on the position of the eyes in the face region, the pixel distance between the left and right eyes is determined as the face size in the face region;

[0023] If the face size is smaller than a preset second threshold, the face size assessment result is determined to be unqualified.

[0024] If the face size is not less than a preset second threshold, a second preset score is generated, and the second preset score is used as the face size evaluation result.

[0025] Furthermore, a method for evaluating the quality of an infrared face image, wherein when the face is determined to be complete based on the face integrity assessment result, and the face size is determined to meet a preset size requirement based on the face size assessment result, image sharpness assessment, image brightness assessment, and image contrast assessment are performed based on the second face image, including:

[0026] When the face integrity assessment result is determined to be neither incomplete nor the face size assessment result to be unqualified, the image sharpness of the second face image is calculated using the following formula:

[0027]

[0028] The image contrast of the second face image is calculated using the following formula:

[0029]

[0030]

[0031] The image brightness of the second face image is calculated using the following formula:

[0032] Luma = LumaStd + LumaMean

[0033] Where i and j are the x and y coordinates of a pixel in the second face image, respectively; P(i,j) is the pixel value of the pixel at position (i,j) in the second face image; w and h are the width and height of the second face image, respectively; Definition is the image sharpness evaluation result of the infrared face image; LumaStd is the image contrast evaluation result of the infrared face image; and Luma is the image brightness evaluation result of the infrared face image.

[0034] Furthermore, the step of generating a quality assessment result for the infrared face image based on the face integrity assessment result, face size assessment result, image sharpness assessment result, image brightness assessment result, and image contrast assessment result includes:

[0035] If the face integrity assessment result is determined to be incomplete or the face size assessment result is determined to be unqualified, the quality assessment result of the infrared face image is determined to be unqualified.

[0036] When the face integrity assessment result is determined to be neither incomplete nor unqualified, and based on the image sharpness assessment result, image brightness assessment result, and image contrast assessment result, the third score is calculated using the following formula:

[0037]

[0038] The first preset score, the second preset score, and the third score are added together to obtain the quality assessment score of the infrared face image;

[0039] If the quality assessment score of the infrared face image is greater than the preset third threshold, the quality assessment result of the infrared face image is determined to be qualified.

[0040] If the quality assessment score of the infrared face image is not greater than a preset third threshold, the quality assessment result of the infrared face image is determined to be unqualified.

[0041] Furthermore, a method for evaluating the quality of an infrared face image, before adding the first preset score, the second preset score, and the third score to obtain the quality assessment score of the infrared face image, further includes:

[0042] When the third score is determined to be less than the preset fourth threshold, the face image enhancement operation is repeated until the third score is not less than the preset fourth threshold, or the quality assessment result of the external face image is determined to be unqualified.

[0043] The face image enhancement operation includes:

[0044] Determine whether the second face image is undergoing face image enhancement for the first time;

[0045] If not, the quality assessment result of the infrared face image corresponding to the second face image is determined to be unqualified;

[0046] If so, the second face image is input into a preset face anti-spoofing model so that the face anti-spoofing model can determine whether the second face image is a real face image;

[0047] If it is determined that the second face image is not a real face image, then the quality assessment result of the infrared face image corresponding to the second face image is determined to be unqualified;

[0048] If it is determined that the second face image is a real face image, then the second face image is input into a preset face enhancement model so that the face enhancement model can optimize the clarity, brightness and contrast of the second face image;

[0049] The optimized second face image is then subjected to image sharpness, image brightness, and image contrast evaluations again. Based on the results of the image sharpness evaluation, image brightness evaluation, and image contrast evaluation, the third score is recalculated.

[0050] Furthermore, in a method for evaluating the quality of infrared face images, both the face anti-spoofing model and the face enhancement model are constructed based on lightweight neural networks.

[0051] Another embodiment of the present invention provides an apparatus for evaluating the quality of infrared face images, comprising:

[0052] Infrared face acquisition module, used to acquire infrared face images;

[0053] An infrared face detection module is used to perform face detection on the infrared face image and obtain the facial key points of the infrared face image.

[0054] An infrared face alignment and denoising module is used to align the facial key points of the infrared face image according to preset standard facial key points to obtain the perspective transformation matrix of the infrared face image and the aligned first face image; according to the perspective transformation matrix, the corresponding pixels in the infrared face image are extracted sequentially and Gaussian processed; the pixels of the Gaussian processed infrared face image are used to replace the corresponding pixels of the first face image to obtain the Gaussian processed second face image;

[0055] The infrared face evaluation module is used to evaluate the face completeness and face size based on the infrared face image; if the face completeness is determined to be complete based on the face completeness evaluation result, and the face size meets the preset size requirements based on the face size evaluation result, then the module performs image clarity, image brightness, and image contrast evaluation based on the second face image; and determines the quality evaluation result of the infrared face image based on any one or more combinations of the face completeness evaluation result, face size evaluation result, image clarity evaluation result, image brightness evaluation result, and image contrast evaluation result.

[0056] Furthermore, in an infrared face image quality evaluation device, the infrared face evaluation module further includes:

[0057] The infrared face anti-counterfeiting submodule is used to determine whether the second face image is a real face;

[0058] The infrared face enhancement submodule is used to optimize the clarity, brightness, and contrast of the second face image.

[0059] The following benefits can be obtained by implementing the present invention:

[0060] This invention first aligns the input infrared face image, then extracts the corresponding pixels from the face image to the face-aligned image based on a perspective transformation matrix, performs Gaussian denoising, and replaces the corresponding pixels in the face-aligned image with the denoised pixels. After traversing all face-aligned images, the face alignment and Gaussian denoising of the input face image are completed. This invention optimizes the face alignment and Gaussian denoising methods, performing Gaussian denoising only on the face region of the infrared face image, thus not affecting the accuracy of the infrared face image quality assessment and avoiding wasted computational resources. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating a method for evaluating the quality of infrared face images according to an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of the structure of an infrared face image quality evaluation device provided in an embodiment of the present invention.

[0063] Figure 3 This is a schematic diagram of an infrared face image quality evaluation device provided in another embodiment of the present invention.

[0064] Figure 4 This is a schematic diagram of the process of Gaussian denoising of a first face image provided by an embodiment of the present invention.

[0065] Figure 5 This is a real-time example of the present invention, which provides several infrared face images of different qualities. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] See Figure 1 This is a flowchart illustrating a method for evaluating the quality of infrared face images according to an embodiment of the present invention, comprising:

[0068] S1. Acquire infrared face image;

[0069] In one embodiment of the present invention, an infrared facial image of a pedestrian is acquired by an access control device equipped with an infrared camera.

[0070] S2. Perform face detection on the infrared face image to obtain the facial key points of the infrared face image;

[0071] In one embodiment of the present invention, the infrared face image obtained by the infrared camera contains other irrelevant factors besides the face. Before performing infrared face image detection, it is necessary to determine the face region in the image and the key point positions of the face region.

[0072] Preferably, the coordinates, width, and height of the face region in the infrared face image are determined to define the face region; the positions of the eyes, bridge of the nose, and corners of the mouth are detected within the face region to generate the facial key points of the infrared face image.

[0073] It needs to be clarified that face detection is performed on the infrared face image to determine the range of the face region, and the detection result is box{x,y,w,h}; where x and y are the coordinates of the upper left corner of the face region, and w and h are the width and height of the face region; within the face region, the coordinates of the left eye, right eye, bridge of the nose, left corner of the mouth, and right corner of the mouth are detected, and the detection result is recorded as {l j |(x j ,y j ),j=1~5}, are the facial key points of the infrared face image.

[0074] S3. According to the preset standard facial key points, align the facial key points of the infrared face image to obtain the perspective transformation matrix of the infrared face image and the aligned first face image.

[0075] In one embodiment of the present invention, the perspective transformation matrix of the infrared face image is obtained using the following formula:

[0076] PT=GetPt(landmarks5_detect,landmarks5_align)

[0077] Where PT is the perspective transformation matrix; GetPT is the method for solving the perspective transformation equation; landmarks5_detect are the coordinates of the left eye, right eye, bridge of the nose, left corner of the mouth, and right corner of the mouth detected in the face region of the infrared face image; and landmarks5_align are the coordinates of the left eye, right eye, bridge of the nose, left corner of the mouth, and right corner of the mouth of the preset standard aligned face.

[0078] The face region of the infrared face image is aligned according to the perspective transformation matrix to obtain the aligned first face image.

[0079] S4. According to the perspective transformation matrix, extract the pixels corresponding to the first face image in the infrared face image in sequence, and perform Gaussian processing; replace the pixels corresponding to the first face image with the pixels of the infrared face image after Gaussian processing to obtain the second face image after Gaussian processing.

[0080] See Figure 4 The diagram below illustrates a process for Gaussian denoising of a first face image according to an embodiment of the present invention. The process includes: traversing the first face image and sequentially obtaining the pixel coordinates (i,j) of the first face image; determining the pixel coordinates (i2,j2) in the infrared face image corresponding to the pixel coordinates (i,j) of the first face image through perspective transformation; determining whether the coordinates (i2,j2) exist in the Gaussian point set; if so, continuing to traverse the first face image and obtaining the next pixel coordinate of the first face image; if not, obtaining the pixel with coordinates (i2,j2) in the infrared face image, performing Gaussian processing, and replacing the pixel with coordinates (i,j) in the first face image with the processed pixel, while simultaneously recording the coordinates (i2,j2) in the Gaussian point set.

[0081] It should be noted that the following formula is used to perform Gaussian processing on the pixels.

[0082]

[0083] Where i and j are the x and y coordinates of the pixels in the infrared face image, respectively, and P (i,j) Let (i,j) be the pixel value of the pixel at position (i,j) in the infrared face image; Gauss(i,j) is the pixel value of the pixel at position (i,j) in the infrared face image after Gaussian processing; Gausskernel 3x3 This is the Gaussian kernel function.

[0084] By implementing this invention, the infrared face image is first aligned, and then Gaussian denoising is performed on the face region of the infrared face image based on the aligned image. This optimizes the existing processing method of first performing full-image Gaussian denoising on the face image and then performing face alignment. This invention only performs Gaussian denoising on the face region of the infrared face image, which does not affect the accuracy of the infrared face image quality assessment and avoids the waste of computing resources.

[0085] S5. Based on the infrared face image, perform face integrity assessment and face size assessment; based on the face integrity assessment result, determine that the face is complete, and based on the face size assessment result, determine that the face size meets the preset size requirements, then perform image clarity assessment, image brightness assessment, and image contrast assessment based on the second face image.

[0086] Preferably, the facial integrity assessment based on the infrared facial image includes: determining the boundary of the facial region based on the coordinate position, width, and height of the facial region in the infrared facial image; if the distance between any boundary of the facial region and the boundary of the infrared facial image is less than a preset first threshold, the facial integrity assessment result is determined to be unqualified; if the distance between the boundary of the facial region and the boundary of the infrared facial image is not less than the preset first threshold, a first preset score is generated, and the first preset score is used as the facial integrity assessment result.

[0087] In one embodiment of the present invention, the detection result of face detection in the infrared face image is box{x,y,w,h}; where x and y are the coordinates of the upper left corner of the face region, and w and h are the width and height of the face region. Based on box{x,y,w,h}, a rectangular face region and its boundary can be determined in the infrared face image. If the distance between any boundary of the rectangular face region and the boundary of the infrared face image is less than a preset first threshold, the face integrity assessment result is determined to be unqualified. If the distance between the boundary of the rectangular face region and the boundary of the infrared face image is not less than the preset first threshold, the face in the infrared face image is determined to be complete, and a first preset score is generated, which is then used as the face integrity assessment result. The first threshold and the first preset score are adjusted according to the specific infrared face image capturing device and its parameters.

[0088] Preferably, the facial size assessment based on the infrared facial image includes: determining the pixel distance between the left and right eyes as the facial size in the facial region based on the position of the eyes in the facial region; if the facial size is less than a preset second threshold, the facial size assessment result is determined to be unqualified; if the facial size is not less than the preset second threshold, a second preset score is generated and the second preset score is used as the facial size assessment result.

[0089] In one embodiment of the present invention, during face detection, the coordinates of the left eye, right eye, bridge of the nose, left corner of the mouth, and right corner of the mouth are generated. The coordinates of the left and right eyes are taken, and the pixel distance between them is calculated as a standard for measuring the face size in the infrared face image. A preset face size threshold is compared with the pixel distance between the left and right eyes. If the pixel distance between the left and right eyes is less than a preset second threshold, it is determined that the face size does not meet the preset size requirement; otherwise, it is determined that the face size meets the preset size requirement, and a preset second score is generated as the face size evaluation result. The second threshold and the second preset score are adjusted according to the specific infrared face image capturing device and its parameters.

[0090] Preferably, the image sharpness assessment, image brightness assessment, and image contrast assessment based on the second face image include:

[0091] Based on the face integrity assessment result, if the face is determined to be complete, and based on the face size assessment result, if the face size meets the preset size requirements, the image clarity of the second face image is calculated using the following formula:

[0092]

[0093] The image contrast of the second face image is calculated using the following formula:

[0094]

[0095]

[0096] The image brightness of the second face image is calculated using the following formula:

[0097] Luma = LumaStd + LumaMean

[0098] Where i and j are the x and y coordinates of a pixel in the second face image, respectively; P(i,j) is the pixel value of the pixel at position (i,j) in the second face image; w and h are the width and height of the second face image, respectively; Definition is the image sharpness evaluation result of the infrared face image; LumaStd is the image contrast evaluation result of the infrared face image; and Luma is the image brightness evaluation result of the infrared face image.

[0099] In one embodiment of the present invention, if the face integrity assessment result is a first preset score, then the face in the infrared face image is confirmed to be complete; if the face size assessment result is a second preset score, then the face size in the infrared face image is confirmed to meet the preset size requirements. When the face integrity assessment result is the first preset score and the face size assessment result is the second preset score, a second face image of the infrared face image is acquired, and its image clarity, brightness, and contrast are assessed.

[0100] S6. Determine the quality assessment result of the infrared face image based on any one or more of the following: face integrity assessment result, face size assessment result, image clarity assessment result, image brightness assessment result, and image contrast assessment result.

[0101] Preferably, when the result of the face integrity assessment is determined to be incomplete or the face size assessment result is determined to be unqualified, the quality assessment result of the infrared face image is determined to be unqualified.

[0102] When the face integrity assessment result is determined to be neither incomplete nor unqualified, and based on the image sharpness assessment result, image brightness assessment result, and image contrast assessment result, the third score is calculated using the following formula:

[0103]

[0104] The first preset score, the second preset score, and the third score are added together to obtain the quality assessment score of the infrared face image. If the quality assessment score of the infrared face image is greater than the preset third threshold, the quality assessment result of the infrared face image is determined to be qualified. If the quality assessment score of the infrared face image is not greater than the preset third threshold, the quality assessment result of the infrared face image is determined to be unqualified.

[0105] In one embodiment of the present invention, if the result of the face integrity assessment is unqualified or the result of the face size assessment is unqualified, then the subsequent image clarity, brightness and contrast assessment of the infrared face image will not be performed, and the assessment result of the infrared face image will be directly determined to be unqualified.

[0106] If the face integrity assessment result is a first preset score and the face size assessment result is a second preset score, the infrared face image is evaluated for image clarity, image brightness, and image contrast, and a third score is calculated according to a preset formula. It should be noted that the third score is between 0 and 100; a higher third score indicates better clarity, brightness, and contrast quality of the infrared face image.

[0107] Finally, the sum of the first preset score, the second preset score, and the third score is used as the quality evaluation score of the infrared face image, and it is determined whether the quality evaluation score is greater than a preset third threshold. If the quality evaluation score of the infrared face image is greater than the preset third threshold, it is determined that the quality evaluation result of the infrared face image is qualified; if the quality evaluation score of the infrared face image is not greater than the preset third threshold, it is determined that the quality evaluation result of the infrared face image is unqualified. Among them, the third threshold is adjusted according to the specific infrared face image capture device and device parameters.

[0108] The present invention improves the accuracy of quality evaluation of infrared face images by performing multi-dimensional evaluations of face integrity, face size, image clarity, image brightness, and image contrast on infrared face images, providing an accurate basis for subsequent work such as whether infrared face images can be used for face recognition and face anti-counterfeiting.

[0109] Preferably, before adding the first preset score, the second preset score, and the third score to obtain the quality evaluation score of the infrared face image, it further includes:

[0110] When it is determined that the third score is less than a preset fourth threshold, the face image enhancement operation is repeatedly executed until the third score is not less than the preset fourth threshold, or it is determined that the quality evaluation result of the infrared face image is unqualified;

[0111] The face image enhancement operation includes:

[0112] Determine whether the second face image is performing the face image enhancement operation for the first time;

[0113] If not, it is determined that the quality evaluation result of the infrared face image corresponding to the second face image is unqualified;

[0114] If so, input the second face image into a preset face anti-counterfeiting model so that the face anti-counterfeiting model determines whether the second face image is a real face image;

[0115] If it is determined that the second face image is not a real face image, it is determined that the quality evaluation result of the infrared face image corresponding to the second face image is unqualified;

[0116] If it is determined that the second face image is a real face image, input the second face image into a preset face enhancement model so that the face enhancement model optimizes the clarity, brightness, and contrast of the second face image;

[0117] The optimized second face image is then subjected to image sharpness, image brightness, and image contrast evaluations again. Based on the results of the image sharpness evaluation, image brightness evaluation, and image contrast evaluation, the third score is recalculated.

[0118] In one embodiment of the present invention, a third score is calculated based on the image sharpness evaluation result, image brightness evaluation result, and image contrast evaluation result. When the third score is less than a fourth threshold, it is determined whether the second face image has undergone face image enhancement operation. If so, the quality evaluation result of the infrared face image corresponding to the second face image is determined to be unqualified. If not, the second face image is input into a preset face anti-spoofing model. When the face anti-spoofing model determines whether the second face image is a real face image, if it determines that the second face image is not a real face image, the quality evaluation result of the infrared face image corresponding to the second face image is determined to be unqualified; if it determines that the second face image is a real face image, the second face image is input into a preset face enhancement model, so that the face enhancement model optimizes the sharpness, brightness, and contrast of the second face image. After optimization, the image sharpness evaluation, image brightness evaluation, and image contrast evaluation are performed on the second face image again, and the third score is recalculated. If the third score of the optimized second face image is not greater than the fourth threshold, the quality assessment result of the infrared face image corresponding to the second face image is determined to be unqualified.

[0119] To better illustrate the above scheme, the following provides a detailed explanation of the preset face anti-spoofing model and the preset face enhancement model;

[0120] Both the face anti-spoofing model and the face enhancement model are built based on lightweight neural networks.

[0121] The construction and training process of the aforementioned face anti-spoofing model includes:

[0122] Several background images without faces, real face images, 2D face images, and 3D face images were collected and labeled as 0, 1, 2, and 3 respectively to obtain a face anti-spoofing dataset.

[0123] The face anti-spoofing model is constructed based on the lightweight neural network MobieNetV2 and trained using the face anti-spoofing dataset. The model's output is compared with the annotation, the model loss is calculated, and the model parameters are updated according to the loss function calculation results. This process is iterated until the model converges, resulting in a well-trained face anti-spoofing model.

[0124] The construction and training process of the face enhancement model includes:

[0125] See Figure 5 This invention provides a real-time example of acquiring several infrared facial images of varying quality. The process involves: writing a data acquisition script to randomly adjust the camera module's built-in parameters, such as exposure and flash frequency, as well as changing the external environment (turning lights on / off, adjusting light intensity, and applying strong light from different angles), thereby capturing several infrared facial images of the same subject with varying quality. The number of subjects is sufficient, and the data distribution in terms of gender, age, and ethnicity is even.

[0126] After face alignment and Gaussian denoising of the aforementioned infrared face images, image sharpness, brightness, and contrast are evaluated, and a third score is calculated based on the evaluation results. According to the third score, infrared face images from the same data collector are grouped into two quality groups: high quality and low quality.

[0127] In one embodiment of the present invention, a face enhancement model based on the lightweight neural network MobieNetV2 is constructed, and its model structure is shown in the table below.

[0128]

[0129]

[0130] During training, taking bath_size=2 as an example, if each batch of data is not shuffled, then the data sampled in the first batch is:

[0131]

[0132] Where forward_img is the model input image, and loos_img is a high-quality image used to calculate the loss value with the model output image.

[0133] During training, the optimized image output by the model is compared with the loos_img image using the following loss function to calculate the quality loss of the model output image:

[0134]

[0135] The model output is a tensor of size h×w×c, which can be converted into an image format to obtain a quality-enhanced image. LP(i,j) is the pixel value at position (i,j) in the loos_img image, and OP(i,j) is the pixel value at position (i,j) in the model output image.

[0136] The above process is iterated continuously until the model converges, resulting in a well-trained face enhancement model.

[0137] This invention addresses the problem of poor image stability in face detection caused by various factors such as external lighting, face viewing angle, and the operation of the infrared module flash, by performing a face image optimization operation on a second face image that does not meet a preset fourth threshold. Without altering the original facial features, it enhances the clarity, brightness, and contrast of the infrared face image, optimizes the face image quality, and improves the accuracy of subsequent face detection and the utilization rate of the infrared face image.

[0138] See Figure 2 The diagram below illustrates the structure of an infrared face image quality evaluation device according to an embodiment of the present invention, comprising:

[0139] Infrared face acquisition module, used to acquire infrared face images;

[0140] An infrared face detection module is used to perform face detection on the infrared face image and obtain the facial key points of the infrared face image.

[0141] An infrared face alignment and denoising module is used to align the facial key points of the infrared face image according to preset standard facial key points to obtain the perspective transformation matrix of the infrared face image and the aligned first face image; according to the perspective transformation matrix, the corresponding pixels in the infrared face image are extracted sequentially and Gaussian processed; the pixels of the Gaussian processed infrared face image are used to replace the corresponding pixels of the first face image to obtain the Gaussian processed second face image;

[0142] The infrared face evaluation module is used to evaluate the face completeness and face size based on the infrared face image; if the face completeness is determined to be complete based on the face completeness evaluation result, and the face size meets the preset size requirements based on the face size evaluation result, then the module performs image clarity, image brightness, and image contrast evaluation based on the second face image; and determines the quality evaluation result of the infrared face image based on any one or more combinations of the face completeness evaluation result, face size evaluation result, image clarity evaluation result, image brightness evaluation result, and image contrast evaluation result.

[0143] Preferred, see Figure 3 The diagram below illustrates the structure of an infrared face image quality evaluation device according to another embodiment of the present invention, which further includes:

[0144] The infrared face anti-counterfeiting submodule is used to determine whether the second face image is a real face;

[0145] The infrared face enhancement submodule is used to optimize the clarity, brightness, and contrast of the second face image.

[0146] An embodiment of this invention provides an apparatus for assessing and optimizing the quality of fundus images. The apparatus first aligns the input infrared face image, then extracts pixels from the face image corresponding to the aligned image based on a perspective transformation matrix, performs Gaussian denoising, and replaces the corresponding pixels in the aligned image with the denoised Gaussian pixels. After traversing all aligned images, the face alignment and Gaussian denoising of the input face image are completed. This invention optimizes the face alignment and Gaussian denoising methods, performing Gaussian denoising only on the face region of the infrared face image, thus maintaining the accuracy of the infrared face image quality assessment and avoiding wasted computational resources.

[0147] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0148] Those skilled in the art will clearly understand that, for convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0149] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0150] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0151] The memory can be used to store the computer program. The processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0152] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0153] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for evaluating quality of an infrared face image, characterized in that, The method comprises the following steps: obtaining an infrared face image; performing face detection on the infrared face image to obtain face key points of the infrared face image; aligning the face key points of the infrared face image according to a preset standard face key point to obtain a perspective transformation matrix of the infrared face image, and aligning a face region of the infrared face image according to the perspective transformation matrix to obtain a first face image after alignment; traversing the first face image to sequentially obtain a first pixel point coordinate of the first face image, determining a second pixel point coordinate corresponding to the first pixel point coordinate of the first face image in the infrared face image through the perspective transformation matrix, judging whether the second pixel point coordinate exists in a Gaussian point set, if yes, continuing to traverse the first face image to obtain a next first pixel point coordinate of the first face image, if not, performing Gaussian processing on a first pixel point of the second pixel point coordinate in the infrared face image, replacing the second pixel point of the first pixel point coordinate in the first face image with the first pixel point after Gaussian processing, and recording the second pixel point coordinate in the Gaussian point set, and obtaining a second face image after Gaussian processing when the traversal is completed; performing face completeness evaluation and face size evaluation on the infrared face image; when it is determined that the face is complete according to the face completeness evaluation result and the size of the face meets the preset size requirement according to the face size evaluation result, performing image definition evaluation, image brightness evaluation and image contrast evaluation on the second face image; determining a quality evaluation result of the infrared face image according to any one or more combinations of the face completeness evaluation result, the face size evaluation result, the image definition evaluation result, the image brightness evaluation result and the image contrast evaluation result.

2. The method for evaluating the quality of an infrared face image according to claim 1, wherein, The face detection on the infrared face image to obtain the face key points of the infrared face image comprises the following steps: determining the coordinate position, width and height of the face region in the infrared face image to determine the face region; detecting the positions of the eyes, the nose bridge and the corners of the mouth in the face region to generate the face key points of the infrared face image.

3. The method of claim 2, wherein the method is used for evaluating the quality of an infrared face image. The face completeness evaluation on the infrared face image comprises the following steps: determining the boundaries of the face region according to the coordinate position, width and height of the face region in the infrared face image; if the distance between any boundary of the face region and the boundary of the infrared face image is less than a preset first threshold value, determining that the face completeness evaluation result is unqualified; if the distance between the boundaries of the face region and the boundary of the infrared face image is not less than the preset first threshold value, generating a first preset score, and taking the first preset score as the face completeness evaluation result.

4. The method for evaluating the quality of an infrared face image according to claim 3, wherein, The face size evaluation on the infrared face image comprises the following steps: determining the pixel distance between the left eye and the right eye as the face size in the face region according to the positions of the eyes in the face region. If the face size is smaller than a preset second threshold, it is determined that the face size evaluation result is unqualified; If the face size is not smaller than the preset second threshold, a second preset score is generated, and the second preset score is taken as the face size evaluation result.

5. The method of claim 4, wherein the method further comprises: When it is determined that the face is complete according to the face completeness evaluation result, and the size of the face meets the preset size requirement according to the face size evaluation result, image definition evaluation, image brightness evaluation and image contrast evaluation are performed on the second face image, including: When it is determined that the face is complete according to the face completeness evaluation result, and the size of the face meets the preset size requirement according to the face size evaluation result, the image definition of the second face image is calculated by using the following formula: The image contrast of the second face image is calculated by using the following formula: The image brightness of the second face image is calculated by using the following formula: wherein i, j are horizontal and vertical coordinates of a pixel in the second face image respectively, P （i,j） is a pixel value of a pixel at position (i, j) in the second face image; w, h are width and height of the second face image respectively; Definition is an image definition evaluation result of the infrared face image; LumaStd is an image contrast evaluation result of the infrared face image; Luma is an image brightness evaluation result of the infrared face image.

6. The method of claim 5, wherein, The quality evaluation result of the infrared face image is generated according to any one or more combinations of the face completeness evaluation result, the face size evaluation result, the image definition evaluation result, the image brightness evaluation result and the image contrast evaluation result, including: When it is determined that the face completeness evaluation result is uncomplete or the face size evaluation result is unqualified, it is determined that the quality evaluation result of the infrared face image is unqualified; When it is determined that the face completeness evaluation result is not uncomplete, and the face size evaluation result is not unqualified, a third score is calculated by using the following formula according to the image definition evaluation result, the image brightness evaluation result and the image contrast evaluation result: The first preset score, the second preset score and the third score are added to obtain the quality evaluation score of the infrared face image; If the quality evaluation score of the infrared face image is greater than a preset third threshold, it is determined that the quality evaluation result of the infrared face image is qualified; If the quality evaluation score of the infrared face image is not greater than the preset third threshold, it is determined that the quality evaluation result of the infrared face image is unqualified.

7. The method of claim 6, wherein the method further comprises: Before the first preset score, the second preset score and the third score are added to obtain the quality evaluation score of the infrared face image, it further includes: When it is determined that the third score is smaller than a preset fourth threshold, the face image enhancement operation is repeatedly performed until the third score is not smaller than the preset fourth threshold, or it is determined that the quality evaluation result of the infrared face image is unqualified; The face image enhancement operation includes: It is determined whether the second face image is subjected to the face image enhancement operation for the first time; If not, it is determined that the quality evaluation result of the infrared face image corresponding to the second face image is unqualified; If yes, the second face image is input into a preset face anti-counterfeiting model, so that the face anti-counterfeiting model determines whether the second face image is a real face image; If it is determined that the second face image is not a real face image, it is determined that the quality evaluation result of the infrared face image corresponding to the second face image is unqualified; If it is determined that the second face image is a real face image, the second face image is input into a preset face enhancement model, so that the face enhancement model optimizes the definition, brightness and contrast of the second face image; The optimized second face image is subjected to image definition evaluation, image brightness evaluation and image contrast evaluation again, and a third score is recalculated according to the image definition evaluation result, the image brightness evaluation result and the image contrast evaluation result.

8. The method of claim 7, wherein the method is used for evaluating the quality of an infrared face image. The face anti-counterfeiting model and the face enhancement model are both constructed based on a lightweight neural network.

9. A device for evaluating the quality of infrared facial images, characterized in that, Comprise: An infrared face acquisition module is configured to acquire an infrared face image; An infrared face detection module is configured to perform face detection on the infrared face image to obtain face key points of the infrared face image; An infrared face alignment and denoising module is configured to align the face key points of the infrared face image according to preset standard face key points to obtain a perspective transformation matrix of the infrared face image, align a face region of the infrared face image according to the perspective transformation matrix to obtain an aligned first face image, traverse the first face image to sequentially obtain first pixel point coordinates of the first face image, determine second pixel point coordinates corresponding to the first pixel point coordinates of the first face image in the infrared face image through the perspective transformation matrix, and determine whether the second pixel point coordinates exist in a Gaussian point set. If yes, the first face image is continuously traversed to obtain a next first pixel point coordinate of the first face image. If no, the first pixel point of the second pixel point coordinates in the infrared face image is subjected to Gaussian processing, the second pixel point of the first pixel point coordinates in the first face image is replaced by the Gaussian-processed first pixel point, and the second pixel point coordinates are recorded in the Gaussian point set. When the traversal is completed, a second face image subjected to Gaussian processing is obtained; An infrared face evaluation module is configured to perform face integrity evaluation and face size evaluation according to the infrared face image, perform image definition evaluation, image brightness evaluation and image contrast evaluation according to the second face image when it is determined that the face is complete according to the face integrity evaluation result and the size of the face meets preset size requirements according to the face size evaluation result, and determine a quality evaluation result of the infrared face image according to any one or more combinations of the face integrity evaluation result, the face size evaluation result, the image definition evaluation result, the image brightness evaluation result and the image contrast evaluation result.

10. The device for evaluating the quality of an infrared face image according to claim 9, wherein, The infrared face evaluation module further comprises: An infrared face anti-counterfeiting submodule is configured to determine whether the second face image is a real face; An infrared face enhancement submodule is configured to optimize the definition, brightness and contrast of the second face image.

Images