Image quality enhancement method based on face recognition

Abstract

The invention relates to the technical field of face recognition, and particularly discloses an image quality enhancement method based on face recognition, comprising the following steps: S1, acquiring video frames and numbering, acquiring a video frame A with the minimum number, and determining an undetermined number; s2, performing ascending sorting on the undetermined numbers, determining a truncation position j in the sorting based on a preset constraint condition, and grouping the video frames based on the truncation position j; removing the grouped video frames, numbering the remaining video frames again, and obtaining a new group again until all the video frames are grouped; and S3, obtaining a target image based on an image fusion technology, obtaining a quality index, calculating a quality score, and determining the target image for quality enhancement based on the quality score. According to the invention, the accuracy and stability of face recognition are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to an image quality enhancement method based on face recognition. Background Art

[0002] Face recognition is a biometric technology that identifies people based on their facial features. It uses an acquisition device to collect images or video streams containing faces, and automatically detects and tracks faces in the images, and then performs facial recognition on the detected faces. It is a series of related technologies, also commonly called portrait recognition or facial recognition.

[0003] Due to factors such as lighting, angle, and occlusion, the collected images may be blurred or distorted, resulting in low image quality, which affects the accuracy and speed of face recognition. For example, under direct strong light, some areas of the face may appear too bright and lose details; or when the face is partially occluded, key features cannot be fully extracted, resulting in recognition difficulties. In addition, different shooting angles may also deform the facial contour, further increasing the difficulty of recognition and reducing recognition efficiency and accuracy. At this time, it is necessary to enhance the quality of images used for face recognition. However, in practical applications, faced with a large number of images with varying quality, how to determine which images have the potential to effectively improve the recognition effect through enhancement technology, and thus select appropriate images for enhancement, has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide an image quality enhancement method based on face recognition to solve the following technical problems: In practical applications, faced with a large number of images with varying quality, how to determine which images have the potential to effectively improve recognition results through enhancement technology and thus select appropriate images for enhancement has become an urgent problem to be solved.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A method for enhancing image quality based on face recognition comprises the following steps: S1: Based on the video captured by the acquisition device when the user performs face recognition, the video is frame-processed to obtain video frames, the video frames are numbered in timeline order, the video frame A with the smallest number is obtained, and the numbers of the video frames whose similarity with the video frame A is greater than a preset value are obtained and recorded as pending numbers; S2: sorting the pending numbers in ascending order, determining a cutoff position j in the sorting based on a preset constraint condition, and grouping the video frames corresponding to the pending numbers from the first position to the jth position in the sorting and the video frame A into the same group; Remove the grouped video frames, renumber the remaining video frames and obtain new groups again until all the video frames have been grouped; S3: Based on the image fusion technology, images in the same group are fused to obtain a target image, and quality indicators of the target image are obtained. The quality indicators include PSNR, SSIM, BRISQUE, and NIQE. The quality score is calculated based on the superior and inferior solution distance method, and the target image C with the largest quality score is obtained. The quality of the target image C is enhanced, and the enhanced target image C is used for face recognition.

[0006] As a further solution of the present invention: in the step S2, the constraint condition of the truncation position j is: the pending number at the i-th position in the sort is i+1 and the pending number at the k-th position in the sort is not k+1, i∈[1,j], k∈[j+1,n], n represents the total number of pending numbers in the sort.

[0007] As a further solution of the present invention: in step S1, the process of obtaining the similarity between the video frame and the video frame A specifically includes: Perform grayscale processing on the video frame A to obtain a grayscale image, which is recorded as image X1; Perform grayscale processing on the video frame to obtain a grayscale image, which is recorded as image X2; Calculate the gray value difference of the same pixel in the image X1 and the image X2, calculate the absolute value, and count the total absolute value. Then the similarity between the video frame and the video frame A is P=η / D, where D represents the total absolute value and η is a preset correction coefficient.

[0008] As a further solution of the present invention: if the total absolute value D=0, it is determined that the similarity degree is greater than the preset value.

[0009] As a further solution of the present invention: in the step S3, enhancing the quality of the target image C includes: denoising, deblurring and frequency domain enhancement.

[0010] As a further solution of the present invention: in the step S1, before the video frames are numbered, the following steps are also included: Identify the skeleton points in the video frame, and obtain the distance between any two skeleton points, where Fxy represents the distance between skeleton point x and skeleton point y; If Fxy<0.8Fxy', then Fxy is recorded as the abnormal distance, and Fxy' represents the preset distance between the bone point x and the bone point y; If the number of abnormal distances is greater than a preset number threshold, the corresponding video frame is determined to be an abnormal frame, and the abnormal frame is deleted from the video frame.

[0011] As a further solution of the present invention: identifying skeleton points in the video frame based on the OpenPose algorithm.

[0012] The beneficial effects of the present invention are as follows: 1) After the video is framed, by detecting whether the distance between the skeleton points is significantly less than the preset value, it can identify frames where the face is greatly turned sideways, blocked, or in abnormal posture, effectively reducing the loss of facial features caused by extreme postures, thereby improving the effectiveness and accuracy of face recognition; 2) By calculating the similarity between different frames and the reference frame A, the video frames with similarity greater than the preset threshold are grouped into the same group, thereby ensuring the similarity of the images in terms of texture, brightness, etc. during fusion. This not only suppresses the fusion distortion caused by excessive differences, but also reduces the noise of the fused image, ultimately obtaining a clearer and more coherent target image; 3) After obtaining the target image, the scheme uses multiple indicators such as PSNR (peak signal-to-noise ratio), SSIM (structural similarity), BRISQUE and NIQE for measurement, covering different dimensions such as reference and no reference, subjective and objective, and introduces the superior and inferior solution distance method to comprehensively evaluate various indicators, thereby improving the objectivity of image quality assessment; 4) The selected optimal target image C may still have residual noise or blur during the fusion process. This solution uses denoising and deblurring to make the image edges sharper and the details richer, thereby improving the recognizability of facial features. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention will be further described below in conjunction with the accompanying drawings.

[0014] Figure 1 It is a flow chart of an image quality enhancement method based on face recognition of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] See also Figure 1 As shown, the present invention is an image quality enhancement method based on face recognition, comprising the following steps: S1: Based on the video captured by the acquisition device when the user performs face recognition, the video is frame-processed to obtain video frames, the video frames are numbered in timeline order, the video frame A with the smallest number is obtained, and the numbers of the video frames whose similarity with the video frame A is greater than a preset value are obtained and recorded as pending numbers; It is understandable that, first, a video of the user performing face recognition is collected using a camera, a smartphone camera, or a monitoring device. At this point, the system will preset the video acquisition parameters, such as resolution (e.g., 1080p or 720p), frame rate (e.g., 25 or 30 frames per second), and color mode to ensure that the acquired video meets the clarity requirements and is easy to process later; the captured video is processed by a video decoding algorithm (e.g., based on the VideoCapture module in OpenCV), and the continuous video stream is decomposed into single frame images at fixed time intervals; after the frame processing, the video frames are numbered according to the timestamp information or acquisition order of each frame, and the numbering usually starts from 1, where the frame with the smallest number (i.e., the first frame) is video frame A, and then the pending number is selected according to the degree of similarity; In a preferred embodiment of the present invention, in step S1, the process of obtaining the similarity between the video frame and the video frame A specifically includes: Perform grayscale processing on the video frame A to obtain a grayscale image, which is recorded as image X1; Perform grayscale processing on the video frame to obtain a grayscale image, which is recorded as image X2; Calculate the gray value difference of the same pixel in the image X1 and the image X2, calculate the absolute value, and count the total absolute value. Then the similarity between the video frame and the video frame A is P=η / D, where D represents the total absolute value and η is a preset correction coefficient; It should be noted that the video frame A and other video frames to be compared are gray-scaled and the color images are converted into gray-scale images. The converted images are recorded as image X1 and image X2 respectively. This conversion not only reduces the computational complexity, but also eliminates the interference of color factors on the similarity calculation. The gray values ​​of the corresponding pixels in image X1 and image X2 are compared, the difference in the gray value of each pixel is calculated and the absolute value is taken, and then the absolute difference of all pixels is accumulated to obtain a total absolute value, which reflects the overall gray difference between the two frames. The greater the overall gray difference, the lower the similarity, and vice versa. In a preferred case of this embodiment, if the total absolute value D=0, it is determined that the similarity is greater than the preset value; It can be understood that the total absolute value D = 0, that is, the two frames are completely consistent; In a preferred embodiment of the present invention, in the step S1, before the video frames are numbered, the following steps are also included: Identify the skeleton points in the video frame, and obtain the distance between any two skeleton points, where Fxy represents the distance between skeleton point x and skeleton point y; If Fxy<0.8Fxy', then Fxy is recorded as the abnormal distance, and Fxy' represents the preset distance between the bone point x and the bone point y; If the number of abnormal distances is greater than a preset number threshold, the corresponding video frame is determined to be an abnormal frame, and the abnormal frame is deleted from the video frame; In a preferred embodiment of the present invention, skeleton points in the video frame are identified based on the OpenPose algorithm; It should be noted that each frame of video is processed using a skeleton point detection algorithm such as OpenPose to automatically detect the positions of the skeleton points of key parts of the human body (such as eyes, nose, ears, etc.). For each pair of key skeleton points detected (for example, left eye and right eye, etc.), the actual distance is calculated using the Euclidean distance formula, which is recorded as Fxy, where x and y represent two key points of the human body respectively; in system initialization or based on historical data, the standard distance Fxy' of each pair of skeleton points in normal frontal shooting is predefined. For example, in the frontal shooting state, the preset distance between the left eye and the right eye may be 50 pixels; For example, in a frame, the preset distance F(left eye, right eye) between the left eye and the right eye is 50 pixels under normal circumstances. If the distance F(left eye, right eye) between the left eye and the right eye detected in the current frame is 38 pixels, 38 is less than 0.8×50 (i.e., 40 pixels), so the distance is determined to be an abnormal distance; The above comparison is performed on all detected pairs of skeleton points in each frame, and the number of abnormal distances is counted. If the number of abnormal distances in a frame exceeds the preset threshold (for example, more than 2), it is considered that the frame as a whole has a large abnormality, which may be due to the abnormal shooting angle caused by the user's posture deviation (such as tilting the head, lowering the head, or partial occlusion), so the frame is judged to be an abnormal frame; in addition to the left eye and right eye pair, if the two pairs of skeleton points of the left ear and right ear and the nose tip and chin also have abnormalities (the actual distances are less than 0.8 times the preset value), then the number of abnormal distances reaches 3 in total, exceeding the set threshold. At this time, the frame is judged to be an abnormal frame and will be removed from the subsequent video frame set, and will no longer participate in subsequent numbering and similarity calculation processing steps; S2: sorting the pending numbers in ascending order, determining a cutoff position j in the sorting based on a preset constraint condition, and grouping the video frames corresponding to the pending numbers from the first position to the jth position in the sorting and the video frame A into the same group; Remove the grouped video frames, renumber the remaining video frames and obtain new groups again until all the video frames have been grouped; In a preferred embodiment of the present invention, in step S2, the constraint condition of the truncation position j is: the pending number at the i-th position in the sort is i+1 and the pending number at the k-th position in the sort is not k+1, i∈[1,j], k∈[j+1,n], n represents the total number of pending numbers in the sort.

[0017] It can be understood that, assuming that the video frames are numbered 1, 2, 3, 4, 5, 6, ..., and the reference frame A is 1, after similarity screening, the set of pending numbers is {2, 3, 5, 6, 7, 10}, and after the set is sorted in ascending order, the order remains 2, 3, 5, 6, 7, 10; Traverse the sorted pending number set and check whether it is continuous starting from the first one: If the number of the first position is 2 (i.e. 1+1), the condition is met; If the number of the second position is 3 (i.e. 2+1), the condition continues to be satisfied; When checking the third position, if the number is not equal to 4 (i.e. 2+1=4), it means that the continuity is interrupted, and the truncation position j is 2; The reference frame A (the frame with the smallest number, usually 1) and the video frames with the to-be-determined numbers within the range of the truncation position j (in this case, the frames with numbers 2 and 3) are grouped into the same group. The video frames in this group are continuous in shooting time and have similar contents, which is suitable for subsequent image fusion; After the grouping is completed, this part of the video frames is removed from the entire video frame set to avoid reuse in subsequent processing; the remaining video frames (in this example, the remaining frames are numbered 5, 6, 7, 10, etc.) are renumbered in chronological order, and then the above steps are repeated; taking the remaining frames {5, 6, 7, 10} as an example, they may become 1, 2, 3, 4 after renumbering. If in the new grouping process, after similarity screening, the pending number set is {2, 3} (part of the original corresponding frames 5, 6, 7), then the truncation position is determined according to the continuity constraint, and finally the new reference frame and the continuous frame are grouped together; S3: Based on the image fusion technology, images in the same group are fused to obtain a target image, and quality indicators of the target image are obtained, wherein the quality indicators include PSNR, SSIM, BRISQUE, and NIQE. A quality score is calculated based on the superior and inferior solution distance method, and a target image C with the largest quality score is obtained. The quality of the target image C is enhanced, and the enhanced target image C is used for face recognition. In a preferred embodiment of the present invention, in step S3, enhancing the quality of the target image C includes: denoising, deblurring and frequency domain enhancement; It is worth noting that for the continuous video frames in the same group, the images are first preprocessed (such as unification of size and brightness normalization), and then the advantages of each frame are fused into a target image using multi-scale fusion or weighted averaging methods. For example, if there are five frames in the same group, the best information of the clear area of ​​each frame can be extracted through pixel-level or feature-level fusion to form a target image with richer details and lower noise; the multi-scale fusion technology based on wavelet transform can be used to decompose the image into low-frequency and high-frequency parts, calculate the fusion weights for each frame respectively, and then reconstruct the final synthetic image; or a simple weighted averaging method can be used, in which the weight of each frame is adaptively adjusted according to the local contrast and texture information of the image; After selecting the target image C, in order to further improve its clarity and detail performance and make it more suitable for face recognition, the following image enhancement processing is performed: 1) Denoising: Use non-local mean, bilateral filtering or deep learning denoising network to suppress random noise in the image; for image C with more noise points, bilateral filtering can smooth the noise while retaining edge information, thereby achieving the purpose of denoising; 2) Deblurring: Use blind deconvolution, Wiener filtering or deep learning-based deblurring algorithm to restore the blur phenomenon caused by motion or out-of-focus in the image. If the target image C has slight motion blur, the blind deconvolution method can be used to estimate the blur kernel, and the iterative algorithm can be used to restore the image clarity; 3) Frequency domain enhancement: Use Fourier transform to convert the image to the frequency domain, and perform enhancement processing on the high-frequency components, such as high-frequency sharpening or bandpass filtering, to highlight the image details; in the frequency domain, appropriately suppress the low-frequency components, and at the same time perform gain processing on the high-frequency components, and then perform inverse Fourier transform to make the image edge and texture information more obvious, further improving the clarity of the key features of the face; After the above fusion, quality evaluation and enhancement processing, the target image C not only reaches the optimal state in terms of overall quality, but also the key details (such as edges, textures and contrast) are significantly improved, which makes the facial features in the image more prominent and provides extremely clear and detailed input for the subsequent face recognition module, thereby greatly improving the recognition accuracy and robustness.

[0018] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method for enhancing image quality based on face recognition, characterized in that: The following steps are involved: S1: Based on the video captured by the acquisition device when the user performs face recognition, the video is frame-processed to obtain video frames, the video frames are numbered in timeline order, the video frame A with the smallest number is obtained, and the numbers of the video frames whose similarity with the video frame A is greater than a preset value are obtained and recorded as pending numbers; S2: sorting the pending numbers in ascending order, determining a cutoff position j in the sorting based on a preset constraint condition, and grouping the video frames corresponding to the pending numbers from the first position to the jth position in the sorting and the video frame A into the same group; Remove the grouped video frames, renumber the remaining video frames and obtain new groups again until all the video frames have been grouped; S3: Based on the image fusion technology, images in the same group are fused to obtain a target image, and quality indicators of the target image are obtained. The quality indicators include PSNR, SSIM, BRISQUE, and NIQE. The quality score is calculated based on the superior and inferior solution distance method, and the target image C with the largest quality score is obtained. The quality of the target image C is enhanced, and the enhanced target image C is used for face recognition.

2. The image quality enhancement method based on face recognition according to claim 1, characterized in that: In step S2, the constraint condition of the truncation position j is: the pending number at the i-th position in the sort is i+1 and the pending number at the k-th position in the sort is not k+1, i∈[1,j], k∈[j+1,n], n represents the total number of pending numbers in the sort.

3. The image quality enhancement method based on face recognition according to claim 1, characterized in that: In step S1, the process of obtaining the similarity between the video frame and the video frame A specifically includes: Perform grayscale processing on the video frame A to obtain a grayscale image, which is recorded as image X1; Perform grayscale processing on the video frame to obtain a grayscale image, which is recorded as image X2; Calculate the gray value difference of the same pixel in the image X1 and the image X2, calculate the absolute value, and count the total absolute value. Then the similarity between the video frame and the video frame A is P=η / D, where D represents the total absolute value and η is a preset correction coefficient.

4. The image quality enhancement method based on face recognition according to claim 3 is characterized in that: If the total absolute value D=0, it is determined that the similarity is greater than the preset value.

5. The image quality enhancement method based on face recognition according to claim 1, characterized in that: In the step S3, enhancing the quality of the target image C includes: denoising, deblurring and frequency domain enhancement.

6. The image quality enhancement method based on face recognition according to claim 1, characterized in that: In the step S1, before numbering the video frames, the following steps are also included: Identify the skeleton points in the video frame, and obtain the distance between any two skeleton points, where Fxy represents the distance between skeleton point x and skeleton point y; If Fxy<0.8Fxy', then Fxy is recorded as the abnormal distance, and Fxy' represents the preset distance between the bone point x and the bone point y; If the number of abnormal distances is greater than a preset number threshold, the corresponding video frame is determined to be an abnormal frame, and the abnormal frame is deleted from the video frame.

7. The image quality enhancement method based on face recognition according to claim 6, characterized in that: The skeleton points in the video frame are identified based on the OpenPose algorithm.

Images