Face recognition method and system based on neural network adaptive image enhancement

Abstract

The invention discloses a face recognition method and system based on neural network adaptive image enhancement, belonging to the field of image recognition technology. The method first evaluates the image quality of an original face image, and determines a first face image according to the quality score; then obtains the quality index of the first face image, and determines the optimization strategy of the first face image according to the quality index; secondly, the first face image is adaptively enhanced according to the optimization strategy to obtain an enhanced face image; finally, the similarity between the first face image and the enhanced face image is evaluated according to contour features and local features, and when the similarity meets the requirements, the enhanced face image is used for face recognition; the method solves the problem of optimization distortion of face images, ensures the similarity of face images before and after enhancement, and improves the accuracy of face recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a face recognition method and system based on neural network adaptive image enhancement. Background Art

[0002] Face recognition technology is a biometric technology that identifies people based on their facial features. The original face images are affected by the shooting environment, resulting in overexposure, noise, blur, and other problems. These image quality problems will seriously affect the accuracy of face recognition, leading to false matches, which will cause a series of security issues. For example, incorrect identity authentication may lead to illegal access, or may cause major misjudgments in security monitoring or identity verification applications. Therefore, it is necessary to use image enhancement technology to enhance the collected face images to improve the accuracy and efficiency of face recognition.

[0003] For example, the patent application with publication number CN116739608A discloses a bank user identity authentication method and system based on face recognition, which enhances the effective information in the image by performing pixel-level processing on the collected face image to improve the accuracy of face recognition; for another example, the patent application with publication number CN109243030A discloses a control method and system for a nighttime contactless access control system, which effectively improves the brightness of the face image, eliminates the shadows and dark areas of the face image, improves the contrast of the face image, and solves the problem of face recognition errors caused by insufficient light intensity at night in traditional face recognition access control systems.

[0004] Existing image enhancement methods only enhance image quality from a single aspect. When the collected facial images have other quality problems, face recognition failure will still occur. Secondly, existing methods do not evaluate image quality before image processing. For facial images with qualified image quality, operation process redundancy will be caused, reducing face recognition efficiency. In addition, existing methods do not perform similarity evaluation on facial images after image enhancement. There is a possibility that facial images after image enhancement are distorted, leading to face recognition errors. Summary of the invention

[0005] In response to the problems existing in the prior art, the present invention provides a face recognition method and system based on neural network adaptive image enhancement, which performs adaptive enhancement processing on the collected face images and performs similarity evaluation through a twin neural network, thereby ensuring the similarity before and after image enhancement and improving the accuracy of face recognition.

[0006] The present invention is achieved through the following technical solutions:

[0007] A face recognition method based on neural network adaptive image enhancement, comprising the following steps:

[0008] Evaluate the image quality of the original face image, and determine a first face image according to the evaluation score, wherein the first face image is an image that does not meet the face recognition requirements;

[0009] Obtaining a quality index of the first facial image, and determining an optimization strategy for the first facial image according to the quality index;

[0010] Adaptively enhancing the first face image according to the optimization strategy to obtain an enhanced face image;

[0011] The similarity between the first face image and the enhanced face image is evaluated based on the contour features and the local features. When the similarity meets the requirements, the enhanced face image is used for face recognition.

[0012] Preferably, the step of evaluating the image quality of the original facial image and determining the first facial image according to the evaluation score includes:

[0013] The original face image is preprocessed, and the quality characteristics of the preprocessed original face image are evaluated. A first face image and a second face image are obtained according to the evaluation scores, and the second face image is an image that meets the face recognition requirements.

[0014] Preferably, the preprocessing of the original facial image is to preprocess the original facial image using a pre-trained deep learning model for face detection and key point positioning.

[0015] Preferably, the quality index includes a brightness index, a noise index and a clarity index.

[0016] Preferably, the step of determining the optimization strategy for the first facial image according to the quality index includes:

[0017] The quality indicators of the first face image are compared with the quality indicators of the standard face image, and the optimization strategy of the first face image is determined according to the quality indicators that do not meet the requirements.

[0018] Preferably, the determination of each quality index of the first facial image includes:

[0019] Determine the brightness index of the first face image according to the brightness function in the OpenCV software library;

[0020] Determining a noise index of the first face image according to a standard deviation of the face image;

[0021] A clarity index of the first facial image is determined according to a variance of a Laplace transform of the facial image.

[0022] Preferably, the adaptively enhancing the first facial image according to the optimization strategy is adaptively enhancing the first facial image according to the optimization strategy in combination with a pre-trained neural network model.

[0023] Preferably, the step of adaptively enhancing the first face image according to the optimization strategy and in combination with a pre-trained neural network model includes:

[0024] The pre-trained Zero-DCE model is used to optimize the brightness of the first face image to obtain a face image with enhanced brightness;

[0025] Using the pre-trained Noise2Void model, denoising the first face image is performed to obtain a denoised face image;

[0026] The pre-trained IKC model is used to deblur the first face image to obtain a face image with enhanced clarity.

[0027] Preferably, the step of evaluating the similarity between the first face image and the enhanced face image based on the contour features and the local features includes:

[0028] The first face image and the enhanced face image are respectively converted into the frequency domain using discrete cosine transform, and the high-frequency components are removed. Then, the time domain images are obtained by inverse discrete cosine transform. The time domain images of the first face image and the enhanced face image are input into the twin neural network to obtain the contour similarity score.

[0029] A multi-task cascade convolutional network is used to obtain the eye images of the first face image and the enhanced face image, and the eye images are input into the twin neural network as feature identification images to obtain local similarity scores;

[0030] The similarity between the first face image and the enhanced face image is evaluated based on the contour similarity score and the local similarity score.

[0031] A face recognition system based on neural network adaptive image enhancement, comprising:

[0032] A quality assessment module, used to assess the image quality of the original face image, and determine a first face image according to the assessment score, wherein the first face image is an image that does not meet the face recognition requirements;

[0033] An optimization strategy module, used to obtain a quality index of the first face image and determine an optimization strategy for the first face image according to the quality index;

[0034] An image enhancement module, used for adaptively enhancing the first face image according to the optimization strategy to obtain an enhanced face image;

[0035] The similarity evaluation module is used to evaluate the similarity between the first face image and the enhanced face image based on the contour features and the local features. When the similarity meets the requirements, the enhanced face image is used for face recognition.

[0036] Compared with the prior art, the present invention has the following beneficial technical effects:

[0037] The present invention provides a face recognition method based on neural network adaptive image enhancement. First, an original face image is evaluated to determine whether the original face image meets the image quality requirement of face recognition. Through this step, the original face images with image quality meeting the requirement are filtered, and only the first face image with image quality not meeting the requirement is optimized, so as to improve the efficiency of face recognition. Then, each quality index of the first face image is obtained, and each quality index is evaluated. An optimization strategy for the first face image is determined according to the quality index not meeting the requirement. An optimization strategy is constructed by using the quality index not meeting the requirement, and the first face image is enhanced in a targeted manner, so as to solve the problem of redundant optimization of face images in the prior art. Finally, a similarity evaluation is performed on the enhanced face image and the first face image, so as to solve the problem of distortion in face image optimization, and ensure the similarity of the face image before and after enhancement, so as to improve the accuracy of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flowchart of a face recognition method based on neural network adaptive image enhancement according to the present invention;

[0039] Figure 2 is a flow chart of a method for evaluating the similarity between a first face image and an enhanced face image according to the present invention;

[0040] Figure 3 The structure diagram of the face recognition system based on neural network adaptive image enhancement of the present invention is shown in FIG. DETAILED DESCRIPTION

[0041] The present invention will be further described in detail below in conjunction with the accompanying drawings, which are intended to explain the present invention rather than to limit it.

[0042] Example 1

[0043] See also Figure 1 , a face recognition method based on neural network adaptive image enhancement, comprising the following steps:

[0044] Step 1: preprocess the original face image to obtain a preprocessed face image.

[0045] The purpose of the preprocessing is to improve the quality of the facial image so as to improve the accuracy of subsequent facial recognition. The preprocessing of the original facial image includes grayscale processing, histogram equalization processing, denoising processing, etc., and at least one preprocessing method is used to preprocess the original facial image; in some embodiments, a combination of multiple preprocessing methods is used to preprocess the original facial image.

[0046] In order to improve the efficiency of preprocessing of the original face image, a pre-trained deep learning model is used to preprocess the original face image. For example, a pre-trained deep learning model MTCNN (Multi-task Cascaded Convolutional Networks) for face detection and key point positioning is used to preprocess the original face image. The original face image is input into the deep learning model, and the deep learning model detects and preprocesses the original face image to obtain a preprocessed face image. The size of the preprocessed face image is 112×112 pixels.

[0047] Step 2: Acquire quality features of the preprocessed face image, evaluate the image quality of the preprocessed face image based on the quality features, and obtain the first face image and the second face image based on the evaluation results.

[0048] The first face image is a face image whose image quality does not meet the requirements, and the first face image is optimized.

[0049] The second facial image is a facial image whose image quality meets the requirements, and facial recognition is performed on the second facial image.

[0050] In some embodiments, the quality features of the preprocessed facial image are parameterized to obtain a quality value, and the quality value is compared with a set quality threshold. When the quality value is less than the quality threshold, the quality of the preprocessed facial image does not meet the requirements and needs to be optimized. When the quality value is greater than the quality threshold, the quality of the preprocessed facial image meets the requirements and facial recognition is performed.

[0051] In some embodiments, a neural network model is used to predict the quality of the preprocessed facial image to obtain a quality score of the preprocessed facial image, the quality score is compared with a set threshold, and whether the quality of the preprocessed facial image meets the requirements is determined based on the comparison result.

[0052] The neural network model is a deep blind image quality assessment model DeepBIQ (Deep Image Quality Assessment), a neural image assessment model NIMA (Neural Image Assessment), an image quality evaluation model based on deep learning, or a no-reference image quality assessment model.

[0053] The DeepBIQ model is preferably used. The DeepBIQ model is an image quality assessment method based on deep learning, which is mainly used to assess the quality of a single image without a reference image. Compared with traditional image quality assessment methods, the DeepBIQ model uses deep learning to automatically learn and extract complex features related to image quality, avoiding the tedious process of manually designing features in traditional methods, and improving the accuracy and efficiency of the assessment.

[0054] The DeepBIQ model is trained using an image dataset to obtain a pre-trained DeepBIQ model. During the training process, the DeepBIQ model uses a convolutional neural network to learn the quality features of the image, predicts the quality score of the image based on the quality features, and outputs the quality score as a real value R. The pre-processed face image is input into the pre-trained DeepBIQ model to obtain the quality score of the pre-processed face image. The real value R of the pre-processed face image is compared with the set threshold value Y1. If the real value R is greater than the set threshold value Y1, the image quality is qualified and the pre-processed face image meets the requirements of face recognition. Otherwise, the pre-processed face image does not meet the requirements of face recognition. Step 3 is executed to optimize the pre-processed face image.

[0055] Step 3: Evaluate the brightness, noise and clarity indicators of the first face image, and determine the optimization strategy of the first face image according to the indicators, as follows:

[0056] S3.1. Determine brightness, noise and clarity indicators of the first face image.

[0057] OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library designed for efficient image processing and computer vision tasks. The brightness related function in the OpenCV software library is used to determine the brightness index Q1 of the first face image;

[0058] Determining a noise index Q2 of the first face image according to a standard deviation of the face image;

[0059] The clarity index Q3 of the first face image is determined according to the variance of the Laplace transform of the face image.

[0060] S3.2. Compare the brightness, noise and clarity indexes of the first face image with those of the standard face image, and determine the optimization strategy for the first face image according to the comparison results.

[0061] The standard face image is an image that can be used for normal face recognition. The brightness, noise and clarity indicators of the standard face image are used as reference standards. The brightness, noise and clarity indicators of the first face image are compared with the corresponding reference standards respectively, and the optimization strategy of the preprocessed face image is determined according to the indicators that exceed the reference standards.

[0062] For example, if the brightness index exceeds the brightness value range corresponding to the reference standard, and the noise index and the clarity index are within the noise value and clarity value range corresponding to the reference standard, the brightness of the first face image is optimized.

[0063] For another example, when the brightness, noise and clarity indicators all exceed the brightness value, noise value and clarity value corresponding to the reference standard, the brightness, noise and clarity of the first face image are optimized.

[0064] Step 4: Adaptively enhance the first face image according to the optimization strategy to obtain an enhanced face image.

[0065] The Zero-DCE (Zero-Reference Deep Curve Estimation) model is a deep learning model for enhancing image brightness. It does not require any reference image for brightness enhancement, but directly learns the brightness adjustment curve from the original image to optimize the image brightness. The pre-trained Zero-DCE model is used to optimize the brightness of the first face image, and the first image is input into the Zero-DCE model to obtain the first face image with enhanced brightness.

[0066] Noise2Voi is a model based on a self-supervised denoising method. It can be trained and denoised without clean, noise-free image samples. It is particularly suitable for practical application scenarios where it is difficult to obtain or create noise-free training data sets. It uses the local information of the image itself to effectively infer the clean image content, thereby achieving the purpose of denoising. The pre-trained Noise2Void model is used to denoise the first face image to obtain the denoised first face image.

[0067] The IKC (Iterative Kernel Correction) model is a deep learning model designed for blind deblurring problems. It can gradually restore the clarity of an image without knowing the exact type and cause of blur, and effectively remove blur from the image. The pre-trained IKC model is used to deblur the first face image to obtain a first face image with enhanced clarity.

[0068] Step 5: Evaluate the similarity between the first face image and the enhanced face image based on the contour features and the local features. When the similarity meets the requirements, the enhanced face image meets the quality requirements for face recognition.

[0069] The similarity evaluation methods include feature-based comparison methods, histogram-based comparison methods, deep learning methods, structural similarity index methods, and hash-based methods.

[0070] In this embodiment, a twin neural network is used to evaluate the similarity between the first face image and the enhanced face image. A twin neural network is a twin network used to learn the similarity of image pairs, and includes two identical subnetworks that share the same parameters. The twin neural network is mainly used to compare whether two inputs are similar or belong to the same category. The twin neural network can directly output a similarity score by comparing and learning paired images. The specific method is as follows:

[0071] See also Figure 2 , the first face image and the enhanced face image are converted into the frequency domain using discrete cosine transform, the high-frequency part is removed, and then the contour image is obtained by inverse discrete cosine transform, and the contour images of the first face image and the enhanced face image are input into the twin neural network to obtain the contour similarity score S1.

[0072] The first face image and the enhanced face image are respectively extracted with MTCNN (Multi-task Cascaded Convolutional Networks) as feature identification images, and the feature identification images are input into the twin neural network to obtain the local similarity score S2.

[0073] The similarity score S between the first face image and the enhanced face image is evaluated based on the contour similarity score S1 and the local similarity score S2.

[0074] The similarity score S=a between the first face image and the enhanced face image S1+b S2, a, b>0 and a+b=1, a is the weight of the first face image, and b is the weight of the enhanced face image.

[0075] Compare the similarity score S with the set threshold value Y2. If the similarity score S is greater than the set threshold value Y2, the enhanced face image meets the face recognition image standard. If the similarity score S is less than the set threshold value Y2, the enhanced face image does not meet the face recognition image standard, and the original face image is recaptured and steps 1-4 are performed.

[0076] Step 6: Input the enhanced face image into the face recognition system for recognition to obtain the identity information corresponding to the face image.

[0077] The face images in the image collection stored in the face recognition system database are compared with the enhanced face image in turn, and the face recognition system returns the most similar face image T whose similarity is greater than a threshold, and the identity information corresponding to the face image T is used as the result of face recognition.

[0078] Example 2

[0079] See also Figure 3 On the other hand, the present invention also provides a face recognition system based on neural network adaptive image enhancement, comprising:

[0080] The quality assessment module 100 is used to assess the image quality of the original face image and determine a first face image according to the assessment score, wherein the first face image is an image that does not meet the face recognition requirements;

[0081] An optimization strategy module 200 is used to obtain a quality index of the first face image and determine an optimization strategy for the first face image according to the quality index;

[0082] An image enhancement module 300 is used to adaptively enhance the first face image according to the optimization strategy to obtain an enhanced face image;

[0083] The similarity evaluation module 400 is used to evaluate the similarity between the first face image and the enhanced face image based on the contour features and the local features, and when the similarity meets the requirements, the enhanced face image is used for face recognition.

[0084] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0085] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used to perform the operation of the above-mentioned face recognition method based on neural network adaptive image enhancement.

[0086] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the face recognition method based on neural network adaptive image enhancement in the above embodiment.

[0087] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, readable storage media, optical storage, etc.) containing computer-usable program codes.

[0088] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0089] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A face recognition method based on neural network adaptive image enhancement, characterized in that: The following steps are involved: Evaluate the image quality of the original face image, and determine a first face image according to the evaluation score, wherein the first face image is an image that does not meet the face recognition requirements; The step of evaluating the image quality of the original facial image and determining the first facial image according to the evaluation score includes: Preprocessing the original face image, evaluating the quality features of the preprocessed original face image, and obtaining a first face image and a second face image according to the evaluation scores, wherein the second face image is an image that meets the face recognition requirements; Obtaining a quality index of the first facial image, and determining an optimization strategy for the first facial image according to the quality index; Adaptively enhancing the first face image according to the optimization strategy to obtain an enhanced face image; The similarity between the first face image and the enhanced face image is evaluated based on the contour features and the local features, and when the similarity meets the requirements, the enhanced face image is used for face recognition; The evaluating the similarity between the first face image and the enhanced face image according to the contour features and the local features includes: The first face image and the enhanced face image are respectively converted into the frequency domain using discrete cosine transform, and the high-frequency components are removed. Then, the time domain images are obtained by inverse discrete cosine transform. The time domain images of the first face image and the enhanced face image are input into the twin neural network to obtain the contour similarity score. A multi-task cascade convolutional network is used to obtain the eye images of the first face image and the enhanced face image, and the eye images are input into the twin neural network as feature identification images to obtain local similarity scores; Evaluating the similarity between the first face image and the enhanced face image according to the contour similarity score and the local similarity score; The similarity score S=a between the first face image and the enhanced face image S1+b S2, a, b>0 and a+b=1, a is the weight of the first face image, and b is the weight of the enhanced face image.

2. The face recognition method based on neural network adaptive image enhancement according to claim 1, characterized in that: The preprocessing of the original face image is to preprocess the original face image using a pre-trained deep learning model for face detection and key point positioning.

3. The face recognition method based on neural network adaptive image enhancement according to claim 1, characterized in that: The quality index includes a brightness index, a noise index and a clarity index.

4. The face recognition method based on neural network adaptive image enhancement according to claim 3 is characterized in that: The step of determining the optimization strategy for the first facial image according to the quality index includes: The quality indicators of the first face image are compared with the quality indicators of the standard face image, and the optimization strategy of the first face image is determined according to the quality indicators that do not meet the requirements.

5. The face recognition method based on neural network adaptive image enhancement according to claim 3 is characterized in that: The determination of each quality index of the first facial image includes: Determine the brightness index of the first face image according to the brightness function in the OpenCV software library; Determining a noise index of the first face image according to a standard deviation of the face image; A clarity index of the first facial image is determined according to a variance of a Laplace transform of the facial image.

6. The face recognition method based on neural network adaptive image enhancement according to claim 1, characterized in that: The adaptively enhancing the first facial image according to the optimization strategy is to adaptively enhance the first facial image according to the optimization strategy in combination with a pre-trained neural network model.

7. The face recognition method based on neural network adaptive image enhancement according to claim 6, characterized in that: The step of adaptively enhancing the first face image according to the optimization strategy and in combination with the pre-trained neural network model includes: The pre-trained Zero-DCE model is used to optimize the brightness of the first face image to obtain a face image with enhanced brightness; Using the pre-trained Noise2Void model, denoising the first face image is performed to obtain a denoised face image; The pre-trained IKC model is used to deblur the first face image to obtain a face image with enhanced clarity.

8. A face recognition system based on neural network adaptive image enhancement, characterized in that: include: A quality assessment module, used to assess the image quality of the original face image, and determine a first face image according to the assessment score, wherein the first face image is an image that does not meet the face recognition requirements; An optimization strategy module, used to obtain a quality index of the first face image and determine an optimization strategy for the first face image according to the quality index; An image enhancement module, used for adaptively enhancing the first face image according to the optimization strategy to obtain an enhanced face image; The similarity evaluation module is used to evaluate the similarity between the first face image and the enhanced face image based on the contour features and the local features. When the similarity meets the requirements, the enhanced face image is used for face recognition.

Images