Face recognition method and system based on image exposure correction network

Abstract

The invention discloses a face recognition method and system based on an image exposure correction network. Firstly, an input image is preprocessed, then the preprocessed image is input into a face image extraction network to obtain a face image, then the face image is input into an image exposure correction network to obtain a normal-exposure face image, and finally the normal-exposure face image is input into a face recognition network to obtain a face recognition result. The image exposure correction network comprises a color space conversion module, an exposure feature extraction network, a brightness mapping network and a detail enhancement module; the exposure feature extraction network comprises a down-sampling layer, a random cutting module and an image encoder A module; and the face recognition network comprises an image encoder B based on the SimCLR. The method has the advantages of stability, independence, rapidness, high efficiency and the like, and the accuracy of face recognition under the condition of abnormal illumination environment can be greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of face recognition, and relates to a face recognition method and system, and in particular to a face recognition method and system based on an image exposure correction network. Background Art

[0002] Since the 1990s, face recognition technology has made great progress. However, during the acquisition and transmission of face images, they may be affected by factors such as changes in lighting, expression, and posture. These factors bring great challenges to face recognition.

[0003] In early research, face recognition mainly relied on geometric feature extraction methods, such as describing facial features by the distance and angle between facial key points [1]. These methods can cope with changes in facial expressions and postures to a certain extent, but are limited by computational complexity and recognition accuracy. Subsequently, algebraic methods based on eigenfaces [2] became increasingly popular. This method extracts the main components of facial images through singular value decomposition (SVD), effectively reducing data dimensions and improving recognition efficiency. However, these methods still have limitations in dealing with illumination changes and occlusion problems. With the development of machine learning, machine learning algorithms have been widely used in the field of face recognition. Methods based on statistical learning, such as support vector machines [3] and boosting algorithms [4], have gradually become mainstream, greatly improving the accuracy and robustness of face recognition.

[0004] In recent years, with the breakthrough of deep learning technology, especially the successful application of convolutional neural network (CNN) [5] in the field of image recognition, face recognition technology has made revolutionary progress. Deep learning models can automatically learn the deep features of face images, effectively overcoming the limitations of traditional methods in dealing with complex scenes. Deep learning models have demonstrated excellent capabilities in processing complex scenes and diverse face images. For example, DeepFace [6] extracts features through a deep convolutional network and achieves efficient recognition on a large database. However, the mathematical modeling of face recognition based on deep learning methods is still not perfect.

[0005] [1]Goldstein, L., Samet, H.,&Teller, S. (1989). Geometric and photometric invariance for object recognition. International Journal of Computer Vision, 3(3), 209-226.

[0006] [2]Turk, M.,&Pentland, A. (1991). Eigenfaces for recognition. Journalof cognitive neuroscience, 3(1), 71-86。

[0007] [3]Platt, J. C. (1999). Using sparsity to improve support vectormachines. In Advances in neural information processing systems (pp. 547-553)。

[0008] [4]Freund, Y.,&Schapire, R. E. (1997). A decision-theoreticgeneralization of on-line learning and an application to boosting. Journal ofcomputer and system sciences, 55(1), 119-139。

[0009] [5]Krizhevsky, A., Sutskever, I.,&Hinton, G. E. (2012). ImageNetclassification with deep convolutional neural networks. In Advances in neuralinformation processing systems (pp. 1097-1105)。

[0010] [6]Taigman, Y., Yang, M., Ranzato, M.,&Wolf, L. (2014). DeepFace:Closing the gap to human-level performance in face verification. InProceedings of the IEEE conference on computer vision and pattern recognition(pp. 1701-1708)。 Summary of the Invention

[0011] In order to solve the problem of low accuracy of current face recognition models under abnormal lighting conditions, the present invention provides a face recognition method and system based on an image exposure correction network, which enhances the accuracy of the face recognition model through the design of the image exposure correction network.

[0012] According to one aspect of the present invention, a face recognition method based on an image exposure correction network is provided, comprising: Inputting the image to be detected into a facial image extraction network to obtain a face image; the facial image extraction network includes a HOG feature extraction module; Inputting the obtained face image into an image exposure correction network to obtain a face image with normal exposure; the image exposure correction network includes a color space conversion module, an exposure feature extraction network, a brightness mapping network and a detail enhancement module; The obtained exposed normal face image is input into a face recognition network to obtain a face recognition result; the face recognition network includes an image encoder based on SimCLR.

[0013] As a further technical solution, the method further includes: Acquire an image to be detected and preprocess the image to be detected.

[0014] As a further technical solution, the exposure feature extraction network includes a downsampling layer, a random cropping module and an image encoder module; the downsampling layer contains 4 basic network structures and 1 sampling layer, each basic network structure includes 2 residual blocks and 1 self-attention layer; the random cropping module is used to provide an image with a fixed resolution; the image encoder module includes a residual neural network and 1 self-attention layer.

[0015] As a further technical solution, the training of the exposure feature extraction network includes: Import the exposure error image dataset; Input the wrong exposure image and the normal exposure image into the same downsampling layer respectively; The downsampled exposure error image and exposure normal image as well as the original exposure error image and exposure normal image are input into the random cropping module respectively; The randomly cropped image is input into the image encoder module, and the output of the image encoder module is projected into the feature space through the decoder; The training data is input into the image encoder module and decoder network for training, and the model parameters are continuously optimized through the standardized temperature-scaled cross entropy loss function to output the trained exposure feature extraction network.

[0016] As a further technical solution, the training of the brightness mapping network includes: Import the exposure error image dataset; The exposure error image and the exposure normal image are trained on the trained frozen exposure feature extraction network using a regularized linear regressor to obtain the brightness mapping relationship.

[0017] As a further technical solution, the implementation of the detail enhancement module includes: The image after brightness mapping is smoothed by using a Gaussian filter to obtain an image with enhanced details.

[0018] As a further technical solution, the training of the face recognition network includes: Import face recognition dataset; Perform random image enhancement on face images; The enhanced face image is input into the image encoder trained based on SimCLR, and the model parameters are continuously optimized through gradient descent and back propagation to obtain a comparative pre-training module, wherein the image encoder includes a basic network structure, which includes 2 residual blocks and 1 self-attention layer, and each residual block consists of 2 convolutional layers.

[0019] According to one aspect of the present invention, a face recognition system based on an image exposure correction network is provided, comprising: A facial image extraction unit, used for inputting the image to be detected into a facial image extraction network to obtain a face image; the facial image extraction network includes a HOG feature extraction module; An image exposure correction unit, used for inputting the obtained face image into an image exposure correction network to obtain a face image with normal exposure; the image exposure correction network includes a color space conversion module, an exposure feature extraction network, a brightness mapping network and a detail enhancement module; The face recognition unit is used to input the obtained exposed normal face image into the face recognition network to obtain the face recognition result; the face recognition network includes an image encoder based on SimCLR.

[0020] According to one aspect of the present invention specification, there is provided a face recognition device based on an image exposure correction network, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the face recognition method based on the image exposure correction network.

[0021] According to one aspect of the present specification, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute the steps of a face recognition method based on an image exposure correction network.

[0022] Compared with the prior art, the present invention has the following beneficial effects: The present invention adopts a face recognition model based on an image correction network to perform face recognition on images with abnormal exposure. The technology combines image restoration and deep learning algorithms. First, HOG is used to obtain the face image features contained in the original input image. Then the face image is converted into a color space from the RGB color space to the HSV color space. Then, an exposure feature extraction network is used to extract features from the face image. Then, a brightness mapping network is used to perform brightness mapping on the extracted face image feature data to obtain a normal exposure image with missing details. Then, a detail enhancement network model is used to enhance the details of the normal exposure image with missing details to obtain a normal exposure face image with enhanced details. Finally, a face recognition network is used to perform face recognition on the normal exposure face image with enhanced details to obtain a face recognition result. Compared with the existing face recognition model, the present invention has the advantages of stability, independence, fastness and efficiency, and can greatly improve the accuracy of face recognition under abnormal lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction is given below to the drawings used in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 is a flow chart of a method according to an embodiment of the present invention; Figure 2 This is a diagram of the image exposure correction network structure according to an embodiment of the present invention; Figure 3 This is a flow chart of exposure correction network training according to an embodiment of the present invention; Figure 4 This is a flow chart of the brightness mapping network training according to an embodiment of the present invention; Figure 5 This is a flow chart of face recognition network training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] It should be noted that: Since accurate recognition is also very important for face recognition in environments with abnormal lighting conditions, in order to improve the accuracy of face recognition in environments with abnormal lighting conditions, it is necessary to perform exposure image correction to improve the face recognition effect. To this end, the present invention proposes a face recognition method and system based on an image exposure correction network to solve the problem of low accuracy of the current face recognition model under abnormal lighting conditions, thereby enhancing the accuracy of the face recognition model.

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention are arbitrarily combined with each other to form a new technical solution. This combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in this field to achieve. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that this combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0027] This embodiment takes a given face recognition data set as an example to further illustrate the present invention. Figure 1 , this embodiment provides a face recognition method based on an image exposure correction network, comprising the following steps: Step 1, preprocessing the input image; Step 2, input the preprocessed image into the facial image extraction network to obtain a face image; Step 3, input the face image into the image exposure correction network to obtain a face image with normal exposure; Step 4: input the exposed normal face image into the face recognition network to obtain the face recognition result.

[0028] The facial image extraction network includes a HOG feature extraction module, whose input is an image to be detected and whose output is a face image.

[0029] Please see Figure 2 The image exposure correction network includes a color space conversion module, an exposure feature extraction network, a brightness mapping network and a detail enhancement module.

[0030] In one implementation, the color space conversion module inputs a human face RGB image and outputs a human face HSV image.

[0031] The exposure feature extraction network includes a downsampling layer, a random cropping module and an image encoder A module. The input dimension, input channel number and output channel number of the entire module are the same as the input image, and the output dimension is a hyperparameter in the design.

[0032] The downsampling layer includes 4 basic network structures and 1 sampling layer, and each basic network structure includes 2 residual blocks and 1 self-attention layer.

[0033] The image encoder A module includes a residual neural network and a self-attention layer.

[0034] The brightness mapping network has an input of an overexposed or underexposed image feature representation and an output of a normally exposed image.

[0035] The detail enhancement module has a detail-deficient image as input and a detail-enhanced image as output.

[0036] The face recognition network is composed of an image encoder B based on SimCLR and outputs face recognition results.

[0037] The SimCLR-based image encoder B includes 2 residual blocks and 1 self-attention layer.

[0038] The image exposure correction network generates a face image with normal exposure and inputs it into the face recognition network to output the face recognition result.

[0039] In one embodiment, in step 1, the input image is preprocessed, including normalization of the input image. The normalization uses scaling normalization to scale all sample data to the same scale and distribute them symmetrically about 0. The specific normalization function is:

[0040] Among them, y is the normalized image and x is the input noise image.

[0041] In one embodiment, in step 2, the facial image extraction network includes HOG feature extraction. The HOG extraction principle is to first grayscale the image, then calculate the gradient of the pixels in the image, and convert the image into HOG form, so that the position of the face in the image can be obtained.

[0042] In one implementation, in step 3, the color space conversion module converts the input RGB face image into an HSV image, and the specific calculation formula is:

[0043] In one implementation, in step 3, the exposure feature extraction network, whose downsampling layer is composed of 4 basic network structures and a sampling function, wherein the basic network structure includes 2 residual blocks and 1 self-attention layer; the downsampling function is composed of a convolution and a Rearrange function, and downsampling is performed by dimension merging; each residual block is composed of 2 convolution layers, and the residual block calculation unit can be expressed as:

[0044] Among them, y is the output of the residual block, is the output of the second convolutional layer in the residual block, and x is the input of the residual block. The residual structure is adopted so that the network "short-circuit" some layers when the depth is deep to ensure the effective transmission of information.

[0045] In one embodiment, in step 3, the exposure feature extraction network has a downsampling layer that adds a self-attention module, and the self-attention module includes three weight matrices for parameter optimization: , used to perform matrix multiplication with the input image to obtain 3 parameter matrices , the specific calculation formula is:

[0046] Among them, x is the output feature of the convolutional layer in the residual neural network; So the weight matrix calculation formula for the entire image is:

[0047] in, for The variance of is the normalization function, and the normalization calculation formula is:

[0048] in, To find the vector The maximum value of .

[0049] Please see Figure 3 In one embodiment, in step 3, the exposure feature extraction network, whose image encoder A module is composed of a residual neural network and a self-attention layer. The training process of the exposure feature extraction network includes the following sub-steps: Step S1, importing an exposure error image dataset; The exposure error image dataset uses Exposure-Errors Dataset and DEPD; Step S2, inputting the exposure error image and the exposure normal image into the same downsampling layer respectively; Input the exposure error image and the exposure normal image into the same downsampling layer respectively. The structure of the downsampling layer is the same as that of the downsampling layer described in step 3 to ensure the consistency of the training environment and the application environment; Step S3, inputting the downsampled exposure error image and the normal exposure image and the original exposure error image and the normal exposure image into a random cropping module respectively, wherein the random cropping module provides an image with a fixed resolution and enriches the diversity of training data; Step S4, the randomly cropped image is input into the image encoder A module, and the output of encoder A is projected into the feature space through the decoder, where the decoder is a multi-layer perceptron (MLP), which reduces the dimension of the representation generated by the encoder. The process can be expressed as:

[0050] Where x represents the input image, and denote the encoder A network and the decoder network respectively, h is the k-dimensional data output by the encoder (k is a hyperparameter in the design), and z is the feature vector of the output of encoder A projected into the feature space through the decoder; Step S5, input the training data into the image encoder A and the decoder network for training, and continuously optimize the model parameters through the cross entropy loss function with standardized temperature scaling; The cross entropy loss function expression of standardized temperature scaling is:

[0051] Where N is the number of images in the batch, is the indicator function, is the temperature parameter, Used to measure the similarity between a and b.

[0052] Please see Figure 4 In one embodiment, in step 3, the image exposure correction network, whose brightness mapping network input is the feature representation of the output of the above-mentioned exposure feature extraction network, and whose output is the image data with normal exposure, the training process of its brightness mapping network includes the following sub-steps: Step S1, importing an exposure error image dataset; The exposure error image dataset uses Exposure-Errors Dataset and DEPD; Step S2, the exposure error image and the exposure normal image are trained on the trained frozen exposure feature extraction network using a regularized linear regressor (ridge regression) to obtain a brightness mapping relationship.

[0053] In one embodiment, in step 3, the detail enhancement module of the image exposure correction network uses a Gaussian filter to smooth the image after brightness mapping to obtain a detail-enhanced image.

[0054] Please see Figure 5 In one embodiment, in step 4, the face recognition network is composed of an image encoder B based on SimCLR, and the training process of the face recognition network includes the following sub-steps: Step S1, importing face recognition data set; The face recognition dataset uses the Glint360K dataset; Step S2, performing random image enhancement on the face image; Perform random image enhancement on face images, including image rotation, image color change, image saturation change, image brightness change, scaling, cropping, etc. Randomly use two image enhancement methods to enhance image data; Step S3, inputting the face image into the image encoder B trained based on SimCLR, continuously optimizing the model parameters through gradient descent and back propagation, and obtaining a comparison pre-training module; Among them, SimCLR is an unsupervised learning method for learning image representation. The core idea is to train the neural network through contrastive learning, so that the model can learn to distinguish whether the views after image enhancement by multiple images come from the same original image.

[0055] The image encoder B consists of a basic network structure, which includes 2 residual blocks and 1 self-attention layer. Each residual block consists of 2 convolutional layers.

[0056] The implementation basis of each embodiment of the present invention is to implement programmed processing through a device with a processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present invention are encapsulated into various modules. Based on this reality, on the basis of the above embodiments, an embodiment of the present invention provides a face recognition system based on an image exposure correction network, which is used to execute a face recognition method based on an image exposure correction network in the above method embodiment.

[0057] The system comprises: a preprocessing unit, which is used to preprocess an input image to obtain a normalized image; a facial image extraction unit, which is used to input the preprocessed image into a facial image extraction network to obtain a face image; an image exposure correction unit, which is used to input the face image into an image exposure correction network to obtain an image with normal exposure; the exposure correction network comprises a color space conversion module, an exposure feature extraction network, a brightness mapping network and a detail enhancement module; the color space conversion module has an RGB face image as input and an HSV face image as output; the exposure feature extraction network comprises a downsampling layer, a random cropping module and an image encoder module, the input dimension, the number of input channels and the number of output channels of the entire module are the same as those of the input image, and the output dimension is a hyperparameter in the design; The downsampling layer includes 4 basic network structures and 1 sampling layer, each basic network structure includes 2 residual blocks and 1 self-attention layer; the image encoder A module includes a residual neural network and 1 self-attention layer; the brightness mapping network, whose input is an overexposed or underexposed image feature representation, and whose output is a normally exposed image; the detail enhancement module, whose input is a detail-missing image, and whose output is a detail-enhanced image; the face recognition unit, used to input the normally exposed image into the SimCLR-based image encoder B, and output the face recognition result; the SimCLR-based image encoder B includes 2 residual blocks and 1 self-attention layer; the image exposure correction network generates a normally exposed face image, and inputs it into the face recognition module, and outputs the face recognition result.

[0058] The specific implementation method of each unit is the same as each step and will not be described in detail in the present invention.

[0059] An embodiment of the present invention provides a face recognition system based on an image exposure correction network. Aiming at the problem that the current face recognition model has low accuracy under abnormal lighting conditions, the system adopts the several modules described above and uses a face recognition model based on an image correction network to perform face recognition on images with abnormal exposure, which can greatly improve the accuracy of face recognition under abnormal lighting conditions.

[0060] It should be noted that the system embodiments provided by the present invention are not only used to implement the methods in the above-mentioned method embodiments, but also used to implement the methods in other method embodiments provided by the present invention. The only difference lies in the setting of corresponding functional modules, and the principles thereof are basically the same as the principles of the above-mentioned system embodiments provided by the present invention. As long as technical personnel in this field refer to the specific technical solutions in other method embodiments on the basis of the above-mentioned system embodiments, obtain corresponding technical means and technical solutions composed of these technical means by combining technical features, and on the premise of ensuring the practicality of the technical solutions, they will improve the modules in the above-mentioned system embodiments to obtain corresponding system class embodiments for implementing the methods in other method class embodiments.

[0061] Based on the same inventive concept as the above embodiment, the embodiment of the present invention further provides a face recognition device based on an image exposure correction network, comprising: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement a face recognition method based on an image exposure correction network as described in any one of claims 1 to 10.

[0062] In summary, the present invention can realize face recognition under abnormal lighting conditions. Compared with the traditional face recognition model, the present invention has the advantages of stability, independence, rapidity and efficiency, and can greatly improve the accuracy of face recognition under abnormal lighting conditions, and has a good prospect for promotion and application.

[0063] The terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions, for example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to the steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A face recognition method based on an image exposure correction network, characterized in that: include: Inputting the image to be detected into a facial image extraction network to obtain a face image; the facial image extraction network includes a HOG feature extraction module; Inputting the obtained face image into an image exposure correction network to obtain a face image with normal exposure; the image exposure correction network includes a color space conversion module, an exposure feature extraction network, a brightness mapping network and a detail enhancement module; The obtained exposed normal face image is input into a face recognition network to obtain a face recognition result; the face recognition network includes an image encoder based on SimCLR.

2. According to claim 1, a face recognition method based on an image exposure correction network is characterized in that: The method further comprises: Acquire an image to be detected and preprocess the image to be detected.

3. According to claim 1, a face recognition method based on an image exposure correction network is characterized in that: The exposure feature extraction network includes a downsampling layer, a random cropping module and an image encoder module; the downsampling layer contains 4 basic network structures and 1 sampling layer, each basic network structure includes 2 residual blocks and 1 self-attention layer; the random cropping module is used to provide an image with a fixed resolution; the image encoder module includes a residual neural network and 1 self-attention layer.

4. According to claim 3, a face recognition method based on an image exposure correction network is characterized in that: The training of the exposure feature extraction network includes: Import the exposure error image dataset; Input the wrong exposure image and the normal exposure image into the same downsampling layer respectively; The downsampled exposure error image and exposure normal image as well as the original exposure error image and exposure normal image are input into the random cropping module respectively; The randomly cropped image is input into the image encoder module, and the output of the image encoder module is projected into the feature space through the decoder; The training data is input into the image encoder module and decoder network for training, and the model parameters are continuously optimized through the standardized temperature-scaled cross entropy loss function to output the trained exposure feature extraction network.

5. The face recognition method based on image exposure correction network according to claim 1, characterized in that: The training of the brightness mapping network includes: Import the exposure error image dataset; The exposure error image and the exposure normal image are trained on the trained frozen exposure feature extraction network using a regularized linear regressor to obtain the brightness mapping relationship.

6. The face recognition method based on image exposure correction network according to claim 1, characterized in that: The implementation of the detail enhancement module includes: The image after brightness mapping is smoothed by using a Gaussian filter to obtain an image with enhanced details.

7. The face recognition method based on image exposure correction network according to claim 1, characterized in that: The training of the face recognition network includes: Import face recognition dataset; Perform random image enhancement on face images; The enhanced face image is input into the image encoder trained based on SimCLR, and the model parameters are continuously optimized through gradient descent and back propagation to obtain a comparative pre-training module, wherein the image encoder includes a basic network structure, which includes 2 residual blocks and 1 self-attention layer, and each residual block consists of 2 convolutional layers.

8. A face recognition system based on an image exposure correction network, characterized in that: include: A facial image extraction unit, used for inputting the image to be detected into a facial image extraction network to obtain a face image; the facial image extraction network includes a HOG feature extraction module; An image exposure correction unit, used for inputting the obtained face image into an image exposure correction network to obtain a face image with normal exposure; the image exposure correction network includes a color space conversion module, an exposure feature extraction network, a brightness mapping network and a detail enhancement module; The face recognition unit is used to input the obtained exposed normal face image into the face recognition network to obtain the face recognition result; the face recognition network includes an image encoder based on SimCLR.

9. A face recognition device based on an image exposure correction network, characterized in that: include: one or more processors; And a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of a face recognition method based on an image exposure correction network as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, which enable the computer to execute the steps of a face recognition method based on an image exposure correction network as described in any one of claims 1 to 7.

Images