Face recognition method, computer readable storage medium and electronic device

Abstract

The application provides a face recognition method, a computer readable storage medium and an electronic device, the method comprising: obtaining an initial face image of a target user and a face infrared image of the target user; analyzing the initial face image using an illumination discrimination model to obtain an image illumination intensity; in the case where the image illumination intensity is less than or equal to a first illumination intensity, recognizing the face infrared image using an infrared imaging technology; in the case where the image illumination intensity is greater than or equal to a second illumination intensity, processing the initial face image and the face infrared image using a de-illumination model and a GAN network to obtain a target face image, and simultaneously recognizing the target face image and the face infrared image to obtain face information of the target user. The problem that the face cannot be accurately monitored under a high-illumination scene in the prior art, resulting in a low face recognition rate, is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of face recognition, in particular, to a face recognition method, a computer readable storage medium and an electronic device. BACKGROUND

[0002] With the advancement of technology, the face recognition accuracy in normal scenarios is very high at present, but in some scenarios, such as high light, facing the sun, lens reflection and other abnormal scenarios, the face recognition is not only low in recognition rate but also low in efficiency. In such abnormal scenarios, the face cannot be detected, resulting in inaccurate positioning and inability to recognize, and the experience is very poor. SUMMARY

[0003] The main purpose of the present application is to provide a face recognition method, a computer readable storage medium and an electronic device, so as to at least solve the problem that the face cannot be accurately detected in the high light scenario in the prior art, resulting in low face recognition rate.

[0004] In order to achieve the above purpose, according to one aspect of the present application, a face recognition method is provided, comprising: acquiring an initial face image of a target user and a face infrared image of the target user, the initial face image being acquired by a camera device; analyzing the initial face image by using a light discrimination model to obtain an image light intensity, the image light intensity being the light intensity of the initial face image captured; in the case that the image light intensity is less than or equal to a first light intensity, recognizing the face infrared image by using infrared imaging technology; in the case that the image light intensity is greater than the first light intensity and less than a second light intensity, directly recognizing the initial face image; in the case that the image light intensity is greater than or equal to the second light intensity, processing the initial face image and the face infrared image by using a de-light model and a GAN network to obtain a target face image, and simultaneously recognizing the target face image and the face infrared image to obtain face information of the target user, the first light intensity being less than the second light intensity, and the de-light model being used to reduce the image light intensity.

[0005] Optionally, in the case that the image light intensity is greater than or equal to the second light intensity, the initial face image and the face infrared image are processed by using a de-light model and a GAN network to obtain a target face image, including: reducing the image light intensity to a preset intensity range by using the de-light model to obtain a de-light image, the minimum value of the preset intensity range being the first light intensity, and the maximum value of the preset intensity range being the second light intensity; performing face reconstruction on the de-light image and the face infrared image by using a GAN network to obtain a reconstructed face image; and fusing the reconstructed face image and the initial face image to obtain the target face image.

[0006] Optionally, the face reconstruction on the de-light image and the face infrared image by using a GAN network to obtain a reconstructed face image includes: averaging and splitting the de-light image into a plurality of sub-images with the same pixel value, the pixel value of the sub-image being the same as the pixel value of the face infrared image, and the number of the sub-images being related to the pixel value of the initial face image and the pixel value of the face infrared image; inputting the face infrared image and each of the sub-images into the GAN network through different channels, respectively, and performing feature extraction on the face infrared image and each of the sub-images to obtain a plurality of groups of feature points, the feature points being used to at least represent the relative positional relationship between each of the sub-images; and performing face reconstruction according to the face infrared image, each group of feature points, and each of the sub-images to obtain the reconstructed face image.

[0007] Optionally, the de-light image is obtained by reducing the image light intensity to a preset intensity range by using the de-light model, including: constructing a de-light model, wherein the de-light model is trained by using a plurality of groups of training data, each of the plurality of groups of training data including a real face image and a rendered image corresponding to the real face image acquired in a historical time period, the light intensity of the real face image being in the preset intensity range, the real face image being a real face image actually photographed, the rendered image corresponding to the real face image being a rendered image obtained by 3D rendering on the real face image, and the light intensity of the rendered image being greater than or equal to the second light intensity; and reducing the image light intensity to the preset intensity range by using the de-light model to obtain the de-light image.

[0008] Optionally, before the image illumination intensity is reduced to a preset intensity range by using the de-illumination model to obtain a de-illumination image, the method further comprises: fusing the real face image and a rendering image corresponding to the real face image to obtain a fused image, a pixel value of the fused image being a sum of a first pixel value and a second pixel value, the first pixel value being a product of a first coefficient and a pixel value of the real face image, and the second pixel value being a product of a second coefficient and a pixel value of the rendering image corresponding to the real face image; processing the fused image by using a generative network to generate a plurality of initial prediction images; performing adversarial training on the plurality of initial prediction images and a plurality of labels to obtain a target prediction image, the illumination intensity of the target prediction image being within the preset intensity range, the labels corresponding one-to-one to the initial prediction images, and the target prediction image being used to predict the de-illumination image of the initial face image.

[0009] Optionally, fusing the reconstructed face image and the initial face image to obtain the target face image comprises: obtaining a first target coefficient and a second target coefficient, a sum of the first target coefficient and the second target coefficient being 1; determining a product of a pixel value of the reconstructed face image and the first target coefficient as a first target pixel value; determining a product of a pixel value of the initial face image and the second target coefficient as a second target pixel value; and determining a sum of the first target pixel value and the second target pixel value as a pixel value of the target face image to obtain the target face image.

[0010] Optionally, obtaining the first target coefficient and the second target coefficient comprises: obtaining a plurality of training coefficient groups, each of the training coefficient groups including two training coefficients, and a sum of the two training coefficients in one of the training coefficient groups being 1; inputting each of the training coefficient groups into an identification training model to identify a same training face image to obtain a plurality of identification results, the identification results corresponding one-to-one to the training coefficient groups; determining an identification rate corresponding to each of the training coefficient groups according to each of the identification results; and determining two coefficients in the training coefficient group with the highest identification rate as the first target coefficient and the second target coefficient, respectively.

[0011] Optionally, the target face image and the face infrared image are recognized at the same time to obtain face information of the target user, including: inputting the target face image and the face infrared image into a face recognition network, the face recognition network being a neural network for face recognition obtained at least according to historical face data; extracting a face feature map from the target face image and the face infrared image, the face feature map being a feature map including a plurality of feature points; inputting the face feature map into a classification recognition model to obtain the face information of the target user, the classification recognition model being a recognition model obtained at least according to historical user data, the historical user data at least including user information of the target user.

[0012] According to another aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium including a stored program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute any one of the face recognition methods when the program runs.

[0013] According to another aspect of the present application, an electronic device is provided, including one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include a program for executing any one of the face recognition methods.

[0014] According to the technical solution of the present application, the face recognition method described above first acquires an initial face image of a target user and a face infrared image of the target user, the initial face image being acquired by a camera device; then analyzes the initial face image using an illumination discrimination model to obtain an image illumination intensity, the image illumination intensity being the illumination intensity of the initial face image captured; finally, in the case that the image illumination intensity is less than or equal to a first illumination intensity, recognizes the face infrared image using infrared imaging technology; in the case that the image illumination intensity is greater than the first illumination intensity and less than a second illumination intensity, directly recognizes the initial face image; in the case that the image illumination intensity is greater than or equal to the second illumination intensity, processes the initial face image and the face infrared image using a de-illumination model and a GAN network to obtain a target face image, and simultaneously recognizes the target face image and the face infrared image to obtain face information of the target user, the first illumination intensity being less than the second illumination intensity, and the de-illumination model being used to reduce the image illumination intensity. This method uses an infrared face image and part of a face image captured by a camera to regenerate a face, and simultaneously inputs the infrared face image into a detection and recognition model, thereby maximizing the retention of real face features and improving the face recognition rate under high light, and solving the problem of low face recognition rate in the prior art due to the inability to accurately monitor a face under high light. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which form a part of this specification, are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification. The embodiments of this application, and their

[0016] Figure 1 A hardware structure block diagram of a mobile terminal for performing a face recognition method according to an embodiment of the present application is shown;

[0017] Figure 2 A flowchart of a face recognition method according to an embodiment of the present application is shown;

[0018] Figure 3 A schematic diagram of a ResNet network according to an embodiment of the present application is shown;

[0019] Figure 4 A flowchart of another face recognition method according to an embodiment of the present application is shown;

[0020] Figure 5 A structure block diagram of a face recognition device according to an embodiment of the present application is shown.

[0021] Among the above-mentioned drawings, the following reference signs are included:

[0022] 102, processor; 104, memory; 106, transmission device; 108, input / output device. DETAILED DESCRIPTION

[0023] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict. The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0024] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] For ease of description, the following describes some nouns or terms related to the embodiments of the present application:

[0027] GAN: Generative adversarial network (GAN) can use a generator and a discriminator to train the input data in an adversarial manner to achieve the desired effect, including style conversion, image generation, etc.

[0028] Style GAN: A new generator network structure inspired by style transfer. The new network structure can decouple and separate the high-level semantic attributes of the image through unsupervised automatic learning, such as the pose and identity of a human face image, and the random changes of the generated image such as freckles and hair. To some extent, it can also control the synthesis;

[0029] Res Net: A common feature extraction and classification model, a deep learning network that uses a deep network to extract high-dimensional features from incoming images, and then classifies the features.

[0030] As introduced in the background, the face recognition in the prior art in the abnormal scene of high light, facing the sun, lens reflection, etc. is not only low in recognition rate but also low in efficiency. To solve the problem that the face cannot be accurately detected in the high light scene in the prior art, resulting in a low face recognition rate, the embodiments of the present application provide a face recognition method, a computer readable storage medium and an electronic device.

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application.

[0032] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking the running on a mobile terminal as an example, Figure 1 is a hardware structure block diagram of a mobile terminal of a face recognition method according to an embodiment of the present application. As Figure 1As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0033] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the face recognition method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] This embodiment provides a face recognition method that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] Figure 2 This is a flowchart of a face recognition method according to an embodiment of this application. Figure 2As shown, the method comprises the following steps:

[0036] In step S201, an initial face image of a target user and a face infrared image of the target user are acquired, and the initial face image is collected by a camera device;

[0037] Specifically, the initial face image is obtained based on a face photo taken by the camera device with the face of the target user directly facing the camera. The initial face image is generally an image with a size of 512*512 pixels, and the face infrared image is generally an infrared image with a size of 256*256 pixels. The pixel values of the initial face image and the face infrared image are not fixed and can be adjusted according to actual conditions as long as they are powers of 2, and the pixel values used in this embodiment are data with high face recognition accuracy trained according to historical data.

[0038] In step S202, an illumination discrimination model is used to analyze the initial face image to obtain an image illumination intensity, and the image illumination intensity is the illumination intensity of the initial face image taken.

[0039] Specifically, the illumination discrimination model is also trained according to historical data. The historical data for training the illumination discrimination model is images with different illumination intensities and the illumination intensities of the images in a historical time period. The illumination discrimination model can quickly distinguish the face recognition application scene, provide a specific face recognition scheme for each scene, and improve the corresponding accuracy.

[0040] In step S203, in the case where the image illumination intensity is less than or equal to a first illumination intensity, the face infrared image is recognized by using infrared imaging technology; in the case where the image illumination intensity is greater than the first illumination intensity and less than a second illumination intensity, the initial face image is directly recognized; in the case where the image illumination intensity is greater than or equal to the second illumination intensity, the initial face image and the face infrared image are processed by using a de-illumination model and a GAN network to obtain a target face image, and the target face image and the face infrared image are simultaneously recognized to obtain face information of the target user, the first illumination intensity is less than the second illumination intensity, and the de-illumination model is used to reduce the image illumination intensity.

[0041] Specifically, the image with the image illumination intensity less than or equal to the first illumination intensity is taken in an irrelevant illumination or low illumination (i.e., a dark scene), the image with the image illumination intensity greater than the first illumination intensity and less than the second illumination intensity is taken in normal illumination, and the image with the image illumination intensity greater than or equal to the second illumination intensity is taken in a high-illumination scene such as facing the sun or lens reflection.

[0042] In the case where the image light intensity is greater than or equal to the second light intensity, the initial face image and the face infrared image are processed by using the de-light model and the GAN network to obtain the specific implementation steps of the target face image as follows:

[0043] In step S301, the image light intensity is reduced to a preset intensity range by using the de-light model to obtain a de-light image, the minimum value of the preset intensity range is the first light intensity, and the maximum value of the preset intensity range is the second light intensity.

[0044] The specific implementation steps of step S301 are as follows:

[0045] In step S3011, a de-light model is constructed, wherein the de-light model is trained by using a plurality of sets of training data, each set of training data in the plurality of sets of training data includes a real face image and a rendered image corresponding to the real face image obtained in a historical time period, the light intensity of the real face image is in the preset intensity range, the real face image is a real face image actually photographed, the rendered image corresponding to the real face image is a rendered image obtained by 3D rendering on the real face image, and the light intensity of the rendered image is greater than or equal to the second light intensity.

[0046] In step S3012, the de-light model is used to reduce the image light intensity to the preset intensity range to obtain the de-light image.

[0047] Specifically, the light intensity of the initial face image can be effectively reduced, and the face recognition accuracy under high intensity can be improved, and the face recognition accuracy can be relatively improved by about 9%. The 3D rendering method is used to generate light effects on the face image under strong light, a large amount of training data is provided for the de-light model, the accuracy of the de-light model can be improved, and the face recognition accuracy under high light can be improved. The face detection accuracy is improved by combining the GAN network with the light discrimination model.

[0048] Before the image light intensity is reduced to the preset intensity range by using the de-light model to obtain the de-light image, the method further includes the following steps:

[0049] In step S401, the real face image and the rendered image corresponding to the real face image are fused to obtain a fused image, the pixel value of the fused image is the sum of a first pixel value and a second pixel value, the first pixel value is the product of a first coefficient and the pixel value of the real face image, and the second pixel value is the product of a second coefficient and the pixel value of the rendered image corresponding to the real face image.

[0050] Step S402, using a generation network to process the above-mentioned fusion image to generate a plurality of initial prediction images;

[0051] Step S403, using a plurality of the above-mentioned initial prediction images and a plurality of labels to perform adversarial training to obtain a target prediction image, the illumination intensity of the target prediction image being within the above-mentioned preset intensity range, the labels corresponding one-to-one to the initial prediction images, and the target prediction image being used to predict the above-mentioned de-illumination image of the initial face image.

[0052] Specifically, this can obtain a more accurate de-illumination image and improve the accuracy of the de-illumination image. The steps S401-S403 are specific training steps for constructing a de-illumination model. The embodiment uses the method in the DeepAdversarial Decomposition paper and independently designs a training set. The training set is composed of a 3D rendering image with strong light and a real picture with normal light. The fusion image is used to generate two prediction images x1 and x2 through a generation network. The labels X1 and X2 are used to perform adversarial training through a designed Crossroad L1 loss. In this way, the face image without illumination can be predicted. The specific network structure can be replaced by other GAN networks.

[0053] Step S302, using a GAN network to perform face reconstruction on the above-mentioned de-illumination image and the above-mentioned face infrared image to obtain a reconstructed face image;

[0054] The specific implementation steps of step S302 are as follows:

[0055] Step S3021, the above-mentioned de-illumination image is evenly split into a plurality of sub-images with the same pixel value, the pixel value of the sub-image being the same as the pixel value of the face infrared image, and the number of the sub-images being related to the pixel value of the initial face image and the pixel value of the face infrared image;

[0056] Step S3022, the face infrared image and each of the sub-images are input into the GAN network through different channels, and feature points are extracted from the face infrared image and each of the sub-images to obtain a plurality of groups of feature points, the feature points being used to at least represent the relative positional relationship between each of the sub-images;

[0057] Step S3023, face reconstruction is performed according to the face infrared image, each group of feature points, and each of the sub-images to obtain the reconstructed face image.

[0058] Specifically, this can maximize the features of the face, improve the face recognition accuracy under high intensity, and specifically can improve the face recognition accuracy by about 4%. The number of sub-images is related to the pixel value of the initial face image and the pixel value of the face infrared image, for example: the initial face image is 512x512 pixels, and the face infrared image is 256x256 pixels. The average split of the de-light image into four 256x256 pixel sub-images ensures that the pixel values of the sub-images and the face infrared image are the same, and on this basis, the face infrared image and the four sub-images are combined into a 5-channel image, the features are extracted, and Style GAN is used for related face reconstruction.

[0059] The reconstruction network used in the above embodiment is Style GAN, and other reconstruction networks can also be used for face reconstruction. The use of Style GAN in the above embodiment can improve the accuracy of face recognition. In addition, in the above embodiment, the initial face image is 512x512 pixels, the face infrared image is 256x256 pixels, and four equal divisions can ensure high face recognition accuracy with simple calculation. In actual application, different pixel values can be used for multiple equal divisions for experiments.

[0060] Step S303, fusing the reconstructed face image and the initial face image to obtain the target face image.

[0061] Specifically, the face infrared image and the initial face image captured by the camera are used for face reconstruction to regenerate the face, and then the regions are fused to maximize the retention of real face features. Using GAN network to recover the face using infrared image and part of the real face picture can solve the problem of face recognition under high light conditions.

[0062] The specific implementation steps of step S303 are as follows:

[0063] Step S3031, obtaining a first target coefficient and a second target coefficient, the sum of the first target coefficient and the second target coefficient is 1;

[0064] The specific implementation steps of step S3031 are as follows:

[0065] Step S501, obtaining a plurality of training coefficient groups, each of the training coefficient groups includes two training coefficients, and the sum of the two training coefficients in one of the training coefficient groups is 1;

[0066] Step S502, inputting each of the training coefficient groups into a recognition training model to recognize the same training face image to obtain a plurality of recognition results, the recognition results correspond one-to-one to the training coefficient groups.

[0067] Step S503, determining the recognition rate corresponding to each of the training coefficient groups according to each of the above recognition results;

[0068] Step S504, determining two coefficients in the training coefficient group with the highest recognition rate as the first target coefficient and the second target coefficient respectively.

[0069] Specifically, the proportional fusion of the generated image and the visible light image can preserve the real face features and improve the accuracy of detection and recognition. The recognition rate can be obtained by the matching rate of the face image and the user data, the first target coefficient can be 0.31, and the second target coefficient can be 0.69. The above coefficients are the coefficient group with the highest recognition rate in the training data.

[0070] Step S3032, determining the product of the pixel value of the reconstructed face image and the first target coefficient as the first target pixel value;

[0071] Step S3033, determining the product of the pixel value of the initial face image and the second target coefficient as the second target pixel value;

[0072] Step S3034, determining the sum of the first target pixel value and the second target pixel value as the pixel value of the target face image to obtain the target face image.

[0073] Specifically, the above embodiment can compensate for the invisible face in the highlight area.

[0074] Wherein, the target face image and the face infrared image are simultaneously recognized to obtain the face information of the target user, and the specific implementation steps are as follows:

[0075] Step S601, inputting the target face image and the face infrared image into a face recognition network, wherein the face recognition network is a neural network for face recognition trained at least according to historical face data;

[0076] Step S602, extracting a face feature map from the target face image and the face infrared image, wherein the face feature map is a feature map including a plurality of feature points;

[0077] Step S603, inputting the face feature map into a classification recognition model to obtain the face information of the target user, wherein the classification recognition model is a recognition model trained at least according to historical user data, and the historical user data at least includes user information of the target user.

[0078] Specifically, since the target face image is obtained after processing, generation and fusion, although the accuracy is high, there will still be a small amount of error, and the face infrared image is the face data of the user directly obtained, so the accuracy is extremely high. Combining the target face image and the face infrared image for detection and recognition can maximize the retention of real face features and improve the accuracy of face recognition.

[0079] The face recognition network is a commonly used Res Net network, and the specific network structure is as shown in the following figure: Figure 3 In the figure, 1x1 represents a convolution operation with a convolution kernel of 1*1, 3x3 represents a convolution operation with a convolution kernel of 3*3, "+" represents direct combination (concat) of feature maps, and 0.5x represents up-sampling, that is, the length and width of the feature map are halved after each convolution. In the figure, the target face image and the face infrared image are input into the face recognition network, and P6 output is the face feature map in the above embodiment. Finally, the face feature map is input into a classification recognition model for recognition. In this way, the face feature map that maximizes the retention of real face features can be obtained according to the target face image and the face infrared image. After the face feature map is recognized, the information of the target user can be obtained more accurately.

[0080] The face recognition method of the present application first acquires an initial face image of a target user and a face infrared image of the target user, and the initial face image is acquired by a camera device. Then, a light discrimination model is used to analyze the initial face image to obtain an image light intensity, which is the light intensity of the initial face image captured. Finally, in the case where the image light intensity is less than or equal to a first light intensity, the face infrared image is recognized by using infrared imaging technology; in the case where the image light intensity is greater than the first light intensity and less than a second light intensity, the initial face image is directly recognized; and in the case where the image light intensity is greater than or equal to the second light intensity, the initial face image and the face infrared image are processed by using a de-lighting model and a GAN network to obtain a target face image, and the target face image and the face infrared image are simultaneously recognized to obtain face information of the target user. The first light intensity is less than the second light intensity, and the de-lighting model is used to reduce the image light intensity. This method uses the infrared face image and part of the face image acquired by the camera to regenerate the face, and simultaneously inputs the infrared face image into the detection and recognition model, thereby maximizing the retention of real face features and improving the face recognition rate under high light, and solving the problem of low face recognition rate in the prior art under high light.

[0081] In order for those skilled in the art to more clearly understand the technical solutions of the present application, the implementation process of the face recognition method of the present application will be described in detail below in conjunction with specific embodiments.

[0082] The embodiment relates to a specific face recognition method, as shown in the following steps. Figure 4 The embodiment relates to a specific face recognition method, as shown in the following steps.

[0083] Step S1: face photos are normalized to 512*512 pixels, and an infrared image is obtained and normalized to 256*256 pixels;

[0084] Step S2: the face pictures are input into an illumination discrimination model, and it is distinguished that the photographed photos belong to the type of illumination, if it is normal illumination, face recognition is directly performed; if it is no illumination or a dark scene, face recognition is directly performed by using infrared imaging technology; if it is a high-illumination scene, step S3 is performed.

[0085] Step S3: a GAN network is used to perform illumination removal on the face pictures, and the illumination-removed images are cross-quadrant divided, and each part image is 256*256 pixels;

[0086] Step S4: a Style GAN is used to perform face reconstruction on the infrared image and the part images divided in the quadrants, and a reconstructed face image is obtained.

[0087] Step S5: the reconstructed 256*256 face image and the 512*512 face photo image in step S1 are fused, finally, the fused image and the infrared image are input into a face recognition system as input to perform face recognition.

[0088] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0089] The embodiment of the present application also provides a face recognition device, and it should be noted that the face recognition device of the embodiment of the present application can be used to execute the face recognition method provided by the embodiment of the present application. The device is used to realize the above-mentioned embodiment and preferred embodiment, and details are not repeated. As used below, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the device described in the following embodiment is preferably realized in software, hardware, or a combination of software and hardware is also possible and is conceived.

[0090] The face recognition device provided by the embodiment of the present application is introduced below.

[0091] Figure 5 FIG. 1 is a schematic diagram of a face recognition device according to the embodiment of the present application. As shown in Figure 5As shown, the device comprises an acquisition unit 10, an analysis unit 20 and an identification unit 30, the acquisition unit 10 is used to acquire the initial face image of the target user and the face infrared image of the target user, the initial face image is collected by a camera device; the analysis unit 20 is used to analyze the initial face image by using an illumination discrimination model, to obtain the image illumination intensity, the image illumination intensity is the illumination intensity of the initial face image shot; the identification unit 30 is used to identify the face infrared image by using infrared imaging technology in the case that the image illumination intensity is less than or equal to the first illumination intensity; directly identify the initial face image in the case that the image illumination intensity is greater than the first illumination intensity and less than the second illumination intensity; in the case that the image illumination intensity is greater than or equal to the second illumination intensity, the initial face image and the face infrared image are processed by using a de-illumination model and a GAN network to obtain a target face image, and the target face image and the face infrared image are identified at the same time to obtain the face information of the target user, the first illumination intensity is less than the second illumination intensity, and the de-illumination model is used to reduce the image illumination intensity.

[0092] The face recognition device of the present application comprises a first acquisition unit, an analysis unit and an identification unit, the first acquisition unit is used to acquire the initial face image of the target user and the face infrared image of the target user, the initial face image is collected by a camera device; the analysis unit is used to analyze the initial face image by using an illumination discrimination model, to obtain the image illumination intensity, the image illumination intensity is the illumination intensity of the initial face image shot; the identification unit is used to identify the face infrared image by using infrared imaging technology in the case that the image illumination intensity is less than or equal to the first illumination intensity; directly identify the initial face image in the case that the image illumination intensity is greater than the first illumination intensity and less than the second illumination intensity; in the case that the image illumination intensity is greater than or equal to the second illumination intensity, the initial face image and the face infrared image are processed by using a de-illumination model and a GAN network to obtain a target face image, and the target face image and the face infrared image are identified at the same time to obtain the face information of the target user, the first illumination intensity is less than the second illumination intensity, and the de-illumination model is used to reduce the image illumination intensity. The device uses the infrared face image and part of the face image collected by the camera to regenerate the face, and at the same time combines the infrared face image to input the detection and identification model, to maximize the retention of real face features, improve the face recognition rate under high light, and solve the problem of low face recognition rate in the prior art under high light.

[0093] In an optional solution, the identifying unit comprises a de-illumination module, a reconstruction module and a first fusion module. The de-illumination module is configured to reduce the illumination intensity of the image to a preset intensity range by using the de-illumination model, to obtain a de-illumination image. The minimum value of the preset intensity range is the first illumination intensity, and the maximum value of the preset intensity range is the second illumination intensity. The reconstruction module is configured to perform face reconstruction on the de-illumination image and the face infrared image by using a GAN network, to obtain a reconstructed face image. The first fusion module is configured to fuse the reconstructed face image and the initial face image, to obtain the target face image.

[0094] In an optional example, the reconstruction module comprises a splitting module, a first input module and a reconstruction submodule. The splitting module is configured to split the de-illumination image into a plurality of sub-images with the same pixel value. The pixel value of the sub-image is the same as the pixel value of the face infrared image. The number of the sub-images is related to the pixel value of the initial face image and the pixel value of the face infrared image. The first input module is configured to input the face infrared image and each of the sub-images into the GAN network by using different channels, and to extract features of the face infrared image and each of the sub-images, to obtain a plurality of groups of feature points. The feature points are used to represent the relative positional relationship between each of the sub-images. The reconstruction submodule is configured to perform face reconstruction according to the face infrared image, each group of feature points and each of the sub-images, to obtain the reconstructed face image. The real face features can be retained in the largest scale. The face recognition problem under high light conditions can be solved.

[0095] As an optional embodiment, the de-illumination module comprises a construction module and a de-illumination submodule. The construction module is configured to construct a de-illumination model. The de-illumination model is trained by using a plurality of groups of training data. Each group of training data comprises a real face image and a corresponding rendered image obtained in a historical time period. The illumination intensity of the real face image is within the preset intensity range. The real face image is a real photographed face image. The corresponding rendered image of the real face image is a rendered image obtained by 3D rendering on the real face image. The illumination intensity of the rendered image is greater than or equal to the second illumination intensity. The de-illumination submodule is configured to reduce the illumination intensity of the image to the preset intensity range by using the de-illumination model, to obtain the de-illumination image. The illumination intensity of the initial face image can be effectively reduced, the face recognition accuracy under high intensity can be improved, and the face recognition accuracy can be relatively improved by about 9%.

[0096] Exemplarily, the device further comprises a second fusion module, a processing module and a training module. The second fusion module is configured to fuse the real face image and the rendered image corresponding to the real face image to obtain a fused image before reducing the image illumination intensity to a preset intensity range by using the above-mentioned de-illumination model to obtain a de-illumination image, wherein a pixel value of the fused image is a sum of a first pixel value and a second pixel value, the first pixel value is a product of a first coefficient and a pixel value of the real face image, and the second pixel value is a product of a second coefficient and a pixel value of the rendered image corresponding to the real face image. The processing module is configured to process the fused image by using a generative network to generate a plurality of initial prediction images. The training module is configured to perform adversarial training by using a plurality of the initial prediction images and a plurality of labels to obtain a target prediction image, wherein an illumination intensity of the target prediction image is within the preset intensity range, the labels correspond one-to-one to the initial prediction images, and the target prediction image is used to predict the de-illumination image of the initial face image. A more accurate de-illumination image can be obtained, and the accuracy of the de-illumination image is improved.

[0097] In the embodiment, the first fusion module comprises a first acquisition module, a first determination module, a second determination module and a third determination module. The first acquisition module is configured to acquire a first target coefficient and a second target coefficient, wherein a sum of the first target coefficient and the second target coefficient is 1. The first determination module is configured to determine a product of a pixel value of the reconstructed face image and the first target coefficient as a first target pixel value. The second determination module is configured to determine a product of a pixel value of the initial face image and the second target coefficient as a second target pixel value. The third determination module is configured to determine a sum of the first target pixel value and the second target pixel value as a pixel value of the target face image, so as to obtain the target face image. The situation that the face in the highlight area is invisible can be compensated.

[0098] In an optional solution, the first acquisition module comprises a second acquisition module, an identification module, a fourth determination module and a fifth determination module. The first acquisition module is configured to acquire a plurality of training coefficient groups, each of the training coefficient groups comprising two training coefficients, and a sum of the two training coefficients in one of the training coefficient groups being 1. The identification module is configured to input each of the training coefficient groups into an identification training model to identify the same training face image to obtain a plurality of identification results, wherein the identification results correspond one-to-one to the training coefficient groups. The fourth determination module is configured to determine an identification rate corresponding to each of the training coefficient groups according to each of the identification results. The fifth determination module is configured to determine the two coefficients in the training coefficient group with the highest identification rate as the first target coefficient and the second target coefficient, respectively. The real face features can be retained to improve the accuracy of detection and identification.

[0099] As an optional solution, the recognition unit comprises a second input module, an extraction module and a third input module, the second input module is configured to input the target face image and the face infrared image into a face recognition network, the face recognition network is a neural network for face recognition trained at least according to historical face data; the extraction module is configured to extract a face feature map from the target face image and the face infrared image, the face feature map is a feature map comprising a plurality of feature points; and the third input module is configured to input the face feature map into a classification recognition model to obtain face information of the target user, the classification recognition model is a recognition model trained at least according to historical user data, and the historical user data at least comprises user information of the target user. The real face features can be retained in the largest scale, and the accuracy of face recognition is improved.

[0100] The face recognition device comprises a processor and a memory, the acquisition unit and the like are stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory. The modules are located in the same processor, or the modules are located in different processors in any combination.

[0101] The processor comprises a core, and the core retrieves the corresponding program unit from the memory. The core can be one or more, and the problem that the face cannot be accurately detected in a high-illumination scene in the prior art, resulting in a low face recognition rate, can be solved by adjusting the core parameters.

[0102] The memory can comprise a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0103] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium comprises a stored program, wherein the program controls a device where the computer readable storage medium is located to execute the face recognition method when the program is running.

[0104] Specifically, the face recognition method comprises:

[0105] In step S201, an initial face image of a target user and a face infrared image of the target user are acquired, and the initial face image is obtained by a camera device;

[0106] Specifically, the initial face image is obtained based on a photograph of a face of a target user taken by a camera device with the face of the target user facing the camera. The initial face image is generally an image with a resolution of 512x512 pixels, and the face infrared image is generally an infrared image with a resolution of 256x256 pixels. The pixel values of the initial face image and the face infrared image are not fixed and can be adjusted according to actual conditions as long as they are powers of 2, and the pixel values used in this embodiment are data with high accuracy of face recognition obtained by training based on historical data.

[0107] In step S202, the initial face image is analyzed by using the illumination discrimination model to obtain an image illumination intensity, which is the illumination intensity of the initial face image taken.

[0108] Specifically, the illumination discrimination model is also obtained by training based on historical data. The historical data for training the illumination discrimination model includes images with different illumination intensities in a historical time period and the illumination intensities of the images. The classification model of illumination is used to quickly distinguish the application scenarios of face recognition, provide a specific face recognition scheme for each scenario, and improve the corresponding accuracy.

[0109] In step S203, in a case where the image illumination intensity is less than or equal to a first illumination intensity, the face infrared image is recognized by using infrared imaging technology; in a case where the image illumination intensity is greater than the first illumination intensity and less than a second illumination intensity, the initial face image is directly recognized; and in a case where the image illumination intensity is greater than or equal to the second illumination intensity, the initial face image and the face infrared image are processed by using a de-illumination model and a GAN network to obtain a target face image, and the target face image and the face infrared image are simultaneously recognized to obtain face information of the target user. The first illumination intensity is less than the second illumination intensity, and the de-illumination model is used to reduce the image illumination intensity.

[0110] Specifically, the image with the image illumination intensity less than or equal to the first illumination intensity is taken in a case of irrelevant illumination or low illumination (i.e., a dark scene), the image with the image illumination intensity greater than the first illumination intensity and less than the second illumination intensity is taken in a case of normal illumination, and the image with the image illumination intensity greater than or equal to the second illumination intensity is taken in a case of high illumination such as sunlight or lens reflection.

[0111] Optionally, in the case that the image light intensity is greater than or equal to the second light intensity, the initial face image and the face infrared image are processed by using a de-light model and a GAN network to obtain a target face image, including: reducing the image light intensity to a preset intensity range by using the de-light model, to obtain a de-light image, the minimum value of the preset intensity range is the first light intensity, and the maximum value of the preset intensity range is the second light intensity; performing face reconstruction on the de-light image and the face infrared image by using the GAN network to obtain a reconstructed face image; and fusing the reconstructed face image and the initial face image to obtain the target face image.

[0112] Optionally, the face reconstruction on the de-light image and the face infrared image by using the GAN network to obtain a reconstructed face image includes: averaging and splitting the de-light image into a plurality of sub-images with the same pixel value, the pixel value of the sub-image is the same as the pixel value of the face infrared image, and the number of the sub-images is related to the pixel value of the initial face image and the pixel value of the face infrared image; inputting the face infrared image and each sub-image into the GAN network through different channels, respectively, and performing feature extraction on the face infrared image and each sub-image to obtain a plurality of groups of feature points, the feature points are used to represent at least the relative position relationship between each sub-image; and performing face reconstruction according to the face infrared image, each group of feature points, and each sub-image to obtain the reconstructed face image.

[0113] Optionally, the de-light image is obtained by reducing the image light intensity to a preset intensity range by using the de-light model, including: constructing a de-light model, wherein the de-light model is trained by using a plurality of groups of training data, each group of training data in the plurality of groups of training data includes a real face image and a rendered image corresponding to the real face image acquired in a historical time period, the light intensity of the real face image is in the preset intensity range, the real face image is a real face image actually photographed, the rendered image corresponding to the real face image is a rendered image obtained by 3D rendering on the real face image, and the light intensity of the rendered image is greater than or equal to the second light intensity; and the de-light image is obtained by reducing the image light intensity to the preset intensity range by using the de-light model.

[0114] Optionally, before the image illumination intensity is reduced to the preset intensity range by using the above-mentioned de-illumination model to obtain the de-illumination image, the method further comprises: fusing the real human face image and the rendering image corresponding to the real human face image to obtain a fused image, the pixel value of the fused image being the sum of a first pixel value and a second pixel value, the first pixel value being the product of a first coefficient and the pixel value of the real human face image, and the second pixel value being the product of a second coefficient and the pixel value of the rendering image corresponding to the real human face image; processing the fused image by using a generative network to generate a plurality of initial prediction images; performing adversarial training on the plurality of initial prediction images and a plurality of labels to obtain a target prediction image, the illumination intensity of the target prediction image being within the preset intensity range, the labels corresponding one-to-one to the initial prediction images, and the target prediction image being used to predict the de-illumination image of the initial human face image.

[0115] Optionally, fusing the reconstructed human face image and the initial human face image to obtain the target human face image comprises: obtaining a first target coefficient and a second target coefficient, the sum of the first target coefficient and the second target coefficient being 1; determining a first target pixel value as the product of the pixel value of the reconstructed human face image and the first target coefficient; determining a second target pixel value as the product of the pixel value of the initial human face image and the second target coefficient; and determining the pixel value of the target human face image as the sum of the first target pixel value and the second target pixel value to obtain the target human face image.

[0116] Optionally, obtaining the first target coefficient and the second target coefficient comprises: obtaining a plurality of training coefficient groups, each of the training coefficient groups comprising two training coefficients, the sum of the two training coefficients in one of the training coefficient groups being 1; inputting each of the training coefficient groups into an identification training model to identify the same training human face image to obtain a plurality of identification results, the identification results corresponding one-to-one to the training coefficient groups; determining an identification rate corresponding to each of the training coefficient groups according to each of the identification results; and determining the two coefficients in the training coefficient group with the highest identification rate as the first target coefficient and the second target coefficient, respectively.

[0117] Optionally, the target face image and the face infrared image are recognized at the same time to obtain face information of the target user, including: inputting the target face image and the face infrared image into a face recognition network, the face recognition network being a neural network for face recognition trained at least according to historical face data; extracting a face feature map from the target face image and the face infrared image, the face feature map being a feature map including a plurality of feature points; inputting the face feature map into a classification recognition model to obtain the face information of the target user, the classification recognition model being a recognition model trained at least according to historical user data, and the historical user data at least including user information of the target user.

[0118] The embodiment of the application provides a processor for running a program, wherein the processor is used for executing the face recognition method.

[0119] Specifically, the face recognition method includes:

[0120] In step S201, an initial face image of a target user and a face infrared image of the target user are obtained, and the initial face image is obtained by a camera device.

[0121] Specifically, the initial face image is obtained based on a face photo taken by the camera device with the face of the target user directly facing the camera, and the initial face image is generally an image with a size of 512*512 pixels, and the face infrared image is generally an infrared image with a size of 256*256 pixels. The pixel values of the initial face image and the face infrared image are not fixed and can be adjusted according to actual conditions, as long as they are powers of 2, and the pixel values used in the embodiment are data with high accuracy of face recognition trained according to historical data.

[0122] In step S202, an illumination discrimination model is used to analyze the initial face image to obtain an image illumination intensity, and the image illumination intensity is the illumination intensity of the initial face image taken.

[0123] Specifically, the illumination discrimination model is also trained according to historical data, and the historical data for training the illumination discrimination model is images with different illumination intensities and the illumination intensities of the images in a historical time period. The classification model of the illumination is used to quickly distinguish the face recognition application scene, provide a specific face recognition scheme for each scene, and improve the corresponding accuracy.

[0124] In step S203, in a case where the image light intensity is less than or equal to a first light intensity, an infrared image of the face is recognized by using an infrared imaging technology; in a case where the image light intensity is greater than the first light intensity and less than a second light intensity, the initial face image is directly recognized; in a case where the image light intensity is greater than or equal to the second light intensity, the initial face image and the infrared image of the face are processed by using a de-lighting model and a GAN network to obtain a target face image, and the target face image and the infrared image of the face are simultaneously recognized to obtain face information of the target user, the first light intensity is less than the second light intensity, and the de-lighting model is used to reduce the image light intensity.

[0125] Specifically, the image with the image light intensity less than or equal to the first light intensity is obtained by shooting in an irrelevant light or low light (i.e., a dark scene), the image with the image light intensity greater than the first light intensity and less than the second light intensity is obtained by shooting in normal light, and the image with the image light intensity greater than or equal to the second light intensity is obtained by shooting in a high light scene such as facing the sun or lens reflection.

[0126] Optionally, in the case where the image light intensity is greater than or equal to the second light intensity, the initial face image and the infrared image of the face are processed by using the de-lighting model and the GAN network to obtain the target face image, including: the image light intensity is reduced to a preset intensity range by using the de-lighting model to obtain a de-lighting image, a minimum value of the preset intensity range is the first light intensity, and a maximum value of the preset intensity range is the second light intensity; a face reconstruction is performed on the de-lighting image and the infrared image of the face by using the GAN network to obtain a reconstructed face image; and the reconstructed face image and the initial face image are fused to obtain the target face image.

[0127] Optionally, the face reconstruction is performed on the de-lighting image and the infrared image of the face by using the GAN network to obtain the reconstructed face image, including: the de-lighting image is evenly split into a plurality of sub-images with the same pixel value, the pixel value of the sub-image is the same as the pixel value of the infrared image of the face, and the number of the sub-images is related to the pixel value of the initial face image and the pixel value of the infrared image of the face; the infrared image of the face and each sub-image are input into the GAN network by using different channels, and feature points are extracted from the infrared image of the face and each sub-image to obtain a plurality of groups of feature points, the feature points are used to at least represent relative positional relationships between the sub-images; and the face reconstruction is performed according to the infrared image of the face, each group of feature points, and each sub-image to obtain the reconstructed face image.

[0128] Optionally, the image light intensity is reduced to a preset intensity range by using the above-mentioned de-illumination model to obtain a de-illumination image, comprising: constructing a de-illumination model, wherein the de-illumination model is trained using a plurality of sets of training data, each set of training data in the plurality of sets of training data comprises a real face image and a rendered image corresponding to the real face image obtained in a historical time period, the light intensity of the real face image is in the preset intensity range, the real face image is a real face image actually photographed, the rendered image corresponding to the real face image is a rendered image obtained by 3D rendering of the real face image, and the light intensity of the rendered image is greater than or equal to the second light intensity; and reducing the image light intensity to the preset intensity range by using the de-illumination model to obtain the de-illumination image.

[0129] Optionally, before the image light intensity is reduced to the preset intensity range by using the above-mentioned de-illumination model to obtain a de-illumination image, the method further comprises: fusing the real face image and the rendered image corresponding to the real face image to obtain a fused image, the pixel value of the fused image being the sum of a first pixel value and a second pixel value, the first pixel value being the product of a first coefficient and the pixel value of the real face image, and the second pixel value being the product of a second coefficient and the pixel value of the rendered image corresponding to the real face image; processing the fused image using a generative network to generate a plurality of initial prediction images; and performing adversarial training using the plurality of initial prediction images and a plurality of labels to obtain a target prediction image, the light intensity of the target prediction image being in the preset intensity range, the labels corresponding one-to-one to the initial prediction images, and the target prediction image being used to predict the de-illumination image of the initial face image.

[0130] Optionally, the reconstructed face image and the initial face image are fused to obtain the target face image, comprising: obtaining a first target coefficient and a second target coefficient, the sum of the first target coefficient and the second target coefficient being 1; determining a first target pixel value as the product of the pixel value of the reconstructed face image and the first target coefficient; determining a second target pixel value as the product of the pixel value of the initial face image and the second target coefficient; and determining the pixel value of the target face image as the sum of the first target pixel value and the second target pixel value to obtain the target face image.

[0131] Optionally, the acquiring the first target coefficient and the second target coefficient comprises: acquiring a plurality of training coefficient groups, each of the training coefficient groups comprising two training coefficients, and the sum of the two training coefficients in each of the training coefficient groups being 1; inputting each of the training coefficient groups into the identification training model to identify the same training face image to obtain a plurality of identification results, the identification results and the training coefficient groups corresponding to each other; determining an identification rate corresponding to each of the training coefficient groups according to each of the identification results; and determining the two coefficients in the training coefficient group with the highest identification rate as the first target coefficient and the second target coefficient respectively.

[0132] Optionally, the face information of the target user is obtained by simultaneously identifying the target face image and the face infrared image, comprising: inputting the target face image and the face infrared image into a face recognition network, the face recognition network being a neural network for face recognition trained at least according to historical face data; extracting a face feature map from the target face image and the face infrared image, the face feature map being a feature map comprising a plurality of feature points; and inputting the face feature map into a classification recognition model to obtain the face information of the target user, the classification recognition model being a recognition model trained at least according to historical user data, the historical user data at least comprising user information of the target user.

[0133] An embodiment of the present application provides a device, the device comprising a processor, a memory, and a program stored on the memory and executable on the processor, and the processor implements at least the following steps when executing the program:

[0134] Step S201, acquiring an initial face image of a target user and a face infrared image of the target user, the initial face image being acquired by a camera device;

[0135] Step S202, analyzing the initial face image by using a light discrimination model to obtain an image light intensity, the image light intensity being a light intensity of the initial face image captured by the camera device;

[0136] In a case that the image light intensity is less than or equal to the first light intensity, the infrared imaging technology is used to recognize the face infrared image; in a case that the image light intensity is greater than the first light intensity and less than a second light intensity, the initial face image is directly recognized; in a case that the image light intensity is greater than or equal to the second light intensity, the initial face image and the face infrared image are processed by using a de-lighting model and a GAN network to obtain a target face image, and the target face image and the face infrared image are simultaneously recognized to obtain face information of the target user, the first light intensity is less than the second light intensity, and the de-lighting model is used to reduce the image light intensity.

[0137] The device herein can be a server, a PC, a PAD, a mobile phone, etc.

[0138] In a case that the image light intensity is greater than or equal to the second light intensity, the initial face image and the face infrared image are processed by using the de-lighting model and the GAN network to obtain the target face image, including: the image light intensity is reduced to a preset intensity range by using the de-lighting model to obtain a de-lighting image, a minimum value of the preset intensity range is the first light intensity, and a maximum value of the preset intensity range is the second light intensity; the face reconstruction is performed on the de-lighting image and the face infrared image by using the GAN network to obtain a reconstructed face image; and the reconstructed face image and the initial face image are fused to obtain the target face image.

[0139] In a case that the image light intensity is greater than or equal to the second light intensity, the initial face image and the face infrared image are processed by using the de-lighting model and the GAN network to obtain the target face image, including: the image light intensity is reduced to a preset intensity range by using the de-lighting model to obtain a de-lighting image, a minimum value of the preset intensity range is the first light intensity, and a maximum value of the preset intensity range is the second light intensity; the face reconstruction is performed on the de-lighting image and the face infrared image by using the GAN network to obtain a reconstructed face image; and the reconstructed face image and the initial face image are fused to obtain the target face image.

[0140] Optionally, the image light intensity is reduced to a preset intensity range by using the above-mentioned de-illumination model to obtain a de-illumination image, comprising: constructing a de-illumination model, wherein the de-illumination model is trained using a plurality of sets of training data, each set of training data in the plurality of sets of training data comprises a real face image and a rendered image corresponding to the real face image obtained in a historical time period, the light intensity of the real face image is in the preset intensity range, the real face image is a real face image actually photographed, the rendered image corresponding to the real face image is a rendered image obtained by 3D rendering of the real face image, and the light intensity of the rendered image is greater than or equal to the second light intensity; and reducing the image light intensity to the preset intensity range by using the de-illumination model to obtain the de-illumination image.

[0141] Optionally, before the image light intensity is reduced to the preset intensity range by using the above-mentioned de-illumination model to obtain a de-illumination image, the method further comprises: fusing the real face image and the rendered image corresponding to the real face image to obtain a fused image, the pixel value of the fused image being the sum of a first pixel value and a second pixel value, the first pixel value being the product of a first coefficient and the pixel value of the real face image, and the second pixel value being the product of a second coefficient and the pixel value of the rendered image corresponding to the real face image; processing the fused image using a generative network to generate a plurality of initial prediction images; and performing adversarial training using the plurality of initial prediction images and a plurality of labels to obtain a target prediction image, the light intensity of the target prediction image being in the preset intensity range, the labels corresponding one-to-one to the initial prediction images, and the target prediction image being used to predict the de-illumination image of the initial face image.

[0142] Optionally, the reconstructed face image and the initial face image are fused to obtain the target face image, comprising: obtaining a first target coefficient and a second target coefficient, the sum of the first target coefficient and the second target coefficient being 1; determining the product of the pixel value of the reconstructed face image and the first target coefficient as a first target pixel value; determining the product of the pixel value of the initial face image and the second target coefficient as a second target pixel value; and determining the sum of the first target pixel value and the second target pixel value as the pixel value of the target face image to obtain the target face image.

[0143] Optionally, the acquiring the first target coefficient and the second target coefficient comprises: acquiring a plurality of training coefficient groups, each of the training coefficient groups comprising two training coefficients, and the sum of the two training coefficients in one of the training coefficient groups being 1; inputting each of the training coefficient groups into the identification training model to identify the same training face image to obtain a plurality of identification results, the identification results and the training coefficient groups being in one-to-one correspondence; determining an identification rate corresponding to each of the training coefficient groups according to the identification results; and determining the two coefficients in the training coefficient group with the highest identification rate as the first target coefficient and the second target coefficient, respectively.

[0144] Optionally, the simultaneously identifying the target face image and the face infrared image to obtain the face information of the target user comprises: inputting the target face image and the face infrared image into a face recognition network, the face recognition network being a neural network for face recognition trained at least according to historical face data; extracting a face feature map from the target face image and the face infrared image, the face feature map being a feature map comprising a plurality of feature points; and inputting the face feature map into a classification identification model to obtain the face information of the target user, the classification identification model being an identification model trained at least according to historical user data, the historical user data at least comprising user information of the target user.

[0145] The application also provides a computer program product adapted to execute a program comprising at least the following steps when executed on a data processing device:

[0146] Step S201: acquiring an initial face image of a target user and a face infrared image of the target user, the initial face image being acquired by a camera device;

[0147] Step S202: analyzing the initial face image by using a light discrimination model to obtain an image light intensity, the image light intensity being the light intensity of the initial face image captured by the camera device;

[0148] Step S203: in a case where the image light intensity is less than or equal to a first light intensity, identifying the face infrared image by using an infrared imaging technology; in a case where the image light intensity is greater than the first light intensity and less than a second light intensity, directly identifying the initial face image; in a case where the image light intensity is greater than or equal to the second light intensity, processing the initial face image and the face infrared image by using a de-lighting model and a GAN network to obtain a target face image, and simultaneously identifying the target face image and the face infrared image to obtain the face information of the target user, the first light intensity being less than the second light intensity, and the de-lighting model being used to reduce the image light intensity.

[0149] Optionally, in the case that the image light intensity is greater than or equal to the second light intensity, the initial face image and the face infrared image are processed by using a de-light model and a GAN network to obtain a target face image, including: reducing the image light intensity to a preset intensity range by using the de-light model to obtain a de-light image, the minimum value of the preset intensity range is the first light intensity, and the maximum value of the preset intensity range is the second light intensity; performing face reconstruction on the de-light image and the face infrared image by using the GAN network to obtain a reconstructed face image; and fusing the reconstructed face image and the initial face image to obtain the target face image.

[0150] Optionally, the face reconstruction on the de-light image and the face infrared image by using the GAN network to obtain a reconstructed face image includes: averaging and splitting the de-light image into a plurality of sub-images with the same pixel value, the pixel value of the sub-image is the same as the pixel value of the face infrared image, and the number of the sub-images is related to the pixel value of the initial face image and the pixel value of the face infrared image; inputting the face infrared image and each sub-image into the GAN network through different channels, respectively, and performing feature extraction on the face infrared image and each sub-image to obtain a plurality of groups of feature points, the feature points are used to represent at least the relative positional relationship between each sub-image; and performing face reconstruction according to the face infrared image, each group of feature points, and each sub-image to obtain the reconstructed face image.

[0151] Optionally, the de-light image is obtained by reducing the image light intensity to a preset intensity range by using the de-light model, including: constructing a de-light model, wherein the de-light model is trained by using a plurality of groups of training data, each group of training data in the plurality of groups of training data includes a real face image and a rendered image corresponding to the real face image acquired in a historical time period, the light intensity of the real face image is in the preset intensity range, the real face image is a real face image actually photographed, the rendered image corresponding to the real face image is a rendered image obtained by 3D rendering on the real face image, and the light intensity of the rendered image is greater than or equal to the second light intensity; and the de-light image is obtained by reducing the image light intensity to the preset intensity range by using the de-light model.

[0152] Optionally, before the image illumination intensity is reduced to the preset intensity range by using the above-mentioned de-illumination model to obtain the de-illumination image, the method further comprises: fusing the real human face image and the rendering image corresponding to the real human face image to obtain a fused image, the pixel value of the fused image being the sum of a first pixel value and a second pixel value, the first pixel value being the product of a first coefficient and the pixel value of the real human face image, and the second pixel value being the product of a second coefficient and the pixel value of the rendering image corresponding to the real human face image; processing the fused image by using a generative network to generate a plurality of initial prediction images; performing adversarial training on the plurality of initial prediction images and a plurality of labels to obtain a target prediction image, the illumination intensity of the target prediction image being within the preset intensity range, the labels corresponding one-to-one to the initial prediction images, and the target prediction image being used to predict the de-illumination image of the initial human face image.

[0153] Optionally, fusing the reconstructed human face image and the initial human face image to obtain the target human face image comprises: obtaining a first target coefficient and a second target coefficient, the sum of the first target coefficient and the second target coefficient being 1; determining a first target pixel value as the product of the pixel value of the reconstructed human face image and the first target coefficient; determining a second target pixel value as the product of the pixel value of the initial human face image and the second target coefficient; and determining the pixel value of the target human face image as the sum of the first target pixel value and the second target pixel value to obtain the target human face image.

[0154] Optionally, obtaining the first target coefficient and the second target coefficient comprises: obtaining a plurality of training coefficient groups, each of the training coefficient groups comprising two training coefficients, the sum of the two training coefficients in one of the training coefficient groups being 1; inputting each of the training coefficient groups into an identification training model to identify the same training human face image to obtain a plurality of identification results, the identification results corresponding one-to-one to the training coefficient groups; determining an identification rate corresponding to each of the training coefficient groups according to each of the identification results; and determining the two coefficients in the training coefficient group with the highest identification rate as the first target coefficient and the second target coefficient, respectively.

[0155] Optionally, the target face image and the face infrared image are recognized simultaneously to obtain face information of the target user, including: inputting the target face image and the face infrared image into a face recognition network, the face recognition network being a neural network for face recognition trained at least according to historical face data; extracting a face feature map from the target face image and the face infrared image, the face feature map being a feature map including a plurality of feature points; inputting the face feature map into a classification recognition model to obtain the face information of the target user, the classification recognition model being a recognition model trained at least according to historical user data, the historical user data including at least user information of the target user.

[0156] It is apparent that those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be centralized on a single computing device or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into individual integrated circuit modules or a single integrated circuit module. Thus, the present application is not limited to any particular combination of hardware and software.

[0157] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0158] The present application is described with reference to the flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0159] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0161] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0162] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, or a combination of non-volatile memories in different forms. The memory is an example of computer readable storage media.

[0163] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0164] It should also be noted that the terms "comprising", "comprises" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0165] From the above description, it can be seen that the above-mentioned embodiments of the present application achieve the following technical effects:

[0166] 1) The above-mentioned face recognition method of the present application first acquires an initial face image of a target user and a face infrared image of the target user, and the initial face image is acquired by a camera device; then a light discrimination model is used to analyze the initial face image to obtain an image light intensity, and the image light intensity is the light intensity of the initial face image captured; finally, in the case that the image light intensity is less than or equal to a first light intensity, an infrared imaging technology is used to recognize the face infrared image; in the case that the image light intensity is greater than the first light intensity and less than a second light intensity, the initial face image is directly recognized; in the case that the image light intensity is greater than or equal to the second light intensity, a de-lighting model and a GAN network are used to process the initial face image and the face infrared image to obtain a target face image, and the target face image and the face infrared image are simultaneously recognized to obtain face information of the target user, the first light intensity is less than the second light intensity, and the de-lighting model is used to reduce the image light intensity. This method uses an infrared face image and part of the face image captured by a camera to regenerate a face, and simultaneously combines the infrared face image to input a detection and recognition model, thereby retaining the real face features in the largest scale, improving the face recognition rate under high light, and solving the problem that the face cannot be accurately monitored under a high light scene in the prior art, resulting in a low face recognition rate.

[0167] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A face recognition method, characterized by, The method comprises the following steps: obtaining an initial face image of a target user and an infrared image of the face of the target user, wherein the initial face image is captured by a camera device; analyzing the initial face image by using an illumination discrimination model to obtain an image illumination intensity, wherein the image illumination intensity is the illumination intensity of the captured initial face image; in a case where the image illumination intensity is less than or equal to a first illumination intensity, recognizing the infrared image of the face by using infrared imaging technology; in a case where the image illumination intensity is greater than the first illumination intensity and less than a second illumination intensity, directly recognizing the initial face image; in a case where the image illumination intensity is greater than or equal to the second illumination intensity, processing the initial face image and the infrared image of the face by using a de-illumination model and a GAN network to obtain a target face image, and simultaneously recognizing the target face image and the infrared image of the face to obtain face information of the target user, wherein the first illumination intensity is less than the second illumination intensity, and the de-illumination model is used to reduce the image illumination intensity; in a case where the image illumination intensity is greater than or equal to the second illumination intensity, processing the initial face image and the infrared image of the face by using a de-illumination model and a GAN network to obtain a target face image, comprising: reducing the image illumination intensity to a preset intensity range by using the de-illumination model to obtain a de-illumination image, wherein the minimum value of the preset intensity range is the first illumination intensity, and the maximum value of the preset intensity range is the second illumination intensity; reconstructing a face by using a GAN network on the de-illumination image and the infrared image of the face to obtain a reconstructed face image; fusing the reconstructed face image and the initial face image to obtain the target face image; averagely splitting the de-illumination image into a plurality of sub-images with the same pixel value, wherein the pixel value of the sub-image is the same as that of the infrared image of the face, and the number of the sub-images is related to the pixel value of the initial face image and the pixel value of the infrared image of the face; inputting the infrared image of the face and each sub-image into the GAN network by using different channels respectively, and extracting features of the infrared image of the face and each sub-image to obtain a plurality of groups of feature points, wherein the feature points are used to represent at least the relative positional relationship between each sub-image; reconstructing a face according to the infrared image of the face, each group of feature points and each sub-image to obtain the reconstructed face image.

2. The face recognition method of claim 1, wherein, reducing the image illumination intensity to a preset intensity range by using the de-illumination model to obtain a de-illumination image, comprising: constructing a de-illumination model, wherein the de-illumination model is trained using a plurality of sets of training data, each set of training data including a real face image and a rendered image corresponding to the real face image, the real face image having an illumination intensity within a preset intensity range, the real face image being a real photographed face image, the rendered image being a rendering image obtained by 3D rendering of the real face image, the rendered image having an illumination intensity greater than or equal to the second illumination intensity, the real face image being obtained in a historical time period; using the de-illumination model to reduce the image illumination intensity to within the preset intensity range to obtain the de-illumination image.

3. The face recognition method of claim 2, wherein, Before using the de-illumination model to reduce the image illumination intensity to within the preset intensity range to obtain the de-illumination image, the method further comprises: fusing the real face image and the rendered image corresponding to the real face image to obtain a fused image, the pixel value of the fused image being the sum of a first pixel value and a second pixel value, the first pixel value being the product of a first coefficient and the pixel value of the real face image, the second pixel value being the product of a second coefficient and the pixel value of the rendered image corresponding to the real face image; using a generative network to process the fused image to generate a plurality of initial prediction images; using a plurality of the initial prediction images and a plurality of labels for adversarial training to obtain a target prediction image, the target prediction image having an illumination intensity within the preset intensity range, the labels corresponding one-to-one to the initial prediction images, the target prediction image being used to predict the de-illumination image of the initial face image.

4. The face recognition method of claim 1, wherein, fusing the reconstructed face image and the initial face image to obtain the target face image, comprising: obtaining a first target coefficient and a second target coefficient, the sum of the first target coefficient and the second target coefficient being 1; determining the product of the pixel value of the reconstructed face image and the first target coefficient as a first target pixel value; determining the product of the pixel value of the initial face image and the second target coefficient as a second target pixel value; determining the sum of the first target pixel value and the second target pixel value as the pixel value of the target face image to obtain the target face image.

5. The face recognition method of claim 4, wherein, obtaining a first target coefficient and a second target coefficient, comprising: obtaining a plurality of training coefficient sets, each training coefficient set including two training coefficients, the sum of the two training coefficients in one training coefficient set being 1; inputting each training coefficient set into an identification training model to identify the same training face image to obtain a plurality of identification results, the identification results corresponding one-to-one to the training coefficient sets; determining the identification rate corresponding to each training coefficient set according to each identification result; determining the two coefficients in the training coefficient set with the highest identification rate as the first target coefficient and the second target coefficient, respectively.

6. The face recognition method according to any one of claims 1 to 4, characterized in that, simultaneously identifying the target face image and the face infrared image to obtain face information of the target user, comprising: input the target face image and the face infrared image into a face recognition network, the face recognition network being a neural network for face recognition trained at least according to historical face data; extract a face feature map from the target face image and the face infrared image, the face feature map being a feature map including a plurality of feature points; input the face feature map into a classification recognition model to obtain face information of the target user, the classification recognition model being a recognition model trained at least according to historical user data, the historical user data including at least user information of the target user.

7. A computer readable storage medium characterized in that, The computer-readable storage medium includes a stored program, wherein the program controls the device where the computer-readable storage medium is located to execute the face recognition method of any one of claims 1 to 6 when the program is running.

8. An electronic device, comprising: comprise: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include a program for executing the face recognition method of any one of claims 1 to 6.

Images