Face recognition method and device, electronic equipment and computer program product

Abstract

The invention discloses a face recognition method and device, electronic equipment and a computer program product. The method comprises the steps that a face visible light image and a corresponding infrared image are acquired; under the condition that the face visible light image is a high-illumination image, performing illumination removal processing on the face visible light image by using an illumination removal model to obtain a face visible light image after illumination removal processing; performing face reconstruction according to the face visible light image after illumination removal processing and the corresponding infrared image to obtain a reconstructed face image; and performing face recognition according to the reconstructed face image, the face visible light image and the infrared image to obtain a face recognition result. According to the face recognition method provided by the embodiment of the invention, the infrared image and the face visible light image are combined to restore the face information and generate the reconstructed face image, so that the problem of inaccurate face recognition in a high-illumination scene is solved, feature information for face recognition is enriched, and the accuracy of face recognition is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to a face recognition method, device, electronic equipment, and computer program product. Background Art

[0002] Face recognition is a biometric technology that extracts and models facial feature information from two-dimensional images and identifies identities.

[0003] With the advancement of technology, the accuracy of face recognition in normal scenarios is already very high. However, in some scenarios, such as high light, facing the sun, lens reflection and other abnormal scenarios, face recognition not only has a low recognition rate but also low efficiency. In such abnormal scenarios, the face cannot be detected, resulting in inaccurate positioning and recognition, which makes the user experience very poor.

[0004] The prior art provides some face recognition solutions. One is a recognition method based on generating faces based on human eyes. By detecting the human eye area and sending it to the GAN network, a face image with high similarity is generated, and then face recognition is performed to overcome the problem of face occlusion.

[0005] The other method is to detect the infrared face area through the infrared face detection model, and then use the infrared face area to generate a visible light face image. Based on the visible light face detection model, the acquired infrared face image is detected and located, and used for subsequent recognition and classification, which can solve the problems of low resolution and unclear image in infrared recognition.

[0006] Another method is a face recognition method based on the combination of near-infrared imaging and deep learning, including near-infrared face image acquisition and liveness detection, and converting infrared faces into visible light images for recognition through the cycleGAN network.

[0007] However, existing face recognition solutions have at least the following technical problems:

[0008] 1) Commonly used infrared face recognition has problems such as poor resolution, low contrast, blurred visual effects, and great influence of ambient light. Using only infrared to recognize faces has poor results.

[0009] 2) In high-light scenes, the effect of pure visible light face detection is poor, there is less available face information, the face cannot be located, and the recognition efficiency is low.

[0010] 3) The recognition method based on human eye generation of face cannot be detected by human eye in high light or when the user wears reflective glasses, which may cause the method to fail. There is also the problem of little available information on the face, which limits the application scenarios.

[0011] 4) The infrared face area is detected through the infrared face detection model, and then the infrared face area is used to generate a visible light face image. This method completely relies on infrared technology to generate visible light face images. Once the infrared face image acquisition resolution is reduced or the acquisition fails, face recognition will not be possible. At the same time, it also cannot solve the problem of loss of facial information caused by high light.

[0012] 5) Face recognition method based on the combination of near-infrared imaging and deep learning. This method only adds liveness detection and ordinary infrared face recognition, and the face recognition effect is not actually improved. Summary of the invention

[0013] The embodiments of the present application provide a face recognition method, device, electronic device, and computer program product to improve the accuracy and efficiency of face recognition.

[0014] The present application embodiment adopts the following technical solutions:

[0015] In a first aspect, an embodiment of the present application provides a face recognition method, the face recognition method comprising:

[0016] Obtain a visible light image of a face and a corresponding infrared image;

[0017] When the visible light image of the face is a high-illumination image, performing illumination removal processing on the visible light image of the face using an illumination removal model to obtain the visible light image of the face after the illumination removal processing;

[0018] Reconstructing the face based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain a reconstructed face image;

[0019] Face recognition is performed based on the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result.

[0020] Optionally, after acquiring the visible light image of the face and the corresponding infrared image, the face recognition method further includes:

[0021] Using an illumination discrimination model to discriminate the visible light image of the human face, and determine the illumination type corresponding to the visible light image of the human face;

[0022] If the illumination type corresponding to the visible light image of the face is normal illumination, directly performing face recognition on the visible light image of the face to obtain a face recognition result;

[0023] If the illumination type corresponding to the visible light image of the face is no illumination or low illumination, extracting iris features from the infrared image, and determining a face recognition result according to the iris features;

[0024] If the illumination type corresponding to the visible light image of the face is high illumination, a step of performing illumination removal processing on the visible light image of the face using an illumination removal model to obtain the visible light image of the face after the illumination removal processing.

[0025] Optionally, the using an illumination discrimination model to discriminate the visible light image of the human face to determine the illumination type corresponding to the visible light image of the human face includes:

[0026] Extracting brightness and gain values ​​from the visible light image of the human face using an illumination discrimination model;

[0027] Determine the illumination type corresponding to the visible light image of the face according to the brightness value and the gain value in the visible light image of the face.

[0028] Optionally, performing face reconstruction based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain the reconstructed face image includes:

[0029] Segmenting the visible light image of the face after the illumination removal process into multiple equal parts to obtain segmented visible light images of the face;

[0030] Merging the segmented visible light image of the face with the corresponding infrared image to obtain a multi-channel image;

[0031] The multi-channel image is processed using a face reconstruction model to obtain a reconstructed face image.

[0032] Optionally, performing face recognition according to the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result includes:

[0033] fusing the reconstructed face image and the face visible light image to obtain a fused face image;

[0034] Face recognition is performed based on the fused face image and the infrared image to obtain a face recognition result.

[0035] Optionally, performing face recognition according to the fused face image and the infrared image to obtain a face recognition result includes:

[0036] extracting facial features from the fused facial image and iris features from the infrared image respectively;

[0037] The facial feature information is compared based on the facial feature and the iris feature to obtain a facial recognition result.

[0038] Optionally, the illumination removal model is trained in the following manner:

[0039] Use 3D rendering to generate illuminated rendered images and unilluminated real images;

[0040] Constructing a training sample pair according to the illuminated rendered image and the unilluminated real image;

[0041] The training samples are used to train an illumination removal model.

[0042] In a second aspect, an embodiment of the present application further provides a face recognition device, the face recognition device comprising:

[0043] An acquisition unit, used to acquire a visible light image of a face and a corresponding infrared image;

[0044] an illumination removal unit, configured to, when the visible light image of the human face is a high-illumination image, perform illumination removal processing on the visible light image of the human face using an illumination removal model to obtain the visible light image of the human face after the illumination removal processing;

[0045] A reconstruction unit, used for performing face reconstruction based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain a reconstructed face image;

[0046] A face recognition unit is used to perform face recognition based on the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result.

[0047] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0048] Processor; and

[0049] A memory arranged to store computer executable instructions which, when executed, cause the processor to perform any of the methods described above.

[0050] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes any of the aforementioned methods.

[0051] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the face recognition method in the embodiments of the present application first obtains a visible light image of the face and a corresponding infrared image; then, when the visible light image of the face is a high-light image, the illumination removal model is used to perform illumination removal processing on the visible light image of the face to obtain a visible light image of the face after illumination removal processing; then, the face is reconstructed based on the visible light image of the face after illumination removal processing and the corresponding infrared image to obtain a reconstructed face image; finally, face recognition is performed based on the reconstructed face image, the visible light image of the face and the infrared image to obtain a face recognition result. The face recognition method in the embodiments of the present application combines the infrared image and the real face image to restore the face and generate a reconstructed face image, which solves the problem of inaccurate face recognition in high-light scenes, enriches the feature information used for face recognition, and greatly improves the accuracy of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 A schematic diagram of a face recognition method in an embodiment of the present application;

[0054] Figure 2 This is a schematic diagram of a face recognition process in an embodiment of the present application;

[0055] Figure 3 This is a structural schematic diagram of a face recognition device in an embodiment of the present application;

[0056] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0058] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0059] The present application embodiment provides a face recognition method, such as Figure 1As shown, a flow chart of a face recognition method in an embodiment of the present application is provided, and the face recognition method at least includes the following steps S110 to S140:

[0060] Step S110, obtaining a visible light image of a face and a corresponding infrared image.

[0061] When performing face recognition, it is necessary to first use a camera device to collect visible light images of the face, and normalize the collected visible light images of the face to a set pixel size, such as 512*512 pixels. At the same time, an infrared image acquisition device is used to collect infrared images, and the infrared images are normalized to a set pixel size, such as 256*256 pixels. Of course, the specific pixel size can be flexibly set by those skilled in the art according to actual needs, and is not specifically limited here.

[0062] The above visible light image of the face is an RGB image, that is, an original color photo obtained by an ordinary optical camera, with a wavelength between 0.450-0.680, usually arranged in RGB16, RGB24, RGB32, etc. The above infrared image, also known as a near-infrared image (Nearing Infrared Image), is an image formed by a photosensitive component receiving the target object's reflection or radiation of the near-infrared band.

[0063] Step S120, when the visible light image of the face is a high-illumination image, use an illumination removal model to perform illumination removal processing on the visible light image of the face to obtain the visible light image of the face after illumination removal processing.

[0064] When the current acquisition scene is a high-light scene, due to the small amount of available facial information, the simple visible light face detection effect is poor and cannot accurately identify facial information. Therefore, the embodiment of the present application adopts a pre-trained illumination removal model to remove the illumination in the visible light image of the face, and obtains the visible light image of the face after the illumination is removed.

[0065] Step S130, performing face reconstruction based on the visible light image of the face after the illumination removal processing and the corresponding infrared image to obtain a reconstructed face image.

[0066] Because both visible light and infrared images of faces will be affected in high-light scenes, it is difficult to ensure the accuracy of recognition by simply relying on any single data source for face recognition. Therefore, the embodiment of the present application combines the visible light image of the face after the light is removed and the corresponding infrared image to reconstruct the face and obtain a reconstructed face image, thereby solving the problem of low face recognition accuracy caused by relying on a single data source under high-light conditions.

[0067] The above-mentioned face reconstruction process can be implemented using a pre-trained Generative Adversarial Network (GAN). The GAN network uses a generator and a discriminator to perform adversarial training on the input data to achieve the desired effect, including style transfer, image generation and other functions.

[0068] The embodiment of the present application can, for example, use the StyleGAN network to achieve face reconstruction. StyleGAN is a new generator network structure designed under the inspiration of style transfer. The new network structure can decouple and separate the high-level semantic attributes of the image through unsupervised automatic learning, such as the posture and identity of the face image, and the random changes of the generated image, such as freckles and hair, can also achieve a certain degree of controlled synthesis. The embodiment of the present application combines the infrared image with part of the real face image, uses the StyleGAN network to restore the face, and generates a new face image, thereby compensating for the influence of high light.

[0069] Of course, it should be noted that the specific face reconstruction model to be used can be flexibly selected by those skilled in the art according to actual needs, and no specific limitation is made here.

[0070] Step S140, performing face recognition according to the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result.

[0071] Although the original visible light images and infrared images of the face collected in high-light scenes are difficult to meet the requirements of face recognition accuracy alone, they can still provide useful face information to a certain extent. Therefore, the embodiments of the present application perform face recognition based on multiple data sources based on the original collected visible light images and infrared images of the face and the above-mentioned reconstructed face images. The information from the multiple data sources complements each other, thereby greatly improving the face recognition accuracy in high-light scenes.

[0072] The face recognition method in the embodiment of the present application combines infrared images and real face images to restore the face and generate a reconstructed face image, which solves the problem of inaccurate face recognition in high-light scenes, enriches the feature information used for face recognition, and greatly improves the accuracy of face recognition.

[0073] In some embodiments of the present application, after acquiring a visible light image of a face and a corresponding infrared image, the face recognition method further includes: using a lighting discrimination model to discriminate the visible light image of the face to determine the lighting type corresponding to the visible light image of the face; if the lighting type corresponding to the visible light image of the face is normal lighting, directly performing face recognition on the visible light image of the face to obtain a face recognition result; if the lighting type corresponding to the visible light image of the face is no lighting or low lighting, extracting iris features from the infrared image, and determining the face recognition result based on the iris features; if the lighting type corresponding to the visible light image of the face is high lighting, executing the step of performing lighting removal processing on the visible light image of the face using an lighting removal model to obtain the visible light image of the face after the lighting removal processing.

[0074] In order to make the face recognition scheme of the embodiment of the present application applicable to different application scenarios, after collecting the visible light image of the face and the corresponding infrared image, the embodiment of the present application can also use the illumination discrimination model to determine the collection scene corresponding to the currently collected visible light image of the face. The collection scenes and the corresponding face recognition schemes mainly include three situations:

[0075] 1) Normal lighting: If the visible light face image is collected under normal lighting, sufficient facial feature information can usually be captured in this scenario. Therefore, the existing face recognition model can be directly used to extract facial features from the collected visible light face image, and then compare the facial feature information.

[0076] 2) No light or low light: If the visible light image of the face is collected in a no light or low light scene, in this case, a lot of effective facial feature point information will be missing in the visible light image, but the infrared image still has a good imaging effect, so the iris feature information in the infrared image can be directly extracted for face recognition. The iris is the ring-shaped part between the black pupil and the white sclera, which contains many interlaced spots, filaments, crowns, stripes, crypts and other detailed features, and is an important feature information for face recognition.

[0077] 3) High light: If the visible light image of the face is collected in a high light scene, the feature information in the visible light image and the infrared image of the face will be affected to varying degrees. Therefore, it is necessary to adopt the improved face recognition scheme of the embodiment of the present application for face recognition.

[0078] It should be noted that the above-mentioned illumination discrimination model adopted in the embodiment of the present application can, for example, adopt methods based on pixel statistics such as average brightness calculation, histogram analysis, etc. to analyze the illumination intensity in the image. Machine learning algorithms such as support vector machine (SVM), random forest, etc. can also be adopted to learn the mapping relationship between illumination intensity and image features through training data sets. Deep learning models such as convolutional neural network (CNN) and other models can also be adopted to automatically learn the recognition rules of illumination intensity through training of a large amount of image data. Of course, the specific illumination discrimination method can be flexibly set by those skilled in the art according to actual needs, and is not specifically limited here.

[0079] In some embodiments of the present application, the use of an illumination discrimination model to discriminate the visible light image of the face to determine the illumination type corresponding to the visible light image of the face includes: extracting brightness values ​​and gain values ​​in the visible light image of the face using the illumination discrimination model; and determining the illumination type corresponding to the visible light image of the face based on the brightness values ​​and gain values ​​in the visible light image of the face.

[0080] When identifying the light intensity in the visible light image of a face, the embodiment of the present application can first extract parameter information related to brightness in the visible light image of the face, for example, it can include the brightness value Y value and the gain value. In image processing, the Y value is usually associated with the brightness component in the color space. In the YUV color space, Y represents brightness (Luminance / Luma), also known as grayscale value. The size of the Y value directly reflects the brightness level of the image. For black and white images, the Y value is all the information of the image; for color images, the Y value is the basis for the brightness of the image. The gain value is also an important parameter in image processing, which is related to the degree of amplification of the image signal. When the gain value is 1, the image signal remains unchanged; when the gain value is greater than 1, the image signal is amplified and the brightness increases; conversely, when the gain value is less than 1, the image signal is attenuated and the brightness decreases.

[0081] Therefore, the embodiments of the present application can combine the Y value and the gain value to determine the light intensity of the current visible light image of the face. Specifically, the brightness value Y and the gain value can be compared with the corresponding interval ranges or thresholds set for different lighting scenes, so as to determine whether the current light intensity belongs to the normal lighting range, the no lighting or low lighting range, or the high lighting range.

[0082] In addition, it should be noted that due to differences in imaging effects of different image acquisition devices, the corresponding lighting judgment thresholds or interval ranges will also be different. Therefore, different judgment conditions can be adopted according to different devices to improve the adaptability and flexibility of lighting judgment.

[0083] In some embodiments of the present application, the face reconstruction is performed based on the visible light image of the face after the illumination removal processing and the corresponding infrared image to obtain the reconstructed face image, which includes: dividing the visible light image of the face after the illumination removal processing into multiple equal parts to obtain a segmented visible light image of the face; merging the segmented visible light image of the face with the corresponding infrared image to obtain a multi-channel image; and processing the multi-channel image using a face reconstruction model to obtain a reconstructed face image.

[0084] When performing face reconstruction, the embodiment of the present application can first divide the visible light image of the face after removing the illumination into multiple equal parts, so that the subsequent face reconstruction model can focus on the important information of different face regions. For example, the visible light image of the face after removing the illumination can be divided into four equal parts, and each small part has a fixed pixel size such as 256*256 pixels. Of course, the number of divisions can be flexibly set according to actual conditions and needs, and is not specifically limited here.

[0085] After that, the real face image after the multi-equal segmentation is fused with the infrared image to obtain a multi-channel image. For example, if the real face image after the multi-equal segmentation is divided into four equal parts, then a 5-channel image can be obtained after fusion with the infrared image. Finally, the 5-channel image is input into the StyleGAN network for face reconstruction to obtain a reconstructed face image. In this way, the embodiment of the present application combines the face information in the infrared image and the information of different face regions in the face visible light image to fully restore a more realistic and richer face image, providing a more accurate and reliable basis for subsequent face recognition.

[0086] In some embodiments of the present application, performing face recognition based on the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result includes: fusing the reconstructed face image and the visible light face image to obtain a fused face image; performing face recognition based on the fused face image and the infrared image to obtain a face recognition result.

[0087] When performing face recognition based on the reconstructed face image of the aforementioned embodiment, the reconstructed face image and the collected original visible light face image can be fused to obtain a fused face image, thereby compensating for the invisible face caused by the highlight area. A specific fusion scheme can be to fuse the pixel information of the original visible light face image and the reconstructed face image in a certain ratio, for example, it can be expressed as: α*original visible light face image+β*reconstructed face image, where α+β=1, and the specific values ​​of α and β can be flexibly set according to actual conditions, and are not specifically limited here.

[0088] The fused face image and the original infrared image are used as the basic data sources for face recognition, and the face feature information is extracted respectively. Finally, the face feature is compared in the database based on the extracted face feature information to obtain the face recognition result. It should be noted that since the reconstructed face image also contains iris feature information, the iris feature can also be extracted from the reconstructed face image.

[0089] In some embodiments of the present application, performing face recognition based on the fused face image and the infrared image to obtain a face recognition result includes: respectively extracting face features in the fused face image and iris features in the infrared image; and comparing face feature information based on the face features and the iris features to obtain a face recognition result.

[0090] When acquiring facial feature information, feature extraction can be performed on the fused facial image and infrared image respectively. For the extraction of facial feature information of the fused facial image, an existing facial recognition model can be used, for example, it can be implemented based on the commonly used ResNet network. ResNet is a common feature extraction and classification model. It uses a deep network to extract high-dimensional features from the input image, and then classifies the features. The deep learning network has been widely used in the field of facial recognition. How to obtain the facial recognition model can be flexibly determined by those skilled in the art in combination with the existing technology, and is not specifically limited here.

[0091] In some embodiments of the present application, the illumination removal model is trained in the following manner: using 3D rendering to generate an illuminated rendered image and a real image without illumination; constructing a training sample pair based on the illuminated rendered image and the real image without illumination; and using the training sample pairs to train the illumination removal model.

[0092] The illumination removal model adopted in the embodiment of the present application can be based on the network structure mentioned in the Deep Adversarial Decomposition paper of CVPR2020, and is trained using a self-designed training set. The paper uses a consistent network structure and a GAN network to solve the problem of separating single overlapping images. Through the separation criterion and the cross loss function, the framework can effectively separate scenes such as removing shadows and removing reflections. The network includes a generator, a separation criterion discriminator, and two Markov discriminators to optimize the visual quality and details of the image.

[0093] The training set independently designed in the embodiment of the present application uses 3D rendering to obtain a rendering image with lighting and a real image without lighting to form a training pair. By inputting a mixed image, two predicted images x1 and x2 are generated through the generative network. The labels X1 and X2 are used respectively for adversarial training through the designed cross loss function. In this way, the face image with the lighting removed can be predicted. The specific network structure can also be replaced by other GAN networks, which is not specifically limited here.

[0094] In order to facilitate the understanding of the above embodiments, Figure 2 As shown, a schematic diagram of a face recognition process in an embodiment of the present application is provided. First, a visible light image and an infrared image of a face are collected, and then the illumination discrimination model is used to determine the illumination condition in the visible light image of the face. If it is a scene with normal illumination, the existing face recognition model is directly used for face recognition. If it is a scene with low illumination or no illumination, the iris feature information can be extracted from the infrared image, and then face comparison is performed based on the iris feature information.

[0095] If it is a high-light scene, the illumination removal model is used to remove the illumination of the visible light image of the face to obtain the visible light image of the face after the illumination is removed. After that, the visible light image of the face after the illumination is removed is divided into multiple equal parts, so that the subsequent face reconstruction model can focus on the important information of different face areas. The real face image after multi-equal division is fused with the infrared image to obtain a multi-channel image, and the multi-channel image is input into the StyleGAN network for face reconstruction to obtain a reconstructed face image. The reconstructed face image is fused with the original visible light face image collected to obtain a fused face image, thereby compensating for the invisible face caused by the highlight area.

[0096] Finally, the fused face image and the original infrared image are used as the basic data sources for face recognition, and the face feature points and iris feature information are extracted respectively. Based on the extracted face feature points and iris feature information, face feature comparison is performed in the database to obtain the face recognition result.

[0097] In summary, the key points of the face recognition method of this application are:

[0098] 1) Combining infrared images and real face images, the face reconstruction model is used to restore the face and generate a reconstructed face image, which solves the problem of low face recognition accuracy in high-light scenes;

[0099] 2) The reconstructed face image incorporates iris information, which increases the feature point information that the face recognition model can capture, greatly improving the accuracy of face recognition;

[0100] 3) A 3D rendering method is used to generate light effects on unilluminated face images, and a self-designed training set is combined with a GAN network to train a lighting removal model, which improves the detection rate and recognition accuracy.

[0101] The face recognition method of the present application uses an illumination discrimination model to quickly distinguish application scenarios of face recognition, provides a specific face recognition solution for each scenario, and improves the corresponding accuracy; at the same time, infrared images are fused with visible light face images of different face areas to regenerate face images. The available information in the newly generated face image includes both iris information and face feature point information, thereby increasing the amount of face information and improving face recognition accuracy.

[0102] Through experimental comparison, we used reconstructed face images to extract iris information and fused face images to extract facial feature information, and performed face comparison. When tested on the same sample, the recognition detection rate and accuracy were increased by 16% and 21% respectively, and the recognition effect was significantly improved.

[0103] The present application also provides a face recognition device 300, such as Figure 3 As shown, a schematic diagram of the structure of a face recognition device in an embodiment of the present application is provided, wherein the face recognition device 300 includes: an acquisition unit 310, an illumination removal unit 320, a reconstruction unit 330 and a face recognition unit 340, wherein:

[0104] An acquisition unit 310 is used to acquire a visible light image of a face and a corresponding infrared image;

[0105] An illumination removal unit 320 is used to perform illumination removal processing on the visible light image of the face using an illumination removal model when the visible light image of the face is a high-illumination image, so as to obtain the visible light image of the face after the illumination removal processing;

[0106] A reconstruction unit 330, configured to perform face reconstruction based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain a reconstructed face image;

[0107] The face recognition unit 340 is used to perform face recognition according to the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result.

[0108] In some embodiments of the present application, the face recognition device 300 also includes: an illumination discrimination unit, which is used to, after acquiring a visible light image of the face and a corresponding infrared image, use an illumination discrimination model to discriminate the visible light image of the face, and determine the illumination type corresponding to the visible light image of the face; the face recognition unit is also used to: if the illumination type corresponding to the visible light image of the face is normal illumination, directly perform face recognition on the visible light image of the face to obtain a face recognition result; if the illumination type corresponding to the visible light image of the face is no illumination or low illumination, extract iris features from the infrared image, and determine the face recognition result based on the iris features; the illumination removal unit is also used to: if the illumination type corresponding to the visible light image of the face is high illumination, perform illumination removal processing on the visible light image of the face using an illumination removal model to obtain a visible light image of the face after illumination removal processing.

[0109] In some embodiments of the present application, the illumination discrimination unit is specifically used to: extract the brightness value and gain value in the visible light image of the face using an illumination discrimination model; and determine the illumination type corresponding to the visible light image of the face based on the brightness value and gain value in the visible light image of the face.

[0110] In some embodiments of the present application, the reconstruction unit 330 is specifically used to: divide the visible light image of the face after the illumination removal processing into multiple equal parts to obtain a segmented visible light image of the face; merge the segmented visible light image of the face with the corresponding infrared image to obtain a multi-channel image; and process the multi-channel image using a face reconstruction model to obtain a reconstructed face image.

[0111] In some embodiments of the present application, the face recognition unit 340 is specifically used to: fuse the reconstructed face image and the visible light face image to obtain a fused face image; perform face recognition based on the fused face image and the infrared image to obtain a face recognition result.

[0112] In some embodiments of the present application, the face recognition unit 340 is specifically used to: respectively extract the facial features in the fused face image and the iris features in the infrared image; compare the facial feature information according to the facial features and the iris features to obtain a face recognition result.

[0113] In some embodiments of the present application, the illumination removal model is trained in the following manner: using 3D rendering to generate an illuminated rendered image and a real image without illumination; constructing a training sample pair based on the illuminated rendered image and the real image without illumination; and using the training sample pairs to train the illumination removal model.

[0114] It can be understood that the above-mentioned face recognition device can implement each step of the face recognition method provided in the aforementioned embodiment, and the relevant explanations about the face recognition method are applicable to the face recognition device and will not be repeated here.

[0115] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0116] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0117] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0118] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a face recognition device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0119] Obtain a visible light image of a face and a corresponding infrared image;

[0120] When the visible light image of the face is a high-illumination image, performing illumination removal processing on the visible light image of the face using an illumination removal model to obtain the visible light image of the face after the illumination removal processing;

[0121] Reconstructing the face based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain a reconstructed face image;

[0122] Face recognition is performed based on the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result.

[0123] The above application Figure 1 The method performed by the face recognition device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in software form. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0124] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the face recognition device in the illustrated embodiment is specifically used to perform:

[0125] Obtain a visible light image of a face and a corresponding infrared image;

[0126] When the visible light image of the face is a high-illumination image, performing illumination removal processing on the visible light image of the face using an illumination removal model to obtain the visible light image of the face after the illumination removal processing;

[0127] Reconstructing the face based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain a reconstructed face image;

[0128] Face recognition is performed based on the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result.

[0129] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0134] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0135] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0138] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A face recognition method, characterized in that: The face recognition method comprises: Obtain a visible light image of a face and a corresponding infrared image; When the visible light image of the face is a high-illumination image, performing illumination removal processing on the visible light image of the face using an illumination removal model to obtain the visible light image of the face after the illumination removal processing; Reconstructing the face based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain a reconstructed face image; Face recognition is performed based on the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result.

2. The face recognition method according to claim 1, characterized in that: After acquiring the visible light image of the face and the corresponding infrared image, the face recognition method further includes: Using an illumination discrimination model to discriminate the visible light image of the human face, and determine the illumination type corresponding to the visible light image of the human face; If the illumination type corresponding to the visible light image of the face is normal illumination, directly performing face recognition on the visible light image of the face to obtain a face recognition result; If the illumination type corresponding to the visible light image of the face is no illumination or low illumination, extracting iris features from the infrared image, and determining a face recognition result according to the iris features; If the illumination type corresponding to the visible light image of the face is high illumination, a step of performing illumination removal processing on the visible light image of the face using an illumination removal model to obtain the visible light image of the face after the illumination removal processing.

3. The face recognition method according to claim 1, characterized in that: The using the illumination discrimination model to discriminate the visible light image of the human face to determine the illumination type corresponding to the visible light image of the human face comprises: Extracting brightness and gain values ​​from the visible light image of the human face using an illumination discrimination model; Determine the illumination type corresponding to the visible light image of the face according to the brightness value and the gain value in the visible light image of the face.

4. The face recognition method according to claim 1, characterized in that: The step of performing face reconstruction based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain the reconstructed face image comprises: Segmenting the visible light image of the face after the illumination removal process into multiple equal parts to obtain segmented visible light images of the face; Merging the segmented visible light image of the face with the corresponding infrared image to obtain a multi-channel image; The multi-channel image is processed using a face reconstruction model to obtain a reconstructed face image.

5. The face recognition method according to claim 1, characterized in that: The performing face recognition according to the reconstructed face image, the visible light face image and the infrared image to obtain the face recognition result comprises: fusing the reconstructed face image and the face visible light image to obtain a fused face image; Face recognition is performed based on the fused face image and the infrared image to obtain a face recognition result.

6. The face recognition method according to claim 5, characterized in that: The performing face recognition according to the fused face image and the infrared image to obtain the face recognition result comprises: extracting facial features from the fused facial image and iris features from the infrared image respectively; The facial feature information is compared based on the facial feature and the iris feature to obtain a facial recognition result.

7. The face recognition method according to claim 1, characterized in that: The illumination removal model is trained in the following way: Use 3D rendering to generate illuminated rendered images and unilluminated real images; Constructing a training sample pair according to the illuminated rendered image and the unilluminated real image; The training samples are used to train an illumination removal model.

8. A face recognition device, characterized in that: The face recognition device comprises: An acquisition unit, used to acquire a visible light image of a face and a corresponding infrared image; an illumination removal unit, configured to, when the visible light image of the human face is a high-illumination image, perform illumination removal processing on the visible light image of the human face using an illumination removal model to obtain the visible light image of the human face after the illumination removal processing; A reconstruction unit, used for performing face reconstruction based on the visible light image of the face after the illumination removal process and the corresponding infrared image to obtain a reconstructed face image; A face recognition unit is used to perform face recognition based on the reconstructed face image, the visible light face image and the infrared image to obtain a face recognition result.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the face recognition method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the face recognition method according to any one of claims 1 to 7 is implemented.

Images