Face recognition method and device and electronic equipment

Abstract

The invention relates to a face recognition method and device and electronic equipment. The face recognition method comprises the steps that a face image is acquired; inputting the face image into a brightness processing network model, and obtaining a brightness correction image after brightness processing through the brightness processing network model; inputting the brightness correction image into a feature extraction network model, and obtaining feature data associated with the face image through the feature extraction network model; and comparing the feature data with pre-stored face feature data to obtain a face recognition result. According to the invention, before face feature extraction is carried out through the feature extraction network model, the brightness processing network model is utilized to carry out brightness correction on the obtained face image, so that the feature extraction network model extracts the face features by utilizing the obtained brightness correction image; therefore, the accuracy of face recognition under the condition that the brightness of the face image is not ideal is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a face recognition method, device and electronic device. Background Art

[0002] Face recognition is to determine a person's identity information based on the image of the face. Compared with biometric recognition such as fingerprint recognition, face recognition has the advantages of easy image collection, no contact, and easy manual verification of recognition results. With the rapid development of e-commerce and electronic payment, face recognition has been widely used. At the same time, face recognition-based access control has also been widely used in smart communities and smart parks. Face recognition has become a widely used visual intelligence technology.

[0003] With the rapid development of deep learning technology, the accuracy of face recognition has been greatly improved. However, under natural conditions (without light shielding or fill light), the collected face images may have too high or too low image brightness due to lighting reasons. Especially when the image brightness is too low, it is difficult to directly recognize the face, resulting in a decrease in the accuracy of face recognition, thus affecting the application of face recognition. Summary of the invention

[0004] In view of this, the present disclosure provides a face recognition method, device and electronic device to improve the accuracy of face recognition when the brightness of a face image is not ideal due to lighting reasons, especially when the brightness of the face image is too low.

[0005] The technical solution of the present disclosure is achieved as follows:

[0006] A face recognition method, comprising:

[0007] Get face image;

[0008] Inputting the face image into a brightness processing network model, and obtaining a brightness corrected image after brightness processing through the brightness processing network model;

[0009] Inputting the brightness-corrected image into a feature extraction network model, and obtaining feature data associated with the face image through the feature extraction network model;

[0010] The feature data is compared with the previously stored facial feature data to obtain a facial recognition result.

[0011] Furthermore, the face recognition method further includes:

[0012] Acquire a face sample data set, wherein the face sample data set includes a plurality of face sample images;

[0013] Inputting all the face sample images in the face sample data set into the brightness processing network model to be trained respectively, and obtaining brightness-corrected sample images corresponding to each of the face sample images respectively through the brightness processing network model to be trained;

[0014] Inputting all the brightness-corrected sample images into the feature extraction network model to be trained respectively, and obtaining feature prediction data corresponding to each of the face sample images respectively through the feature extraction network model to be trained;

[0015] Inputting all the feature prediction data into the identity classifier to be trained respectively, and obtaining the identity prediction results corresponding to each of the face sample images respectively through the identity classifier to be trained;

[0016] Obtaining, according to the brightness-corrected sample image and the face sample image, an illumination smoothing loss between the brightness-corrected sample image and the face sample image, wherein the brightness-corrected sample image corresponds to the face sample image;

[0017] According to all the feature prediction data and the illumination cluster center to which each of the face sample images belongs, the face feature loss of the same person at different illumination cluster centers and the face feature loss of different people at the same illumination cluster center are obtained;

[0018] Obtaining a face identity classification loss according to the identity prediction result and the identity label of the face sample image, wherein the identity prediction result corresponds to the face sample image;

[0019] Obtaining an overall loss according to the illumination smoothing loss, the face feature loss of the same person with different illumination cluster centers, the face feature loss of different people with the same illumination cluster centers, and the face identity classification loss;

[0020] The network model including the brightness processing network model to be trained, the feature extraction network model to be trained and the identity classifier to be trained is trained according to the overall loss to obtain the brightness processing network model and the feature extraction network model.

[0021] Furthermore, the facial feature loss of the same person with different illumination cluster centers and the facial feature loss of different people with the same illumination cluster center are obtained based on all the feature prediction data and the illumination cluster center to which each of the face sample images belongs, including:

[0022] According to the illumination cluster center to which each of the face sample images belongs, obtaining from the face sample data set a first face sample image pair belonging to the same face and belonging to different illumination cluster centers, and a second face sample image pair belonging to different faces and belonging to the same illumination cluster center;

[0023] Obtaining the face feature loss of the cluster center of different illuminations of the same person according to two feature prediction data obtained by sequentially inference of the brightness processing network model to be trained and the feature extraction network model to be trained on the two face sample images in the first face sample image pair;

[0024] According to the two feature prediction data obtained by sequentially inferring the two face sample images in the second face sample image pair through the brightness processing network model to be trained and the feature extraction network model to be trained, the face feature loss of the cluster center of different people with the same illumination is obtained.

[0025] Furthermore, the face recognition method further includes:

[0026] According to the multiple illumination clustering centers obtained by illumination clustering processing of the multiple face sample images, the illumination clustering center to which each of the face sample images belongs is obtained.

[0027] Further, the illumination clustering process includes:

[0028] Normalizing the face sample image to a set size to obtain a normalized face sample image;

[0029] Dividing the normalized face sample image into a plurality of illumination regions of the same size;

[0030] Acquire illumination features of each of the illumination areas from the normalized face sample image;

[0031] Obtaining the overall illumination feature of the face sample image according to the illumination feature of each illumination area;

[0032] Clustering the overall illumination features of all the face sample images in the face sample data set to obtain a plurality of illumination clustering centers;

[0033] According to the overall illumination feature of each of the face sample images and the plurality of illumination cluster centers, the illumination cluster center to which each of the face sample images belongs is obtained.

[0034] Further, the two face sample images in the first face sample image pair are sequentially inferred by the brightness processing network model to be trained and the feature extraction network model to be trained to obtain the face feature loss of the cluster center of different illuminations of the same person, including:

[0035] The normalized feature prediction data is obtained by the following formula:

[0036] Fni=Fi / norm(Fi)

[0037] Fnj=Fj / norm(Fj)

[0038] Wherein, Fni represents the normalized feature prediction data of one face sample image in the first face sample image pair, Fnj represents the normalized feature prediction data of the other face sample image in the first face sample image pair, norm(X) represents taking the modulus of the vector X, Fi represents feature prediction data of one face sample image in the first face sample image pair, and Fj represents feature prediction data of the other face sample image in the first face sample image pair;

[0039] The loss between the feature prediction data of the two face sample images in the first face sample image pair is obtained by the following formula:

[0040] Lsame_ij=1-Fni*Fnj

[0041] Wherein, Lsame_ij is the loss between the feature prediction data of the two face sample images in the first face sample image pair;

[0042] The face feature loss of the cluster center of the same person under different illuminations is obtained by the following formula:

[0043] Lsame=∑Lsame_ij

[0044] Wherein, Lsame represents the facial feature loss of the cluster center of the same person under different illumination conditions, and ij represents all first face sample image pairs in the face sample data set.

[0045] Further, the two feature prediction data obtained by sequentially inferring the brightness processing network model to be trained and the feature extraction network model to be trained based on the two face sample images in the second face sample image pair to obtain the face feature loss of the cluster center of the same illumination for different persons include:

[0046] The normalized feature prediction data is obtained by the following formula:

[0047] Fnk=Fk / norm(Fk)

[0048] Fnl=Fl / norm(Fl)

[0049] Wherein, Fnk represents the normalized feature prediction data of one face sample image in the second face sample image pair, Fnl represents the normalized feature prediction data of the other face sample image in the second face sample image pair, norm(X) represents taking the modulus of the vector X, Fk represents feature prediction data of one face sample image in the second face sample image pair, and Fl represents feature prediction data of the other face sample image in the second face sample image pair;

[0050] The loss between the feature prediction data of the two face sample images in the second face sample image pair is obtained by the following formula:

[0051] Ldiff_kl=Fnk*Fnl

[0052] Wherein, Ldiff_kl is the loss between the feature prediction data of the two face sample images in the second face sample image pair;

[0053] The face feature loss of the cluster center of different people with the same illumination is obtained by the following formula:

[0054] Ldiff=∑Ldiff_kl

[0055] Wherein, Ldiff represents the face feature loss of the cluster center of different persons with the same illumination, and kl represents all the second face sample image pairs in the face sample data set.

[0056] Furthermore, the overall loss is obtained according to the illumination smoothing loss, the face feature loss of the same person with different illumination cluster centers, the face feature loss of different people with the same illumination cluster centers, and the face identity classification loss, including:

[0057] The overall loss is obtained by the following formula:

[0058] L=Ls*Ws+Lid*Wid+Lsame*Wsame+Ldiff*Wdiff

[0059] Among them, L represents the overall loss, Ls represents the illumination smoothing loss, Lid represents the face identity classification loss, Lsame represents the face feature loss of the same person in different illumination cluster centers, Ldiff represents the face feature loss of different people in the same illumination cluster center, Ws is the weighting coefficient of Ls, Wid is the weighting coefficient of Lid, Wsame is the weighting coefficient of Lsame, and Wdiff is the weighting coefficient of Ldiff.

[0060] A face recognition device, comprising:

[0061] An image acquisition module is configured to acquire a face image;

[0062] A brightness correction module is configured to input the face image into a brightness processing network model, and obtain a brightness-corrected image after brightness processing through the brightness processing network model;

[0063] A feature extraction module is configured to input the brightness-corrected image into a feature extraction network model, and obtain feature data associated with the face image through the feature extraction network model;

[0064] The data comparison module is configured to compare the feature data with pre-stored facial feature data to obtain a facial recognition result.

[0065] An electronic device, comprising:

[0066] processor;

[0067] a memory for storing executable instructions for the processor;

[0068] Wherein, the processor is configured to execute the executable instructions to implement the face recognition method as described in any one of the above items.

[0069] It can be seen from the above scheme that the face recognition method, device and electronic device disclosed in the present invention use a brightness processing network model to perform brightness correction on the acquired face image before extracting face features through a feature extraction network model, so that the feature extraction network model extracts face features using the obtained brightness corrected image. The face recognition method, device and electronic device disclosed in the present invention combine the brightness processing network model and the feature extraction network model, thereby improving the accuracy of face recognition when the brightness of the face image is not ideal.

[0070] On this basis, in order to obtain a more optimized effect, in the present disclosure, when jointly training the brightness processing network model and the feature extraction network model, a scheme of illumination clustering is designed and proposed in combination with the needs of face recognition. On the basis of the face identity classification loss used in face recognition, two face feature losses for illumination clustering are proposed, and an illumination smoothing loss constraint based on illumination characteristics is used for the brightness processing network model. Finally, these losses are combined and the identity classifier is assisted to train the network model, which not only ensures the visual effect of the network model for processing images with undesirable brightness, but also improves the accuracy of face recognition, especially greatly improves the accuracy of face recognition of dark light images. Since the brightness processing network model and feature extraction network model used can be lightweight networks, the face recognition method, device and electronic device disclosed in the present disclosure also realize end-to-end processing of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 is a flow chart of a face recognition method according to an exemplary embodiment;

[0072] Figure 2 is a schematic diagram of a process of training a joint network according to an exemplary embodiment;

[0073] Figure 3 is a structural diagram of a joint network training process according to an exemplary embodiment

[0074] Figure 4 is a schematic diagram of a process of obtaining two types of face feature losses according to an exemplary embodiment;

[0075] Figure 5 is a schematic diagram of a process for obtaining the face feature loss of cluster centers of the same person under different illuminations according to an exemplary embodiment;

[0076] Figure 6 is a schematic diagram of a process for obtaining face feature loss of cluster centers of different persons with the same illumination according to an exemplary embodiment;

[0077] Figure 7 is a schematic diagram of a process of illumination clustering processing according to an exemplary embodiment;

[0078] Figure 8 is a schematic diagram of an application scenario flow chart of a face recognition method according to an exemplary embodiment;

[0079] Fig. 9 is a schematic diagram of the logical structure of a face recognition device according to an exemplary embodiment;

[0080] Fig.10 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0081] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below with reference to the accompanying drawings and examples.

[0082] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0083] The main purpose of existing low-light image processing technologies is to improve the visual effect of low-light images. These existing low-light image processing technologies do not take into account subsequent face recognition. Although the visual effect of the image is improved, face recognition shows that the improvement of face recognition effect by existing low-light image processing technologies is very limited and cannot meet the needs of face recognition in actual scenes.

[0084] In view of the above problems existing in the existing dark light image processing technology, the embodiments of the present disclosure propose a face recognition method, device and electronic device combined with brightness processing, and use the brightness processing network model and the feature extraction network model to optimize the image brightness processing and face recognition as an end-to-end network model as a whole, which can improve the accuracy of face recognition when the image brightness is too high or too low, especially when the image brightness is too low, the processing effect of the dark light image is better. In the embodiments of the present disclosure, when training the network model, the brightness processing of the image adopts the illumination smoothing loss constraint based on the illumination characteristics, and at the same time, combined with the needs of face recognition, illumination clustering is designed and proposed. On the basis of the face identity classification loss commonly used in face recognition, two face feature losses for illumination clustering are proposed; finally, these losses are combined and the identity classifier is assisted to train the network model, which not only ensures the visual effect of the network model processing the image with unsatisfactory brightness, but also improves the accuracy of face recognition, especially greatly improves the accuracy of face recognition of dark light images.

[0085] Figure 1 is a flow chart of a face recognition method according to an exemplary embodiment. Figure 1 As shown, the face recognition method mainly includes the following steps 101 to 104.

[0086] Step 101: Acquire a face image.

[0087] In an exemplary embodiment, the facial image may be directly acquired by a camera.

[0088] Step 102: input the face image into the brightness processing network model, and obtain a brightness corrected image after brightness processing through the brightness processing network model.

[0089] The brightness processing network model is used to correct the brightness of the face image. In an exemplary embodiment, the brightness processing network model may adopt a convolutional neural network (CNN) structure. In an exemplary embodiment, the brightness processing network model may include a convolutional layer, an activation layer, etc., and may adopt a CNN network model such as U-Net (U-Net: Convolutional Networks for Biomedical Image Segmentation), requiring the network input and final output to be of the same size.

[0090] Step 103: input the brightness corrected image into a feature extraction network model, and obtain feature data associated with the face image through the feature extraction network model.

[0091] The feature extraction network model is used to extract feature data of the face image from the brightness corrected image. In an exemplary embodiment, the feature extraction network model may adopt a convolutional neural network (CNN) structure. In an exemplary embodiment, the feature extraction network model may adopt a commonly used face recognition model such as MobileFaceNet (MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices) and other network models.

[0092] Step 104: Compare the feature data with the previously stored facial feature data to obtain a facial recognition result.

[0093] In an illustrative embodiment, the pre-stored facial feature data can be obtained through the brightness processing network model and the feature extraction network model in the embodiment of the present disclosure. For example, when registering a face, the registration facial image is input into the brightness processing network model to obtain a registration brightness-corrected image after brightness processing, and then the registration brightness-corrected image is input into the feature extraction network model to obtain the pre-stored facial feature data.

[0094] The face recognition method of the disclosed embodiment adds a brightness processing network model before the feature extraction network model. The brightness processing network model and the feature extraction network model together form a joint network to extract feature data from the face image, thereby improving the accuracy of face recognition when the image brightness is too high or too low, and in particular, improving the feature extraction effect of dark light images.

[0095] In order to ensure that the joint network composed of the brightness processing network model and the feature extraction network model can improve the recognition accuracy of facial images with different brightness, before recognizing the facial image, the face recognition method of the embodiment of the present disclosure also includes a process of training the joint network. Figure 2 is a schematic diagram of a process of training a joint network according to an exemplary embodiment, Figure 3 is a structural diagram of a joint network training process according to an exemplary embodiment, such as Figure 2 and Figure 3 As shown, the training process mainly includes the following steps 201 to 209.

[0096] Step 201: Obtain a face sample data set, where the face sample data set includes a plurality of face sample images.

[0097] In an illustrative embodiment, the face sample data set includes at least two face sample images belonging to the same face and belonging to different illumination cluster centers, and the face sample data set also includes at least two face sample images belonging to different faces and belonging to the same illumination cluster center.

[0098] Step 202: input all the face sample images in the face sample data set into the brightness processing network model to be trained respectively, and obtain brightness-corrected sample images corresponding to each face sample image respectively through the brightness processing network model to be trained.

[0099] Step 203: input all brightness-corrected sample images into the feature extraction network model to be trained, and obtain feature prediction data corresponding to each face sample image through the feature extraction network model to be trained.

[0100] Step 204: input all feature prediction data into the identity classifier to be trained respectively, and obtain the identity prediction results corresponding to each face sample image respectively through the identity classifier to be trained.

[0101] For example, if the face sample data set contains 1.28 million face sample images, multiple batches of face sample images can be selected from the 1.28 million face sample images during training, and training can be performed using each batch of face sample images. In an example, each batch of face sample images includes 128 face sample images. In step 202, the selected 128 face sample images are sequentially input into the brightness processing network model to be trained to obtain 128 brightness-corrected sample images, and the 128 brightness-corrected sample images correspond one-to-one to each of the 128 face sample images; in step 203, the 128 brightness-corrected sample images are sequentially input into the feature extraction network model to be trained to obtain 128 feature prediction data, and the 128 feature prediction data correspond one-to-one to each of the 128 face sample images; in step 204, the 128 feature prediction data are sequentially input into the identity classifier to be trained to obtain 128 identity prediction results, and the 128 identity prediction results correspond one-to-one to each of the 128 face sample images.

[0102] In an exemplary embodiment, the identity classifier to be trained adopts a softmax classifier to be trained.

[0103] Step 205: Obtain an illumination smoothing loss between the brightness-corrected sample image and the face sample image according to the brightness-corrected sample image and the face sample image, wherein the brightness-corrected sample image corresponds to the face sample image.

[0104] In an exemplary embodiment, the illumination smoothing loss is obtained by the following formula:

[0105] Ls=∑(Gx(abs(I2-I1))+Gy(abs(I2-I1)))

[0106] Among them, I1 is the face sample image, I2 is the brightness corrected sample image obtained by the face sample image I1 through the brightness processing network model to be trained, Gx and Gy represent the gradients in the x and y directions respectively, abs() represents taking the absolute value of the result in the brackets, Ls is the illumination smoothing loss, and ∑ represents the sum.

[0107] Step 206: According to all feature prediction data and the illumination cluster center to which each face sample image belongs, the face feature loss of the same person at different illumination cluster centers and the face feature loss of different people at the same illumination cluster center are obtained.

[0108] Step 207: Obtain a face identity classification loss based on the identity prediction result and the identity label of the face sample image, wherein the identity prediction result corresponds to the face sample image.

[0109] In an illustrative embodiment, for the softmax classifier, the identity classification loss in face recognition can be directly used, or an improved loss such as ArcFace loss (see ArcFace: Additive Angular Margin Lossfor Deep Face Recognition, published on January 23, 2018) can be used as the face identity classification loss.

[0110] Step 208: Obtain the overall loss based on the illumination smoothing loss, the face feature loss of the same person with different illumination cluster centers, the face feature loss of different people with the same illumination cluster centers, and the face identity classification loss.

[0111] In an exemplary embodiment, step 208 may specifically include:

[0112] The overall loss is obtained by the following formula:

[0113] L=Ls*Ws+Lid*Wid+Lsame*Wsame+Ldiff*Wdiff

[0114] Among them, L represents the overall loss, Ls represents the illumination smoothing loss, Lid represents the face identity classification loss, Lsame represents the face feature loss of the same person in different illumination cluster centers, Ldiff represents the face feature loss of different people in the same illumination cluster center, Ws is the weighting coefficient of Ls, Wid is the weighting coefficient of Lid, Wsame is the weighting coefficient of Lsame, and Wdiff is the weighting coefficient of Ldiff.

[0115] Step 209: Train the network model including the brightness processing network model to be trained, the feature extraction network model to be trained, and the identity classifier to be trained according to the overall loss to obtain the brightness processing network model and the feature extraction network model.

[0116] In an illustrative embodiment, step 209 may include, according to the overall loss, adjusting the parameters of the brightness processing network model to be trained, the feature extraction network model to be trained, and the identity classifier to be trained through back propagation until the loss value of the overall loss function converges to the expected value or reaches the number of training iterations. After the training is completed, the brightness processing network model and the feature extraction network model are retained. The identity classifier to be trained is to assist the training of the brightness processing network model to be trained and the feature extraction network model to be trained to realize the extraction of feature data of the face image. After the training is completed, when the application of face recognition is implemented, the identity classifier is no longer needed, but the feature data associated with the face image obtained by the feature extraction network model is compared with the face feature data stored in advance to determine whether the recognized face image is a legitimate user.

[0117] Figure 4 FIG. 1 is a flow chart of obtaining two types of face feature losses according to an exemplary embodiment. Figure 4 As shown, step 206 may specifically include the following steps 401 to 403.

[0118] Step 401, according to the illumination cluster center to which each face sample image belongs, obtain from the face sample data set a first face sample image pair belonging to the same face and belonging to different illumination cluster centers, and a second face sample image pair belonging to different faces and belonging to the same illumination cluster center.

[0119] In order to ensure accurate recognition of the same person under different lighting conditions, in the exemplary embodiment, the first face sample image pair is at least one pair. In order to ensure accurate recognition of different people under the same lighting conditions, in the exemplary embodiment, the second face sample image pair is at least one pair.

[0120] Step 402: Obtain the facial feature loss of the cluster center of different illumination for the same person based on two feature prediction data obtained by sequentially inferring the brightness processing network model to be trained and the feature extraction network model to be trained based on the two facial sample images in the first facial sample image pair.

[0121] Figure 5 FIG. 1 is a flow chart of obtaining the face feature loss of cluster centers of the same person under different illumination conditions according to an exemplary embodiment. Figure 5 As shown, in the exemplary embodiment, step 402 may specifically include the following steps 501 to 503.

[0122] Step 501: obtain normalized feature prediction data by the following formula:

[0123] Fni=Fi / norm(Fi)

[0124] Fnj=Fj / norm(Fj)

[0125] Wherein, Fni represents the normalized feature prediction data of one face sample image in the first face sample image pair, Fnj represents the normalized feature prediction data of the other face sample image in the first face sample image pair, norm(X) represents taking the modulus of the vector X, X represents the vector in the brackets, for example, X can represent Fi in norm(Fi), and X can also represent Fj in norm(Fj), Xi represents the i-th element in the vector X, Fi represents the feature prediction data of one face sample image in the first face sample image pair, and Fj represents the feature prediction data of the other face sample image in the first face sample image pair.

[0126] Step 502: Obtain the loss between the feature prediction data of two face sample images in the first face sample image pair by the following formula:

[0127] Lsame_ij=1-Fni*Fnj

[0128] Among them, Lsame_ij is the loss between the feature prediction data of the two face sample images in the first face sample image pair.

[0129] Step 503: The face feature loss of the cluster center of the same person under different illuminations is obtained by the following formula:

[0130] Lsame=∑Lsame_ij

[0131] Among them, Lsame represents the face feature loss of the cluster center of the same person under different illumination, and ij represents all the first face sample image pairs in the face sample dataset.

[0132] Step 403: Obtain the face feature loss of the same illumination cluster center of different persons based on the two face sample images in the second face sample image pair, by sequentially inferring the brightness processing network model to be trained and the feature extraction network model to be trained to obtain two feature prediction data.

[0133] Figure 6 FIG. 1 is a flow chart of obtaining the face feature loss of cluster centers of different people with the same illumination according to an exemplary embodiment. Figure 6 As shown, in the exemplary embodiment, step 403 may specifically include the following steps 601 to 603.

[0134] Step 601: obtain normalized feature prediction data by the following formula:

[0135] Fnk=Fk / norm(Fk)

[0136] Fnl=Fl / norm(Fl)

[0137] Wherein, Fnk represents the normalized feature prediction data of one face sample image in the second face sample image pair, Fnl represents the normalized feature prediction data of the other face sample image in the second face sample image pair, norm(X) represents taking the modulus of the vector X, Fk represents feature prediction data of one face sample image in the second face sample image pair, and Fl represents feature prediction data of the other face sample image in the second face sample image pair.

[0138] Step 602: Obtain the loss between the feature prediction data of two face sample images in the second face sample image pair by the following formula:

[0139] Ldiff_kl=Fnk*Fnl

[0140] Wherein, Ldiff_kl is the loss between the feature prediction data of the two face sample images in the second face sample image pair.

[0141] Step 603: The face feature loss of cluster centers of different people with the same illumination is obtained by the following formula:

[0142] Ldiff=∑Ldiff_kl

[0143] Among them, Ldiff represents the face feature loss of cluster centers of different people with the same illumination, and kl represents all the second face sample image pairs in the face sample dataset.

[0144] In the above description, illumination cluster centers are involved. For example, in step 201, it is mentioned that the face sample data set contains at least two face sample images belonging to the same face and belonging to different illumination cluster centers, and the face sample data set also contains at least two face sample images belonging to different faces and belonging to the same illumination cluster center; in step 206, it is mentioned that according to all feature prediction data and the illumination cluster center to which each face sample image belongs, the face feature loss of the same person with different illumination cluster centers and the face feature loss of different people with the same illumination cluster center are obtained. Based on this, the face recognition method of the embodiment of the present disclosure further includes steps for obtaining illumination cluster centers, specifically including:

[0145] According to the multiple illumination clustering centers obtained by illumination clustering processing of the multiple face sample images, the illumination clustering center to which each face sample image belongs is obtained.

[0146] Figure 7 is a schematic diagram of a process of illumination clustering processing according to an exemplary embodiment. Figure 7 As shown, in the exemplary embodiment, the illumination clustering process mainly includes the following steps 701 to 706 .

[0147] Step 701: Normalize the sample face image to a set size to obtain a normalized sample face image.

[0148] Normalizing the face sample images to a set size refers to normalizing each face sample image to a set size.

[0149] In an illustrative embodiment, the face image can be normalized to a size of 112*112 (pixels) based on the five-point positions of the face in the face sample image (including the center of the left eye, the center of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth) and the reference positions corresponding to the five-point positions to obtain a normalized face sample image.

[0150] Step 702: Divide the normalized face sample image into a plurality of illumination regions of the same size.

[0151] In an exemplary embodiment, the normalized face sample image may be divided into 3×3 regions of the same size.

[0152] Step 703: Obtain illumination features of each illumination area from the normalized face sample image.

[0153] In an exemplary embodiment, the illumination feature may be, for example, a histogram feature. For example, for each of the 3×3 regions of equal size divided by the normalized face sample image, a histogram feature hi of each region is obtained as the illumination feature of each region, where hi represents the histogram feature of any one of all the regions.

[0154] Among them, extracting histogram features is a commonly used image feature extraction method, which is mainly used to count and represent the color distribution and brightness distribution of an image. The following is a common method for extracting histogram features:

[0155] 1) Color space selection: First, select an appropriate color space, such as any one of the color spaces such as RGB, HSV, Lab, etc. Selecting an appropriate color space can better represent the color distribution of the image.

[0156] 2) Image segmentation: If you need to extract histogram features from a specific area, you can first segment the image and extract the required area.

[0157] 3) Division and grouping: Divide the color space into several small intervals or groups, such as dividing the RGB space into 256 groups (0-255).

[0158] 4) Statistical analysis: Perform statistical analysis on the pixels in each group and calculate the number or proportion of pixels in each group.

[0159] 5) Feature representation: The number or proportion of each group obtained by statistics is used as the representation of the histogram feature. The histogram features of all dimensions can be concatenated to form a feature vector.

[0160] 6) Normalization: The feature vector can be normalized so that each dimension of the feature vector has equal importance.

[0161] Step 704: Obtain the overall illumination feature of the face sample image according to the illumination feature of each illumination area.

[0162] In an exemplary embodiment, the illumination feature may be obtained by, for example, concatenating the histogram features of each region. For example, for a normalized face sample image divided into 3×3 regions of equal size, the histogram features of all regions are concatenated together to obtain the illumination feature of the entire normalized face sample image, and the illumination feature of the entire normalized face sample image may be expressed as L=(h1, h2, ..., h9), where L represents the illumination feature of the normalized face sample image, h1 represents the histogram feature of the first region, h2 represents the histogram feature of the second region, and h9 represents the histogram feature of the ninth region.

[0163] Step 705: cluster the overall illumination features of all face sample images in the face sample data set to obtain multiple illumination cluster centers.

[0164] In an exemplary embodiment, various clustering methods may be used to cluster the overall illumination features of all face sample images in the face sample data set. For example, a mean-shift clustering method may be used for clustering.

[0165] Among them, mean clustering is a density-based non-parametric clustering method. Mean clustering starts from the points in the data set, calculates the centroid (mean) of each point and moves the centroid to the location with the largest local density, repeating this process until the centroid no longer moves or converges to the local maximum point. Finally, the data points near the convergence point are assigned to the centroid to form a cluster. Mean clustering is an iterative process that does not require the number of clusters to be specified in advance. The working principle of mean clustering is to determine the density of points by calculating the distance between sample points, and moving the centroid in each iteration to maximize the density. This process brings a group of sample points together and forms clusters.

[0166] In addition to mean clustering, you can also use K-means clustering, hierarchical clustering, density clustering, model-based clustering, grid-based clustering, spectral clustering, fuzzy clustering and other clustering methods.

[0167] Step 706: Obtain the illumination cluster center to which each face sample image belongs based on the overall illumination characteristics of each face sample image and the multiple illumination cluster centers.

[0168] In an illustrative embodiment, when the mean clustering method is used, the illumination cluster center to which the face sample image belongs can be determined by the distance between the overall illumination features of the face sample image and the cluster center. For example, a measurement method such as Euclidean distance can be used to calculate the distance between the overall illumination features of each face sample image and each illumination cluster center. For a certain face sample image, the illumination cluster center closest to its overall illumination features is determined as the illumination cluster center to which it belongs.

[0169] K-means clustering is similar to mean clustering. It also determines the illumination cluster center to which the face sample image belongs through the distance between the overall illumination characteristics of the face sample image and the cluster center.

[0170] The hierarchical clustering method is a method based on merging and splitting, in which each sample (the overall illumination feature of the face sample image) is initially regarded as a separate cluster, and then the clusters are gradually merged or split by calculating the similarity and distance between different samples (the overall illumination feature of the face sample image). To determine which cluster center the sample (the overall illumination feature of the face sample image) belongs to, the similarity or distance between the sample (the overall illumination feature of the face sample image) and each cluster is calculated, and it is assigned to the cluster with the largest similarity or distance.

[0171] Density clustering is clustering by assigning samples (overall illumination features of face sample images) to areas with sufficient density. To determine which cluster center a sample (overall illumination features of face sample images) belongs to, the sample (overall illumination features of face sample images) is divided based on its density and its relationship with other samples (overall illumination features of face sample images).

[0172] Figure 8FIG. 1 is a flow chart of an application scenario of a face recognition method according to an exemplary embodiment. The application scenario is the overall process from model training to application implementation. Figure 8 As shown, the application scenario mainly includes the following steps 801 to 820.

[0173] Step 801: Obtain a face sample data set, where the face sample data set includes a plurality of face sample images, and then execute step 802.

[0174] The face sample data set includes at least two face sample images belonging to the same face and belonging to different illumination cluster centers, and the face sample data set also includes at least two face sample images belonging to different faces and belonging to the same illumination cluster center.

[0175] Step 802 , normalize each face sample image to a set size to obtain a normalized face sample image, and then execute step 803 .

[0176] Step 803: Divide each normalized face sample image into a plurality of illumination areas of the same size, and then execute step 804.

[0177] Step 804 : for each normalized human face sample image, obtain the overall illumination feature of each human face sample image according to the illumination feature of each illumination area, and then execute step 805 .

[0178] Step 805 , clustering the overall illumination features of all face sample images in the face sample data set to obtain multiple illumination clustering centers, and then executing step 806 .

[0179] Step 806 : According to the overall illumination characteristics of each face sample image and multiple illumination cluster centers, obtain the illumination cluster center to which each face sample image belongs, and then execute step 807 .

[0180] Step 807, according to the illumination cluster center to which each face sample image belongs, obtain from the face sample data set a first face sample image pair belonging to the same face and belonging to different illumination cluster centers, and a second face sample image pair belonging to different faces and belonging to the same illumination cluster center, and then execute step 808.

[0181] In an illustrative embodiment, each face sample image in the face sample data set corresponds to an identity label, which indicates the identity of the corresponding face sample image. If the identity labels corresponding to two or more face sample images in the face sample data set are the same, it indicates that the two or more face sample images are face sample images of the same person.

[0182] After step 806, each face sample image in the face sample data set also corresponds to its own illumination cluster center. In step 807, the first face sample image pair and the second face sample image pair can be obtained according to the identity label corresponding to each face sample image in the face sample data set and the illumination cluster center corresponding to each face sample image, wherein the identity labels between the two face sample images in the first face sample image pair are the same and the illumination cluster centers are different, and the identity labels between the two face sample images in the second face sample image pair are different and the illumination cluster centers are the same.

[0183] Step 808: input all the face sample images in the face sample data set into the brightness processing network model to be trained respectively, obtain the brightness corrected sample images corresponding to each face sample image respectively through the brightness processing network model to be trained, and then execute step 809.

[0184] Step 809: input all brightness-corrected sample images into the feature extraction network model to be trained respectively, obtain feature prediction data corresponding to each face sample image respectively through the feature extraction network model to be trained, and then execute step 810.

[0185] Step 810: input all feature prediction data into the identity classifier to be trained respectively, and obtain the identity prediction results corresponding to each face sample image respectively through the identity classifier to be trained, and then execute steps 811, 812, 813 and 814. After completing steps 811, 812, 813 and 814, execute step 815.

[0186] Step 811: Obtain an illumination smoothing loss between the brightness-corrected sample image and the face sample image according to the brightness-corrected sample image and the face sample image, wherein the brightness-corrected sample image corresponds to the face sample image.

[0187] Among them, the illumination smoothing loss is obtained by the following formula:

[0188] Ls=∑(Gx(abs(I2-I1))+Gy(abs(I2-I1)))

[0189] Among them, I1 is the face sample image, I2 is the brightness corrected sample image obtained by the face sample image I1 through the brightness processing network model to be trained, Gx and Gy represent the gradients in the x and y directions respectively, abs() represents taking the absolute value of the result in the brackets, Ls is the illumination smoothing loss, and ∑ represents the sum.

[0190] Step 812: Obtain the facial feature loss of the same person under different illumination clustering centers based on the two facial sample images in the first facial sample image pair, by sequentially inferring the brightness processing network model to be trained and the feature extraction network model to be trained to obtain two feature prediction data.

[0191] Wherein, step 812 specifically includes:

[0192] The normalized feature prediction data is obtained by the following formula:

[0193] Fni=Fi / norm(Fi)

[0194] Fnj=Fj / norm(Fj)

[0195] Wherein, Fni represents the normalized feature prediction data of one face sample image in the first face sample image pair, Fnj represents the normalized feature prediction data of the other face sample image in the first face sample image pair, norm(X) represents taking the modulus of the vector X, Fi represents feature prediction data of one face sample image in the first face sample image pair, and Fj represents feature prediction data of the other face sample image in the first face sample image pair;

[0196] The loss between the feature prediction data of the two face sample images in the first face sample image pair is obtained by the following formula:

[0197] Lsame_ij=1-Fni*Fnj

[0198] Wherein, Lsame_ij is the loss between the feature prediction data of the two face sample images in the first face sample image pair;

[0199] The face feature loss of the cluster center of the same person under different illuminations is obtained by the following formula:

[0200] Lsame=∑Lsame_ij

[0201] Among them, Lsame represents the face feature loss of the cluster center of the same person under different illumination, and ij represents all the first face sample image pairs in the face sample dataset.

[0202] Step 813: Obtain the facial feature loss of the same illumination cluster center for different persons based on the two facial sample images in the second facial sample image pair, by sequentially inferring the brightness processing network model to be trained and the feature extraction network model to be trained to obtain two feature prediction data.

[0203] Wherein, step 813 specifically includes:

[0204] The normalized feature prediction data is obtained by the following formula:

[0205] Fnk=Fk / norm(Fk)

[0206] Fnl=Fl / norm(Fl)

[0207] Wherein, Fnk represents the normalized feature prediction data of one face sample image in the second face sample image pair, Fnl represents the normalized feature prediction data of the other face sample image in the second face sample image pair, norm(X) represents taking the modulus of the vector X, Fk represents feature prediction data of one face sample image in the second face sample image pair, and Fl represents feature prediction data of the other face sample image in the second face sample image pair;

[0208] The loss between the feature prediction data of the two face sample images in the second face sample image pair is obtained by the following formula:

[0209] Ldiff_kl=Fnk*Fnl

[0210] Wherein, Ldiff_kl is the loss between the feature prediction data of the two face sample images in the second face sample image pair;

[0211] The face feature loss of cluster centers of different people with the same illumination is obtained by the following formula:

[0212] Ldiff=∑Ldiff_kl

[0213] Among them, Ldiff represents the face feature loss of cluster centers of different people with the same illumination, and kl represents all the second face sample image pairs in the face sample dataset.

[0214] Step 814: Obtain the face identity classification loss according to the identity prediction result and the identity label of the face sample image, wherein the identity prediction result corresponds to the face sample image.

[0215] In an illustrative embodiment, the identity classifier can be a softmax classifier. For the softmax classifier, the commonly used identity classification loss in face recognition can be directly adopted, or an improved loss such as ArcFace loss (see ArcFace: Additive Angular Margin Loss for Deep Face Recognition, published on January 23, 2018) can be adopted as the face identity classification loss.

[0216] Step 815, obtain the overall loss based on the illumination smoothing loss, the face feature loss of the same person with different illumination cluster centers, the face feature loss of different people with the same illumination cluster centers, and the face identity classification loss, and then execute step 816.

[0217] Among them, the overall loss is obtained by the following formula:

[0218] L=Ls*Ws+Lid*Wid+Lsame*Wsame+Ldiff*Wdiff

[0219] Among them, L represents the overall loss, Ls represents the illumination smoothing loss, Lid represents the face identity classification loss, Lsame represents the face feature loss of the same person in different illumination cluster centers, Ldiff represents the face feature loss of different people in the same illumination cluster center, Ws is the weighting coefficient of Ls, Wid is the weighting coefficient of Lid, Wsame is the weighting coefficient of Lsame, and Wdiff is the weighting coefficient of Ldiff.

[0220] Step 816: train the network model including the brightness processing network model to be trained, the feature extraction network model to be trained, and the identity classifier to be trained according to the overall loss to obtain the brightness processing network model and the feature extraction network model, and then execute step 817.

[0221] In an illustrative embodiment, step 816 may specifically include: adjusting the parameters of the brightness processing network model to be trained, the feature extraction network model to be trained, and the identity classifier to be trained through back propagation according to the overall loss until the loss value of the overall loss function converges to the expected value or reaches the number of training iterations.

[0222] After completing step 816, the brightness processing network model and the feature extraction network model are retained.

[0223] Step 817, obtain the face image, and then execute step 818.

[0224] Among them, the face image comes directly from the camera collection.

[0225] Step 818: input the face image into the brightness processing network model, obtain the brightness corrected image after brightness processing through the brightness processing network model, and then execute step 819.

[0226] Step 819: input the brightness corrected image into the feature extraction network model, obtain feature data associated with the face image through the feature extraction network model, and then execute step 820.

[0227] Step 820: Compare the feature data with the previously stored facial feature data to obtain a facial recognition result.

[0228] The pre-stored facial feature data may be obtained by using steps 817 to 819 after the training of the brightness processing network model and the feature extraction network model is completed and before the face recognition is performed. For example, when registering a face, the registration face image is input into the brightness processing network model to obtain a brightness-corrected registration image after brightness processing, and then the registration brightness-corrected image is input into the feature extraction network model to obtain facial feature data, and then the facial feature data is stored, that is, the pre-stored facial feature data. The pre-stored facial feature data is stored together with the identity information of the person associated with the pre-stored facial feature data, such as the person's identification number, name, etc. When performing face recognition, the obtained facial feature data is compared with the pre-stored facial feature data. If the two match, the identity information for face recognition is obtained, and on this basis, the user of face recognition can be authorized to proceed to the next action or operation; if the two do not match, the recognition fails, and on this basis, the user of face recognition is rejected to proceed to the next action or operation.

[0229] The face recognition method of the embodiment of the present disclosure uses a brightness processing network model to correct the brightness of the acquired face image before extracting face features through a feature extraction network model, so that the feature extraction network model uses the obtained brightness corrected image to extract face features. The face recognition method of the embodiment of the present disclosure combines the brightness processing network model and the feature extraction network model, thereby improving the accuracy of face recognition when the brightness of the face image is not ideal. On this basis, in order to obtain a more optimized effect, in the face recognition method of the embodiment of the present disclosure, when the brightness processing network model and the feature extraction network model are jointly trained, a scheme of illumination clustering is designed and proposed in combination with the needs of face recognition. On the basis of the face identity classification loss used in face recognition, two face feature losses for illumination clustering are proposed, and an illumination smoothing loss constraint based on illumination characteristics is used for the brightness processing network model. Finally, these losses are combined and the identity classifier is assisted to train the network model, which not only ensures the visual effect of the network model processing images with unideal brightness, but also improves the accuracy of face recognition, especially greatly improves the accuracy of face recognition of dark light images. Since the brightness processing network model and feature extraction network model used can be lightweight networks, the face recognition method of the embodiment of the present disclosure also realizes end-to-end processing of face recognition.

[0230] Fig. 9 is a schematic diagram of the logical structure of a face recognition device according to an exemplary embodiment. Fig. 9 As shown, the face recognition device mainly includes an image acquisition module 901, a brightness correction module 902, a feature extraction module 903 and a data comparison module 904.

[0231] The image acquisition module 901 is configured to acquire a face image;

[0232] The brightness correction module 902 is configured to input the face image into the brightness processing network model, and obtain a brightness corrected image after brightness processing through the brightness processing network model;

[0233] The feature extraction module 903 is configured to input the brightness corrected image into the feature extraction network model, and obtain feature data associated with the face image through the feature extraction network model;

[0234] The data comparison module 904 is configured to compare the feature data with the previously stored facial feature data to obtain a facial recognition result.

[0235] In an exemplary embodiment, the face recognition device further includes:

[0236] A sample data acquisition module is configured to acquire a face sample data set, where the face sample data set includes a plurality of face sample images;

[0237] The sample brightness correction module is configured to input all the face sample images in the face sample data set into the brightness processing network model to be trained, and obtain the brightness corrected sample images corresponding to each face sample image through the brightness processing network model to be trained;

[0238] A sample feature extraction module is configured to input all brightness-corrected sample images into the feature extraction network model to be trained, and obtain feature prediction data corresponding to each face sample image through the feature extraction network model to be trained;

[0239] The sample identity classification module is configured to input all feature prediction data into the identity classifier to be trained, and obtain the identity prediction results corresponding to each face sample image through the identity classifier to be trained;

[0240] An illumination smoothness loss obtaining module is configured to obtain an illumination smoothness loss between the brightness corrected sample image and the face sample image according to the brightness corrected sample image and the face sample image, wherein the brightness corrected sample image corresponds to the face sample image;

[0241] The face feature loss obtaining module is configured to obtain face feature losses of the same person with different illumination cluster centers and face feature losses of different people with the same illumination cluster centers based on all feature prediction data and the illumination cluster center to which each face sample image belongs;

[0242] A face identity classification loss obtaining module is configured to obtain a face identity classification loss according to an identity prediction result and an identity label of a face sample image, wherein the identity prediction result corresponds to the face sample image;

[0243] The overall loss obtaining module is configured to obtain the overall loss according to the illumination smoothing loss, the face feature loss of the same person with different illumination cluster centers, the face feature loss of different people with the same illumination cluster centers, and the face identity classification loss;

[0244] The training module is configured to execute training on a network model including a brightness processing network model to be trained, a feature extraction network model to be trained and an identity classifier to be trained according to the overall loss to obtain a brightness processing network model and a feature extraction network model.

[0245] In an exemplary embodiment, the face feature loss obtaining module includes:

[0246] The sample image pair obtaining submodule is configured to obtain, from the face sample data set, first face sample image pairs belonging to the same face and belonging to different illumination cluster centers, and second face sample image pairs belonging to different faces and belonging to the same illumination cluster center, according to the illumination cluster center to which each face sample image belongs;

[0247] The first face feature loss obtaining submodule is configured to execute two feature prediction data obtained by sequentially inferring the brightness processing network model to be trained and the feature extraction network model to be trained based on the two face sample images in the first face sample image pair, so as to obtain face feature losses of cluster centers of different illuminations of the same person;

[0248] The second facial feature loss acquisition submodule is configured to execute two feature prediction data obtained by inference based on the two facial sample images in the second facial sample image pair through the brightness processing network model to be trained and the feature extraction network model to be trained, so as to obtain the facial feature loss of the same illumination clustering center of different people.

[0249] In an exemplary embodiment, the face recognition device further includes:

[0250] The illumination cluster center acquisition module is configured to execute the illumination clustering processing based on the multiple face sample images to obtain the illumination cluster center to which each face sample image belongs.

[0251] In an exemplary embodiment, the illumination cluster center acquisition module includes:

[0252] A normalization submodule is configured to perform normalization of the face sample image to a set size to obtain a normalized face sample image;

[0253] An illumination region division submodule is configured to divide the normalized face sample image into a plurality of illumination regions of the same size;

[0254] A regional illumination feature acquisition submodule is configured to acquire illumination features of each illumination region from the normalized face sample image;

[0255] The overall illumination feature acquisition submodule is configured to obtain the overall illumination feature of the face sample image according to the illumination feature of each illumination area;

[0256] A clustering submodule is configured to perform clustering of the overall illumination features of all face sample images in the face sample data set to obtain a plurality of illumination clustering centers;

[0257] The cluster center determination submodule is configured to obtain the illumination cluster center to which each face sample image belongs based on the overall illumination characteristics of each face sample image and multiple illumination cluster centers.

[0258] In an exemplary embodiment, the first face feature loss obtaining submodule is further configured to execute:

[0259] The normalized feature prediction data is obtained by the following formula:

[0260] Fni=Fi / norm(Fi)

[0261] Fnj=Fj / norm(Fj)

[0262] Wherein, Fni represents the normalized feature prediction data of one face sample image in the first face sample image pair, Fnj represents the normalized feature prediction data of the other face sample image in the first face sample image pair, norm(X) represents taking the modulus of the vector X, Fi represents feature prediction data of one face sample image in the first face sample image pair, and Fj represents feature prediction data of the other face sample image in the first face sample image pair;

[0263] The loss between the feature prediction data of the two face sample images in the first face sample image pair is obtained by the following formula:

[0264] Lsame_ij=1-Fni*Fnj

[0265] Wherein, Lsame_ij is the loss between the feature prediction data of the two face sample images in the first face sample image pair;

[0266] The face feature loss of the cluster center of the same person under different illuminations is obtained by the following formula:

[0267] Lsame=∑Lsame_ij

[0268] Among them, Lsame represents the face feature loss of the cluster center of the same person under different illumination, and ij represents all the first face sample image pairs in the face sample dataset.

[0269] In an exemplary embodiment, the second facial feature loss obtaining submodule is further configured to execute:

[0270] The normalized feature prediction data is obtained by the following formula:

[0271] Fnk=Fk / norm(Fk)

[0272] Fnl=Fl / norm(Fl)

[0273] Wherein, Fnk represents the normalized feature prediction data of one face sample image in the second face sample image pair, Fnl represents the normalized feature prediction data of the other face sample image in the second face sample image pair, norm(X) represents taking the modulus of the vector X, Fk represents feature prediction data of one face sample image in the second face sample image pair, and Fl represents feature prediction data of the other face sample image in the second face sample image pair;

[0274] The loss between the feature prediction data of the two face sample images in the second face sample image pair is obtained by the following formula:

[0275] Ldiff_kl=Fnk*Fnl

[0276] Wherein, Ldiff_kl is the loss between the feature prediction data of the two face sample images in the second face sample image pair;

[0277] The face feature loss of cluster centers of different people with the same illumination is obtained by the following formula:

[0278] Ldiff=∑Ldiff_kl

[0279] Among them, Ldiff represents the face feature loss of cluster centers of different people with the same illumination, and kl represents all the second face sample image pairs in the face sample dataset.

[0280] In an exemplary embodiment, the overall loss obtaining module is further configured to execute:

[0281] The overall loss is obtained by the following formula:

[0282] L=Ls*Ws+Lid*Wid+Lsame*Wsame+Ldiff*Wdiff

[0283] Among them, L represents the overall loss, Ls represents the illumination smoothing loss, Lid represents the face identity classification loss, Lsame represents the face feature loss of the same person in different illumination cluster centers, Ldiff represents the face feature loss of different people in the same illumination cluster center, Ws is the weighting coefficient of Ls, Wid is the weighting coefficient of Lid, Wsame is the weighting coefficient of Lsame, and Wdiff is the weighting coefficient of Ldiff.

[0284] The face recognition device of the embodiment of the present disclosure uses a brightness processing network model to correct the brightness of the acquired face image before extracting face features through a feature extraction network model, so that the feature extraction network model uses the obtained brightness corrected image to extract face features. The face recognition device of the embodiment of the present disclosure combines the brightness processing network model and the feature extraction network model, thereby improving the accuracy of face recognition when the brightness of the face image is not ideal. On this basis, in order to obtain a more optimized effect, in the embodiment of the present disclosure, when the face recognition device jointly trains the brightness processing network model and the feature extraction network model, a scheme of illumination clustering is designed and proposed in combination with the needs of face recognition. On the basis of the commonly used face identity classification loss used in face recognition, two face feature losses for illumination clustering are proposed, and an illumination smoothing loss constraint based on illumination characteristics is used for the brightness processing network model. Finally, these losses are combined and the identity classifier is assisted to train the network model, which not only ensures the visual effect of the network model processing images with unideal brightness, but also improves the accuracy of face recognition, especially greatly improves the accuracy of face recognition of dark light images. Since the brightness processing network model and feature extraction network model used can be lightweight networks, the face recognition device of the embodiment of the present disclosure also realizes end-to-end processing of face recognition.

[0285] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0286] Among them, regarding the face recognition device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the face recognition method, and will not be elaborated here.

[0287] It should be noted that: the above embodiment only illustrates the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0288] Fig.10It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. In some embodiments, the electronic device is a server. The electronic device 1000 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1001 and one or more memories 1002, wherein at least one program code is stored in the memory 1002, and the at least one program code is loaded and executed by the processor 1001 to implement the face recognition method provided by the above-mentioned various embodiments. Of course, the electronic device 1000 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the electronic device 900 may also include other components for implementing device functions, which will not be repeated here.

[0289] In an exemplary embodiment, a computer-readable storage medium including at least one instruction is also provided, such as a memory including at least one instruction. The at least one instruction can be executed by a processor in a computer device to complete the face recognition method in the above embodiment.

[0290] Optionally, the above-mentioned computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may include ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device, etc.

[0291] In an exemplary embodiment, a computer program product is also provided, including one or more instructions, which can be executed by a processor of a computer device to complete the face recognition method provided by the above-mentioned embodiments.

[0292] The above are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A face recognition method, comprising: Get face image; Inputting the face image into a brightness processing network model, and obtaining a brightness corrected image after brightness processing through the brightness processing network model; Inputting the brightness-corrected image into a feature extraction network model, and obtaining feature data associated with the face image through the feature extraction network model; The feature data is compared with the previously stored facial feature data to obtain a facial recognition result.

2. The face recognition method according to claim 1, characterized in that: The face recognition method further comprises: Acquire a face sample data set, wherein the face sample data set includes a plurality of face sample images; Inputting all the face sample images in the face sample data set into the brightness processing network model to be trained respectively, and obtaining brightness-corrected sample images corresponding to each of the face sample images respectively through the brightness processing network model to be trained; Inputting all the brightness-corrected sample images into the feature extraction network model to be trained respectively, and obtaining feature prediction data corresponding to each of the face sample images respectively through the feature extraction network model to be trained; Inputting all the feature prediction data into the identity classifier to be trained respectively, and obtaining the identity prediction results corresponding to each of the face sample images respectively through the identity classifier to be trained; Obtaining, according to the brightness-corrected sample image and the face sample image, an illumination smoothing loss between the brightness-corrected sample image and the face sample image, wherein the brightness-corrected sample image corresponds to the face sample image; According to all the feature prediction data and the illumination cluster center to which each of the face sample images belongs, the face feature loss of the same person at different illumination cluster centers and the face feature loss of different people at the same illumination cluster center are obtained; Obtaining a face identity classification loss according to the identity prediction result and the identity label of the face sample image, wherein the identity prediction result corresponds to the face sample image; Obtaining an overall loss according to the illumination smoothing loss, the face feature loss of the same person with different illumination cluster centers, the face feature loss of different people with the same illumination cluster centers, and the face identity classification loss; The network model including the brightness processing network model to be trained, the feature extraction network model to be trained and the identity classifier to be trained is trained according to the overall loss to obtain the brightness processing network model and the feature extraction network model.

3. The face recognition method according to claim 2, characterized in that: The method of obtaining face feature loss of the same person at different illumination cluster centers and face feature loss of different people at the same illumination cluster center based on all the feature prediction data and the illumination cluster center to which each face sample image belongs includes: According to the illumination cluster center to which each of the face sample images belongs, obtaining from the face sample data set a first face sample image pair belonging to the same face and belonging to different illumination cluster centers, and a second face sample image pair belonging to different faces and belonging to the same illumination cluster center; Obtaining the face feature loss of the cluster center of different illuminations of the same person according to two feature prediction data obtained by sequentially inference of the brightness processing network model to be trained and the feature extraction network model to be trained on the two face sample images in the first face sample image pair; According to the two feature prediction data obtained by sequentially inferring the two face sample images in the second face sample image pair through the brightness processing network model to be trained and the feature extraction network model to be trained, the face feature loss of the cluster center of different people with the same illumination is obtained.

4. The face recognition method according to claim 2, characterized in that: The face recognition method further comprises: According to the multiple illumination clustering centers obtained by illumination clustering processing of the multiple face sample images, the illumination clustering center to which each of the face sample images belongs is obtained.

5. The face recognition method according to claim 4, characterized in that: The illumination clustering process comprises: Normalizing the face sample image to a set size to obtain a normalized face sample image; Dividing the normalized face sample image into a plurality of illumination regions of the same size; Acquire illumination features of each of the illumination areas from the normalized face sample image; Obtaining the overall illumination feature of the face sample image according to the illumination feature of each illumination area; Clustering the overall illumination features of all the face sample images in the face sample data set to obtain a plurality of illumination clustering centers; According to the overall illumination feature of each of the face sample images and the plurality of illumination cluster centers, the illumination cluster center to which each of the face sample images belongs is obtained.

6. The face recognition method according to claim 3, characterized in that: The method of obtaining the face feature loss of the cluster center of different illumination for the same person by sequentially inferring the two face sample images in the first face sample image pair through the brightness processing network model to be trained and the feature extraction network model to be trained to obtain the two feature prediction data includes: The normalized feature prediction data is obtained by the following formula: Fni=Fi / norm(Fi) Fnj=Fj / norm(Fj) Wherein, Fni represents the normalized feature prediction data of one face sample image in the first face sample image pair, Fnj represents the normalized feature prediction data of the other face sample image in the first face sample image pair, norm(X) represents taking the modulus of the vector X, Fi represents feature prediction data of one face sample image in the first face sample image pair, and Fj represents feature prediction data of the other face sample image in the first face sample image pair; The loss between the feature prediction data of the two face sample images in the first face sample image pair is obtained by the following formula: Lsame_ij=1-Fni*Fnj Wherein, Lsame_ij is the loss between the feature prediction data of the two face sample images in the first face sample image pair; The face feature loss of the cluster center of the same person under different illuminations is obtained by the following formula: Lsame=∑Lsame_ij Wherein, Lsame represents the facial feature loss of the cluster center of the same person under different illumination conditions, and ij represents all first face sample image pairs in the face sample data set.

7. The face recognition method according to claim 3, characterized in that: The method of obtaining the face feature loss of the cluster center of the same illumination for different persons according to the two face sample images in the second face sample image pair is performed by sequentially passing the brightness processing network model to be trained and the feature extraction network model to be trained to obtain the two feature prediction data, including: The normalized feature prediction data is obtained by the following formula: Fnk=Fk / norm(Fk) Fnl=Fl / norm(Fl) Wherein, Fnk represents the normalized feature prediction data of one face sample image in the second face sample image pair, Fnl represents the normalized feature prediction data of the other face sample image in the second face sample image pair, norm(X) represents taking the modulus of the vector X, Fk represents feature prediction data of one face sample image in the second face sample image pair, and Fl represents feature prediction data of the other face sample image in the second face sample image pair; The loss between the feature prediction data of the two face sample images in the second face sample image pair is obtained by the following formula: Ldiff_kl=Fnk*Fnl Wherein, Ldiff_kl is the loss between the feature prediction data of the two face sample images in the second face sample image pair; The face feature loss of the cluster center of different people with the same illumination is obtained by the following formula: Ldiff=∑Ldiff_kl Wherein, Ldiff represents the face feature loss of the cluster center of different persons with the same illumination, and kl represents all the second face sample image pairs in the face sample data set.

8. The face recognition method according to claim 2, characterized in that: The overall loss is obtained according to the illumination smoothing loss, the face feature loss of the same person with different illumination cluster centers, the face feature loss of different people with the same illumination cluster centers, and the face identity classification loss, including: The overall loss is obtained by the following formula: L=Ls*Ws+Lid*Wid+Lsame*Wsame+Ldiff*Wdiff Among them, L represents the overall loss, Ls represents the illumination smoothing loss, Lid represents the face identity classification loss, Lsame represents the face feature loss of the same person in different illumination cluster centers, Ldiff represents the face feature loss of different people in the same illumination cluster center, Ws is the weighting coefficient of Ls, Wid is the weighting coefficient of Lid, Wsame is the weighting coefficient of Lsame, and Wdiff is the weighting coefficient of Ldiff.

9. A face recognition device, characterized in that: include: An image acquisition module is configured to acquire a face image; A brightness correction module is configured to input the face image into a brightness processing network model, and obtain a brightness-corrected image after brightness processing through the brightness processing network model; A feature extraction module is configured to input the brightness-corrected image into a feature extraction network model, and obtain feature data associated with the face image through the feature extraction network model; The data comparison module is configured to compare the feature data with pre-stored facial feature data to obtain a facial recognition result.

10. An electronic device, characterized in that: include: processor; a memory for storing executable instructions for the processor; The processor is configured to execute the executable instructions to implement the face recognition method as described in any one of claims 1 to 8.

Images