Face recognition enhancement method
By classifying face images and constructing training sample sets for frontal and non-frontal faces, and using convolutional neural networks and generative adversarial networks to process facial features, the problem of face recognition algorithms being affected by environment and angle is solved, thereby improving recognition accuracy and training efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FUJIAN JOYUSING TECHNOLOGY CO LTD
- Filing Date
- 2022-12-26
- Publication Date
- 2026-07-03
AI Technical Summary
Current facial recognition algorithms are greatly affected by lighting conditions, facial angle, expression, and training data, and lack effective processing of input image features, resulting in low recognition accuracy.
By randomly collecting and classifying face images, a training sample set of frontal and non-frontal faces is constructed. A frontal and non-frontal face recognition network is built. Convolutional neural networks and generative adversarial networks are used to extract effective features and generate high-quality frontal face images. The training process is decomposed to improve recognition accuracy.
It effectively filters facial features, improves the accuracy of face recognition, reduces training difficulty and speeds up the network training process, and ensures consistent recognition accuracy for frontal and non-frontal face images.
Smart Images
Figure CN116246316B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of facial recognition technology, specifically to a method for enhancing facial recognition. Background Technology
[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. With the application of deep learning networks, the accuracy of facial recognition algorithms is increasing. However, many environmental factors still affect the accuracy of facial recognition, and training facial recognition systems is also very challenging.
[0003] The accuracy of face recognition algorithms is affected by factors such as lighting conditions, different angles of the face, different expressions, age, and the sheer volume of training data. In the existing technology CN113705358A, a multi-angle profile frontalization method based on feature mapping, the Light CNN face recognition model is first used as a feature extractor to extract deep features from the profile input image and the real frontal image, learning the mapping relationship between profile features and frontal features to obtain model M. Next, a Generative Adversarial Network (GAN) is used as the backbone network, with the profile image as the input image. Model M maps the profile features to frontal features, while the encoder in the GAN network extracts profile image features. The two sets of features are concatenated along the channel dimension as input features for the decoder, ultimately outputting a realistic virtual frontal face image. However, existing technologies lack processing of the features extracted from the input image, making it impossible to determine the effectiveness of the extracted features, which poses a potential risk to the accuracy of the subsequent frontal image construction. Summary of the Invention
[0004] To address the problems existing in the prior art, this invention proposes a face recognition enhancement method.
[0005] The technical solution of the present invention is as follows:
[0006] On the one hand, this invention proposes a method for enhancing face recognition, the specific steps of which include:
[0007] Randomly collect facial images of people with different identities, and filter out facial images whose faces are obscured. Then classify the filtered facial image set into frontal facial images and non-frontal facial images.
[0008] Acquire several sets of frontal face feature images and several non-frontal face feature images. Each set of frontal face feature images includes any two frontal face images. Add person identification labels to the two frontal face images in each set of frontal face feature images to form a frontal face training sample set. Then add person identification labels to the non-frontal face images to form a non-frontal face training sample set.
[0009] A frontal face recognition network is constructed, comprising a face effective feature calculation network and a feature comparison network. The face effective feature calculation network is a convolutional neural network. First, the frontal face feature image is input into the face effective feature calculation network. The weights of the left and right face effective features in the frontal face feature image are obtained through a softmax network. The left and right face effective feature values in the frontal face feature image are obtained through a split method. The left and right face effective feature values and their weights are superimposed to calculate the effective feature value of the frontal face feature image. Then, the effective feature value of the frontal face feature image obtained after processing by the face effective feature calculation network is used as the input of the feature comparison network. The output is the similarity between frontal face feature images. The goal is to minimize the difference between the corresponding feature values of frontal face images with the same identity label. Iterative training is performed, and the optimal frontal face recognition network is output after the iteration ends.
[0010] A non-frontal face recognition network is constructed, which includes a GAN network and a frontal face recognition network. Non-frontal face images are input into the GAN network to generate corresponding high-quality frontal face images. Then, the corresponding high-quality frontal face images are input into the trained optimal frontal face recognition network for identity recognition.
[0011] This study utilizes frontal and non-frontal face recognition networks to identify individuals from both frontal and non-frontal face images.
[0012] As a preferred embodiment, the specific structure of the effective facial feature calculation network is as follows:
[0013] After the feature image is input into the face effective feature calculation network, the input Layer1 is obtained through the convolutional network, and then the output of Layer1 is input into two hidden layers.
[0014] The first hidden layer generates Layer 4 from the output of Layer 1 through a convolutional and fully connected network, and generates Layer 5 from the output of Layer 4 through a softmax network. The output of Layer 5 contains two values, Layer 5[0] and Layer 5[1], which represent the weights of the effective feature values of the left and right faces of the feature image, respectively, and the sum of Layer 5[0] and Layer 5[1] is 1.
[0015] The second hidden layer splits Layer 1 into Layer 2 and Layer 3 using a split operation. The sum of the product of the outputs of Layer 2 and Layer 3 and the output of Layer 5 is Layer 6. The specific formula is as follows:
[0016]
[0017] In the formula, Layer2 is the effective feature value of the left face, layer3 is the effective feature value of the right face, Layer5[0] is the weight of the left face, and Layer5[1] is the weight of the right face;
[0018] Finally, layer6 is processed through convolution and fully connected operations to obtain the effective feature values of the feature image.
[0019] In a preferred embodiment, the effective feature values of the frontal face feature images obtained after processing by the face effective feature calculation network are used as the input of the feature comparison network, and the output is the similarity between frontal face feature images. In the iterative training step, with the goal of minimizing the difference between the corresponding feature values of frontal face images with the same identity label, a loss function is constructed to measure the difference in feature values between two frontal face images. Specifically, the loss function is:
[0020]
[0021] In the formula, FeatureLA is the feature value of the frontal face image of A, and FeatureLB is the feature value of the frontal face image of B; LayerP obtains a 1-dimensional vector output layer through a fully connected dense layer. If FeatureLA and FeatureLB are the same person, the output is 0, otherwise the output is 1.
[0022] In a preferred embodiment, the GAN network includes a Generator network and a Discriminator network. First, a non-frontal face image is input into the Generator network to generate a frontal face image corresponding to the identity. Then, the generated frontal face image corresponding to the identity and a known frontal face image corresponding to the identity are input into the Discriminator network for adversarial learning, so that the Generator network outputs a frontal face image corresponding to the non-frontal face image.
[0023] On the other hand, the present invention proposes a face recognition enhancement system, comprising:
[0024] Image collection and processing module: randomly collects several facial images of people with different identities, filters out facial images whose front faces are obscured, and then classifies the filtered facial image set into frontal facial images and non-frontal facial images;
[0025] Feature image acquisition module: Acquires several sets of frontal face feature images and several non-frontal face feature images. Each set of frontal face feature images includes any two frontal face images. Adds person identification labels to the two frontal face images in each set of frontal face feature images to form a frontal face training sample set. Then, adds person identification labels to the non-frontal face images to form a non-frontal face training sample set.
[0026] Frontal Face Recognition Module: A frontal face recognition network is constructed, which includes a face effective feature calculation network and a feature comparison network. The face effective feature calculation network is a convolutional neural network. First, the frontal face feature image is input into the face effective feature calculation network. The weights of the left and right face effective features in the frontal face feature image are obtained through a softmax network. The left and right face effective feature values in the frontal face feature image are obtained through a split method. The left and right face effective feature values and their weights are superimposed to calculate the effective feature value of the frontal face feature image. Then, the effective feature value of the frontal face feature image obtained after processing by the face effective feature calculation network is used as the input of the feature comparison network. The output is the similarity between frontal face feature images. The goal is to minimize the difference between the corresponding feature values of frontal face images with the same identity label. Iterative training is performed, and the optimal frontal face recognition network is output after the iteration ends.
[0027] Non-frontal face recognition module: Construct a non-frontal face recognition network, which includes a GAN network and a frontal face recognition network. Input non-frontal face images into the GAN network to generate corresponding high-quality frontal face images, and then input the corresponding high-quality frontal face images into the above-trained optimal frontal face recognition network for identity recognition.
[0028] As a preferred embodiment, the specific structure of the effective facial feature calculation network is as follows:
[0029] After the feature image is input into the face effective feature calculation network, the input Layer1 is obtained through the convolutional network, and then the output of Layer1 is input into two hidden layers.
[0030] The first hidden layer generates Layer 4 from the output of Layer 1 through a convolutional and fully connected network, and generates Layer 5 from the output of Layer 4 through a softmax network. The output of Layer 5 contains two values, Layer 5[0] and Layer 5[1], which represent the weights of the effective feature values of the left and right faces of the feature image, respectively, and the sum of Layer 5[0] and Layer 5[1] is 1.
[0031] The second hidden layer splits Layer 1 into Layer 2 and Layer 3 using a split operation. The sum of the product of the outputs of Layer 2 and Layer 3 and the output of Layer 5 is Layer 6. The specific formula is as follows:
[0032]
[0033] In the formula, Layer2 is the effective feature value of the left face, layer3 is the effective feature value of the right face, Layer5[0] is the weight of the left face, and Layer5[1] is the weight of the right face;
[0034] Finally, layer6 is processed through convolution and fully connected operations to obtain the effective feature values of the feature image.
[0035] In a preferred embodiment, the effective feature values of the frontal face feature images obtained after processing by the face effective feature calculation network are used as the input of the feature comparison network, and the output is the similarity between frontal face feature images. In the iterative training step, with the goal of minimizing the difference between the corresponding feature values of frontal face images with the same identity label, a loss function is constructed to measure the difference in feature values between two frontal face images. Specifically, the loss function is:
[0036]
[0037] In the formula, FeatureLA is the feature value of the frontal face image of A, and FeatureLB is the feature value of the frontal face image of B; LayerP obtains a 1-dimensional vector output layer through a fully connected dense layer. If FeatureLA and FeatureLB are the same person, the output is 0, otherwise the output is 1.
[0038] In a preferred embodiment, the GAN network includes a Generator network and a Discriminator network. First, a non-frontal face image is input into the Generator network to generate a frontal face image corresponding to the identity. Then, the generated frontal face image corresponding to the identity and a known frontal face image corresponding to the identity are input into the Discriminator network for adversarial learning, so that the Generator network outputs a frontal face image corresponding to the non-frontal face image.
[0039] On the other hand, the present invention proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a face recognition enhancement method according to any embodiment of the present invention.
[0040] On the other hand, the present invention proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a face recognition enhancement method as described in any embodiment of the present invention.
[0041] The present invention has the following beneficial effects:
[0042] 1. This invention provides a facial effective feature calculation network. After training with a large amount of data, it can filter out effective features from all extracted features and quantify the effective features of the face, which facilitates the calculation of the frontal face recognition network and improves the accuracy of face recognition.
[0043] 2. This invention uses a GAN network to convert non-frontal face images into frontal images while ensuring the consistency of identity information corresponding to the face. It can guarantee the accuracy of recognition regardless of whether the image is frontal or non-frontal.
[0044] 3. This invention decomposes the face recognition network into two parts: a frontal face recognition network and a face recognition network, which are trained separately. This reduces the difficulty of face training and accelerates the network training process, while also improving the accuracy of network recognition. Attached Figure Description
[0045] Figure 1 This is a flowchart of the method of the present invention;
[0046] Figure 2 A network structure diagram for calculating effective facial features;
[0047] Figure 3 This is a diagram of the feature matching network structure. Detailed Implementation
[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0049] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.
[0050] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0051] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.
[0052] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.
[0053] Example 1:
[0054] See Figure 1 A method for enhancing facial recognition, comprising the following steps:
[0055] Several facial images of people with different identities are randomly collected, and facial images with obscured faces are removed. The facial images are then classified into frontal facial images and non-frontal facial images.
[0056] In practice, collecting facial images of people with different identities can yield different facial features, while collecting a large number of facial images of people with the same identity can yield similar facial features.
[0057] Acquire several sets of frontal face feature images and several non-frontal face feature images. Each set of frontal face feature images includes any two frontal face images. Add person identification labels to the two frontal face images in each set of frontal face feature images to form a frontal face training sample set. Then add person identification labels to the non-frontal face images to form a non-frontal face training sample set.
[0058] A frontal face recognition network is constructed, comprising a face effective feature calculation network and a feature comparison network. The face effective feature calculation network is a convolutional neural network. First, the frontal face feature image is input into the face effective feature calculation network. The weights of the left and right face effective features in the frontal face feature image are obtained through a softmax network. The left and right face effective feature values in the frontal face feature image are obtained through a split method. The left and right face effective feature values and their weights are superimposed to calculate the effective feature value of the frontal face feature image. Then, the effective feature value of the frontal face feature image obtained after processing by the face effective feature calculation network is used as the input of the feature comparison network. The output is the similarity between frontal face feature images. The goal is to minimize the difference between the corresponding feature values of frontal face images with the same identity label. Iterative training is performed, and the optimal frontal face recognition network is output after the iteration ends.
[0059] A non-frontal face recognition network is constructed, which includes a GAN network and a frontal face recognition network. Non-frontal face images are input into the GAN network to generate corresponding high-quality frontal face images. Then, the corresponding high-quality frontal face images are input into the trained optimal frontal face recognition network for identity recognition.
[0060] The optimal facial recognition network model is used to identify any person in an image.
[0061] As a preferred embodiment of this example, see Figure 2 The specific structure of the facial effective feature calculation network is as follows:
[0062] After the feature image is input into the face feature calculation network, the input Layer 1 is obtained through the convolutional network, and then the output of Layer 1 is input into two hidden layers.
[0063] The first hidden layer generates Layer 4 from the output of Layer 1 through a convolutional and fully connected network, and generates Layer 5 from the output of Layer 4 through a softmax network. The output of Layer 5 contains two values, Layer 5[0] and Layer 5[1], which represent the weights of the effective feature values of the left and right faces of the feature image, respectively, and the sum of Layer 5[0] and Layer 5[1] is 1.
[0064] The second hidden layer splits Layer 1 into Layer 2 and Layer 3 using a split operation. The sum of the product of the outputs of Layer 2 and Layer 3 and the output of Layer 5 is Layer 6. The specific formula is as follows:
[0065]
[0066] In the formula, Layer2 is the effective feature value of the left face, layer3 is the effective feature value of the right face, Layer5[0] is the weight of the left face, and Layer5[1] is the weight of the right face;
[0067] Finally, layer 6 is processed through convolution, full connection, and other operations to obtain the effective feature values of the feature image.
[0068] In practice, the actual input feature image size is 80*80 pixels. It is input into the Input and passes through the convolutional network to obtain Layer 1 with a size of 20*20. At this time, the output of Layer 1 is input into two hidden layers.
[0069] The first layer generates Layer4 with a size of 2 through convolution and fully connected networks. The output of Layer4 is then used to generate Layer5 with a size of 2 through a softmax network. At this point, the sum of the two output values Layer5[0] and Layer5[1] of Layer5 is 1.
[0070] The second layer splits Layer1 into two layers, Layer2 and Layer3, with a size of 10*20. The sum of the outputs of Layer2 and Layer3 and the output of Layer5 is Layer6 with a size of 10*20. Layer6 = Layer2 * Layer5[0] + Layer3 * Layer5[1].
[0071] Then, layer 6 is processed through convolution, fully connected layers, and other operations to generate the feature layer FeatureL of the face, which has a size of 128.
[0072] In a preferred embodiment of this invention, the effective feature values of the frontal face feature images obtained after processing by the face effective feature calculation network are used as the input of the feature comparison network, and the output is the similarity between frontal face feature images. In the iterative training step, with the goal of minimizing the difference between the corresponding feature values of frontal face images with the same identity label, a loss function is constructed to measure the difference in feature values between two frontal face images. Specifically, the loss function is:
[0073]
[0074] In the formula, FeatureLA is the feature value of the frontal face image of A, and FeatureLB is the feature value of the frontal face image of B; LayerP obtains a 1-dimensional vector output layer through a fully connected dense layer. If FeatureLA and FeatureLB are the same person, the output is 0, otherwise the output is 1.
[0075] In practice, two frontal face images, FaceImageA and FaceImageB, are input into the network to obtain the features of the two faces, namely FeatureLA and FeatureLB. The absolute value of the difference between FeatureLA and FeatureLB is used to obtain LayerP = abs(FeatureLA - FeatureLB). LayerP is passed through a fully connected dense layer to obtain the output layer, which is a 1-dimensional vector. If FaceImageA and FaceImageB are the same person, the output is 0; if they are different people, the output is 1.
[0076] The network structure diagram of the feature matching network is as follows: Figure 3 As shown.
[0077] In a preferred embodiment of this example, the GAN network includes a Generator network and a Discriminator network. First, a non-frontal face image is input into the Generator network to generate a frontal face image corresponding to the identity. Then, the generated frontal face image corresponding to the identity and a known frontal face image corresponding to the identity are input into the Discriminator network for adversarial learning, so that the Generator network outputs a frontal face image corresponding to the non-frontal face image.
[0078] In practice, a non-frontal face image is input into the input field. A Generator feature output layer, LayerW, with a size of 256 is generated through a convolutional and fully connected network. LayerW then passes through the Generator network and outputs Goutput, with a size of 80*80.
[0079] Example 2:
[0080] A facial recognition enhancement system includes:
[0081] Image collection and processing module: randomly collects several facial images of people with different identities, filters out facial images whose front faces are obscured, and then classifies the filtered facial image set into frontal facial images and non-frontal facial images;
[0082] Feature image acquisition module: Acquires several sets of frontal face feature images and several non-frontal face feature images. Each set of frontal face feature images includes any two frontal face images. Adds person identification labels to the two frontal face images in each set of frontal face feature images to form a frontal face training sample set. Then, adds person identification labels to the non-frontal face images to form a non-frontal face training sample set.
[0083] Frontal Face Recognition Module: A frontal face recognition network is constructed, which includes a face effective feature calculation network and a feature comparison network. The face effective feature calculation network is a convolutional neural network. First, the frontal face feature image is input into the face effective feature calculation network. The weights of the left and right face effective features in the frontal face feature image are obtained through a softmax network. The left and right face effective feature values in the frontal face feature image are obtained through a split method. The left and right face effective feature values and their weights are superimposed to calculate the effective feature value of the frontal face feature image. Then, the effective feature value of the frontal face feature image obtained after processing by the face effective feature calculation network is used as the input of the feature comparison network. The output is the similarity between frontal face feature images. The goal is to minimize the difference between the corresponding feature values of frontal face images with the same identity label. Iterative training is performed, and the optimal frontal face recognition network is output after the iteration ends.
[0084] Non-frontal face recognition module: Construct a non-frontal face recognition network, which includes a GAN network and a frontal face recognition network. Input non-frontal face images into the GAN network to generate corresponding high-quality frontal face images, and then input the corresponding high-quality frontal face images into the above-trained optimal frontal face recognition network for identity recognition.
[0085] In a preferred embodiment of this invention, the specific structure of the effective facial feature calculation network is as follows:
[0086] After the feature image is input into the face effective feature calculation network, the input Layer1 is obtained through the convolutional network, and then the output of Layer1 is input into two hidden layers.
[0087] The first hidden layer generates Layer 4 from the output of Layer 1 through a convolutional and fully connected network, and generates Layer 5 from the output of Layer 4 through a softmax network. The output of Layer 5 contains two values, Layer 5[0] and Layer 5[1], which represent the weights of the effective feature values of the left and right faces of the feature image, respectively, and the sum of Layer 5[0] and Layer 5[1] is 1.
[0088] The second hidden layer splits Layer 1 into Layer 2 and Layer 3 using a split operation. The sum of the product of the outputs of Layer 2 and Layer 3 and the output of Layer 5 is Layer 6. The specific formula is as follows:
[0089]
[0090] In the formula, Layer2 is the effective feature value of the left face, layer3 is the effective feature value of the right face, Layer5[0] is the weight of the left face, and Layer5[1] is the weight of the right face;
[0091] Finally, layer6 is processed through convolution and fully connected operations to obtain the effective feature values of the feature image.
[0092] In a preferred embodiment of this invention, the effective feature values of the frontal face feature images obtained after processing by the face effective feature calculation network are used as the input of the feature comparison network, and the output is the similarity between frontal face feature images. In the iterative training step, with the goal of minimizing the difference between the corresponding feature values of frontal face images with the same identity label, a loss function is constructed to measure the difference in feature values between two frontal face images. Specifically, the loss function is:
[0093]
[0094] In the formula, FeatureLA is the feature value of the frontal face image of A, and FeatureLB is the feature value of the frontal face image of B; LayerP obtains a 1-dimensional vector output layer through a fully connected dense layer. If FeatureLA and FeatureLB are the same person, the output is 0, otherwise the output is 1.
[0095] In a preferred embodiment of this example, the GAN network includes a Generator network and a Discriminator network. First, a non-frontal face image is input into the Generator network to generate a frontal face image corresponding to the identity. Then, the generated frontal face image corresponding to the identity and a known frontal face image corresponding to the identity are input into the Discriminator network for adversarial learning, so that the Generator network outputs a frontal face image corresponding to the non-frontal face image.
[0096] Example 3:
[0097] This embodiment of an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a face recognition enhancement method according to any embodiment of the present invention.
[0098] Example 4:
[0099] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a face recognition enhancement method according to any embodiment of the present invention.
[0100] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.
Claims
1. A face recognition enhancement method, characterized by, The specific steps include: Randomly collect facial images of people with different identities, and filter out facial images whose faces are obscured. Then classify the filtered facial image set into frontal facial images and non-frontal facial images. Acquire several sets of frontal face feature images and several non-frontal face feature images. Each set of frontal face feature images includes any two frontal face images. Add person identification labels to the two frontal face images in each set of frontal face feature images to form a frontal face training sample set. Then add person identification labels to the non-frontal face images to form a non-frontal face training sample set. A frontal face recognition network is constructed, comprising a face effective feature calculation network and a feature comparison network. The face effective feature calculation network is a convolutional neural network. First, the frontal face feature image is input into the face effective feature calculation network. The weights of the left and right face effective features in the frontal face feature image are obtained through a softmax network. The left and right face effective feature values in the frontal face feature image are obtained through a split method. The left and right face effective feature values and their weights are superimposed to calculate the effective feature value of the frontal face feature image. Then, the effective feature value of the frontal face feature image obtained after processing by the face effective feature calculation network is used as the input of the feature comparison network. The output is the similarity between frontal face feature images. The goal is to minimize the difference between the corresponding feature values of frontal face images with the same identity label. Iterative training is performed, and the optimal frontal face recognition network is output after the iteration ends. A non-frontal face recognition network is constructed, which includes a GAN network and a frontal face recognition network. Non-frontal face images are input into the GAN network to generate corresponding high-quality frontal face images. Then, the corresponding high-quality frontal face images are input into the trained optimal frontal face recognition network for identity recognition. This study utilizes frontal and non-frontal face recognition networks to identify individuals from both frontal and non-frontal face images.
2. The face recognition enhancement method according to claim 1, characterized in that, The specific structure of the effective facial feature calculation network is as follows: After the feature image is input into the face effective feature calculation network, the input Layer1 is obtained through the convolutional network, and then the output of Layer1 is input into two hidden layers. The first hidden layer generates Layer 4 from the output of Layer 1 through a convolutional and fully connected network, and generates Layer 5 from the output of Layer 4 through a softmax network. The output of Layer 5 contains two values, Layer 5[0] and Layer 5[1], which represent the weights of the effective feature values of the left and right faces of the feature image, respectively, and the sum of Layer 5[0] and Layer 5[1] is 1. The second hidden layer splits Layer 1 into Layer 2 and Layer 3 using a split operation. The sum of the product of the outputs of Layer 2 and Layer 3 and the output of Layer 5 is Layer 6. The specific formula is as follows: In the formula, Layer2 is the effective feature value of the left face, layer3 is the effective feature value of the right face, Layer5[0] is the weight of the left face, and Layer5[1] is the weight of the right face; Finally, layer 6 is processed through convolution and fully connected operations to obtain the effective feature values of the feature image.
3. The face recognition enhancement method according to claim 2, characterized in that, In the iterative training step, which uses the effective feature values of the frontal face feature images obtained after processing by the face effective feature calculation network as the input of the feature comparison network and the output as the similarity between frontal face feature images, the goal is to minimize the difference between the corresponding feature values of frontal face images with the same identity label. A loss function is constructed to measure the difference in feature values between two frontal face images. Specifically, the loss function is: In the formula, FeatureLA is the feature value of the frontal face image of A, and FeatureLB is the feature value of the frontal face image of B; LayerP obtains a 1-dimensional vector output layer through a fully connected dense layer. If FeatureLA and FeatureLB are the same person, the output is 0, otherwise the output is 1.
4. The face recognition enhancement method according to claim 1, characterized in that, The GAN network includes a Generator network and a Discriminator network. First, a non-frontal face image is input into the Generator network to generate a frontal face image corresponding to the identity. Then, the generated frontal face image corresponding to the identity and a known frontal face image corresponding to the identity are input into the Discriminator network for adversarial learning, so that the Generator network outputs the frontal face image corresponding to the non-frontal face image.
5. A facial recognition enhancement system, characterized in that, include: Image collection and processing module: randomly collects several facial images of people with different identities, filters out facial images whose front faces are obscured, and then classifies the filtered facial image set into frontal facial images and non-frontal facial images; Feature image acquisition module: Acquires several sets of frontal face feature images and several non-frontal face feature images. Each set of frontal face feature images includes any two frontal face images. Adds person identification labels to the two frontal face images in each set of frontal face feature images to form a frontal face training sample set. Then, adds person identification labels to the non-frontal face images to form a non-frontal face training sample set. Frontal Face Recognition Module: A frontal face recognition network is constructed, which includes a face effective feature calculation network and a feature comparison network. The face effective feature calculation network is a convolutional neural network. First, the frontal face feature image is input into the face effective feature calculation network. The weights of the left and right face effective features in the frontal face feature image are obtained through a softmax network. The left and right face effective feature values in the frontal face feature image are obtained through a split method. The left and right face effective feature values and their weights are superimposed to calculate the effective feature value of the frontal face feature image. Then, the effective feature value of the frontal face feature image obtained after processing by the face effective feature calculation network is used as the input of the feature comparison network. The output is the similarity between frontal face feature images. The goal is to minimize the difference between the corresponding feature values of frontal face images with the same identity label. Iterative training is performed, and the optimal frontal face recognition network is output after the iteration ends. Non-frontal face recognition module: Construct a non-frontal face recognition network, which includes a GAN network and a frontal face recognition network. Input non-frontal face images into the GAN network to generate corresponding high-quality frontal face images, and then input the corresponding high-quality frontal face images into the above-trained optimal frontal face recognition network for identity recognition.
6. The face recognition enhancement system according to claim 5, characterized in that, The specific structure of the effective facial feature calculation network is as follows: After the feature image is input into the face effective feature calculation network, the input Layer1 is obtained through the convolutional network, and then the output of Layer1 is input into two hidden layers. The first hidden layer generates Layer 4 from the output of Layer 1 through a convolutional and fully connected network, and generates Layer 5 from the output of Layer 4 through a softmax network. The output of Layer 5 contains two values, Layer 5[0] and Layer 5[1], which represent the weights of the effective feature values of the left and right faces of the feature image, respectively, and the sum of Layer 5[0] and Layer 5[1] is 1. The second hidden layer splits Layer 1 into Layer 2 and Layer 3 using a split operation. The sum of the product of the outputs of Layer 2 and Layer 3 and the output of Layer 5 is Layer 6. The specific formula is as follows: In the formula, Layer2 is the effective feature value of the left face, layer3 is the effective feature value of the right face, Layer5[0] is the weight of the left face, and Layer5[1] is the weight of the right face; Finally, layer 6 is processed through convolution and fully connected operations to obtain the effective feature values of the feature image.
7. A face recognition enhancement system according to claim 6, characterized in that, In the iterative training step, which uses the effective feature values of the frontal face feature images obtained after processing by the face effective feature calculation network as the input of the feature comparison network and the output as the similarity between frontal face feature images, the goal is to minimize the difference between the corresponding feature values of frontal face images with the same identity label. A loss function is constructed to measure the difference in feature values between two frontal face images. Specifically, the loss function is: In the formula, FeatureLA is the feature value of the frontal face image of A, and FeatureLB is the feature value of the frontal face image of B; LayerP obtains a 1-dimensional vector output layer through a fully connected dense layer. If FeatureLA and FeatureLB are the same person, the output is 0, otherwise the output is 1.
8. A face recognition enhancement system according to claim 5, characterized in that, The GAN network includes a Generator network and a Discriminator network. First, a non-frontal face image is input into the Generator network to generate a frontal face image corresponding to the identity. Then, the generated frontal face image corresponding to the identity and a known frontal face image corresponding to the identity are input into the Discriminator network for adversarial learning, so that the Generator network outputs the frontal face image corresponding to the non-frontal face image.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a face recognition enhancement method as described in any one of claims 1 to 4.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the face recognition enhancement method as described in any one of claims 1 to 4.