A face recognition method

Through deep neural networks and attention modules, and combined with softmax and triplet-center loss, the problem of low face recognition rate in natural environments is solved, and more efficient feature extraction and robust recognition is achieved.

CN113989906BActive Publication Date: 2024-11-15JIANGSU UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111418910.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-26
Publication Date
2024-11-15
Estimated Expiration
2041-11-26

AI Technical Summary

Technical Problem

In natural real environments, face images are easily affected by occlusion, and the prior art is difficult to achieve efficient face recognition. Traditional methods such as SRC require image alignment and excessive neural network parameters lead to high computational difficulties and overfitting problems.

Method used

Deep neural network is used for image cropping and alignment, 128-dimensional features are extracted through deep convolutional neural network, attention module is introduced to enhance feature vectors, combined with softmax loss and triplet-center loss loss, using improved facenet network framework and inception- ResNet-v2 model, introducing position and coordinate attention modules to improve the robustness of feature extraction and model training.

Benefits of technology

It improves the accuracy and robustness of face recognition, reduces the complexity of model training and the risk of overfitting, and improves the recognition rate in an occlusion environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113989906B_ABST
    Figure CN113989906B_ABST
Patent Text Reader

Abstract

The present invention discloses a face recognition method, comprising the following steps: firstly, preprocessing the input face image with an improved deep neural network, including face detection and cropping, then constructing a neural network introducing a dual attention mechanism, and training the entire network with a data set to eliminate the influence of various occlusions in reality, and finally performing face recognition on the test data set through the trained system. The present invention adopts a combination of softmax loss and triplet‑center loss as a loss function to improve the accuracy and speed of the training model, thereby improving the recognition rate of the entire system; introducing an attention mechanism in the improved facenet network framework, so that the system can have better robustness and a higher recognition rate while solving the problems of too many parameters, difficult training and overfitting.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to computer vision, and in particular to a face recognition method. Background Art

[0002] As a research hotspot in the field of computer vision and pattern recognition in recent years, face recognition technology has developed very maturely and has a high recognition rate under the constraints of the commercial field.

[0003] However, compared to constrained environments, in natural and real environments, face images are often affected by adverse factors such as various forms of occlusion during collection, resulting in the existing face recognition performance being far from meeting practical requirements.

[0004] At present, the technologies for effectively recognizing occluded faces mainly include the traditional sparse representation classification (SRC) for face recognition, but SRC requires that all face images must be aligned, which makes it difficult to apply to real scenes. In addition, there is also face recognition technology based on neural networks, but traditional neural networks have too many parameters and are prone to overfitting; the network is huge, the calculation is difficult, and it is difficult to implement. In addition, the gradient is diffuse, and it is difficult to optimize the model. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a face recognition method that solves the problem of low face recognition rate under real occlusion.

[0006] Technical solution: A face recognition method described in the present invention comprises the following steps:

[0007] (1) Input a face image and crop and align the image using a deep neural network;

[0008] (2) extracting features from the image: mapping the image to a 128-dimensional feature space through a deep convolutional neural network to obtain a corresponding 128-dimensional feature vector;

[0009] (3) Introduce an attention module to enhance the feature vector: The 128-dimensional feature vector obtained in the previous step is passed through the attention mechanism module to obtain feature vectors with different adaptive weights, and then feature normalization is used to speed up the convergence speed;

[0010] (4) Training the attention neural network model: Input the training set data prepared in advance into the system, then run the MTCNN algorithm to detect the avatar and crop the image to 160×160, then extract features through the neural network, and enhance features through the position attention module and the coordinate attention module, then normalize through the L2 regularized norm, calculate the loss through the loss function, calibrate the parameters, and finally output the training set model;

[0011] (5) Face recognition verification: Input the image to be identified. After the preprocessing steps, the trained model is used to calculate the Euclidean distance between the two to complete the identification of the sample. If the Euclidean distance is less than 1, it is the same face.

[0012] The deep convolutional neural network described in step (1) is the MTCNN project in the facenet network framework; its main operation is: input an image, pass it through P-Net, R-Net, and O-Net in sequence, perform face detection and face image cropping, and output an image size of 160×160.

[0013] The deep neural network described in step (2) is inception-resnet-v2; the steps of using its basic framework for feature extraction are: using a picture of size 299×299×3 as the network input, firstly processing the data with three classic 3×3 convolutions to obtain a feature map with 64 channels, then passing through a pooling layer and a 3×3 convolution layer respectively, outputting two feature maps in parallel, passing the concatenated feature map through two convolution layers of different sizes, and then pooling and convolving the two output feature maps to obtain a feature map with a feature size of 35×35×384; then passing through inception-Re The snet-A module performs 1×1 and 3×3 convolution and pooling operations, and the output feature is 35×35×384. After parallel pooling and convolution by the reduction-A module, the output feature is 17×17×1024. After passing through the inception-Resnet-B module, the output feature is 17×17×1024. After passing through the reduction-B module, the output feature is 8×8×1536. After passing through the inception-Resnet-C, the output feature is 8×8×1536. Finally, the final output is obtained after performing the pooling layer, dropout layer and softmax layer.

[0014] The attention module described in step 3 is specifically: introducing a position attention module after the reduction-A and reduction-B modules; introducing a coordinate attention module after the inception-Resnet-C module.

[0015] The working principle of the position attention module is as follows: Input a feature map A∈R C×H×W , pass it through the convolutional layer to generate three feature maps B, C, D∈R C×H×W , after B and C operations, the attention map S is calculated through the softmax layer ji ;

[0016]

[0017] Where N = H × W, S ji The influence of the i-th position on the j-th position, D is expanded into R C×N The matrix is ​​multiplied by S, then multiplied by the scale parameter, and summed over feature A to get the final output:

[0018] The working principle of the coordinate attention module is: pool the input feature map in the x and y axis directions, then connect the pooling results and obtain nonlinear data through the activation function, and then divide the input results into two groups of feature maps in the horizontal and vertical directions, and then use sigmoid to obtain the attention weight data after the two groups of features are respectively convolved by 1×1. Finally, the input Feature Map data is multiplied by the horizontal and vertical weights to obtain the coordinate attention output feature map, which can be written as the following formula:

[0019]

[0020] where x c (i,j) is the input Feature Map, is the weight in the horizontal direction, is the weight in the vertical direction, y c (i,j) Map of output features.

[0021] The loss function described in step (4) defines a loss function L that combines softmax loss and triplet-center loss, and its calculation formula is:

[0022]

[0023]

[0024] Among them, L s is the softmax loss, L tc is the triplet-center loss, Used to balance the two losses, x i is the test sample y i The category of cyi Yes i The sample center of the class, m is the number of samples, k is the number of classifications, f(x i ) represents the test sample x i Network feature embedding output.

[0025] A computer storage medium stores a computer program, which implements the above-mentioned face recognition method when executed by a processor.

[0026] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned face recognition method when executing the computer program.

[0027] Beneficial effects: Compared with the prior art, the present invention has the following advantages: 1. The combination of softmax loss and triplet-center loss is used as the loss function to improve the accuracy and speed of the training model, thereby improving the recognition rate of the entire system; 2. The attention mechanism is introduced into the improved facenet network framework, so that the system can have better robustness and higher recognition rate while solving the problems of too many parameters, training difficulties and overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a face recognition model diagram under facial occlusion of the present invention;

[0029] Figure 2 This is the structure diagram of the feature extraction neural network inception-ResNet-v2 of the present invention;

[0030] Figure 3 It is the stem structure in the neural network of the present invention;

[0031] Figure 4 It is the position attention module in the present invention;

[0032] Figure 5 It is the coordinate attention module in the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0034] like Figure 1 As shown, a face recognition method comprises the following steps:

[0035] (1) Input a face image and crop and align the image using a deep neural network;

[0036] The MTCNN algorithm of the facenet framework is used to simultaneously complete face detection and positioning as well as face cropping, and the image is cropped to 160×160.

[0037] (2) extracting features from the image: mapping the image to a 128-dimensional feature space through a deep convolutional neural network to obtain a corresponding 128-dimensional feature vector;

[0038] Construct the inception-ResNet-v2 deep convolutional neural network for feature extraction. This network combines ResNet and GoogleNet. Figure 2 As its network structure, it is then connected to a stem structure for preprocessing the data before entering the inception module, and then connected to 3 types of 14 inception-Resnet modules and two reduction modules.

[0039] Using a picture of size 299×299×3 as the network input, first use three classic 3×3 convolutions to process the data to get a feature map with 64 channels, then pass it through a pooling layer and a 3×3 convolution layer respectively, output two feature maps in parallel, pass the concatenated feature map through two convolution layers of different sizes, and then pool and convolve the two output feature maps to get a feature map of 35×35×384. The network structure is shown in the figure below. Figure 3 Then, the inception-Resnet-A module performs 1×1 and 3×3 convolution and pooling operations, and the output feature is 35×35×384. After parallel pooling and convolution by the reduction-A module, the output feature is 17×17×1024. After the inception-Resnet-B module, the output feature is 17×17×1024. After the reduction-B module, the output feature is 8×8×1536. After the inception-Resnet-C module, the output feature is 8×8×1536. Finally, the pooling layer, dropout layer and softmax layer are performed to obtain the final output.

[0040] (3) Introduce an attention module to enhance the feature vector: The 128-dimensional feature vector obtained in the previous step is passed through the attention mechanism module to obtain feature vectors with different adaptive weights, and then feature normalization is used to speed up the convergence speed;

[0041] A position attention module is introduced after the reduction-A and reduction-B modules, and a coordinate attention module is introduced after the inception-Resnet-C module. The structure diagram of the position attention module is as follows Figure 4As shown, input a feature map A∈R C×H×W , pass it through the convolutional layer to generate three feature maps B, C, D∈R C×H×W , where the attention map is calculated through the softmax layer after B and C operations:

[0042]

[0043] Where N = H × W, S ji The influence of the i-th position on the j-th position, D is expanded into R C×N The matrix is ​​multiplied by S, then multiplied by the scale parameter, and summed over feature A to get the final output: E j It includes the relationship between the occlusion of the entire input image and selects the unoccluded area based on the attention focus.

[0044] The coordinate attention module (CA) is as follows Figure 5 As shown in Figure 2, CA encodes the relationship between channel attention and long-range dependencies through two steps: embedding position information and generating coordinate attention.

[0045] 1) Embedding of location information

[0046] Input feature map A∈R C×H×W, The feature map is then pooled in the horizontal and vertical directions to obtain the x- and y-axis related position information of the feature map.

[0047] The obtained one-dimensional feature in the horizontal direction is:

[0048] The obtained one-dimensional feature in the vertical direction is:

[0049] The above two transformations aggregate features along two spatial directions respectively, generating a pair of direction-aware feature maps. These two transformations also allow our attention block to capture long-term dependencies in one spatial direction and preserve precise location information in another spatial direction, which helps the network locate objects of interest more accurately.

[0050] 2) Coordinate Attention Generation

[0051] The horizontal and vertical pooling results are connected together and the attention map is obtained through a 1×1 convolution:

[0052] f=δ(F1([z h ,z w ]))

[0053] F1 is the concatenation operation of the horizontal and vertical pooling results, f∈R C / r×(H+W) After the activation function obtains nonlinear data, the input results are divided into two groups of feature maps in the horizontal and vertical directions:

[0054] g h =σ(F h (f h ))

[0055] g w =σ(F w (f w ))

[0056] After these two sets of features are convolved by 1×1, sigmoid is used to obtain the attention weight data. Finally, the input Feature Map data is multiplied by the horizontal and vertical weights to obtain the coordinate attention output feature map, which can be written as the following formula:

[0057]

[0058] As mentioned above, attention along the horizontal and vertical directions is applied to the input tensor simultaneously. Each element in the two attention maps reflects whether the object of interest exists in the corresponding row and column. This encoding process can make our coordinate attention more accurate in locating the exact location of the object of interest, thus helping the whole model to better recognize.

[0059] (4) Training the attention neural network model: Input the training set data prepared in advance into the system, then run the MTCNN algorithm to detect the avatar and crop the image to 160×160, then extract features through the neural network, and enhance features through the position attention module and the coordinate attention module, then normalize through the L2 regularized norm, calculate the loss through the loss function, calibrate the parameters, and finally output the training set model;

[0060] In this invention, in order to solve the problem that the triplet function is difficult to train and the process is time-consuming, and considering that the distance between classes should be increased while minimizing the distance within the class, we define a loss function L that combines the softmax loss with the triplet-center loss.

[0061]

[0062]

[0063] Among them, L s is the softmax loss, L tcis the triplet-center loss, Used to balance the two losses, x i is the test sample y i The category of c yi Yes i The sample center of the class, m is the number of samples, k is the number of classifications, f(x i ) represents the test sample x i Network feature embedding output. Each time, samples with different occlusion positions and different occlusion areas are selected for training, and the trained classification model is saved.

[0064] (5) Face recognition verification: Input the image to be identified. After the preprocessing steps, the trained model is used to calculate the Euclidean distance between the two to complete the identification of the sample. If the Euclidean distance is less than 1, it is the same face.

[0065] Input the picture you want to identify into the system, then run the MTCNN algorithm to detect the head portrait and crop the picture to 160×160, then perform feature extraction through the improved neural network of the present invention, calculate the Euclidean distance between the pictures, and determine whether it is a person.

Claims

1. A face recognition method, characterized in that: The following steps are involved: (1) Input a face image, and crop and align the image through a deep neural network; the deep neural network is the MTCNN project in the facenet network framework; (2) extracting features from the image: mapping the image to a 128-dimensional feature space through a deep convolutional neural network to obtain a corresponding 128-dimensional feature vector; the deep convolutional neural network is inception-resnet-v2; (3) Introducing an attention module to enhance the feature vector: The 128-dimensional feature vector obtained in the previous step is passed through the attention mechanism module to obtain feature vectors with different adaptive weights, and then feature normalization is performed to accelerate the convergence speed; the attention module is specifically: a position attention module is introduced after the reduction-A and reduction-B modules; a coordinate attention module is introduced after the inception-Resnet-C module; (4) Training the attention neural network model: Input the prepared training set data into the deep neural network, then run the MTCNN algorithm to detect the face and crop the image to 160×160, then extract the features through the neural network, and enhance the features through the position attention module and the coordinate attention module, then normalize it through the L2 regularized norm, and then calculate the loss through the loss function, correct the parameters, and finally output the training set model; the loss function defines a loss function L that combines the softmax loss and the triplet-center loss, and its calculation formula is: Among them, L s is the softmax loss, L tc is the triplet-center loss, Used to balance the two losses, x i is the i-th test sample, y i is the true category label of the i-th test sample, c yi Yes i The sample center of the class, m is the number of samples, k is the number of classifications, f(x i ) represents the test sample x i Network feature embedding output; (5) Face recognition verification: Input the image to be identified. After the preprocessing step, the trained model is used to calculate the Euclidean distance between the images to complete the identification of the sample.

2. A face recognition method according to claim 1, characterized in that: The working principle of the position attention module is as follows: Input a feature map A∈R C×H×W , pass it through the convolutional layer to generate three feature maps B, C, D∈R C×H×W , after B and C operations, the attention map S is calculated through the softmax layer ji ; Where N = H × W, S ji The influence of the i-th position on the j-th position, D is expanded into R C×N Matrix and S ji Multiply them together, then multiply by the scale parameter, and sum feature A to get the final output:

3. A face recognition method according to claim 1, characterized in that: The working principle of the coordinate attention module is as follows: the input feature map is pooled in the x and y axis directions, and then the pooling results are connected and the nonlinear data is obtained through the activation function. Then, the input result is re-divided into two groups of feature maps in the horizontal and vertical directions. After these two groups of features are convolved by 1×1, sigmoid is used to obtain the attention weight data; finally, the input Feature Map data is multiplied by the horizontal and vertical weights to obtain the coordinate attention output feature map, which can be written as the following formula: Among them, x c (i,j) is the input Feature Map, is the weight in the horizontal direction, is the weight in the vertical direction, y c (i,j) Map of output features.

4. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a face recognition method as described in any one of claims 1 to 3 is implemented.

5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, a face recognition method as described in any one of claims 1 to 3 is implemented.