Method for improving face recognition performance by using polynomial operator to assist neural network training
Through the combination of parallel architecture learning network and polynomial operator module, the FaceNet model with fixed parameters is used to solve the problems of neural network model complexity and computing resource consumption in face recognition technology, and efficient feature extraction and recognition performance is achieved.
Patent Information
- Application Number
- CN202411182265.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the existing facial recognition technology, the high complexity of neural network models and computing resource consumption problems exist, especially in terms of training time and computing resource requirements.
Using polynomial operator-assisted neural network training method, the combination of learning network and polynomial operator modules is used to use the FaceNet model with fixed parameters to reduce the computing resource requirements during the training process and improve the stability and accuracy of feature extraction. .
It significantly reduces the computing resource requirements during the training process, improves the model's adaptability and recognition accuracy, and maintains efficient recognition performance especially when processing complex data.
Smart Images

Figure CN119068531A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision and artificial intelligence technology, and in particular to a method for improving face recognition performance by using polynomial operators to assist neural network training. Background Art
[0002] In face recognition technology, neural network models are widely used in image processing and pattern recognition tasks. Existing face recognition technology mainly relies on deep convolutional neural networks (CNN) and recurrent neural networks (RNN), such as classic architectures such as ResNet and VGG. These networks extract different levels of features from the input image through multiple layers of convolution, pooling and activation functions, and classify them through fully connected layers. Although these classic networks have achieved remarkable results in feature extraction and classification accuracy, as the network depth increases, the number of model parameters also increases, resulting in a significant increase in training time and computing resource requirements.
[0003] In order to improve training efficiency and reduce computing resource consumption, researchers have proposed a variety of optimization methods. For example, pruning technology reduces the complexity and computational complexity of the model by removing redundant neurons and connections in the neural network, but excessive pruning may lead to degraded model performance; quantization technology converts floating-point parameters in the neural network into low-precision integer parameters to reduce the storage and computational overhead of the model, but this method may affect the accuracy of the model in some cases; knowledge distillation significantly reduces the number of model parameters and computational requirements by training smaller student models to learn the behavior of large teacher models, but the student model may not be able to fully match the performance of the teacher model, especially when dealing with complex tasks, there may be inference delays and performance degradation.
[0004] Although these methods have alleviated the training time and resource consumption problems of neural networks to a certain extent, they mainly focus on the structural optimization of the neural network itself, while ignoring the possibility of improving model performance through external auxiliary methods. In recent years, fine-tuning technology has gradually attracted attention as an effective optimization method. Fine-tuning adjusts a small number of parameters based on the pre-trained model so that the model can quickly adapt to new tasks and improve accuracy. Compared with fully updating all parameters, fine-tuning significantly reduces the demand for computing resources. Summary of the invention
[0005] The present invention aims to solve the problems of high model complexity and parameter update computing resource consumption of neural networks in face recognition tasks, and provides a method for improving face recognition performance by using polynomial operators to assist neural network training.
[0006] To solve the above problems, the present invention is achieved through the following technical solutions:
[0007] The method for improving face recognition performance by using polynomial operators to assist neural network training includes the following steps:
[0008] Step 1: Obtain a sample image data set, and use a face detection algorithm to detect face regions in each sample image of the sample image data set, and then pre-process these face regions to obtain a sample face region image data set;
[0009] Step 2: Select anchor samples, positive samples, and negative samples from the sample face area image dataset to construct sample triples to obtain a sample triplet dataset; wherein the anchor sample is a sample face area image of a certain person, the positive sample is another sample face area image of a certain person, and the negative sample is a sample face area image of another person;
[0010] Step 3, construct a parallel architecture learning network; the parallel architecture learning network includes a first learning branch, a second learning branch and a fusion layer; the first learning branch is composed of a convolutional layer and a fully connected layer in series, and the output of the convolutional layer is connected to the input of the fully connected layer; the second learning branch is a face recognition network that has been trained and has fixed parameters; the input of the convolutional layer and the input of the face recognition network together form the input of the parallel architecture learning network, the output of the fully connected layer and the output of the face recognition network are connected to the input of the fusion layer, and the output of the fusion layer forms the output of the parallel architecture learning network;
[0011] Step 4: Use the sample triplet data set to train the parallel architecture learning network to obtain a parallel architecture learning model;
[0012] Step 5: Obtain a dataset of identifiable person images, and use a face detection algorithm to detect face regions in each identifiable person image in the dataset, and then pre-process these face regions to obtain a dataset of identifiable person face region images;
[0013] Step 6: Send the identifiable face region image data set to the parallel architecture learning model to extract the first feature vector, the third feature vector and the final feature vector of each identifiable face region image;
[0014] Step 7: Based on the final feature vectors of the face region images of each identifiable person extracted in step 6, the average value of all the final feature vectors of the same identifiable person is used as the control feature vector of the identifiable person;
[0015] Step 8: Based on the first feature vector and the third feature vector of each identifiable person's face region image extracted in step 6 and the identifiable person's comparison feature vector obtained in step 7, a polynomial operator module is constructed; the feature vector coefficient w of the polynomial operator module is:
[0016]
[0017] Where: U1 is the row concatenation vector of the first eigenvector, U1 = [u 11 ,u 12 ,…,u 1K ] T ,u 11 ,u 12 ,…,u 1K They represent the first feature vectors extracted from the 1st, 2nd, …, Kth identifiable face area images; U3 is the row concatenation vector of the third feature vector, U3=[u 31 ,u 32 ,…,u 3K ] T ,u 31 ,u 32 ,…,u 3K They represent the third feature vectors extracted from the 1st, 2nd, …, Kth identifiable face region images respectively; Z is the row concatenation vector of the reference feature vector, Z = [z1, z2, …, z K ] T ,z1,z2,…,z K denote the comparison feature vectors of identifiable persons corresponding to the 1st, 2nd, ..., Kth identifiable person face region images respectively; K is the number of identifiable person face region images in the identifiable person face region image set;
[0018] Step 9: Replace the fully connected layer in the parallel architecture learning model with a polynomial operator module to obtain a face recognition model;
[0019] Step 10: Obtain an image of a person to be identified, and use a face detection algorithm to detect a face region in the image of the person to be identified, and then pre-process the face region to obtain an image of the face region of the person to be identified;
[0020] Step 11: Send the facial region image of the person to be identified into the face recognition model, extract the final feature vector of the person to be identified, calculate the similarity between the extracted final feature vector of the person to be identified and the reference feature vectors of each stored identifiable person, and give the recognition result.
[0021] In the above step 1, MTCNN is used as the face detection algorithm.
[0022] The preprocessing process of the above step 1 is as follows: first, the face is rotated and scaled to a standard posture using the detected facial key points of the face area to obtain an aligned image; then, the face area is cropped out based on the aligned image; then, the pixel values of the cropped face area are normalized to obtain a normalized image; finally, the size of the face area image is adjusted to meet the input size requirements of the deep learning model to obtain a sample face area image.
[0023] In the above step 3, the face recognition network in the parallel architecture learning network is FaceNet.
[0024] The learning process of step 4 above is as follows: the first feature vector extracted by the first learning branch and the second feature vector extracted by the second learning branch are merged to obtain the final feature vector, and the triplet loss is calculated based on the final feature vector. While keeping the parameters of the second learning branch unchanged, the parameters of the first learning branch are updated through back propagation to minimize the triplet loss and obtain a parallel architecture learning model.
[0025] In step 4 above, the triple loss L(A,P,N) is:
[0026]
[0027] Where ||·||2 represents the Euclidean distance, f(A) is the extracted feature vector of the anchor sample in the final feature vector, f(P) is the extracted feature vector of the positive sample in the final feature vector, f(N) is the extracted feature vector of the negative sample in the final feature vector, α is a hyperparameter, i=1,2,...,M, and M is the number of sample triplets.
[0028] In the above step 6, the first feature vector is the feature vector output by the convolution layer of the first learning branch of the parallel architecture learning model; the third feature vector is the feature vector output by the second learning branch of the parallel architecture learning model; and the final feature vector is the feature vector output by the fusion layer of the parallel architecture learning model.
[0029] In the above step 9, the face recognition model includes a first learning branch, a second learning branch and a fusion layer; the first learning branch is composed of a convolutional layer and a polynomial operator module in series, and the output of the convolutional layer is connected to the input of the polynomial operator module; the second learning branch is a face recognition network; the input of the convolutional layer and the input of the face recognition network together form the input of the face recognition model, the output of the polynomial operator module and the output of the face recognition network are connected to the input of the fusion layer, and the output of the fusion layer forms the output of the face recognition model.
[0030] In the above step 11, the similarity between the final feature vector of the person to be identified and the reference feature vectors of each identifiable person is calculated using the Euclidean distance calculation.
[0031] Compared with the prior art, the present invention has the following characteristics:
[0032] 1. By utilizing the efficient feature extraction capabilities of the FaceNet model with fixed parameters, the stability and accuracy of feature extraction are significantly improved, while reducing the problem of gradient disappearance during training.
[0033] 2. It is proposed that based on the FaceNet structure, the feature extraction of the convolutional layer and the fully connected layer is combined for training, which significantly reduces the computing resource requirements during training while maintaining efficient model performance.
[0034] 3. By combining the feature optimization of FaceNet and the polynomial operator module, it avoids the need to train again through back propagation when a new task arrives, effectively improving the adaptability of the face recognition model to new tasks and maintaining a high recognition accuracy when processing complex data. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 This is a schematic diagram of the face recognition system;
[0036] Figure 2 Schematic diagram of the principle of learning network for parallel architecture;
[0037] Figure 3 Schematic diagram of the face recognition model. DETAILED DESCRIPTION
[0038] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with specific examples and with reference to the accompanying drawings.
[0039] The overall design of the face recognition system mainly includes face detection module, face preprocessing module, face representation module and face recognition module. Its structure is as follows: Figure 1 As shown in the figure. The face detection module uses a residual neural network to detect the face area in the input image or video frame, and gives the location and confidence of the face. The face preprocessing module aligns, crops, normalizes and other operations on the detected faces for subsequent feature extraction and recognition. The face representation module uses a deep learning framework to build and train a face recognition model, extracts discriminative feature vectors from the preprocessed face image data, adjusts the model configuration, and reports the trained model. The face recognition module is used to calculate the similarity between facial features and give recognition results.
[0040] The present invention proposes a method for improving face recognition performance by using polynomial operators to assist neural network training, comprising the following steps:
[0041] Phase 1: Use the sample image dataset to train the parallel architecture learning network to obtain a parallel architecture learning model.
[0042] Step 1: Obtain a sample image data set, and use a face detection algorithm to detect the face regions in each sample image of the sample image data set, and then preprocess these face regions to obtain a sample face region image data set.
[0043] The sample image dataset is used to train the parallel architecture learning network. The sample image dataset contains the largest number of people, which can be obtained from existing portrait datasets or intercepted security camera video streams. There are more than one sample image for each person in the sample image dataset, including images of the person from different angles.
[0044] Face detection is a computer technology for identifying faces in digital images. It is a key and basic step in various face recognition methods. Whether the face can be accurately detected from a given image through an algorithm, whether it exists, and whether the extracted area meets the standards for subsequent processing will directly affect the recognition rate of the system. Common face detection algorithms include Haar features, HOG features, MTCNN, etc. The present invention selects MTCNN (Multi-Task Cascaded Convolutional Networks) as a face detection algorithm to detect the face area in the sample image data set and obtain the position and confidence of the face. MTCNN is a multi-task cascaded convolutional neural network based on deep learning, which is specifically used for face area and key point detection. MTCNN divides the face detection task into three stages, which are implemented using three cascaded convolutional neural networks (P-Net, R-Net and O-Net). First, P-Net (Proposal Network) quickly generates candidate face regions and performs preliminary screening; then, R-Net (Refine Network) further screens and accurately locates these candidate regions; finally, O-Net (Output Network) generates the final detection results based on precise positioning and simultaneously detects five key points (two eyes, nose, and two corners of the mouth). This multi-task learning framework not only improves the accuracy and robustness of detection, but also provides key point information at the same time, which is helpful for subsequent face alignment and recognition operations. MTCNN has high detection speed and accuracy, and is suitable for face detection tasks under various complex backgrounds and lighting conditions. After the input sample image I passes through MTCNN, the detected face region F and facial key points K are obtained:
[0045] F,K=MTCNN(I)
[0046] The detected face area is subjected to preprocessing operations such as cropping, alignment and normalization to obtain the final preprocessed image (sample face area image) F resized First, use the detected facial features K to rotate and scale the face to a standard pose to obtain the aligned image F aligned Then, after the alignment, the image F aligned Based on the above, we cut out the face area F cropped , remove the redundant background. Then, the cropped face area Fcropped The pixel values are normalized to [-1,1] to obtain the normalized image F normalized , in order to improve the accuracy of model training. Finally, adjust the size of the face area image (such as 96×96 pixels) to meet the input size requirements of the deep learning model, and obtain the final preprocessed image F resized .
[0047] Step 2: Select anchor samples, positive samples, and negative samples from the sample face area image dataset to construct sample triplets and obtain a sample triplet dataset.
[0048] Select anchor samples, positive samples and negative samples from the sample face area image dataset to construct sample triples {(A i ,P i ,N i )}; where i = 1, 2, ..., M, M is the number of sample triplets; A i is the i-th anchor sample, which is a sample face region image of a person (the current person) selected from the sample face region image dataset; P i is the i-th positive sample, which is another sample face area image of a person (the current person) selected from the sample face area image dataset; N i is the i-th negative sample, which is a sample face region image of another person (other than the current person) selected from the sample face region image dataset. The M sample triplets are formed into a sample triplet dataset.
[0049] Step 3: Build a parallel architecture learning network.
[0050] See also Figure 2 The parallel architecture learning network includes a first learning branch, a second learning branch and a fusion layer. The first learning branch is composed of a convolutional layer and a fully connected layer in series, and the output of the convolutional layer is connected to the input of the fully connected layer. The second learning branch is a face recognition network that has been trained and has fixed parameters. The input of the convolutional layer and the input of the face recognition network together form the input of the parallel architecture learning network, the output of the fully connected layer and the output of the face recognition network are connected to the input of the fusion layer, and the output of the fusion layer forms the output of the parallel architecture learning network.
[0051] The present invention uses the FaceNet network as a face recognition network. FaceNet is an advanced deep learning model specifically used for face recognition and verification. It can effectively capture the detailed feature vectors of the face and has strong robustness to various lighting changes, posture changes and expression changes. The network structure of the convolution layer and the fully connected layer is connected in parallel with the trained and fixed parameter face recognition network (FaceNet) network, thereby parallelizing the architecture learning network. In this parallel architecture learning network: the second learning branch (FaceNet) maps the face image to a high-dimensional feature space through a deep convolutional neural network (DCNN), in which similar faces have a smaller Euclidean distance, while different faces have a larger Euclidean distance; the first learning branch (convolution and fully connected parts) further enhances the expression ability of the FaceNet network by adding a small number of parameters, and at the same time provides a network architecture foundation for the subsequent polynomial operator module; the fusion layer fuses the feature vector parameters extracted by the two to obtain the final feature vector z as follows:
[0052] z=u2+u3
[0053] in, represents the feature vector extracted by the first learning branch in the parallel architecture, Represents the feature vector extracted by the second learning branch.
[0054] The present invention uses a FaceNet network that has been trained and has solidified parameters, and solidifies the network parameters of the FaceNet network during the learning process without updating, which not only retains the powerful feature vector extraction ability of the FaceNet network, but also reduces the complexity and time of training. On this basis, the present invention adjusts a small number of parameters in combination with an external module (the first learning branch) on the basis of a pre-trained model (a FaceNet network that has been trained and has solidified parameters of the second learning branch), so that the model can quickly adapt to new tasks and improve accuracy. Compared with comprehensively updating all parameters, the fine-tuning strategy combined with the external module reduces the amount of parameters that need to be updated by only adjusting a small number of task-related external modules, significantly reducing the demand for computing resources, thereby improving the operating efficiency of the model. While maintaining or even improving the accuracy of the model, the computing cost and storage requirements are greatly reduced, achieving faster training speed and lower computing overhead, and enhancing the adaptability and recognition accuracy of the model on different tasks through simple training, so that the model can adapt to new data and tasks more efficiently, and is particularly suitable for application scenarios with limited resources.
[0055] It is of great significance to use a trained model and fine-tune it for face recognition, which can significantly save time and computing resources, because the pre-trained model has learned common features on a large-scale data set, and fine-tuning only requires a small amount of training on specific data to complete the optimization. This method can improve the performance of the model, especially in the case of insufficient data, by fine-tuning to adapt to the characteristics of specific tasks, thereby improving recognition accuracy.
[0056] Step 4: Use the sample triplet data set to train the parallel architecture learning network to obtain a parallel architecture learning model.
[0057] During the training process of the parallel architecture learning network: the first feature vector extracted by the first learning branch and the second feature vector extracted by the second learning branch are fused to obtain the final feature vector, and the triplet loss is calculated based on the final feature vector; while keeping the network parameters of the FaceNet network in the second learning branch unchanged, the network parameters of the convolutional layer and the fully connected layer in the first learning branch are updated through back propagation to minimize the triplet loss function, optimize the performance of the parallel architecture learning network, and obtain the parallel architecture learning model.
[0058] The triplet loss function minimizes the distance between the anchor sample and the positive sample and maximizes the distance between the anchor sample and the negative sample, making similar face feature vectors closer and different face feature vectors farther away, thereby improving the accuracy and robustness of recognition. The triplet loss L(A,P,N) is:
[0059]
[0060] Among them, ||·||2 represents the Euclidean distance, f(A) is the extracted feature vector of the anchor sample in the final feature vector (the anchor sample vector A=[A1,A2,…,A M ] T The feature vector extracted by the parallel architecture learning network is sent to the parallel architecture learning network), and f(P) is the extracted feature vector of the positive sample in the final feature vector (the positive sample vector P = [P1, P2, ..., P M ] T The feature vector extracted by the parallel architecture learning network is sent to the parallel architecture learning network), f(N) is the extracted feature vector of the negative sample in the final feature vector (the negative sample vector N = [N1, N2, ..., N M ] T The feature vector extracted by the parallel architecture learning network is sent to the parallel architecture learning network), and α is a hyperparameter (margin) used to control the distance interval between positive and negative samples.
[0061] By training the parallel architecture learning network, the parallel architecture learning network can effectively process difficult samples and learn a robust feature vector representation, so that it can maintain high recognition accuracy under different lighting, posture and expression changes, reduce the misrecognition rate, and improve the reliability and effectiveness of the parallel architecture learning model in practical applications.
[0062] Phase 2: Send the recognizable human image dataset into the parallel architecture learning model for inference to obtain the polynomial operator module, and build a face recognition model based on the polynomial operator module.
[0063] Step 5: Obtain a dataset of identifiable person images, and use a face detection algorithm to detect the face regions in each identifiable person image in the dataset, and then preprocess these face regions to obtain a dataset of identifiable person face region images.
[0064] The identifiable person image dataset is used to build an identifiable person database and determine the polynomial operator module of the face recognition model. The people involved in the identifiable person image dataset are the people that the face recognition system can eventually recognize. For example, when the face recognition system is the access control system for teachers and students of a certain university, the people included in its identifiable person image dataset are the portraits of all teachers and students of the university. There is more than one identifiable person image for each person in the identifiable person image dataset, including images of the person from different angles.
[0065] The image processing method of step 5 for the images in the recognizable person image dataset is the same as the image processing method of step 1 for the images in the sample image dataset.
[0066] Step 6: Send the identifiable face region image dataset to the parallel architecture learning model to extract the first eigenvector, the second eigenvector, the third eigenvector and the final eigenvector of each identifiable face region image.
[0067] The first feature vector u1 is the feature vector output by the convolution layer of the first learning branch of the parallel architecture learning model. The second feature vector u2 is the feature vector output by the fully connected layer of the first learning branch of the parallel architecture learning model. The third feature vector u3 is the feature vector output by the second learning branch of the parallel architecture learning model. The final feature vector z is the feature vector output by the fusion layer of the parallel architecture learning model.
[0068] Step 7: Based on the final feature vectors of the facial region images of each identifiable person extracted in step 6, the average value of all the final feature vectors of the same identifiable person is used as the comparison feature vector of the identifiable person.
[0069] For each identifiable person j, its identifiable person comparison feature vector Z j for:
[0070]
[0071] Among them, z jn is the nth final feature vector of the jth identifiable person (the final feature vector extracted from the nth identifiable face region image of the jth identifiable person fed into the parallel architecture learning model), N j is the number of identifiable face area images of the jth person.
[0072] A database of identifiable persons is constructed using the reference feature vectors of all identifiable persons, and the database of identifiable persons is stored in the memory of the person recognition system. In the subsequent face recognition process, the final feature vector of the person to be recognized is compared with the reference feature vectors of each identifiable person in the database of identifiable persons.
[0073] Step 8: Based on the first feature vector and the third feature vector of each identifiable person's face region image extracted in step 6 and the identifiable person's comparison feature vector obtained in step 7, a polynomial operator module is constructed.
[0074] In the most ideal case, the final feature vectors of the same person are the same, that is, their Euclidean distance is 0. Therefore, in order to better optimize the network, we use the second feature vector u 2k Replace it with the control feature vector, that is, the control feature vector Z of the same person that is stored j→k Extract feature vector u with FaceNet 3k The difference is taken as the ideal output of the polynomial operator module, and the input u of the fully connected layer is 1k As the input of the polynomial operator module, the eigenvector coefficient w of the polynomial operator module is solved by the least squares method as follows:
[0075]
[0076] Where: U1 is the row concatenation vector of the first eigenvector, U1 = [u 11 ,u 12 ,…,u 1K ] T ,u 11 ,u 12 ,…,u 1K They represent the first feature vectors extracted from the 1st, 2nd, ..., Kth identifiable face region images; U3 is the row concatenation vector of the third feature vector, U3 = [u 31 ,u 32 ,…,u 3K ] T ,u 31 ,u 32 ,…,u 3KThey represent the third feature vectors extracted from the 1st, 2nd, …, Kth identifiable face region images respectively; Z is the row concatenation vector of the reference feature vector, Z = [z1, z2, …, z K ] T ,z1,z2,…,z K They respectively represent the control feature vectors of identifiable persons corresponding to the 1st, 2nd, …, Kth identifiable person face area images; K is the number of identifiable person face area images in the identifiable person face area image set.
[0077] Since the parallel architecture learning network has been trained in stage one, the trained parallel architecture learning network (parallel architecture learning model) can be used to construct the polynomial operator module. Therefore, if the face recognition system needs to be updated (such as adding a new identifiable person or replacing an identifiable person), there is no need to re-train the parallel architecture learning network in stage one. Instead, the updated identifiable person image data set can be directly fed into the trained parallel architecture learning network to complete the update process of the face recognition system, which greatly shortens the training time.
[0078] Step 9: Replace the fully connected layer in the parallel architecture learning model with a polynomial operator module to obtain a face recognition model.
[0079] See also Figure 3 The face recognition model includes a first learning branch, a second learning branch and a fusion layer. The first learning branch is composed of a convolution layer and a polynomial operator module in series, and the output of the convolution layer is connected to the input of the polynomial operator module. The second learning branch is the face recognition network. The input of the convolution layer and the input of the face recognition network together form the input of the face recognition model, the output of the polynomial operator module and the output of the face recognition network are connected to the input of the fusion layer, and the output of the fusion layer forms the output of the face recognition model.
[0080] The calculated polynomial operator is used to replace the fully connected layer, thereby improving the accuracy of training, and when a new category of face images is introduced, there is no need to re-perform back propagation calculations, which significantly improves the training speed. In this way, the model can not only classify faces more accurately, but also achieve rapid adaptation and efficient updates when faced with new data, further optimizing the overall performance of the face recognition system.
[0081] Stage 3: Send the image of the person to be identified into the face recognition model to achieve face recognition.
[0082] Step 10: Obtain an image of a person to be identified, and use a face detection algorithm to detect a face region in the image of the person to be identified, and preprocess the face region to obtain an image of the face region of the person to be identified.
[0083] The image processing method of the image of the person to be recognized in step 10 is the same as the image processing method of the image in the sample image dataset in step 1 and the image processing method of the image in the identifiable person image dataset in step 5.
[0084] Step 11: Send the facial region image of the person to be identified into the face recognition model, extract the final feature vector of the person to be identified, calculate the similarity between the extracted final feature vector of the person to be identified and the reference feature vectors of each stored identifiable person, and give the recognition result.
[0085] First, the facial region image of the person to be identified is sent to the face recognition model for inference to obtain the final feature vector of the person to be identified.
[0086] Next, by calculating the final feature vector z of the person to be identified new The comparison feature vector Z of each identifiable person stored in the face recognition system j The Euclidean distance d between j To determine their similarity; where the Euclidean distance d j for:
[0087]
[0088] Finally, the minimum value min(d j ) is compared with the threshold: if min(d j ) is less than the threshold, then the person to be identified is determined to be j ) is the same person, and that person is output as the result; otherwise, the person to be identified is deemed different from all identifiable persons, and the result of “unknown” or “not in the database” is output.
[0089] The invention discloses a method for improving face recognition performance by using polynomial operators to assist neural network training, and belongs to the fields of computer vision and artificial intelligence. First, a face detection algorithm is used to detect the face area in an input image or video frame, and a triple sample is constructed after a preprocessing operation is performed on the detected face. A network architecture with a parallel structure added on the basis of a face recognition network with fixed parameters is adopted, and training is performed using triple samples. By fixing the initial parameters, only a small number of key parameters need to be adjusted in the subsequent training process, thereby simplifying the training process. After the training is completed, the trained model is saved, and the feature vectors of the anchor samples in each category are calculated and averaged to obtain the feature vectors of each category and store them. Then, all parameters in the entire network architecture are fixed, and a polynomial operator is used to replace the fully connected layer part. According to the input and ideal output of the fully connected layer, the characteristic coefficients of the polynomial operator module are solved using the least squares method. The polynomial operator is combined with the neural network. When there is a new category of face image, it is no longer necessary to re-train through back propagation, optimize the training process of the neural network, and improve the training efficiency and recognition accuracy of the model. Finally, the similarity between facial features is determined by calculating the Euclidean distance between the new feature vector and the stored feature vector, and finally the recognition result is given.
[0090] The present invention combines a polynomial operator with a face recognition network, and by fixing the parameters of the main face recognition network, only the polynomial operator part is trained to optimize the training process of the neural network, so as to improve the training efficiency, adaptability and recognition accuracy of the face recognition model. The feature vectors of each category are stored using a parallel architecture, and the feature vectors of all categories are spliced row by row to obtain the final feature vector matrix. When a new category of human image arrives, the trained network architecture is directly used to calculate its feature vector, avoiding back propagation for retraining, and the feature vector matrix is expanded for storage, which is used for subsequent solution of the ideal output of the polynomial operator module. The present invention not only optimizes the training process, effectively reduces the number of parameters that need to be adjusted in the model training process, reduces the consumption of computing resources, but also can ensure the recognition accuracy of the model, and is particularly suitable for processing large-scale data sets, and can significantly speed up the training speed and improve the adaptability of the model on new tasks.
[0091] It should be noted that although the embodiments of the present invention described above are illustrative, they are not intended to limit the present invention, and therefore the present invention is not limited to the above specific embodiments. Without departing from the principles of the present invention, any other embodiments obtained by those skilled in the art under the guidance of the present invention are deemed to be within the protection of the present invention.
Claims
1. A method for improving face recognition performance by using polynomial operators to assist neural network training, characterized in that: The steps include: Step 1: Obtain a sample image data set, and use a face detection algorithm to detect face regions in each sample image of the sample image data set, and then pre-process these face regions to obtain a sample face region image data set; Step 2: Select anchor samples, positive samples, and negative samples from the sample face area image dataset to construct sample triples to obtain a sample triplet dataset; wherein the anchor sample is a sample face area image of a certain person, the positive sample is another sample face area image of a certain person, and the negative sample is a sample face area image of another person; Step 3, construct a parallel architecture learning network; the parallel architecture learning network includes a first learning branch, a second learning branch and a fusion layer; the first learning branch is composed of a convolutional layer and a fully connected layer in series, and the output of the convolutional layer is connected to the input of the fully connected layer; the second learning branch is a face recognition network that has been trained and has fixed parameters; the input of the convolutional layer and the input of the face recognition network together form the input of the parallel architecture learning network, the output of the fully connected layer and the output of the face recognition network are connected to the input of the fusion layer, and the output of the fusion layer forms the output of the parallel architecture learning network; Step 4: Use the sample triplet data set to train the parallel architecture learning network to obtain a parallel architecture learning model; Step 5: Obtain a dataset of identifiable person images, and use a face detection algorithm to detect face regions in each identifiable person image in the dataset, and then pre-process these face regions to obtain a dataset of identifiable person face region images; Step 6: Send the identifiable face region image data set to the parallel architecture learning model to extract the first feature vector, the third feature vector and the final feature vector of each identifiable face region image; Step 7: Based on the final feature vectors of the face region images of each identifiable person extracted in step 6, the average value of all the final feature vectors of the same identifiable person is used as the control feature vector of the identifiable person; Step 8: Based on the first feature vector and the third feature vector of each identifiable person's face region image extracted in step 6 and the identifiable person's comparison feature vector obtained in step 7, a polynomial operator module is constructed; the feature vector coefficient w of the polynomial operator module is: Where: U1 is the row concatenation vector of the first eigenvector, U1=[u 11 ,u 12 ,…,u 1K ] T ,u 11 ,u 12 ,…,u 1K They represent the first feature vectors extracted from the 1st, 2nd, ..., Kth identifiable face region images; U3 is the row concatenation vector of the third feature vector, U3 = [u 31 ,u 32 ,…,u 3K ] T ,u 31 ,u 32 ,…,u 3K They represent the third feature vectors extracted from the 1st, 2nd, …, Kth identifiable face region images respectively; Z is the row concatenation vector of the reference feature vector, Z = [z1, z2, …, z K ] T ,z1,z2,…,z K denote the comparison feature vectors of identifiable persons corresponding to the 1st, 2nd, ..., Kth identifiable person face region images respectively; K is the number of identifiable person face region images in the identifiable person face region image set; Step 9: Replace the fully connected layer in the parallel architecture learning model with a polynomial operator module to obtain a face recognition model; Step 10: Obtain an image of a person to be identified, and use a face detection algorithm to detect a face region in the image of the person to be identified, and then pre-process the face region to obtain an image of the face region of the person to be identified; Step 11: Send the facial region image of the person to be identified into the face recognition model, extract the final feature vector of the person to be identified, calculate the similarity between the extracted final feature vector of the person to be identified and the reference feature vectors of each stored identifiable person, and give the recognition result.
2. The method for improving face recognition performance by using polynomial operator-assisted neural network training according to claim 1 is characterized in that: In step 1, MTCNN is used as the face detection algorithm.
3. The method for improving face recognition performance by using polynomial operator-assisted neural network training according to claim 1 is characterized in that: The preprocessing process of step 1 is as follows: first, the face is rotated and scaled to a standard posture using the detected facial key points of the face area to obtain an aligned image; then, the face area is cropped out based on the aligned image; then, the pixel values of the cropped face area are normalized to obtain a normalized image; finally, the size of the face area image is adjusted to meet the input size requirements of the deep learning model to obtain a sample face area image.
4. The method for improving face recognition performance by using polynomial operator-assisted neural network training according to claim 1 is characterized in that: In step 3, the face recognition network in the parallel architecture learning network is FaceNet.
5. The method for improving face recognition performance by using polynomial operator-assisted neural network training according to claim 1 is characterized in that: The learning process of step 4 is as follows: the first feature vector extracted by the first learning branch and the second feature vector extracted by the second learning branch are merged to obtain the final feature vector, and the triplet loss is calculated based on the final feature vector. While keeping the parameters of the second learning branch unchanged, the parameters of the first learning branch are updated through back propagation to minimize the triplet loss and obtain a parallel architecture learning model.
6. The method for improving face recognition performance by using polynomial operator-assisted neural network training according to claim 5 is characterized in that: In step 4, the triplet loss L(A,P,N) is: Where ||·||2 represents the Euclidean distance, f(A) is the extracted feature vector of the anchor sample in the final feature vector, f(P) is the extracted feature vector of the positive sample in the final feature vector, f(N) is the extracted feature vector of the negative sample in the final feature vector, α is a hyperparameter, i=1,2,...,M, and M is the number of sample triplets.
7. According to the method of improving face recognition performance by using polynomial operators to assist neural network training as described in claim 1, in step 6, the first feature vector is the feature vector output by the convolution layer of the first learning branch of the parallel architecture learning model; the third feature vector is the feature vector output by the second learning branch of the parallel architecture learning model; and the final feature vector is the feature vector output by the fusion layer of the parallel architecture learning model.
8. The method for improving face recognition performance by using polynomial operator-assisted neural network training according to claim 1, characterized in that: In step 9, the face recognition model includes a first learning branch, a second learning branch and a fusion layer; the first learning branch is formed by connecting a convolution layer and a polynomial operator module in series, and the output of the convolution layer is connected to the input of the polynomial operator module; The second learning branch is the face recognition network; the input of the convolutional layer and the input of the face recognition network together form the input of the face recognition model, the output of the polynomial operator module and the output of the face recognition network are connected to the input of the fusion layer, and the output of the fusion layer forms the output of the face recognition model.
9. The method for improving face recognition performance by using polynomial operator-assisted neural network training according to claim 1, characterized in that: In step 11, the similarity between the final feature vector of the person to be identified and the reference feature vectors of each identifiable person is calculated using Euclidean distance calculation.
