A small sample face recognition method and system

Abstract

The present application relates to a kind of small sample face recognition method and system, belong to face recognition technical field, solve the problem of the recognition result deviation of present face sample quantity short time being big. Including obtaining face picture, face picture is input to the face recognition network trained, obtain face feature vector, construct face image library;Face recognition network extracts the feature of public face dataset by training encoding network and decoding network, and the feature of large sample face dataset in it is migrated to the feature of small sample face dataset and is obtained by training fine-grained network;Real-time acquisition video picture is detected and preprocessed, obtain the face picture to be identified, input to the face recognition network trained, obtain the feature vector to be identified;Based on Euclidean distance, obtain the face picture corresponding to the face feature vector in face image library with the Euclidean distance minimum and less than threshold value of the feature vector to be identified, as recognition result. The accuracy of small sample face recognition is improved.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and in particular to a small-sample facial recognition method and system. Background Technology

[0002] Face recognition technology is one of the fundamental and long-standing research topics in computer vision. With the development of deep convolutional neural networks and large-scale datasets, deep learning-based face recognition technology has made unprecedented progress and is widely applied to real-life needs. Given a pair of image or video frames as input, a face recognition neural network outputs facial features, thereby achieving face recognition.

[0003] While current face recognition methods based on neural network models can achieve good detection results, the weight files obtained from training the models are large, resulting in a huge number of parameters and computational costs, leading to long processing times. Furthermore, previous deep neural network face recognition algorithms required training with large datasets of face samples to achieve good recognition results. However, when the number of training samples for a single category is small, the model lacks the ability to represent a limited number of face images, reducing recognition accuracy. Most publicly available face datasets, such as WiderFace, MegaFace, LFW, and MS1MV2, exhibit severe class imbalance, i.e., a long-tail effect. From an ethnic perspective, Caucasians and Greater Caucasians dominate, accounting for approximately 72% of the dataset, while East Asians and Black Africans account for approximately 8%. From an age perspective, data from the 17-46 age group dominates, accounting for approximately 65%. From a facial image perspective, frontal view data accounts for approximately 52%, half-face view data accounts for approximately 16%, and oblique side view data accounts for approximately 32%. The aforementioned imbalances in the distribution of data categories result in the Arcface model, based on this data, being a biased estimation model.

[0004] Moreover, most face recognition models are optimized and trained by adding a regularization factor to the softmax cost function. This method has two drawbacks: 1) It requires a large dataset and a large number of face samples in each face ID, otherwise the trained model will have difficulty having good generalization ability; 2) When there is a large number of class imbalances in the data, the model trained in this way often has a large bias, resulting in poor accuracy on data with fewer classes, and thus inaccurate face recognition results. Summary of the Invention

[0005] Based on the above analysis, the embodiments of the present invention aim to provide a face recognition method and system to solve the problem of large deviation in recognition results when the number of existing face samples is small.

[0006] On one hand, embodiments of the present invention provide a small-sample face recognition method, comprising the following steps:

[0007] Acquire face images, input the face images into a trained face recognition network to obtain face feature vectors, and construct a face image database based on the face feature vectors and their corresponding face images; the face recognition network extracts features from publicly available face datasets by training encoding and decoding networks, and then transfers the features from large-sample face datasets to features from small-sample face datasets to train a fine-grained network.

[0008] The system detects and preprocesses real-time captured video images to obtain face images to be recognized. These images are then input into a trained face recognition network to obtain feature vectors to be recognized. Based on Euclidean distance, the system selects the face image corresponding to the face feature vector in the face image database that has the smallest Euclidean distance to the feature vector to be recognized and is less than a threshold. This face image is then used as the recognition result.

[0009] Based on a further improvement of the above method, using Euclidean distance, the face image corresponding to the face feature vector in the face image database that has the smallest Euclidean distance to the feature vector to be identified and is less than a threshold is obtained, including:

[0010] Cluster analysis is performed on the facial feature vectors in the facial image database to obtain the categories of each vector;

[0011] Calculate the Euclidean distance between the feature vector to be identified and the cluster center of each vector category. Take the vector category corresponding to the smallest Euclidean distance as the comparison category. Then calculate the Euclidean distance between the feature vector to be identified and the face feature vector in the comparison category. Obtain the face image corresponding to the face feature vector with the smallest Euclidean distance and less than the threshold.

[0012] Based on further improvements to the above method, the face recognition network extracts features from publicly available face datasets by training encoding and decoding networks, and then transfers features from large-sample face datasets to features from small-sample face datasets to train a fine-grained network, including:

[0013] The publicly available face datasets are preprocessed and divided into large-sample datasets and small-sample datasets based on the number of face images of the same person in each category.

[0014] Based on a large sample dataset, after training the encoding and decoding networks, the features of the publicly available face dataset are extracted by the encoding network and added to the feature set; the features of the large sample face dataset are then transferred to the features of the small sample face dataset, and the feature set is updated.

[0015] Based on the updated feature set, construct triplet feature samples and train a fine-grained network using the triplet function as the loss function.

[0016] Based on further improvements to the above method, the publicly available face dataset is preprocessed, including:

[0017] The RetinaFace facial landmark detection model was used to detect landmarks in face images from a publicly available face dataset, and the coordinates of the landmarks were obtained. The landmarks include: left eye, right eye, left corner of mouth, right corner of mouth, and nose.

[0018] By horizontally flipping a face image, the coordinates of key points after the flip are obtained;

[0019] Calculate the absolute distance between the coordinates of each key point in each face image and the coordinates of the corresponding key point after flipping, and sum them to obtain the difference value of each face image before and after flipping.

[0020] Filter out face images with a difference value greater than the difference threshold to obtain a preprocessed public face dataset.

[0021] Further improvements to the above method involve training the encoder and decoder networks using a large sample dataset, including:

[0022] Construct encoding and decoding networks, set initial network parameters for each, and obtain the reconstructed network;

[0023] Based on a large sample dataset, the L2 regression function is used as the loss function, and the SGD stochastic gradient descent method is used to optimize and reconstruct the network parameters. After training, the determined encoding network model parameters are obtained.

[0024] Based on further improvements to the above method, the encoding network includes multiple bottleneck layers, one flattening layer, and one fully connected layer. The first bottleneck layer includes three convolutional layers, and the other bottleneck layers each include four convolutional layers. Each convolutional layer includes one convolutional operation, one batch normalization, and one linear rectifier. The decoding network includes one fully connected layer, one reshape layer, and multiple upsampling layers. The number of upsampling layers in the decoding network is the same as the number of bottleneck layers in the encoding network. Each upsampling layer includes one transposed convolutional layer and three convolutional layers. The fine-grained network has a multi-layer structure, with each layer including one fully connected layer, one batch normalization, and one linear rectifier.

[0025] Based on the further improvement of the above method, the input of the encoding network is sample data in M×M×3 format, 96≤M≤168; the output of the encoding network is a feature vector of N×1, 256≤N≤320.

[0026] Further improvements to the above method involve transferring features from a large-sample face dataset to features from a small-sample face dataset, including:

[0027] Based on the features of a large sample face dataset, calculate the mean of the feature vectors of each class of samples, and construct the covariance matrix of the large sample.

[0028] PCA is used to reduce the dimensionality of the covariance matrix, and the first S eigenvectors are taken to obtain the feature matrix;

[0029] Based on the feature matrix and the features of the large-sample face dataset, the following formula is used to perform transfer learning on the features of each small-sample face dataset:

[0030] z jk =u jp +QQ T (z lq -u lr )

[0031] Among them, z jk Let u be the feature vector of the k-th sample in the j-th class of the small sample dataset. jp Let z be the mean of the sample vectors of the j-th class in the small sample dataset. lq Let u be the feature vector of the q-th sample from the l-th class of a large dataset. lr Let be the mean of the l-th class sample vectors randomly selected from the large sample dataset, and Q be the feature matrix.

[0032] Based on further improvements to the above method, triplet feature samples are constructed according to the updated feature set, including:

[0033] The algorithm sequentially selects a feature vector from the feature set as an anchor point, then sequentially selects a feature vector belonging to the same class as the current anchor point as a positive example, forming a tuple with the anchor point. Finally, iterates through the features and selects a feature vector belonging to a different class than the current anchor point. When the absolute value of the difference between the distance between feature vectors of the same class and the distance between feature vectors of different classes is less than the distance threshold, the feature vector is used as a negative example and added to the current tuple, thus constructing a triplet vector sample.

[0034] On the other hand, embodiments of the present invention provide a small-sample face recognition system, including:

[0035] The face image library construction module is used to acquire face images, input the face images into a trained face recognition network to obtain face feature vectors, and construct a face image library based on the face feature vectors and their corresponding face images. The face recognition network extracts features from publicly available face datasets by training encoding and decoding networks, and then transfers the features from large-sample face datasets to features from small-sample face datasets to train a fine-grained network.

[0036] The face recognition module is used to detect and preprocess real-time captured video images to obtain the face image to be recognized, which is then input into the trained face recognition network to obtain the feature vector to be recognized. Based on Euclidean distance, the face image corresponding to the face feature vector with the smallest Euclidean distance to the feature vector to be recognized and less than the threshold is obtained from the face image database and used as the recognition result.

[0037] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0038] 1. The parameters of the encoding network are optimized by using reconstruction regression and adaptive gradient descent optimization methods through encoding and decoding, so that the facial image features extracted from the encoding network have rich image semantic features;

[0039] 2. By using transfer learning, the semantics of images in small samples are enriched, and the error caused by data imbalance is reduced, enabling the face recognition network to correctly match and recognize the side profile of the same person based on the frontal face image in a small sample, thereby improving the recognition accuracy.

[0040] 3. By training the encoding network and decryption network, the parameters of the encoding network are solidified. Then, based on the output of the encoding network, a fine-grained network is trained. By repeating these two stages, the overall face recognition network is optimized, enabling the face recognition network to have good generalization ability and high recognition accuracy.

[0041] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0042] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0043] Figure 1 This is a flowchart of a small-sample face recognition method according to Embodiment 1 of the present invention;

[0044] Figure 2 This is a schematic diagram of the face recognition network in Embodiment 1 of the present invention. Detailed Implementation

[0045] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0046] Example 1

[0047] A specific embodiment of the present invention discloses a small-sample face recognition method, such as... Figure 1 As shown, it includes the following steps:

[0048] S11: Obtain face images, input the face images into the trained face recognition network to obtain face feature vectors, and construct a face image database based on the face feature vectors and their corresponding face images; the face recognition network extracts features from publicly available face datasets by training encoding and decoding networks, and transfers the features of large-sample face datasets to the features of small-sample face datasets to train a fine-grained network.

[0049] It should be noted that facial images can be acquired based on the preset application scenario. This can be done by detecting historical video images containing faces, or by collecting ID photos of individuals. The acquired facial images are processed using the RetinaFace facial landmark detection model to detect five key facial landmarks: left eye, right eye, left corner of the mouth, right corner of the mouth, and nose. Affine transformations are then used to correct the face. The corrected image is then resized and cropped to fit the input size of the facial recognition network.

[0050] Preprocessed face images are input into a trained face recognition network. The output face feature vectors, rich in semantic features, are stored in a face image database for comparison during real-time face recognition. The face image corresponding to the face feature vector is the output result when a match is found.

[0051] The face recognition network in this embodiment includes an encoding network, a decoding network, and a fine-grained network. These are pre-built and trained before constructing the face image database. The encoding network is used to extract features from face images. The decoding network, together with the encoding network, optimizes the parameters of the encoding network using reconstruction regression and adaptive gradient descent optimization methods, so that the parameters output by the encoding network have rich image semantic features. The fine-grained network is used to enable the output features to separate different types of faces, so that the Euclidean distance between the face feature vectors of the same person is small, and the Euclidean distance between the face feature vectors of different people is large.

[0052] When training on a publicly available face dataset, the encoding and decoding networks are trained first. After training, the parameters of the encoding network are fixed, and then the fine-grained network is trained. Because face images collected in real-world applications often exhibit class imbalance and limited facial angles, to ensure the trained face recognition network can adapt to these scenarios and possesses high recognition accuracy and low error, during training, large and small sample sets are created based on the number of each class in the publicly available face dataset. The rich semantic features of the large samples are then transferred to the features of the small samples, improving the discriminative power of the fine-grained network.

[0053] Specifically, the training process for a face recognition network includes:

[0054] 1) Preprocess the publicly available face datasets, dividing them into large-sample datasets and small-sample datasets based on the number of face images of the same person in each category.

[0055] It should be noted that publicly available face datasets include: WiderFace, MegaFace, LFW, MS1MV2, etc. Preprocessing of publicly available face datasets includes:

[0056] The RetinaFace facial landmark detection model was used to detect landmarks in face images from a publicly available face dataset, and the coordinates of the landmarks were obtained. The landmarks include: left eye, right eye, left corner of mouth, right corner of mouth, and nose.

[0057] By horizontally flipping a face image, the coordinates of key points after the flip are obtained;

[0058] Calculate the absolute distance between the coordinates of each key point in each face image and the coordinates of the corresponding key point after flipping, and sum them to obtain the difference value of each face image before and after flipping.

[0059] Filter out face images with a difference value greater than the difference threshold to obtain a preprocessed public face dataset.

[0060] It should be noted that in many cases, using a completely profile face to identify a face ID can easily cause confusion, and even manual identification has a high probability of misidentification. This embodiment filters out most of the more skewed face images in this way, which greatly reduces the possibility of the network failing to converge during training due to these abnormal data.

[0061] After preprocessing the publicly available face dataset, it is divided into large-sample and small-sample datasets based on the number of face images of the same person in each category. The specific face categories and division rules are determined according to the actual situation. For example, face categories include: ethnicity, age, face image perspective, etc. If the ethnicity category is selected, the large-sample dataset is composed of data with >= 50 face images of the same person in each ethnicity category, that is, each face ID contains more than 50 face images from different angles. The small-sample dataset is composed of data from different ethnicities than the large-sample dataset, and the number of face images of the same person is < 5, that is, each face ID has at most 4 face images, mostly frontal.

[0062] It should be noted that large and small sample datasets can be further divided into training and test sets. The training set is used to train the network, while the test set is used to test the network's performance, preventing overfitting and undertraining of the network to the training dataset. This embodiment uses both training and test sets for network training, which are standard practices and will not be elaborated upon separately.

[0063] 2) After training the encoding and decoding networks based on the large sample dataset, extract the features of the publicly available face dataset from the encoding network and put them into the feature set; transfer the features of the large sample face dataset to the features of the small sample face dataset and update the feature set.

[0064] Specifically, an encoding network and a decoding network are constructed, and initial network parameters are set for each to obtain the reconstructed network. The input to the encoding network is the same as the input to the entire face recognition network. The purpose of the encoding network is to encode and compress high-dimensional image data into a low-dimensional vector feature with rich image semantics.

[0065] In this embodiment, the input to the encoding network is face image data in M×M×3 format, 96≤M≤168, and the output is a feature vector of N×1, 256≤N≤320.

[0066] It's important to note that when the face size exceeds 168 pixels, adding more face pixels does not improve the model's feature vector representation ability; instead, it increases the number of model parameters, leading to slower inference performance. Conversely, a value below 96 will reduce the model's recognition accuracy. Since a low-dimensional feature vector needs to be output, the dimension of N cannot be too high, otherwise the compression purpose will be lost. When the model dimension is below 256, the variance of the feature vector statistics begins to decrease, thus reducing the richness of facial features contained in the feature vector. Therefore, a value of 320 is chosen as the upper bound for N, and 256 as the lower bound.

[0067] Preferably, since the kernel size in the convolution operator is usually chosen to be 7 or 3, M is selected as a number that is divisible by 7 or 3 within the numerical range (96, 168).

[0068] The encoding network consists of multiple bottleneck layers, one flatten layer, and one fully connected dense layer. The first bottleneck layer includes three convolutional layers, and the other bottleneck layers each include four convolutional layers. Each convolutional layer includes one convolutional operation (conv), one batch normalization (batch normalization), and one linear rectifier (relu).

[0069] Preferably, such as Figure 2 As shown, the encoding network includes 4 bottleneck layers. The kernel size of the first convolutional layer in the first bottleneck layer is set to 7, and the kernel size of the other 4 convolutional layers in the bottleneck layer is set to 3, with only the first convolutional layer being downsampled by 1 / 2.

[0070] The purpose of the decoding network is to decode the output of the encoding network back into the original image output, with the final output feature vector being the same as the input of the encoding network. Although the output of the decoding network cannot completely recover the information of the original image after training, the goal is not to recover the original image information, but to optimize the parameters of the encoding network by using reconstruction regression and adaptive gradient descent optimization methods through encoding and decoding, so that the output features of the encoding network have rich image semantic features.

[0071] The decoding network consists of one fully connected dense layer, one reshape layer, and multiple upsampling layers. The number of upsampling layers in the decoding network is the same as the number of bottleneck layers in the encoding network. Each upsampling layer includes one transposed convolutional layer and three convolutional layers.

[0072] For example, in Figure 2 In this model, a 6-layer encoding network and a 6-layer decoding network are constructed. The input of the encoding network is 112×112×3, where 112 and 112 represent the height and width of the image, respectively, and 3 represents the number of color gamut channels of the image, i.e., an RGB image; the output is 320×1; the output of the decoding network is 112×112×3.

[0073] Based on a large sample dataset, using L2 regression as the loss function, the SGD stochastic gradient descent method is used to optimize and reconstruct the network parameters, resulting in a determined encoding network model parameter after training. The preprocessed public face dataset is then input into the trained encoding network, and the resulting feature vectors are used as features of either the large-sample or small-sample face dataset, and added to the feature set.

[0074] Because large-sample data contains images of the same face taken from multiple angles, and varies due to factors such as different hairstyles, age groups, and lighting conditions, the data samples are diverse. Therefore, transfer learning is used to transfer features from large-sample data to small-sample data, reducing recognition bias caused by data imbalance and enabling the correct matching and recognition of profile images of the same person based on frontal images. Specifically, the features transferred from the large-sample face dataset to the small-sample face dataset include:

[0075] ① Based on the features of the large-sample face dataset, calculate the mean of the feature vectors of each class of samples and construct the covariance matrix of the large sample.

[0076] It should be noted that this embodiment assumes that the feature vectors follow a Gaussian distribution. The arithmetic mean of each component of the feature vector for each class of samples is calculated to obtain the mean of the feature vector for each class of samples. The covariance matrix of the large sample is then constructed according to the following formula:

[0077]

[0078] Among them, u ir Let z be the mean of the i-th class sample vectors in a large dataset. ik Let be the feature vector of the k-th sample in the i-th class of the large sample dataset, m be the total number of samples in the i-th class of the large sample dataset, and n be the total number of sample classes in the large sample dataset.

[0079] ② Perform PCA dimensionality reduction on the covariance matrix and take the first S eigenvectors to obtain the eigenma matrix.

[0080] For example, if each sample feature vector is 320×1 and S is set to 100, the resulting feature matrix is ​​320×100.

[0081] ③ Based on the feature matrix and the features of the large-sample face dataset, perform transfer learning on the features of each small-sample face dataset using the following formula:

[0082] z jk =u jp +QQ T (z lq -u lr ) Formula (2)

[0083] Among them, z jk Let u be the feature vector of the k-th sample in the j-th class of the small sample dataset. jp Let z be the mean of the sample vectors of the j-th class in the small sample dataset. lq Let u be the feature vector of the q-th sample from the l-th class of a large dataset. lrLet be the mean of the l-th class sample vectors randomly selected from the large sample dataset, and Q be the feature matrix.

[0084] After transfer learning of each feature in a small sample face dataset, the feature set is updated.

[0085] 3) Based on the updated feature set, construct triplet feature samples and train a fine-grained network using the triplet function as the loss function.

[0086] It should be noted that the updated feature set includes features from both the large-sample face dataset and the small-sample face dataset after transfer learning. Based on the updated feature set, triplet feature samples are constructed, including:

[0087] The algorithm iterates through the feature set, taking a feature vector as an anchor point, and then taking a feature vector belonging to the same class as the current anchor point as a positive example. This positive example is then combined with the anchor point to form a binary pair. Finally, it iterates through the feature set and takes a feature vector belonging to a different class than the current anchor point. When the absolute value of the difference between the distance between feature vectors of the same class and the distance between feature vectors of different classes is less than the distance threshold, this feature vector is taken as a negative example and added to the current binary pair, thus constructing a triplet vector sample.

[0088] Specifically, the absolute value of the difference between the distances between feature vectors of the same class and the distances between feature vectors of different classes is less than a distance threshold, as expressed by the following formula:

[0089]

[0090]

[0091] Among them, z a For the current anchor point, z p As a current positive example, z n As the current counterexample, α represents the distance threshold, α > 0, and norm(z) represents normalizing the feature z into a unit vector.

[0092] The fine-grained network has a multi-layer structure, with each layer consisting of a fully connected dense layer, a batch normalization layer, and a ReLU linear rectifier.

[0093] For example, in Figure 2 In this fine-grained network, there is a two-layer structure. The first layer consists of a fully connected dense layer, followed by a batch normalization layer and a ReLU linear rectifier. The second layer is the same as the first layer, and it outputs a feature vector for face recognition.

[0094] After constructing the triplet feature samples, they are input into the fine-grained network in the order of anchor point, positive example, and negative example, with the data volume in each batch being an integer multiple of 3. This yields the face feature vectors corresponding to the triplet feature samples. The fine-grained network is then trained using the triplet function as the loss function, and the fine-grained network parameters are optimized.

[0095] Specifically, the loss function is shown in the following equation:

[0096]

[0097] in, and α represents the feature vector output by the fine-grained network after the anchor point, positive example, and negative example in the r-th triplet feature sample are input into the network, R is the total number of triplet feature samples, and α is the distance threshold in formula (4).

[0098] S12: Detect and preprocess the real-time acquired video images to obtain the face image to be recognized, input it into the trained face recognition network to obtain the feature vector to be recognized; based on Euclidean distance, obtain the face image corresponding to the face feature vector in the face image database that has the smallest Euclidean distance to the feature vector to be recognized and is less than the threshold, and use it as the recognition result.

[0099] It should be noted that the real-time captured video images are preprocessed into the input size of the face recognition network according to the method in step S11, and then input into the trained face recognition network. After passing through the encoding network and fine-grained network, the feature vector to be recognized is obtained. The feature vector is then compared and recognized with the feature vectors in the face image database, including:

[0100] Cluster analysis is performed on the facial feature vectors in the facial image database to obtain the categories of each vector;

[0101] Calculate the Euclidean distance between the feature vector to be identified and the cluster center of each vector category. Take the vector category corresponding to the smallest Euclidean distance as the comparison category. Then calculate the Euclidean distance between the feature vector to be identified and the face feature vector in the comparison category. Obtain the face image corresponding to the face feature vector with the smallest Euclidean distance and less than the threshold.

[0102] It should be noted that the threshold for actual face recognition is set according to the accuracy required in the actual scenario.

[0103] Compared with existing technologies, this embodiment provides a small-sample face recognition method that optimizes the encoding network parameters using reconstruction regression and adaptive gradient descent optimization methods through encoding and decoding. This results in face image features extracted from the encoding network possessing rich semantic features. Through transfer learning, the semantics of small-sample images are enriched, reducing errors caused by data imbalance. This enables the face recognition network to correctly match and recognize profile face images of the same person based on small-sample frontal face images, improving recognition accuracy. By training the encoding and decryption networks, the encoding network parameters are solidified. Then, based on the output of the encoding network, a fine-grained network is trained. By repeating these two stages, the overall face recognition network is optimized, giving it good generalization ability and high recognition accuracy.

[0104] Example 2

[0105] Another embodiment of the present invention discloses a small-sample face recognition system, thereby implementing the face recognition method in Embodiment 1. The specific implementation of each module is described in the corresponding description in Embodiment 1. The system includes:

[0106] The face image library construction module is used to acquire face images, input the face images into a trained face recognition network to obtain face feature vectors, and construct a face image library based on the face feature vectors and their corresponding face images. The face recognition network extracts features from publicly available face datasets by training encoding and decoding networks, and then transfers the features from large-sample face datasets to features from small-sample face datasets to train a fine-grained network.

[0107] The face recognition module is used to detect and preprocess real-time captured video images to obtain the face image to be recognized, which is then input into the trained face recognition network to obtain the feature vector to be recognized. Based on Euclidean distance, the face image corresponding to the face feature vector with the smallest Euclidean distance to the feature vector to be recognized and less than the threshold is obtained from the face image database and used as the recognition result.

[0108] Since the small-sample face recognition system in this embodiment can draw on the relevant aspects of the aforementioned face recognition method, and this is a repetition, it will not be repeated here. Because this system embodiment shares the same principle as the aforementioned method embodiment, it also possesses the corresponding technical effects of the aforementioned method embodiment.

[0109] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0110] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A small-sample face recognition method, characterized in that, Includes the following steps: Acquire a face image, input the face image into a trained face recognition network to obtain a face feature vector, and construct a face image library based on the face feature vector and its corresponding face image; The face recognition network is obtained by training an encoding network and a decoding network to extract features from a publicly available face dataset, fixing the encoding network parameters, and then transferring the features from the large-sample face dataset to the features from the small-sample face dataset to train a fine-grained network. The process includes: preprocessing the publicly available face dataset, dividing it into large-sample and small-sample datasets based on the number of face images of the same person in each category; training the encoding and decoding networks based on the large-sample dataset, then extracting features from the publicly available face dataset using the encoding network and adding them to a feature set; and transferring the features from the large-sample face dataset to the features from the small-sample face dataset. The features of the dataset are analyzed, and the feature set is updated. Based on the updated feature set, triplet feature samples are constructed, and a fine-grained network is trained using the triplet function as the loss function. The transfer of features from the large-sample face dataset to the small-sample face dataset includes: calculating the mean of the feature vectors of each class of samples based on the features of the large-sample face dataset, and constructing the covariance matrix of the large sample; performing PCA dimensionality reduction on the covariance matrix, and taking the first S feature vectors to obtain the feature matrix; based on the feature matrix and the features of the large-sample face dataset, the following formula is used to perform transfer learning on the features of each small-sample face dataset: ， in, For the first in a small sample dataset The first class of samples The feature vector of each sample For the first in a small sample dataset Class sample vector mean, The first one randomly selected from the large sample dataset The first class of samples The feature vector of each sample The first one randomly selected from the large sample dataset Class sample vector mean, The characteristic matrix; The system detects and preprocesses real-time captured video images to obtain face images to be recognized. These images are then input into a trained face recognition network, where they pass through the encoding network and fine-grained network to obtain the feature vector to be recognized. Based on Euclidean distance, the system obtains the face image corresponding to the face feature vector in the face image database that has the smallest Euclidean distance to the feature vector to be recognized and is less than a threshold. This face image is then used as the recognition result.

2. The small-sample face recognition method according to claim 1, characterized in that, The step of obtaining the face image corresponding to the face feature vector in the face image database that has the smallest Euclidean distance to the feature vector to be identified and is less than a threshold, based on Euclidean distance, includes: Cluster analysis is performed on the facial feature vectors in the facial image database to obtain the categories of each vector; Calculate the Euclidean distance between the feature vector to be identified and the cluster center of each vector category, take the vector category corresponding to the smallest Euclidean distance as the comparison category, and then calculate the Euclidean distance between the feature vector to be identified and the face feature vector in the comparison category, and obtain the face image corresponding to the face feature vector with the smallest Euclidean distance and less than the threshold.

3. The small-sample face recognition method according to claim 2, characterized in that, The preprocessed publicly available face dataset includes: The RetinaFace facial landmark detection model was used to detect landmarks in facial images from a publicly available facial dataset, and the coordinates of the landmarks were obtained. The landmarks include: left eye, right eye, left corner of mouth, right corner of mouth, and nose. By horizontally flipping a face image, the coordinates of key points after the flip are obtained; Calculate the absolute distance between the coordinates of each key point in each face image and the coordinates of the corresponding key point after flipping, and sum them to obtain the difference value of each face image before and after flipping. Filter out face images with a difference value greater than the difference threshold to obtain a preprocessed public face dataset.

4. The small-sample face recognition method according to claim 2, characterized in that, The process of training the encoder and decoder networks based on a large sample dataset includes: Construct encoding and decoding networks, set initial network parameters for each, and obtain the reconstructed network; Based on a large sample dataset, the L2 regression function is used as the loss function, and the SGD stochastic gradient descent method is used to optimize and reconstruct the network parameters. After training, the determined encoding network model parameters are obtained.

5. The small-sample face recognition method according to claim 2, characterized in that, The encoding network includes multiple bottleneck layers, one flattening layer, and one fully connected layer. The first bottleneck layer includes three convolutional layers, and the other bottleneck layers each include four convolutional layers. Each convolutional layer includes one convolutional operation, one batch normalization, and one linear rectifier. The decoding network includes one fully connected layer, one reshape layer, and multiple upsampling layers. The number of upsampling layers in the decoding network is the same as the number of bottleneck layers in the encoding network. Each upsampling layer includes one transposed convolutional layer and three convolutional layers. The fine-grained network has a multi-layer structure, with each layer including one fully connected layer, one batch normalization, and one linear rectifier.

6. The small-sample face recognition method according to claim 4, characterized in that, The input to the encoding network is sample data in M×M×3 format, where 96≤M≤168; the output of the encoding network is a feature vector of N×1, where 256≤N≤320.

7. The small-sample face recognition method according to claim 2, characterized in that, The step of constructing triplet feature samples based on the updated feature set includes: The algorithm sequentially selects a feature vector from the feature set as an anchor point, then sequentially selects a feature vector belonging to the same class as the current anchor point as a positive example, forming a tuple with the anchor point. Finally, iterates through the features and selects a feature vector belonging to a different class than the current anchor point. When the absolute value of the difference between the distance between feature vectors of the same class and the distance between feature vectors of different classes is less than the distance threshold, the feature vector is used as a negative example and added to the current tuple, thus constructing a triplet vector sample.

8. A small-sample face recognition system, characterized in that, The system is used to perform the small-sample face recognition method as described in any one of claims 1-7, including: The face image library construction module is used to acquire face images, input the face images into a trained face recognition network to obtain face feature vectors, and construct a face image library based on the face feature vectors and their corresponding face images. The face recognition network extracts features from publicly available face datasets by training an encoding network and a decoding network, and then transfers the features from large-sample face datasets to features from small-sample face datasets to train a fine-grained network. The face recognition module is used to detect and preprocess real-time captured video images to obtain face images to be recognized, which are then input into a trained face recognition network to obtain a feature vector to be recognized. Based on Euclidean distance, the module obtains the face image corresponding to the face feature vector in the face image database that has the smallest Euclidean distance to the feature vector to be recognized and is less than a threshold, and uses this as the recognition result.

Images