A face recognition method based on sample-level contrastive learning

Abstract

The application discloses a face recognition method based on sample-level contrast learning, which is characterized by comprising the following steps: obtaining a public face dataset; processing the dataset, and cutting the face part; designing a training framework combining sample-level contrast loss and classification loss; designing a targeted sample-level contrast loss function; combining the feature extraction network, classification loss and contrast loss in the framework, and training until convergence. Compared with the prior art, the application combines sample-level contrast loss and classification loss in the face recognition task requiring a high-quality dataset, increases the distance between different image features through the contrast loss, and the improved discrimination in the high-dimensional space helps the model to achieve better performance in the recognition task. The designed loss enhances the effect of contrast supervision, avoids the interference of the supervision signals between the contrast and classification losses, and finally realizes accurate face recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition, and particularly relates to a face recognition method based on sample-level contrast learning. BACKGROUND

[0002] Face recognition results play a very important role in security. Through a face recognition algorithm, it can be quickly verified whether a person matches a target, or a target face feature is compared with an existing face feature in a database to identify the identity of a person. The face recognition task often requires the model to be able to test the open dataset, that is, to perform face verification and face retrieval on the face images of persons other than the training data. Face verification requires determining whether two images belong to the same person, and face retrieval requires identifying images of the same person as a given image from a large number of candidate images. For the face recognition task, there are mainly two types of existing technologies: one is a class-level contrast learning method, which is similar to the processing process when solving the problem, relying on face categories to construct pairs of face image combinations for relationship optimization, but not fully utilizing the discrimination between different persons during training, and the recognition accuracy has room for improvement. The second is a classification-based method that optimizes the class judgment of independent face pictures, but does not match the processing process when solving the problem, and the model performance declines due to the difference in tasks. The joint use of these two methods has appeared in the early development of deep learning, but as the research on classification loss deepens, the performance of joint use of the two cannot exceed the use of classification loss alone.

[0003] Sample-level contrast learning is an unsupervised learning method that improves the model's understanding of the input by proxy tasks that constrain the features of the image itself. A common practice for proxy tasks is to transform the same picture into two copies with the same semantics but different images through different data enhancement processing. In the small batch training process, the images in the same batch are transformed into a copy pool through enhancement, and at this time the sample-level contrast learning requires to be able to determine whether any two copies in the copy pool come from the same image. Since the face recognition task belongs to a large number of fine-grained tasks, many factors will affect the accuracy of recognition, including the age, pose of the picture person specific to the task, and factors such as lighting and occlusion involved in general image recognition. The former should be considered in the specific application process, and the latter often appears in large-scale data sets used for training. These factors make face recognition a still challenging task.

[0004] The existing face recognition is prone to interference between contrast and classification loss supervision signals, slow training and inference speed, low accuracy and poor effect of face recognition. SUMMARY

[0005] The face recognition method based on sample-level contrast learning is designed to solve the problems in the prior art, adopts a framework compatible with sample-level contrast learning and classification learning, and a contrast loss for assisting classification learning, helps the model to adapt to image comparison in the subsequent inference stage on the premise of maintaining the differentiation of a large number of category training, realizes the face recognition algorithm with fast training and inference speed, and combines the sample-level contrast loss and the classification loss in the face recognition task requiring a high-quality data set, the distance between different image features is increased through the contrast loss, and the differentiation degree in the high-dimensional space is improved, which helps the model to obtain better performance in the identification task. The designed loss enhances the effect of contrast supervision, avoids the interference of the supervision signals between the contrast and classification losses, and finally realizes accurate face recognition effect. Both the comparison information between a large number of face categories of the training data and the inference paradigm in the application process can be introduced into the training process, the interference of the supervision signals caused by simultaneously using the category-level contrast learning and the classification learning is effectively avoided, the accuracy of the face recognition task is greatly improved, the training speed and the inference speed are considered, and the method has a certain application value for person detection in security.

[0006] The face recognition method based on sample-level contrast learning is designed to solve the problems in the prior art, adopts a framework compatible with sample-level contrast learning and classification learning, and a contrast loss for assisting classification learning, helps the model to adapt to image comparison in the subsequent inference stage on the premise of maintaining the differentiation of a large number of category training, realizes the face recognition algorithm with fast training and inference speed, and combines the sample-level contrast loss and the classification loss in the face recognition task requiring a high-quality data set, the distance between different image features is increased through the contrast loss, and the differentiation degree in the high-dimensional space is improved, which helps the model to obtain better performance in the identification task. The designed loss enhances the effect of contrast supervision, avoids the interference of the supervision signals between the contrast and classification losses, and finally realizes accurate face recognition effect. Both the comparison information between a large number of face categories of the training data and the inference paradigm in the application process can be introduced into the training process, the interference of the supervision signals caused by simultaneously using the category-level contrast learning and the classification learning is effectively avoided, the accuracy of the face recognition task is greatly improved, the training speed and the inference speed are considered, and the method has a certain application value for person detection in security.

[0007] Step S1: Acquisition of face data set

[0008] The public face data set is downloaded, and the image characters in the training set and the test set are avoided to cross, so that the open data set test is realized.

[0009] Step S2: Processing of data set

[0010] Face detection and positioning are performed on each picture, the face image part is cut, and then standardization processing is performed. The training set and the test set are processed according to the same process, so that consistency is ensured.

[0011] Step S3: Design of training framework

[0012] A special face recognition training framework combining sample-level contrast loss and classification loss is designed. In order to realize lossless fusion of the two training methods, the face recognition training framework is based on a general classification learning framework, and specifically includes: a trunk network f for extracting features, a projection layer g for mapping features, and a classification loss module L for classifying using features class In order to add sample-level contrast learning to the training process, while avoiding the destruction of the recognizable nature of the face image by the general data enhancement method, the present application extracts the feature f(x i The extracted feature f(x iPerform two random discard operations σ, and calculate the result of the random discard σ(f(x)). i Projection is performed using a projection layer g. For the i-th image (i = 0, 1, 2, ..., B) in a batch (assuming batch size B), two distinct but semantically identical feature vector copies are obtained, denoted as h. i and h i+B During mini-batch training, copies generated from each image can form a copy pool, and a pre-designed contrastive loss Li can be used. contra Constrain it. Simultaneously, constrain the two feature copies (h) i ,h i+B Using the classification loss module L respectively class To impose constraints, the model uses three losses, as expressed in equation (a) below:

[0013]

[0014] Where λ is the weighted average of the contrast loss; L is the average loss of the frame over a batch.

[0015] This invention primarily uses classification loss, therefore ensuring the overall weight of the classification loss is 1, and for each feature's two feature copies (h... j ,h j+B The classification loss is averaged.

[0016] Step S4: Design of the contrast loss function

[0017] Design a sample-level contrastive learning loss to assist classification loss, adding an additional discriminative boundary m to the commonly used InfoNCE loss for sample-level contrastive learning. c That is, the following equation (b) is expressed as:

[0018]

[0019] Where s is the scaling parameter for adjusting the output distribution of the Softmax function; exp is an exponential function with base e; and sim(h1,h2) is the similarity metric function expressed in equation (c) below:

[0020]

[0021] Here, the same scale parameter as in general classification loss is used, instead of the scale parameter form in InfoNCE, for subsequent integration. Discriminant boundary m c This increases the difficulty for the model to distinguish between feature copy pairs (from the same image feature), promotes the model's mining of deeper information, and improves the supervision effect of contrastive loss. To select appropriate discrimination boundary parameters, this invention [addresses] m cThe following (d) to (f) exponential moving averages are used:

[0022]

[0023]

[0024] Maxneg i = max(sim(h i , h j )), j e [1+B, 2B], y i ≠ y j (f);

[0025] wherein y i represents the label of the i-th sample within the batch, k represents the k-th iteration, a represents a control parameter of the moving average, and is set to 0.99; Maxneg i represents the maximum intra-batch similarity of the target copy of the non-same image, and is initialized

[0026] To avoid interference of the contrast learning on the classification loss, the feature copy relationship calculation between the same-class (person) images is excluded in the contrast loss, and only the relationship calculation between the same-image feature copy and the different-class image feature copy is used. At this time, the above (b) can be updated to the following (g) formula:

[0027]

[0028] The contrast loss promotes greater differentiation between different persons in the image during the training process, while requiring the same image of the same person to have robust recognition degree (recognize the random discard result). This brings better generalization to face recognition, and can better encode the feature of the face image that does not appear in the training data for the recognition task.

[0029] Step S5: training of the combined framework

[0030] The combined training framework includes a feature extraction trunk network, a classification loss and a sample-level contrast loss. ArcFace is a loss function that is easy to implement and has good effect, which is used as the classification loss L ArcFace in the training process and is defined by the following (h) formula:

[0031]

[0032] wherein W j is the j-th column of the classification layer weight and can represent the j-th face class. θ j is the image feature copy h i and W jCosine similarity between the image feature and the image feature copy h j ) is represented by the following (i) formula i Cosine similarity between the image feature and the image feature copy h j Cosine similarity between the image feature and the image feature copy h

[0033]

[0034] In the initial stage of training, due to the insufficient representation ability of the image feature, the contrast learning is not added, but the random discard is performed on the feature, and the single image feature is used for classification learning. In the middle and later stages of training, twice random discard is used, the sample-level contrast learning loss designed in step S4 is added, and two image features are used for classification learning.

[0035] Step S6: design appropriate model evaluation indicators and test the prediction effect of the model.

[0036] Compared with the prior art, the present application has the advantages that by introducing random discard into the classification method, a semantic-invariant image feature copy is generated, and then sample-level contrast learning can be used. At the same time, the framework only uses random discard, and the additional computational burden generated can be ignored. The designed contrast loss not only increases the distance between different image features, but also avoids the interference of the supervision signal with the classification loss. This improvement in discrimination in high-dimensional space helps the model to obtain better performance in the identification task, and finally realizes accurate face recognition effect. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is the flowchart of the present application;

[0038] Figure 2 is a schematic diagram of the framework structure combining sample-level contrast learning loss and classification loss;

[0039] Figure 3 is a schematic diagram of the calculation process and dynamic updating process of the sample-level contrast loss;

[0040] Figure 4 is a schematic diagram of the geometric interpretation of the sample-level contrast loss;

[0041] Figure 5 is a schematic diagram of the ResNet network structure. DETAILED DESCRIPTION

[0042] Referring to Figure 1 , the present application adopts a training framework combining sample-level contrast learning loss and classification loss, uses the feature extraction network in the combined framework to perform classification loss and contrast loss, and trains to convergence. The specific face recognition includes the following steps:

[0043] Step S1: obtaining a face data set

[0044] Download the public face dataset and avoid the intersection of image characters in the training set and the test set to realize the open dataset test;

[0045] Step S2: processing of the dataset

[0046] Face detection and positioning are performed on each picture of the training set and the test set, the face image part is cropped, and then standardized processing is performed;

[0047] Step S3: design of the training framework

[0048] A face recognition model training framework is designed, which combines sample-level contrast loss and classification loss, and uses random dropout for feature enhancement;

[0049] Step S4: design of the contrast loss function

[0050] A sample-level contrast learning loss function L is designed for auxiliary classification loss contra The contrast loss promotes the generation of greater distinction between images of different characters in the training process, and can avoid conflict with the supervised classification loss;

[0051] Step S5: training of the combined framework

[0052] ResNet-100 is used as the feature extraction network and ArcFace is used as the classification loss function L ArcFace and the sample-level contrast learning loss function L designed in step S4 contra The combined training framework is trained to convergence to achieve accurate face recognition effect. In the initial stage of the training, single image features are used for classification learning, and random dropout is performed on the features. In the middle and later stages of the training, two random dropouts are used, and the sample-level contrast learning loss function designed in step S4 is added, and two image features are used for classification learning.

[0053] The present application will be further described below in conjunction with the drawings and specific embodiments, but not as a limitation of the present application. The technical solutions in the embodiments of the present application are described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0054] Embodiment 1

[0055] A face recognition method based on sample-level contrast learning specifically includes the following steps:

[0056] Step S1: obtaining of the face dataset

[0057] Download public face dataset, including MS1MV2, LFW, CALFW, CPLFW, AgeDB, CFP-FP. Among them, MS1MV2 is used as the training set, and the other data sets are used as the test set. For the tasks that exist in the training set and the test set at the same time, all the pictures belonging to them are deleted from the training set to realize the open data set test.

[0058] Step S2: processing the data set

[0059] Each image is detected and positioned using MTCNN for face key points, and then the image is processed using a similarity transformation according to the detected five face key points of both eyes, nose and two corners of the mouth, and then the processed face image is cropped to obtain the image used for training. The image needs to be scaled to 112x112 pixels before being input into the model, and each RGB pixel will be subtracted by 127.5 and then divided by 128 for normalization. The training set and the test set are processed in this way.

[0060] Step S3: design of the training framework

[0061] Referring to Figure 2 , in order to enable the lossless fusion of the two training methods, the present application designs a special face recognition training framework combined with sample-level contrast loss and classification loss. The framework is based on a general classification learning framework, including: a trunk network f for extracting features, a projection layer g for mapping the features, and a classification loss module L for classifying using the features class , in order to add sample-level contrast learning to the training process while avoiding the destruction of the recognizability of face images by general data enhancement methods, the present application performs two random discard operations σ on the extracted feature f(x i ) of the i-th sample x i ) in a batch of size B. The random discarded results σ(f(x i )) are projected using the projection layer g to obtain two distinguished but semantically identical feature vector copies (h i , h i+B ). In the mini-batch training process, the copies generated using each image can form a copy pool, and a pre-designed contrast loss L contra is used to constrain it. Preferably, the classification loss module L i is used to constrain the two feature copies (h i+B , h class ) respectively. The model is constrained using the three losses represented by the following formula (a):

[0062]

[0063] where λ is the weighting for the contrast loss.

[0064] The present application mainly uses classification loss, so as to ensure that the overall weight of the classification loss is 1, and the classification losses of two feature copies (h j ,h j+B ) of each feature are averaged. L is the average loss of the framework on a batch.

[0065] Step S4: design a sample-level contrastive learning loss for auxiliary classification loss

[0066] The commonly used InfoNCE loss for sample-level contrastive learning adds an additional discrimination boundary m c , that is, the following formula (b) is expressed as:

[0067]

[0068] Wherein, s is a scale parameter for adjusting the output distribution of the Softmax function; exp is an exponential function with constant e as the base; sim(h1, h2) is a similarity measure function expressed by the following formula (c):

[0069]

[0070] Here, the same scale parameter as the general classification loss is used, and the scale parameter form in InfoNCE is not used, so as to be integrated subsequently. The discrimination boundary m c Increases the difficulty of the model to distinguish the feature copy pair (from the same image feature), promotes the model to mine deep information, and improves the supervision effect of the contrastive loss.

[0071] In order to select a suitable discrimination boundary parameter, the present application uses the following exponential moving average (d)~(f) for m c

[0072]

[0073]

[0074] Maxneg i = max(sim(h i ,h j )),j∈[1+B,2B],y i ≠y j (f).

[0075] Wherein, y i represents the label of the i-th sample in the batch, k represents the k-th iteration, and a represents the control parameter of the moving average, which is set to 0.99. Maxneg i represents the maximum non-image batch similarity of the target copy, which is initialized​

[0076] Referring to Figure 3 To avoid the interference of contrast learning on the classification loss, the feature copy relationship calculation between images of the same category (person) is excluded in the contrast loss, and only the relationship calculation between the feature copies of the same image and the feature copies of different category images is used. At this time, the above (b) can be updated to the following (g) formula:

[0077]

[0078] Referring to Figure 4 The contrast loss promotes greater differentiation between images of different persons during the training process, while requiring the same image of the same person to have robust recognition degree (recognition of random discard results). This brings better generalization to face recognition, and can better encode the features of more images of faces that do not appear in the training data for the recognition task.

[0079] Referring to Figure 4 a, in the classification-based model, the focus is on whether the similarity of the image feature h i to its label category is better than its similarity to other categories . This feature distribution adjustment mode can achieve good results in the case of using a classification boundary, but ignores the relationship processing between image features, and there is a case where the features of non-similar images are closer in high-dimensional space.

[0080] Referring to Figure 4 b, the similarity problem between the above image features will lead to incorrect discrimination in the face recognition application stage based on image comparison. According to the encoding of the image, non-similar images may have higher feature similarity between them, and thus incorrectly judge the relationship between the images. The addition of contrast learning will alleviate this problem, which narrows the distance between images with the same semantics in high-dimensional space, and makes the distance between images with different semantics further apart.

[0081] Step S5: training of the combined framework

[0082] The combined training framework includes the feature extraction trunk network, the classification loss, and the sample-level contrast loss. ArcFace is a loss function that is easy to implement and has good results, which is used as the classification loss L ArcFace in the training process defined by the following (h) formula:

[0083]

[0084] where W j is the jth column of the classification layer weight, which can represent the jth face category. θ jis an image feature copy h i W j cosine similarity between h j and W i : j

[0085]

[0086] The ArcFace method performs scale-free optimization on the high-dimensional feature vector obtained by the feature extractor, replacing the sample-class similarity score represented by in the general Softmax classification with the scale-free cos(θ j ), avoiding the influence of the sample feature vector length and the class feature vector length on the similarity measurement. Since the lengths of the two kinds of feature vectors are determined by the image quality and the number of images under each class respectively, such a loss function enables the model to treat samples of different image qualities and person classes with different numbers of images more fairly, obtaining a more evenly distributed feature space. The discriminant boundary parameter m in it processes the angle between the high-dimensional space feature vectors, increasing the discrimination between the feature vectors, which helps the model to learn the details of the person in the image and encode them as feature vectors with higher person discrimination.

[0087] Refer to Figure 5 ​The feature extraction network model f used in the application is ResNet-100, which is different from the commonly used ResNet network, and there is no 1*1 window size Conv convolution network in the backbone network. ResNet includes 4 stage modules, each stage module includes a plurality of sub-modules, and each sub-module uses a Conv convolutional neural network, a Batch Normalization batch normalization, and a PReLU activation function. When ResNet was proposed, due to the limitation of computing power, a convolutional layer with a window size of 1*1, a step of 1, and a channel number transformation was used to replace a convolutional layer with a window size of 3*3, a step of 2, and a channel number unchanged. The application uses a complete 3*3 Conv convolutional layer, and the use sequence is Batch Normalization, Conv, Batch Normalization, PReLU, Conv, and Batch Normalization. The input of each module will be directly added to the aforementioned network as the final sub-module output. For the first sub-module of each stage, due to the existence of channel number transformation, a convolutional layer with a window size of 1*1 and a step of 2 is used to process the sub-module input, and then added to the output of the aforementioned module. The ResNet-100 used in the application uses 3, 13, 30, and 3 sub-modules in each sub-stage, and the output dimensions of each sub-stage are 256, 512, 1024, and 2048, respectively.

[0088] The feature projection network g used in the application is a single-layer MLP followed by Batch Normalization. The main purpose of this network is to project high-dimensional features to a lower dimension while supporting subsequent calculations. The structure of Batch Normalization+Dropout is used between the feature extraction trunk network f (ResNet) and the feature projection network g to create contrast features, and network g avoids the empty features caused by directly using Dropout. This network structure takes into account the difficulty of real application and the requirements of technical implementation: general sample-level contrast learning methods rely on data augmentation before feature extraction and need to extract features for two samples of an image respectively. This training process doubles the computational overhead, and for face recognition tasks that require a large amount of data for long-time training, it will cause a huge computational burden. The application uses random dropout on the calculation results of the feature extraction network, which not only meets the needs of contrast learning for two related but different features, but also avoids additional computational overhead. At the same time, the selection of ResNet as the feature extraction network also takes into account that it does not include a module that generates random results, so that each random dropout is processed on the real model output feature, and then optimized through the loss function.

[0089] In the initial stage of training, due to the insufficient representation ability of image features, the contrast learning is not added, but the features are randomly discarded, and a single image feature copy is used for classification learning, and the label of the classification learning is the person category to which the image belongs. Since the number of person categories in the data set is large (more than 80,000), it is difficult to judge, and the coding ability of the model will be greatly improved, and the prediction accuracy of the image person will also be greatly improved. In the middle and later stages of training, two feature vector copies belonging to the same image are obtained by using random discarding twice. The two feature copies are respectively used for classification learning loss to constrain the category to which they belong. Since the semantics of the two vector copies are equivalent, the effect of discriminating them should also be equivalent, so the weights of the two losses are set to 0.5 respectively. Since the discarding of the features is random, the two feature copies actually only have part of the information of the original features. In the process of classification supervision, part of the features of the copy are required to be able to judge the person, and this measure will improve the quality of the coded features.

[0090] The present application uses two feature copies belonging to the same image, and more feature copies belonging to different images and different categories in the same training batch, and the contrast loss designed in step S4 is applied. The loss uses feature copies with the same semantics and supervises the semantic relationship between them, requiring the feature copies of the same image to be consistent in direction in high-dimensional space, and the feature copies of different categories and different images to be far away from each other in direction in high-dimensional space. This supervision is closely related to the implementation stage of the face recognition task, and is also a missing part in classification learning. For two image features (h1, h2) belonging to two persons, in the high-dimensional space in the classification scenario, since they are closer to the category center of each other , the direction is closer, and the judgment is correct. But in the face recognition task without category feature vectors, the distance between the two features may be less than the set threshold and thus judged as the same person, or the distance between the two features is less than the distance between one of them and other image features in the candidate set and thus judged as the same person. These two possible error forms are not explicitly supervised in classification learning, and by introducing sample-level contrast learning, the coding ability of the feature extraction model is further improved, thereby improving the distinguishability of the image features in high-dimensional space.

[0091] The contrast loss designed in step S4 cancels the similarity measurement between different image copies of the same class in the calculation process to avoid interference with the supervision signal of the classification loss. Considering the dominant position of the classification loss-based training method in the current face recognition task and the performance of the contrast method beyond the class level, the sample-level contrast learning loss should be used as an auxiliary loss that does not interfere with the supervision signal. Otherwise, the contrast loss and the classification loss will simultaneously supervise the image features, and the superimposed effect may cancel each other out, making the encoding effect of the feature extractor worse than using the classification loss alone.

[0092] To enable the contrast loss to more effectively supervise the different-class image feature copies, a dynamic discrimination boundary parameter m is added in step S4 c . The parameter is updated according to the difference between the similarity of the same image feature copies and the similarity of the different-class image feature copies, ensuring that the feature copies are supervised throughout the training process, and the constraint effect is controlled and will not cause the loss to be too large to continue optimization.

[0093] The present embodiment uses an SGD optimizer with a momentum of 0.9, a learning rate of 0.1, and a current learning rate of 1 / 10 of the original at the 8th, 14th, and 20th rounds. The model ends training at the 24th round, the weight decay is set to 0.0005, and the total batch size is 512. For the loss functions in steps S4 and S5, the present embodiment sets the weight factor of the contrast loss λ = 0.05, the discrimination boundary parameter of the classification loss m = 0.5, and the scale parameter of the classification loss and the contrast loss s = 64.

[0094] The present embodiment uses 4 blocks of NVIDIA A100 Tensor Core GPUs for distributed training, the operating system is 64-bit CentOS 7.4, the Intel(R) Xeon(R) Gold 6132 CPU is used at 2.60 GHz, and the memory size is 384 GB. The required software tools include: Python 3.8, CUDA 10.3, cuDNN 8.0, and PyTorch 1.10.0. Other alternative software tools and hardware devices can also be used to implement the techniques and steps described in the method of the present application. The software and hardware environment involved in the present embodiment is not a limitation of the present application.

[0095] Step S6: Design appropriate model evaluation indicators and test the prediction effect of the model

[0096] During the test, the classification loss layer and Dropout are discarded, and the result of the projection layer g is used as the face image feature. The evaluation criteria of the method of the application include face verification accuracy (Ver), correct acceptance rate (TAR) under a certain false acceptance rate (FPR), and face retrieval accuracy (Id). The method of the application is first trained using training data, and then the model is tested using data that does not overlap with the training data. The classification layer and random dropout are not used in the test stage. For LFW, AgeDB, CFP-FP, CALFW, CPLFW and other data sets, the face verification accuracy is used as the measurement; for IJB-B and IJB-C data sets containing about ten thousand pairs of face verification data, the face detection of TAR@FAR=0.0001 is used as the measurement; for MegaFace containing one million face images, the face detection of TAR@FAR=0.000001 and the face retrieval accuracy are used as the measurements. The indicators are defined by the following (j)-(m):

[0097]

[0098]

[0099]

[0100]

[0101] Wherein, TP, TN, FP and FN represent the number of true positives, true negatives, false positives and false negatives defined at the face image pair level. If the prediction about the image pair combination belongs to the same person is correct, it is considered as true positive, otherwise it is considered as false negative; N represents the number of person categories in the data set; M i represents the number of images contained in the i-th person category; is a symbolic function.

[0102] If the prediction about the image pair combination does not belong to the same person is correct, it is true negative, otherwise it is false positive. TAR@FAR measures the correct acceptance rate under the premise of a lower false acceptance rate (all errors are single digits). Id measures the matching accuracy of M i images of N persons in a million irrelevant candidate sets containing one image of the same person, is a symbolic function, which is 1 when the jth image of the i-th person is correctly judged, otherwise it is 0.

[0103] The method of the present application designs a framework based on sample-level contrast loss, introduces random dropout into the classification method, thereby generating a semantic invariant image feature copy, and then sample-level contrast learning can be used. Meanwhile, the framework only uses random dropout, and the additional computational burden generated can be ignored. The designed contrast loss not only increases the distance between different image features, but also avoids the interference of the supervision signal with the classification loss. This distinction in high-dimensional space helps the model to achieve better performance in the identification task. Finally, an accurate face recognition effect is realized.

[0104] The above merely describes the preferred embodiments of the present application, and is not intended to limit the implementation and protection scope of the present application. It should be realized by those skilled in the art that any equivalent replacement and obvious changes made according to the content of the present application should be included in the protection scope of the present application.

Claims

1. A face recognition method based on sample-level contrastive learning, characterized in that, The method adopts a training framework combining sample-level contrast loss and classification loss, uses a feature extraction network in the combined framework to perform classification loss and contrast loss, and is trained to convergence, and the specific face recognition includes the following steps: Step S1: obtaining a face data set Download the public face data set, and avoid the intersection of image characters in the training set and the test set to realize the open data set test; Step S2: processing of the data set Face detection and positioning are performed on each picture of the training set and the test set, the face image part is cropped, and then standardization processing is performed; Step S3: design of the training framework A face recognition model training framework combining sample-level contrast loss and classification loss is designed, and random dropout is used for enhancement on the features; Step S4: design of the contrast loss function Design a sample-level contrastive learning loss function for auxiliary classification loss Make the contrastive loss promote the generation of greater distinction between images of different characters in the training process, and avoid conflict with the supervised classification loss; Step S5: training of the combined framework ResNet-100 is used as the feature extraction network and ArcFace is used as the classification loss function and the sample-level contrastive learning loss function designed in step S4 The training framework is trained to convergence to achieve accurate face recognition effect. In the initial stage of training, single image features are used for classification learning, and the features are randomly discarded. In the middle and later stages of training, two random discards are used, the sample-level contrastive learning loss function designed in step S4 is added, and two image features are used for classification learning. The contrast loss function of the step S4 is defined by the following (b) formula: （b）； wherein is a similarity measure function, is a scale parameter that adjusts the output distribution of the Softmax function, is an exponential function with base ; and is a decision boundary; is a similarity measure function defined by the following (c) formula: （c）； The discrimination boundary An exponential moving average is defined using the following equations (d) to (f): （d）； （e）； （f）； wherein, represents a label of the i-th sample within the batch; represents the i-th iteration; represents a control parameter of the moving average; represents the maximum intra-batch similarity of non-same images of the target copy.​​ In the contrast loss, the feature copy interrelation calculation between the images of the same category is eliminated, and only the relationship calculation between the feature copies of the same image and the feature copies of different category images is used. At this time, the above (b) can be updated to the following (g) formula: (g).

2. The face recognition method based on sample-level contrastive learning according to claim 1, characterized in that, The data set processing uses MTCNN to perform face key point detection and positioning, the face positions of each image are cropped according to the key points, and each RGB pixel is subtracted by 127.5 and then divided by 128 for normalization processing.

3. The face recognition method based on sample-level contrastive learning according to claim 1, characterized in that, The enhancement on the random discard in the step S3 is twice random discard operation on the result of the trunk network used for extracting the feature The result of the random discard Using a projection layer Projecting, for the i-th image in a batch B, to get two distinguished but semantically identical feature vector copies respectively denoted as and Using a classification loss module Constraining, and using a contrastive loss module represented by the following (a) formula on the combination Constraining: (a);​ wherein, is the average loss of the framework on a batch; is the weighting of the contrastive loss; is the contrastive loss, during the mini-batch training process, the copies generated by each image can constitute a copy pool, and a pre-designed contrastive loss is constrained is the size of the batch; i is the traversal of .

4. The face recognition method based on sample-level contrastive learning according to claim 1, characterized in that, The step S5 uses ResNet-100 as a feature extraction network, the main stem of which is completely dependent on a convolution network with a window size of 3x3, and the ArcFace as a classification loss function is defined by the following (h) formula during the training process: （h）； in, It is the first weight of the classification layer. Column can represent the first Personal face categories; It is an image feature copy and Angle in higher-dimensional space; Image feature copy represented by the following formula (i) and Cosine similarity: (i).

Images