A face recognition method based on quality perception
By using a quality-aware face recognition training framework that combines classification and contrastive loss functions, the problem of low recognition accuracy for images of different quality is solved, achieving efficient recognition on images of varying quality and improving the accuracy and adaptability of practical applications.
Patent Information
- Application Number
- CN202310786828.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-30
- Publication Date
- 2026-01-02
- Estimated Expiration
- 2043-06-30
AI Technical Summary
Existing technologies have low accuracy in recognizing face images of varying quality, making it difficult to adapt to the problem of inconsistent image quality in real-world application scenarios.
A quality-aware face recognition training framework is adopted, which combines classification loss function and contrast loss function. The image quality is estimated during training through the image quality awareness module. Different learning methods are used to process high-quality and low-quality images respectively. A data augmentation module and a feature extraction network are constructed to realize an end-to-end network architecture.
It improves the accuracy and adaptability of the model in mixed-quality face image recognition tasks, and enhances the recognition performance in practical application scenarios, especially the recognition performance of low-quality images.
Smart Images
Figure CN116704585B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of face recognition, and in particular to a face recognition method based on quality perception. BACKGROUND
[0002] Face information is biological characteristic information with unchangeability and uniqueness. With the rapid development of artificial intelligence technology and the improvement of economic development level, under the leadership of big data application, face recognition has been widely applied in smart cities, security markets, online payments and other fields, and has become the most widely used biological recognition technology. The existing method has achieved high accuracy on some high-quality image datasets. However, in actual application, due to the influence of factors such as equipment, weather, and light, the quality of face images varies greatly, and the existing method cannot take into account the characteristics of face images of different qualities, so the actual recognition accuracy is greatly reduced.
[0003] After decades of efforts by researchers, the performance of the existing method for high-quality face image recognition has been relatively satisfactory, but when the related method is applied to low-quality images, the accuracy is significantly reduced. To solve the problem of low-quality face recognition, researchers have proposed quality-independent methods, trying to use pre-trained high-quality models to learn low-quality images. However, due to the loss of some information with good distinguishability, the recognition effect on high-quality data is reduced. Some other methods reduce the weight of low-quality images during training to reduce the impact of low-quality images on model training, but this goes against the goal of learning low-quality images. Another type of method uses an additional super-resolution module to reconstruct low-quality images, and then performs face recognition on the reconstructed images to improve accuracy. However, this method requires additional training of the super-resolution module, cannot achieve an end-to-end network architecture, and has high training and testing overhead. In addition, the training of the super-resolution module still requires a trained feature extraction module to achieve the reconstruction of high-quality images.
[0004] The existing face recognition is learned for single-quality face image (such as high-quality or low-quality) recognition task, and the recognition effect of the trained model on different quality images is significantly reduced. It is better to solve the problem that in actual application scenarios (such as video monitoring), the quality of the face images taken is uneven due to factors such as distance, light, and occlusion, and it is difficult to simultaneously adjust the corresponding images of different qualities, greatly reducing the applicability of face recognition methods in different application scenarios, and the low accuracy of face recognition, which has wide application prospect and commercial value. SUMMARY
[0005] The present application aims at the deficiencies of the prior art and provides a quality perception-based face recognition method. The quality perception-based face recognition training framework is constructed by using a face image quality perception and classification method. The training framework combines a classification loss function for high-quality face image learning and a contrast loss function for low-quality face image learning. The face image quality is estimated in advance in the model training process. Different training methods are used for images of different qualities. The finally obtained model can be suitable for feature extraction of face images of different qualities. The effect is better than that of the existing face recognition algorithm. The data enhancement module, the feature extraction network, the quality perception module, the classification loss, and the contrast loss in the combined framework are trained to convergence. The method is used in the face recognition task of the mixed quality dataset. The face image quality is measured. Different learning methods are used for high-quality and low-quality images. It is helpful to improve the performance of the model in the mixed quality face image recognition task. It can be applied to the recognition of faces of different image qualities. The adaptability and accuracy of the face recognition method in the actual application scene are improved.
[0006] The specific technical solution of the present application is: a quality perception-based face recognition method, characterized in that, in view of the characteristics of face images of different qualities, a quality perception-based face recognition training framework and a classification loss function for high-quality face image learning are designed by using a face image quality perception and classification method. Different image qualities are used for face recognition. The specific steps include the following steps:
[0007] Step S1: acquisition of the dataset
[0008] The public face dataset is downloaded, including high-quality and low-quality face images. The images of the training set and the test set are avoided to cross. The open dataset test is realized.
[0009] Step S2: processing of the dataset
[0010] Face detection and positioning are performed on each picture. The face image part is cropped, and then standardized. The training set and the test set are processed according to the same process to ensure consistency.
[0011] Step S3: construction of the face recognition model
[0012] A quality perception-based face recognition model is designed. The model combines a classification learning and a contrast learning framework, including: a data enhancement module Aug, a trunk network F for extracting features, an image quality perception module z, a classification loss module L using features for classification, and a contrast loss L for contrast learning. class contra For better learning of different quality face images, the image quality perception module z is used to estimate the quality of input images in each batch learning process, and the input images are divided into high-quality and low-quality images. For high-quality images, the classification loss L contra is used for learning; for low-quality images, the contrast loss L class is used for learning, and the model is constrained by using two losses.
[0013] Step S4: design of the image quality perception module z
[0014] For the ith input image, the feature vector h i is calculated through the feature extraction network, and the modulus z i ||h i || of the feature vector is used as an image quality indicator, and all images in a batch are divided into high-quality and low-quality images according to the value of z i .
[0015] Step S5: learning of the classification loss function
[0016] The classification loss function L contra is used for learning for high-quality images.
[0017]
[0018] wherein cos (θ j ) is the cosine similarity of two feature vectors,
[0019] 3 j is the jth column of the classification layer weight, which can represent the jth face class; θ j is the angle between the feature vector h and 3 j in the high-dimensional space; s is a scale parameter for adjusting the output distribution of the Softmax function; exp is an exponential function with base e; and B is the number of samples in a training batch.
[0020] Step S6: learning of the contrast loss function
[0021] For low-quality images, the contrast loss function is used for learning, and the formula is as follows:
[0022]
[0023] wherein cosh1,h2) is the cosine similarity of two feature vectors, h G and h k are the feature vectors of the input image and the image after enhancement, respectively, and y Gis the label of the image; s is the scale parameter to adjust the output distribution of the Softmax function; exp is the exponential function with base e; Q is the number of sample pairs.
[0024] The contrast loss promotes the greater distinction of images of different persons in the training process, while requiring the robust recognition of the same image of the same person, which brings better generalization to face recognition, and especially better feature learning effect for low-quality images that are difficult to identify.
[0025] Step S7: training of the face recognition model
[0026] The training framework is combined, including the feature extraction trunk network, the quality perception module, the classification loss and the contrast loss, and the face recognition model is trained to convergence.
[0027] Step S8: designing appropriate model evaluation indexes to test the prediction effect of the face recognition model.
[0028] Compared with the prior art, the present application has the advantages that by measuring the quality of the input image, different learning methods are used for high-quality and low-quality images respectively, targeted learning is realized, the model can process face recognition problems of different qualities at the same time, and finally the face recognition accuracy of mixed quality is improved. The contrast loss promotes the greater distinction of images of different persons in the training process, while requiring the robust recognition of the same image of the same person, which brings better generalization to face recognition, and especially better feature learning effect for low-quality images that are difficult to identify. BRIEF DESCRIPTION OF DRAWINGS
[0029] Fig. 1 It is a face image of different quality in Example 1;
[0030] Fig. 2 It is a schematic diagram of the training framework of the face recognition model;
[0031] Fig. 3 It is a schematic diagram of the trunk network ResNet network structure. DETAILED DESCRIPTION
[0032] The present application will be further described below in combination with the drawings and specific embodiments, but not as a limitation of the present application. Example 1
[0033] Referring to Figs. 1-3 , the following steps are performed for the quality perception-based face recognition method:
[0034] Step S1: acquisition of the data set
[0035] Referring to Fig. 1, download public face dataset, collect different quality face images belonging to different people, specifically including: VGGFace, LFW, AgeDB, CFP-FP, CPLFW, CALFW, SCface, Tinyface, IJB-B. Among them, VGGFace is used as the training set, and the other data sets are used as the test set. For the task that exists in the training set and the test set at the same time, all the pictures belonging to it are deleted from the training set to realize the open data set test.
[0036] Step S2: processing the data set
[0037] Each image is detected and positioned using MTCNN face key points, and then the image is processed using a similarity transformation according to the detected five face key points of both eyes, nose and two corners of the mouth, and then the processed face image is cropped to obtain an image for training. The image needs to be scaled to 112x112 pixels before being input into the model, and each RGB pixel will be subtracted by 127.5 and then divided by 128 for normalization. The training set and the test set are processed in this way.
[0038] Step S3: construction of the face recognition model
[0039] Referring to Fig. 2 , a special quality-aware face recognition training framework is designed, in order to enable the model to better learn different quality face images, the present application designs a model training framework, as shown in Fig. 2 The proposed framework combines general classification learning and contrast learning framework, including data augmentation module Aug, trunk network F for feature extraction, quality-aware module z, classification loss module L class for classification using features, and contrast loss L contta for contrast learning. In order to better learn different quality face images, the present application is based on the image quality perception module, in the learning process of each batch of samples with a size of B, the image quality of each sample x i input is estimated, which is divided into high-quality and low-quality images. For high-quality images, classification loss L contra is used for learning; for low-quality images, contrast loss L class is used for learning, and the model is constrained using two losses.
[0040] Step S4: design of the quality-aware module
[0041] For each sample x i Ci=1,2,…,B) in a batch with a size of B, the feature vector h i is calculated through the feature extraction network, and here the norm z i =||hi || as a measure of image quality. For all samples {x i |i = 1, 2, …, B}, the average value μ i (i = 1, 2, …, B) of all z z and the standard deviation σ z .
[0042] z i is normalized by the following (c) formula:
[0043]
[0044] where c is a normalization coefficient, so that all values are located in the interval (-1, 1).
[0045] According to the value of , all images input in a batch are divided into high-quality and low-quality images, and learning of classification loss and contrast loss is performed using the following (d) ~ (f), respectively, to calculate the classification loss contrast loss and the overall loss
[0046]
[0047]
[0048]
[0049] where c is a threshold value for distinguishing high-quality and low-quality images, and is taken as 0.2 in this embodiment.
[0050] Step S5: Learning of classification loss function
[0051] For high-quality images, learning is performed using the following (a) formula classification loss function;
[0052]
[0053] where cos(θ j ) is the cosine similarity of two feature vectors,
[0054] 3 j is the jth column of the classification layer weight, which can represent the jth face class; θ j is the angle between the image feature copy h and 3 j in the high-dimensional space; s is a scale parameter for adjusting the output distribution of the Softmax function; exp is the exponential function with base e; and B is the number of samples in a training batch.
[0055] Step S6: Learning of contrastive loss function
[0056] For low-quality images, learning is performed using the following (b) formula for the contrastive loss function:
[0057]
[0058] wherein cosh1, h2) is the cosine similarity of two feature vectors,
[0059] h G , h k are the feature vectors of the input image and the image after enhancement, respectively, y G is the label of the image; s is a scale parameter for adjusting the output distribution of the Softmax function; exp is the exponential function with base e; and Q is the number of sample pairs.
[0060] The contrastive loss promotes greater differentiation between images of different persons during the training process, while requiring the same image of the same person to have robust recognition. This brings better generalization to face recognition, and better feature learning effect can be achieved for low-quality images that are difficult to recognize.
[0061] Step S7: Combining the training framework, including the feature extraction trunk network, the quality perception module, the classification loss, and the contrastive loss, to train the face recognition model to convergence.
[0062] Referring to Fig. 3The feature extraction network model used in the present application is ResNet-100, which is different from the commonly used ResNet network, and there is no Conv convolution network with a window size of 1*1 in the backbone network. ResNet includes 4 stage modules, each stage module includes a plurality of sub-modules, and each sub-module uses a Conv convolutional neural network, a Batch Normalization batch normalization, and a PReLU activation function. When ResNet was proposed, due to the limitation of computing power, a convolutional layer with a window size of 1*1, a step of 1, and a channel number transformation was used to replace a convolutional layer with a window size of 3*3, a step of 2, and a constant channel number, and the present application uses a complete 3*3 Conv convolutional layer, and the use sequence is Batch Normalization, Conv, Batch Normalization, PReLU, Conv, and Batch Normalization. The input of each module will be directly added to the aforementioned network as the final sub-module output. For the first sub-module of each stage, due to the existence of channel number transformation, a convolutional layer with a window size of 1*1 and a step of 2 is used to process the sub-module input, and then added to the output of the aforementioned module. The number of sub-modules used in each sub-stage of the used ResNet-100 is 3, 13, 30, and 3 respectively, and the output dimensions of each sub-stage are 256, 512, 1024, and 2048 respectively. The ResNet model structure is shown in Fig. 2
[0063] The present application uses data enhancement to generate a positive sample pair in contrast learning. Specifically, for a sample x G , a new sample x k is obtained through data enhancement, and is input into the trunk network to extract features together to form a positive sample pair (h G , h k ). In this embodiment, the data enhancement method used includes image rotation, random scaling, and cutting.
[0064] In this embodiment, the SGD optimizer with a momentum of 0.9 is used, the learning rate is set to 0.1, the current learning rate is changed to 1 / 10 of the original at the 8th, 14th, and 20th rounds, the model is trained at the 24th round, the weight decay is set to 0.0005, the total batch size is 512, the scale parameter s of the classification loss and the contrast loss is 64.
[0065] The present embodiment uses 4 blocks with NVIDIA A100 Tensor Core GPU for distributed training, the operating system is 64-bit CentOS 7.4, uses Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz, and the memory size is 384GB. The required software tools include Python 3.8, CUDA 10.3, cuDNN 8.0 and PyTorch 1.10.0. Other alternative software tools and hardware devices can also be used to implement the techniques and steps described in the method of the present application. The software and hardware environment involved in the present embodiment is not a limitation of the present application.
[0066] Step S8: Design appropriate model evaluation indicators and test the prediction effect of the model.
[0067] The evaluation criteria of the present application include face verification accuracy (Ver), correct acceptance rate (TAR) under a certain false acceptance rate (FPR) and face retrieval accuracy (Id). The present application first uses training data for training, and then uses data that does not overlap with the training data to test the model. The classification layer and random dropout are not used in the test stage. For LFW, AgeDB, CFP-FP, CALFW, CPLFW and other data sets, the face verification accuracy is used for measurement; for IJB-B and IJB-C data sets containing about ten thousand pairs of face verification data, the face detection of TAR@FAR=0.0001 is used for measurement; for MegaFace containing one million face images, the face detection of TAR@FAR=0.000001 and the face retrieval accuracy are used for measurement. The definitions of various indicators are shown in (g)-(j) as follows:
[0068]
[0069]
[0070]
[0071]
[0072] Wherein, TP, TN, FP and FN represent the number of true positives, true negatives, false positives and false negatives defined at the level of face image pairs. If the prediction about the image pair combination belongs to the same person is correct, it is considered as true positive, otherwise it is considered as false negative. If the prediction about the image pair combination does not belong to the same person is correct, it is true negative, otherwise it is false positive. TAR@FAR measures the correct acceptance rate under the premise of a lower false acceptance rate (all errors are single digits). Id measures the matching accuracy of M pictures of x individuals in a million irrelevant candidate sets containing one same person image, is a symbolic function, when the jth picture of the ith person is correct, it is 1, otherwise it is 0.
[0073] The application designs a face recognition framework based on quality perception, measures the quality of input images, adopts different learning methods for high-quality and low-quality images respectively, realizes targeted learning, can make the model process different quality face recognition problems at the same time, and finally improves the face recognition accuracy of mixed quality. The above only describes the preferred embodiments of the application, and does not limit the implementation and protection scope of the application. For those skilled in the art, it should be realized that any equivalent replacement and obvious changes made by applying the content of the specification and drawings should be included in the protection scope of the application.
Claims
1. A face recognition method based on quality perception, characterized in that, The facial recognition method specifically includes the following steps: Step S1: Data Collection Download publicly available high-quality and low-quality face images as training and testing sets, remove overlapping images from the training and testing sets, and implement open dataset testing. Step S2: Dataset Processing Face detection and localization are performed on each image in the training and test sets, and the face images are cropped and then standardized. Step S3: Construction of the face recognition model The system includes: a data augmentation module Aug, an image quality perception module z, and a classification loss function. and contrast loss function The face recognition model framework, wherein the data augmentation module Aug is a feature extraction trunk network. The image quality perception module z measures a size of... During the learning process of each batch of samples, the input sample is estimated. The image quality is analyzed and categorized into high-quality and low-quality images. Step S4: Acquisition of high-quality and low-quality images For the input image, a torso network is used to extract features. Calculate its feature vector, use the magnitude of the feature vector as an image quality indicator, and classify all images in a batch of input images into high-quality and low-quality images; Step S5: Learning the classification loss function For high-quality images, the classification loss function is used as shown in equation (a). To learn; （a）； in, ; The weight of the classification layer Column, representing the first Personal face categories; For image feature copy and Angle in higher-dimensional space; To adjust the scaling parameter of the output distribution of the Softmax function; It is a constant The base is an exponential function; B is the number of samples in the training batch; Step S6: Learning the contrast loss function For low-quality images, the contrast loss function in equation (b) below is used. To learn: （b）； in, The cosine similarity between two feature vectors. ; These are the feature vectors of the input image and the enhanced image, respectively. Labels for images; To adjust the scaling parameter of the output distribution of the Softmax function; It is a constant An exponential function with base π; Q is the number of sample pairs; Step S7: Training the face recognition model The combination includes: a feature extraction trunk network. Image quality perception module z, classification loss function and contrast loss function The training framework is used to train the face recognition model until it converges; The trunk network For ResNet-100; the image quality sensing module z processes each sample in a batch of size B. The feature vector is calculated through a feature extraction network. And with the modulus of the feature vector As a measure of image quality, for all samples Calculate all average and standard deviation and will Standardization is performed using the following formula (c): （c）； in, It is a normalization coefficient that makes all The value is located in the interval .
2. The face recognition method based on quality perception according to claim 1, characterized in that, The classification loss function and contrast loss function according to The values are used to classify all input images in a batch into high-quality and low-quality images, and the classification loss is calculated using equations (d) to (f) below. Comparative loss and total loss : （d）； （e）； （f）； in, A threshold for distinguishing between high-quality and low-quality images.
3. The face recognition method based on quality perception according to claim 1, characterized in that, The trunk network The ResNet-100 consists of four stage modules, each containing several sub-modules. Each sub-module uses a 3×Conv convolutional neural network, Batch Normalization, and PReLU activation. The ResNet replaces the 3×3 convolutional layers with a stride of 2 and a window size of 2 with a 1×1 window size and a stride of 1, using the Batch Normalization, Conv, Batch Normalization, PReLU, Conv, Batch Normalization sequence. The input to each stage module is directly summed with the aforementioned network to obtain the final sub-module output. For the first sub-module in each stage, a 1×1 window size and a stride of 2 convolutional layer is used to process the sub-module input, which is then added to the output of the aforementioned module. The number of sub-modules used in each ResNet-100 stage is 3, 13, 30, and 3, respectively, and the output dimensions of each stage are 256, 512, 1024, and 2048, respectively.
4. The quality-aware face recognition method according to claim 1, characterized in that, The data augmentation module Aug uses image rotation, random scaling, and cropping methods to perform data augmentation on a sample. A new sample is obtained through data augmentation. , and Inputting the torso network together Extract features to form a positive sample pair .