Method for improving remote face recognition effect

Abstract

The invention provides a method for improving a remote face recognition effect. The method comprises the following steps: S1, screening remote face data collected by a camera and a part of an open source face data set to serve as training data of a generative adversarial network cycled and training; s2, extracting different numbers of pictures according to the face data of each face in the face training data set and the fuzzy degree of the face; s3, performing simulated remote face data generation on the picture extracted in the step S2 by using cycled, and filtering the generated data through a face recognition model; and S4, adding the data in the step S3 into a face recognition training set for training, and realizing the improvement of the recognition of the face recognition model on the remote face data according to a final test result. The method is simple, the long-distance face recognition effect can be greatly improved finally, and the problem that long tails of a data set are distributed unevenly can be solved to a certain extent.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a method for improving long-distance face recognition effect. Background Art

[0002] With the development of science and technology, especially the development of AI technology, the field of image recognition has attracted more and more attention. In the prior art, the implementation scheme closest to the present invention includes, for example, "A method for enhancing long-distance facial images for recognition" with application number 201710757108.9, which uses a clustering algorithm to divide the facial area into blocks, and then uses a nonlinear transformation based on a combination of gamma transformation and logarithmic transformation to restore facial image detail information. File feature extraction is then performed based on the facial image extracted and processed by the convolutional neural network, and then the similarity between the image and the base database is calculated as the recognition result.

[0003] However, the defects of the prior art are:

[0004] 1. The existing method is based on an online approach, which will greatly affect the actual algorithm processing time.

[0005] 2. The training datasets of existing methods lack real-life long-distance face data.

[0006] 3. Existing methods make it difficult to collect long-distance facial images of corresponding people in open source facial datasets.

[0007] In addition, terms commonly used in the prior art include:

[0008] 1. Face detection: The face detection model, given a picture, will return the location of the face frame.

[0009] 2. Face alignment: extract the face part from the original image and scale it to a certain size.

[0010] 3. Training set: data that the network uses to learn.

[0011] 4. Cyclegan network: Generative adversarial network, which can complete the image style transfer without pairing images. In layman's terms, given two kinds of data, it can learn the relationship between the two data, given one kind of data, it can generate another kind of data. (For example, ordinary horse to zebra, face without glasses to face with glasses.)

[0012] 5. Fuzzy model: can determine the degree of blur of an image.

[0013] 6. Gamma transformation: can correct bleached images or images that are too dark.

[0014] 7. Logarithmic transformation: can enhance the details of the dark part of an image.

[0015] 8. Open source face dataset: Celeba face dataset.

[0016] 9. Base database: a collection of face recognition registration images.

[0017] 10. Long-tail distribution: In the training samples, a small part of the head contains more pictures, and the rest of the samples contain fewer pictures. In particular, each person in the face recognition data set has a different number of face pictures, some people have too many pictures, and some people have too few pictures (for example, most people have about 20 pictures, some people have only 2 pictures, and some people have 40 pictures). In simple terms, the number of face pictures for each person is uneven. Summary of the invention

[0018] In order to solve the above problems, the purpose of this application is to:

[0019] 1. By imitating the imaging effect of real long-distance faces, it is possible to generate simulated long-distance face data corresponding to the face image, thereby improving the recognition effect of long-distance faces.

[0020] 2. Reduce the online running time, start with the training data set, and improve the effect of face recognition.

[0021] Specifically, the present invention provides a method for improving long-distance face recognition effect, the method comprising the following steps:

[0022] S1. Filter the data of distant human faces collected by the camera and a part of the open source face dataset to serve as the training data of the generative adversarial network CycleGAN and train it;

[0023] S2. extracting different numbers of images according to the face data of each face in the face training data set and the blur degree of the face;

[0024] S3. Use cyclegan to generate long-distance face data from the image extracted in step S2, and filter the generated data through the face recognition model;

[0025] S4. Add the data in step S3 to the face recognition training set for training, and finally test the result to improve the face recognition model's recognition of long-distance face data.

[0026] The step S1 further comprises:

[0027] S1.1, the preparation of the CycleGAN training data set includes:

[0028] (1) Training data A, which is used to construct a real long-distance face dataset:

[0029] (1).1, collect the data recorded by the camera, and find the position and size of the face frame through face detection, including: obtain the frame and size of the face through the self-trained face detection model, the self-trained face detection model is a face detection model trained by yolov5, the model can output the position of the face frame, the face frame information: the coordinates of the upper left corner and the lower right corner of the face frame, and the size of the face can be obtained through coordinate calculation;

[0030] (1).2, filter out the part of the face frame below 60 by the size of the frame, that is, determine the face at a long distance;

[0031] (2) Training data B, which is used to train the CycleGAN network, allowing the CycleGAN network to learn the changes from close-up faces to long-distance faces:

[0032] (2).1, randomly select pictures of different people from the open source face dataset. The number of pictures selected needs to be equal to the number of long-distance face datasets collected and screened;

[0033] (2).2, the image in (2).1 is filtered out by a fuzzy model to obtain a relatively clear image; the fuzzy model is a fuzzy module in the face quality model, which can obtain the degree of blur of the face. The degree of blur of each face is obtained by the face quality model trained by itself. The degree of blur is a decimal result output by the model, and faces with a degree of blur below 0.4 are filtered out;

[0034] S1.2, training CycleGAN, learning the mapping from training data B to training data A, the purpose is to turn a clear picture into a blurry picture at a long distance under camera imaging; CycleGAN is a neural network, prepares two data sets, and trains the network to learn the mapping relationship between the two data sets.

[0035] The step S2 further comprises: extracting the original image of the CycleGAN model for generating data:

[0036] S2.1, count the number of pictures of each person in the open source face dataset: each person in the face dataset has a folder, the person is represented by an id or name in face recognition, and the folder contains all the pictures of this person. The number of pictures in the folder can be obtained by using the glob package in the python code, that is, len(glob.glob(img_flod)), where img_flod is the folder of each person. In this way, the number of face pictures of each person can be counted by the len() function;

[0037] S2.2, according to the number of pictures of each person in the open source face dataset, choose how many pictures to take for each person, take all the pictures if there are fewer pictures, and take a certain number of pictures if there are more pictures: if a person has less than 10 pictures, take all of them; if the number is more than 10 but less than 25, take 25 minus the current number of pictures; if the number to be taken is greater than the current number of pictures, take all of them; if the number is more than 30, do not take pictures;

[0038] S2.3, using the fuzzy model to judge the quality of the face image of the data selected in step S2.2, and deleting the blurred data. The quality judgment can be achieved by setting a threshold as needed, and the data below the threshold will be deleted.

[0039] The step S3 further comprises: generating simulated long-distance face data through the trained CycleGAN model and filtering:

[0040] S3.1, the images selected in step S2.3 are used to generate long-distance face data through the trained CycleGAN model;

[0041] S3.2, filter the data generated by cyclegan through a face recognition network, wherein the face recognition network uses adaface (res101), a model trained with webface12M; where res101 represents the network structure used to distinguish the models;

[0042] The data of realistic long-distance faces is simulated and generated through the CycleGAN network and filtered through the face recognition model.

[0043] In step S3.2, the filtering operation is achieved by training the face recognition network and evaluating it:

[0044] Through the previously trained face recognition model, the data generated by step S3.1 is filtered through a high threshold to ensure that the retained images are of good quality: the open source face recognition model adaface is used to obtain the similarity between the original face image and the face image generated by cyclegan, which is a decimal of 0-1. When the similarity is greater than 0.4, the image is retained and considered to be of good quality. If it is less than 0.4, it is deleted.

[0045] The step S4 further comprises: adding the data processed in step S3 to the face recognition training data set for training and testing:

[0046] S4.1, add the data filtered in step S3.2 to the open source face dataset to train the model; S4.2, evaluate the model trained in step S4.1 on the open source test set LFW. The test results show that its accuracy is improved, and the accuracy on the actual long-distance face test set is improved.

[0047] Therefore, the advantages of the present application are: the method is simple, and can ultimately greatly improve the effect of long-distance face recognition, and can solve the problem of uneven long-tail distribution of the data set to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0049] Figure 1 is a schematic diagram of the process of this method. DETAILED DESCRIPTION

[0050] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0051] like Figure 1 As shown, the present application proposes a method for improving the effect of long-distance face recognition. The method starts with a training data set and supplements the long-distance face data missing in the training data set; the ultimate goal is to improve the recognition accuracy of the face recognition model for long-distance faces, which specifically includes the following steps:

[0052] Step S1. Screen a portion of the data of the long-distance human face collected by the camera and the open source human face data set as the training data of the generative adversarial network CycleGAN and train it; the portion is equivalent to the number of pictures actually collected, and then synchronized to a level during the subsequent training, with the number difference being no more than 20;

[0053] The step S1 further comprises:

[0054] S1.1, the preparation of the CycleGAN training data set includes:

[0055] (1) Training data A, which is used to construct a real long-distance face dataset:

[0056] (1).1, collect the data recorded by the camera, and find the position and size of the face frame through face detection, including: obtain the frame and size of the face through the self-trained face detection model, the self-trained face detection model is a face detection model trained by yolov5, the model can output the position of the face frame, the face frame information: the upper left corner coordinates and the lower right corner coordinates of the face frame, the size of the face can be obtained by coordinate calculation, the face detection and blur model can refer to Baidu AI address https: / / ai.baidu.com / tech / face / detect;

[0057] (1).2, filter out the face frame with a size below 60 by the frame size, that is, determine the distant faces. The frame size of 60 is selected as the filtering condition because: the face frame size is obtained by step (1).1, and the frame size of the distant faces, that is, the face frame size of the face beyond 7m in the data is basically below 60;

[0058] (2) Training data B, which is used to train the CycleGAN network, allowing the CycleGAN network to learn the changes from close-up faces to long-distance faces:

[0059] (2).1, randomly select pictures of different people from the open source face dataset. The number of pictures selected needs to be equal to the number of long-distance face datasets collected and screened;

[0060] (2).2, filter out relatively clear pictures from the pictures in (2).1 through a blur model; the blur model is a blur module in a face quality model, which can obtain the blur degree of a face. The face quality model is a model trained by itself, for example, it can be named face_blur. The blur degree of each face is obtained through the face quality model trained by itself. The blur degree is a decimal result output by the model, and faces with blur degrees below 0.4 are filtered out (according to experience, faces with blur degrees above 0.4 are relatively clear);

[0061] S1.2, training CycleGAN, learning the mapping from training data B to training data A, the purpose is to turn a clear picture into a long-distance blurry picture under camera imaging;

[0062] CycleGAN is a neural network. Two data sets are prepared and the network is trained to learn the mapping relationship between the two data sets. This is the existing technology and will not be repeated here. You can refer to the article "Introduction and Implementation Effect of CycleGAN" by Qu Hongze.

[0063] Step S2. Extract different numbers of pictures according to the facial data and blur degree of each face in the face training data set; the blur degree can be clear as long as the facial features are visible, and blurry as long as the facial features are not visible, and the output of the blur model in the face attribute model in the face attribute model is 0.4 as the boundary (greater than clear, less than blurry); the different numbers are further explained in the following step S2.2.

[0064] The step S2 further comprises: extracting the original image of the CycleGAN model for generating data:

[0065] S2.1, count the number of pictures of each person in the open source face dataset (because we want to minimize the long-tail distribution of the training set): each person in the face dataset has a folder, and the person is represented by an id or name in face recognition. The folder contains all the pictures of this person. The glob package in the python code can be used to get the number of pictures in the folder, i.e., len(glob.glob(img_flod)), where img_flod is the folder of each person. In this way, the number of face pictures of each person can be counted through the len() function;

[0066] S2.2, according to the number of pictures of each person in the open source face dataset, choose how many pictures to take for each person (take out all the pictures if there are fewer, and take out a certain number of pictures if there are more). If a person has less than 10 pictures, take out all of them. If the number of pictures is greater than 10 and less than 25, take (25-the current number of pictures). If the number of pictures to be taken is greater than the current number of pictures, take out all of them. If the number of pictures is greater than 30, do not take any pictures.

[0067] S2.3, the data selected in step S2.2 is used to judge the quality of the face image through the fuzzy model, and the fuzzy data is deleted. The quality judgment can be achieved by setting a threshold as needed, and the data below the threshold is deleted.

[0068] Step S3. Use CycleGAN to generate long-distance face data from the image extracted in step S2, and filter the generated data through the face recognition model;

[0069] The step S3 further comprises: generating simulated long-distance face data through the trained CycleGAN model and filtering:

[0070] S3.1, the image selected in step S2.3 is used to generate fake long-distance face data through the trained CycleGAN model (because it is a model-generated image, it is fake;

[0071] S3.2, filter the data generated by cyclegan through the face recognition network, the face recognition network uses adaface (res101), a model trained with webface12M, and there is an open source model download on github; where res101 represents the network structure used to distinguish the model;

[0072] The data of realistic long-distance faces is simulated and generated through the CycleGAN network, and then filtered through existing face recognition models (such as AdaFace (RES101)).

[0073] In step S3.2, the filtering operation is achieved by training the face recognition network and evaluating it:

[0074] Through the previously trained face recognition model, the data generated by step S3.1 is filtered through a high threshold to ensure that the retained images are of good quality: the open source face recognition model adaface is used to obtain the similarity between the original face image and the face image generated by cyclegan, which is a decimal of 0-1. When the similarity is greater than 0.4, the image is retained and considered to be of good quality. If it is less than 0.4, it is deleted.

[0075] Since the data generated in step S3 is added to the open source face dataset to train the model, there is a risk: when evaluating the trained model on the open source test set LFW, the accuracy drops from the original 98% to 82%. The reason for this is that the data generated by cyclegan is not stable, and there will be face distortion and huge noise data. Therefore, the method further includes step S4:

[0076] This method is to improve the accuracy of face recognition for long-distance face data from the perspective of the data set. The method of enhancing the data set is used to improve the adaptability of face recognition to long-distance face data. Most open source data sets on the Internet do not have real long-distance faces. Through the method of this application, real long-distance faces are simulated to supplement the data set, thereby improving the accuracy of face recognition (mainly long-distance face recognition is more accurate).

[0077] Step S4. Add the data in step S3 to the face recognition training set for training, and finally test the result to improve the face recognition model's recognition of long-distance face data.

[0078] The step S4 further comprises: adding the data processed in step S3 to the face recognition training data set for training and testing:

[0079] S4.1, add the data filtered in step S3.2 to the open source face dataset to train the model; S4.2, evaluate the model trained in step S4.1 on the open source test set LFW. The test results show that its accuracy is improved (from the original 98% to 99%), and its accuracy on the actual long-distance face test set is improved (from the original 64.7% to 86.3%).

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for improving long-distance face recognition effect, characterized in that: The method comprises the following steps: S1. Filter the data of distant human faces collected by the camera and a part of the open source face dataset to serve as the training data of the generative adversarial network CycleGAN and train it; S2. extracting different numbers of images according to the face data of each face in the face training data set and the blur degree of the face; S3. Use cyclegan to generate long-distance face data from the image extracted in step S2, and filter the generated data through the face recognition model; S4. Add the data in step S3 to the face recognition training set for training, and finally test the result to improve the face recognition model's recognition of long-distance face data.

2. The method for improving long-distance face recognition according to claim 1, characterized in that: The step S1 further comprises: S1.1, the preparation of the CycleGAN training data set includes: (1) Training data A, which is used to construct a real long-distance face dataset: (1).1, collect the data recorded by the camera, and find the position and size of the face frame through face detection, including: obtain the frame and size of the face through the self-trained face detection model, the self-trained face detection model is a face detection model trained by yolov5, the model can output the position of the face frame, the face frame information: the coordinates of the upper left corner and the lower right corner of the face frame, and the size of the face can be obtained through coordinate calculation; (1).2, filter out the part of the face frame below 60 by the size of the frame, that is, determine the face at a long distance; (2) Training data B, which is used to train the CycleGAN network, allowing the CycleGAN network to learn the changes from close-up faces to long-distance faces: (2).1, randomly select pictures of different people from the open source face dataset. The number of pictures selected needs to be equal to the number of long-distance face datasets collected and screened; (2).2, the image in (2).1 is filtered out by a fuzzy model to obtain a relatively clear image; the fuzzy model is a fuzzy module in the face quality model, which can obtain the degree of blur of the face. The degree of blur of each face is obtained by the face quality model trained by itself. The degree of blur is a decimal result output by the model, and faces with a degree of blur below 0.4 are filtered out; S1.2, training CycleGAN, learning the mapping from training data B to training data A, the purpose is to turn a clear picture into a blurry picture at a long distance under camera imaging; CycleGAN is a neural network, prepares two data sets, and trains the network to learn the mapping relationship between the two data sets.

3. The method for improving long-distance face recognition according to claim 1, characterized in that: The step S2 further comprises: extracting the original image used by the cyclegan model to generate data: S2.1, count the number of pictures of each person in the open source face dataset: each person in the face dataset has a folder, the person is represented by an id or name in face recognition, and the folder contains all the pictures of this person. The number of pictures in the folder can be obtained by using the glob package in the python code, that is, len(glob.glob(img_flod)), where img_flod is the folder of each person. In this way, the number of face pictures of each person can be counted by the len() function; S2.2, according to the number of pictures of each person in the open source face dataset, choose how many pictures to take for each person, take all the pictures if there are fewer pictures, and take a certain number of pictures if there are more pictures: if a person has less than 10 pictures, take all of them; if the number is more than 10 but less than 25, take 25 minus the current number of pictures; if the number to be taken is greater than the current number of pictures, take all of them; if the number is more than 30, do not take pictures; S2.3, using the fuzzy model to judge the quality of the face image of the data selected in step S2.2, and deleting the blurred data. The quality judgment can be achieved by setting a threshold as needed, and the data below the threshold will be deleted.

4. The method for improving long-distance face recognition according to claim 1, characterized in that: The step S3 further comprises: generating simulated long-distance face data through the trained CycleGAN model and filtering: S3.1, the images selected in step S2.3 are used to generate long-distance face data through the trained CycleGAN model; S3.2, filter the data generated by cyclegan through a face recognition network, wherein the face recognition network uses adaface (res101), a model trained with webface12M; res101 indicates the network structure used to distinguish the models; The data of realistic long-distance faces is simulated and generated through the CycleGAN network, and then filtered through the face recognition model.

5. The method for improving long-distance face recognition effect according to claim 4, characterized in that: In step S3.2, the filtering operation is achieved by training the face recognition network and evaluating it: Through the previously trained face recognition model, the data generated by step S3.1 is filtered through a high threshold to ensure that the retained images are of good quality: the open source face recognition model adaface is used to obtain the similarity between the original face image and the face image generated by cyclegan, which is a decimal of 0-1. When the similarity is greater than 0.4, the image is retained and considered to be of good quality. If it is less than 0.4, it is deleted.

6. The method for improving long-distance face recognition according to claim 1, characterized in that: The step S4 further comprises: adding the data processed in step S3 to the face recognition training data set for training and testing: S4.1, adding the data filtered in step S3.2 to the open source face dataset to train the model; S4.2, evaluate the model trained in step S4.1 on the open source test set LFW. The test results show that its accuracy is improved, and the accuracy on the actual long-distance face test set is improved.

Images