A face data cleaning method based on difficult sample mining

Abstract

The present invention discloses a facial data cleaning method based on difficult sample mining, comprising the following steps: S1 obtaining a list of facial images in the same folder, S2 extracting facial image features using a trained facial recognition model, S3 obtaining simple samples, S4 obtaining difficult samples, and S5 directly deleting the difficult sample images, retaining the simple sample images, merging the images identified as "simple samples" in S3, and outputting the final cleaned facial image data. The present invention can accurately detect dirty data in facial images, that is, it can better handle problems such as photos of different individuals and low-quality facial images (large-angle postures, large-scale occlusion, and large-area blur) in facial data.

Description

Technical Field

[0001] The present invention relates to a face data cleaning method, and in particular to a face data cleaning method based on difficult sample mining. Background Art

[0002] With the development of deep learning technology in recent years, facial recognition accuracy has greatly improved and has been successfully applied in various scenarios, including supermarket payment, park check-in, and airport security. Optimizing facial recognition models requires a large amount of training data. Currently, most open-source facial training data is collected from the internet, which contains a large amount of noisy data. For example, facial photos with the same ID may show different people, or may contain large angles, pose variations, large occlusions, or significant blurring. Therefore, it is essential to clean facial data before use.

[0003] The patent "Facial Data Cleansing Method and Apparatus" (application number CN201810930672.0) discloses a facial data cleaning method: first facial data to be processed is obtained through a network search or pre-stored images, and the first facial features of the target person are obtained based on the first facial data to be processed. Then, based on the first facial features, the first facial features that do not need to be cleaned are screened out, and the first facial data that does not need to be cleaned, the second facial data that needs to be deleted, and the third facial data that needs to be cleaned are screened out from each facial data. Finally, the first facial data and the fourth facial data are determined as the first target facial data.

[0004] It has obvious disadvantages: first, the quality of the facial data to be processed almost determines the success or failure of the subsequent algorithm strategy; first, the acquisition of facial data to be processed also increases the workload of the algorithm.

[0005] The noise in facial data mainly manifests itself in the following ways: facial photos with the same ID have large-angle posture changes, large-scale occlusion, and large degrees of blur; photos with the same ID are mixed with different facial images. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a face data cleaning method based on difficult sample mining, which determines difficult samples of face images through similarity distribution statistics, improves the face data cleaning effect, and solves the problems existing in the background technology.

[0007] The face data cleaning method based on difficult sample mining of the present invention is implemented by the following technical solution, including the following steps:

[0008] S1. Get a list of face images in the same folder;

[0009] S2. Use the trained face recognition model to extract facial image features and calculate cosine similarity between each pair; then sort the similarities in descending order;

[0010] S3. Obtain simple samples; define images with a similarity greater than 0.5 and a feature norm greater than 10 as "simple samples" and save the image names in the name_list_simple list;

[0011] S4. Obtain difficult samples; define images with a similarity lower than 0.2 or a feature norm lower than 10 as "suspected difficult samples" and save the image names and occurrence frequencies into the name_list_min list; count the difficult samples based on name_list_min;

[0012] S5. Delete the difficult sample images directly, retain the simple sample images, merge the images that have been determined as "simple samples" in S3, and output the final cleaned face image data.

[0013] As a preferred technical solution, in S1, when training the face recognition model, the training data is generally divided into different folders according to the personnel category, that is, the face images in the same folder belong to the same person ID; first, a list of files in the same directory is obtained for subsequent processing.

[0014] As a preferred technical solution, in S3, high-quality training data can provide more discriminative information in face recognition model training, which is more conducive to model training and accelerates model convergence. Simple sample mining is divided into the following steps:

[0015] S31. Input the image list in the same folder;

[0016] S32. Use the pre-trained model to extract facial image feature values ​​and calculate cosine similarity between two vectors. Cosine similarity measures the similarity between two vectors by measuring the cosine value of the angle between them. Given two vectors A and B, their cosine similarity is calculated as:

[0017]

[0018] The range of cosine similarity cos(θ) is between 0 and 1. If it is closer to 1, the similarity between the two vectors is higher; if it is closer to 0, the similarity between the two vectors is lower.

[0019] S33. Arrange the cosine similarities in descending order;

[0020] S34. According to the cosine similarity of the image pairs and the size of the image feature norm, “simple samples”, “intermediate class samples”, “suspected difficult samples” and “difficult samples” are screened.

[0021] As a preferred technical solution, in S4, the "suspected difficult samples" obtained in S3 are input. This batch of data may contain various types of data, so it is necessary to further mine it, which specifically includes the following steps:

[0022] S41. Use the suspected difficult sample obtained in S3 as the input of this step.

[0023] S42. The algorithm execution process is as follows: if the suspected difficult sample is a difficult sample, then theoretically the cosine similarity between the image and any other image is relatively low. Therefore, the frequency of difficult samples appearing in name_list_min will be relatively high. Based on this argument, the name_list_min list is traversed. If the highest frequency minus the second highest frequency is greater than the threshold thres_3 (set to 5 here), then the image corresponding to the highest frequency can be considered as a difficult sample and can be deleted directly.

[0024] As a preferred technical solution, in S5, the samples retained after S2 and S3 are executed are merged and output as a cleaned face training image.

[0025] The beneficial effects of the present invention are:

[0026] 1. The present invention can accurately detect dirty data in facial images, that is, it can better handle problems such as photos of different people in facial data, low-quality facial images (large-angle posture, large-range occlusion, large-area blur), etc.

[0027] 2. The present invention does not require additional facial data support and can complete facial data cleaning using existing samples and trained models.

[0028] 3. The present invention uses the cleaned data to train the face recognition model, effectively improving the model recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 It is the overall flow chart of the algorithm of the present invention;

[0031] Figure 2 This is a simple sample mining flow chart of the present invention;

[0032] Figure 3This is a flowchart of the difficult sample mining process of the present invention;

[0033] Figure 4 This is an example diagram of a face image pair according to the present invention. DETAILED DESCRIPTION

[0034] All features disclosed in this specification, or all steps in the disclosed methods or processes, except mutually exclusive features and / or steps, can be combined in any manner.

[0035] like Figure 1 As shown, a face data cleaning method based on difficult sample mining of the present invention includes the following steps:

[0036] S1. Get a list of face images in the same folder;

[0037] S2. Use the trained face recognition model to extract facial image features and calculate cosine similarity between each pair; then sort the similarities in descending order;

[0038] S3. Obtain simple samples; define images with a similarity greater than 0.5 and a feature norm greater than 10 as "simple samples" and save the image names in the name_list_simple list;

[0039] S4. Obtain difficult samples; define images with a similarity lower than 0.2 or a feature norm lower than 10 as "suspected difficult samples" and save the image names and occurrence frequencies into the name_list_min list; count the difficult samples based on name_list_min;

[0040] S5. Delete the difficult sample images directly, retain the simple sample images, merge the images that have been determined as "simple samples" in S3, and output the final cleaned face image data.

[0041] In this embodiment, in S1, when training the face recognition model, the training data is generally divided into different folders according to the personnel category, that is, the face images in the same folder belong to the same person ID; first, a list of files in the same directory is obtained for subsequent processing.

[0042] In this embodiment, in S3, in the face recognition model training, high-quality training data can provide more discriminant information, which is more conducive to model training and accelerates the model convergence speed; Figure 2 As shown, simple sample mining is divided into the following steps:

[0043] S31. Input the image list in the same folder;

[0044] S32. Use the pre-trained model to extract facial image feature values ​​and calculate cosine similarity between two vectors. Cosine similarity measures the similarity between two vectors by measuring the cosine value of the angle between them. Given two vectors A and B, their cosine similarity is calculated as:

[0045]

[0046] The range of cosine similarity cos(θ) is between 0 and 1. If it is closer to 1, the similarity between the two vectors is higher; if it is closer to 0, the similarity between the two vectors is lower.

[0047] S33. Arrange the cosine similarities in descending order;

[0048] S34. According to the cosine similarity of the image pairs and the size of the image feature norm, “simple samples”, “intermediate class samples”, “suspected difficult samples” and “difficult samples” are screened.

[0049] First, let’s explain the terms involved in this step:

[0050] (1) Norm

[0051] The norm is used to measure the length or size of each vector in a vector or matrix space. In face recognition, the L2 norm, which measures the Euclidean distance, is commonly used. The formula is as follows:

[0052]

[0053] (2) Image sample classification

[0054] In this step, facial image data is divided into several categories according to the difficulty of recognition: "easy samples", "intermediate samples", "suspected difficult samples", and "difficult samples". The following describes them respectively:

[0055] "Simple sample": The facial features are clear, the facial features are distinct, and there are no large-scale occlusions, large-angle postures, large-area blurs, etc. Figure 4 As shown in (a).

[0056] "Intermediate samples": facial features are relatively clear, facial features are relatively obvious, and there are no problems such as large-scale occlusion, large-angle posture, and large-area blur.

[0057] "Suspected difficult samples": The algorithm determines that the image is difficult to recognize. The image generally has problems such as occlusion, posture, blur, etc. Figure 4 (b), (c), and (d) respectively demonstrate the problems of large-scale occlusion, large-angle posture, and large-area blur.

[0058] “Difficult samples”: images that are judged by the algorithm to have serious occlusion, posture, and blur issues.

[0059] Next, the simple sample mining process is described:

[0060] (1) If the cosine similarity between the image pairs is greater than 0.5 and the image feature norm is greater than 10, the image names involved are saved in the name_list_max list. The frequency of the image names in the name_list_max list is counted. If the frequency of an image name is greater than 1, the image is output as a simple sample and its name is recorded in the name_list_simple list;

[0061] (2) If the cosine similarity between the image pairs is less than 0.5 and greater than 0.2, or if the cosine similarity between the image pairs is greater than 0.5 but the image feature norm is less than or equal to 10, then the sample is output as a suspected difficult sample;

[0062] (3) If the cosine similarity between the image pairs is greater than 0.5, and the image feature norm is greater than 10, but the frequency of occurrence is less than or equal to 1, then the sample is considered a difficult sample;

[0063] (4) If the cosine similarity between the image pairs is less than 0.5 and greater than 0.2, the output is an intermediate class sample.

[0064] In this embodiment, in S4, the “suspected difficult samples” obtained in S3 are input. This batch of data may contain various types of data, so further mining is required, such as Figure 4 As shown in Figure 2, difficult sample mining specifically includes the following steps:

[0065] S41. Use the suspected difficult sample obtained in S3 as the input of this step;

[0066] S42. The algorithm executes as follows: If a suspected difficult sample is a difficult sample, then theoretically the cosine similarity between this image and any other image is relatively low. Therefore, the frequency of difficult samples appearing in name_list_min will be relatively high. Based on this argument, the name_list_min list is traversed. If the highest frequency minus the second highest frequency is greater than the threshold thres_3 (here set to 5), then the image corresponding to the highest frequency is considered a difficult sample and can be directly deleted.

[0067] For example, name_list_min = {'2_1234':12,'2_1235':4,'2_1236':1}, where the dictionary key represents the image name and the value represents the frequency of occurrence.

[0068] '2_1235' is 8 higher, so we can determine that '2_1234' is a difficult sample;

[0069] For images that cannot be determined as difficult samples, such as '2_1235' and '2_1236' in the above example, the method to further determine whether they are difficult samples is to check whether the image name appears in the simple sample list name_list_simple. If it appears there, it is considered a simple sample. If it does not appear there, it is considered a difficult sample and is directly deleted.

[0070] In this embodiment, in S5, the samples retained after S2 and S3 are executed are merged and output as a cleaned face training image.

[0071] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that are not conceived through creative work should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection defined in the claims.

Claims

1. A face data cleaning method based on difficult sample mining, characterized in that: The following steps are involved: S1. Get a list of face images in the same folder; S2. Use the trained face recognition model to extract facial image features and calculate cosine similarity between each pair; then sort the similarities in descending order; S3. Get simple samples; define images with a similarity greater than 0.5 and a feature norm greater than 10 as "simple samples" and save the image names in the name_list_sample list; S4. Obtain difficult samples; define images with a similarity lower than 0.2 or a feature norm lower than 10 as "suspected difficult samples" and save the image names and occurrence frequencies in the name_list_min list; S5. Count difficult samples based on name_list_min; S6. Delete the difficult sample images directly, retain the simple sample images, merge the images identified as "simple samples" in S3, and output the final cleaned face image data; In S1, when training the face recognition model, the training data is divided into different folders according to the person category, that is, the face images in the same folder belong to the same person ID; first, a list of files in the same directory is obtained for subsequent processing; In S3, high-quality training data can provide more discriminative information during face recognition model training, which is more conducive to model training and accelerates model convergence. Simple sample mining is divided into the following steps: S31. Input the image list in the same folder; S32. Use the pre-trained model to extract facial image feature values ​​and calculate cosine similarity between two vectors. Cosine similarity measures the similarity between two vectors by measuring the cosine value of the angle between them. Given two vectors A and B, their cosine similarity is calculated as follows: Cosine similarity The range is between 0 and 1. The closer it is to 1, the higher the similarity between the two vectors; the closer it is to 0, the lower the similarity between the two vectors. S33. Arrange the cosine similarities in descending order. S34. Filter "simple samples", "intermediate samples", "suspected difficult samples", and "difficult samples" based on the cosine similarity of the image pairs and the size of the image feature norm. In S4, the "suspected difficult samples" obtained in S3 are input. This batch of data may contain various types of data, so further mining is required. Specifically, the following steps are included: S41. Use the suspected difficult samples obtained in S3 as input for this step; S42. The algorithm execution process is as follows: if the suspected difficult sample is a difficult sample, then theoretically the cosine similarity between the image and any other image is relatively low. Therefore, the frequency of difficult samples appearing in name_list_min will be relatively high. Based on this argument, the name_list_min list is traversed. If the highest frequency minus the second highest frequency is greater than the threshold thres_3, then the image corresponding to the highest frequency is considered to be a difficult sample and can be deleted directly. In S6, the samples retained after S2 and S3 are executed are merged and output as the cleaned face training image.

Images