A large-scale masked face dataset annotation method and system
Through high-frequency conversion and preclustering technology, the labeling accuracy problem of face image data sets in the face occlusion is solved, efficient and accurate labeling results are achieved, and the correctness of labeling is ensured through manual review.
Patent Information
- Application Number
- CN202311147711.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-06
- Publication Date
- 2025-03-28
- Estimated Expiration
- 2043-09-06
AI Technical Summary
In the prior art, when face occlusion is marked with face image data sets, facial occlusion will greatly reduce the labeling accuracy, and the accuracy of the Dlib face detection algorithm is not high, which affects the labeling results.
Low-frequency noise is removed through high-frequency conversion, high-frequency texture components of face images are extracted, the relationship matrix between high-frequency texture components is calculated, pre-clustering and object detection are performed, and finally the correctness of the label is ensured through manual review.
Improve the accuracy of labeling when the face image is blocked, simplifies the labeling process, reduces time and labor costs, and ensures the correctness of labeling results.
Smart Images
Figure CN117173765B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of dataset annotation, and more specifically, to a large-scale masked face dataset annotation method and system. Background Art
[0002] With the rapid development of large-scale artificial intelligence models, the demand for large-scale training data is also increasing. Although it is relatively convenient to collect data, the tediousness of data labeling cannot be ignored. Therefore, high-quality labeling of massive data has become a severe challenge. Traditional manual data labeling methods are not only inefficient, but also prone to labeling errors. They can only label small amounts of high-quality data. In this context, it is particularly important to seek efficient and accurate labeling methods.
[0003] The prior art proposes a facial data annotation method and system, comprising the following steps: obtaining preset position information of preset key points in a facial image to be annotated; obtaining annotation information in the facial image to be annotated according to the preset position information of the preset key points and the geometric position relationship between different preset key points; the method infers other annotation information in the facial image to be annotated through preset key points at known positions in the facial image to be annotated and the geometric position information between the preset key points, and can accurately annotate large-scale facial data; however, because the method relies on the geometric position relationship between the preset key points, when the face of the facial image is blocked by an obstruction, the accuracy of annotating the facial image data set using the method will be greatly reduced.
[0004] The prior art also proposes a face occlusion detection algorithm, which includes the following steps: face image preprocessing, cutting the collected multiple face images according to the face position information and the coordinates of the face key points to obtain multiple facial features images as the basic data set; wherein the facial features information is obtained by the Dlib face detection algorithm; data augmentation, augmenting the basic data set; data labeling and classification, classifying the augmented images, including occlusion, eyes, mouth and nose, labeling the augmented images of different categories, and using the labeled data as training samples for A convolutional neural network model is trained; the trained convolutional neural network model is then used to detect face images; the method proposes a method for automatically labeling a face image data set when the face of the face image is blocked by an occluder; however, the facial features information of this method is directly obtained through the Dlib face detection algorithm, and the accuracy of the facial features information directly obtained through the Dlib face detection algorithm is not high, thereby affecting the accuracy of the labeling result; and the method uses a data augmentation operation, and during the data augmentation process, the natural morphology of the face may be destroyed, resulting in a low accuracy of the labeling result. Summary of the invention
[0005] In order to overcome the defect of the above-mentioned prior art that a face image dataset cannot be accurately labeled when the face of the face image is blocked by an occluder, the present invention provides a large-scale masked face dataset labeling method and system that can accurately label.
[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:
[0007] A large-scale masked face dataset annotation method includes the following steps:
[0008] S1: collect face images with occluded faces to form a face image dataset;
[0009] S2: Perform high-frequency conversion on each face image with occluded face in the face image dataset, remove low-frequency noise in the face image with occluded face, obtain high-frequency texture components corresponding to the face image with occluded face, and form a high-frequency texture component set;
[0010] S3: Calculate the relationship matrix between any two high-frequency texture components in the high-frequency texture component set;
[0011] S4: pre-clustering the face image dataset using the relationship matrix to obtain a pre-clustering matrix;
[0012] S5: Use the pre-clustering matrix and the target detection algorithm to predict the face target bounding box of the face image dataset, and obtain a set of face target bounding box prediction results;
[0013] S6: annotating the face image dataset based on the face target bounding box prediction result set to obtain a pre-annotated face image dataset;
[0014] S7: Correct the pre-labeled face image dataset to obtain a fully labeled face image dataset.
[0015] The present invention also proposes a system for labeling a large-scale masked face dataset to implement the above-mentioned large-scale masked face dataset labeling method. The system comprises:
[0016] The acquisition module is used to collect face images with occluded faces to form a face image dataset;
[0017] A high-frequency conversion module is used to perform high-frequency conversion on each face image with an occluded face in the face image data set, remove low-frequency noise in the face image with an occluded face, obtain high-frequency texture components corresponding to the face image with an occluded face, and form a high-frequency texture component set;
[0018] A relationship matrix calculation module, used to calculate the relationship matrix between any two high-frequency texture components in the high-frequency texture component set;
[0019] A pre-clustering module, used to pre-cluster the face image data set using the relationship matrix to obtain a pre-clustering matrix;
[0020] A face target bounding box prediction module is used to predict the face target bounding box of a face image dataset using a pre-clustering matrix and a target detection algorithm to obtain a face target bounding box prediction result set;
[0021] A pre-labeling module is used to label the face image dataset based on the face target bounding box prediction result set to obtain a pre-labeled face image dataset;
[0022] The correction module is used to correct the pre-labeled face image dataset to obtain a complete labeled face image dataset.
[0023] The present invention also proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the processor executes the steps of the large-scale masked face dataset labeling method proposed by the present invention.
[0024] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0025] The present invention first extracts high-frequency texture components of facial images, removes low-frequency noise, and restores the occluded parts of facial images. Then, through a pre-clustering method, facial images that may belong to the same person are pre-clustered into the same category. After completing the pre-clustering, target detection is performed according to the clustering type, which can better utilize the similarities between facial images of the same person and improve the accuracy of target detection. Finally, through manual review, the correctness of the annotation of the facial image data set is ensured, thereby achieving the purpose of accurately annotating the facial image data set when the face of the facial image is occluded. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 This is a flow chart of a large-scale masked face dataset annotation method according to Example 1;
[0027] Figure 2 This is a schematic diagram of the process of pre-clustering a face image dataset in Example 1;
[0028] Figure 3 This is the overall framework diagram of the system for annotating a large-scale masked face dataset of Example 2. DETAILED DESCRIPTION
[0029] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;
[0030] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;
[0031] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0032] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0033] Example 1
[0034] This embodiment proposes a large-scale masked face dataset annotation method. Figure 1 Schematic diagram of the process of the large-scale masked face dataset annotation method of this embodiment.
[0035] The large-scale masked face dataset annotation method proposed in this embodiment includes the following steps:
[0036] S1: collect face images with occluded faces to form a face image dataset;
[0037] S2: Perform high-frequency conversion on each face image with occluded face in the face image dataset, remove low-frequency noise in the face image with occluded face, obtain high-frequency texture components corresponding to the face image with occluded face, and form a high-frequency texture component set;
[0038] S3: Calculate the relationship matrix between any two high-frequency texture components in the high-frequency texture component set;
[0039] S4: pre-clustering the face image dataset using the relationship matrix to obtain a pre-clustering matrix;
[0040] S5: Use the pre-clustering matrix and the target detection algorithm to predict the face target bounding box of the face image dataset, and obtain a set of face target bounding box prediction results;
[0041] S6: annotating the face image dataset based on the face target bounding box prediction result set to obtain a pre-annotated face image dataset;
[0042] S7: Correct the pre-labeled face image dataset to obtain a fully labeled face image dataset.
[0043] In the specific implementation process, the high-frequency texture components of the face image are first extracted, the low-frequency noise is removed, and the occluded part of the face image is restored. Then, the face images that may belong to the same person are pre-clustered into the same category through the pre-clustering method. After the pre-clustering is completed, target detection is performed according to the clustering type, which can better utilize the similarities between face images of the same person and improve the accuracy of target detection, thereby improving the accuracy of labeling the face image data set using the target detection results, and simplifying the labeling process, greatly reducing the time and labor cost required for labeling. Finally, through manual review, the correctness of the labeling of the face image data set is ensured, so as to achieve the purpose of accurately labeling the face image data set when the face of the face image is occluded.
[0044] In an optional embodiment, before performing high-frequency conversion on each face image with an obstructed face in the face image data set, each pixel point in each face image with an obstructed face in the face image data set is represented in the form of a matrix element to obtain a pixel matrix corresponding to the face image with an obstructed face, thereby forming a pixel matrix set;
[0045] Optionally, the specific process of calculating the high-frequency texture component is:
[0046] Using high frequency filters Extracting high-frequency texture components from a pixel matrix set;
[0047] Among them, the i-th pixel matrix P in the pixel matrix set is converted into a high-frequency texture component h i The function expression is:
[0048]
[0049] Where (m,n) represents the pixel coordinates, (u,v) represents the pixel coordinates in the frequency domain; H and W represent the length and width of the face image with the face occluded in the face image dataset, respectively.
[0050] In this embodiment, the cutoff frequency and order of the high-frequency filter need to be preset. The extraction of the high-frequency texture component is closely related to the cutoff frequency of the high-frequency filter, and the coarseness of the texture features in the high-frequency texture component is related to the order of the high-frequency filter. If the cutoff frequency of the high-frequency filter is set too low, too much low-frequency noise that should be removed will be retained. If it is set too high, useful high-frequency texture information may be removed. Only by selecting the appropriate cutoff frequency and order of the high-frequency filter can the noise be removed while retaining the key high-frequency texture information.
[0051] Optionally, the specific process of calculating the high-frequency texture component is:
[0052] Using two-dimensional discrete Fourier transform, each pixel matrix in the pixel matrix set is converted into a frequency domain form to obtain a number of frequency domain signals; all frequency domain signals form a frequency domain signal set; using a high-frequency filter to filter the frequency domain signal set, remove low-frequency noise in the frequency domain signal set, and obtain high-frequency texture components corresponding to the face image with the face occluded;
[0053] Among them, the i-th pixel matrix P in the pixel matrix set is converted into a frequency domain signal X i The function expression is:
[0054]
[0055] Convert the i-th pixel matrix P in the pixel matrix set into the frequency domain signal X i The matrix expression of is:
[0056]
[0057] Where B1 and B2 represent the basis matrices determined by H and W respectively;
[0058] Using a high pass filter The frequency domain signal X i Converted into high-frequency texture component h i The calculation expression is:
[0059]
[0060] An image can be decomposed into two parts: structure and texture. The structural information reflects the overall framework of the image, including important descriptive information such as the edge of the image, while the texture information reflects the details in the image framework.
[0061] In this embodiment, the low-frequency signal in the frequency domain signal is removed by a high-pass filter to remove low-frequency noise such as occlusion, shadow, exposure, etc. in the face image, while retaining the high-frequency texture information such as the contour and wrinkles of the face image. Then, a two-dimensional inverse Fourier transform is used to restore the face image after noise removal, so that the similarity of face images belonging to the same person is higher, thereby improving the accuracy of subsequent pre-clustering operations.
[0062] In an optional embodiment, step S3 is specifically as follows:
[0063] S3.1: Calculate the co-correlation between any two high-frequency texture components in the high-frequency texture component set;
[0064] Assume that any two high-frequency texture components are the ath high-frequency texture component and the bth high-frequency texture component, and the ath high-frequency texture component is h a , the bth high-frequency texture component is h b, the co-correlation cov between the a-th high-frequency texture component and the b-th high-frequency texture component ab The calculation expression is:
[0065]
[0066] In the formula, vec(·) represents the function used for vectorization, represents the Frobenius norm;
[0067] S3.2: Calculate the relationship matrix between any two high-frequency texture components in the high-frequency texture component set to obtain the relationship matrix W;
[0068] The ath high-frequency texture component h a and the bth high-frequency texture component h b The calculation expression of the relationship matrix is:
[0069]
[0070]
[0071] Where N represents the total number of high-frequency texture components.
[0072] In this optional embodiment, the relationship matrix can contain the relationship between data of multiple dimensions, rather than just a single feature, so that the subsequent pre-clustering process can consider more comprehensive data relationships, which helps to better capture the structure of the face image dataset and obtain more accurate pre-clustering results.
[0073] In an alternative embodiment, Figure 2 This is a schematic diagram of the process of pre-clustering a face image dataset in this embodiment; step S4 is specifically as follows:
[0074] S4.1: Calculate the degree matrix D corresponding to the high-frequency texture component set;
[0075]
[0076] S4.2: Using the degree matrix D, calculate the Laplacian matrix L corresponding to the high-frequency texture component set:
[0077] L=DW
[0078] S4.3: Use the Laplace matrix L to construct a pre-clustering matrix optimization model, specifically:
[0079]
[0080] Where Tr represents the trace of the matrix, and I represents the identity matrix;
[0081] Among them, F qrepresents the objective function of the pre-clustering matrix optimization model, F represents the pre-clustering matrix, which is the quantity to be determined of the pre-clustering matrix optimization model, each row of the pre-clustering matrix represents each clustering type, each column represents the high-frequency texture component corresponding to each face image with the face occluded, each column has and only has one element value of 1, and the other element values are all 0, and the element with an element value of 1 indicates that the clustering type corresponding to the column where the element is located is the type indicated by the row; the initial value of each element in the pre-clustering matrix is zero or a random value;
[0082] S4.4: In F T Under the constraint condition of DF=I, the pre-clustering matrix optimization model is iteratively updated. a When it reaches the minimum or the number of iterations reaches the preset value, the iteration ends and the pre-clustering matrix is obtained.
[0083] In this optional embodiment, face image data is concentrated through pre-clustering, and face images that may belong to the same person are pre-clustered into the same category, so as to improve the accuracy of face target bounding box prediction results of subsequent target detection.
[0084] In an optional embodiment, step S5 is specifically as follows:
[0085] S5.1: extracting, row by row, the face images with occluded faces corresponding to the high-frequency texture components indicated by the columns where the elements with element values of 1 are located in the pre-clustering matrix, to obtain a number of face image sub-datasets; wherein any face image sub-dataset is composed of face images with occluded faces extracted from any row of the pre-clustering matrix;
[0086] S5.2: Use the Yolo object detection algorithm to predict the face target bounding box for all face image sub-datasets to obtain a set of face target bounding box prediction results;
[0087] Among them, for the k-th face image sub-dataset Y k The specific process of predicting the bounding box of a face target is as follows:
[0088] The face image sub-dataset Y k Divide any image in into a cell, predict several face target bounding boxes for each cell, and obtain the face target bounding box prediction result;
[0089] Among them, the size of the cell and the number of bounding boxes are preset values;
[0090] When the face image sub-dataset Y k Any image in is a face image sub-dataset Y k The fth face image I in f When the face image I fThe corresponding face target bounding box prediction result z kf The expression is:
[0091] z kf ={(x kf1 ,y kf1 , w kf1 ,h kf1 , g kf1 ),...,(x kfe ,y kfe , w kfe ,h kfe , g kfe ), ...}
[0092] x kfe =σ(tx kfe )+cx kfe ,
[0093] y kfe =σ(ty kfe )+cy kfe ,
[0094] w kfe =pw kfe *exp(tw kfe ),
[0095] h kfe =ph kfe *exp(th kfe )
[0096] Where σ(·) represents the Sigmoid function; x kfe and kfe Represents the bounding box B e The horizontal and vertical coordinates of the center coordinates, w kfe and h kfe Represents the bounding box B e Width and height of tx kfe Andty kfe Represents the bounding box B e The horizontal and vertical offsets of the predicted center coordinates, tw kfe and kfe Represents the bounding box B e The predicted width and predicted height of cx kfe andcy kfe , respectively represent the bounding box B e The horizontal and vertical coordinates of the corresponding cell center coordinates; p represents the value used to adjust the bounding box B e The parameter of the aspect ratio; exp(·) represents the exponential function; g kfe Represents the bounding box B eThe probability of belonging to the kth cluster type, g kfe By e Calculated using the Softmax activation function.
[0097] In this optional embodiment, after completing the pre-clustering, target detection is performed according to the clustering type, which can better utilize the similarities between facial images of the same person and improve the accuracy of target detection.
[0098] In an optional embodiment, the process of correcting the pre-labeled face image dataset is:
[0099] Upload the face target bounding box prediction result set and the face image dataset to Labelbox, and use Labelbox's automatic labeling algorithm to automatically label the face image dataset to obtain a pre-labeled face image dataset;
[0100] Import the pre-annotated face image dataset into the annotation tool, and check whether the automatically annotated target bounding boxes and corresponding category labels are correct one by one. If the automatic annotation results are found to be incorrect, manually adjust the position of the target bounding box or correct its category label; if the automatic annotation misses the target, manually add the bounding box and category label to the target; if all targets in the face image dataset are correctly annotated, stop checking and obtain the fully annotated face image dataset.
[0101] In this embodiment, automatic labeling of the face image dataset through Labelbox can effectively avoid subjective errors caused by manual labeling, and greatly reduce the time and labor cost required for labeling, thereby bringing more accurate and consistent labeling results; on this basis, manual review is used to correct targets that may be missed by automatic labeling or labels that are not correctly labeled, further improving the labeling quality of the completed face dataset.
[0102] Example 2
[0103] This embodiment applies the large-scale masked face dataset annotation method proposed in Embodiment 1, and proposes the following performance comparison example:
[0104] This embodiment uses the X26 face image dataset and the AR face image dataset, where the X26 face image dataset contains 260 face images of 10 different people with different expressions, lighting conditions, and masks such as sunglasses and scarves; the AR face image dataset contains 2600 face images of 100 different people, where the facial expressions, lighting conditions, and masks of glasses and scarves of each face image are different.
[0105] When using this scheme to annotate the X26 face image dataset, the number of rows of the pre-clustering matrix is set to 19, the Butterworth filter is used as the high-frequency filter, and the order and cutoff frequency of the Butterworth filter are set to 2 and 0.09 respectively;
[0106] When using this scheme to annotate the AR face image dataset, the number of rows of the pre-clustering matrix is set to 200, the Butterworth filter is used as the high-frequency filter, and the order and cutoff frequency of the Butterworth filter are set to 2 and 0.1 respectively;
[0107] Performance comparison results:
[0108] The maximum ACC (accuracy) of the X26 face image dataset annotated using the k-means algorithm is 20.00%;
[0109] The maximum ACC of the X26 face image dataset annotated using the SC algorithm is 26.92%;
[0110] The maximum ACC of the X26 face image dataset annotated using this scheme is 99.62%;
[0111] The maximum ACC of the AR face image dataset annotated using the k-means algorithm is 13.12%;
[0112] The maximum ACC of the AR face image dataset annotated using the SC algorithm is 12.77%;
[0113] The maximum ACC of the AR face image dataset annotated using this scheme is 72.28%;
[0114] In summary, the accuracy of the method proposed in the present invention in labeling face image datasets is significantly better than that of the k-means algorithm and the SC algorithm.
[0115] Example 3
[0116] This embodiment proposes a system for labeling a large-scale masked face dataset, which is used to implement a method for labeling a large-scale masked face dataset proposed in Embodiment 1.
[0117] Figure 3 This is an overall framework diagram of the system for annotating a large-scale masked face dataset in accordance with the present embodiment.
[0118] The system for labeling a large-scale masked face dataset includes:
[0119] The acquisition module is used to collect face images with occluded faces to form a face image dataset;
[0120] A high-frequency conversion module is used to perform high-frequency conversion on each face image with an occluded face in the face image data set, remove low-frequency noise in the face image with an occluded face, obtain high-frequency texture components corresponding to the face image with an occluded face, and form a high-frequency texture component set;
[0121] A relationship matrix calculation module, used to calculate the relationship matrix between any two high-frequency texture components in the high-frequency texture component set;
[0122] A pre-clustering module, used to pre-cluster the face image data set using the relationship matrix to obtain a pre-clustering matrix;
[0123] A face target bounding box prediction module is used to predict the face target bounding box of a face image dataset using a pre-clustering matrix and a target detection algorithm to obtain a face target bounding box prediction result set;
[0124] A pre-labeling module is used to label the face image dataset based on the face target bounding box prediction result set to obtain a pre-labeled face image dataset;
[0125] The correction module is used to correct the pre-labeled face image dataset to obtain a complete labeled face image dataset.
[0126] It can be understood that the system of this embodiment is applied to the method of the above-mentioned embodiment 1, and the options in the above-mentioned embodiment 1 are also applicable to this embodiment, so they will not be described repeatedly here.
[0127] Example 4
[0128] This embodiment proposes a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the processor executes the steps of the large-scale masked face dataset labeling method proposed in Example 1.
[0129] It can be understood that the computer device of this embodiment is applied to the method of the above-mentioned embodiment 1, and the options in the above-mentioned embodiment 1 are also applicable to this embodiment, so they will not be described repeatedly here.
[0130] The same or similar reference numerals correspond to the same or similar components;
[0131] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;
[0132] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A large-scale masked face dataset annotation method, characterized in that: The following steps are involved: S1: collect face images with occluded faces to form a face image dataset; S2: Perform high-frequency conversion on each face image with occluded face in the face image dataset, remove low-frequency noise in the face image with occluded face, obtain high-frequency texture components corresponding to the face image with occluded face, and form a high-frequency texture component set; S3: Calculate the relationship matrix between any two high-frequency texture components in the high-frequency texture component set; The S3 step is as follows: S3.1: Calculate the co-correlation between any two high-frequency texture components in the high-frequency texture component set; Assume that any two high-frequency texture components are the ath high-frequency texture component and the bth high-frequency texture component, and the ath high-frequency texture component is h a , the bth high-frequency texture component is h b , the co-correlation between the a-th high-frequency texture component and the b-th high-frequency texture component cov(h a ,h b ) is calculated as: In the formula, vec(·) represents the function used for vectorization, represents the Frobenius norm; S3.2: Calculate the relationship matrix between any two high-frequency texture components in the high-frequency texture component set to obtain the relationship matrix W; The ath high-frequency texture component h a and the bth high-frequency texture component h b The calculation expression of the relationship matrix between is: Where N represents the total number of high-frequency texture components; S4: pre-clustering the face image dataset using the relationship matrix to obtain a pre-clustering matrix; The specific steps of S4 are: S4.1: Calculate the degree matrix D corresponding to the high-frequency texture component set; S4.2: Using the degree matrix D, calculate the Laplacian matrix L corresponding to the high-frequency texture component set: L=DW S4.3: Use the Laplace matrix L to construct a pre-clustering matrix optimization model, specifically: Where Tr represents the trace of the matrix, and I represents the identity matrix; Among them, F q represents the objective function of the pre-clustering matrix optimization model, F represents the pre-clustering matrix, which is the quantity to be determined of the pre-clustering matrix optimization model, each row of the pre-clustering matrix represents each clustering type, each column represents the high-frequency texture component corresponding to each face image with the face occluded, each column has and only has one element value of 1, and the other element values are all 0, and the element with an element value of 1 indicates that the clustering type corresponding to the column where the element is located is the type indicated by the row; the initial value of each element in the pre-clustering matrix is zero or a random value; S4.4: In F T Under the constraint condition of DF=I, the pre-clustering matrix optimization model is iteratively updated. q When it reaches the minimum, or the number of iterations reaches the preset value, the iteration ends and the pre-clustering matrix is obtained; S5: Use the pre-clustering matrix and the target detection algorithm to predict the face target bounding box of the face image dataset, and obtain a set of face target bounding box prediction results; S6: annotating the face image dataset based on the face target bounding box prediction result set to obtain a pre-annotated face image dataset; S7: Correct the pre-labeled face image dataset to obtain a fully labeled face image dataset.
2. The large-scale masked face dataset annotation method according to claim 1, characterized in that: Before performing high-frequency conversion on each face image with an occluded face in the face image dataset, each pixel point in each face image with an occluded face in the face image dataset is represented in the form of a matrix element to obtain a pixel matrix corresponding to the face image with an occluded face to form a pixel matrix set.
3. The large-scale masked face dataset annotation method according to claim 2, characterized in that: The specific process of calculating the high-frequency texture component is: Using high frequency filters Extracting high-frequency texture components from a pixel matrix set; Among them, the i-th pixel matrix P in the pixel matrix set is converted into a high-frequency texture component h i The function expression is: Where (m,n) represents the pixel coordinates, (u,v) represents the pixel coordinates in the frequency domain; H and W represent the length and width of the face image with the face occluded in the face image dataset, respectively; Indicates the use of high frequency filters Based on this formula, the pixel matrix P is converted into high-frequency texture components h i .
4. The large-scale masked face dataset annotation method according to claim 2, characterized in that: The specific process of calculating the high-frequency texture component is: Using two-dimensional discrete Fourier transform, each pixel matrix in the pixel matrix set is converted into a frequency domain form to obtain a number of frequency domain signals; all frequency domain signals form a frequency domain signal set; using a high-frequency filter to filter the frequency domain signal set, remove low-frequency noise in the frequency domain signal set, and obtain high-frequency texture components corresponding to the face image with the face occluded; Among them, the i-th pixel matrix P in the pixel matrix set is converted into a frequency domain signal X i The function expression is: Convert the i-th pixel matrix P in the pixel matrix set into the frequency domain signal X i The matrix expression of is: Where B1 and B2 represent the basis matrices determined by H and W respectively; Using a high pass filter The frequency domain signal X i Converted into high-frequency texture component h i The calculation expression is: In the formula, Indicates the use of high-pass filter Based on this formula, the pixel matrix P is converted into high-frequency texture components h i .
5. The large-scale masked face dataset annotation method according to any one of claims 1 to 4, characterized in that: The specific steps of S5 are as follows: S5.1: extracting, row by row, the face images with occluded faces corresponding to the high-frequency texture components indicated by the columns where the elements with element values of 1 are located in the pre-clustering matrix, to obtain a number of face image sub-datasets; wherein any face image sub-dataset is composed of face images with occluded faces extracted from any row of the pre-clustering matrix; S5.2: Use the Yolo object detection algorithm to predict the face target bounding box for all face image sub-datasets to obtain a set of face target bounding box prediction results; Among them, for the k-th face image sub-dataset Y k The specific process of predicting the bounding box of a face target is as follows: The face image sub-dataset Y k Divide any image in into a cell, predict several face target bounding boxes for each cell, and obtain the face target bounding box prediction result; Among them, the size of the cell and the number of bounding boxes are preset values; When the face image sub-dataset Y k Any image in is a face image sub-dataset Y k The fth face image I in f When the face image I f The corresponding face target bounding box prediction result z kf The expression is: z kf ={(x kf1 ,y kf1 ,w kf1 ,h kf1 ,g kf1 ),...,(x kfe ,y kfe ,w kfe ,h kfe ,g kfe ),...} x kfe =σ(tx kfe )+cx kfe , and kfe =o(you kfe )+cy kfe , w kfe =pw kfe *exp(tw kfe ), h kfe =ph kfe *exp(th kfe ) Where σ(·) represents the Sigmoid function; x kfe and kfe Represents the bounding box B e The horizontal and vertical coordinates of the center coordinates, w kfe and h kfe Represents the bounding box B e Width and height of tx kfe Andty kfe Represents the bounding box B e The horizontal and vertical offsets of the predicted center coordinates, tw kfe and kfe Represents the bounding box B e The predicted width and predicted height of cx kfe andcy kfe , respectively represent the bounding box B e The horizontal and vertical coordinates of the corresponding cell center coordinates; p represents the value used to adjust the bounding box B e The parameter of the aspect ratio; exp(·) represents the exponential function; g kfe Represents the bounding box B e The probability of belonging to the kth cluster type, g kfe By e Calculated using the Softmax activation function.
6. The large-scale masked face dataset annotation method according to claim 5, characterized in that: The process of correcting the pre-annotated face image dataset is: Upload the face target bounding box prediction result set and the face image dataset to Labelbox, and use Labelbox's automatic labeling algorithm to automatically label the face image dataset to obtain a pre-labeled face image dataset; Import the pre-annotated face image dataset into the annotation tool, and check whether the automatically annotated target bounding boxes and corresponding category labels are correct one by one. If the automatic annotation results are found to be incorrect, manually adjust the position of the target bounding box or correct its category label; if the automatic annotation misses the target, manually add the bounding box and category label to the target; if all targets in the face image dataset are correctly annotated, stop checking and obtain the fully annotated face image dataset.
7. A system for labeling a large-scale masked face dataset, used to implement the large-scale masked face dataset labeling method according to any one of claims 1 to 6, characterized in that: include: The acquisition module is used to collect face images with occluded faces to form a face image dataset; A high-frequency conversion module is used to perform high-frequency conversion on each face image with an occluded face in the face image data set, remove low-frequency noise in the face image with an occluded face, obtain high-frequency texture components corresponding to the face image with an occluded face, and form a high-frequency texture component set; A relationship matrix calculation module, used to calculate the relationship matrix between any two high-frequency texture components in the high-frequency texture component set; A pre-clustering module, used to pre-cluster the face image data set using the relationship matrix to obtain a pre-clustering matrix; A face target bounding box prediction module is used to predict the face target bounding box of a face image dataset using a pre-clustering matrix and a target detection algorithm to obtain a face target bounding box prediction result set; A pre-labeling module is used to label the face image dataset based on the face target bounding box prediction result set to obtain a pre-labeled face image dataset; The correction module is used to correct the pre-labeled face image dataset to obtain a complete labeled face image dataset.
8. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, characterized in that: When the computer-readable instructions are executed by the processor, the processor performs the steps of the large-scale masked face dataset labeling method according to any one of claims 1 to 6.