Method for obtaining offline quantization calibration set of face detection model

Abstract

The present application relates to the field of computer vision, and particularly relates to a method for obtaining an offline quantization calibration set of a face detection model, after obtaining a face detection dataset, each sample in the face detection dataset is classified according to different scenes, and the scene quality label, the face density label, the face average area label, the face average orientation label and the face average quality label corresponding to each sample in the face detection dataset are determined, then a plurality of samples meeting the requirements are screened out from the face detection dataset to form the offline quantization calibration set of the face detection model. By marking and storing each sample in the face detection dataset, the present application automatically selects a plurality of representative and diverse samples to form the offline quantization calibration set of the face detection model, which greatly improves the work efficiency compared with manually preparing the offline quantization calibration set of the face detection model, and is simple to operate and has strong scalability.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically to a method for obtaining an offline quantization calibration set for a face detection model. Background Technology

[0002] Model quantization refers to converting the floating-point operations of a neural network into fixed-point operations to accelerate inference, reduce device power consumption, and minimize storage space. It is mainly divided into Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT). Typically, chip manufacturers perform quantization operations before deploying deep learning models to edge devices, providing offline quantization solutions based on calibration sets. This requires chip users to select 50-100 typical scene images as a calibration set to collect data distribution information, including minimum / maximum values, optimal thresholds based on entropy theory, and quantization factors based on symmetric quantization. Ultimately, these quantization parameters are recorded in the newly generated quantized model. A reasonable calibration set can reduce accuracy loss during the quantization process.

[0003] However, the current method of quickly and accurately selecting typical scene image samples as the offline quantization calibration set for face detection models can only rely on manual selection, which is relatively inefficient. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for obtaining an offline quantization calibration set for face detection models. This method involves labeling samples in the face detection dataset according to different dimensions, such as scene ID, scene quality, face density, average face orientation, average face quality, and average face size, to automatically select a more representative and diverse sample calibration set.

[0005] The objective of this invention is achieved through the following scheme: a method for obtaining an offline quantization calibration set for a face detection model. After obtaining the face detection dataset, each sample in the face detection dataset is classified, and each sample in the face detection dataset is processed according to the following steps:

[0006] 1) Determine the scene quality label corresponding to each sample in the face detection dataset;

[0007] 2) Determine the face density label corresponding to each sample in the face detection dataset;

[0008] 3) Determine the average face area label for each sample in the face detection dataset;

[0009] 4) Determine the average face orientation label for each sample in the face detection dataset;

[0010] 5) Determine the average face quality label for each sample in the face detection dataset;

[0011] Finally, several samples that meet the requirements are selected from the face detection dataset to form an offline quantization calibration set for the face detection model.

[0012] Preferably, after obtaining the face detection dataset, the type of the face detection dataset is determined, and the samples in the face detection dataset are classified in the following manner:

[0013] ① If the face detection dataset is a public dataset, extract the features of each sample and cluster them using a feature matching algorithm to obtain multiple cluster centers. Set a corresponding scene ID label for each cluster center and use different scene ID labels to classify each sample in the face detection dataset.

[0014] ② If the face detection dataset is a self-collected dataset, set a corresponding scene ID label for each sample collection device, and use different scene ID labels to classify each sample in the face detection dataset.

[0015] Preferably, the scene quality label includes "black and white" and "color", and the specific detection steps are as follows:

[0016] ① Set a threshold for judging black and white images;

[0017] ② Divide the sample into three channels: R, G, and B. Calculate the difference between the color information of each pixel coordinate in each channel, and count the number of pixel coordinates where the difference between the color information in each channel is zero. Determine the relationship between this number and the threshold for judging black and white images.

[0018] If the number is greater than or equal to the black and white image judgment threshold, then the scene quality label of the sample is set to "black and white".

[0019] If the number is less than the black and white image judgment threshold, then the scene quality label of the sample is set to "color".

[0020] Preferably, the scene quality label further includes "blurred" and "clear," and the specific detection steps are as follows:

[0021] ① Set a threshold for judging blurry images;

[0022] ② Convert the sample to a grayscale image, perform a Laplacian transform, calculate the variance of the pixels within the grayscale image, and determine the relationship between this variance and the blur threshold for judging the image:

[0023] If the variance is less than or equal to the blurry image judgment threshold, then the scene quality label of the sample is set to "blurry".

[0024] If the variance is greater than the blurry image judgment threshold, then the scene quality label of the sample is set to "clear".

[0025] Preferably, the scene quality label further includes "color cast" and "no color cast". The specific steps to determine whether the scene quality label corresponding to each sample in the face detection dataset is "color cast" or "no color cast" are as follows:

[0026] ① Set a threshold for judging color cast images;

[0027] ② Convert the sample from the RGB color space to the CIELAB color space, and calculate the color cast value of the sample according to the following formula:

[0028]

[0029] In the formula, da represents the sample in the CIELAB color space. * The mean of the components on the axis, Ma is the value of the sample in the CIELAB color space a. * The variance of the components on the axis, db is the sample in the CIELAB color space b * The mean of the components on the axis, Mb is the value of the sample in the CIELAB color space. * The variance of the components on the axis, where K is the color cast value of the sample;

[0030] ③ Determine the relationship between the color cast value of the sample and the color cast image judgment threshold:

[0031] If the color cast value of the sample is greater than or equal to the color cast image judgment threshold, then the scene quality label of the sample is set to "color cast".

[0032] If the color cast value of the sample is less than the color cast image judgment threshold, then the scene quality label of the sample is set to "no color cast".

[0033] Preferably, the scene quality label also includes "normal brightness", "too dark", and "too bright", and the specific detection steps are as follows:

[0034] ① Set a normal brightness judgment threshold, which includes an upper limit and a lower limit for normal brightness judgment;

[0035] ② Convert the sample to a grayscale image and calculate the mean value of the pixels within the grayscale image;

[0036] ③ Determine the relationship between the mean value of pixels within a grayscale image and the normal brightness threshold:

[0037] If the mean value of the pixels inside the grayscale image is less than or equal to the lower limit of normal brightness judgment, then the scene quality label of the sample is set to "too dark".

[0038] If the average value of the pixels inside the grayscale image is within the normal brightness judgment threshold range, then the scene quality label of the sample is set to "normal brightness".

[0039] If the average value of the pixels inside the grayscale image is greater than or equal to the upper limit of normal brightness judgment, then the scene quality label of the sample is set to "too bright".

[0040] Preferably, the specific method for determining the face density label corresponding to each sample in the face detection dataset is as follows:

[0041] 2-1) Set a face density threshold, which includes an upper limit and a lower limit for face density;

[0042] 2-2) Read the annotation information of the sample, count the total number of faces in the sample, and determine the relationship between the total number of faces in the sample and the face density threshold:

[0043] If the total number of faces is less than the lower limit of face density, then the face density label of the sample is set to "low density".

[0044] If the total number of faces is within the face density threshold range, then the face density label of the sample is set to "medium density".

[0045] If the total number of faces exceeds the upper limit of face density, the face density label of the sample is set to "high density".

[0046] Preferably, the specific method for determining the average face area label corresponding to each sample in the face detection dataset is as follows:

[0047] 3-1) Set a threshold for the average face area, which includes an upper limit and a lower limit for the average face area;

[0048] 3-2) Read the annotation information of the sample, calculate the area of ​​each face in the sample, and determine the relationship between the average face area in the sample and the average face area threshold:

[0049] If the average face area is less than the lower limit of the average face area, then the average face area label of the sample is set to "small face".

[0050] If the average face area is within the threshold range, then the average face area label of the sample is set to "average face".

[0051] If the average face area is greater than the upper limit of the average face area, then the average face area of ​​the sample is labeled as "large face".

[0052] Preferably, the specific method for determining the average face orientation label corresponding to each sample in the face detection dataset is as follows:

[0053] 4-1) Set the orientation deflection threshold;

[0054] 4-2) Read the annotation information of the sample and use the head pose estimation network to calculate the pitch angle, yaw angle and roll angle corresponding to each annotated face image in the sample;

[0055] 4-3) Calculate the average of the sum of the absolute values ​​of each pitch angle, the average of the sum of the absolute values ​​of each yaw angle, and the average of the sum of the absolute values ​​of each roll angle, and determine the relationship between each average value and the heading deflection threshold:

[0056] If all average values ​​are less than the orientation deflection threshold, then the average orientation label of the face in this sample is set to "frontal orientation".

[0057] If any average value is greater than or equal to the orientation deflection threshold, the average orientation label of the face in that sample is set to "side orientation".

[0058] Preferably, the specific method for determining the average quality label of each sample in the face detection dataset is as follows:

[0059] 5-1) Set a face quality threshold, which includes an upper limit and a lower limit for face quality;

[0060] 5-2) Read the annotation information of any sample, and send each annotated face image in the sample into the face quality detection network to obtain the corresponding face quality score;

[0061] 5-3) After summing the face quality scores of each labeled face image, calculate the average and compare the resulting average with the face quality threshold:

[0062] If the average value is less than the lower limit of face quality, then the average face quality label of the sample is set to "low quality face".

[0063] If the average value is within the face quality threshold range, then the average face quality label of the sample is set to "medium quality face".

[0064] If the average value is greater than the upper limit of face quality, then the average face quality label of the sample is set to "high quality face".

[0065] Preferably, a sample reduction threshold is set. Before selecting a number of qualified samples from the face detection dataset to form the offline quantization calibration set of the face detection model, all samples corresponding to scene types with a sample number lower than the sample reduction threshold are deleted to reduce the face detection dataset.

[0066] Preferably, the specific method for selecting a number of qualified samples from the face detection dataset to form the offline quantization calibration set for the face detection model is as follows:

[0067] (1) Set scene quality control threshold, face density control threshold, average face area control threshold, average face orientation control threshold, and average face quality control threshold.

[0068] Preferably, the scene quality control threshold includes a normal color sample quantity threshold and a normal black and white sample quantity threshold;

[0069] Preferably, the face density control threshold includes a low-density sample quantity threshold, a medium-density sample quantity threshold, and a high-density sample quantity threshold;

[0070] Preferably, the average face area control threshold includes a threshold for the number of small face samples, a threshold for the number of medium-sized face samples, and a threshold for the number of large face samples.

[0071] Preferably, the average face orientation control threshold includes a threshold for the number of front-facing samples and a threshold for the number of side-facing samples;

[0072] Preferably, the average face quality control threshold includes a low-quality sample quantity threshold, a medium-quality sample quantity threshold, and a high-quality sample quantity threshold;

[0073] (2) Based on the scene quality control threshold, several normal color samples and normal black and white samples are randomly selected from the face detection dataset to form a scene quality sample set;

[0074] The normal color sample is a sample that simultaneously possesses four scene quality labels: "color", "clear", "no color cast", and "normal brightness".

[0075] The normal black and white sample is a sample that simultaneously possesses four scene quality labels: "black and white", "clear", "no color cast", and "normal brightness".

[0076] Preferably, the number of normal color samples in the scene quality sample set is a normal color sample quantity threshold, and the number of normal black and white samples in the scene quality sample set is a normal black and white sample quantity threshold.

[0077] (3) Based on the face density control threshold, randomly select samples with face density labels of "low density", "medium density" and "high density" from the face detection dataset to form a face density sample set.

[0078] Preferably, the number of samples labeled "low density" in the face density sample set is the low density sample number threshold, the number of samples labeled "medium density" is the medium density sample number threshold, and the number of samples labeled "high density" is the high density sample number threshold.

[0079] (4) Based on the average face area control threshold, randomly select samples with average face area labels of "small face", "medium face" and "large face" from the face detection dataset to form a sample set of average face area.

[0080] Preferably, the number of samples labeled "small face" in the average face area sample set is the threshold for the number of small face samples, the number of samples labeled "medium density" is the threshold for the number of medium-sized face samples, and the number of samples labeled "high density" is the threshold for the number of large face samples.

[0081] (5) Based on the average face orientation control threshold, randomly select samples with average face orientation labels of "frontal orientation" and "side orientation" from the face detection dataset to form an average face orientation sample set;

[0082] Preferably, the number of samples with the average face orientation label "frontal orientation" in the face average orientation sample set is the frontal orientation sample number threshold, and the number of samples with the average face orientation label "side orientation" is the side orientation sample number threshold.

[0083] (6) Based on the average face quality control threshold, randomly select samples with average face quality labels of "low quality", "medium quality" and "high quality" from the face detection dataset to form an average face quality sample set.

[0084] Preferably, the number of samples with the average face quality label "low quality" in the face average quality sample set is the low quality sample number threshold, the number of samples with the average face quality label "medium quality" is the medium quality sample number threshold, and the number of samples with the average face quality label "high quality" is the high quality sample number threshold.

[0085] (7) Remove duplicate samples from the scene quality sample set, face density sample set, face average area sample set, face average orientation sample set, and face average quality sample set, and use the remaining samples to form the offline quantization calibration set for the face detection model.

[0086] The advantages of this invention are as follows:

[0087] ① This invention sets corresponding attribute labels for each sample in the face detection dataset, which is simple to operate and highly scalable;

[0088] ② This invention utilizes a computer to automatically select several representative and diverse samples to form an offline quantization calibration set for the face detection model. Compared with the process of manually creating an offline quantization calibration set for the face detection model, this invention is simple to operate, highly scalable, and extremely efficient. Attached Figure Description

[0089] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0090] like Figure 1 As shown, a method for obtaining an offline quantization calibration set for a face detection model involves first determining the type of the face detection dataset after obtaining it, and then classifying the samples in the face detection dataset according to the following method:

[0091] ① If the face detection dataset is a public dataset, extract the features of each sample and cluster them using a feature matching algorithm to obtain multiple cluster centers. Set a corresponding scene ID label for each cluster center and use different scene ID labels to classify each sample in the face detection dataset.

[0092] For example, if the face detection dataset is the public dataset Widerface, the first step is to extract the feature point set of each sample image in the Widerface dataset in batches. Then, a sample image is randomly selected as a cluster center. Using a feature matching algorithm, the number of feature points that match the sample image with the other sample images in the Widerface dataset is calculated. If the quotient of the number of matching feature points divided by the number of feature points of the sample image itself is greater than 0.1, then the two sample images are considered to belong to the same category and are labeled with the same scene ID label. After traversing all samples, another sample image is randomly selected from the remaining samples as a cluster center, and the above process is repeated until all sample images are labeled.

[0093] In this embodiment, the feature point extraction algorithms include SIFT algorithm, Kaze / Akaze algorithm, and Superpoint (Self-Supervised Interest Point Detection and Description) algorithm; the feature matching algorithms include nearest neighbor matching algorithm, brute force matching algorithm, and Superglue (Learning Feature Matching with Graph Neural Networks) matching algorithm.

[0094] ② If the face detection dataset is a self-collected dataset, set a corresponding scene ID label for each sample collection device, and use different scene ID labels to classify each sample in the face detection dataset.

[0095] After classifying the samples in the face detection dataset, process each sample in the face detection dataset according to the following steps:

[0096] 1) Determine the scene quality label corresponding to each sample in the face detection dataset;

[0097] In this embodiment, the scene quality labels include "black and white" and "color", and the determination method is as follows:

[0098] ① Set a black and white image judgment threshold. In this embodiment, the black and white image judgment threshold is 5% of the total number of pixels of the sample to be processed;

[0099] ② Divide the sample into three channels: R, G, and B. Calculate the difference between the color information of each pixel coordinate in each channel, and count the number of pixel coordinates where the difference between the color information in each channel is zero. Determine the relationship between this number and the threshold for judging black and white images.

[0100] If the number is greater than or equal to the black and white image judgment threshold, then the scene quality label of the sample is set to "black and white".

[0101] If the number is less than the black and white image judgment threshold, then the scene quality label of the sample is set to "color".

[0102] In this embodiment, the scene quality labels also include "blurred" and "sharp," and the specific detection steps are as follows:

[0103] ① Set a blurry image judgment threshold. In this embodiment, the blurry image judgment threshold is set to 100.

[0104] ② Convert the sample to a grayscale image, perform a Laplacian transform, calculate the variance of the pixels within the grayscale image, and determine the relationship between this variance and the blur threshold for judging the image:

[0105] If the variance is less than or equal to the blurry image judgment threshold, then the scene quality label of the sample is set to "blurry".

[0106] If the variance is greater than the blurry image judgment threshold, then the scene quality label of the sample is set to "clear".

[0107] In this embodiment, the scene quality label also includes "color bias" and "no color bias". The specific steps to determine whether the scene quality label corresponding to each sample in the face detection dataset is "color bias" or "no color bias" are as follows:

[0108] ① Set a threshold for judging color cast images. In this embodiment, the threshold for judging color cast images is set to 1.5.

[0109] ② Convert the sample from the RGB color space to the CIELAB color space, and calculate the color cast value of the sample according to the following formula:

[0110]

[0111] In the formula, da represents the sample in the CIELAB color space. * The mean of the components on the axis, Ma is the value of the sample in the CIELAB color space a. * The variance of the components on the axis, db is the sample in the CIELAB color space b * The mean of the components on the axis, Mb is the value of the sample in the CIELAB color space. * The variance of the components on the axis, where K is the color cast value of the sample;

[0112] ③ Determine the relationship between the color cast value of the sample and the color cast image judgment threshold:

[0113] If the color cast value of the sample is greater than or equal to the color cast image judgment threshold, then the scene quality label of the sample is set to "color cast".

[0114] If the color cast value of the sample is less than the color cast image judgment threshold, then the scene quality label of the sample is set to "no color cast".

[0115] In this embodiment, the scene quality labels also include "normal brightness", "too dark", and "too bright", and the specific detection steps are as follows:

[0116] ① Set a normal brightness judgment threshold, which includes an upper limit and a lower limit for normal brightness judgment;

[0117] In this embodiment, the upper limit value for normal brightness judgment is 200, and the lower limit value for normal brightness judgment is 40;

[0118] ② Convert the sample to a grayscale image and calculate the mean value of the pixels within the grayscale image;

[0119] ③ Determine the relationship between the mean value of pixels within a grayscale image and the normal brightness threshold:

[0120] If the mean value of the pixels inside the grayscale image is less than or equal to the lower limit of normal brightness judgment, then the scene quality label of the sample is set to "too dark".

[0121] If the average value of the pixels inside the grayscale image is within the normal brightness judgment threshold range, then the scene quality label of the sample is set to "normal brightness".

[0122] If the average value of the pixels inside the grayscale image is greater than or equal to the upper limit of normal brightness judgment, then the scene quality label of the sample is set to "too bright".

[0123] 2) Determine the face density label for each sample in the face detection dataset using the following method:

[0124] 2-1) Set a face density threshold, which includes an upper limit and a lower limit for face density;

[0125] In this embodiment, the upper limit of face density is 10, and the lower limit of face density is 5;

[0126] 2-2) Read the annotation information of the sample, count the total number of faces in the sample, and determine the relationship between the total number of faces in the sample and the face density threshold:

[0127] If the total number of faces is less than the lower limit of face density, then the face density label of the sample is set to "low density".

[0128] If the total number of faces is within the face density threshold range, then the face density label of the sample is set to "medium density".

[0129] If the total number of faces exceeds the upper limit of face density, the face density label of the sample is set to "high density".

[0130] 3) Determine the average face area label for each sample in the face detection dataset using the following method:

[0131] 3-1) Set a threshold for the average face area, which includes an upper limit and a lower limit for the average face area;

[0132] In this embodiment, the upper limit of the average face area is 9216 pixels, and the lower limit of the average face area is 1024 pixels.

[0133] 3-2) Read the annotation information of the sample, count the area of ​​each face in the sample, calculate the average face area of ​​the sample (i.e., the sum of the areas of each face divided by the number of faces in the sample), and determine the relationship between the average face area of ​​the sample and the average face area threshold:

[0134] If the average face area is less than the lower limit of the average face area, then the average face area label of the sample is set to "small face".

[0135] If the average face area is within the threshold range, then the average face area label of the sample is set to "average face".

[0136] If the average face area is greater than the upper limit of the average face area, then the average face area of ​​the sample is labeled as "large face".

[0137] 4) Determine the average face orientation label for each sample in the face detection dataset as follows:

[0138] 4-1) Set the orientation deflection threshold. In this embodiment, the orientation deflection threshold is 20°.

[0139] 4-2) Read the annotation information of the sample and use the head pose estimation network to calculate the pitch angle, yaw angle and roll angle corresponding to each annotated face image in the sample;

[0140] In this embodiment, the head pose estimation networks used include HopeNet (Fine-Grained Head Pose Estimation Without Keypoints), img2pose (Face Alignment and Detection via 6DoF, Face Pose Estimation), and FSA-Net (Learning Fine-Grained Structure Aggregation for Head Pose Estimation from a Single Image).

[0141] 4-3) Calculate the average of the sum of the absolute values ​​of each pitch angle, the average of the sum of the absolute values ​​of each yaw angle, and the average of the sum of the absolute values ​​of each roll angle, and determine the relationship between each average value and the heading deflection threshold:

[0142] If all average values ​​are less than the orientation deflection threshold, then the average orientation label of the face in this sample is set to "frontal orientation".

[0143] If any average value is greater than or equal to the orientation deflection threshold, the average orientation label of the face in that sample is set to "side orientation".

[0144] 5) Determine the average face quality label for each sample in the face detection dataset as follows:

[0145] 5-1) Set a face quality threshold, which includes an upper limit and a lower limit for face quality;

[0146] In this embodiment, the upper limit of face quality is 0.6, and the lower limit of face quality is 0.3;

[0147] 5-2) Read the annotation information of any sample, and send each annotated face image in the sample into the face quality detection network to obtain the corresponding face quality score;

[0148] In this embodiment, the face quality detection networks used include EQFace (A Simple Explicit Quality Network for Face Recognition), SER-FIQ (Unsupervised Estimation of Face ImageQuality Based on Stochastic Embedding Robustness), and PLQ (Pixel-Level Face ImageQuality Assessment for Explainable Face Recognition).

[0149] 5-3) After summing the face quality scores of each labeled face image, calculate the average and compare the resulting average with the face quality threshold:

[0150] If the average value is less than the lower limit of face quality, then the average face quality label of the sample is set to "low quality face".

[0151] If the average value is within the face quality threshold range, then the average face quality label of the sample is set to "medium quality face".

[0152] If the average value is greater than the upper limit of face quality, then the average face quality label of the sample is set to "high quality face".

[0153] Finally, several samples that meet the requirements are selected from the face detection dataset to form an offline quantization calibration set for the face detection model. The specific method is as follows:

[0154] (1) Set scene quality control threshold, face density control threshold, average face area control threshold, average face orientation control threshold, and average face quality control threshold.

[0155] In this embodiment, the scene quality control threshold includes a normal color sample quantity threshold and a normal black and white sample quantity threshold;

[0156] The face density control thresholds include a low-density sample quantity threshold, a medium-density sample quantity threshold, and a high-density sample quantity threshold.

[0157] The average face area control threshold includes a threshold for the number of small face samples, a threshold for the number of medium-sized face samples, and a threshold for the number of large face samples.

[0158] The average face orientation control threshold includes a threshold for the number of front-facing samples and a threshold for the number of side-facing samples.

[0159] The average face quality control threshold includes a low-quality sample quantity threshold, a medium-quality sample quantity threshold, and a high-quality sample quantity threshold.

[0160] (2) Based on the scene quality control threshold, several normal color samples and normal black and white samples are randomly selected from the face detection dataset to form a scene quality sample set;

[0161] The normal color sample is a sample that simultaneously possesses four scene quality labels: "color", "clear", "no color cast", and "normal brightness".

[0162] The normal black and white sample is a sample that simultaneously possesses four scene quality labels: "black and white", "clear", "no color cast", and "normal brightness".

[0163] In this embodiment, the number of normal color samples in the scene quality sample set is a normal color sample quantity threshold, and the number of normal black and white samples in the scene quality sample set is a normal black and white sample quantity threshold.

[0164] (3) Based on the face density control threshold, randomly select samples with face density labels of "low density", "medium density" and "high density" from the face detection dataset to form a face density sample set.

[0165] In this embodiment, the number of samples labeled "low density" in the face density sample set is the low density sample number threshold, the number of samples labeled "medium density" is the medium density sample number threshold, and the number of samples labeled "high density" is the high density sample number threshold.

[0166] (4) Based on the average face area control threshold, randomly select samples with average face area labels of "small face", "medium face" and "large face" from the face detection dataset to form a sample set of average face area.

[0167] In this embodiment, the number of samples labeled "small face" in the average face area sample set is the threshold for the number of small face samples, the number of samples labeled "medium density" is the threshold for the number of medium-sized face samples, and the number of samples labeled "high density" is the threshold for the number of large face samples.

[0168] (5) Based on the average face orientation control threshold, randomly select samples with average face orientation labels of "frontal orientation" and "side orientation" from the face detection dataset to form an average face orientation sample set;

[0169] In this embodiment, the number of samples with the average face orientation label "frontal orientation" in the average face orientation sample set is the frontal orientation sample number threshold, and the number of samples with the average face orientation label "side orientation" is the side orientation sample number threshold.

[0170] (6) Based on the average face quality control threshold, randomly select samples with average face quality labels of "low quality", "medium quality" and "high quality" from the face detection dataset to form an average face quality sample set.

[0171] In this embodiment, the number of samples with the average face quality label "low quality" in the face average quality sample set is the low quality sample quantity threshold, the number of samples with the average face quality label "medium quality" is the medium quality sample quantity threshold, and the number of samples with the average face quality label "high quality" is the high quality sample quantity threshold.

[0172] (7) Remove duplicate samples from the scene quality sample set, face density sample set, face average area sample set, face average orientation sample set, and face average quality sample set, and use the remaining samples to form the offline quantization calibration set for the face detection model.

[0173] It is worth noting that, in order to streamline the face detection dataset and improve quantization accuracy, a sample streamlining threshold is usually set. Before selecting a number of qualified samples from the face detection dataset to form the offline quantization calibration set of the face detection model, all samples corresponding to scene types with a sample count lower than the sample streamlining threshold are deleted. That is, the samples in the final offline quantization calibration set of the face detection model cannot have scene quality labels such as "blurry", "color cast", "too dark", or "too bright". Such samples are extreme samples that will affect quantization accuracy and need to be deleted to ensure that all samples in the final offline quantization calibration set of the face detection model are representative of each scene type.

[0174] In addition, the thresholds involved in this embodiment are all empirical values, and the steps to determine each threshold are as follows:

[0175] ① Determine the initial settings for each threshold based on the offline quantization scheme provided by the chip manufacturer;

[0176] ② Obtain the offline quantization calibration set of the face detection model according to the method described in this invention;

[0177] ③ The face detection model offline quantization calibration set is used to quantize the deep learning model to be processed according to the offline quantization scheme provided by the chip manufacturer, and it is determined whether the accuracy loss of the quantization process meets the requirements of the offline quantization scheme. If it does not meet the requirements, the values ​​of each threshold are adjusted, and step ② is repeated until the accuracy loss of the quantization process meets the requirements of the offline quantization scheme.

[0178] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications made to the present invention by those skilled in the art without departing from the spirit of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for obtaining an offline quantization calibration set for a face detection model, characterized in that, Includes the following steps: 1) After obtaining the face detection dataset, determine the type of the face detection dataset and classify the samples in the face detection dataset according to the following method: 1-1) If the face detection dataset is a public dataset, extract the features of each sample and cluster them using a feature matching algorithm to obtain multiple cluster centers. Set a corresponding scene ID label for each cluster center and use different scene ID labels to classify each sample in the face detection dataset. 1-2) If the face detection dataset is a self-collected dataset, set a corresponding scene ID label for each sample collection device, and use different scene ID labels to classify each sample in the face detection dataset; 2) Process each sample in the face detection dataset as follows: 2-1) Determine the scene quality label corresponding to each sample in the face detection dataset; 2-2) Determine the face density label corresponding to each sample in the face detection dataset; 2-3) Determine the average face area label for each sample in the face detection dataset; 2-4) Determine the average face orientation label for each sample in the face detection dataset; 2-5) Determine the average quality label of each sample in the face detection dataset; 3) Based on the offline quantization scheme provided by the chip manufacturer, set the initial values ​​for scene quality control threshold, face density control threshold, average face area control threshold, average face orientation control threshold, and average face quality control threshold: 3-1) The scene quality control thresholds include the normal color sample quantity threshold and the normal black and white sample quantity threshold; 3-2) The face density control thresholds include low-density sample quantity thresholds, medium-density sample quantity thresholds, and high-density sample quantity thresholds; 3-3) The average face area control threshold includes a threshold for the number of small face samples, a threshold for the number of medium-sized face samples, and a threshold for the number of large face samples; 3-4) The average face orientation control threshold includes a threshold for the number of front-facing samples and a threshold for the number of side-facing samples; 3-5) The average face quality control thresholds include a low-quality sample quantity threshold, a medium-quality sample quantity threshold, and a high-quality sample quantity threshold; 4) Select several qualified samples from the face detection dataset according to the following method to form the offline quantization calibration set for the face detection model: 4-1) Based on the scene quality control threshold, randomly select several normal color samples and normal black and white samples from the face detection dataset to form a scene quality sample set: 4-1-1) The normal color sample mentioned above is a sample that simultaneously possesses four scene quality labels: "color", "clear", "no color cast", and "normal brightness"; 4-1-2) The normal black and white sample mentioned above is a sample that simultaneously possesses four scene quality labels: "black and white", "clear", "no color cast", and "normal brightness". 4-1-3) The number of normal color samples in the scene quality sample set is a normal color sample quantity threshold, and the number of normal black and white samples in the scene quality sample set is a normal black and white sample quantity threshold; 4-2) Based on the face density control threshold, randomly select samples with face density labels of "low density", "medium density" and "high density" from the face detection dataset to form a face density sample set; 4-2-1) The number of samples with the face density label "low density" in the face density sample set is the threshold number of low density samples; 4-2-2) The number of samples labeled "medium density" is the threshold number of medium density samples; 4-2-3) The number of samples labeled "high density" is the threshold for the number of high-density samples; 4-3) Based on the average face area control threshold, randomly select samples with average face area labels of "small face", "medium face" and "large face" from the face detection dataset to form an average face area sample set. 4-3-1) The number of samples with the average face area label "small face" in the sample set of average face area is the threshold number of small face samples; 4-3-2) The number of samples with the average face area labeled "medium density" is the threshold for the number of medium-sized face samples; 4-3-3) The number of samples labeled "high density" for average face area is the threshold for the number of large face samples; 4-4) Based on the average face orientation control threshold, randomly select samples with average face orientation labels of "frontal orientation" and "side orientation" from the face detection dataset to form an average face orientation sample set; 4-4-1) The number of samples in the average face orientation sample set labeled "frontal orientation" is the threshold number of frontal orientation samples; 4-4-2) The number of samples with the average face orientation label "side orientation" is the threshold number of side orientation samples; 4-5) Based on the average face quality control threshold, randomly select samples with average face quality labels of "low quality", "medium quality" and "high quality" from the face detection dataset to form an average face quality sample set. 4-5-1) The number of samples in the face average quality sample set labeled "low quality" is the low quality sample number threshold. 4-5-2) The number of samples with the average face quality label of "medium quality" is the threshold number of medium quality samples; 4-5-3) The number of samples with the average face quality label of "high quality" is the threshold for the number of high-quality samples; 4-6) Remove duplicate samples from the scene quality sample set, face density sample set, face average area sample set, face average orientation sample set, and face average quality sample set, and use the remaining samples to form the offline quantization calibration set for the face detection model. 5) Obtain the optimal values ​​of scene quality control threshold, face density control threshold, average face area control threshold, average face orientation control threshold, and average face quality control threshold using the following methods, and obtain the optimal offline quantization calibration set for the face detection model: 5-1) The face detection model offline quantization calibration set is used to perform model quantization processing on the deep learning model to be processed according to the offline quantization scheme provided by the chip manufacturer; 5-2) Determine whether the accuracy loss of the quantization process meets the requirements of the offline quantization scheme. If not, adjust the value of each threshold in step 3) and repeat step 4) to regenerate the offline quantization calibration set of the face detection model until the accuracy loss of the quantization process meets the requirements of the offline quantization scheme.

2. The method for obtaining an offline quantization calibration set for a face detection model according to claim 1, characterized in that, The scene quality labels include "black and white" and "color", and the specific detection steps are as follows: ① Set a threshold for judging black and white images; ② Divide the sample into three channels: R, G, and B. Calculate the difference between the color information of each pixel coordinate in each channel, and count the number of pixel coordinates where the difference between the color information in each channel is zero. Determine the relationship between this number and the threshold for judging black and white images. If the number is greater than or equal to the black and white image judgment threshold, then the scene quality label of the sample is set to "black and white". If the number is less than the black and white image judgment threshold, then the scene quality label of the sample is set to "color". The scene quality labels also include "blurred" and "clear," and the specific detection steps are as follows: ① Set a threshold for judging blurry images; ② Convert the sample to a grayscale image, perform a Laplacian transform, calculate the variance of the pixels within the grayscale image, and determine the relationship between this variance and the blur threshold for judging the image: If the variance is less than or equal to the blurry image judgment threshold, then the scene quality label of the sample is set to "blurry". If the variance is greater than the blurry image judgment threshold, then the scene quality label of the sample is set to "clear". The scene quality labels also include "color cast" and "no color cast," and the specific detection steps are as follows: ① Set a threshold for judging color cast images; ② Convert the sample from the RGB color space to the CIELAB color space, and calculate the color cast value of the sample according to the following formula: ； In the formula, is the color space a of the sample in CIELAB. * The mean of the components on the axis is the value of the sample in the CIELAB color space a. * The variance of the components on the axis is the variance of the sample in the CIELAB color space b. * The mean of the components on the axis is the value of the sample in the CIELAB color space b. * The variance of the components on the axis is the color cast value of the sample; ③ Determine the relationship between the color cast value of the sample and the color cast image judgment threshold: If the color cast value of the sample is greater than or equal to the color cast image judgment threshold, then the scene quality label of the sample is set to "color cast". If the color cast value of the sample is less than the color cast image judgment threshold, then the scene quality label of the sample is set to "no color cast". The scene quality labels also include "normal brightness", "too dark", and "too bright", and the specific detection steps are as follows: ① Set a normal brightness judgment threshold, which includes an upper limit and a lower limit for normal brightness judgment; ② Convert the sample to a grayscale image and calculate the mean value of the pixels within the grayscale image; ③ Determine the relationship between the mean value of pixels within a grayscale image and the normal brightness judgment threshold: If the mean value of the pixels inside the grayscale image is less than or equal to the lower limit of normal brightness judgment, then the scene quality label of the sample is set to "too dark". If the average value of the pixels inside the grayscale image is within the normal brightness judgment threshold range, then the scene quality label of the sample is set to "normal brightness". If the average value of the pixels inside the grayscale image is greater than or equal to the upper limit of normal brightness judgment, then the scene quality label of the sample is set to "too bright".

3. The method for obtaining an offline quantization calibration set for a face detection model according to claim 1, characterized in that, The specific method for determining the face density label corresponding to each sample in the face detection dataset is as follows: 2-1) Set a face density threshold, which includes an upper limit and a lower limit for face density; 2-2) Read the annotation information of the sample, count the total number of faces in the sample, and determine the relationship between the total number of faces in the sample and the face density threshold: If the total number of faces is less than the lower limit of face density, then the face density label of the sample is set to "low density". If the total number of faces is within the face density threshold range, then the face density label of the sample is set to "medium density". If the total number of faces exceeds the upper limit of face density, then the face density label of the sample is set to "high density".

4. The method for obtaining an offline quantization calibration set for a face detection model according to claim 1, characterized in that, The specific method for determining the average face area label for each sample in the face detection dataset is as follows: 3-1) Set a threshold for the average face area, which includes an upper limit and a lower limit for the average face area; 3-2) Read the annotation information of the sample, calculate the area of ​​each face in the sample, and determine the relationship between the average face area in the sample and the average face area threshold: If the average face area is less than the lower limit of the average face area, then the average face area label of the sample is set to "small face". If the average face area is within the threshold range, then the average face area label of the sample is set to "average face". If the average face area is greater than the upper limit of the average face area, then the average face area of ​​the sample is labeled as "large face".

5. The method for obtaining an offline quantization calibration set for a face detection model according to claim 1, characterized in that, The specific method for determining the average face orientation label for each sample in the face detection dataset is as follows: 4-1) Set the orientation deflection threshold; 4-2) Read the annotation information of the sample and use the head pose estimation network to calculate the pitch angle, yaw angle and roll angle corresponding to each annotated face image in the sample; 4-3) Calculate the average of the sum of the absolute values ​​of each pitch angle, the average of the sum of the absolute values ​​of each yaw angle, and the average of the sum of the absolute values ​​of each roll angle, and determine the relationship between each average value and the heading deflection threshold: If all average values ​​are less than the orientation deflection threshold, then the average orientation label of the face in this sample is set to "frontal orientation". If any average value is greater than or equal to the orientation deflection threshold, the average orientation label of the face in that sample is set to "side orientation".

6. The method for obtaining an offline quantization calibration set for a face detection model according to claim 1, characterized in that, The specific method for determining the average face quality label for each sample in the face detection dataset is as follows: 5-1) Set a face quality threshold, which includes an upper limit and a lower limit for face quality; 5-2) Read the annotation information of any sample, and send each annotated face image in the sample into the face quality detection network to obtain the corresponding face quality score; 5-3) After summing the face quality scores of each labeled face image, calculate the average and compare the resulting average with the face quality threshold: If the average value is less than the lower limit of face quality, then the average face quality label of the sample is set to "low quality face". If the average value is within the face quality threshold range, then the average face quality label of the sample is set to "medium quality face". If the average value is greater than the upper limit of face quality, then the average face quality label of the sample is set to "high quality face".

7. The method for obtaining an offline quantization calibration set for a face detection model according to claim 1, characterized in that, Before selecting a number of qualified samples from the face detection dataset to form the offline quantization calibration set for the face detection model, a sample reduction threshold is set. All samples corresponding to scene types with a sample count lower than the sample reduction threshold are deleted to reduce the face detection dataset.

Images