Effective face detection method, device, equipment, storage medium and program product

Abstract

The application discloses an effective face detection method, device and equipment, a storage medium and a program product, and is applied to the field of image detection. The method comprises the following steps: obtaining a to-be-detected person image and a downstream task type; inputting the to-be-detected person image into an effective face recognition model to obtain face coordinate information of each face in the to-be-detected person image and a face out-of-focus blur score; determining a face size of the face according to the face coordinate information; screening a first effective face with a face size greater than a preset face size threshold from the faces; in the case that the number of the first effective face is greater than a preset threshold, classifying the first effective face according to the face out-of-focus blur score of the first effective face to obtain an in-focus face category and / or an out-of-focus face category; and determining the first effective face belonging to the in-focus face category as a target effective face. According to the embodiment of the application, the efficiency of effective face detection can be improved, and the computing resources can be saved.

Description

Technical Field

[0001] This application belongs to the field of image detection technology, and in particular relates to an effective face detection method, apparatus, device, computer storage medium, and computer program product. Background Technology

[0002] Rapid and accurate facial image screening technology has a wide and urgent application demand in news media, the photography industry, social media platforms, and security monitoring. With the popularization of digital photography equipment, the amount of image data is growing explosively. How to automatically and efficiently select effective facial images from massive amounts of photos has become a key technical link in improving the efficiency of workflows in related industries.

[0003] A valid face image is a face image that, in a specific application scenario, is large enough and within the depth of field of a camera lens, and can provide valuable information for subsequent tasks (such as face recognition, key point detection, and subjective evaluation by the human eye).

[0004] An invalid face image is a face image that is too small or too blurry because it is outside the depth of field of the camera lens, making subsequent tasks impossible or unable to provide valuable information.

[0005] Currently, effective face detection schemes have high computational costs and their detection efficiency needs to be improved. Summary of the Invention

[0006] This application provides an effective face detection method, apparatus, device, computer storage medium, and computer program product, which can improve the efficiency of effective face detection while saving computing resources.

[0007] On the one hand, embodiments of this application provide an effective face detection method, which includes:

[0008] Obtain the image of the person to be detected and the type of downstream task;

[0009] The image of the person to be detected is input into the effective face recognition model to obtain the face coordinate information of each face in the image and the face blur score; the effective face recognition model includes a baseline face recognition model, and the prediction head part of the baseline face recognition model includes a recognition model and an image quality evaluation model; the recognition model is used to identify whether there is a face in the image, and the image quality evaluation model is used to score the blur of the recognized face.

[0010] For each face, the face size is determined based on the face coordinate information;

[0011] From all faces, select the first valid face whose size is greater than a preset face size threshold; the preset face size threshold is determined according to the downstream task type.

[0012] If the number of first valid faces exceeds a preset threshold, the first valid faces are classified according to their out-of-focus blur score to obtain in-focus face categories and / or out-of-focus face categories.

[0013] The first valid face belonging to the in-focus face category is identified as the target valid face.

[0014] In some possible implementations, the method also includes:

[0015] Obtain an image sample group, which includes at least one original image sample;

[0016] For each original image sample, the original image sample is scaled multiple times with different scaling ratios in order of increasing scaling ratio to obtain multiple scaled image samples.

[0017] Based on the size of the original face detection box in the original image sample, the scaled face detection box and its size are obtained by scaling the image sample according to multiple different scaling ratios.

[0018] Based on each scaled face detection bounding box, the scaled face region image is cropped from the corresponding scaled image sample;

[0019] Calculate the similarity between each scaled face region image and the baseline face region image, where the baseline face region image is the original face region image in the original image sample.

[0020] Based on multiple similarities and their corresponding scaled face detection box sizes, the relationship between similarity and face detection box size is fitted to obtain a fitting curve;

[0021] Identify the abrupt change points of the curve from the fitted curve;

[0022] The size of the face detection box corresponding to the mutation point is determined as the preset face size threshold.

[0023] In some possible implementations, the downstream task is a face recognition task, which calculates the similarity between each scaled face region image and a baseline face region image, including:

[0024] Extract scaled face features from each scaled face region image;

[0025] Calculate the similarity between each scaled face feature and the baseline face feature, which is the face feature extracted from the original face region image in the first original image sample.

[0026] In some possible implementations, the downstream task is a facial landmark detection task, which calculates the similarity between each scaled face region image and a baseline face region image, including:

[0027] Extract scaled facial key points from each scaled face region image;

[0028] The similarity between each scaled facial landmark and the baseline facial landmark is calculated. The baseline facial landmark is the facial landmark detected from the original face region image in the second original image sample.

[0029] In some possible implementations, the downstream task type is a subjective evaluation task by the human eye, and the methods also include:

[0030] Determine the initial face size threshold based on the characteristics of human vision;

[0031] Based on the operational characteristic curve method of the test object, the initial face size threshold is adjusted to obtain the minimum face size that is irrelevant to identification; the minimum face size that is irrelevant to identification is the preset face size threshold.

[0032] In some possible implementations, the first valid face is classified based on its out-of-focus blur score to obtain in-focus face categories and / or out-of-focus face categories, including:

[0033] The first effective face sequence is obtained by sorting the face blur scores of the first effective faces in descending order.

[0034] Calculate the gradient of the face blur score between every two adjacent valid faces in the first valid face sequence to obtain the gradient sequence;

[0035] The binary classification threshold is determined based on the gradient in the gradient sequence;

[0036] The first valid face with a blur score greater than the binary classification threshold is classified as an in-focus face, and the first valid face with a blur score less than or equal to the binary classification threshold is classified as an out-of-focus face.

[0037] In some possible implementations, the binary classification threshold is determined based on the gradient in the gradient sequence, including:

[0038] The gradient outlier value in the gradient sequence is detected by the quartile method, and the gradient outlier value is set as the i-th gradient in the gradient sequence.

[0039] The binary classification threshold is determined based on the face blur score of the i-th first valid face in the first valid face sequence.

[0040] In some possible implementations, outlier values ​​in the gradient sequence are detected using the quartile method, including:

[0041] Calculate the first and third quartiles of multiple gradients in a gradient sequence;

[0042] Calculate the interquartile range based on the first and third quartiles;

[0043] Outliers are determined based on the third quartile and the interquartile range.

[0044] On the other hand, embodiments of this application provide an effective face detection device, which includes:

[0045] The acquisition module is used to acquire the image of the person to be detected and the type of the downstream task;

[0046] The input module is used to input the image of the person to be detected into the effective face recognition model to obtain the face coordinate information of each face in the image of the person to be detected and the face blur score; the effective face recognition model includes a baseline face recognition model, and the prediction head part of the baseline face recognition model includes a recognition model and an image quality evaluation model; the recognition model is used to identify whether there is a face in the image, and the image quality evaluation model is used to score the blur score of the recognized face;

[0047] The determination module is used to determine the face size of each face based on the face coordinate information.

[0048] The filtering module is used to select the first valid face from all faces whose face size is greater than a preset face size threshold; the preset face size threshold is determined according to the downstream task type.

[0049] The classification module is used to classify the first effective faces according to the face blur score of the first effective faces when the number of first effective faces is greater than a preset threshold, so as to obtain the in-focus face category and / or out-of-focus face category.

[0050] The determination module is also used to determine the first valid face belonging to the in-focus face category as the target valid face.

[0051] In another aspect, embodiments of this application provide an effective face detection device, the device including: a processor and a memory storing computer program instructions;

[0052] The processor implements the effective face detection method described above when executing computer program instructions.

[0053] In another aspect, embodiments of this application provide a computer storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, they implement the effective face detection method described above.

[0054] In another aspect, embodiments of this application provide a computer program product in which instructions are executed by the processor of an electronic device, causing the electronic device to perform the effective face detection method described above.

[0055] The effective face detection method, apparatus, device, computer storage medium, and computer program product of this application embodiment, when performing effective face detection, firstly acquires the image of the person to be detected and the downstream task type; then, inputs the image of the person to be detected into the effective face recognition model to obtain the face coordinate information and face blur score of each face in the image of the person to be detected; then, determines the face size of the face based on the face coordinate information; and filters out the first effective faces whose face size is greater than a preset face size threshold; then, if the number of the first effective faces is greater than the preset threshold, classifies the first effective faces according to the face blur score of the first effective faces to obtain in-focus face category and / or out-of-focus face category; finally, the first effective face belonging to the in-focus face category is determined as the target effective face. On the one hand, this method integrates the recognition model and the image quality evaluation model into a single effective face recognition model, avoiding the problems of high computational overhead and low detection efficiency caused by serializing multiple independent models in traditional schemes. On the other hand, when screening effective faces, the preset face size threshold is not fixed, but can be dynamically determined adaptively according to the type of downstream task. By accurately screening out target effective faces that meet both size requirements and are clearly in focus, the amount of data to be processed by downstream tasks (such as face recognition and key point detection) is greatly reduced, which improves the efficiency of effective face detection while saving computational resources. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating an effective face detection method provided in one embodiment of this application;

[0058] Figure 2 This is a schematic diagram of the first process for determining a face size threshold according to an embodiment of this application;

[0059] Figure 3This is a schematic diagram of the second process for determining a face size threshold provided in one embodiment of this application;

[0060] Figure 4 This is a schematic diagram illustrating the relationship between face similarity and face image size according to one embodiment of this application;

[0061] Figure 5 This is a schematic diagram of a third process for determining a face size threshold provided in one embodiment of this application;

[0062] Figure 6 This is a flowchart illustrating S160 provided in one embodiment of this application;

[0063] Figure 7 This is a flowchart illustrating S1603 provided in one embodiment of this application;

[0064] Figure 8 This is a schematic diagram of the structure of an effective face detection device provided in another embodiment of this application;

[0065] Figure 9 This is a schematic diagram of the structure of an effective face detection device provided in another embodiment of this application. Detailed Implementation

[0066] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0067] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0068] Before introducing the embodiments of this application, the terminology involved in the embodiments of this application will be introduced first:

[0069] Valid Face: Refers to a face that is large enough and within the depth of field of a camera lens in a specific application scenario, and can provide valuable information for subsequent tasks (such as face recognition, key point detection, and subjective evaluation by the human eye).

[0070] Invalid Face: A face that is too small or too blurry to be used for subsequent tasks or to provide valuable information.

[0071] In-focus face: A face that is in sharp focus and within the depth of field of the camera lens.

[0072] Out-of-focus face: A face that is outside the depth of field of the camera lens and is blurred due to bokeh.

[0073] Maximum side length of face bounding box (face_max_side): The maximum value of the width or height of the face detection bounding box.

[0074] PSNR (Peak Signal-to-Noise Ratio): A full-reference image quality evaluation metric used to measure the difference between two images; a higher value indicates that the image quality is closer.

[0075] L1 distance (Manhattan distance): The sum of the absolute differences in each dimension between two vectors, used to measure the similarity of facial feature vectors or key point coordinates. The smaller the value, the more similar they are.

[0076] The ROC method (Receiver Operating Characteristic) assesses diagnostic accuracy or image quality by statistically analyzing observer judgments.

[0077] Currently, effective face detection schemes have high computational costs and their detection efficiency needs to be improved.

[0078] To address the problems of the prior art, embodiments of this application provide an effective face detection method, apparatus, device, computer storage medium, and computer program product. The effective face detection method provided in this application embodiment is described below.

[0079] Figure 1 A flowchart illustrating an effective face detection method provided in one embodiment of this application is shown. Figure 1As shown, the effective face detection method in this application embodiment includes the following steps S110-S180.

[0080] S110, Obtain the image of the person to be detected and the downstream task type.

[0081] For example, the image of the person to be detected can be an image containing a face, and the downstream task type is used to indicate the purpose of the image of the person to be detected. As an example, the downstream task type can be a face recognition task, a facial landmark detection task, and a human eye subjective evaluation task.

[0082] For example, a face recognition task could be used to identify who is looking at a face; a facial landmark detection task could be used to locate the positions of key facial features, such as the eyes, nose, and mouth. A subjective evaluation task is used to present a facial image to others, requiring the image to be visually clear.

[0083] S120. Input the image of the person to be detected into the effective face recognition model to obtain the face coordinate information of each face in the image of the person to be detected and the face blur score.

[0084] For example, an effective face recognition model can be a multi-task learning model, which includes a baseline face recognition model, such as the YOLO series or any neural network model like RetainFace.

[0085] The prediction head of the baseline face recognition model includes a recognition model and an image quality assessment model; the recognition model is used to identify whether a face exists in the image, and the image quality assessment model is used to score the out-of-focus blur of the recognized face.

[0086] It should be noted that traditional benchmark face recognition models only output face coordinates and confidence scores. In this embodiment, the prediction head of the benchmark face recognition model has been improved. The prediction head not only includes the recognition model but also incorporates an image quality assessment model. Upon detecting a face, it can directly provide a score indicating whether the face is out of focus. This score quantifies the sharpness of the face; a higher score generally indicates greater sharpness (in-focus), while a lower score indicates greater blurriness (out-of-focus).

[0087] For example, when using image quality assessment models to score the out-of-focus blur of recognized faces, the corresponding image quality assessment models differ depending on the type of downstream task. As an example, when the downstream task is face recognition, image quality assessment models related to face recognition, such as FaceQNet, can be selected; when the downstream task is facial landmark detection or directly related to subjective human evaluation, general image quality assessment models such as LIQE can be used.

[0088] As an example, when selecting an image quality assessment model, it is also necessary to create a blurred background dataset. For instance, the method for creating a blurred background dataset is as follows: The subject and background figures are approximately 3 meters apart. A fixed-position, fixed-focus lens is used to focus on the subject, ensuring their face is in focus. Then, different apertures are used to take pictures, obtaining images with varying degrees of blur. After shooting, the blurred background faces are cropped and sorted according to aperture from smallest to largest, with the degree of blurriness increasing from clear to sharp. The image quality assessment model is then used to score the face sequence, which is a sequence of background figures from multiple images. If the score results are monotonically increasing or decreasing, it indicates that the image quality assessment model meets the requirements.

[0089] S130. For each face, determine the face size based on the face coordinate information.

[0090] After inputting the image of the person to be detected into the effective face recognition model, the model outputs the coordinate information of all faces in the image. ,in,( () represents the coordinates of the top-left corner of the face detection bounding box. This represents the coordinates of the bottom right corner of the face detection bounding box.

[0091] When determining the face size based on face coordinate information, for example, the face size is the maximum side length of the face detection bounding box. That is, the maximum value of the width and height of the face detection box.

[0092]

[0093] S140. From all faces, select the first valid face whose face size is greater than the preset face size threshold.

[0094] When filtering valid faces based on face size, different valid face size thresholds can be set according to the downstream task type. .

[0095]

[0096] in, The determination process can be found in subsequent steps S210-S280, through... It can filter out valid faces that meet the size requirements.

[0097] It should be noted that the preset face size threshold is not fixed, but can be determined based on the type of downstream task. This is because different tasks have different requirements for image quality. For face recognition tasks, a sufficient number of pixels are needed to extract subtle features, so the size threshold is set to the highest level. For facial landmark detection, the facial features need to be clearly visible, so the threshold can be set to a medium level. For subjective evaluation tasks, the requirements are relatively low; as long as the human eye can clearly see who it is, the threshold is set to the lowest level.

[0098] S150. Determine whether the number of the first valid faces is greater than the preset threshold.

[0099] After selecting the first valid face from each face, it is necessary to further determine the number of the first valid faces. If the number of the first valid faces is not greater than a preset threshold, execute S160; if the number of the first valid faces is greater than the preset threshold, execute S170.

[0100] S160, Output the valid face of the target.

[0101] As a more concrete example, the preset threshold can be 4. It should be noted that if the number of "first valid faces" is too small (e.g., only 1-3), there is no need to perform complex sharpness analysis. It can be assumed that all first valid faces are valid faces, and all first valid faces can be directly output as target valid faces.

[0102] S170. Based on the facial blur score of the first effective face, classify the first effective face to obtain the in-focus face category and / or out-of-focus face category.

[0103] If the number of first valid faces exceeds a preset threshold, the first valid faces can be classified into in-focus faces and / or out-of-focus faces based on their out-of-focus blur rating. In-focus faces are those that are clearly in focus and within the depth of field of the camera lens, while out-of-focus faces are those that are blurred due to bokeh and outside the depth of field of the camera lens.

[0104] S180. The first valid face belonging to the in-focus face category is identified as the target valid face.

[0105] For example, a clearly focused face within the depth of field of the camera lens can be output as the target valid face.

[0106] The effective face detection method, apparatus, device, computer storage medium, and computer program product of this application embodiment, when performing effective face detection, firstly acquires the image of the person to be detected and the downstream task type; then, inputs the image of the person to be detected into the effective face recognition model to obtain the face coordinate information and face blur score of each face in the image of the person to be detected; then, determines the face size of the face based on the face coordinate information; and filters out the first effective faces whose face size is greater than a preset face size threshold; then, if the number of the first effective faces is greater than the preset threshold, classifies the first effective faces according to the face blur score of the first effective faces to obtain in-focus face category and / or out-of-focus face category; finally, the first effective face belonging to the in-focus face category is determined as the target effective face. On the one hand, this method integrates the recognition model and the image quality evaluation model into a single effective face recognition model, avoiding the problems of high computational overhead and low detection efficiency caused by serializing multiple independent models in traditional schemes. On the other hand, when screening effective faces, the preset face size threshold is not fixed, but can be dynamically determined adaptively according to the type of downstream task. By accurately screening out target effective faces that meet both size requirements and are clearly in focus, the amount of data to be processed by downstream tasks (such as face recognition and key point detection) is greatly reduced, which improves the efficiency of effective face detection while saving computational resources.

[0107] To determine the preset face size threshold, as an example, such as... Figure 2 As shown, the method may further include steps S210-S280.

[0108] S210. Obtain image sample groups.

[0109] For example, the image sample group may include at least one original image sample. In order to improve detection accuracy, the original image sample may be a high-resolution human image. The image sample needs to contain a clear frontal human face and the face size should be large to ensure that the face can still be detected with high quality in the initial scaling stage. In addition, the image sample needs to cover different genders, ages and skin colors to ensure the comprehensiveness and representativeness of the sample.

[0110] S220. For each original image sample, scale the original image sample multiple times with different scaling ratios in order of increasing scaling ratio to obtain multiple scaled image samples.

[0111] For example, image samples can be scaled proportionally. As a more specific example, each image can be scaled proportionally in sequence, with each scaling ratio being 99.5% of the original size, until the image size is reduced to 1% of the original image, thus forming a single control variable. Except for the resolution decreasing, i.e. the face size changing, the range and proportion of the rest of the face image remain unchanged.

[0112] S230. Based on the size of the original face detection box in the original image sample, scale it according to multiple different scaling ratios to obtain the scaled face detection box and its size corresponding to each of the scaled image samples.

[0113] After each scaling step, the detection box coordinates are scaled proportionally based on the face detection results of the original image, resulting in the new position and size of the face box in the scaled image. .

[0114] S240. Based on each scaled face detection box, crop the scaled face region image from the corresponding scaled image sample.

[0115] By cropping out the facial region from each scaled image, you can obtain a series of facial images of the same person with the same expression, but with continuously varying resolutions (sizes).

[0116] S250. Calculate the similarity between each scaled face region image and the baseline face region image.

[0117] For example, the baseline face region image is the original face region image in the original image sample, which is the largest and clearest face cropped from the original high-definition image, representing the best image quality.

[0118] When calculating similarity, each scaled and cropped image can be input into a downstream task model (such as a face recognition model) to extract its features and compare them with the features of a baseline face to calculate the similarity. For example, for face recognition, L1 distance or cosine distance is commonly used for similarity calculation; the larger the distance, the lower the similarity. For keypoint detection, the error in the keypoint coordinates can be calculated.

[0119] S260. Based on multiple similarities and their corresponding scaled face detection box sizes, fit the relationship between similarity and face detection box size to obtain a fitting curve.

[0120] After obtaining multiple pairs of face detection bounding box sizes and similarity data, these data pairs are plotted in a coordinate system, and a smooth curve is fitted using polynomial fitting. This curve visually demonstrates the law of detection performance decay with size.

[0121] S270. Determine the abrupt change points of the curve from the fitted curve.

[0122] For example, in regions with large dimensions, the curve is very flat (performance remains almost unchanged as the size decreases); however, when the size shrinks to a certain point, the curve begins to show an "elbow," and performance drops sharply. This "elbow" is the inflection point.

[0123] S280. Determine the size of the face detection box corresponding to the mutation point as the preset face size threshold.

[0124] Read the value corresponding to the mutation point on the X-axis. This value is a preset face size threshold. When the face size is greater than the threshold, it means that the downstream task can work stably, and the face is a valid face; when the face size is smaller than the threshold, the downstream task performance is unreliable, and the face is an invalid face.

[0125] It's important to note that traditional methods rely on manual experience to set fixed size thresholds, which are highly subjective and lack data support. This new method constructs a "single-variable" experimental environment ranging from high-resolution original images to continuously scaled images, and quantitatively analyzes the performance degradation with size. This transforms the threshold determination process from a subjective decision-making process into an objective data analysis task, ensuring the scientific validity and reliability of the obtained threshold and avoiding problems caused by manually setting it too high (missing valid faces) or too low (incomplete screening). Furthermore, the core of this method lies in directly measuring the response of key indicators for downstream tasks (such as recognition similarity and keypoint error) to changes in face size. By fitting curves and locating performance mutation points, the determined threshold closely matches the actual performance boundaries of downstream tasks. This means that "valid faces" selected based on this threshold can effectively guarantee the success rate and accuracy of downstream tasks (such as face recognition and keypoint detection), thereby greatly improving the effectiveness of the entire valid face selection scheme.

[0126] It should be noted that the preset face size threshold is determined based on the downstream task type. For example, the downstream task type can be a face recognition task, a face landmark detection task, or a human eye subjective evaluation task.

[0127] In the case where the downstream task is a face recognition task, as an example, when calculating the similarity between each scaled face region image and the baseline face region image, firstly, scaled face features are extracted from each scaled face region image; then, the similarity between each scaled face feature and the baseline face feature is calculated, where the baseline face feature is the face feature extracted from the original face region image in the first original image sample.

[0128] As an example, such as Figure 3 As shown, when the downstream task type is a face recognition task, the preset face size threshold is determined in the following way:

[0129] S310, Obtain the first image sample group.

[0130] For example, the first image sample group includes at least one first original image sample;

[0131] S320. Scale the first original image sample by multiple different scaling ratios in order of increasing scaling ratio to obtain multiple scaled first image samples.

[0132] S330. Based on the size of the original face detection box in the first original image sample, scale it according to multiple different scaling ratios to obtain the scaled face detection box and its size corresponding to each of the multiple scaled first image samples.

[0133] S340. Based on each scaled face detection box, crop the scaled face region image from the corresponding scaled first image sample.

[0134] S350. Extract scaled face features from each scaled face region image.

[0135] For example, each scaled face region image can be input into a face recognition model to extract scaled face features.

[0136] S360. Calculate the similarity between each scaled face feature and the baseline face feature.

[0137] For example, the baseline facial features are facial features extracted from the original facial region image in the first original image sample.

[0138] When calculating the similarity between each scaled face feature and the baseline face feature, the Euclidean distance (L1 distance) between each scaled face feature and the baseline face feature can be calculated as a face recognition similarity evaluation index.

[0139]

[0140] Where A and B are the N-dimensional facial feature vectors obtained by the face recognition model through inference calculation of the reference face image and the proportionally scaled-down face image, respectively. L1(A,B) is used to evaluate similarity, ranging from 0 to positive infinity, and the smaller the Euclidean distance between the two, the more similar they are.

[0141] S370. Based on multiple similarities and their respective scaled face detection box sizes, fit the relationship between similarity and face detection box size to obtain the first fitting curve.

[0142] According to L1 and The correspondence is obtained by fitting the data using polynomials or other methods to obtain L1 and Approximate calculation formula:

[0143]

[0144] Where y represents L1, x represents a, b, c, d, e, f are the polynomial fitting parameters, and T1 and C1 represent constants.

[0145] S380. Determine the first abrupt change point of the curve from the first fitted curve.

[0146] like Figure 4 As shown, Figure 4 This is a schematic diagram of the first fitted curve, used to represent the relationship between face similarity and face image size. The horizontal axis represents face size, and the vertical axis represents the similarity to a reference face (L1). The fitting results clearly show that the curve has an inflection point, denoted as [insert inflection point here]. ,exist Greater than At this time, the similarity (L1) between the scaled facial features and the baseline facial features is relatively small; the smaller the similarity, the more similar the facial recognition model (the region of good performance of the facial recognition model, i.e., the facial recognition model has high recognition accuracy); this indicates that in Greater than At that time, facial recognition accuracy was independent of face size; however, in Less than The nonlinear surge in the curve indicates that the model's recognition ability is completely ineffective at this small face image size. Correspondingly, the curve will show an "inflection point (at C1)" or a "cliff-like drop" (the area of ​​rapid performance deterioration). This inflection point C1 is the critical position where the model's performance changes from quantitative to qualitative change, and this "mutation point" is the signal that the model begins to fail.

[0147] S390. Determine the size of the face detection box corresponding to the first mutation point as the preset face size threshold.

[0148] For example, a constant can be As a threshold for determining the effective face size for face recognition tasks .

[0149] It's important to note that this method doesn't use general image quality metrics. Instead, it directly uses the downstream face recognition model itself for similarity calculations. This ensures that the final similarity curve directly reflects the impact of face size variations on the performance of a specific recognition model. The size threshold determined in this way effectively guarantees that for faces larger than this threshold, when input into the recognition model, sufficiently stable and accurate features can be extracted, thus ensuring a high success rate and reliability for subsequent face recognition tasks. Furthermore, different pre-trained face recognition models (such as ArcFace and FaceNet) exhibit significant differences in robustness to low-resolution faces due to variations in their network structure and training data. By changing the baseline model used for feature extraction, a unique optimal size threshold can be "tailor-made" for each specific face recognition model, maximizing the performance potential of each model.

[0150] In the case where the downstream task is facial landmark detection, as an example, when calculating the similarity between each scaled face region image and the baseline face region image, firstly, scaled facial landmarks are extracted from each scaled face region image. As an example, facial landmarks can be key points unrelated to facial expressions, such as stable points like the corners of the eyes, the alar points of the nose, and the corners of the mouth. Then, the similarity between each scaled facial landmark and the baseline facial landmarks, which are facial landmarks detected from the original face region images in the second original image sample, is calculated.

[0151] As an example, such as Figure 5 As shown, when the downstream task type is facial landmark detection, the preset face size threshold is determined in the following way:

[0152] S610, Obtain the second image sample group.

[0153] The second image sample group includes at least one second original image sample.

[0154] S620. Scale the second original image sample by multiple different scaling ratios in ascending order of scaling ratio to obtain multiple scaled second image samples.

[0155] S630. Based on the size of the original face detection box in the second original image sample, scale it according to multiple different scaling ratios to obtain the scaled face detection box and its size corresponding to each of the multiple scaled second image samples.

[0156] S640. Based on each scaled face detection box, crop the scaled face region image from the corresponding scaled second image sample.

[0157] S650: Extract scaled facial key points from each scaled facial region image.

[0158] As an example, facial landmarks can be landmarks unrelated to facial expressions. By selecting expression-independent landmarks, on the one hand, errors in results caused by differences in facial expressions in the test images are avoided; on the other hand, expression-independent landmarks can be used for calculations such as facial pose estimation, ensuring the practical value of the landmarks used in the calculation.

[0159] S660. Calculate the similarity between each scaled facial landmark and the baseline facial landmark.

[0160] For example, the reference facial landmarks are facial landmarks detected from the original facial region image in the second original image sample.

[0161] When calculating the similarity between each scaled facial landmark and the baseline facial landmark, the Euclidean distance S between each scaled facial landmark and the baseline facial landmark can be calculated as an evaluation index for facial landmark detection error.

[0162]

[0163] Where n is the number of facial expression-irrelevant key points, and k is the coordinate dimension. Let represent the coordinates of the d-th dimension of the i-th expression-irrelevant keypoint in the reference face image and the proportionally scaled-down face image, respectively. S is used to evaluate the keypoint detection error, ranging from 0 to positive infinity; the smaller the Euclidean distance between the two images, the more similar they are.

[0164] S670. Based on multiple similarities and their corresponding scaled face detection box sizes, fit the relationship between similarity and face detection box size to obtain the second fitting curve.

[0165] Based on the key point detection error S and The correspondence between them is also obtained by fitting the data using polynomials or other methods to obtain S and Approximate calculation formula:

[0166]

[0167] Where y represents S, and x represents a, b, c, d, e, f are the polynomial fitting parameters, and T2 and C2 represent constants.

[0168] S480. Determine the second abrupt change point of the curve from the second fitted curve.

[0169] It should be noted that the second fitted curve is similar to the first fitted curve, and specific attached figures are not provided here. The trend of the fitted curve and the position of the inflection point can be found by referring to [the original text]. Figure 4 The fitting results also clearly show an inflection point, denoted as . ,exist Greater than At that time, the accuracy of facial landmark detection is independent of face size, and is smaller than 1000 mm. Nonlinear surges sometimes occur.

[0170] S490. Determine the size of the face detection box corresponding to the second mutation point as the preset face size threshold.

[0171] For example, the constant C2 can be used as a threshold for determining the effective face size for a facial landmark detection task. .

[0172] It should be noted that this method directly uses the output of the downstream facial landmark detection model for similarity calculation, so that the final evaluation curve directly reflects the impact of facial size changes on landmark localization accuracy. The size threshold determined in this way can most effectively ensure that the landmark detection results of all faces larger than this threshold can reach an acceptable accuracy range, thereby directly guaranteeing the accuracy and stability of downstream facial landmark detection.

[0173] When the downstream task type is a subjective evaluation task, an initial face size threshold can be determined based on the characteristics of human vision. Then, based on the operating characteristic curve (ROC) method of the test object, the initial face size threshold is adjusted to obtain the minimum face size that is irrelevant to identification. The minimum face size that is irrelevant to identification is the preset face size threshold.

[0174] Based on the characteristics of human vision, the angular resolution of the human eye is approximately 1 arcminute (about 0.0167 degrees). At a normal visual distance (about 25 cm), the smallest object size that can be distinguished is about 70 micrometers. Actual resolution requires at least 3 pixels to form a recognizable shape (one pixel each for the pupil and upper and lower eyelids). Using the operating characteristic curve (ROC) method based on the test object, and through subjective quality evaluation, the facial recognition quality of face images of different sizes was assessed to determine the smallest recognizable irrelevant face size. .

[0175] By setting initial thresholds based on the visual resolution of the human eye (such as angular resolution and minimum distinguishable pixel count), the final size standard is matched with the actual perceptual ability of human observers. This ensures that the selected "effective faces" meet the basic requirements for clear facial recognition by the human eye, thereby guaranteeing the image quality for subjective tasks such as display, browsing, and review.

[0176] As an example, such as Figure 6 As shown, S160 may include:

[0177] 1601. Based on the facial blur score of the first valid face, sort them in descending order to obtain the first valid face sequence.

[0178] The first valid faces selected (i.e. faces that meet the size requirements) are sorted from high to low according to their blur score. The higher the score, the clearer the face, resulting in a face sequence from "sharpest" to "blurries".

[0179] 1602. Calculate the gradient of the face blur score between every two adjacent valid faces in the first valid face sequence to obtain the gradient sequence.

[0180] For example, the gradient is used to represent the drasticness of the rating change. The difference between the ratings of two adjacent faces in the sequence (i.e., the gradient) is calculated to obtain the gradient sequence.

[0181] For example: in the sequence, the first place has a score of 100, the second place has a score of 98, and the gradient is 2; the second place has a score of 98, the third place has a score of 60, and the gradient is 38. This gradient sequence reflects the "smoothness" or "jump" of the decrease in sharpness.

[0182] 1603. Determine the binary classification threshold based on the gradient in the gradient sequence.

[0183] As an example, such as Figure 7 As shown, S1603 may include:

[0184] S16031. Detect gradient outliers in the gradient sequence using the quartile method, and set the gradient outlier as the i-th gradient in the gradient sequence.

[0185] As an example, when detecting gradient outliers in a gradient sequence using the quartile method, the gradient sequence is analyzed to find points that indicate a "cliff-like drop" in sharpness. Typically, the quartile method is used to identify outliers in the gradient.

[0186] First, calculate the first and third quartiles of multiple gradients in the gradient sequence; then, calculate the interquartile range based on the first and third quartiles; finally, determine the outliers based on the third quartile and the interquartile range.

[0187] Since the first quartile, third quartile, and interquartile range are all dynamically calculated from the gradient sequence of the current image, the outlier determination criterion of this method is entirely determined by the current data distribution. This allows it to automatically adapt to the overall differences and dispersion of sharpness scores between different images. Regardless of whether the gradient values ​​are generally large or small, it can automatically adjust the sensitivity and has extremely strong generalization ability.

[0188] The specific calculation steps for the quartile method are as follows:

[0189] First quartile (Q1): Represents the value less than or equal to 25% of the data points.

[0190] The third quartile (Q3): represents the value that 75% of the data points are less than or equal to.

[0191] Interquartile Range (IQR): IQR = Q3 - Q1 = 3 - 1 = 2.

[0192] Calculate outlier boundaries:

[0193] Lower limit: Q1 - 1.5 × IQR

[0194] Upper limit: Q3 + 1.5 × IQR

[0195] In the current task, any gradient greater than Q3 + 1.5 × IQR is considered an outlier.

[0196] S16032. Determine the binary classification threshold based on the face blur score of the i-th first valid face in the first valid face sequence.

[0197] For example, from all outliers, select the outlier with the largest gradient value (i.e., the most significant jump point). Let i be the gradient index corresponding to this outlier. i (i.e., G[i]) G [ i [This is an outlier]. The first outlier indicates an unusually large change in sharpness at that point, and the score corresponding to this point is the ideal binary classification threshold.

[0198] By automatically detecting outliers in gradient sequences using quartile statistics, the process of determining the binary classification threshold is transformed into a purely mathematical calculation problem. This avoids the subjectivity and arbitrariness caused by relying on human experience to set the threshold, ensuring the objectivity and consistency of each classification decision and improving the repeatability and reliability of the method.

[0199] 1604. Determine whether the facial blur score is greater than the binary classification threshold.

[0200] If the face blur score is greater than the binary classification threshold, execute S1605; if the face blur score is less than or equal to the binary classification threshold, execute S1606.

[0201] For example, the median or other statistics can also be determined as a threshold based on the gradient in the gradient sequence.

[0202] 1605. Classify the first valid face into the in-focus face category.

[0203] 1606. Classify the first valid face into the out-of-focus face category.

[0204] For example, a face sequence is divided into two parts by a found dynamic threshold. Faces above the threshold (i.e., the sharpest part) are classified as in-focus faces, and faces at or below the threshold are classified as out-of-focus faces.

[0205] By analyzing the relative distribution and gradient changes of clear faces within a single image, a binary classification threshold is dynamically determined. This allows the classification criteria to automatically adapt to different shooting scenarios, lens optical characteristics (such as aperture and focal length), and the photographer's creative intent. Regardless of whether the overall image is clear or blurry, it can accurately identify the "relatively clearest" focal person, thus solving the key problem of poor generalization ability with fixed thresholds.

[0206] The effective face detection method, apparatus, device, computer storage medium, and computer program product of this application embodiment, when performing effective face detection, firstly acquires the image of the person to be detected and the downstream task type; then, inputs the image of the person to be detected into the effective face recognition model to obtain the face coordinate information and face blur score of each face in the image of the person to be detected; then, determines the face size of the face based on the face coordinate information; and filters out the first effective faces whose face size is greater than a preset face size threshold; then, if the number of the first effective faces is greater than the preset threshold, classifies the first effective faces according to the face blur score of the first effective faces to obtain in-focus face category and / or out-of-focus face category; finally, the first effective face belonging to the in-focus face category is determined as the target effective face. On the one hand, this method integrates the recognition model and the image quality evaluation model into a single effective face recognition model, avoiding the problems of high computational overhead and low detection efficiency caused by serializing multiple independent models in traditional schemes. On the other hand, when screening effective faces, the preset face size threshold is not fixed, but can be dynamically determined adaptively according to the type of downstream task. By accurately screening out target effective faces that meet both size requirements and are clearly in focus, the amount of data to be processed by downstream tasks (such as face recognition and key point detection) is greatly reduced, which improves the efficiency of effective face detection while saving computational resources.

[0207] Based on the effective face detection method provided in the above embodiments, this application also provides specific implementations of an effective face detection device. Please refer to the following embodiments.

[0208] First see Figure 8 The effective face detection device 900 provided in this application embodiment includes the following modules:

[0209] The acquisition module 910 is used to acquire the image of the person to be detected and the type of the downstream task.

[0210] The input module 920 is used to input the image of the person to be detected into the effective face recognition model to obtain the face coordinate information of each face in the image of the person to be detected and the face blur score; the effective face recognition model includes a baseline face recognition model, and the prediction head part of the baseline face recognition model includes a recognition model and an image quality evaluation model; the recognition model is used to identify whether there is a face in the image, and the image quality evaluation model is used to score the blur score of the recognized face;

[0211] The determination module 930 is used to determine the face size of each face based on the face coordinate information.

[0212] The filtering module 940 is used to filter out the first valid face from the various faces whose face size is greater than a preset face size threshold; the preset face size threshold is determined according to the downstream task type;

[0213] The classification module 950 is used to classify the first effective faces according to the face blur score of the first effective faces when the number of first effective faces is greater than a preset threshold, so as to obtain the in-focus face category and / or out-of-focus face category.

[0214] The determination module 930 is also used to determine the first valid face belonging to the in-focus face category as the target valid face.

[0215] In some possible implementations, module 910 is also used for:

[0216] Obtain an image sample group, which includes at least one original image sample;

[0217] For each original image sample, the original image sample is scaled multiple times with different scaling ratios in order of increasing scaling ratio to obtain multiple scaled image samples.

[0218] Based on the size of the original face detection box in the original image sample, the scaled face detection box and its size are obtained by scaling the image sample according to multiple different scaling ratios.

[0219] Based on each scaled face detection bounding box, the scaled face region image is cropped from the corresponding scaled image sample;

[0220] Calculate the similarity between each scaled face region image and the baseline face region image, where the baseline face region image is the original face region image in the original image sample.

[0221] Based on multiple similarities and their corresponding scaled face detection box sizes, the relationship between similarity and face detection box size is fitted to obtain a fitting curve;

[0222] Identify the abrupt change points of the curve from the fitted curve;

[0223] The size of the face detection box corresponding to the mutation point is determined as the preset face size threshold.

[0224] In some possible implementations, the downstream task is a face recognition task, which calculates the similarity between each scaled face region image and a baseline face region image, including:

[0225] Extract scaled face features from each scaled face region image;

[0226] Calculate the similarity between each scaled face feature and the baseline face feature, which is the face feature extracted from the original face region image in the first original image sample.

[0227] In some possible implementations, the downstream task is a facial landmark detection task, which calculates the similarity between each scaled face region image and a baseline face region image, including:

[0228] Extract scaled facial key points from each scaled face region image;

[0229] The similarity between each scaled facial landmark and the baseline facial landmark is calculated. The baseline facial landmark is the facial landmark detected from the original face region image in the second original image sample.

[0230] In some possible implementations, the downstream task type is a human subjective evaluation task, and module 930 is also used for:

[0231] Determine the initial face size threshold based on the characteristics of human vision;

[0232] Based on the operational characteristic curve method of the test object, the initial face size threshold is adjusted to obtain the minimum face size that is irrelevant to identification; the minimum face size that is irrelevant to identification is the preset face size threshold.

[0233] In some possible implementations, the classification module 950 is also used for:

[0234] The first effective face sequence is obtained by sorting the face blur scores of the first effective faces in descending order.

[0235] Calculate the gradient of the face blur score between every two adjacent valid faces in the first valid face sequence to obtain the gradient sequence;

[0236] The binary classification threshold is determined based on the gradient in the gradient sequence;

[0237] The first valid face with a blur score greater than the binary classification threshold is classified as an in-focus face, and the first valid face with a blur score less than or equal to the binary classification threshold is classified as an out-of-focus face.

[0238] In some possible implementations, module 930 is also used for:

[0239] The gradient outlier value in the gradient sequence is detected by the quartile method, and the gradient outlier value is set as the i-th gradient in the gradient sequence.

[0240] The binary classification threshold is determined based on the face blur score of the i-th first valid face in the first valid face sequence.

[0241] In some possible implementations, outlier values ​​in the gradient sequence are detected using the quartile method, including:

[0242] Calculate the first and third quartiles of multiple gradients in a gradient sequence;

[0243] Calculate the interquartile range based on the first and third quartiles;

[0244] Outliers are determined based on the third quartile and the interquartile range.

[0245] The various modules of the effective face detection device provided in this application embodiment can achieve... Figure 1 The functions of each step in the provided effective face detection method, and the corresponding technical effects they achieve, will not be elaborated here for the sake of brevity.

[0246] Figure 9 A schematic diagram of the hardware structure of the effective face detection device provided in the embodiments of this application is shown.

[0247] An effective face detection device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0248] Specifically, the processor 1001 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0249] Memory 1002 may include mass storage for data or instructions. For example, and not limitingly, memory 1002 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1002 may include removable or non-removable (or fixed) media. Where appropriate, memory 1002 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1002 is non-volatile solid-state memory.

[0250] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0251] The processor 1001 reads and executes computer program instructions stored in the memory 1002 to implement any of the effective face detection methods in the above embodiments.

[0252] In one example, the effective face detection device may further include a communication interface 1003 and a bus 1010. For example, Figure 9 As shown, the processor 1001, memory 1002, and communication interface 1003 are connected through bus 1010 and complete communication with each other.

[0253] The communication interface 1003 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0254] Bus 1010 includes hardware, software, or both, that couples components of a device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1010 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0255] Furthermore, in conjunction with the effective face detection methods described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the effective face detection methods described in the above embodiments.

[0256] This application also provides a computer program product, including a computer program that, when executed, implements any of the effective face detection methods described in the above embodiments.

[0257] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0258] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0259] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0260] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0261] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An effective face detection method, characterized in that, include: Obtain the image of the person to be detected and the type of downstream task; The image of the person to be detected is input into an effective face recognition model to obtain the face coordinate information and the face blur score of each face in the image of the person to be detected; the effective face recognition model includes a baseline face recognition model, and the prediction head part of the baseline face recognition model includes a recognition model and an image quality evaluation model; the recognition model is used to identify whether there is a face in the image, and the image quality evaluation model is used to score the blur of the recognized face; For each face, the face size is determined based on the face coordinate information; From the various faces, a first valid face whose size is greater than a preset face size threshold is selected; the preset face size threshold is determined according to the downstream task type. If the number of the first valid faces is greater than a preset threshold, the first valid faces are classified according to the face blur score of the first valid faces to obtain in-focus face category and / or out-of-focus face category. The first valid face belonging to the in-focus face category is determined as the target valid face; The step of classifying the first valid face based on its out-of-focus blur score to obtain in-focus face categories and / or out-of-focus face categories includes: The first valid face sequence is obtained by sorting the face blur scores of the first valid faces in descending order. Calculate the gradient of the face blur score between every two adjacent valid faces in the first valid face sequence to obtain the gradient sequence; The binary classification threshold is determined based on the gradient in the gradient sequence; Faces with out-of-focus blur scores greater than the first valid face in the binary classification threshold are classified as in-focus faces, and faces with out-of-focus blur scores less than or equal to the first valid face in the binary classification threshold are classified as out-of-focus faces. The step of determining the binary classification threshold based on the gradient in the gradient sequence includes: The gradient outlier value in the gradient sequence is detected by the quartile method, and the gradient outlier value is set as the i-th gradient in the gradient sequence; The binary classification threshold is determined based on the face blur score of the i-th first valid face in the first valid face sequence.

2. The effective face detection method according to claim 1, characterized in that, The method further includes: Obtain an image sample group, wherein the image sample group includes at least one original image sample; For each original image sample, the original image sample is scaled by multiple different scaling ratios in order of increasing scaling ratio to obtain multiple scaled image samples. Based on the size of the original face detection box in the original image sample, the scaled face detection box and its size are obtained according to the multiple different scaling ratios for each of the multiple scaled image samples. Based on each of the scaled face detection boxes, the scaled face region image is cropped from the corresponding scaled image sample; Calculate the similarity between each scaled face region image and the reference face region image, where the reference face region image is the original face region image in the original image sample; Based on the multiple similarities and their respective corresponding scaled face detection box sizes, the relationship between the similarity and the face detection box size is fitted to obtain a fitting curve; Determine the abrupt change points of the curve from the fitted curve; The size of the face detection box corresponding to the mutation point is determined as a preset face size threshold.

3. The effective face detection method according to claim 2, characterized in that, The downstream task type is a face recognition task, and the step of calculating the similarity between each scaled face region image and the baseline face region image includes: Scaled face features are extracted from each of the scaled face region images; The similarity between each scaled face feature and the baseline face feature is calculated. The baseline face feature is a face feature extracted from the original face region image in the first original image sample.

4. The effective face detection method according to claim 2, characterized in that, The downstream task type is a facial landmark detection task, and the step of calculating the similarity between each scaled face region image and the baseline face region image includes: Scaled facial key points are extracted from each of the scaled facial region images; The similarity between each scaled facial landmark and a baseline facial landmark is calculated. The baseline facial landmark is a facial landmark detected from the original facial region image in the second original image sample.

5. The effective face detection method according to claim 1, characterized in that, The downstream task type is a subjective evaluation task based on human vision, and the method further includes: Determine the initial face size threshold based on the characteristics of human vision; Based on the operational characteristic curve method of the test object, the initial face size threshold is adjusted to obtain the minimum face size that is irrelevant to identification; the minimum face size that is irrelevant to identification is a preset face size threshold.

6. The effective face detection method according to claim 1, characterized in that, The step of detecting gradient outliers in the gradient sequence using the quartile method includes: Calculate the first and third quartiles of multiple gradients in the gradient sequence; Calculate the interquartile range based on the first quartile and the third quartile; Outlier values ​​are determined based on the third quartile and the interquartile range.

7. An effective face detection device, characterized in that, The device includes: The acquisition module is used to acquire the image of the person to be detected and the type of the downstream task; The input module is used to input the image of the person to be detected into the effective face recognition model to obtain the face coordinate information and the face blur score of each face in the image of the person to be detected; the effective face recognition model includes a baseline face recognition model, and the prediction head part of the baseline face recognition model includes a recognition model and an image quality evaluation model; the recognition model is used to identify whether there is a face in the image, and the image quality evaluation model is used to score the blur of the identified face; The determination module is used to determine the face size of each face based on the face coordinate information. The filtering module is used to filter out the first valid faces from the various faces whose face size is greater than a preset face size threshold; the preset face size threshold is determined according to the downstream task type; The classification module is used to classify the first effective faces according to the face blur score of the first effective faces when the number of the first effective faces is greater than a preset threshold, so as to obtain the in-focus face category and / or out-of-focus face category. The determining module is further configured to determine the first valid face belonging to the in-focus face category as the target valid face; The classification module is also used for: The first valid face sequence is obtained by sorting the face blur scores of the first valid faces in descending order. Calculate the gradient of the face blur score between every two adjacent valid faces in the first valid face sequence to obtain the gradient sequence; The binary classification threshold is determined based on the gradient in the gradient sequence; Faces with out-of-focus blur scores greater than the first valid face in the binary classification threshold are classified as in-focus faces, and faces with out-of-focus blur scores less than or equal to the first valid face in the binary classification threshold are classified as out-of-focus faces. The step of determining the binary classification threshold based on the gradient in the gradient sequence includes: The gradient outlier value in the gradient sequence is detected by the quartile method, and the gradient outlier value is set as the i-th gradient in the gradient sequence; The binary classification threshold is determined based on the face blur score of the i-th first valid face in the first valid face sequence.

8. An effective face detection device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the effective face detection method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the effective face detection method as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the effective face detection method as described in any one of claims 1-6.

Images