Face detection method and related device

Abstract

The embodiment of the present application provides a face detection method, comprising: acquiring a to-be-detected image, wherein the to-be-detected image comprises a truncated face; performing multi-scale feature extraction on the to-be-detected image to obtain multi-scale intermediate features of the to-be-detected image, wherein the multi-scale features comprise intermediate features of a plurality of preset scales; performing face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the to-be-detected image, wherein the face detection result comprises a truncated face estimation result. The face detection and truncated face estimation processing are performed on the multi-scale intermediate features of the to-be-detected image to obtain the face detection result comprising the truncated face estimation result. The semantic information of the truncated face can be used to estimate the pixel region of the truncated part, so that the face remains in a symmetrical and centered state, which is beneficial to the comparison and verification in the subsequent face identity recognition process, thereby improving the accuracy of face identity recognition.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a face detection method and related equipment. Background Art

[0002] With the development of artificial intelligence, such as the widespread application of image processing technology, face detection is currently one of the most thoroughly studied sub-fields of object detection. It has strong application value in intelligent surveillance, identity verification, human-computer interaction, social interaction, and entertainment, and is also the first step in the entire face recognition algorithm. In actual application scenarios, face truncation due to the limited field of view of the camera is common. The truncation of the face cannot be properly corrected and aligned, which has a significant impact on subsequent face identification. Directly using the detected incomplete face due to truncation for face identification will have a negative impact on the accuracy of face identification, reducing it. Summary of the Invention

[0003] Embodiments of the present invention provide a face detection method and related equipment. By performing face detection and truncated face estimation processing on multi-scale intermediate features of an image to be detected, a face detection result including a truncated face estimation result is obtained. The semantic information of the truncated face can be used to estimate the pixel area of ​​the truncated portion, so that the face remains symmetrical and centered, which is beneficial to the comparison and verification steps in the subsequent face identity recognition process, thereby improving the accuracy of face identity recognition.

[0004] In a first aspect, an embodiment of the present invention provides a face detection method, the method comprising:

[0005] Acquire an image to be detected, wherein the image to be detected includes a truncated face;

[0006] Performing multi-scale feature extraction on the image to be detected to obtain multi-scale intermediate features of the image to be detected, wherein the multi-scale features include intermediate features of multiple preset scales;

[0007] Performing face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected, wherein the face detection result includes a truncated face estimation result.

[0008] Optionally, performing face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected includes:

[0009] Performing feature fusion on the multi-scale intermediate features at the multiple preset scales to obtain multi-scale fused features;

[0010] Performing face detection processing on the multi-scale fusion features to obtain candidate face frames that meet preset conditions;

[0011] performing truncated face estimation processing on the multi-scale fusion feature to obtain a truncated estimation value;

[0012] Calculating a truncated estimated face frame based on the candidate face frame and the truncation estimated value;

[0013] A face detection result of the image to be detected is calculated based on the candidate face frame and the truncated estimated face frame.

[0014] Optionally, performing face detection processing on the multi-scale fusion features to obtain a candidate face frame that meets preset conditions includes:

[0015] Performing face detection processing on the multi-scale fusion features to obtain category confidence and location confidence of the face frame;

[0016] A candidate face frame that meets preset conditions is calculated based on the category confidence and the location confidence.

[0017] Optionally, calculating the truncated estimated face frame based on the candidate face frame and the truncation estimated value includes:

[0018] Performing face detection processing on the multi-scale fusion features to obtain position information of the face frame;

[0019] A truncated estimated face frame is calculated based on the position information of the candidate face frame and the truncation estimate value.

[0020] Optionally, calculating a face detection result of the image to be detected based on the candidate face frame and the truncated estimated face frame includes:

[0021] Perform non-maximum suppression on candidate face frames at multiple scales to obtain the target face frame;

[0022] Perform non-maximum suppression on the truncated estimated face frames at multiple scales to obtain the target truncated estimated face frames;

[0023] The target face frame and the target truncated estimated face frame are output to the image to be detected to obtain a face detection result of the image to be detected.

[0024] Optionally, performing non-maximum suppression processing on the candidate face frames at multiple scales to obtain the target face frame includes:

[0025] Mapping the candidate face frames at the multiple scales according to the size of the image to be detected to obtain multiple face frames to be processed;

[0026] The multiple face frames to be processed are subjected to non-maximum suppression processing to obtain a target face frame.

[0027] Optionally, performing non-maximum suppression on the truncated estimated face frames at multiple scales to obtain a target truncated estimated face frame includes:

[0028] Mapping the truncated estimated face frames at the multiple scales according to the size of the image to be detected to obtain multiple truncated estimated face frames to be processed;

[0029] The multiple truncated estimated face frames to be processed are subjected to non-maximum suppression processing to obtain target truncated estimated face frames.

[0030] In a second aspect, an embodiment of the present invention provides a face detection device, comprising:

[0031] An acquisition module, configured to acquire an image to be detected, wherein the image to be detected includes a truncated face;

[0032] An extraction module is used to perform multi-scale feature extraction on the image to be detected to obtain multi-scale intermediate features of the image to be detected, wherein the multi-scale features include intermediate features of multiple preset scales;

[0033] A processing module is used to perform face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected, wherein the face detection result includes a truncated face estimation result.

[0034] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and runnable on the processor, wherein when the processor executes the computer program, the steps in the face detection method provided in the embodiment of the present invention are implemented.

[0035] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the face detection method provided in the embodiment of the invention are implemented.

[0036] In an embodiment of the present invention, an image to be detected is obtained, wherein the image to be detected includes a truncated face; multi-scale feature extraction is performed on the image to be detected to obtain multi-scale intermediate features of the image to be detected, wherein the multi-scale features include intermediate features of multiple preset scales; face detection and truncated face estimation processing are performed on the multi-scale intermediate features to obtain a face detection result for the image to be detected, wherein the face detection result includes a truncated face estimation result. By performing face detection and truncated face estimation processing on the multi-scale intermediate features of the image to be detected to obtain a face detection result including a truncated face estimation result, the semantic information of the truncated face can be used to estimate the pixel area of ​​the truncated portion, so that the face remains symmetrical and centered, which is beneficial to the comparison and verification steps in the subsequent face identity recognition process, thereby improving the accuracy of face identity recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 is a flow chart of a face detection method provided by an embodiment of the present invention;

[0039] Figure 2 is a structural diagram of a joint network model provided by an embodiment of the present invention;

[0040] Figure 3 is a schematic diagram of a face detection result provided by an embodiment of the present invention;

[0041] Figure 4 is a flowchart of another face detection method provided by an embodiment of the present invention;

[0042] Figure 5 1 is a structural diagram of a face detection device provided by an embodiment of the present invention;

[0043] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0045] See Figure 1 , Figure 1 FIG. 1 is a flow chart of a face detection method provided by an embodiment of the present invention. Figure 1 As shown, the face detection method includes the following steps:

[0046] 101. Obtain an image to be detected.

[0047] In an embodiment of the present invention, a target person can be photographed by a camera set at a designated location to obtain an image of the target person to be detected, or the image to be detected can be obtained by a user uploading a facial image of the target person. The above-mentioned image to be detected includes a truncated face, and the above-mentioned truncated face can be understood as a face contour being truncated by the edge of the image, resulting in an incomplete face. For example, if the photographed face appears in the lower left corner of the image, the lower left corner of the face may be truncated; if the photographed face appears in the lower right corner of the image, the lower right corner of the face may be truncated; if the photographed face appears in the upper left corner of the image, the upper left corner of the face may be truncated; if the photographed face appears in the upper right corner of the image, the upper right corner of the face may be truncated; if the photographed face appears directly below the image, the lower part of the face may be truncated.

[0048] 102. Perform multi-scale feature extraction on the image to be detected to obtain multi-scale intermediate features of the image to be detected.

[0049] In an embodiment of the present invention, the multi-scale features may include intermediate features of multiple preset scales. The multi-scale feature extraction may be performed using a deep neural network. In the deep neural network, the scales of the image to be detected and the intermediate features may be reduced by downsampling, thereby obtaining intermediate features of different scales.

[0050] Multi-scale feature extraction can be performed on the image to be detected using a network backbone module. This network backbone module can be a commonly used backbone module for object detection networks, such as those from the ResNet, MobileNet, and ShuffleNet series. The network backbone module is used to extract deep semantic features from the input image and is typically composed of multiple stages of convolutional modules. Downsampling is performed through the downsampling layer before each stage, so that the spatial resolution of the convolutional module in each stage is half that of the previous stage, while the channel dimension is generally doubled. For example, if the spatial resolution of the image to be detected is 2048*2048*1, where 2048*2048 represents the spatial resolution and 1 represents the channel dimension, after one stage of convolution, the resulting resolution is 1024*1024*2, where 1024*1024 represents the spatial resolution and 2 represents the channel dimension.

[0051] In an embodiment of the present invention, features of corresponding scales can be extracted from the third, fourth, and fifth stages, thereby obtaining intermediate features at scales of 1 / 8, 1 / 16, and 1 / 32 relative to the image to be detected. For the convenience of description, the embodiment of the present invention refers to the intermediate features of the above three scales as C3, C4, and C5, respectively.

[0052] The aforementioned multiple preset scales may be 1 / 8, 1 / 16, or 1 / 32 of the scale of the image to be detected. In the embodiments of the present invention, the number and size of the preset scales are not limited. By extracting multi-scale intermediate features from the image to be detected, facial features at multiple scales can be obtained, thereby improving the accuracy of subsequent face detection.

[0053] 103. Perform face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected.

[0054] In an embodiment of the present invention, the face detection result includes a truncated face estimation result. The face detection result may be information about a passing face frame in the image to be detected, including the face category confidence, face frame positioning confidence, center point coordinates, and width and height attribute information.

[0055] The truncated face estimation result may be truncated estimated face frame information in the image to be detected, and the truncated estimated face frame information includes center point coordinates and width and height attribute information.

[0056] Specifically, in the step of performing face detection and truncated face estimation on multi-scale intermediate features to obtain the face detection result of the image to be detected, the multi-scale intermediate features can be fused at multiple preset scales to obtain multi-scale fused features; face detection is performed on the multi-scale fused features to obtain a candidate face frame that meets preset conditions; truncated face estimation is performed on the multi-scale fused features to obtain a truncated estimated value; a truncated estimated face frame is calculated based on the candidate face frame and the truncated estimated value; and the face detection result of the image to be detected is calculated based on the candidate face frame and the truncated estimated value.

[0057] The extracted multi-scale intermediate features can be fused to enhance the exchange of feature information at different scales and improve the accuracy of face detection. The multi-scale intermediate features can be fused by a feature fusion module. The feature fusion module can adopt some commonly used feature fusion networks, such as feature pyramid network (FPN), bidirectional feature pyramid network (BiFPN), etc. It should be noted that the feature fusion network of the present invention is not limited in any way, and other algorithms, models and networks that can achieve feature fusion are also acceptable.

[0058] In an embodiment of the present invention, when the network backbone module extracts features, the intermediate features of a specific stage layer will be extracted and input into the next feature fusion module to provide multi-scale fusion features. The intermediate features C3, intermediate features C4 and intermediate features C5 are fused by the feature fusion module, and the feature fusion module will regenerate new multi-scale features. For ease of explanation, the embodiment of the present invention is referred to as fusion feature P3, fusion feature P4 and fusion feature P5. Since the scale space of each layer of features is different, the number of input channels is different. For example, the scale space of the fusion feature P3 is 1 / 8 of the image to be detected, and the corresponding number of output channels is 256; the scale space of the fusion feature P4 is 1 / 16 of the image to be detected, and the corresponding number of output channels is 512; the scale space of the fusion feature P5 is 1 / 32 of the original image, and the corresponding number of output channels is 1024.

[0059] The multi-scale fusion features can be used for face detection to obtain detection frames at various scales. Detection frames that do not belong to the face frames are discarded to obtain candidate face frames at various scales. The multi-scale fusion features can be used for face detection using a face localization model. The face localization module is applied to the fusion features P3, P4, and P5 output by the feature fusion module to perform face localization. In an embodiment of the present invention, the face localization module can have three different output branches: a category prediction branch, a position prediction branch, and an IoU perception branch.

[0060] The category prediction branch can be used to predict the confidence level of a face. Its feature layer size is H×W×C, where H represents the height of the current output feature layer, W represents the width of the current output feature layer, and C represents the total number of predicted categories. It should be noted that because this invention only detects faces, C can be equal to 1. The output of this branch indicates the confidence level that the detection box belongs to a face.

[0061] The position prediction branch is used to predict the position of the detected face. Its feature size is H×W×4, where H represents the height of the current output feature layer and W represents the width of the current output feature layer. However, it outputs four layers of features, which are further represented as the center point coordinates and width and height attribute information of the predicted detection box.

[0062] The IoU perception branch is used to assist in improving the positioning accuracy of the face prediction module. Its feature size is H×W×1, and its output is the confidence level of the face frame positioning.

[0063] Before each output branch, a series of convolutional layers can be combined to further extract multi-scale fusion features to produce different prediction functions. The number of channels in each convolutional layer is 256, and the position prediction branch and the IoU perception branch share some convolutional layers.

[0064] Multi-scale fusion features can be processed for truncated face estimation, thereby obtaining truncated estimates at various scales. This can be done using the face truncation estimation module. Like the face localization module, this module is based on the multi-scale fusion features output by the feature fusion module. At different scales, an additional output branch structure is created. This structure is similar to that of the face localization module, using a series of convolutional layers to convolve the FPN multi-scale fusion features. The final output feature size is H×W×4, with each layer's feature channels used to predict truncated pixels in the top, bottom, left, and right directions of the face frame. This output branch performs estimation via regression. Since the face truncation estimation module runs in parallel with the face localization module, it has little impact on face detection.

[0065] It should be noted that while the network structures of the face localization module and the face truncation prediction module are essentially the same, each branch learns different parameters and implements different functions because it targets different learning objectives during training. The cls branch learns the class probability. If a position corresponds to the center of the face landmark, the value at that position is 1, while all other positions are 0. Therefore, the learning result of the cls branch represents the face classification probability at that position. The pos branch learns the x- and y-axis deviations of the face landmark center relative to the current position and the width and height of the target through regression. Therefore, the pos branch has the function of localizing the face frame. The IoU branch learns the IoU value between the predicted and landmark boxes, which indicates the positioning confidence of the detection box. The reg branch of the face truncation prediction module learns the pixel deviations of the face landmark before and after truncation in the four directions, namely, top, bottom, left, and right. This allows for truncation prediction. The above network backbone module, feature fusion module, face positioning module, and face truncation estimation module can be connected to obtain a joint network model, as shown in the following example: Figure 2 As shown, Figure 2 This is a structural diagram of a joint network model provided by an embodiment of the present invention.

[0066] Specifically, in the step of performing face detection processing on the multi-scale fusion features to obtain a candidate face frame that meets the preset conditions, face detection processing is performed on the multi-scale fusion features to obtain the category confidence and location confidence of the face frame; based on the category confidence and location confidence, the candidate face frame that meets the preset conditions is calculated.

[0067] The above-mentioned category confidence can be obtained by outputting the above-mentioned category prediction branch, and the above-mentioned location confidence can be obtained by outputting the above-mentioned IoU perception branch. The total confidence of the detection frame for all positioning points is calculated: total confidence = category confidence value × location confidence. When the total confidence of a detection frame is lower than the set threshold, it indicates that the detection frame is not a face frame. Therefore, the detection frames with total confidence below the set threshold can be filtered out, and the detection frames with total confidence below the set threshold are obtained as candidate face frames. The coordinates of the candidate face frames can be calculated by combining the POS feature. Specifically, it can be assumed that the coordinates of the current positioning point are (x, y), the predicted POS offsets in the x and y directions at this positioning point are dx and dy respectively, and the width and height of the detection frame are w and h. The coordinates of the center point of the positioning frame are then (x+dx, y+dy), thereby obtaining the coordinates of the candidate face frames at multiple scales.

[0068] The truncated estimated face frame can be calculated based on the position information of the candidate face frame and the truncation estimate value. Furthermore, the final output feature of the face truncation estimate module is of size H×W×4. Each layer of feature channels is used to predict the truncated pixels in the four directions of the face frame, namely the top, bottom, left, and right. The face truncation estimate output features index the estimated deviation values ​​yt, yb, xl, and xr in the top, bottom, left, and right directions under the positioning point. Therefore, the coordinates of the upper left corner vertex of the final truncated estimated face frame are (x+dx-w / 2-xl, y+dy-h / 2-yt) and the coordinates of the lower right vertex are (x+dx+w / 2+xr, y+dy+h / 2+yb). For candidate face frames of different scales, truncated estimated face frames of different scales can be obtained by using the truncation estimate value.

[0069] Specifically, after obtaining candidate face frames at multiple scales, non-maximum suppression can be performed on the candidate face frames at multiple scales to filter out duplicate candidate face frames in highly overlapping areas, thereby obtaining a target face frame. After obtaining truncated estimated face frames at multiple scales, non-maximum suppression can be performed on the truncated estimated face frames at multiple scales to filter out duplicate truncated estimated face frames in highly overlapping areas, thereby obtaining a target truncated estimated face frame. The target face frame and the target truncated estimated face frame are output to the image to be detected to obtain a face detection result for the image to be detected.

[0070] Furthermore, the candidate face frames at multiple scales can be mapped according to the size of the image to be detected to obtain multiple face frames to be processed; non-maximum suppression is performed on the multiple face frames to be processed to obtain a target face frame. Alternatively, the truncated estimated face frames at multiple scales can be mapped according to the size of the image to be detected to obtain multiple truncated estimated face frames to be processed; non-maximum suppression is performed on the multiple truncated estimated face frames to be processed to obtain a target truncated estimated face frame.

[0071] After obtaining the target face frame, the target face frame can be output to the image to be detected, so that the face detection result can be obtained. The face detection result is the image in the face frame. After obtaining the target truncated estimated face frame, the target truncated estimated face frame can be output to the image to be detected, so that the truncated face estimation result can be obtained. The truncated face estimation result is the image in the truncated face estimation frame. Figure 3 As shown, Figure 3 This is a schematic diagram of a face detection result provided by an embodiment of the present invention.

[0072] Optional, please refer to Figure 4 , Figure 4 FIG. 1 is a flow chart of another face detection method provided by an embodiment of the present invention. Figure 4As shown, the main process of the face detection method provided by the embodiment of the present invention includes: after the input image undergoes certain image preprocessing, it is input into the backbone network module to extract multi-scale features, such as the intermediate features C3, intermediate features C4, and intermediate features C5 described above; the features extracted by the backbone network module are input into the feature fusion module for feature fusion, and the fused multi-scale features are output, such as the fused features P3, fused features P4, and fused features P5 described above; face positioning and truncation estimation are performed on the fused feature P4. The face positioning and truncation estimation functions are implemented as follows: ①, the fused feature P4 passes through the three branches of the face positioning module to output the CLS feature, the POS feature, and the IOU feature respectively, and passes through the truncation estimation module to output the truncated estimated Reg feature; ②, the total confidence of the detection frame of all positioning points is calculated, and the total confidence = the classification confidence value of the CLS feature × the positioning confidence value of the IOU feature; ③, the face positioning points below the set threshold are filtered out; ④, for the remaining high-confidence face positioning points, the coordinates of the face detection frame are calculated in combination with the POS feature. The calculation method is: the scale of the current feature layer is H×W, then there are H×W grids, and the upper left vertex of the grid where the center point is located is used as the positioning point. Assuming that the current positioning point coordinates are (x, y), the predicted bias of pos under this positioning point in the x and y directions is dx, dy, and the grid width and height are w, h. The coordinates of the center point of the positioning frame are (x+dx, y+dy), the coordinates of the upper left vertex of the positioning frame are (x+dx-w / 2, y+dy-h / 2), and the coordinates of the lower right vertex of the positioning frame are (x+dx+w / 2, y+dy+h / 2), thereby obtaining the coordinates of the face frame before truncation estimation; ⑤. According to the high-confidence face positioning point and the face truncation estimation output features in ④, index the estimated deviation values ​​yt, yb, xl and xr in the upper, lower and left directions under the positioning point, so that the coordinates of the upper left vertex of the face frame after the final truncation estimation can be obtained as (x+dx-w / 2-xl, y+dy-h / 2-yt) and the coordinates of the lower right vertex of the face frame are (x+dx+w / 2+xr, y+dy+h / 2+yb). ⑥. Map the coordinates of the face detection frame after truncation estimation obtained at the current feature layer back to the original image scale. Similar face positioning and truncation estimation operations are also performed on the fusion features P3 and P5. All truncated estimated face frames detected on the fusion feature P3, fusion feature P4 and fusion feature P5 feature layers are integrated, and the repeated positioning frames in the high overlapping areas are filtered out using the non-maximum suppression algorithm. The final truncated estimated face detection frames of all faces under the current input image are output.

[0073] In an embodiment of the present invention, an image to be detected is obtained, wherein the image to be detected includes a truncated face; multi-scale feature extraction is performed on the image to be detected to obtain multi-scale intermediate features of the image to be detected, wherein the multi-scale features include intermediate features of multiple preset scales; face detection and truncated face estimation processing are performed on the multi-scale intermediate features to obtain a face detection result for the image to be detected, wherein the face detection result includes a truncated face estimation result. By performing face detection and truncated face estimation processing on the multi-scale intermediate features of the image to be detected to obtain a face detection result including a truncated face estimation result, the semantic information of the truncated face can be used to estimate the pixel area of ​​the truncated portion, so that the face remains symmetrical and centered, which is beneficial to the comparison and verification steps in the subsequent face identity recognition process, thereby improving the accuracy of face identity recognition.

[0074] It should be noted that the face detection method provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can perform face detection.

[0075] Optional, see Figure 5 , Figure 5 FIG. 1 is a structural diagram of a face detection device provided by an embodiment of the present invention. Figure 5 As shown, the device includes:

[0076] An acquisition module 501 is configured to acquire an image to be detected, wherein the image to be detected includes a truncated face;

[0077] An extraction module 502 is configured to perform multi-scale feature extraction on the image to be detected to obtain multi-scale intermediate features of the image to be detected, wherein the multi-scale features include intermediate features of multiple preset scales;

[0078] The processing module 503 is configured to perform face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected, wherein the face detection result includes a truncated face estimation result.

[0079] Optionally, the processing module 503 includes:

[0080] A fusion submodule, configured to fuse the multi-scale intermediate features at the plurality of preset scales to obtain multi-scale fused features;

[0081] A first processing submodule is configured to perform face detection processing on the multi-scale fusion features to obtain a candidate face frame that meets preset conditions;

[0082] A second processing submodule is configured to perform truncated face estimation processing on the multi-scale fusion features to obtain a truncated estimation value;

[0083] A first calculation submodule is configured to calculate a truncated estimated face frame based on the candidate face frame and the truncation estimate value;

[0084] The second calculation submodule is configured to calculate a face detection result of the image to be detected based on the candidate face frame and the truncated estimated face frame.

[0085] Optionally, the first processing submodule includes:

[0086] a first processing unit, configured to perform face detection processing on the multi-scale fusion features to obtain a category confidence and a location confidence of a face frame;

[0087] The first calculation unit is configured to calculate a candidate face frame that meets a preset condition based on the category confidence and the location confidence.

[0088] Optionally, the first computing submodule includes:

[0089] A second processing unit is configured to perform face detection processing on the multi-scale fusion features to obtain position information of a face frame;

[0090] The second calculation unit is configured to calculate a truncated estimated face frame based on the position information of the candidate face frame and the truncation estimation value.

[0091] Optionally, the second computing submodule includes:

[0092] The third processing unit is used to perform non-maximum suppression processing on the candidate face frames at multiple scales to obtain the target face frame;

[0093] a fourth processing unit, configured to perform non-maximum suppression on the truncated estimated face frames at multiple scales to obtain a target truncated estimated face frame;

[0094] The output unit is used to output the target face frame and the target truncated estimated face frame to the image to be detected to obtain a face detection result of the image to be detected.

[0095] Optionally, the third processing unit includes:

[0096] A first mapping subunit is configured to map the candidate face frames at the multiple scales according to the size of the image to be detected to obtain multiple face frames to be processed;

[0097] The first processing sub-unit is configured to perform non-maximum suppression processing on the multiple face frames to be processed to obtain a target face frame.

[0098] Optionally, the fourth processing unit includes:

[0099] a second mapping subunit, configured to map the truncated estimated face frames at the multiple scales according to the size of the image to be detected, to obtain multiple truncated estimated face frames to be processed;

[0100] The second processing sub-unit is configured to perform non-maximum suppression processing on the multiple truncated estimated face frames to be processed to obtain a target truncated estimated face frame.

[0101] It should be noted that the face detection device provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can perform face detection.

[0102] The face detection device provided in the embodiment of the present invention can implement each process implemented by the face detection method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0103] See also Figure 6 , Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, it includes: a memory 602, a processor 601, and a computer program of a face detection method stored in the memory 602 and executable on the processor 601, wherein:

[0104] The processor 601 is configured to call the computer program stored in the memory 602 and execute the following steps:

[0105] Acquire an image to be detected, wherein the image to be detected includes a truncated face;

[0106] Performing multi-scale feature extraction on the image to be detected to obtain multi-scale intermediate features of the image to be detected, wherein the multi-scale features include intermediate features of multiple preset scales;

[0107] Performing face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected, wherein the face detection result includes a truncated face estimation result.

[0108] Optionally, the processor 601 performs face detection and truncation face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected, including:

[0109] Performing feature fusion on the multi-scale intermediate features at the multiple preset scales to obtain multi-scale fused features;

[0110] Performing face detection processing on the multi-scale fusion features to obtain candidate face frames that meet preset conditions;

[0111] performing truncated face estimation processing on the multi-scale fusion feature to obtain a truncated estimation value;

[0112] Calculating a truncated estimated face frame based on the candidate face frame and the truncation estimated value;

[0113] A face detection result of the image to be detected is calculated based on the candidate face frame and the truncated estimated face frame.

[0114] Optionally, the processor 601 performs face detection processing on the multi-scale fusion features to obtain a candidate face frame that meets preset conditions, including:

[0115] Performing face detection processing on the multi-scale fusion features to obtain category confidence and location confidence of the face frame;

[0116] A candidate face frame that meets preset conditions is calculated based on the category confidence and the location confidence.

[0117] Optionally, the processor 601 calculates the truncated estimated face frame based on the candidate face frame and the truncation estimate value, including:

[0118] Performing face detection processing on the multi-scale fusion features to obtain position information of the face frame;

[0119] A truncated estimated face frame is calculated based on the position information of the candidate face frame and the truncation estimate value.

[0120] Optionally, the processor 601 calculates the face detection result of the image to be detected based on the candidate face frame and the truncated estimated face frame, including:

[0121] Perform non-maximum suppression on candidate face frames at multiple scales to obtain the target face frame;

[0122] Perform non-maximum suppression on the truncated estimated face frames at multiple scales to obtain the target truncated estimated face frames;

[0123] The target face frame and the target truncated estimated face frame are output to the image to be detected to obtain a face detection result of the image to be detected.

[0124] Optionally, the processor 601 performs non-maximum suppression processing on the candidate face frames at multiple scales to obtain the target face frame, including:

[0125] Mapping the candidate face frames at the multiple scales according to the size of the image to be detected to obtain multiple face frames to be processed;

[0126] The multiple face frames to be processed are subjected to non-maximum suppression processing to obtain a target face frame.

[0127] Optionally, the processor 601 performs non-maximum suppression on the truncated estimated face frames at multiple scales to obtain a target truncated estimated face frame, including:

[0128] Mapping the truncated estimated face frames at the multiple scales according to the size of the image to be detected to obtain multiple truncated estimated face frames to be processed;

[0129] The multiple truncated estimated face frames to be processed are subjected to non-maximum suppression processing to obtain target truncated estimated face frames.

[0130] The electronic device provided by the embodiment of the present invention can implement each process implemented by the face detection method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0131] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the various processes of the face detection method or the application-side face detection method provided in the embodiment of the present invention, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0132] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0133] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A face detection method for truncated face estimation, characterized in that: The following steps are involved: Acquire an image to be detected, wherein the image to be detected includes a truncated face; Performing multi-scale feature extraction on the image to be detected through a network backbone module to obtain multi-scale intermediate features of the image to be detected, wherein the multi-scale features include intermediate features of multiple preset scales; Performing face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected, wherein the face detection result includes a truncated face estimation result; The performing face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected includes: The multi-scale intermediate features are subjected to feature fusion at the plurality of preset scales by a feature fusion module to obtain multi-scale fusion features; Performing face detection processing on the multi-scale fusion features through a face positioning module to obtain a candidate face frame that meets preset conditions; The multi-scale fusion feature is subjected to truncated face estimation processing by a face truncation estimation module to obtain a truncation estimation value. Specifically, the face truncation estimation module uses a series of convolutional layers to convolve the multi-scale fusion feature, and the size of the final output feature is H×W×4. The feature channel of each layer is used to predict the truncated pixels in the four directions of the top, bottom, left, and right of the face frame, and the estimation is performed through regression. The face truncation estimation module is parallel to the face positioning module. Where H represents the height of the output feature layer, W represents the width of the output feature layer, and 4 represents the total number of predicted categories is 4; According to the candidate face frame and the truncation estimated value, a truncated estimated face frame is calculated; the multi-scale fusion feature is subjected to face detection processing to obtain position information of the candidate face frame; according to the position information of the candidate face frame and the truncation estimated value, a truncated estimated face frame is calculated; specifically, assuming that the current positioning point coordinates are (x, y), the positioning point prediction bias in the x direction and y direction are dx, dy respectively, the width and height of the detection frame are w, h respectively, then the center point of the candidate face frame is The coordinates are (x+dx, y+dy). Based on the position information of the candidate face frame and the truncation estimate value, the truncated estimated face frame is calculated. The specific face truncation estimate output features are indexed by the estimated deviation values ​​yt, yb, xl and xr in the up, down, left and right directions under the positioning point. The final truncated estimated face frame has the upper left corner vertex coordinates (x+dx-w / 2-xl, y+dy-h / 2-yt) and the lower right vertex coordinates (x+dx+w / 2+xr, y+dy+h / 2+yb); A face detection result of the image to be detected is calculated based on the candidate face frame and the truncated estimated face frame.

2. The method according to claim 1, wherein The performing face detection processing on the multi-scale fusion features to obtain a candidate face frame that meets preset conditions includes: Performing face detection processing on the multi-scale fusion features to obtain category confidence and location confidence of the face frame; A candidate face frame that meets preset conditions is calculated based on the category confidence and the location confidence.

3. The method according to claim 1, wherein The calculating the truncated estimated face frame according to the candidate face frame and the truncation estimated value includes: Performing face detection processing on the multi-scale fusion features to obtain position information of the face frame; A truncated estimated face frame is calculated based on the position information of the candidate face frame and the truncation estimate value.

4. The method according to claim 3, wherein The step of calculating a face detection result of the image to be detected based on the candidate face frame and the truncated estimated face frame includes: Perform non-maximum suppression on candidate face frames at multiple scales to obtain the target face frame; Perform non-maximum suppression on the truncated estimated face frames at multiple scales to obtain the target truncated estimated face frames; The target face frame and the target truncated estimated face frame are output to the image to be detected to obtain a face detection result of the image to be detected.

5. The method according to claim 4, wherein The non-maximum suppression processing is performed on the candidate face frames at multiple scales to obtain the target face frame, including: Mapping the candidate face frames at the multiple scales according to the size of the image to be detected to obtain multiple face frames to be processed; The multiple face frames to be processed are subjected to non-maximum suppression processing to obtain a target face frame.

6. The method according to claim 4, wherein The step of performing non-maximum suppression on the truncated estimated face frames at multiple scales to obtain a target truncated estimated face frame includes: Mapping the truncated estimated face frames at the multiple scales according to the size of the image to be detected to obtain multiple truncated estimated face frames to be processed; The multiple truncated estimated face frames to be processed are subjected to non-maximum suppression processing to obtain target truncated estimated face frames.

7. A face detection device, characterized in that: The device comprises: An acquisition module, configured to acquire an image to be detected, wherein the image to be detected includes a truncated face; An extraction module is used to perform multi-scale feature extraction on the image to be detected to obtain multi-scale intermediate features of the image to be detected, wherein the multi-scale features include intermediate features of multiple preset scales; a processing module, configured to perform face detection and truncated face estimation processing on the multi-scale intermediate features to obtain a face detection result of the image to be detected, wherein the face detection result includes a truncated face estimation result; The processing module is further configured to: The multi-scale intermediate features are subjected to feature fusion at the plurality of preset scales by a feature fusion module to obtain multi-scale fusion features; Performing face detection processing on the multi-scale fusion features through a face positioning module to obtain a candidate face frame that meets preset conditions; The multi-scale fusion feature is subjected to truncated face estimation processing by a face truncation estimation module to obtain a truncation estimation value. Specifically, the face truncation estimation module uses a series of convolutional layers to convolve the multi-scale fusion feature, and the size of the final output feature is H×W×4. The feature channel of each layer is used to predict the truncated pixels in the four directions of the top, bottom, left, and right of the face frame, and the estimation is performed through regression. The face truncation estimation module is parallel to the face positioning module. Where H represents the height of the output feature layer, W represents the width of the output feature layer, and 4 represents the total number of predicted categories is 4; According to the candidate face frame and the truncation estimated value, a truncated estimated face frame is calculated; the multi-scale fusion feature is subjected to face detection processing to obtain position information of the candidate face frame; according to the position information of the candidate face frame and the truncation estimated value, a truncated estimated face frame is calculated; specifically, assuming that the current positioning point coordinates are (x, y), the positioning point prediction bias in the x direction and y direction are dx, dy respectively, the width and height of the detection frame are w, h respectively, then the center point of the candidate face frame is The coordinates are (x+dx, y+dy). Based on the position information of the candidate face frame and the truncation estimate value, the truncated estimated face frame is calculated. The specific face truncation estimate output features are indexed by the estimated deviation values ​​yt, yb, xl and xr in the up, down, left and right directions under the positioning point. The final truncated estimated face frame has the upper left corner vertex coordinates (x+dx-w / 2-xl, y+dy-h / 2-yt) and the lower right vertex coordinates (x+dx+w / 2+xr, y+dy+h / 2+yb); A face detection result of the image to be detected is calculated based on the candidate face frame and the truncated estimated face frame.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the face detection method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the face detection method according to any one of claims 1 to 6.

Images