Face detection method in complex scenes based on dynamic fusion of multi-domain features and CIOU optimization

Abstract

The present invention discloses a complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization, which belongs to the technical field of face detection in complex scenes. The method comprises the following steps: obtaining a face detection dataset; constructing a complex scene face detection network; training the complex scene face detection network with a face detection training set to obtain a complex scene face detection model; inputting the face detection image to be detected into the complex scene face detection model, and outputting the target detection position and detection results. The present invention significantly improves the network's detection accuracy for small target faces; can more accurately distinguish the differences between prediction frames, thereby optimizing the model training process and obtaining more accurate detection results; at the same time, it also introduces Varifocal Loss, dynamically adjusting the weights of difficult samples through a difficult and easy sample balance mechanism, guiding the model learning direction, and enhancing its detection accuracy and adaptability in complex scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of face detection in complex scenes, and specifically relates to a complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization. Background Art

[0002] Face detection technology, as a core component of facial recognition systems, directly impacts the accuracy of target detection and identification. It is a key technology for intelligent surveillance systems in modern public security. In crowded public places like train stations, airports, and shopping malls, traditional manual surveillance methods, constrained by the physiological limitations of human vision (e.g., the inability to focus on multiple objects simultaneously and difficulty maintaining attention for extended periods), struggle to meet the demands of high-precision, real-time identification and tracking of individuals. In complex scenarios, such as those characterized by fluctuating lighting, object occlusion, and large variations in face size, the efficiency and accuracy of traditional surveillance methods decline significantly, making it difficult to effectively respond to emergencies. With the rapid development of artificial intelligence (AI), face detection technology has emerged as a key breakthrough in addressing this issue. By incorporating advanced computer vision algorithms, surveillance systems can capture facial information in real time and, combined with facial recognition technology, provide rapid warnings and precise tracking of suspicious individuals or violations. This not only significantly improves the efficiency of public security surveillance but also provides strong technical support for intelligent security, smart transportation, and other fields. However, most face detection scenarios face challenges, such as fluctuating lighting, densely packed objects, and large variations in face size, presenting numerous challenges.

[0003] Traditional face detection methods primarily use a large number of sample images containing faces to train models and generate face detection templates, or classifiers, to handle binary classification. However, face detection templates can only accept images of specific sizes, and these artificially designed features are easily affected by various external conditions, such as lighting conditions, camera angles, and occlusion between faces. In complex scenes like dense crowds, traditional face detection methods can no longer guarantee performance. However, with the continuous advancement of deep learning technology, particularly convolutional neural networks in computer vision and image processing, their feature capture methods have significantly surpassed the accuracy of traditional machine learning detection and have been widely applied in various fields of face detection.

[0004] Deep learning-based face detection techniques can be broadly categorized into two types. One uses a cascaded approach as a feature extractor and classifier to detect faces. While these methods achieved significant early success, cascaded detectors suffer from several drawbacks, such as difficulty in training, cumbersome model adjustments, and slow detection speeds. Consequently, the other type of face detection algorithm, based on improved general object detection algorithms, has become the primary focus of current face detection research. These algorithms often consider a wider range of features, resulting in stronger feature extraction capabilities and greater adaptability.

[0005] In recent years, addressing the challenges of low-quality face detection, such as large object scale variations and densely packed small objects, a single neural network-based real-time multi-scale face detection method has been proposed. This method improves recall and reduces false positive rates through a scale-compensated anchor matching strategy and label maximization. Existing methods enhance feature extraction through an attention mechanism and a weighted bidirectional feature pyramid, optimizing the loss function of the RetinaFace algorithm to focus more on face location. PyramidBox has also been proposed to enhance small face detection by leveraging relevant information and a data-anchor-sampling strategy. However, these methods ignore the mismatch between classification scores and localization accuracy in single-stage object detectors and fail to consider the class imbalance in dense scenes. This results in a large number of negative samples that affect model training and degrade detection performance. Existing methods generate a large number of hard negative mining (HNM) samples to balance the data distribution and perform multi-scale retraining, thereby improving model robustness. Furthermore, the generated detection bounding boxes are converted to ellipses to more tightly enclose the face region. Existing technologies improve resolution by introducing sub-pixel convolution, using Focal loss to balance the weights of positive and negative samples, and CIou loss to more accurately reflect the degree of overlap between the predicted and true frames in the detection problem. Tinaface balances the positioning and classification scores by introducing an IOU branch, and enhances the detection accuracy of small objects through the DIOU loss function. YOLO-FaceV2 improves the detection accuracy of occluded objects by introducing an exclusion loss and a weighted loss function to balance the sample distribution. However, these methods suffer from the problem that target information is easily lost during feature extraction and fusion, resulting in poor detection accuracy for low-quality faces.

[0006] In summary, the shortcomings of the existing technology are:

[0007] 1. The problem of target information loss during feature extraction and fusion is mainly reflected in the insufficient interaction between shallow and deep features in existing methods, as well as the limitations of the extraction and fusion strategies of target features in complex scenes.

[0008] Existing methods often overlook the interaction between the detailed information contained in shallow features and the semantic information contained in deeper features during feature extraction and fusion. Most networks fuse features of different scales through repeated upsampling and downsampling using a feature pyramid network (FPN). However, this approach can easily lead to feature loss of small or low-quality objects during repeated upsampling and downsampling. This information loss is particularly severe when dealing with multi-scale and densely packed objects. Furthermore, existing methods have limitations in their feature fusion strategies. Although multi-scale feature fusion techniques are widely used in object detection tasks, most methods fail to fully utilize both the detailed information of shallow features and the semantic information of deeper features. This deficiency results in insufficient localization and classification capabilities when dealing with complex scenarios (such as occluded objects, changing illumination, and multi-scale objects), further compromising detection accuracy and robustness.

[0009] 2. When dealing with dense scenes, existing methods fail to effectively solve the problem of class imbalance, resulting in a large number of negative samples interfering with the model training direction and affecting detection performance.

[0010] Existing methods, when dealing with dense scenes, suffer from a significant disparity in the number of positive and negative samples due to class imbalance. Negative samples far outnumber positive samples, making it difficult for the model to effectively learn the features of positive samples during training. Furthermore, negative samples interfere with the model's training direction, causing the model to over-focus on negative samples and neglect learning the features of positive samples, thus affecting detection performance. Finally, existing methods fail to fully utilize the feature information of positive samples, resulting in insufficient detection capabilities for positive samples in dense scenes, further reducing detection accuracy. Summary of the Invention

[0011] The purpose of the present invention is to address the above-mentioned deficiencies in the prior art and to provide a complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization, so as to solve the problems of low detection accuracy and sample imbalance for low-quality faces such as small targets in dense scenes. Deep learning, as one of the mainstream methods in face detection tasks, has demonstrated superior performance far exceeding traditional face detection methods. However, most neural network architectures in deep learning lack the interaction between shallow features and deep features, as well as limitations in the extraction and fusion strategies of target features in complex scenes. Especially in dense scenes, the detection accuracy of small targets and low-quality faces is low, and the sample imbalance problem further aggravates the difficulty of model training. In response to the problems of low detection accuracy and sample imbalance for low-quality faces such as small targets in dense scenes, the technical solution adopted by the present invention is:

[0012] A complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization includes the following steps:

[0013] S1. Obtain a face detection dataset and preprocess the face detection dataset to obtain a face detection training set;

[0014] S2. Build a complex scene face detection network, which includes a feature extraction network, a cross-domain fusion feature pyramid network, and a multi-task detection head;

[0015] S3. Use the face detection training set to train the complex scene face detection network to obtain a complex scene face detection model;

[0016] S4. Input the face detection image to be detected into the complex scene face detection model, and output the target detection position and detection results.

[0017] This method constructs a face detection network for dense environments. The detection network consists of three components: a cross-domain fusion feature pyramid network and a multi-task detection head. The feature extraction component uses the ResNet50 model, which has been widely demonstrated to have strong feature extraction capabilities across various tasks. In the feature fusion pyramid, input features are fused from the bottom up, and bilinear interpolation is used to align features of different scales. The fused features are then fed into a multi-domain dynamic fusion module (MDFM) for adaptive fusion to prevent information loss. To adapt to the dense and highly variable scales of face detection targets, additional features E1 and E2 are added, resulting in six feature maps at different scales. These are then fed into a shared context information exchange module (Inception Module) for further fusion. The fused features are then fed into the multi-task detection head, which is divided into a classification branch, a localization branch, and a CIOU branch to predict the classification, offset, and CIOU score, respectively. Finally, the offsets are decoded to obtain the final coordinates, which are then combined with the classification and CIOU scores to perform non-maximum suppression to obtain the final detection result.

[0018] Furthermore, S3 specifically includes the following sub-steps:

[0019] S31, inputting the face detection image into the feature extraction network to perform multi-level feature extraction and output a multi-scale feature group;

[0020] S32, input the multi-scale feature group into the cross-domain fusion feature pyramid network for feature fusion, and output six-level fusion features;

[0021] S33, input the six-level fusion features into the multi-task detection head for classification detection, and output the target detection position and detection results;

[0022] S34. Calculate the multi-task joint loss function to optimize the network parameters of complex scene face detection;

[0023] S35, perform multiple rounds of iterative training on S33 and S34;

[0024] S36. Test the trained complex scene face detection network, select the network model with the best performance during the test, and then obtain the complex scene face detection model.

[0025] Furthermore, S31 specifically includes the following sub-steps:

[0026] S311, inputting the face detection image into the feature extraction network, and outputting a first basic feature map C2, a second basic feature map C3, a third basic feature map C4, and a fourth basic feature map C5 respectively through four residual stages;

[0027] S312: Input the first basic feature map C2, the second basic feature map C3, the third basic feature map C4 and the fourth basic feature map C5 into the 1×1 convolution layer respectively, and output multi-scale feature groups with the same number of channels respectively. , i=1, 2, 3, 4.

[0028] Furthermore, S32 specifically includes the following sub-steps:

[0029] S321, multi-scale feature group , i=1, 2, 3, 4 input cross-domain fusion feature pyramid network, where the multi-scale feature group , i=1 performs convolution operation to obtain additional features E1 and E2;

[0030] Multi-scale feature group , i=2, 3, 4 and transition feature map , i=1, 2, 3 are added element by element to generate basic fusion features ;

[0031] S322, basic fusion features Input into the spatial domain branch and the frequency domain branch respectively to generate the spatial domain weight matrix and frequency domain weight matrix ;

[0032] S323, the spatial weight matrix and frequency domain weight matrix After addition, Sigmoid activation is performed to generate the fusion weight matrix W;

[0033] S324, based on the weight matrix W multi-scale feature group and transition feature map Perform dynamic weighted fusion to obtain four intermediate fusion features;

[0034] S325 , input the four intermediate fusion features and the additional features E1 and E2 into the shared context information exchange module for feature fusion, and output a six-level fusion feature.

[0035] Furthermore, in S322, the basic fusion features Input the spatial branch to generate the spatial weight matrix , the process specifically includes:

[0036] Basic fusion features Perform 1×1 convolution operation, reduce the number of channels to 64, and use three-level parallel dilated convolution to extract multiple multi-scale spatial features. After fusing and adding the information of multiple multi-scale spatial features, the spatial domain weight matrix is ​​generated through channel splicing and 1×1 convolution. .

[0037] Furthermore, in S322, the basic fusion features In the input frequency domain branch, generate the frequency domain weight matrix , the process specifically includes:

[0038] Basic fusion features Perform fast Fourier transform to separate the real and imaginary components, process the frequency domain features through 3×3 convolution and ReLU activation, and then reconstruct the spatial domain features through inverse Fourier transform to generate the frequency domain weight matrix .

[0039] Furthermore, S33 specifically includes:

[0040] The six-level fusion features are input into the classification branch, localization branch, and CIOU branch of the multi-task detection head respectively, and the detection results including the target detection category, confidence score, bounding box coordinates and quality score are output;

[0041] Among them, the classification branch is used to predict whether the current detection box is a target;

[0042] The positioning branch is used to predict the offset of the current detection box to the prior box;

[0043] The CIOU branch is used to predict the CIOU score of the current detection box.

[0044] Furthermore, in the CIOU branch, the CIOU score of the current detection box is predicted, which specifically includes the following steps:

[0045] S331, extracting geometric information of the candidate box and the real box;

[0046] S332. Based on the geometric information, calculate the CIOU score of the candidate box and the ground truth box:

[0047]

[0048] Where, The ratio of the overlapping area of ​​the candidate box and the real box to the area of ​​the union of the two; is the Euclidean distance between the center point of the candidate box and the center point of the real box; is the diagonal length of the real frame; The difference between the width-to-height ratio of the candidate frame and the width-to-height ratio of the real frame; is the weight coefficient;

[0049] S333. Perform weighted fusion of the CIOU score and the output of the classification branch to generate a comprehensive score:

[0050]

[0051] Where λ is the weight coefficient, which is used to balance the contribution of classification score and CIOU score;

[0052] S334. Sort the candidate frames based on the comprehensive scores, filter out false positive frames using the non-maximum suppression algorithm, and retain the best candidate frame as the final detection result.

[0053] Furthermore, in S34, the multi-task joint loss function is expressed as:

[0054]

[0055] Where, represents the multi-task joint loss function; is the classification loss; For positioning loss; Losses for CIOU branch; 、 and The corresponding loss weight.

[0056] Furthermore, in S36, the performance indicators during the test include: mean average precision , F1 score, precision Pr and recall Re.

[0057] The complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization provided by the present invention has the following beneficial effects:

[0058] 1. The present invention improves the feature fusion strategy, adopts a dual-branch structure to extract global features and local features from the frequency domain and spatial domain respectively, and uses a dynamic weighted fusion mechanism to avoid the loss of effective information in the feature fusion process, thereby significantly improving the network's detection accuracy for small target faces. In response to problems such as dense targets and difficulty in distinguishing differences in prediction frames in complex scenes, the present invention further designs a prediction branch based on CIOU, which optimizes the prediction target of the traditional IOU branch by introducing shape factors, so that the model can more accurately distinguish the differences between prediction frames, thereby optimizing the model training process and obtaining more accurate detection results. In addition, to solve the problem of sample imbalance, the present invention introduces Varifocal Loss, which dynamically adjusts the weights of difficult samples through a difficult and easy sample balance mechanism, guides the model learning direction, and enhances its detection accuracy and adaptability in complex scenes.

[0059] 2. The present invention combines CIOU with the CIOU to design a CIOU prediction branch to optimize the IOU branch prediction target, obtain a more accurate CIOU score, and introduces Varifocal Loss to guide IOU branch learning, alleviate the category imbalance problem, and enhance the detection performance and adaptability of the model in complex scenarios.

[0060] 3. The complex scene face detection network based on dynamic fusion of multi-domain features and CIOU optimization proposed in this invention shows excellent performance in face detection tasks in dense scenes compared with other mainstream target detection algorithms, and achieves the best detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a schematic diagram of the structure of a complex scene face detection network according to an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the structure of a multi-domain feature dynamic fusion module according to an embodiment of the present invention;

[0063] Figure 3 This is a diagram illustrating the overlap of CIOU prediction branches according to an embodiment of the present invention;

[0064] Figure 4 This is a flowchart of a complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0066] The complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization in this embodiment is described in detail. Figure 4 , which specifically includes the following:

[0067] S1. Obtain a face detection dataset and preprocess the face detection dataset;

[0068] In some embodiments, the face detection dataset is the Wider Face dataset, currently the largest and most authoritative face detection benchmark dataset, containing 32,203 images and nearly 400,000 faces. The Wider Face dataset not only annotates faces but also includes additional information such as blur and occlusion. It covers faces in a variety of poses, lighting conditions, scales, and scenarios. This diversity makes the dataset more realistic and challenging.

[0069] In a specific embodiment, the Wider Face dataset is downloaded and divided into two parts, the dataset resolution is The Wider Face dataset is preprocessed and divided into training and test sets according to the agreed ratio; data augmentation is performed on the training and test sets.

[0070] In a specific embodiment, the entire Wider Face dataset contains 61 different real-life scenes and is divided into three subsets: simple, general, and difficult according to the difficulty of detecting each face. Most of the faces in the difficult subset are low-quality faces with small scale, dense occlusion, or blur. The detection accuracy on the difficult subset is the most challenging and best reflects the model's detection performance. Therefore, the experiment in this embodiment mainly compares the detection performance of the model on the difficult subset.

[0071] S2. Build a complex scene face detection network DF-FACE (Face Detection with FeatureFusion), refer to Figure 1 ,The complex scene face detection network includes feature extraction network, cross-domain fusion feature pyramid network and multi-task detection head;

[0072] In some embodiments, the complex scene face detection network is based on the TinaFace network structure and optimizes its application in face detection in dense environments.

[0073] The complex scene face detection network includes: feature extraction network, cross-domain fusion feature pyramid network and multi-task detection head.

[0074] In some embodiments, the feature extraction component uses the ResNet50 model, widely used for its powerful feature extraction capabilities. Its output features at different scales are aligned by 1×1 convolution and then fed into a feature pyramid for fusion. Within the feature pyramid, input features are aligned at different scales using bilinear interpolation before being fed into the Multi-Domain Feature Dynamic Fusion Module (MDFM) for adaptive fusion to prevent information loss. To accommodate the densely packed objects and large scale variations in face detection, additional features are added and fed into a shared context information exchange module for further fusion. The inspection head consists of three branches: the classification branch (cls) predicts whether the current detection bounding box is an object; the localization branch (loc) predicts the offset of the current detection bounding box relative to the prior bounding box; and the CIOU branch (ciou) predicts the CIOU score of the current detection bounding box. The offset is then decoded to obtain the coordinates. The final result is obtained through NMS based on the classification and CIOU scores.

[0075] S3. Use the training set to train the complex scene face detection network to obtain a complex scene face detection model. The training process of this step specifically includes the following sub-steps:

[0076] S31, input the face detection image into the feature extraction network for multi-level feature extraction, output four layers of basic feature maps (downsampled 4, 8, 16, and 32 times compared to the input image respectively), and the four layers of basic feature maps are then aligned to the channel dimension by 1×1 convolution to obtain a multi-scale feature set. ;

[0077] In one specific embodiment, the feature extraction network is built based on the ResNet50 architecture, which performs multi-level feature extraction on the input image through four residual stages. The residual structure of ResNet50 effectively solves the gradient vanishing problem in deep networks and can extract multi-scale features, making it suitable for object detection tasks in complex scenes. In addition, its powerful feature expression ability and wide applicability enable it to perform well in various computer vision tasks.

[0078] Based on the ResNet50 architecture, the multi-level feature extraction in step S31 specifically includes the following sub-steps:

[0079] S311: Input the face detection image into the feature extraction network, and output the first basic feature map C2, the second basic feature map C3, the third basic feature map C4, and the fourth basic feature map C5 respectively through the initial convolution layer and the four residual stages. The specific process includes:

[0080] Phase 1: The face detection image is downsampled by a factor of 4 through the initial convolution layer (7×7 convolution kernel, stride 2) and the maximum pooling layer (3×3 pooling kernel, stride 2). The downsampled input is then passed to the first residual stage consisting of 3 residual blocks to generate the first basic feature map C2, with a spatial resolution 4 times lower than the input image.

[0081] Second stage: The first basic feature map C2 is converted into the second basic feature map C3 through the second residual stage containing 4 residual blocks, and the spatial resolution is downsampled 8 times compared with the input image;

[0082] The third stage: The second basic feature map C3 is converted into the third basic feature map C4 through the third residual stage containing 6 residual blocks. The spatial resolution is 16 times lower than that of the input image.

[0083] Fourth stage: The third basic feature map C4 is generated into the fourth basic feature map C5 through the fourth residual stage containing 3 residual blocks, and the spatial resolution is downsampled 32 times compared with the input image.

[0084] It should be noted that, in this step S311, the first stage represents the order of operations, which represents the first step; the first residual stage represents the first residual module in the residual network.

[0085] S312: Input the first basic feature map C2, the second basic feature map C3, the third basic feature map C4 and the fourth basic feature map C5 into the 1×1 convolution layer respectively, and output multi-scale feature groups with the same number of channels respectively. , i=1, 2, 3, 4;

[0086] in:

[0087] Corresponding to C2 (downsampling by 4 times, the number of channels adjusted to 256);

[0088] Corresponding to C3 (downsampling by 8 times, the number of channels adjusted to 256);

[0089] Corresponding to C4 (downsampling 16 times, the number of channels adjusted to 256);

[0090] Corresponding to C5 (downsampling by a factor of 32, the number of channels adjusted to 256).

[0091] S32, input the multi-scale feature group into the cross-domain fusion feature pyramid network for feature fusion, and output six-level fusion features;

[0092] In some embodiments, the multi-scale feature set Input the cross-domain fusion feature pyramid network to perform feature fusion. Specifically, the multi-scale feature group Fusion is performed in a bottom-up direction. In order to align features of different scales, the bilinear interpolation method is used to align the bottom layer features with the upper layer features to obtain the input features. , the original features and transition characteristics The multi-domain feature dynamic fusion module is input, and the spatial domain features and frequency domain features are adaptively weighted and fused through the channel attention mechanism. In order to adapt to the characteristics of dense face detection targets and large variations in face scale, additional features E1 and E2 are added on the basis of the four-layer features to obtain feature maps of six different scales. The six feature maps of different scales are then input into the shared context information exchange module (Inception Module) for feature fusion.

[0093] refer to Figure 2 , which is a structural diagram of the multi-domain feature dynamic fusion module (MDFM) designed by the present invention; feature fusion is often an effective method to improve the accuracy of face detection. However, existing feature fusion methods lack the interaction between shallow features and deep features, and have limitations in the extraction and fusion strategies of target features in complex scenes. The multi-domain feature dynamic fusion module (MDFM) extracts feature information from the spatial domain and frequency domain respectively through local attention branches and global attention branches. The local attention branch captures features under multi-scale receptive fields through dilated convolutions with different dilation rates, while the global attention branch uses fast Fourier transform to convert features from the spatial domain to the frequency domain, thereby obtaining the positional relationship between global features. Finally, the original features are weightedly fused through adaptive weights, effectively improving the model's ability to process complex information and enhancing the detection performance of low-quality targets.

[0094] In a specific embodiment, S32 specifically includes the following sub-steps:

[0095] S321, multi-scale feature group , i=1, 2, 3, 4 input cross-domain fusion feature pyramid network, where the multi-scale feature group , i=1 performs convolution operation to obtain additional features E1 and E2, which are downsampled by 64 and 128 times compared with the input image respectively;

[0096] Multi-scale feature group , i=2, 3, 4 and transition feature map , i=1, 2, 3 are added element by element to generate basic fusion features ;

[0097] S322, basic fusion features Input into the spatial domain branch and the frequency domain branch respectively to generate the spatial domain weight matrix and frequency domain weight matrix ;

[0098] Among them, the basic fusion features Input the spatial branch to generate the spatial weight matrix , the process specifically includes:

[0099] Basic fusion features Perform 1×1 convolution operation, reduce the number of channels to 64, and use three-level parallel dilated convolution (expansion rate 1, 2, 3) to extract multiple multi-scale spatial features. After fusing and adding the information of multiple multi-scale spatial features, the spatial domain weight matrix is ​​generated through channel splicing and 1×1 convolution. .

[0100] The basic fusion features In the input frequency domain branch, generate the frequency domain weight matrix , the process specifically includes:

[0101] Basic fusion features Perform fast Fourier transform to separate the real and imaginary components, process the frequency domain features through 3×3 convolution and ReLU activation, and then reconstruct the spatial domain features through inverse Fourier transform to generate the frequency domain weight matrix .

[0102] S323, the spatial weight matrix and frequency domain weight matrix After addition, Sigmoid activation is performed to generate the fusion weight matrix W;

[0103] S324, based on the weight matrix W multi-scale feature group and transition feature map Perform dynamic weighted fusion to obtain four intermediate fusion features;

[0104] S325 , input the four intermediate fusion features and the additional features E1 and E2 into the shared context information exchange module for feature fusion, and output a six-level fusion feature.

[0105] S33, input the six-level fusion features into the multi-task detection head for classification detection, and output the target detection position and detection results; input the six-level fusion features into the classification branch, positioning branch, and CIOU branch in the multi-task detection head respectively, and output the detection results including the target detection category, confidence score, bounding box coordinates, and quality score;

[0106] Among them, the classification branch is used to predict whether the current detection box is a target;

[0107] The positioning branch is used to predict the offset of the current detection box to the prior box;

[0108] The CIOU branch is used to predict the CIOU score of the current detection box.

[0109] In one embodiment, reference Figure 3 , which is a demonstration of the overlap of the CIOU prediction branch. Existing target detection algorithms suffer from a mismatch between classification scores and positioning accuracy. The CIOU prediction branch, by introducing shape matching factors, optimizes the limitations of the traditional IOU branch and can more accurately evaluate the quality of candidate boxes. By weighted fusion of classification scores and CIOU scores, the model can effectively filter out false positive boxes and improve detection accuracy;

[0110] in:

[0111] Predicting the CIOU score of the current detection box includes the following steps:

[0112] S331, extracting geometric information of the candidate box and the real box;

[0113] First, the geometric information of the candidate box is compared with that of the real box, and the difference in the overlapping area, center point distance, and width-to-height ratio between the candidate box and the real box is calculated as the basis for calculating the CIOU score.

[0114] S332. Based on the geometric information, calculate the CIOU score of the candidate box and the ground truth box:

[0115]

[0116] Where, The ratio of the overlapping area of ​​the candidate box and the real box to the area of ​​the union of the two; is the Euclidean distance between the center point of the candidate box and the center point of the real box; is the diagonal length of the real frame; The difference between the width-to-height ratio of the candidate frame and the width-to-height ratio of the real frame; is the weight coefficient, which is used to balance the influence of the center point distance and the width-to-height ratio;

[0117] S333. Perform weighted fusion of the CIOU score and the output result of the classification branch (classification score) to generate a comprehensive score:

[0118]

[0119] Where λ is the weight coefficient, which is used to balance the contribution of classification score and CIOU score;

[0120] S334: Sort the candidate boxes based on the comprehensive scores and filter out false positive boxes using the non-maximum suppression (NMS) algorithm, retaining the best candidate boxes as the final detection results. Specifically, first sort the candidate boxes in descending order based on the comprehensive scores, then remove the candidate boxes with high overlap, and finally output the detection results.

[0121] S34. Calculate the multi-task joint loss function to optimize the network parameters of complex scene face detection;

[0122] In one embodiment, Focal Loss is used as the classification loss. , through the modulation coefficient and focus parameters Dynamically adjust the weight distribution of difficult and easy samples, where In order to increase the probability of the model predicting a positive sample, the recognition accuracy of difficult-to-classify samples is optimized first to alleviate the problem of unbalanced sample distribution.

[0123] In a specific embodiment, DIoU Loss is used as the positioning loss. By minimizing the center point distance and overlapping area between the predicted box and the true box, the model convergence is accelerated and the accuracy of the model's target position prediction is enhanced.

[0124] In a specific embodiment, Varifocal Loss is used as the CIOU branch loss. , dynamically adjust the weights through the difficult and easy sample rebalancing mechanism, strengthen the geometric shape consistency constraints (overlap rate, center point distance and aspect ratio), and further improve the accuracy and robustness of the detection frame.

[0125] Based on the above three types of losses, multi-task joint optimization is achieved through weighted summation, and the back-propagation algorithm is used to update the network parameters to ensure that the model achieves optimal performance in classification, localization, and detection box quality assessment tasks;

[0126] in:

[0127] The multi-task joint loss function is expressed as:

[0128]

[0129] Where, represents the multi-task joint loss function; is the classification loss; For positioning loss; Losses for CIOU branch; 、 and The corresponding loss weight.

[0130] S35, perform multiple rounds of iterative training on S33 and S34;

[0131] In a specific embodiment, after 300 rounds of training for S33 and S34, the test set is passed to the trained network for testing at the end of each round of training, all indicators of the face detection network currently tested are calculated, and the parameters of the model with the best indicators are retained;

[0132] S36. Test the trained complex scene face detection network, select the network model with the best performance during the test, and then obtain the complex scene face detection model.

[0133] In one embodiment, the performance indicators during the test include the average precision , F1 score, precision rate Pr and recall rate Re key indicators, and the detection effect diagram generated by the visualization module.

[0134] S4. Input the face detection image to be detected into the complex scene face detection model, and output the target detection position and detection results.

[0135] In some embodiments, to further illustrate the effectiveness of the present method, the present method is compared with other existing methods. For fair comparison, the officially released code of the other methods is used and the experimental settings are followed. All methods are implemented in the same computing environment and quantitative and qualitative analyses are performed simultaneously.

[0136] Other methods include:

[0137] Method 1 is S3FD, which achieves efficient target detection through multi-scale feature fusion and cascade structure, especially showing good real-time performance and accuracy in face detection tasks.

[0138] Method 2 is Face R-CNN, which is based on the R-CNN framework and significantly improves the accuracy of face detection through region proposal network (RPN) and RoI pooling.

[0139] Method 3 is RetinaFace, which introduces the attention mechanism and weighted bidirectional feature pyramid, optimizes the loss function, makes the model pay more attention to the face position, and improves the detection accuracy.

[0140] Method 4 is Tinaface, which improves the detection accuracy of small targets by introducing the IOU branch to balance the classification and positioning scores and adopting the DIOU loss function.

[0141] Method 5 is YOLO5Face, which optimizes the face detection task based on YOLOV5 and further improves the detection accuracy by improving the loss function and anchor box strategy.

[0142] Method 6 is YOLO-Face, which balances the sample distribution and improves the detection accuracy of occluded targets by introducing exclusion loss and weighted loss function.

[0143] Method 7 is YOLO-FaceV2, which further optimizes YOLO-Face and improves detection performance by improving the loss function and feature extraction network.

[0144] Method 8 is MFPIDet, which is improved for overlapping and small objects. It enhances the detection ability of small and overlapping objects through multi-scale feature fusion and attention mechanism.

[0145] Method 9 is YOLOV9, an excellent method in the general detection field. By improving the network structure and optimizing the algorithm, it significantly improves detection efficiency and accuracy. Through comparative experiments with these methods, we verified the advantages and innovation of our method in face detection tasks.

[0146] Since the method of the present invention is mainly aimed at face detection in dense environments, Table 1 shows the indicators of 10 different network structures on the test set (which are divided into three subsets: simple, general and difficult according to the difficulty of detecting each face). .

[0147] Table 1 Comparative experimental results, evaluation indicators are ,Bold indicates the optimal value, and * indicates the results in the paper that are directly used.

[0148]

[0149] In order to further evaluate the detection performance of the model of the present invention, several comparative methods with better performance in the comparative experiments were selected, and their maximum F1 scores and corresponding precision and recall rates at various thresholds were analyzed. The experimental results are shown in Table 2. The F1 score can measure the comprehensive performance between precision and recall. In both simple and general subsets, the method of the present invention achieved the best results, achieving a good balance between precision and recall. In the difficult subset, the recall rate of the method of the present invention at the maximum F1 score reached the highest. Combined with Table 1, it can be seen that the proposed method performs well overall on the difficult subset, proving that the model can maintain a high precision and recall rate at most thresholds, has a stronger ability to identify and locate targets, has stronger adaptability and robustness, and is more suitable for complex scenarios in reality.

[0150] Table 2 Maximum F1 scores and corresponding precision Pr and recall Re under various thresholds. Bold indicates the optimal value.

[0151]

[0152] Finally, we selected several challenging scenes from the dataset and compared the average accuracy of each model on these difficult subsets to verify whether the proposed method can improve detection accuracy in these complex and densely populated scenes, enhancing the detection performance of low-quality faces. The experimental results are shown in Table 3. The proposed method achieved optimal accuracy in crowded scenes such as People Marching, Group, and Parade, demonstrating its ability to effectively improve detection accuracy in dense scenes. In scenes such as Festival and Sport_Fan, where faces are blurred, lighting varies greatly, and most faces are obscured by decorations, the proposed method achieved accuracy that, while lower than that of other scenes, still far surpassed that of other methods.

[0153] Comprehensive analysis shows that the method of the present invention demonstrates excellent performance in multiple scenarios, is more competitive in difficult subset detection tasks containing a large number of low-quality face targets, and is more suitable for face detection tasks in complex real-world scenarios.

[0154] Table 3 Average accuracy of difficult subsets in each scenario ,Bold indicates the optimal value.

[0155]

[0156] The effectiveness of each proposed module is verified through ablation experiments. The evaluation dataset of the ablation experiments uses the difficult subset of the Wider Face dataset.

[0157] Table 4 Ablation experiment results. Indicates that the module has been used. Indicates that the module is not used

[0158]

[0159] To verify whether the modules proposed in this paper can effectively improve the model's low-quality face detection accuracy in dense scenes, we conducted ablation experiments on a difficult subset. The experimental results are shown in Table 4. When adding modules one by one to the baseline model, significant improvements in average precision and recall are observed.

[0160] Although the specific embodiments of the invention are described in detail in conjunction with the accompanying drawings, this should not be construed as limiting the scope of protection of this patent. Within the scope described by the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.

Claims

1. A complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization, characterized by: The following steps are involved: S1. Obtain a face detection dataset and preprocess the face detection dataset to obtain a face detection training set; S2. Build a complex scene face detection network, which includes a feature extraction network, a cross-domain fusion feature pyramid network, and a multi-task detection head; S3. Use the face detection training set to train the complex scene face detection network to obtain a complex scene face detection model; S4, input the face detection image to be detected into the complex scene face detection model, and output the target detection position and detection results; The S3 specifically includes the following sub-steps: S31, inputting the face detection image into the feature extraction network to perform multi-level feature extraction and output a multi-scale feature group; S32, input the multi-scale feature group into the cross-domain fusion feature pyramid network for feature fusion, and output six-level fusion features; S33, input the six-level fusion features into the multi-task detection head for classification detection, and output the target detection position and detection results; S34. Calculate the multi-task joint loss function to optimize the network parameters of complex scene face detection; S35, perform multiple rounds of iterative training on S33 and S34; S36, testing the trained complex scene face detection network, selecting the network model with the best performance during the test, and then obtaining a complex scene face detection model; The S31 specifically includes the following sub-steps: S311, inputting the face detection image into the feature extraction network, and outputting a first basic feature map C2, a second basic feature map C3, a third basic feature map C4, and a fourth basic feature map C5 respectively through four residual stages; S312: Input the first basic feature map C2, the second basic feature map C3, the third basic feature map C4 and the fourth basic feature map C5 into the 1×1 convolution layer respectively, and output multi-scale feature groups with the same number of channels respectively. , i =1, 2, 3, 4; The S32 specifically includes the following sub-steps: S321, multi-scale feature group , i =1, 2, 3, 4 input cross-domain fusion feature pyramid network, where the multi-scale feature group , i =1 to perform convolution operation to obtain additional features E1 and E2; Multi-scale feature group , i =2, 3, 4 and transition characteristic diagram , i =1, 2, 3 are added element by element to generate basic fusion features , where the transition feature map The bilinear interpolation method is used to align the bottom layer features with the previous layer features to obtain the input features; S322, basic fusion features Input into the spatial domain branch and the frequency domain branch respectively to generate the spatial domain weight matrix and frequency domain weight matrix ; S323, the spatial weight matrix and frequency domain weight matrix After addition, Sigmoid activation is performed to generate the fusion weight matrix W; S324, based on the weight matrix W multi-scale feature group and transition feature map Perform dynamic weighted fusion to obtain four intermediate fusion features; S325, input the four intermediate fusion features and the additional features E1 and E2 into the shared context information exchange module for feature fusion, and output a six-level fusion feature; In the above S322, the basic fusion feature Input the spatial branch to generate the spatial weight matrix , the process specifically includes: Basic fusion features Perform 1×1 convolution operation, reduce the number of channels to 64, and use three-level parallel dilated convolution to extract multiple multi-scale spatial features. After fusing and adding the information of multiple multi-scale spatial features, the spatial domain weight matrix is ​​generated through channel splicing and 1×1 convolution. ; In the above S322, the basic fusion feature In the input frequency domain branch, generate the frequency domain weight matrix , the process specifically includes: Basic fusion features Perform fast Fourier transform to separate the real and imaginary components, process the frequency domain features through 3×3 convolution and ReLU activation, and then reconstruct the spatial domain features through inverse Fourier transform to generate the frequency domain weight matrix ; The S33 specifically includes: The six-level fusion features are input into the classification branch, localization branch, and CIOU branch of the multi-task detection head respectively, and the detection results including the target detection category, confidence score, bounding box coordinates and quality score are output; Among them, the classification branch is used to predict whether the current detection box is a target; The positioning branch is used to predict the offset of the current detection box to the prior box; CIOU branch, used to predict the CIOU score of the current detection box; In the CIOU branch, the CIOU score of the current detection box is predicted, which includes the following steps: S331, extracting geometric information of the candidate box and the real box; S332. Based on the geometric information, calculate the CIOU score of the candidate box and the ground truth box: Where, The ratio of the overlapping area of ​​the candidate box and the real box to the area of ​​the union of the two; is the Euclidean distance between the center point of the candidate box and the center point of the real box; is the diagonal length of the real frame; The difference between the width-to-height ratio of the candidate frame and the width-to-height ratio of the real frame; is the weight coefficient; S333. Perform weighted fusion of the CIOU score and the output of the classification branch to generate a comprehensive score: Comprehensive score = λ•Classification score + (1-λ)•CIOU score Where, λ is the weight coefficient, which is used to balance the contribution of classification score and CIOU score; S334. Sort the candidate frames based on the comprehensive scores, filter out false positive frames using the non-maximum suppression algorithm, and retain the best candidate frame as the final detection result.

2. The complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization according to claim 1 is characterized in that: In S34, the multi-task joint loss function is expressed as: Where, represents the multi-task joint loss function; is the classification loss; For positioning loss; Losses for CIOU branch; 、 and The corresponding loss weight.

3. The complex scene face detection method based on dynamic fusion of multi-domain features and CIOU optimization according to claim 1 is characterized in that: In S36, the performance indicators during the test include: average accuracy , F1 score, precision Pr and recall Re.

Images