A face detection method, device, and medium based on dual-stream feature extraction

Abstract

This invention provides a face detection method, apparatus, and medium based on dual-stream feature extraction, belonging to the field of computer vision technology. The method includes: acquiring face images to establish a dataset; preprocessing the dataset and dividing it into training and testing sets; constructing a face detection model, which includes a YOLOv11 branch, an MViTv2 branch, a feature fusion module, and a detection head. The YOLOv11 branch uses the backbone network of the YOLOv11 network to extract local features, the MViTv2 branch uses an improved MViTv2 model to extract global features, the feature fusion module fuses local and global features at corresponding scales, and the detection head outputs detection results based on the fused features; training the face detection model using the training set data; acquiring face images to be detected and inputting them into the trained face detection model to obtain detection results. This invention effectively alleviates the difficulty of matching overlapping and occluded targets in dense scenes, further improving detection performance.

Description

Technical Field

[0001] This invention relates to a face detection method, apparatus, and medium based on dual-stream feature extraction, belonging to the field of computer vision technology. Background Technology

[0002] With the continuous development of deep learning technology, object detection algorithms have been widely applied in various fields such as intelligent monitoring, autonomous driving, and behavior analysis. Especially in high-density environments such as classrooms, subways, and shopping malls, improving detection accuracy and recall has become a research focus. In recent years, the YOLO series of algorithms has become the mainstream single-stage object detection method due to its fast detection speed and high accuracy. Among them, YOLOv11, as the latest evolution of this series, has optimized the model structure and loss function while maintaining high efficiency, making it suitable for various complex scenarios.

[0003] However, in practical applications, especially in environments with severe occlusion (such as a student in a classroom being partially obscured by others), relying solely on the local visual features extracted by the YOLOv11 backbone (usually a convolutional neural network structure, such as CSPDarknet) often fails to identify severely occluded targets or large targets. In such cases, the detector's recall rate drops significantly, failing to meet practical deployment requirements.

[0004] To address the aforementioned issues, the visual Transformer architecture has been increasingly adopted in object detection in recent years due to its superior global modeling capabilities. Multiscale Vision Transformer v2 (MViTv2), as an efficient multiscale visual Transformer, possesses strong global feature extraction capabilities and can model dependencies between distant regions in an image. Therefore, it performs exceptionally well in handling targets that are locally invisible but whose context can be inferred, thus compensating for the insufficient local receptive field of convolutional networks. However, the original MViTv2 has high computational overhead and does not adequately consider the differentiation between small and large targets. Summary of the Invention

[0005] The purpose of this invention is to provide a face detection method, device, and medium based on dual-stream feature extraction, which effectively alleviates the difficulty of matching overlapping and occluded targets in dense scenes and further improves detection performance.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] A face detection method based on dual-stream feature extraction includes the following steps:

[0008] Collect facial images to create a dataset, and then preprocess the dataset to divide it into a training set and a test set;

[0009] A face detection model is constructed, which includes a YOLOV11 branch, an MViTv2 branch, a feature fusion module, and a detection head. The YOLOV11 branch uses the backbone network of the YOLOV11 network to extract local features, and the MViTv2 branch uses an improved MViTv2 model to extract global features. The feature fusion module fuses the local features and global features at corresponding scales, and the detection head outputs the detection result based on the fused features.

[0010] The face detection model is trained using the training set data to obtain the trained face detection model;

[0011] Collect the face image to be detected, input it into the trained face detection model, and obtain the detection result.

[0012] Preferably, the improved MViTv2 model includes four modules;

[0013] The first module includes convolutional layers, DW convolutional layers, CPE, and RoPE;

[0014] The second module includes window attention mechanism, Halo attention mechanism, SwiGLU activation function and feedforward neural network;

[0015] The third module includes window attention mechanism, deformable attention mechanism, SwiGLU activation function and feedforward neural network;

[0016] The fourth module includes a deformable attention mechanism and a sparse global connection layer;

[0017] The second module's output features reduce redundancy through Token Merge, and the third module's output features also reduce redundancy through Token Merge.

[0018] A cross-scale bridging module is added between the second module and the third module, and a cross-scale bridging module is added between the third module and the fourth module.

[0019] Preferably, the feature fusion module specifically includes: the YOLOv11 branch outputting feature maps at three scales. , , The MViTv2 branch outputs feature maps at three scales. , , ;

[0020] The feature map With feature map Feature fusion The feature map With feature map Feature fusion The feature map With feature map Feature fusion .

[0021] Preferably, the detection head includes a YOLO Head and a Transformer Head, wherein the YOLO Head is used for detection. , The target, the Transformer Head, is used for detection. The goal is that the Transformer Head employs a 2- or 3-layer deformable attention mechanism.

[0022] Preferably, the prediction results of YOLO Head and Transformer Head are fused by area threshold routing and NMS strategy.

[0023] Preferably, the face detection model training phase uses an improved Dynamic ATSS label allocation strategy to screen positive samples and combines it with a soft label supervision mechanism for training.

[0024] Preferably, the improved Dynamic ATSS tag assignment strategy includes:

[0025] For each ground truth bounding box, calculate its Euclidean distance to the anchor point or the center of the anchor point's free grid at all levels of feature points;

[0026] Filter candidate points, retaining those whose Euclidean distance is less than the set radius inside the true bounding box. The calculation formula is as follows:

[0027] ,

[0028] in, The width of the actual bounding box. The height of the actual bounding box. For weight fusion.

[0029] Preferably, a distillation mechanism is introduced during the training phase to perform knowledge distillation on the YOLO Head through the output of the Transformer Head, thereby improving its global perception capability in complex scenarios.

[0030] A face detection device based on dual-stream feature extraction includes a processor and a memory storing program instructions, wherein the processor is configured to execute the face detection method based on dual-stream feature extraction when running the program instructions.

[0031] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the face detection method based on dual-stream feature extraction.

[0032] The advantages of this invention are:

[0033] This invention achieves complementarity between global and local features by combining the local feature extraction capabilities of YOLOv11 with the optimized global modeling capabilities of MViTv2. The YOLOv11 backbone excels at extracting local details such as edges and textures from images, while the MViTv2 branch effectively captures long-range dependencies and contextual information in images through a multi-scale visual Transformer architecture. This combination significantly improves detection accuracy in heavily occluded or multi-object-dense scenes, effectively addressing the problem of insufficient local receptive fields in traditional convolutional networks.

[0034] This invention employs a differentiated processing strategy for targets of different sizes. The YOLO Head focuses on the rapid and efficient detection of small and medium-sized targets, maintaining its lightweight advantage; while the Transformer Head, through its lightweight design, enhances the detection accuracy and stability for targets of a wider range. This avoids the shortcomings of a single detection path in handling targets of different sizes, thus improving the overall stability and reliability of the detection system.

[0035] This invention introduces a Token Merge mechanism and a cross-scale bridging module into the MViTv2 branch, effectively reducing redundant computation while maintaining three-scale feature output. Token Merge progressively reduces the number of tokens, while the cross-scale bridging module enhances semantic consistency between features at different scales, avoiding redundant feature extraction.

[0036] This invention introduces an improved Dynamic ATSS label allocation strategy and combines it with a center-radius positive sample selection method based on a softfusion mechanism. This strategy effectively alleviates the difficulty of matching overlapping and occluded targets in dense scenes by dynamically calculating the distance from candidate positive samples to the center point of the ground truth bounding box and introducing a radius constraint mechanism and a soft label supervision mechanism. Furthermore, dedicated rules for large target matching further improve the accuracy of large target detection, enabling the overall detection system to perform excellently in various complex scenarios.

[0037] This invention maintains the core YOLO structure, optimizing and improving only the MViTv2 branch, and integrating the Transformer Head and the optimized MViTv2 as modular components into the overall network. This makes the system more flexible and convenient to deploy in practice, allowing for flexible adjustment of the model size based on computing resources and application requirements. Furthermore, this method eliminates the need for decoupling and retraining feature maps, further improving the efficiency and feasibility of engineering implementation. Attached Figure Description

[0038] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0039] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0040] Figure 2 This is a schematic diagram of the model structure of the present invention.

[0041] Figure 3 This is a schematic diagram of the improved MViTv2 model of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1

[0044] An improved face detection method based on two-stream feature extraction is as follows:

[0045] MViTv2 structure optimization: Introducing mechanisms such as hybrid attention (window attention + deformable attention), token merge, and cross-scale bridge to maintain the output features at three scales (1 / 8, 1 / 16, 1 / 32), thereby reducing redundant computation and enhancing the modeling ability for small and occluded targets.

[0046] Detection head differentiation design: YOLO Head is continued to be used for small and medium targets; the large target detection branch (1 / 32 feature layer) is replaced with a lightweight Transformer Head (based on the DeformableDETR idea), which utilizes global context to improve the detection accuracy and stability of large-scale targets.

[0047] Furthermore, to further improve the target detector's ability to assign labels to small targets and occluded targets, this invention introduces an improved label assignment strategy, Dynamic ATSS (Dynamic Adaptive Training Sample Selection), and proposes a center_radius positive sample selection method based on the softfusion mechanism.

[0048] The main innovations of this tag assignment module include:

[0049] Dynamic ATSS candidate positive sample selection: For each ground truth box (GT), calculate the Euclidean distance from the feature point to the anchor or anchor-free grid center point in all layers, and select the top-k nearest neighbors of each layer as candidate positive samples.

[0050] The center_radius constraint mechanism further filters candidate points, retaining only those whose centers are within a certain radius inside the ground truth box, thus eliminating edge-irrelevant interference.

[0051] The soft fusion center_radius strategy merges multiple radius definitions and controls the weights of the two methods through weighting. The weighting coefficients are predicted by the network or automatically selected from a fixed set of candidates, so that the center_radius can be dynamically adjusted according to the image context.

[0052] Soft label supervision mechanism: The confidence of candidate positive samples not only comes from IoU, but also combines information such as predicted class score and spatial location. The soft_target design is introduced so that the network can adaptively adjust the weight distribution of positive samples.

[0053] Through the above mechanism, the present invention fully integrates context, spatial and category information in the positive sample allocation stage, effectively alleviating the difficulty of matching overlapping and occluded targets in dense scenes, and further improving detection performance.

[0054] Specifically, such as Figures 1-2 As shown, it includes the following steps:

[0055] S1: Collect face images to build a dataset, and then divide the dataset into training and test sets after preprocessing;

[0056] S2: Construct a face detection model, which includes a YOLOV11 branch, an MViTv2 branch, a feature fusion module, and a detection head. The YOLOV11 branch uses the backbone network of the YOLOV11 network to extract local features, and the MViTv2 branch uses an improved MViTv2 model to extract global features. The feature fusion module fuses the local features and global features at corresponding scales, and the detection head outputs the detection result based on the fused features.

[0057] S3: Train the face detection model using the training set data to obtain the trained face detection model;

[0058] S4: Collect the face image to be detected, input it into the trained face detection model, and obtain the detection result.

[0059] As a refinement of the above embodiments, such as Figure 3 As shown, the improved MViTv2 model includes four modules;

[0060] The first module includes convolutional layers, DW convolutional layers, CPE (Conditional Position Encoder), and RoPE.

[0061] The second module includes window attention mechanism, Halo attention mechanism, SwiGLU activation function and feedforward neural network.

[0062] The third module includes window attention mechanism, deformable attention mechanism, SwiGLU activation function and feedforward neural network.

[0063] The fourth module includes a deformable attention mechanism and a sparse global connection layer.

[0064] The second module outputs features with reduced redundancy through Token Merge, and the third module outputs features with reduced redundancy through Token Merge.

[0065] A cross-scale bridging module is added between the second module and the third module, and a cross-scale bridging module is added between the third module and the fourth module.

[0066] As a refinement of the above embodiments, the feature fusion module specifically includes: the YOLOv11 branch outputs feature maps at three scales. , , The MViTv2 branch outputs feature maps at three scales. , , .

[0067] The feature map With feature map Feature fusion The feature map With feature map Feature fusion The feature map With feature map Feature fusion .

[0068] As a refinement of the above embodiments, the detection head includes a YOLO Head and a Transformer Head, wherein the YOLO Head is used for detection. , The target, the Transformer Head, is used for detection. The goal is that the Transformer Head employs a 2- or 3-layer deformable cross-attention mechanism.

[0069] As a refinement of the above embodiments, the prediction results of YOLO Head and Transformer Head are fused through area threshold routing and NMS strategy.

[0070] As a refinement of the above embodiments, the face detection model training phase is based on an improved Dynamic ATSS label allocation strategy for positive sample selection, and training is combined with a soft label supervision mechanism.

[0071] As a refinement of the above embodiments, the improved Dynamic ATSS tag allocation strategy includes:

[0072] For each ground truth bounding box, calculate its Euclidean distance to the anchor point or the center of the anchor point's free grid at all levels of feature points;

[0073] Filter candidate points, retaining those whose Euclidean distance is less than the set radius inside the true bounding box. The calculation formula is as follows:

[0074] ,

[0075] in, The width of the actual bounding box. The height of the actual bounding box. For weight fusion.

[0076] As a refinement of the above embodiments, a distillation mechanism is introduced during the training phase. The YOLO Head is distilled with the output of the Transformer Head to improve its global perception capability in complex scenarios.

[0077] Top-k removal: Unlike traditional ATSS, this strategy removes the restriction on top-k candidates and relies only on the central region constraint to screen positive samples, making it closer to the actual detection density.

[0078] Dynamic threshold allocation: In candidate positive samples, the threshold for separating positive and negative samples is dynamically set using statistical methods based on the IoU between the positive sample and the corresponding ground truth bounding box, thereby improving the robustness of label allocation.

[0079] Example 2

[0080] This disclosure also provides a face detection device based on dual-stream feature extraction, including a processor and a memory. Optionally, the device may further include a communication interface and a bus. The processor, communication interface, and memory can communicate with each other via the bus. The communication interface can be used for information transmission. The processor can call logical instructions in the memory to execute the face detection method based on dual-stream feature extraction described in the above embodiments.

[0081] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0082] Memory, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor executes the program instructions / modules stored in the memory to perform functional applications and data processing, thereby implementing the face detection method based on dual-stream feature extraction in the above embodiments.

[0083] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory.

[0084] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the aforementioned face detection method based on dual-stream feature extraction.

[0085] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0086] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0087] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A face detection method based on dual-stream feature extraction, characterized in that, Includes the following steps: Collect facial images to create a dataset, and then preprocess the dataset to divide it into a training set and a test set; A face detection model is constructed, which includes a YOLOV11 branch, an MViTv2 branch, a feature fusion module, and a detection head. The YOLOV11 branch uses the backbone network of the YOLOV11 network to extract local features, and the MViTv2 branch uses an improved MViTv2 model to extract global features. The feature fusion module fuses the local features and global features at corresponding scales, and the detection head outputs the detection result based on the fused features. The face detection model is trained using the training set data to obtain the trained face detection model; Collect the face image to be detected, input it into the trained face detection model, and obtain the detection result; The improved MViTv2 model consists of four modules in sequence; The first module includes convolutional layers, DW convolutional layers, CPE, and RoPE; The second module includes window attention mechanism, Halo attention mechanism, SwiGLU activation function and feedforward neural network; The third module includes window attention mechanism, deformable attention mechanism, SwiGLU activation function and feedforward neural network; The fourth module includes a deformable attention mechanism and a sparse global connection layer; The second module's output features reduce redundancy through Token Merge, and the third module's output features also reduce redundancy through TokenMerge. A cross-scale bridging module is added between the second module and the third module, and a cross-scale bridging module is added between the third module and the fourth module.

2. The face detection method based on dual-stream feature extraction according to claim 1, characterized in that, The feature fusion module specifically includes: the YOLOv11 branch outputs feature maps at three scales. , , The MViTv2 branch outputs feature maps at three scales. , , ; The feature map With feature map Feature fusion The feature map With feature map Feature fusion The feature map With feature map Feature fusion .

3. The face detection method based on dual-stream feature extraction according to claim 2, characterized in that, The detection head includes a YOLO Head and a Transformer Head, the YOLO Head being used for detection. , The target, the Transformer Head, is used for detection. The goal is that the Transformer Head employs a 2- or 3-layer deformable attention mechanism.

4. The face detection method based on dual-stream feature extraction according to claim 3, characterized in that, The prediction results of YOLO Head and Transformer Head are fused by area threshold routing and NMS strategy.

5. The face detection method based on dual-stream feature extraction according to claim 1, characterized in that, The face detection model is trained using an improved Dynamic ATSS label allocation strategy to select positive samples and combined with a soft label supervision mechanism.

6. The face detection method based on dual-stream feature extraction according to claim 5, characterized in that, The improved Dynamic ATSS tag assignment strategy includes: For each ground truth bounding box, calculate its Euclidean distance to the anchor point or the center of the anchor point's free grid at all levels of feature points; Filter candidate points, retaining those whose Euclidean distance is less than the set radius inside the true bounding box. The calculation formula is as follows: ， in, The width of the actual bounding box. The height of the actual bounding box. For weight fusion.

7. The face detection method based on dual-stream feature extraction according to claim 5, characterized in that, A distillation mechanism is introduced during the training phase to perform knowledge distillation on the YOLO Head through the output of the Transformer Head, thereby improving its global perception capability in complex scenarios.

8. A face detection device based on dual-stream feature extraction, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the face detection method based on dual-stream feature extraction as described in any one of claims 1-7 when running the program instructions.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the face detection method based on dual-stream feature extraction as described in any one of claims 1-7.

Images