Face detection method for dense crowd in complex environment

Abstract

The present application is directed to the problem of dense crowd face detection in complex environments, and designs the field of face detection in computer vision. The method provides a model named deep and compact face detection (DCFD), which is referred to as DCFD for short. The method comprises: building an input layer of the DCFD model; building a main feature extraction network of the DCFD model, using an EfficientNetV2 network to replace the original main network module, and using a large core attention mechanism to replace a traditional convolution block to give the network long end support and realize the ability of global receptive field; building the connection between the main network of the DCFD model and the feature module, introducing an ECA attention mechanism to strengthen the feature extraction capability of the model, so that the model pays more attention to the detected target; building a feature fusion module of the DCFD model: using an improved NAS-FPN image pyramid to replace the original FPN structure. The pedestrian dataset is sent into the improved RetinaFcae model, and the Adam optimizer is used for training to obtain the DCFD model and apply it to dense crowd face detection in complex environments.

Description

Technical Field

[0001] This application relates to the field of CV (Computer Vision), and more specifically, to a method for pedestrian face detection. Background Technology

[0002] Face detection is a crucial component of face recognition. Only by detecting faces and extracting relevant information can it be truly applied to real-world needs, and it is now widely used in security, traffic management, and military fields. However, in complex environments, such as complex backgrounds, low resolution, invisibility, and insufficient signal, face detection still faces significant challenges. These challenges drive further in-depth research into the field of face detection, especially in densely populated scenes, which presents a particularly challenging task.

[0003] Recently, with the widespread application of artificial intelligence in various fields and the emergence of deep learning, facial recognition technology has been integrated into people's daily production and life. In this field, the face detection module, as a key component, has achieved significant progress and success. Face detection is an important part of facial recognition; only by detecting faces and extracting relevant information can it be truly applied to practical needs. It is now widely used in security, traffic management, military, and other fields. However, in complex environments, such as complex backgrounds, low resolution, invisibility, and insufficient signal, face detection still faces significant challenges.

[0004] These challenges have spurred deeper research into face detection, especially in densely populated environments where overcoming these difficulties is crucial. Face detection primarily follows two main approaches: feature-based methods and deep learning-based methods.

[0005] Traditional face detection algorithms typically involve extracting image features and using classifiers for classification. This approach suffers from limitations in accuracy and efficiency when faced with complex situations such as dense crowds, congestion, and changing lighting. Therefore, seeking more advanced methods to address these challenges has become an urgent need.

[0006] Existing patent application CN118470767A discloses a face detection method and apparatus based on a YOLOv8 object detection model. First, a YOLOv8-based object detection model is constructed and trained. Then, the face image to be detected is input into the trained object detection model to obtain the recognition result. The object detection model includes a backbone network, a neck network, and a prediction head. The backbone network includes sequentially arranged convolutional modules and C2f modules, as well as a feature pyramid module at the end. The second convolutional block in the bottleneck block of the C2f module is a deformable convolution DCNv2. The feature pyramid module introduces a large separable convolutional attention mechanism LSKA between the splicing module (which fused pooling layer features) and the second convolutional block.

[0007] CN118570261A discloses a low-power, lightweight face detection and tracking method and system. The method includes the following steps: S1, acquiring face images using a camera and initializing high resolution and frame rate; S2, adaptively adjusting the resolution of the acquired face images, processing them through a face detection model, identifying faces in the images through the face detection model, and outputting face detection confidence and face detection bounding boxes; S3, adjusting the resolution based on the face confidence; S4, tracking the detected faces continuously using a face detection and tracking algorithm; S5, analyzing the face tracking results and adaptively adjusting the frame rate; S6, outputting the adjusted frame rate to the camera to acquire face images, and repeatedly executing steps S2 to S5 to achieve adaptive adjustment of resolution and frame rate.

[0008] CN118506412A discloses a face detection method for low-light scenes based on unsupervised domain adaptation. It includes: Step 1: using the Wideface dataset as the source domain and the low-light face dataset as the target domain; Step 2: constructing a low-light scene face detection algorithm consisting of a feature extraction network, an unsupervised domain adaptation component based on backpropagation, and a face detection branch network. This invention proposes a low-light scene face detection algorithm based on unsupervised domain adaptation. The algorithm consists of three main parts: a feature extraction network, an unsupervised domain adaptation component based on backpropagation, and a face detection branch network. RetinaFace is used as the face detection branch network, and the unsupervised domain adaptation component based on backpropagation is used to reduce the feature distribution difference between the source and target domain images in an unsupervised manner.

[0009] However, the aforementioned face detection methods have some problems: the pedestrian environment is not complex enough, and the detection accuracy is only improved under low power consumption and low light conditions; they do not cover face detection in dense crowds; and there is no clear solution for face detection in complex environments with dense crowds. Summary of the Invention

[0010] This invention provides a face detection method for complex environments, comprising: face detection and facial landmark localization. The face detection part uses an improved EfficientNetV2 network as the backbone network. Based on the complex environment, a lightweight and compact face detection method is proposed, called DCFD (Deep and Compact Face Detection), which includes:

[0011] 1) Construct the input layer of the DCFD model;

[0012] 2) Construct the backbone feature extraction network of the DCFD model, replace the original backbone network module with EfficientNetV2 network, and replace the traditional convolutional block with a large kernel attention mechanism to give the network the ability to support the long end and realize the global receptive field, so as to facilitate the full extraction of coarse-grained global features from the image.

[0013] 3) Establish the connection between the backbone network and feature modules of the DCFD model, introduce the ECA attention mechanism, enhance the model's ability to extract features, and make the model pay more attention to the detected target;

[0014] 4) Constructing the feature fusion module for the DCFD model: The improved NAS-FPN image pyramid is used to replace the original FPN structure;

[0015] 5) Feed the pedestrian dataset into the improved RetinaFcae model with an input image size of 640×640, epoch=150, and train it using the Adam optimizer to obtain the best DCFD model for subsequent face detection.

[0016] This application may yield one or more beneficial effects:

[0017] (1) Use the lightweight EfficientNetV2 network and introduce large kernel attention to better integrate local context information and consider long-distance dependencies, so that the model attention is more focused on facial features;

[0018] (2) The loss function was optimized by using the focus loss function instead of the cross-entropy loss function to balance the training of positive and negative samples and reduce the loss of the easily trained positive samples.

[0019] (3) An improved efficient channel attention mechanism (ECA) is introduced between the backbone network feature extraction network and the feature fusion module to improve the utilization rate of feature information in the feature fusion module;

[0020] (4) The feature fusion module was reconstructed, and an improved NAS-FPN image pyramid was used instead of the original FPN. This improvement enhances the detection accuracy for small faces and improves detection performance in densely populated environments.

[0021] The accuracy and robustness. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating an algorithm implementation of the present invention;

[0023] Figure 2 This is a block diagram of a DCFD network architecture according to an embodiment of the present invention;

[0024] Figure 3 This is a block diagram of an improved MBConv structure according to the present invention;

[0025] Figure 4 This is a structural diagram of an LKA module according to an embodiment of the present invention;

[0026] Figure 5 This is a block diagram of an improved ECA attention mechanism structure according to the present invention;

[0027] Figure 6 This is a network structure diagram of a NAS-FPN according to an embodiment of the present invention;

[0028] Figure 7 This is a block diagram of a NAS-FPN architecture according to an embodiment of the present invention. Detailed Implementation

[0029] To address the challenges of face detection in complex environments, particularly in densely populated areas, RetinaFace underwent a series of key improvements, resulting in a significant performance enhancement.

[0030] Upgraded EfficientNetV2 network architecture: An improved lightweight EfficientNetV2 network is adopted as the backbone of RetinaFace, coupled with a large-kernel attention mechanism, aiming to extract facial features more effectively. Through algorithm optimization, not only is detection accuracy guaranteed, but network overhead is also reduced, resulting in exceptional performance in face detection in densely populated scenes. Reconstruction of the spatial attention module: A spatial attention module is introduced into the ECA attention mechanism to better focus on the representation of spatial information, which not only improves the model's generalization ability but also more effectively captures the spatial relationships of faces in dense crowds.

[0031] Improved Dense Loss Function: To achieve finer-grained model tuning, an improved dense loss function is adopted, enabling the model to more directly handle pixel-level predictions. This results in more accurate predictions of model structure and surroundings in face detection tasks within dense crowds. Spatial Fusion Module Reconstruction: Improved NAS-FPN achieves adaptive gain for image targets. This not only reduces the number of model parameters and improves computational efficiency but also achieves more accurate computation, providing a more robust solution for subsequent practical applications of face detection.

[0032] The present invention will now be described in further detail with reference to the accompanying drawings:

[0033] Figure 1 This is a flowchart illustrating an algorithm implementation of the present invention. As shown, this implementation method includes the following steps:

[0034] Step 1: Preprocess the dataset;

[0035] Step 1-1. Systematically label, classify, and filter the WiderFace dataset to create a new WiderFace subset containing approximately 29,000 images.

[0036] Step 1-2. Select the LFW dataset as the second dataset for face detection;

[0037] Step 2: Build and train an improved RetinaFace model for face detection. This invention is based on the RetinaFace model and improves it for face detection in dense crowds. The resulting improved RetinaFace model structure is named the DCFD model, as shown in the diagram. Figure 2 As shown, it includes an input layer, a backbone network layer, a feature fusion layer, a prediction layer, a decoding layer, and NMS. The construction methods of each layer are shown below.

[0038] Step 2-1: Construct the DCFD input layer and perform Mosaic data augmentation. By stitching four images into one image through flipping, scaling, and color gamut changes within the region, the processing efficiency is greatly enhanced. The adaptive image scaling method is as follows: First, the size of the training image is specified as 640×640 (unit: pixels). The image is scaled. Then, the size of the horizontal and vertical pixels after scaling is calculated, represented as x and y, respectively, where x = x × min{x / 640, y / 640}, y, = y × min{x / 640, y / 640}. If x < 640, then a black border of height [(640-x) % 64] / 2 is added above and below the corresponding x height to finally form an image of size 640×640. The same applies to the y direction. The min operation means taking the minimum value within the curly braces, and % means taking the remainder.

[0039] Step 2-2: Construct an improved DCFD model backbone feature extraction network. Use an improved EfficientNetV2 (its network structure is shown in Table 1) instead of the original RetinaFace network as the backbone network, and introduce large kernel attention. Concatenate the pooled convolutional features from the main MBConv module of EfficientNetV2, and then input them as a whole into the fully connected layer. Its main architecture includes 1×1 ordinary convolutions to increase dimensionality, followed by k×k depthwise convolutions, where k can be 3×3 or 5×5, such as... Figure 3 As shown. These structural layers utilize the Swish activation function and combine it with the SE (Squeeze-and-Excitation) module. This effectively captures low-level texture information and high-level semantic information of the image, thereby improving detection capabilities. LKA convolutional blocks are used instead of traditional convolutional blocks, such as... Figure 4 The LKA module structure is shown below. The goal of this innovative design is to endow the network with the ability to support long ends and achieve a global receptive field, thereby facilitating the extraction of coarse-grained global features from images. The introduction of the LKA module further compresses the output channels and the number of repeating layers in EfficientNetV2, enabling the extraction of multi-dimensional and multi-scale fine-grained features from global image features. The improved EfficientNetV2 algorithm network structure is shown in Table 1. In Table 1, SE represents the self-attention module, where 0.25 is the coefficient of the first fully connected layer in the SE module, representing one-quarter of the number of channels in the input feature matrix of this module. K represents the convolution kernel size, while the coefficients after MBConv are the dilation factor.

[0040] Table 1 Network Structure of the Improved EfficientNetV2 Algorithm

[0041]

[0042] Steps 2-3: Construct the DCFD model feature fusion module. An improved, efficient channel attention mechanism (ECA) is introduced between the backbone network feature extraction network and the feature fusion module, such as... Figure 5 As shown, while retaining the one-dimensional convolutional advantages of the ECA attention mechanism, a spatial attention module (SA module) is added to better capture features in different spaces in face images. This new feature aims to enhance global information attention and improve network performance. Through one-dimensional convolution, the feature extraction capability is improved without dimensionality reduction, thereby improving model performance with minimal parameters and computational cost. This improvement achieves adaptability to face detection in complex backgrounds. The size of the convolution kernel k is related to the number of channels C. The larger the value of C, the larger the value of K. γ and b are nonlinear parameters, where γ is set to 2 and b is set to 1, as shown in formula (1):

[0043]

[0044] The feature fusion module was restructured, replacing the original feature pyramids (FPNs) with an improved NAS-FPN image pyramid, such as... Figure 6 As shown, Figure 6 In the diagram, P3-P7 on the left represent the input feature layers, while P3-P7 within the white box represent the output feature layers. GP stands for Global Pooling, which uses the semantic information contained in higher layers as a global feature map for computation. RCB stands for ReLU-Conv-BatchNorm, indicating that the data has undergone ReLU activation, batch processing, and batch normalization. The Feature Pyramid Network (FPN) is used to handle variations in objects at different scales. The original FPN network structure can extract diverse feature maps from different dimensions of different networks. However, directly performing 1×1 convolutions on each extracted feature map to reduce the number of kernels would lead to an overly computationally intensive task. As network depth increases, the resolution of the FPN decreases, affecting the detection of key points on small targets like faces. The DCFD algorithm uses the EfficientNetV2 network as its backbone. To improve network accuracy without increasing the number of network parameters and computational cost, a NAS-FPN structure is used to recombine and fuse feature maps. High-scale features are globally pooled, multiplied by low-scale features after 1×1 convolutions, and then summed to obtain the final output. In this way, the high-level feature maps, through global pooling and 1×1 convolution, guide the computation of low-level feature maps in the form of weights, achieving more compact feature fusion. The improved network structure diagram is as follows: Figure 7 As shown. This improvement aims to optimize the feature extraction and fusion mechanism of the network, thereby enhancing the performance of the DCFD algorithm in target detection tasks;

[0045] Steps 2-4. Perform three types of predictions, including classification prediction, face bounding box regression, and face landmark localization. Use the focus loss function instead of the cross-entropy loss function to balance the training of positive and negative samples and reduce the loss on easily trained positive samples. The focus loss function is introduced, and its unique feature is its ability to adaptively focus on difficult samples. That is, it assigns higher weights to samples that are easily misclassified, while reducing the weights to relatively easy-to-classify samples. This mechanism helps the model focus more on key regions, thereby improving the performance of face detection. The cross-entropy loss function used is shown in Equation (2):

[0046]

[0047] yi is the label value of the i-th sample, which can be 0 or 1. Pi is the prediction probability of the model that the i-th sample is a positive sample, with a value of [0,1]. This loss function is used to predict the difference between the current model's prediction and the actual label. If the sample is a positive sample (yi = 1), the closer pi is to 1, the better the positive sample can be predicted, as shown in formula (3):

[0048] CE = -log(pi) (3)

[0049] The closer pi is to 0, the better it can predict negative samples, as shown in formula (4):

[0050] CE = -log(1-pi) (4)

[0051] To address the problems of imbalanced samples and difficulty in classification, a focus loss function is introduced.

[0052] It adjusts the loss by introducing a focus parameter, and its prediction accuracy Pt is defined as shown in formula (5):

[0053]

[0054] To address the imbalanced sample problem, a weighting factor α is introduced, with a value ranging from [0,1]. When yi = 1, training sample i is considered a positive sample, and its weight is reset to α; while when yi = 0, training sample i is considered a negative sample, and its weight is set to (1-α). Based on the definition of this weighting factor α, a weighted cross-entropy formula is defined to adjust the cross-entropy loss, as shown in formula (6):

[0055] CE(p t α t )=-α t log(p t (6)

[0056] A weighting factor αt is introduced to adjust the weight ratio of positive and negative samples in the cross-entropy to address the problem of imbalanced sample data. To address the issue of distinguishing "hard" samples, an adjustment factor γ with a range of [0, +∞] is introduced into the cross-entropy loss function. The resulting function can be expressed as formula (7):

[0057] CE(p t ,γ)=-(1-p t ) γ log(p t (7)

[0058] The modulating factor γ can be used in the following way:

[0059] ① When a sample is misclassified and pt is small (indicating a sample that is difficult to classify), adjusting the parameter (1-Pt)γ close to 1 has almost no effect on the loss function. This means that the model will maintain a large loss in this case, making people pay more attention to these difficult-to-classify samples;

[0060] ② When the sample pt is 1 (a sample that is correctly classified), adjust the parameter (1-Pt)γ to 0. This means that the loss for a sample that is correctly classified is 0, and the model will not pay attention to such samples.

[0061] ③ When the sample pt is 0.9 (a correctly classified sample that is easy to classify) and γ = 2, adjust the parameter (1-Pt)γ to 0.01, and the sample loss function is reduced to 1 / 100 of the original, so that the model pays less attention to such easily classified samples;

[0062] ④ When γ = 0, the focus loss function becomes the cross-entropy loss function, no longer considering the difficulty of classifying the samples, and all samples are given the same weight;

[0063] Formula (8) for the focus loss function is used to dynamically adjust the loss function based on the classification of the samples and the adjustment factor γ, so as to better process the samples.

[0064] FL(p t )=-α(1-p t ) γ log(p t (8)

[0065] In face recognition tasks, the presence of a large number of prior bounding boxes can lead to most samples being treated as background. This corresponds to the role of the γ parameter in the FocalLoss function, which is introduced to make the model pay more attention to samples that are difficult to identify. This directly causes the model to misidentify a large range of background samples as faces, resulting in low confidence scores. In the subsequent non-maximum suppression (NMS) stage, low-confidence faces are usually excluded.

[0066] The results show that introducing the focus loss function does improve the model's ability to detect small faces. However, in the post-processing stage, due to excessive prior boxes and oversensitivity detection, the number of faces is very large. In some "hard images," there may be tens of thousands of faces (most of which can be eliminated), which significantly slows down the NMS algorithm. Therefore, to address this issue, this paper adopts a strategy of using the cross-entropy loss function in the first 145 cycles. The advantage of this strategy is that it reduces the number of prior boxes, thereby reducing the burden on the NMS stage.

[0067] Steps 2-5. Adjust the prediction results from the previous step and refine the positions of the face bounding box and facial landmarks;

[0068] Steps 2-6. Using EIOU as the method for NMS, highly overlapping prediction boxes are effectively removed;

[0069] Steps 2-7. Input the WiderFace dataset and the LFW dataset into the DCFD model. The training parameters are shown in Table 2.

[0070] Table 2 Training parameter configuration

[0071]

[0072] The ablation experiments on the WiderFace dataset are shown in Table 3:

[0073] Table 3 Ablation experiments on the WiderFace dataset

[0074]

[0075] The ablation experiments on the LFW dataset are shown in Table 4:

[0076] Table 4 Ablation Experiments on the LFW Dataset

[0077]

Claims

1. A pedestrian face detection method based on an improved RetinaFace architecture, characterized in that, include: Step 1: Obtain sample data, which includes different face detection datasets, and perform data preprocessing; Step 2: Build a lightweight and compact face detection model, DCFD, for face detection; Build and train the DCFD model, including: (1) Construct the input layer of the DCFD model and perform Mosaic data augmentation. The training image size is specified as 640×640. The image is scaled up, and then the size of the horizontal and vertical pixels after scaling is calculated, which are represented as x and y respectively. Where x = x × min{x / 640, y / 640}, y = y × min{x / 640, y / 640}. If x < 640, then add a black border of height [(640-x)%64] / 2 above and below the corresponding x height to finally make up an image of size 640×640. The operation in the y direction is the same. Where % means the remainder operation. (2) Construct the backbone feature extraction network of the DCFD model, replace the original backbone network module with the improved EfficientNetV2 network, and replace the traditional convolutional block with the large kernel attention mechanism. When improving the EfficientNetV2 network, the convolutional features pooled by the MBConv module of EfficientNetV2 are concatenated and then input as a whole into the fully connected layer. Its architecture includes a 1×1 ordinary convolution to increase the dimension, followed by a k×k depthwise convolution, where k can be 3×3 or 5×5, and the use of the Swish activation function combined with the SE module; LKA convolutional blocks are used to replace the traditional convolutional blocks. (3) Establish the connection between the backbone network and feature modules of the DCFD model and introduce an improved ECA attention mechanism. While retaining the one-dimensional convolutional advantage of the ECA attention mechanism, a spatial attention module is added to form an improved ECA attention mechanism. (4) Constructing the feature fusion module of the DCFD model: The improved NAS-FPN image pyramid is used to replace the original FPN structure. The NAS-FPN structure is used to reassemble and fuse the feature maps. The high-scale features are globally pooled, multiplied with the low-scale features by 1×1 convolution, and then added to finally output the feature map. (5) The pedestrian dataset was fed into the improved RetinaFcae model with an input image size of 640×640, epoch=150, and trained using the Adam optimizer to obtain the best DCFD model for subsequent face detection. In the first 145 epochs, the cross-entropy loss function was used, and then the focus loss function was used instead of the cross-entropy loss function to balance the training of positive and negative samples. EIOU was used as the non-maximum suppression (NMS) method to remove highly overlapping prediction boxes.

2. The face detection method based on the improved ReinaFace according to claim 1, characterized in that, Step 1 includes: (1) Collect pedestrian datasets, i.e. collect pedestrian face datasets based on the training dataset of the improved RetinaFace model: systematically label, classify and filter the WiderFace dataset to create a new WiderFace data subset; (2) Collect the LFW dataset, which includes 13,233 face images covering 5,749 unique identities, reflecting the diversity of face changes in different ages, lighting and poses, and simulating complex conditions in the real world.

Images