Dense crowd face detection method in complex environment

Abstract

The invention relates to the field of face detection in computer vision, and aims to solve the problem of face detection of dense crowds in a complex environment. According to the method, a model named as deep dense face detection (Deep and Compact Face Detection) is provided, and the model is called as DCFD for short. The method comprises the following steps: constructing an input layer of a DCFD model; the method comprises the following steps: establishing a DCFD model backbone feature extraction network, replacing an original backbone network module with an OfficientNetV2 network, replacing a traditional convolution block with a large kernel attention mechanism, and endowing the network with long-end support and global receptive field realization capability; establishing a relation between a DCFD model backbone network and a feature module, introducing an ECA attention mechanism, enhancing the feature extraction capability of the model, and enabling the model to pay more attention to a detected target; constructing a feature fusion module of the DCFD model; and replacing an original FPN structure with an improved NAS-FPN image pyramid. And the pedestrian data set is sent to an improved RetinaFcae model, an Adam optimizer is used for training, and a DCFD model is obtained and is used for dense crowd face detection in a complex environment.

Description

Technical Field

[0001] The present application relates to the field of CV (Computer Vision), and more specifically, to a method for pedestrian face detection. Background Art

[0002] Face detection is an important part of face recognition. Only when the face is detected and the relevant content is extracted can it be truly applied to practical needs. It is now widely used in security, traffic management, military and other fields. However, in complex environments, face detection still faces huge challenges due to uncontrollable factors such as complex background, low resolution, invisible, insufficient signal, etc. It is these challenges that prompt more in-depth research in the field of face detection, especially in crowded scenes, which is a challenging task.

[0003] Recently, with the widespread application of artificial intelligence in various fields and the introduction of deep learning, face recognition technology has been integrated into people's daily production and life. In this field, the face detection module, as a key link, has made remarkable progress and achievements. Face detection is an important part of face recognition. Only when the face is detected and the relevant content is extracted can it be truly applied to actual needs. It is now widely used in security, traffic management, military and other fields. However, in complex environments, face detection still faces huge challenges due to uncontrollable factors such as complex background, low resolution, invisibility, and insufficient signal.

[0004] It is these challenges that prompt more in-depth research in the field of face detection, especially in crowded scenes, where the above difficulties need to be overcome. There are two main directions of face detection, namely feature-based methods and deep learning-based methods.

[0005] Traditional face detection algorithms usually involve extracting image features and using classifiers for classification. This method has certain accuracy and efficiency limitations in complex situations such as dense crowds, congestion, and lighting changes. Therefore, it has become an urgent requirement to seek more advanced methods to cope with these challenges.

[0006] The existing patent application CN118470767A discloses a face detection method and device based on the YOLOv8 target detection model. First, a target detection model based on YOLOv8 is constructed and trained, and then the face image to be detected is input into the trained target detection model to obtain a recognition result. The target detection model includes a backbone network, a neck network and a prediction head; the backbone network includes a convolution module and a C2f module arranged in sequence, and a feature pyramid module at the end; the second convolution block in the bottleneck block of the C2f module is a deformable convolution DCNv2; the feature pyramid module introduces a large separation convolution attention mechanism LSKA between the splicing module of the splicing fusion pooling layer features and the second convolution block.

[0007] CN118570261A discloses a low-power lightweight face detection and tracking method and system, which includes the following steps: S1, using a camera to collect face images, initializing high resolution and frame rate; S2, adaptively adjusting the resolution of the collected face images, processing them through a face detection model, identifying faces in the images through the face detection model, and outputting face detection confidence and face detection frames; S3, adjusting the resolution according to the face confidence; S4, using a face detection and tracking algorithm to track the detected faces in consecutive frames; S5, analyzing the face tracking results, and adaptively adjusting the frame rate; S6, outputting the adjusted frame rate to the camera to collect face images, and cyclically executing steps S2 to S5 to achieve adaptive adjustment of resolution and frame rate.

[0008] CN118506412A discloses a method for face detection in low-light scenes based on unsupervised domain adaptation. It includes: step 1: using the WIDERFACE dataset as the source domain and the low-light face dataset as the target domain; step 2: constructing a low-light scene face detection algorithm consisting of a feature extraction network, an unsupervised domain adaptation component based on back propagation, and a face detection branch network. The present invention proposes a low-light scene face detection algorithm based on unsupervised domain adaptation. The algorithm consists of three parts: a feature extraction network, an unsupervised domain adaptation component based on back propagation, and a face detection branch network, wherein RetinaFace is used as the face detection branch network, and an unsupervised domain adaptation component based on back propagation is used to shorten the feature distribution difference between the source domain and the target domain images in an unsupervised manner.

[0009] However, there are some problems with the above-mentioned face detection methods: the environment in which pedestrians are located is not complex enough, and the detection accuracy is only improved under low-power and low-light scene conditions; it does not involve face detection in dense crowds; and there is no clear solution for face detection in complex environments and dense crowds. Summary of the invention

[0010] The present invention provides a face detection method in a complex environment, which includes: face detection and face key point positioning. The face detection part uses an improved EfficientNetV2 network as the backbone network. According to the complex environment, a lightweight and compact face detection method called DCFD (Deep and Compact Face Detection) is proposed. The method includes: (1) Build the input layer of the DCFD model; (2) Build a DCFD model backbone feature extraction network, use the EfficientNetV2 network to replace the original backbone network module, and use the large-core attention mechanism to replace the traditional convolutional block, giving the network the ability to support long ends and achieve global receptive field, so as to facilitate the extraction of coarse-grained global features from the image; (3) Build the connection between the DCFD model backbone network and the feature module, introduce the ECA attention mechanism, enhance the model's ability to extract features, and make the model pay more attention to the detected target; (4) Build the feature fusion module of the DCFD model: use the improved NAS-FPN image pyramid to replace the original FPN structure; (5) The pedestrian dataset is sent to the improved RetinaFcae model with an input image size of 640×640, epoch=150, and trained using the Adam optimizer to obtain the optimal DCFD model for subsequent face detection.

[0011] The present application can achieve one or more beneficial effects: (1) Use the lightweight EfficientNetV2 network and introduce large kernel attention to better integrate local context information and consider long-distance dependencies, so that the model can focus more on facial features; (2) The loss function is optimized, and the focal loss function is used instead of the cross entropy loss function to balance the training of positive and negative samples and reduce the loss of positive samples that are easy to train; (3) An improved efficient channel attention mechanism (ECA) is introduced between the backbone network feature extraction network and the feature fusion module to improve the utilization of feature information in the feature fusion module; (4) The feature fusion module is reconstructed and the improved NAS-FPN image pyramid is used to replace the original FPN. This improvement improves the detection accuracy of small faces and enhances the accuracy and robustness of detection in dense crowd environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A flowchart of an algorithm implementation of the present invention; Figure 2A DCFD network architecture module diagram of an embodiment of the present invention; Figure 3 A module diagram of an improved MBConv structure of the present invention; Figure 4 A structural diagram of an LKA module according to an embodiment of the present invention; Figure 5 A module diagram of an improved ECA attention mechanism structure of the present invention; Figure 6 A network structure diagram of a NAS-FPN according to an embodiment of the present invention; Figure 7 The NAS-FPN architecture module diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0013] To address the challenges of face detection in complex environments, especially in dense crowds, a series of key improvements have been made to significantly improve performance.

[0014] Upgrade of EfficientNetV2 network structure: The improved lightweight EfficientNetV2 network is used as the backbone network of RetinaFace, and the large-core attention mechanism is used to extract facial features more effectively. Through algorithm optimization, not only the detection accuracy is guaranteed, but also the network overhead is reduced, making face detection in dense crowd scenes more outstanding. Reconstruction of spatial attention module: The spatial attention module is introduced into the ECA attention mechanism to better focus on the expression of spatial information, which not only improves the generalization ability of the model, but also more effectively captures the spatial relationship of faces in dense crowds.

[0015] Improved dense loss function: In order to achieve more fine-grained model adjustment, an improved dense loss function is used, which enables the model to process pixel-level predictions more directly, making the model more accurate in predicting the model structure and surroundings in the face detection task in dense crowds. Spatial fusion module reconstruction: Improved NAS-FPN to achieve adaptive gain for image targets. It not only reduces the number of model parameters and improves computational efficiency, but also obtains more accurate computational accuracy, providing a more robust solution for the actual application scenarios of subsequent face detection problems.

[0016] The present invention will be further described in detail below with reference to the accompanying drawings Figure 1 This is a flowchart of an algorithm implementation of the present invention. As shown, this implementation method includes the following steps: Step 1: Preprocess the dataset; Step 1-1: Systematically label, classify, and filter the WiderFace dataset to create a new WiderFace dataset subset. The new dataset contains approximately 29,000 images. Step 1-2: Select the LFW dataset as the second dataset for face detection.

[0017] Step 2: Build and train an improved RetinaFace model for face detection. Based on the RetinaFace model, this paper improves the face detection problem in dense crowds and obtains an improved RetinaFace model structure diagram named DCFD model, as shown in Figure 2 As shown, it includes input end, backbone network layer, feature fusion layer, prediction layer, decoding layer and NMS. The construction method of each layer is as follows; Step 2-1: Build the DCFD input layer and perform Mosaic data enhancement. By splicing four pictures into one picture through flipping, scaling, and color gamut changes within the area, the processing efficiency is greatly improved; the adaptive picture scaling method is: first, the size of the training picture is set to 640×640 (unit: pixel), and the image is scaled. Secondly, the size of the horizontal and vertical pixels after scaling is calculated, which are represented as x, and y, respectively, where x = x×min{x / 640, y / 640}, y, = y×min{x / 640, y / 640}, if x<640, then the corresponding x height is added with black edges of [(640-x)%64] / 2 height, and finally a 640×640 picture is formed. The same operation is performed in the y direction, where the min operation is represented by taking the smallest value in the curly brackets, and % is represented by the remainder operation; Step 2-2: Build an improved DCFD model backbone feature extraction network, use the improved EfficientNetV2 (its network structure is shown in Table 1) instead of the original RetinaFace network as the backbone network, and introduce large core attention. The convolutional features after pooling of the main module MBConv module of EfficientNetV2 are connected in series, and then input into the fully connected layer as a whole. Its main architecture includes 1×1 ordinary convolution for increasing the dimension, followed by k×k deep convolution, where the specific value of k can be 3×3 or 5×5, such as Figure 3 As shown in Figure 2, these structural layers use the Swish activation function and combine it with the SE (Squeeze-and-Excitation) module. They can effectively capture the underlying texture information and high-level semantic information of the image, thereby improving the detection capability. The LKA convolution block is used to replace the traditional convolution block, such as Figure 4The LKA module structure is shown in the figure. The goal of this innovative design is to give the network the ability to support the long end and achieve a global receptive field, so as to facilitate the full extraction of coarse-grained global features from the image. The introduction of the LKA module further compresses the output channels and repeated layers in EfficientNetV2, so that multi-dimensional and multi-scale fine-grained features can be extracted from global image features. The improved network structure of the EfficientNetV2 algorithm is shown in Table 1. The SE in Table 1 represents the self-attention module, where 0.25 is the coefficient of the first fully connected layer in the SE module, indicating one-fourth of the number of channels of the feature matrix input to the module. K represents the convolution kernel size, and the coefficient after MBConv is the expansion multiple; Table 1 Improved EfficientNetV2 algorithm network structure Step 2-3: Build the DCFD model feature fusion module. Introduce an improved efficient channel attention mechanism (ECA) between the backbone network feature extraction network and the feature fusion module, such as Figure 5 As shown in the figure, while retaining the one-dimensional convolution advantage of the ECA attention mechanism, a spatial attention module (SA module) is added to better capture the features of different spaces in the face image. This new feature is designed to enhance global information attention and improve network performance. Through one-dimensional convolution, the ability of feature extraction is improved without dimensionality reduction, thereby improving model performance with minimal parameters and computation. This improvement achieves the adaptability of face detection under complex backgrounds. The size of the convolution kernel k is related to the number of channels C. The larger the value of C, the larger the value of K. γ and b are nonlinear parameters, where γ is set to 2 and b is set to 1, as shown in formula (1): The feature fusion module is reconstructed, and the improved NAS-FPN image pyramid is used to replace the original feature pyramid (feature pyramid networks, referred to as FPN), such as Figure 6 As shown, Figure 6P3-P7 on the left side of the figure represent the input feature layer, and P3-P7 in the white background box represent the output feature layer. GP stands for the global pooling layer, which calculates the semantic information contained in the high layer as the global feature map. RCB stands for ReLU-Conv-BatchNorm, which means that it has been processed by ReLU activation function, batch operation and batch normalization. Feature pyramid network (FPN) is used to process the changes of objects at different scales. The original FPN network structure can extract diversified feature maps from different dimensions of different networks. However, if 1×1 convolution is directly performed on each extracted feature map to reduce the number of convolution kernels, the calculation task will be too heavy. When the network depth increases, the resolution of FPN will decrease, which will affect the key point detection of small target faces. The DCFD algorithm uses the EfficientNetV2 network as the backbone network. In order to improve the network accuracy without increasing the number of network parameters and calculations, the NAS-FPN structure is used to regroup and fuse the feature maps. By globally pooling the high-scale features, multiplying them with the low-scale features through 1×1 convolution and then adding them, the final output is output. In this way, the high-level feature map guides the calculation of the low-level feature map in the form of weights through global pooling and 1×1 convolution, achieving a tighter feature fusion. The improved network structure is shown in the figure below. Figure 7 This improvement aims to optimize the network's feature extraction and fusion mechanism, thereby improving the performance of the DCFD algorithm in target detection tasks; Step 2-4: Perform three types of predictions, including classification prediction, face regression box, and face key point location. The focus loss function is used instead of the cross entropy loss function to balance the training of positive and negative samples and reduce the loss of positive samples that are easy to train. The focus loss function is introduced. Its uniqueness lies in its ability to adaptively focus on difficult samples, that is, it gives higher weights to samples that are easily misclassified, while reducing the weights for samples that are relatively easy to classify. This mechanism helps the model focus more on key areas, thereby improving the performance of face detection. The cross entropy loss function used is shown in formula (2): yi is the label value of the ith sample, which can be 0 or 1. Pi is the model's predicted probability that the ith sample is a positive sample, and its value is [0, 1]. This loss function is used to predict the difference between the current model's prediction and the actual label. If the sample is a positive sample (yi = 1), the closer pi is to 1, the better the prediction of the positive sample can be, as shown in formula (3): CE=-log(pi) (3) When pi is closer to 0, the negative samples can be better predicted, as shown in formula (4): CE=-log(1-pi) (4) In order to solve the problems of sample imbalance and difficulty in classification, the focal loss function is introduced, which adjusts the loss by introducing a focal parameter. Its prediction accuracy Pt is defined as shown in formula (5). To solve the problem of sample imbalance, a weight factor α is introduced, and the value range of this factor is between [0, 1]. When yi = 1, the training sample i is regarded as a positive sample, and its weight is reset to α; when yi = 0, the training sample i is regarded as a negative sample, and the weight is set to (1-α). Based on the definition of this weight factor α, a weighted cross entropy formula is defined to adjust the cross entropy loss. , as shown in formula (6): CE(p t , α t )=-α t log(p t ) (6) The weight factor αt is introduced to adjust the weight ratio of positive and negative samples in the cross entropy to solve the problem of sample data imbalance. In order to solve the problem of distinguishing "hard" samples, an adjustment factor γ with a range of [0, +∞] is introduced into the cross entropy loss function. The introduced function can be expressed as formula (7): CE(p t ,γ)=-(1-p t ) γ log(p t ) (7) The mode of action of regulatory factor gamma can be used as follows: ① When the sample is misclassified and pt is small (indicating a sample that is difficult to classify), adjusting the parameter (1-Pt)γ close to 1 has almost no effect on the result of the loss function. This means that the model will maintain a large loss in this case, making people pay more attention to these difficult-to-classify samples; ② When the sample pt is 1 (a completely correctly classified sample), the adjustment parameter (1-Pt)γ is 0, which means that the loss of the completely correctly classified sample is 0, and the model will not pay attention to such samples; ③ When the sample pt is 0.9 (correct sample that is easy to classify) and γ = 2, the parameter (1-Pt)γ is adjusted to 0.01, and the sample loss function is reduced to 1 / 100 of the original, so that the model pays less attention to such easy-to-classify samples; ④When γ=0, the focal loss function becomes the cross entropy loss function, which no longer considers the difficulty of classifying samples, and all samples are given the same weight; Formula (8) of the focal loss function dynamically adjusts the loss function according to the classification of the sample and the adjustment factor γ to better process the sample; FL(p t)=-α(1-p t ) γ log(p t ) (8) In the face recognition task, the presence of a large number of prior boxes may cause most samples to be regarded as model background. This corresponds to the role of the γ parameter in the FocalLoss function. The purpose of introducing the γ parameter is to make the model pay more attention to samples that are difficult to recognize. This will directly cause the model to mistakenly identify the background as a face in a large range and give a lower confidence score. In the subsequent non-maximum suppression (NMS) stage, low-confidence faces are usually excluded; From the results, it can be seen that the introduction of the focal loss function does improve the model's ability to detect small faces, but in the post-processing stage, the number of faces is large due to too many prior boxes and overly sensitive detection. In some "difficult images", there may be thousands of faces (most of which can be eliminated), which will significantly slow down the operation of the NMS algorithm. Therefore, to solve the above problem, this paper adopts a strategy of using the cross entropy loss function in the first 145 cycles. The advantage of this strategy is that it reduces the number of its prior boxes, thereby reducing the burden on the NMS stage;

[0018] Step 2-5: Adjust the prediction results of the previous step and improve the position of the face frame and facial key points.

[0019] Step 2-6: Use EIOU as the NMS method to effectively remove highly overlapping prediction boxes.

[0020] Step 2-7: Send the WiderFace dataset and the LFW dataset to the DCFD model. The training parameters are shown in Table 2: Table 2 Training parameter configuration The ablation experiment of the WideFace dataset is shown in Table 3: Table 3 Ablation experiment of WiderFace dataset The ablation experiment of LFW dataset is shown in Table 4: Table 4 Ablation experiment on LFW dataset

Claims

1. A pedestrian face detection method based on improved RetinaFace, characterized in that: include: (1) Obtaining sample data, the sample data including different face detection data sets, and performing data preprocessing; (2) Build a lightweight and compact face detection model DCFD (Deep and Compact Face Detection) for face detection.

2. The face detection method based on improved ReinaFace according to claim 1, characterized in that: The step 1 comprises: (1) Collect pedestrian datasets, that is, for the improved RetinaFace model training dataset, collect pedestrian face datasets: systematically label, classify and filter the WiderFace dataset to create a new WiderFace data subset; (2) Collect the LFW dataset, which includes 13,233 face images covering 5,749 independent identities. It reflects the diversity of faces in different ages, lighting, and postures, simulating the complex conditions in the real world.

3. The pedestrian face detection method based on improved RetinaFace according to claim 1, characterized in that: Step 2 of building and training the DCFD model includes: (1) Build the input layer of the DCFD model; (2) Build a DCFD model backbone feature extraction network, use the EfficientNetV2 network to replace the original backbone network module, and use the large-core attention mechanism to replace the traditional convolution block, giving the network the ability to support long ends and achieve global receptive field, so as to facilitate the full extraction of coarse-grained global features from the image; (3) Build the connection between the DCFD model backbone network and the feature module, introduce the ECA attention mechanism, enhance the model's ability to extract features, and make the model pay more attention to the detected target; (4) Build the feature fusion module of the DCFD model: use the improved NAS-FPN image pyramid to replace the original FPN structure; (5) The pedestrian dataset is sent to the improved RetinaFcae model with an input image size of 640×640, epoch=150, and trained using the Adam optimizer to obtain the optimal DCFD model for subsequent face detection.

Images