A face detection method based on multi-scale feature fusion

Abstract

The present application belongs to the field of target detection of computer vision, and discloses a face detection method based on multi-scale feature fusion, which comprises the following steps: 1, inputting a visible light image into a face target detection network to output a plurality of multi-dimensional matrix feature maps; 2, inputting the plurality of multi-dimensional matrix feature maps into a selective multi-scale feature pyramid module to perform multi-scale feature fusion; 3, inputting the image after multi-scale feature fusion into an asymmetric convolution perception module to perform feature refining and enlarged receptive field, and adopting an asymmetric convolution module to splice the multi-scale feature layers; and 4, outputting a plurality of feature maps for predicting targets of different sizes. The present application improves the feature pyramid of selective multi-scale fusion for feature fusion, uses a new module for the backbone network of target detection, can increase the detection performance at a very small cost, and improves the face detection performance.

Description

Technical Field

[0001] This invention belongs to the field of target detection in computer vision, specifically relating to a face detection method based on multi-scale feature fusion. Background Technology

[0002] Face detection is a biometric recognition technology that aims to locate all faces in an image and mark them with bounding boxes. With the rapid development of deep learning, image processing, and pattern recognition technologies, many face detection methods based on deep neural networks have emerged in recent years. These methods can handle face detection problems involving multiple categories, multiple scales, and densely packed small targets in unconstrained environments, significantly improving the performance and speed of face detection. Currently, face detection is widely used in many important industries and fields such as public security, security, customs, finance, military, airports, border crossings, and security systems.

[0003] Mainstream face detection algorithms include geometric feature-based methods, template-based methods, and deep learning-based methods. Among them, deep learning-based face detection algorithms are currently the most popular and fastest. This method detects faces by training a convolutional neural network. Its advantage is that it can automatically extract discriminative features from the original image, thereby achieving fast and accurate face detection.

[0004] Multi-scale fusion is primarily used in face detection to address the problem of detecting faces at different scales, thereby improving accuracy and speed. Currently, the mainstream multi-scale fusion methods include: using input images at multiple scales and then fusing the results, such as MTCNN; using pyramid structures to fuse feature maps at different levels, such as FPN, SPP, PSP, and ASPP; and using attention mechanisms or dilated convolutions to enhance the receptive field and feature representation, such as FAN, ConvMixer, and TridentNet.

[0005] The above methods have achieved good results in the field of face detection. However, due to limitations in computing power and scene requirements, the multi-scale fusion methods do not consider the importance of features at different scales to the detection results, resulting in poor fusion effects, little performance improvement, and a decrease in detection speed. Therefore, it is particularly important to find a way to fully fuse information with limited computing power for detecting faces in different scenes. Summary of the Invention

[0006] To address the aforementioned technical issues, this invention proposes a multi-scale fusion face detection method. This method performs multi-scale fusion using Selective-Scale-FPN and expands the receptive field through the AC-Context Module. Simultaneously, it concatenates multi-scale information to further improve detection performance. The method is validated on the WiderFace dataset, and experiments demonstrate that the algorithm of this invention has better detection performance than existing methods.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0008] This invention is a face detection method based on multi-scale feature fusion, which includes the following steps:

[0009] Step 1: Input the visible light image into the face object detection network and output multiple multi-dimensional matrix feature maps. The face object detection network is based on ResNet.

[0010] Step 2: Input multiple multi-dimensional matrix feature maps into the Selective-Scale-FPN module for multi-scale feature fusion. Based on the feature pyramid, a multi-core attention mechanism is used for weighted fusion.

[0011] Step 3: Input the asymmetric convolutional perception module AC-Context Module to refine features and expand the receptive field. Use the asymmetric convolution module AC Block to stitch together multi-scale feature layers.

[0012] Step 4: Output multiple feature maps to predict targets of different sizes.

[0013] A further improvement of the present invention is that the mathematical formula of the Selective-Scale-FPN is as follows:

[0014] P i =W i ×F i +W i+1 ×F i+1

[0015] Where P i W is the output feature layer of the i-th layer. i With W i+1 These are the multi-core attention weights of the i-th layer and the (i+1)-th layer, respectively, obtained from the multi-core attention mechanism.

[0016] A further improvement of the present invention is that the equivalent formula of the multi-core attention mechanism is as follows:

[0017] s = FC(GAP(F i+F i+1 ))

[0018] W i W i+1 =softmax(FC(s),FC(s))

[0019] Where F i With F i+1 These are the i-th and (i+1)-th input feature layers, respectively. GAP is global average pooling, FC is a fully connected layer, and softmax is the softmax function.

[0020] A further improvement of the present invention is that the formula for the asymmetric convolution module AC Block in step 3 is as follows:

[0021] O = Conv 3×3 (P)+Conv 3×1 (P)+Conv 1×3 (P)

[0022] Where Conv 3×3 Conv 3×1 Conv 1×3 These represent convolution operations with kernel sizes of 3×3, 3×1, and 1×3, respectively.

[0023] A further improvement of the present invention is that: the asymmetric convolutional sensing module AC-Context Module described in step 3 consists of five asymmetric convolutional modules AC Blocks. Assuming that the number of channels of P is C, its mathematical formula is as follows:

[0024]

[0025] AC_Block(A|in_c=X,out_c=Y) means that AC_Block takes A as input, has X as its number of channels, and outputs Y as its number of channels after the operation of AC_Block; Concat() refers to concatenating the number of channels in the feature layer.

[0026] The beneficial effects of this invention are:

[0027] This invention proposes a Selective-Scale-FPN based on FPN for feature fusion. Selective-Scale-FPN achieves a feature pyramid structure that fully integrates features and considers multi-scale fusion by adding a multi-core attention mechanism, thereby improving face detection performance.

[0028] This invention proposes a novel AC-Context Module for the backbone network of object detection, which can increase detection performance at a very low cost. The AC-Context Module can consider multi-scale information and has a stronger feature extraction capability, which is more helpful for face detection.

[0029] This invention utilizes the convolutional operation technology of computers and the high-speed computing performance of GPUs to achieve fast and high-precision face detection; it uses ResNet network for feature extraction; it uses Selective-Scale-FPN for feature fusion and AC-Context Module for feature refinement, thus achieving full fusion of information and rich semantic information, which greatly improves the performance of face detection. Attached Figure Description

[0030] Figure 1 This is a schematic diagram illustrating the principle of the face detection network method based on multi-scale fusion of the present invention.

[0031] Figure 2 This is a schematic diagram of the multi-core attention mechanism of the present invention.

[0032] Figure 3 This is a schematic diagram of the AC-Context Module of the present invention.

[0033] Figure 4 This is a schematic diagram of the AC Block of the present invention. Detailed Implementation

[0034] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.

[0035] like Figure 1 As shown, this invention is a face detection method based on multi-scale feature fusion, trained in an end-to-end manner, and specifically includes the following steps:

[0036] Step 1: Acquire and preprocess training image data. The COCO dataset was chosen as the training image data. This dataset is designed for human visual cognition and understanding and contains many scene categories. Each category contains 400 images. The WiderFace dataset was used for training, containing 32,203 images categorized into 61 classes based on the event scene type. The training, validation, and test sets were then divided into groups of 40%, 10%, and 50% respectively. Data augmentation used color dithering with a probability of 0.5: brightness, contrast, saturation, and hue.

[0037] Step 2: Input the visible light image into the face detection network, and output multiple multi-dimensional matrix feature maps. The face detection network is based on ResNet.

[0038] The backbone extraction network in this embodiment is ResNet50. The ResNet50 network consists of five convolutional layers, each followed by a residual module to learn the residual mapping between the input and output. The ResNet50 network structure has a total of 50 layers, including one input layer and 49 convolutional layers. The outputs of the last four stages will serve as the input to Selective-Kernel-FPN.

[0039] Step 3: Input the multiple multi-dimensional matrix feature maps described in Step 1 into the Selective-Scale-FPN module for multi-scale feature fusion.

[0040] The Selective-Scale-FPN module, based on the Feature Pyramid (FPN), calculates the weights for the fusion of the higher-level feature map (upsampled) and the lower-level feature map (after undergoing a multi-core attention mechanism) when fusing two feature layers.

[0041] The mathematical formula for the Seleetive-Seale-FPN module of the multi-scale feature pyramid is as follows:

[0042] P i =W i ×F i +W i+1 ×F i+1

[0043] Where P i W is the output feature layer of the i-th layer. i With W i+1 These are the multi-core attention weights of the i-th layer and the (i+1)-th layer, respectively, obtained from the multi-core attention mechanism.

[0044] Multi-core attention mechanisms such as Figure 2 As shown, it utilizes global information from the feature layer to assign different weights to feature layer information at different scales. The mathematical expression of the multi-core attention mechanism is as follows:

[0045] s = FC(GAP(F i +F i+1 ))

[0046] W i W i+1 =softmax(FC(s),FC(s))

[0047] Where r i With r i+1 These are the i-th and (i+1)-th input feature layers, respectively. GAP stands for Global Average Pooling, FC stands for Fully Connected Layer, and softmax stands for Softmax Function.

[0048] Step 4: Input the image after multi-scale feature fusion in Step 2 into the asymmetric convolutional perception module AC-ContextModule for feature refinement and receptive field expansion. The asymmetric convolution module AC Block is used to stitch together the multi-scale feature layers.

[0049] Step 5, AC-Context Module as follows Figure 3 As shown, the AC-Context Module consists of five AC Blocks, as follows: Figure 4 As shown, each AC block consists of parallel convolutional kernels of 3×3, 3×1, and 1×3, and its formula is as follows:

[0050] O = Conv 3×3 (P)+Conv 3×1 (P)+Conv 1×3 (P)

[0051] Where Conv 3×3 Conv 3×1 Conv 1×3 These represent convolution operations with kernel sizes of 3×3, 3×1, and 1×3, respectively. After the AC-Context Module extracts features from the input feature layer, assuming the number of channels in the input feature layer is C, the specific mathematical formula is as follows:

[0052]

[0053] Where AC_Block(A|in_c=X,out_c=Y) means that AC_Block takes A as input, has X channels, and outputs Y channels after AC_Block's operation; Concat() concatenates the feature layers along the channel dimension. The change in the number of channels after five AC_Block operations is as follows: The final output is the concatenation of the feature layers of the 1st, 3rd, and 5th AC_Block convolutions, with the number of channels in the output feature layer remaining at C.

[0054] Step 5: Output multiple feature maps to predict targets of different sizes.

[0055] During training, the overall loss function L of the algorithm is defined as follows:

[0056]

[0057] Where L cls and L box These are the classification loss and localization loss, respectively, with λ being the balancing parameter. The class labels are for the true positive samples. Classification loss Li cls Using the FocalLoss loss function, defined as

[0058]

[0059] Where p i To improve the network's classification prediction results, α is added as a balancing factor to address the uneven distribution of positive and negative samples, while γ alleviates the issue of easy and difficult samples. Localization loss L... box Using the L1 loss function, defined as

[0060]

[0061] Where t i The network predicted bounding box results. The label box is set to |·|, which indicates the calculation of the absolute value.

[0062] Training and validation were performed on the WiderFace dataset. The SGD optimizer was used, with an initial learning rate of 1e-3. The learning rate decreased by a factor of 10 at iterations of 50,000 and 60,000, respectively. Momentum was set to 0.9, weight decay to 5e-4, and epochs to 150. Experimental results show that adding two multi-scale fusion methods improved mAP by 0.06%, 0.34%, and 3.06% on the Easy, Medium, and Hard validation datasets, respectively, with a particularly significant performance improvement on the Hard validation dataset.

[0063] This invention proposes a face detection network based on multi-scale fusion to quickly detect faces. It proposes to use ResNet as the basic architecture, perform effective multi-scale fusion through Selective-Scale-FPN, and refine features through the AC-Context Module to fully extract contextual semantics.

[0064] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A face detection method based on multi-scale feature fusion, trained in an end-to-end manner, characterized in that: The face detection method comprises the following steps: Step 1, input the visible light image into a face target detection network to output a plurality of multi-dimensional matrix feature maps, wherein the face target detection network is based on ResNet; Step 2, input the plurality of multi-dimensional matrix feature maps in step 1 into a selective multi-scale feature pyramid (Selective-Scale-FPN) module for multi-scale feature fusion; Step 3, input the image after the multi-scale feature fusion in step 2 into an asymmetric convolution perception module (AC-Context Module) for feature refinement and enlarged receptive field, and adopt an asymmetric convolution module (AC Block) to splice the multi-scale feature layers; Step 4, output a plurality of feature maps for predicting targets of different sizes, wherein: In the step 2, the selective multi-scale feature pyramid (Selective-Scale-FPN) module is based on a feature pyramid (FPN) and adopts a multi-core attention mechanism for weighted fusion; In step 3, the expression of the asymmetric convolution module (AC Block) is as follows: O = Conv 3×3 (P) + Conv 3×1 (P) + Conv 1×3 (P) where Conv 3×3 , Conv 3×1 , Conv 1×3 represent convolution operations with kernel size of 3x3, 3x1, 1x3, respectively; The asymmetric convolution perception module (AC-Context Module) is composed of five AC Blocks, and the mathematical formula is as follows, assuming that the channel number of P is C: Wherein AC_Block(A|in_c=X,out_c=Y) means that the input of AC_Block is A, the channel number is X, and the output channel number is Y after the operation of AC_Block, and Concat() means splicing the feature layers in the channel number.

2. The method according to claim 1, characterized in that: In the step 2, the mathematical formula of the selective multi-scale feature pyramid (Selective-Scale-FPN) module is as follows: P i = W i × F i + W i+1 × F i+1 where P i is the output feature layer of the i-th layer, W i is the multi-kernel attention weight of the i-th layer, and W i+1 is the multi-kernel attention weight of the i+1-th layer, obtained by the multi-kernel attention mechanism.

3. The method according to claim 2, characterized in that: The mathematical expression of the multi-core attention mechanism is as follows: s = FC(GAP(F i + F i+1 )) W i ,W i+1 = softmax(FC(s), FC(s)) wherein F i with F i+1 are the i-th and i+1-th input feature layers, respectively, GAP is a Global Average Pooling, FC is a fully connected layer, and softmax is a softmax function.

Images