Face detection method and system based on improved YOLOv5s

Abstract

The present application relates to the technical field of image processing, and more particularly to a face detection method and system based on improved YOLOv5, comprising collecting face image data and performing data enhancement on the face image; a FasterNet backbone feature extraction network is built, the training set image is input into the FasterNet backbone feature extraction network to generate initial features with different depth information; a feature enhancement module is used to fuse the features output by the concat of the part shallow feature map in the backbone and the deep feature map after up-sampling; a C3_NAM module is used to replace the C3 module of the BottleNeck layer to suppress the low-weight features of the face and thus improve the weight of the face key point information. The present application solves the problem of existing face detection methods that have many network parameters and low detection speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a face detection method and system based on improved YOLOv5. BACKGROUND

[0002] Face detection is the basis of face image analysis applications, covering applications such as face recognition and verification, face-based target tracking, and face expression analysis. With the application of deep learning, face detection has made important breakthroughs. Although face detection models have reached a very high accuracy in controlled environments, they still face many challenges in uncontrolled environments. The face density, face size variation range, face being partially occluded, and light variation in uncontrolled environments such as shopping malls, airports, and stations have a great impact on detection accuracy, and existing detection methods still have room for improvement.

[0003] To solve the above problems, many researchers have proposed many special face detection methods in terms of model structure, which require larger-scale convolutional neural networks. The increase in the number of neural network layers naturally loads more parameters and a large amount of matrix operations, which makes the entire algorithm and system become more and more complex, and the parameter amount also becomes larger and larger. It is difficult to ensure that the detection speed can match the use in real scenes while improving the face detection accuracy. SUMMARY

[0004] In view of the deficiencies of the existing method, the present application solves the problem of too many network parameters and low detection speed of the existing method.

[0005] The technical scheme adopted by the present application is: a face detection method based on improved YOLOv5 includes the following steps:

[0006] Step one, collect face image data and perform data enhancement on the face image;

[0007] Further, the data enhancement includes translation, rotation, scaling, flipping, random cropping, adjusting the hue, exposure, and saturation of the image.

[0008] Step two, build a FasterNet backbone feature extraction network, input the training set image into the FasterNet backbone feature extraction network to generate initial features with different depth information;

[0009] Further, the FasterNet backbone feature extraction network includes a CBS-FasterNet module composed of a CBS and a FasterNet Block, four CBS-FasterNet modules are cascaded, and the fourth CBS-FasterNet module is connected with an SPPF and a CBS module to output up-sampling.

[0010] Step three, using the feature enhancement module to fuse the part of shallow feature maps in the backbone with the deep feature maps after up-sampling to output the fused features through concat;

[0011] Further, step three specifically includes:

[0012] Step 31, in the 11th layer, the feature maps obtained in the P2 layer are first subjected to a CBS convolution with a size of 1x1 and a step of 1 to increase the channel dimension; then in the 12th layer, a CBS convolution with a size of 3x3 and a step of 2 is used to compress the feature maps; and the three feature layers are concatenated with the features extracted from the layer where the feature P3 is located and the features extracted by up-sampling in the 10th layer to obtain the feature maps Q1;

[0013] Step 32, in the 17th layer, the feature maps obtained in the P1 layer are first subjected to a CBS convolution with a size of 1x1 and a step of 1 to increase the channel dimension; then in the 18th layer, a CBS convolution with a size of 3x3 and a step of 2 is used to compress the feature maps; and the three feature layers are concatenated with the features extracted from the layer where the feature P2 is located and the features extracted by up-sampling in the 16th layer to obtain the feature maps Q2.

[0014] Step four, using a C3_NAM module to replace the C3 module of the BottleNeck layer to suppress the low-weight features of the face and thus improve the weight of the face key point information;

[0015] Further, step four specifically includes:

[0016] The three C3_NAM modules are used to allocate weights to the output features Q2, Q3 and Q4 according to the BN scale factor to generate BQ2, BQ3 and BQ4 with weight feature information;

[0017] Further, the formula of the BN scale factor is:

[0018]

[0019] wherein, μ B is the mean, σ B is the standard deviation, γ and β are trainable affine transformation parameters, ∈ is error compensation, and B in is the input feature.

[0020] Further, the face detection system based on the improved YOLOv5s includes a memory for storing instructions executable by a processor, and the processor is configured to execute the instructions to implement the face detection method based on the improved YOLOv5s.

[0021] Further, a computer readable medium storing computer program code, the computer program code, when executed by a processor, implements the face detection method based on the improved YOLOv5.

[0022] Advantages of the present application:

[0023] 1. The FasterNet feature extraction network is built to realize the lightweight of the model, and the problem of large model parameter quantity is solved.

[0024] 2. The shallow features of the main part and the deep features of the Neck part are fused to realize local information cross-channel interaction and improve the face detection performance, and the face detection still has high accuracy under complex conditions.

[0025] 3. The normalization attention mechanism is used to suppress less significant feature information, improve the weight of face detail feature information, and alleviate the problem of excessive decline of face detection accuracy caused by the reduction of model parameter quantity.

[0026] 4. The method can be applied to face prediction in various complex environments, including dense, occlusion, weak light and the like. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is the improved YOLOv5 network structure diagram of the present application;

[0028] Figure 2 is the FasterNet Block structure diagram of the present application;

[0029] Figure 3 is the NAM attention mechanism and feature weight fusion schematic diagram of the present application;

[0030] Figure 4 is the C3_NAM structure diagram of the present application;

[0031] Figure 5 is the overall experimental effect diagram of the present application;

[0032] Figure 6 is the effect diagram of the present application in the face dense condition;

[0033] Figure 7 is the effect diagram of the present application in the partial face occlusion condition;

[0034] Figure 8 is the effect diagram of the present application in the weak light condition. DETAILED DESCRIPTION

[0035] The present application will be further described below in conjunction with the drawings and examples, which are simplified schematic diagrams and only schematically show the basic structure of the present application, and therefore only show the structures related to the present application.

[0036] As Figure 1 shown, the face detection method and system based on improved YOLOv5s includes the following steps:

[0037] Step one, collect face image data, and perform data enhancement on the face image, and divide it into training set and validation set according to the proportion;

[0038] Data enhancement includes translation, rotation, scaling, flipping, random cropping, adjusting image hue, exposure, and saturation, etc. Get 16106 real scene face images, expand sample data, of which 12880 images are used as training set, and 3226 images are used as validation set. The face in each image is labeled.

[0039] Step two, build FasterNet backbone feature extraction network, input the training set image into FasterNet backbone feature extraction network to generate initial features P1, P2, P3, P4 with different depth information;

[0040] The FasterNet network is used to build the backbone network of YOLOv5s. The FasterNet backbone feature extraction network includes: CBS-FasterNet module composed of CBS and FasterNet Block, four CBS-FasterNet modules are cascaded, and the fourth CBS-FasterNet module is connected with SPPF and CBS module to output up-sampling.

[0041] As Figure 2 The FasterNet block uses partial convolution (PConv) as the convolution operator of the feature extraction network. PConv applies regular convolution Conv on a part of the input channel for spatial feature extraction, and the remaining channels remain unchanged. Two regular convolution (Conv 1x1) layers are added to the partial convolution PConv layer, and the batch normalization layer (BN) and the activation layer (ReLU) are placed in the middle position between the two regular convolution Conv layers, which are used for spatial down-sampling and channel number expansion. Four CBS-FasterNet modules are used to obtain four initial features with different scales: 40x64x64 (P1), 80x32x32 (P2), 160x16x16 (P3), and 320x8x8 (P4).

[0042] Step three, use the feature enhancement module to concatenate the shallow features P1, P2, P3, P4 in the backbone with the deep feature maps after up-sampling to output features Q1, Q2, Q3, Q4 through concat fusion operation;

[0043] The feature enhancement module is as Figure 1As shown in the middle dashed box, two CBS convolution modules of different sizes and steps are used to fuse the shallow and deep feature maps through a feature enhancement module.

[0044] Specifically, the method comprises the following steps:

[0045] Step 31, after the shallow feature P2 layer, the feature map is down-sampled, and the feature map obtained at the 11th layer of the feature P2 layer is first subjected to CBS convolution with a size of 1x1 and a step of 1 to increase the channel dimension, and the channel number is 128; then, at the 12th layer, CBS convolution with a size of 3x3 and a step of 2 is used to compress the feature map, so that the feature map size is compressed to 32x32 which can be fused; and the three feature layers are fused (concat) with the features extracted from the feature P3 layer and the features extracted by up-sampling at the 10th layer, to obtain a feature map Q1 with rich feature information;

[0046] Step 32, the obtained feature map Q1 is subjected to residual, standardization and up-sampling operations to obtain a new feature layer, and the feature layer is the 16th layer; after the shallow feature P1 layer, the feature map is down-sampled, and the feature map obtained at the 17th layer of the feature P1 layer is first subjected to CBS convolution with a size of 1x1 and a step of 1 to increase the channel dimension, and the channel number is 64; then, at the 18th layer, CBS convolution with a size of 3x3 and a step of 2 is used to compress the feature map, so that the feature map size is compressed to 16x16 which can be fused; and the three feature layers are fused (concat) with the features extracted from the feature P2 layer and the features extracted by up-sampling at the 16th layer, to obtain a feature map Q2 with rich feature information.

[0047] Step four, using the C3_NAM module to distribute weights according to the BN scale factor for the output features Q2, Q3 and Q4, to generate BQ2, BQ3 and BQ4 with weight feature information, and to learn features according to the weight feature information;

[0048] As shown in Figure 3 , 4 , the C3_NAM is a channel and spatial attention submodule that fuses weights according to the BN scale factor, and uses the standard deviation σ B to represent the importance of the weight; the 20th, 23rd and 26th layers at the tail of the BottleNeck are improved, the C3 module is replaced by the C3_NAM module, the low-weight features that are not very significant in face detection are suppressed, and the weight of the key point information of the face part is improved.

[0049] The features Q2, Q3 and Q4 obtained from the previous layer are fused according to the BN scale factor to generate feature maps BQ2, BQ3 and BQ4 with weight feature information, and the feature maps of three scales BQ2-BQ4 are reduced in dimension by 1x1 convolution to extract high-level features, and the high-level features are used to predict the face position to realize face detection under complex conditions, and the scale sizes of 32x32x128, 16x16x256 and 8x8x512 are output respectively; wherein the BN scale factor is as follows:

[0050]

[0051] wherein μ B is the mean, σ B is the standard deviation, γ and β are trainable affine transformation parameters, ∈ is error compensation, and B in is the input feature.

[0052] Face detection under complex conditions is performed by using the learned feature information.

[0053] In the model training stage, the labeled samples are divided into a training set and a validation set according to a ratio of 4:1, the training set is input into the network to start learning, and the mosaic data augmentation is used in the training process to learn the feature representation of the blurred target and improve the adaptability and robustness of the model. After the training set image size is resized to 640x640, the network starts training, and the momentum used in the training is 0.937, the SGD random gradient descent is used for optimization, the initial learning rate is 0.01, the model weight decay coefficient is 0.0005, the positioning loss box_loss used is CI0U loss, and data jitter, horizontal flip, rotation angle, adjustment of image hue, exposure and saturation are also used to generate more training data to improve the detection ability and generalization ability of the model. The training and test configuration is Intel(R)Core(TM)i9-9900K CPU@3.60GHZ, 128GB RAM, NVIDIA GeForce RTX 2080Ti GPU, the batch_size in the training stage is set to 16, and the epoch is set to 300 rounds. When the network is trained to about 270 rounds, it basically converges, then the samples in the validation set are input for recognition verification, and part of the predicted results are as shown in Figure 5 Figure 5 ​The displayed data in the middle consists of two parts of the identified type and the confidence degree considered as the type, wherein the face detection confidence degree of the extremely small size is 0.6-0.7, the detection confidence degree of the occluded part and the weak light part is distributed in the interval of 0.7-0.87; the detection confidence degree of the normal face size is above 0.9. Overall, if there are a large number of faces in the same image, and the size span of the faces is large, the smaller the face size is, the smaller the confidence degree of the detected face is, because in the same image, the details of the smaller size device are relatively not clear enough, resulting in the loss of part of the feature information during identification.

[0054] As Figures 6-8 The training result shows that the average accuracy rate reaches 72%, the model parameter quantity is reduced from 7022326 to 3196282, reduced by 54.4%, the floating point operation number FLOPs is reduced from 15.9 GFLOPs to 7.3 GFLOPs, reduced by 54.0%, the delay is significantly reduced, and the above results fully verify the effectiveness of the method.

[0055] Based on the above ideal embodiments according to the present application, through the above description, relevant personnel can make various changes and modifications without deviating from the technical idea of the present application. The technical scope of the present application is not limited to the contents in the specification, and must be determined according to the scope of claims.

Claims

1. A face detection method based on improved YOLOv5s, characterized in that, The method comprises the following steps: Step one, collecting face image data and performing data enhancement on the face image; Step two, building a FasterNet backbone feature extraction network, inputting the training set image into the FasterNet backbone feature extraction network to generate initial features with different depth information; The FasterNet backbone feature extraction network comprises a CBS-FasterNet module composed of a CBS and a FasterNet Block, four CBS-FasterNet modules are cascaded, and the fourth CBS-FasterNet module is connected with an SPPF and a CBS module to output up-sampling; Step three, using a feature enhancement module to fuse the features output by concatenating the part of shallow feature maps in the backbone and the deep feature maps after up-sampling; Step three specifically comprises: Step 31, obtain the feature map in the 11th layer The feature map obtained in the 11th layer is first subjected to a CBS convolution with a size of 1x1 and a step of 1 to raise the channel dimension; then, in the 12th layer, a CBS convolution with a size of 3x3 and a step of 2 is used to compress the feature map; and the feature map obtained in the 12th layer is concatenated with the feature map obtained by up-sampling the feature extracted from the 10th layer to obtain a feature map ​​ Step 32, obtain the feature map in the 17th layer The feature map obtained in the 17th layer is first subjected to a CBS convolution with a size of 1x1 and a step of 1 to increase the channel dimension; then, in the 18th layer, a CBS convolution with a size of 3x3 and a step of 2 is used to compress the feature map; and the feature map obtained in the 18th layer is concatenated with the feature map obtained in the 16th layer after being up-sampled to obtain a feature map ​​ Step four, using a C3_NAM module to replace the C3 module of the BottleNeck layer to suppress the low-weight features of the face and improve the weight of the key point information of the face.

2. The face detection method based on improved YOLOv5 according to claim 1, wherein, The data enhancement includes translation, rotation, scaling, flipping, random cropping, adjusting image hue, exposure, and saturation.

3. The face detection method based on improved YOLOv5 according to claim 1, wherein, Utilizing three C3_NAM modules to output features , , Weight distribution is performed according to the BN scale factor, and , , with weight feature information is generated.

4. The face detection method based on improved YOLOv5 according to claim 1, wherein, The formula of the BN scale factor is: wherein, is the mean, is the standard deviation, and are trainable affine transformation parameters, is the error compensation, is the input feature.

5. The face detection system based on improved YOLOv5s, characterized in that, It comprises: a memory for storing instructions executable by the processor; a processor for executing the instructions to implement the face detection method based on the improved YOLOv5s according to any one of claims 1-4.

6. A computer readable medium having stored thereon a computer program code, characterized in that, The computer program code implements the face detection method based on the improved YOLOv5s according to any one of claims 1-4 when executed by the processor.

Images