A real-time small face detection method based on improved YOLOv5

Abstract

The present invention seeks to protect a real-time small face detection method based on improved YOLOv5, which belongs to the field of computer vision. The method comprises the following steps: Step 1. YOLOv5 is used as the basic framework for small face detection. SKAttention is added to the YOLOv5 small face detection model. The weight of the channel is obtained by dynamically calculating each convolution kernel, and the results of each convolution kernel are dynamically fused, which allows the neural network to pay more attention to small faces and improve the accuracy of small face detection in real scenes. Step 2. A small face feature enhancement module is designed to upsample the feature map so that the feature map continues to expand, and the shallow feature map is fused with the deep feature map. Step 3. In order to meet the real-time requirements, the YOLOv5 model is adopted, and the backbone feature extraction network C3 is replaced with a lighter ShuffleNetV2 network to achieve lightweight network model and balance speed and accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular to a method for detecting small human faces. Background Art

[0002] Face detection is the foundation of facial image analysis applications, including face recognition and verification, face-based object tracking, facial expression analysis, and facial attribute recognition (i.e., gender / age recognition). With the development of deep learning, face detection has achieved remarkable success. Although face detection models in controlled environments have achieved high accuracy, face detection in unconstrained environments still faces many challenges. In unconstrained environments, faces often vary in scale and are affected by other factors such as occlusion, lighting, and facial expression. Scale variation has a significant impact on detection accuracy.

[0003] To address the problem of face detection in real-world scenarios, many researchers have proposed numerous specialized face detection methods from various perspectives, including model structure, data augmentation, and label assignment. This has led to increasingly complex algorithms and systems. Representative methods include Multitask Cascade CNN (Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks) and RetinaFace (RetinaFace: Single-stage Dense Face Localization in the Wild). These methods, while core to the connection between face detection and facial landmark detection, utilize facial landmarks as supervisory information, increasing network complexity and computational effort, hindering their practical application in real-world scenarios.

[0004] The representative method of YOLOv5 application in the field of small target detection is 'The Improved YOLOV5 Algorithm and Its Application in Small Target Detection'. It mainly clusters the dimensions and aspect ratios of target images. The improved YOLOv5 can better detect small targets, thereby significantly improving the recall rate and average precision of small targets.

[0005] Face detection belongs to the field of target detection and is regarded as a general target detection task. In order to solve the application problem of small face detection in real scenes, this paper proposes a real-time small face detection method based on improved YOLOv5.

[0006] CN114022878A, the present invention belongs to the technical field of computer vision and pattern recognition, deep learning and string tomato maturity detection. In view of the relatively low detection accuracy of tomatoes, slow inference speed, poor adaptability and accuracy to complex scenes of traditional image processing algorithms, and the lack of deep learning in string tomato detection methods and image data set processing, a real-time string tomato detection method based on improved YOLOv5 is disclosed. The method is based on the YOLOv5 concept and introduces an attention mechanism to mine deeper and better image feature information. The method can meet practical requirements in terms of detection accuracy and inference speed, and realizes application in artificial intelligence embedded devices.

[0007] Disadvantages: CN114022878A has insufficient data, with only 2,000 training set images; GioU is used as the loss function

[0008] Difference: The attention mechanism used is different; the loss function used is different.

[0009] (1) 1. When the detection box and the ground-truth box are included, GIOU degenerates into IOU. 2. When the two boxes intersect, the network converges slowly. The loss function in yolov5 is changed to Eiou, which calculates the difference between the width and height instead of the aspect ratio.

[0010] (2) The face dataset WIDER Face used in this paper is a public dataset, including 32,203 images with a total of 393,703 annotated faces. This dataset is more difficult and challenging. Many studies have studied it, including Poly-NL: Linear Complexity Non-local Layers with Polynomials; RetinaFace: Single-stage Dense Face Localization in the Wild; and CenterFace: Joint Face Detection and Alignment Using Face as Point. Summary of the Invention

[0011] The present invention aims to solve the above problems in the prior art. A real-time small face detection method based on improved YOLOv5 is proposed. The technical solution of the present invention is as follows:

[0012] A real-time small face detection method based on improved YOLOv5, comprising the following steps:

[0013] Step 1: Divide the face dataset into a training set and a test set. The training set is preprocessed as follows: Dataset cleaning: Check whether the images and label files in the dataset are complete. If not, delete the redundant images and label files. At the same time, delete the anchor boxes with too small aspect ratios. After deleting, check whether there are empty label files. If so, delete them as well. Obtain adaptive anchor boxes based on the Kmeans clustering algorithm.

[0014] Step 2: Based on the YOLOv5 small face detection model, the SK selective convolution kernel attention is introduced. The channel weight is obtained by dynamically calculating each convolution kernel, and the results of each convolution kernel are dynamically fused.

[0015] Step 3: Design a small face feature enhancement module. The small face feature enhancement module is as follows: after the 17th layer, continue to upsample the feature map to make the feature map continue to expand, and at the same time, fuse the feature map obtained at the 20th layer with the feature map of the 2nd layer (concat) to obtain a larger feature map for small target detection, upsample the feature map to make the feature map continue to expand, and fuse the shallow feature map with the deep feature map;

[0016] Step 4: Use the YOLOv5 model and replace the backbone feature extraction network C3 with the ShuffleNetV2 network;

[0017] Step 5: Apply the trained deep learning model for small face detection to small face detection in real scenes.

[0018] Furthermore, the step 1: dividing the face dataset into a training set and a test set, preprocessing the training set, and obtaining an adaptive anchor frame according to a clustering algorithm specifically includes the following steps:

[0019] A1. Perform data augmentation on the image data in the training set by performing operations such as translation, rotation, changing the aspect ratio of the image, and adjusting the saturation to increase the sample data;

[0020] A2. Perform cluster analysis on the real anchor frames in the training set through clustering algorithms to obtain accurate anchor frames as the predetermined values ​​of the small face detection network.

[0021] Furthermore, A2, performing cluster analysis on the real anchor frames in the training set through a clustering algorithm to obtain accurate anchor frames as predetermined values ​​for the small face detection network, specifically includes:

[0022] Yolov5 not only uses k-means clustering to generate anchors, but also uses the Genetic Algorithm to perform mutations on the results of k-means clustering.

[0023] 1. Read the width and height of each image in the training set and the width and height of all anchor boxes

[0024] 2. Scale the maximum width and height of each image proportionally to the specified size img_size

[0025] 3. Change the anchor box from relative coordinates to absolute coordinates (multiply by the scaled width and height)

[0026] 4. Filter anchor boxes and retain anchor boxes with width and height greater than or equal to two pixels

[0027] 5. Use k-means clustering to get n anchor boxes (dropping the k-means package involves a whitening operation)

[0028] 6. Use a genetic algorithm to randomly mutate the width and height of the anchor box. If the effect is better after mutation, assign the mutated result to the anchor box. If the effect is worse after mutation, skip it. The default mutation is 1000 times. 7. Return the final mutated anchor box.

[0029] Furthermore, step 2: based on the small face detection model of YOLOv5, SK attention is introduced, the weight of each convolution kernel is obtained by dynamically calculating the channel, and the results of each convolution kernel are dynamically fused, which specifically includes the following steps:

[0030] B1. For any given feature map X∈R H×W×c , H, W, c represent the width, height, and number of channels of the image respectively, R represents the feature map, after two transformations and and They are all composed of depthwise convolution, with normalization and activation functions running in sequence. The traditional 5*5 convolution is replaced by a dilated convolution with a size of 2, and the convolution kernel size is 3*3. Different branches use different convolution kernels to process the input X features, and obtain the feature maps after multi-branch processing.

[0031] B2. Control the information flow from multiple branches to carry information of different scales into the neurons of the next layer; first, fuse the results from multiple branches: Then U passes through E gp Global average pooling

[0032]

[0033] E gp ,i,j,uc(i,j) respectively represent the global spatial information, the height of the channel from 1 to H, the width of the channel from 1 to W, and the channel statistical information, and H*W represents the spatial dimension;

[0034] Furthermore, the SK attention is introduced to obtain the channel weight by dynamically calculating each convolution kernel

[0035]

[0036] in F scale( u c , s c ) refers to the scalar s c Channel-wise multiplication of the feature map.

[0037] Furthermore, the step 3: a small face feature enhancement module is designed to further expand the feature map and fuse the shallow feature map with the deep feature map, which specifically includes the following steps:

[0038] A small face feature enhancement module is designed. After the 17th layer, the feature map is upsampled to further expand the feature map. At the same time, the feature map obtained at the 20th layer is fused (concat) with the feature map at the 2nd layer to obtain a larger feature map for small target detection.

[0039] C1. Upsample the feature map to expand it; use the nearest neighbor interpolation method for upsampling;

[0040] C2. Fuse the shallow feature map with the deep feature map. The features extracted by the shallow network are closer to the input and contain more pixel information. Some fine-grained information is some color, texture, edge, and angular information of the image. However, due to fewer convolutions, its semantics is lower and there is more noise. The features extracted by the deep network are some information about the overall image. Some coarse-grained information contains more abstract information, namely semantic information. After the 17th layer, the feature map is continued to be upsampled so that the feature map continues to expand. At the same time, the feature map obtained by the 20th layer is fused (concat) with the feature map of the 2nd layer to obtain a larger feature map for small target detection.

[0041] Furthermore, in step 4: adopting the YOLOv5 model and replacing the backbone feature extraction network C3 with the ShuffleNetV2 network, the specific steps are as follows:

[0042] In addition to the ShuffleNetV2 network, the YOLOv5 model also includes CBL, Resunit, SPP, and Focus.

[0043] Furthermore, step 5 applies the deep learning model for small face detection to small face detection in real scenes, specifically including:

[0044] 1. Use the graphical tool 'Auto Py To Exe' to import the trained best.pt file into the weights folder and generate an exe file.

[0045] 2. Double-click to run the exe file, which supports image detection, video detection, and camera detection.

[0046] The advantages and beneficial effects of the present invention are as follows:

[0047] The present invention mainly addresses the currently popular face detection task based on deep convolutional neural networks, treating face detection as a separate computer vision problem. However, the accuracy and speed of small face detection cannot meet the requirements of real-time detection. Small face detection is treated as a general target detection task. From a data perspective, features such as posture, scale, occlusion, illumination, and blurring of the face also appear in other general target detection tasks. Unique facial attributes, such as expression and makeup, also correspond to shape and color changes in general target detection problems. YOLOv5 can be applied in the field of small target detection. A small face detector based on YOLOv5 is developed, incorporating the SE attention mechanism to improve mean average precision (mAP) performance. To meet the requirements of real-time detection, a small model is adopted, and the backbone network is modified to ShuffleNetV2 to increase the speed of small face detection. This method achieves good detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The present invention provides a preferred embodiment of a real-time small face detection method network framework based on YOLOv5.

[0049] Figure 2 This is the small face feature enhancement module of the present invention. DETAILED DESCRIPTION

[0050] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0051] The technical solution of the present invention to solve the above technical problems is:

[0052] The embodiment of the present invention is based on the YOLOv5 target detection framework as the basic framework, see https: / / github.com / ultralytics / yolov5 / tree / v6.0 The SK attention mechanism is added to allow the network to focus more on small faces, improving detection accuracy. The backbone network is replaced with ShuffleNetv2 to achieve a lightweight network model and balance speed and accuracy.

[0053] The present invention will be further described below with reference to the accompanying drawings:

[0054] As attached Figure 1 As shown in the figure, the design process of a real-time small face detection method network framework based on YOLOv5 includes the following steps:

[0055] Step 1: Divide the face dataset into a training set and a test set, preprocess the training set, and obtain adaptive anchor frames based on the clustering algorithm;

[0056] Step 2: Based on the YOLOv5 small face detection model, SK attention is introduced. By dynamically calculating the channel weight of each convolution kernel, the results of each convolution kernel are dynamically fused, which improves the detection effect by making the network pay more attention to the target to be detected.

[0057] Step 3: A small face feature enhancement module is designed to upsample the feature map to further expand it. At the same time, the shallow feature map is fused with the deep feature map to further improve the small face detection accuracy.

[0058] Step 4: To address the problem of increased computational complexity and decreased inference detection speed after adding the detection layer, in order to meet real-time requirements, the YOLOv5 model was adopted and the backbone feature extraction network C3 was replaced with the lighter ShuffleNetV2 network to achieve a lightweight network model and balance speed and accuracy.

[0059] Step 5: Apply the trained deep learning model for small face detection to small face detection in real scenes.

[0060] Preferably, the step 1: dividing the face dataset into a training set and a test set, preprocessing the training set, and obtaining an adaptive anchor frame according to a clustering algorithm specifically comprises the following steps:

[0061] A1. Perform data augmentation on the image data in the training set. By performing operations such as translation, rotation, changing the aspect ratio of the image, and adjusting the saturation, we increase the sample data and improve the robustness of the small face detection model.

[0062] A2. Perform cluster analysis on the real anchor frames in the training set through clustering algorithms to obtain accurate anchor frames as the predetermined values ​​of the small face detection network.

[0063] Preferably, the step 2: based on the small face detection model of YOLOv5, introducing SK attention and adaptively adjusting the receptive field specifically includes the following steps:

[0064] B1. Split: For any given feature map X∈R H×W×c , after two transformations and It should be noted that and They are all composed of depthwise convolution, normalization and activation functions are run in sequence. To further improve efficiency, the traditional convolution 5*5 is replaced by dilated convolution to 2, and the convolution kernel size is 3*3. Different branches use different convolution kernels to process the input X features, and obtain the feature maps after multi-branch processing.

[0065] B2. Fusion: Control the information flow from multiple branches to carry information of different scales into the neurons of the next layer. First, the results from multiple branches are fused: Then U passes through F gp Global average pooling

[0066]

[0067] Preferably, the step 3: designing a small face feature enhancement module, specifically includes the following steps:

[0068] C1. Upsample the feature map to expand it.

[0069] C2, fuse the shallow feature map with the deep feature map to obtain a larger feature map, further improving the accuracy of small face detection;

[0070] Preferably, the lightweight feature network ShuffleNetV2 has the following specific steps:

[0071] D1. When the number of input channels is not equal to the number of output channels, the memory access cost will increase, so keeping the number of input channels equal to the number of output channels can minimize the memory access cost.

[0072] D2. In group convolution, you cannot simply increase the number of groups to improve accuracy. Too many groups will increase the amount of calculation and reduce the running speed.

[0073] Preferably, the step 4: the trained small face detection deep learning model can be run in real time in the terminal device, specifically comprising the following steps:

[0074] E1. Test the performance of the deep learning model for small face detection based on the improved YOLOv5;

[0075] E2. Evaluate whether the model performance meets the actual application requirements;

[0076] E3. Apply the deep learning model for small face detection to small face detection in real scenes.

[0077] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0078] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0079] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0080] The above embodiments should be understood as merely illustrating the present invention and not as limiting the scope of protection of the present invention. After reading the contents of the present invention, technicians may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A real-time small face detection method based on improved YOLOv5, characterized in that: The following steps are involved: Step 1: Divide the face dataset into a training set and a test set. The training set is preprocessed as follows: Dataset cleaning: Check whether the images and label files in the dataset are complete. If not, delete the redundant images and label files. At the same time, delete the anchor boxes with too small aspect ratios. After deleting, check whether there are empty label files. If so, delete them as well. Obtain adaptive anchor boxes based on the Kmeans clustering algorithm. Step 2: Based on the YOLOv5 small face detection model, the SK selective convolution kernel attention is introduced. The channel weight is obtained by dynamically calculating each convolution kernel, and the results of each convolution kernel are dynamically fused. Step 3: Design a small face feature enhancement module. The small face feature enhancement module is as follows: after the 17th layer, continue to upsample the feature map to make the feature map continue to expand, and at the same time, fuse the feature map obtained at the 20th layer with the feature map of the 2nd layer (concat) to obtain a larger feature map for small target detection, upsample the feature map to make the feature map continue to expand, and fuse the shallow feature map with the deep feature map; Step 4: Use the YOLOv5 model and replace the backbone feature extraction network C3 with the ShuffleNetV2 network; Step 5: Apply the trained deep learning model for small face detection to small face detection in real scenes.

2. A real-time small face detection method based on improved YOLOv5 according to claim 1, characterized in that, Step 1: Divide the face dataset into a training set and a test set, preprocess the training set, and obtain an adaptive anchor frame based on a clustering algorithm, specifically including the following steps: A1. Perform data augmentation on the image data in the training set by performing operations such as translation, rotation, changing the aspect ratio of the image, and adjusting the saturation to increase the sample data; A2. Perform cluster analysis on the real anchor frames in the training set through clustering algorithms to obtain accurate anchor frames as the predetermined values ​​of the small face detection network.

3. A real-time small face detection method based on improved YOLOv5 according to claim 2, characterized in that: A2, performing cluster analysis on the real anchor frames in the training set through a clustering algorithm to obtain accurate anchor frames as predetermined values ​​for the small face detection network, specifically includes: Yolov5 not only uses k-means clustering to generate anchors, but also uses the Genetic Algorithm to perform mutation on the results of k-means clustering.

1. Read the width and height of each image in the training set and the width and height of all anchor boxes; 2. Scale the maximum width and height of each image proportionally to the specified size img_size; 3. Change the anchor box from relative coordinates to absolute coordinates, that is, multiply it by the scaled width and height; 4. Filter anchor boxes and retain those with a width and height greater than or equal to two pixels; 5. Use k-means clustering to obtain n anchor boxes; 6. Use a genetic algorithm to randomly mutate the width and height of the anchor box. If the mutation results in a better result, assign the result to the anchor box. If the mutation results in a worse result, skip it. The default mutation is 1000 times.

7. Return the final mutated anchor frame.

4. A real-time small face detection method based on improved YOLOv5 according to claim 1, characterized in that: Step 2: Based on the small face detection model of YOLOv5, SK attention is introduced, the weight of each convolution kernel is obtained by dynamically calculating the channel, and the results of each convolution kernel are dynamically fused. Specifically, the following steps are included: B1. For any given feature map X∈R H×W×c , H, W, c represent the width, height, and number of channels of the image respectively, R represents the feature map, after two transformations and and They are all composed of depthwise convolution, with normalization and activation functions running in sequence. The traditional 5*5 convolution is replaced by a dilated convolution with a size of 2, and the convolution kernel size is 3*3. Different branches use different convolution kernels to process the input X features, and obtain the feature maps after multi-branch processing. B2. Control the information flow from multiple branches to carry information of different scales into the neurons of the next layer; first, fuse the results from multiple branches: Then U passes through F gp Global average pooling F gp , i, j, uc(i, j) respectively represent the global spatial information, the height of the channel from 1 to H, the width of the channel from 1 to W, and the channel statistical information, and H*W represents the spatial dimension.

5. A real-time small face detection method based on improved YOLOv5 according to claim 4, characterized in that: The SK attention is introduced to obtain the channel weight by dynamically calculating each convolution kernel in F scale (u c ,s c ) refers to the scalar s c Channel-wise multiplication of the feature map.

6. A real-time small face detection method based on improved YOLOv5 according to claim 1, characterized in that: Step 3: A small face feature enhancement module is designed to further expand the feature map and fuse the shallow feature map with the deep feature map. Specifically, the following steps are included: A small face feature enhancement module is designed. After the 17th layer, the feature map is upsampled to further expand the feature map. At the same time, the feature map obtained at the 20th layer is fused (concat) with the feature map at the 2nd layer to obtain a larger feature map for small target detection. C1. Upsample the feature map to expand it; use the nearest neighbor interpolation method for upsampling; C2. Fuse the shallow feature map with the deep feature map. The features extracted by the shallow network are closer to the input and contain more pixel information. Some fine-grained information is some color, texture, edge, and angular information of the image. However, due to fewer convolutions, its semantics is lower and there is more noise. The features extracted by the deep network are some information about the overall image. Some coarse-grained information contains more abstract information, namely semantic information. After the 17th layer, the feature map is continued to be upsampled so that the feature map continues to expand. At the same time, the feature map obtained by the 20th layer is fused (concat) with the feature map of the 2nd layer to obtain a larger feature map for small target detection.

7. A real-time small face detection method based on improved YOLOv5 according to claim 1, characterized in that: Step 4: Use the YOLOv5 model and replace the backbone feature extraction network C3 with the ShuffleNetV2 network. The specific steps are as follows: In addition to the ShuffleNetV2 network, the YOLOv5 model also includes CBL, Resunit, SPP, and Focus.

8. A real-time small face detection method based on improved YOLOv5 according to claim 1, characterized in that: Step 5 applies the deep learning model for small face detection to small face detection in real scenes, specifically including:

1. Use the graphical tool 'Auto Py To Exe' to import the trained best.pt file into the weights folder to generate an exe file; 2. Double-click to run the exe file, which supports image detection, video detection, and camera detection.

Images