A lightweight small target detection method and device that integrates multi-scale features

Abstract

This invention discloses a lightweight small object detection method and apparatus that integrates multi-scale features. It constructs a dataset for small object detection, dividing it into training and validation sets. The Backbone in the original YOLOv5 model is replaced with a lightweight feature extraction network, Shuffle-Block. Shuffle-Block replaces the ordinary 1×1 convolution operation of the original branch with grouped convolution, using channel shuffling to facilitate information flow between different groups and enhance feature communication. A GCA attention module is introduced into the Neck layer of the YOLOv5 model, enabling the fully connected layer with fixed weights to generate an attention map with a global receptive field. An Adaptive Spatial Feature Fusion (ASFF) module is introduced to address inconsistencies within the feature pyramid in the YOLOv5 model. The training set is input into the improved YOLOv5 model to obtain optimal weights, and finally, the validation set is input into the network to output the detection results. This invention significantly reduces the number of parameters and computational cost of the object detection model, and improves the detection speed of small objects, especially for devices with limited computing power.

Description

Technical Field

[0001] This invention pertains to the application of deep learning in the field of computer vision, specifically a lightweight method and apparatus for small object detection that integrates multi-scale features. Background Art

[0002] Object detection involves identifying specific object categories and their precise locations within a given image. Small object detection is a crucial research area within object detection, with significant applications in remote sensing image target recognition, infrared imaging target recognition, and agricultural pest and disease identification. In object detection, targets with a pixel value less than 0.12% of the entire image or a pixel value smaller than 32*32 are typically defined as small objects. Due to the low resolution and high noise levels of small objects, the features extracted after multiple convolutional layers are often insignificant, making the detection of small objects in images very challenging.

[0003] Currently, there are two main types of algorithms for small object detection: one-stage algorithms such as YOLO and SSD have strong real-time performance and fast model inference speed, but their accuracy is not high enough; the other-stage algorithms are the RCNN series, which have high accuracy, but slow training speed and high difficulty in real-time deployment.

[0004] In recent years, most research has focused on improving the accuracy of small target detection. However, these methods have not considered practical application issues, and the high computational cost significantly reduces their real-time performance. For example, the existing invention patent document CN115331126A, "Construction and Detection Method of a Small Target Detection Model Based on YOLOv5," adds a small target detection layer, introduces a GMA attention mechanism, and expands the detection size of the feature fusion anchor box. This method can improve the accuracy of small target detection, but it increases the number of parameters and the complexity of the algorithm. The existing invention patent document CN115410039A, "Coal Foreign Object Detection System and Method Based on Improved YOLOv5 Algorithm," uses a lightweight network structure and an ECA attention mechanism to improve accuracy, but it only targets a single small target and has significant limitations for detecting multiple types of small targets in the same background.

[0005] Existing technologies increase model depth and stack parameters in pursuit of small target detection accuracy, resulting in slow model detection speed and a large number of parameters, which is not conducive to the application of low-end and mid-range equipment, while ignoring practical applications. Summary of the Invention

[0006] Purpose of the invention: In order to solve the problems of redundant parameters, slow speed, low accuracy and weak connection between shallow and deep semantic information in existing target detection methods for small target detection, the present invention provides a lightweight small target detection method and device that integrates multi-scale features.

[0007] Technical Solution: This invention provides a lightweight small target detection method that integrates multi-scale features, specifically including the following steps:

[0008] (1) Preprocess the pre-built dataset for small object detection and divide the dataset into training set and validation set;

[0009] (2) Replace the Backbone in the original YOLOv5 model with the lightweight feature extraction network Shuffle_Block; Shuffle-Block changes the ordinary 1×1 convolution operation of the original branch to group convolution, and helps information flow between different groups through channel shuffling to enhance feature communication.

[0010] (3) Introduce the GCA attention module into the NECK layer of the YOLOV5 model to generate an attention map with a global receptive field in the fully connected layer with fixed weights.

[0011] (4) Introduce the adaptive spatial feature fusion module ASFF to solve the inconsistency within the feature pyramid in the YOLOv5 model;

[0012] (5) Input the training set into the improved YOLOv5 model to obtain the optimal weights, and finally input the validation set into the network to output the detection results.

[0013] Further, the preprocessing procedure in step (1) is as follows:

[0014] The original dataset format was converted to YOLO format. The high-resolution images in the original dataset were first segmented, and an overlap region was set between two small images. After the small images were detected, all the bounding boxes were placed into the large image. An NMS operation was performed on the entire large image to delete many duplicate bounding boxes in the overlap region. Some regions in the original image contained a large number of dense small targets that could not be labeled. These regions were occluded to reduce the impact of these regions on accuracy.

[0015] Further, in step (2), the Shuffle-Block, with stride=1, divides the C channels of the input feature map into two branches. One branch remains unchanged, while the other branch first performs a 1×1 grouped convolution operation to reduce the number of parameters. Then, Channel Shuffle is performed to reorganize the feature map and increase channel connections. Next, a 3×3 depthwise separable convolution is performed to compress the model parameters. Then, a 1×1 convolution is performed, followed by a channel shuffling. Finally, the channel is concatenated with the other branch through Concat. Finally, the feature map is reorganized through Channel Shuffle operation. BN and ReLU operations are performed after each 1×1 convolution.

[0016] Furthermore, the implementation process of step (3) is as follows:

[0017] The fully connected (FC) layer is decomposed into a horizontal and a vertical FC layer. The GCA attention module first performs a downsampling operation on the input feature map using global average pooling, followed by 1×1 convolution, batch normalization (BN), a horizontal fully connected layer, BN, and ReLU, a vertical fully connected layer, BN, and ReLU. The feature map obtained by bilinear interpolation is then the original size, and finally the feature weight information is output through the SI GMOID activation function.

[0018] Furthermore, the implementation process of step (4) is as follows:

[0019] In the Head layer, the original YOLOv5 model feature fusion network PAnet is replaced with the ASFF module. The ASFF module adaptively learns the weight parameters of feature fusion at each level, then uses 1×1 convolution to compress it to the original number of channels, outputting feature layers ASFF-1, ASFF-2, and ASFF-3, which are finally input into the prediction network.

[0020] Based on the same inventive concept, the present invention also provides a lightweight small target detection device that integrates multi-scale features, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the above-mentioned lightweight small target detection method that integrates multi-scale features.

[0021] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are: the present invention addresses the limited computing power of low-end devices by adopting a lightweight feature extraction network and a multi-feature fusion method, which significantly reduces the number of parameters and computing costs of the target detection model and improves the detection speed of small targets. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention;

[0023] Figure 2 This is a schematic diagram of the Shuffle-Block module proposed in this invention;

[0024] Figure 3 This is a schematic diagram of the CA attention mechanism proposed in this invention.

[0025] Figure 4 This is a schematic diagram of the ASFF feature fusion structure proposed in this invention;

[0026] Figure 5 This is a schematic diagram of the small target detection network model structure based on the improved YOLOv5 proposed in this invention;

[0027] Figure 6 This is a screenshot showing the detection results of this invention on the VisDrone2019 dataset. Detailed Implementation

[0028] The present invention will be described in further detail below with reference to the accompanying drawings.

[0029] This invention provides a lightweight small target detection method that integrates multi-scale features, improving the YOLOv5 model. Specifically, the improved YOLOv5 model comprises three parts: a Backbone layer, a Neck layer, and a Head layer. The Backbone layer replaces the original YOLOv5 CSPDarkNet structure with a lightweight feature extraction module, shuffle_block. Specifically, for a single CBRM operation, layers one through six are shuffle_blocks with different input parameters. The Neck layer introduces a GCA attention model based on the original YOLOv5 to improve the representational power of the lightweight model. The Head layer is modified to enhance detection capabilities by replacing the existing model with an ASFF feature fusion module. Figure 1 As shown, the specific steps include:

[0030] Step 1: Construct a dataset for small object detection and divide the dataset into a training set and a validation set.

[0031] The dataset was collected using different drone platforms under different scenarios, weather and lighting conditions, including categories such as pedestrians, cars, and bicycles. Different backgrounds and types can better improve the generalization ability of the detection model.

[0032] The dataset used in this embodiment is the Visdrone 2019 dataset, a publicly available drone image feature set, including 6471 training samples and 1610 test samples. The dataset format labeled in the Visdrone dataset is converted to YOLO format. Because many images in the original dataset have excessively high resolution, simple downsampling would result in excessive downsampling factor, easily leading to data loss. However, excessive downsampling factor would cause a large number of feature maps to stack during network forward propagation, greatly wasting computational resources and affecting inference speed. Therefore, this embodiment first segments the high-resolution images in the original dataset. To avoid target segmentation during the segmentation process, an overlap region is set between the two smaller images. After the smaller images are detected, all bounding boxes are placed into the larger image, and an NMS operation is performed on the entire larger image to delete many duplicate bounding boxes in the overlap region. Some areas in the original images contain a large number of densely packed small targets that cannot be labeled. This embodiment occludes these areas to reduce their impact on accuracy.

[0033] Step 2: Replace the original YOLOv5 Backbone layer CSPDarkNet structure with the lightweight feature extraction module shuffle_block.

[0034] ShuffleNetv2, proposed by Megvii, is a high-efficiency, lightweight network widely used in low-end devices (such as mobile phones). Like DenseNet, ShuffleNetv2's architecture implements a feature reuse pattern, resulting in high accuracy and higher efficiency. The ShuffleNetv2 network uses grouped pointwise product and a bottleneck structure to reduce MAC (memory access cost). This invention proposes Shuffle-Block, which replaces the original YOLOv5 backbone with a lightweight feature extraction network, Shuffle_Block. Specifically, a CBRM unit first extracts features from the initial image. The CBRM includes a Conv layer, a BN layer, a Rilu activation function, and a max-pooling layer. These features are then input into the first-layer Shuffle_Block module, making the input matrix equal to the output matrix, thus minimizing the MAC and significantly reducing the number of parameters in the original YOLOv5 network model. To further reduce network fragmentation and address the slow model speed caused by other element-level operations, layers three through seven use Shuffle_Block modules of different scales, with the principle being the same as described above.

[0035] As an improved feature extraction network for YOLOv5, to ensure that the number of input channels matches the number of output channels, such as... Figure 2As shown, a simple channel split operator is introduced to ensure that the channel widths are equal after splitting. With stride=1, the C channels of the input feature map are split into two branches. To minimize network fragmentation, one branch remains unchanged. Unlike ShuffleNetv2, the other branch first undergoes a 1×1 grouped convolution operation to reduce the number of parameters, followed by channel shuffling to reorganize the feature map and increase channel connections. Then, a 3×3 depthwise separable convolution is performed to compress the model parameters, followed by a 1×1 convolution, then a channel shuffle, and finally concatenated with the other branch through concat. This ensures that the number of output channels is consistent with the number of input channels. Finally, the feature map is reorganized through a channel shuffle operation. After each 1×1 convolution, Batch Normalization (BN) and ReLU operations are performed to accelerate model convergence and prevent gradient vanishing. YOLOv5 uses the Shuffle-Block module for feature extraction, employs pointwise grouped convolution to reduce computation, and uses channel shuffle to solve the feature communication problem between different groups. Unlike ShuffleNetV2, this invention first performs grouped convolution in the main branch to further reduce the parameter redundancy caused by 1×1 convolution, and then performs feature map reorganization to strengthen the connection.

[0036] Step 3: Introduce the GCA attention module into the Neck layer of the YOLOV5 model, so that the fully connected layer with fixed weights generates an attention map with a global receptive field.

[0037] This invention introduces an attention mechanism into the neck enhancement module. Self-attention has the advantage of long-range modeling capability, but its computational complexity increases quadratically with the input resolution, which is very unfavorable for high-resolution images. Small object detection often requires high-resolution input. To better locate small objects, such as… Figure 3The GCA attention module proposed in this invention is a highly efficient and versatile attention mechanism that utilizes a simpler and easier-to-implement fully connected layer (FC) with fixed weights to generate an attention map with a global receptive field. Specifically, the FC layer is decomposed into a horizontal and a vertical FC layer. The GCA attention module first performs a downsampling operation on the input feature map using global average pooling, followed by a 1×1 convolution, batch normalization (BN), a horizontal fully connected layer, BN, ReLU, a vertical fully connected layer, BN, ReLU, and then the feature map obtained by bilinear interpolation is the original size. Finally, the feature weight information is output through a sigmoid activation function. By downsampling in the horizontal and vertical directions, the size of the features is reduced, and the width and height of the features are scaled to half of their original length, which reduces Flops by 75%. The feature maps with specific information embedded in these two directions are encoded into two attention maps, each capturing the long dependencies of the input feature map along a spatial direction. The two attention maps are then applied to the input feature map through multiplication to emphasize the representations of interest. It complements the input feature map, which is beneficial for accurately capturing information about small targets.

[0038] The improved YOLOv5 lightweight feature extraction network extracts features from images of different sizes, resulting in three effective feature layers, which are then input into the Neck layer. To enhance the expressive power of the lightweight network's features, this embodiment introduces a GCA attention mechanism in the Neck layer, as detailed below. Figure 5 As shown, the final small object detection network model based on YOLOv5 is obtained. The proposed GCA attention module is added to the eleventh, sixteenth and twentieth layers of the improved YOLOv5 network to enhance the weights for small object detection, thereby improving detection accuracy while reducing the number of parameters.

[0039] Step 4: This invention changes the feature fusion method of the head prediction module in the target detection model and introduces the adaptive spatial feature fusion module ASFF. ASFF is used to solve the inconsistency within the feature pyramid in YOLOv5.

[0040] When a target is classified as positive in one layer, the corresponding target region is classified as negative in other layers. Large targets are detected in one layer, and small targets in another. However, multi-size detection in the network doesn't just detect a specific region; it detects the entire image. During feature fusion, much useless information from other layers is also incorporated. ASFF enables the network to learn how to spatially filter features at other levels, thus retaining only useful information for combination. In FPN-type networks, features are heuristically selected, detecting large targets from high-level semantic information and small targets from low-level semantic information.

[0041] In the Head layer, the original YOLOv5 model feature fusion network PAnet is replaced with an ASFF module. The feature maps obtained after passing through the Neck layer are three different scales and are input into the improved Head layer before being fed into the ASFF module. Figure 4 As shown, the ASFF module adaptively learns the weight parameters for feature fusion at each level, then uses 1×1 convolutions to compress the data back to the original number of channels, outputting feature layers ASFF-1, ASFF-2, and ASFF-3, which are finally input into the prediction network. By adaptively learning the spatial weights for feature fusion, more hierarchical features can be extracted, thereby improving the accuracy of small object detection.

[0042] Step 5: Input the training set into the small object detection network model based on YOLOv5 to obtain the optimal weights, and finally input the validation set into the detection network model to output the detection results.

[0043] Based on the same inventive concept, the present invention also provides a lightweight small target detection device that integrates multi-scale features, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the above-mentioned lightweight small target detection method that integrates multi-scale features.

[0044] The present invention was tested on the VisDrone2019 dataset. In order to make a direct comparison of these methods, the embodiments of the present invention use mAP and Runtime as evaluation metrics for model performance, as shown in Table 1.

[0045] Table 1 Comparison of experimental results for different models

[0046] Model Runtime(h) <![CDATA[ Params (M)]]> <![CDATA[ GFLOPs ]]> <![CDATA[ mAP (%)]]> YOLOv5n 7.4 1.87 4.6 32.6 YOLOv5s 7.66 7.2 15.8 35.8 YOLOv5l 9.73 46.5 107.8 40 Ours 7.54 9.3 16.5 38

[0047] Table 1 shows the detection results of the widely used model. Compared with other advanced models, in this embodiment of the invention, the model depth and channel width are set to 1, which is consistent with YOLOv5l. However, YOLOv5l has 46.5M parameters and can only be experimentally applied on some large cloud devices, resulting in poor real-time performance. In contrast, the parameter count of this embodiment is only 9.3M, greatly reducing the complexity of the model and making it applicable to low-end devices, thus expanding its applicability. Although the accuracy is not as high as YOLOv5l, compared with the fastest detection speeds YOLOv5s and YOLOv5n, this embodiment of the invention has made significant improvements in both speed and accuracy, and the detection effect is as follows. Figure 6 As shown, this significantly improves the false detection rate of small target detection, which will be more beneficial for practical applications.

Claims

1. A lightweight small target detection method that integrates multi-scale features, characterized in that, Includes the following steps: (1) Preprocess the pre-built dataset for small object detection and divide the dataset into training set and validation set; (2) Replace the Backbone in the original YOLOv5 model with the lightweight feature extraction network Shuffle_Block; Shuffle-Block changes the ordinary 1×1 convolution operation of the original branch to group convolution, and helps information flow between different groups through channel shuffling to enhance feature communication. (3) In YOLO V 5. N of the model ECK The GCA attention module is introduced into the layer, enabling the fully connected layer with fixed weights to generate an attention graph with a global receptive field; (4) Introduce the adaptive spatial feature fusion module ASFF to solve the inconsistency within the feature pyramid in the YOLOv5 model; (5) Input the training set into the small object detection network model based on YOLOv5 to obtain the optimal weights, and finally input the validation set into the network to output the detection results; In step (2), the Shuffle-Block, with stride=1, divides the C channels of the input feature map into two branches. One branch remains unchanged, while the other branch first undergoes a 1×1 grouped convolution operation to reduce the number of parameters. Then, Channel Shuffle is performed to reconstruct the feature map and increase channel connections. Next, a 3×3 depthwise separable convolution is performed to compress the model parameters. Then, a 1×1 convolution is performed, followed by a channel shuffling. Finally, the channel is concatenated with the other branch through Concat. Finally, the feature map is reconstructed through Channel Shuffle. BN and ReLU operations are performed after each 1×1 convolution. The implementation process of step (3) is as follows: The fully connected (FC) layer is decomposed into one horizontal and one vertical FC layer. The GCA attention module first performs a downsampling operation on the input feature map using global average pooling, followed by 1×1 convolutions, batch normalization (BN), and then a horizontal fully connected layer, BN, and R. ELU Vertical fully connected layer, BN, R ELU Then, the feature map obtained through bilinear interpolation is used as the original size, and finally, S... I The GMOID activation function outputs feature weight information; The implementation process of step (4) is as follows: In the Head layer, the original YOLOv5 model feature fusion network PAnet is replaced with the ASFF module. The ASFF module adaptively learns the weight parameters of feature fusion at each level, then uses 1×1 convolution to compress it to the original number of channels, outputting feature layers ASFF-1, ASFF-2, and ASFF-3, which are finally input into the prediction network.

2. The lightweight small target detection method fusing multi-scale features according to claim 1, characterized in that, The preprocessing process described in step (1) is as follows: The original dataset format was converted to YOLO format. The high-resolution images in the original dataset were first segmented, and an overlap region was set between two small images. After the small images were detected, all the bounding boxes were placed into the large image. An NMS operation was performed on the entire large image to delete many duplicate bounding boxes in the overlap region. Some regions in the original image contained a large number of dense small targets that could not be labeled. These regions were occluded to reduce the impact of these regions on accuracy.

3. A lightweight small target detection device integrating multi-scale features, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the lightweight small target detection method that fuses multi-scale features according to any one of claims 1 to 2.

Images