Real-time small target detection method based on deep learning
The improved YOLOv7 algorithm with residual channels, deformable convolution, and spatial-channel attention enhances small target detection accuracy and reduces computational load, addressing low detection accuracy and resource demands in complex scenarios.
Patent Information
- Application Number
- CN202411240807.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-27
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing single-stage target detection algorithms face challenges in small target scenarios due to dense targets, complex backgrounds, and interference factors like lighting and shooting angles, leading to low detection accuracy and high computational resource demands.
A real-time small target detection method using an improved YOLOv7 algorithm with a residual channel, deformable convolution, and spatial-channel attention mechanism (CBAM) to enhance feature extraction, combined with lightweight modifications such as depth-wise separable convolution and adjusted anchor boxes, to reduce model size and improve accuracy.
The method achieves improved detection accuracy and reduced computational requirements, enabling real-time small target detection with a smaller model size and faster processing, while maintaining high detection performance.
Smart Images
Figure CN119206412A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of deep learning and computer vision, and specifically relates to a real-time small target detection method based on deep learning. Background Art
[0002] With the emergence of convolutional neural networks, deep learning technology has developed rapidly, and computer vision based on deep learning has gradually been applied to people's daily lives in recent years. In the field of computer vision, target detection algorithms based on deep learning are mainly divided into two categories: single-stage target detection algorithms and two-stage target detection algorithms. The single-stage target detection regards the positioning and classification in target detection as a regression problem. The two-stage target detection algorithm first proposes candidate areas and then classifies the candidate areas. Therefore, the two-stage target detection algorithm takes more time than the single-stage algorithm. However, with the development of the single-stage target detection algorithm, the detection accuracy of the two-stage algorithm is gradually inferior to that of the single-stage algorithm. Therefore, the single-stage algorithm has become the mainstream method in the target detection algorithm.
[0003] The existing mainstream single-stage algorithms such as YOLO and SSD are widely used in autonomous driving, pedestrian counting, license plate detection, traffic signal recognition, etc., which greatly facilitate people's daily life. However, the existing algorithms still have the following problems in small target detection scenarios: dense targets lead to ineffective complete detection and recognition; complex backgrounds and too many interference factors, such as changes in lighting and shooting angles; and serious deformation of objects due to shooting height issues, which is not conducive to detection. Summary of the invention
[0004] In order to solve the problem of low detection accuracy in small target scenarios, the present invention proposes a real-time small target detection method based on deep learning.
[0005] On the one hand, the present disclosure solves the problem of low detection accuracy of existing target detection in small target scenes and reduces the computing resources required by the model. The method includes the following steps:
[0006] Step 1: Configure the deep learning software environment and the target detection algorithm environment based on deep learning;
[0007] Step 2: Build a small target image dataset, obtain real small target images, annotate and divide the images, and summarize the annotated labels to obtain a label set;
[0008] Step 3: Establish an improved YOLOv7 target detection algorithm. First, establish a residual channel to receive the feature map obtained after feature extraction from the backbone network. Then, process the received feature map through deformable convolution and spatial-channel attention mechanism CBAM to enhance the model's attention to small targets.
[0009] Step 4: Lightweight the model. By lightweighting the network Neck part, the model parameters can be greatly reduced, and the computing resources required for model reasoning can be reduced to meet the real-time requirements of the model.
[0010] Step 5: Train the model to obtain a real-time small target detection model.
[0011] Step six: Use the real-time small target detection model to perform real-time detection.
[0012] Specific:
[0013] The target detection algorithm in step 1 is YOLOv7 in the single-stage algorithm;
[0014] The small target dataset described in step 2 is a synthetic dataset. First, the real small target images available on the Internet are used and the images taken by drones are merged to form an extended dataset. After collecting the images, the images are annotated and the labels are summarized into corresponding sets. Finally, the obtained dataset is divided into training set, validation set and test set in a fixed ratio of 7:2:1.
[0015] The improved algorithm described in step three first establishes a residual channel to receive feature maps of different scales obtained after feature extraction from the network backbone, and then sends these feature maps of different scales to the efficient spatial-channel attention mechanism CBAM for processing. The channel attention mechanism in CBAM assigns weights to each channel, assigns larger weights to more important channels, and assigns smaller weights to unimportant channels. Generally speaking, the more important weights range from 0.8 to 0.95, and the unimportant weights range from 0.1 to 0.5. By assigning weights to channels, the model's attention to small target objects is increased, and the spatial attention mechanism extracts important information from the entire image by performing pooling operations on the image. Secondly, deformable convolution is added to the neck part of the model. The features of the variable convolution can extract features more effectively. An effective feature map can be formed to facilitate model learning;
[0016] The model is lightweight as described in step 4. First, the size of the maximum pooling kernel of the feature pyramid in the model is adjusted from the original 5, 9, 13 to 3, 5, 9 to minimize the loss of small target features. In the final head part, the ordinary convolution is replaced by the depthwise separable convolution and a new clustering algorithm is used to obtain a detection anchor box that is more suitable for small targets.
[0017] The fused data set used for model training described in step 5 first collects the small target images publicly available on the Internet, then merges the real small target images taken by drones and other equipment to form a fused data set, and then annotates the data images through label annotation tools to obtain the images and labels required for training; secondly, the data set obtained after preprocessing needs to be randomly divided into training set, validation set and test set according to 7:2:1, and the training set and validation set are used as the training data of the model to train the model. After the model training is completed, the model is tested with the test set in the data set;
[0018] In step 6, the model is used to detect small targets in real time. First, it is necessary to use a drone or other device to take real-time photos and transmit the images to an electronic device used for detection in real time. The electronic device needs to have at least one processor and a storage unit that can store programs and images. The model obtained after training is used to perform real-time detection on the transmitted images and output the detection results in real time.
[0019] Beneficial effects of the present invention:
[0020] The present invention is based on a real-time small target detection method based on deep learning. First, a feature extraction method is designed. Based on the combination of residual channel and space-channel attention mechanism, feature maps of different scales formed by the trunk are obtained through residual channel, and then these feature maps are weighted to each channel through space-channel attention mechanism. Generally speaking, the channel weights obtained by CBAM attention mechanism are generally between 0.8-0.95 for important channels and between 0.1-0.5 for unimportant channels, while space attention mechanism can effectively extract important information from the image. By trimming the model, the model parameters can be effectively reduced. The model parameters after trimming can be reduced by about 1 / 4, and the detection accuracy can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 It is a flow chart of a real-time small target detection method based on deep learning;
[0022] Figure 2 It is the structure diagram of the residual channel attention mechanism;
[0023] Figure 3 It is a deformable convolution structure diagram;
[0024] Figure 4 This is the improved ELAN module structure diagram; Specific implementation plan
[0025] In order to more clearly demonstrate the technical solutions and features of the present invention, the present invention is explained below with reference to the accompanying drawings, but the present invention is not limited to the examples.
[0026] Embodiment 1:
[0027] A real-time small target detection method based on deep learning, the detection method comprising:
[0028] Step 1. Configure the environment: Configure the deep learning-based target detection model environment. This example uses the Linux 5.4.0 operating system, Pytorch version 1.11.0, Cuda 11.3, the GPU used in this example is RTX 3090, and the CPU is Intel (R) Xeon (R) Gold 6330 CPU@2.00GHz The baseline target detection model is the open source target detection model YOLOv7 on Github.
[0029] Step 2: Collect data: Build a small target dataset, obtain real low-light images, annotate the images, and summarize them into a labeled dataset. This example first uses the public dataset Visdrone2019, which contains a total of 10 categories, namely pedestrians, people, cars, vans, buses, trucks, motorcycles, bicycles, awning tricycles, and tricycles. The dataset contains a total of 10,209 images. In addition, this example also uses DJI drones to control drones at a certain height to shoot street scenes to supplement the dataset. Finally, 12,000 images are obtained, and the dataset is divided into training set, validation set, and test set ratios of 7:2:1.
[0030] Step 3: Establish the improved YOLOv7 small target detection algorithm. First, use the residual branch to receive feature maps of different scales formed after the trunk is downsampled, and simply fuse them so that the shallow semantic information and deep features complement each other. The fused feature map is passed into the CBAM attention mechanism. CBAM is an attention module that combines channel attention and spatial attention. It consists of CAM (Channel Attention Module) and SAM (Spatial Attention Module); the CAM module contains a maximum pooling, average pooling, and two fully connected layers. First, the input feature map F (H*W*C) is respectively subjected to global maximum pooling and global average pooling to obtain two 1*1*C feature maps. Then, by sending these two feature maps to the fully connected layer, the processed feature map is sigmoid activated to obtain the channel weight of the feature map. The SAM module receives the feature map after CAM processing, and first undergoes a global maximum pooling operation and a global average pooling operation based on Channel to obtain two H*W*1 feature maps. Then, these two feature maps are spliced based on Channel, and then a 7*7 convolution operation is performed to reduce the dimension of Channel to 1. Then, a sigmoid activation is performed to generate the final spatial attention feature map. After experimental comparison, it is found that with the benchmark architecture of ResNet50, the best effect is achieved by performing the channel attention operation first and then the spatial attention operation.
[0031] Step 4: Lightweight the model to ensure that the model meets the needs of real-time detection. According to the definition of small targets in the COCO dataset, targets with target pixels smaller than 32*32 are generally defined as small targets. In the YOLO target detection network, the original anchor boxes are [12, 16, 19, 36, 40, 28], [36, 75, 76, 55, 72, 146], [142, 110, 192, 243, 459, 401], which are too large for small targets. Therefore, by using a clustering algorithm to form a new small target detection anchor box, the detection accuracy of the model can be improved without increasing the model parameters. In the feature pyramid part of the model, the original maximum pooling kernels of 5, 9, and 13 are changed to 3, 5, and 9, further reducing the receptive field of the model to match the characteristics of small targets. In the head part of the network, in order to further reduce the computational complexity of the network, the depth-wise separable convolution is used to replace the ordinary 3*3 convolution in ELAN. Assuming that the number of channels of the input feature map is m, the size of the convolution kernel is k*k, and the size of the output feature map is (n, h′, w′), the computational cost of the standard convolution is n*h′*w′*k*k*m, and the depth-wise separable convolution is composed of DWConv and PWConv. In the above scenario, the computational cost of DWConv is m*h′*w′*k*k, the computational cost of PWConv is n*h′*w′*1*1*m, and the depth-wise separable convolution = DWConv+PWConv. In summary, the computational cost of the depth-wise separable convolution is approximately 1 / k of that of the ordinary convolution. 2 , which is about 1 / 9 of the original, greatly reducing the depth and width of the model. The reasoning speed of the model is improved, and since the backbone composition of the network is not changed, the impact of replacing the convolution operation in the deep layer of the network on the model detection accuracy is relatively small, so there is no particularly large loss in detection accuracy.
[0032] Step 5: Train the model to obtain a real-time small target detection model. The data set in step 2 is divided into a training set, a validation set, and a test set according to a ratio of 7:2:1 and imported into this example for training, validation, and testing, and finally a small target detection model and its corresponding model parameters are generated.
[0033] Step 6: Use the above model to detect small targets. First, it is necessary to use a drone or other device to take real-time photos and transmit the images to the electronic device used for detection. The electronic device needs to have at least one processor and a storage unit that can store programs and images. Use the model obtained after training to perform real-time detection on the images transmitted back and output the detection results in real time.
[0034] The model of the present invention exceeds YOLOv7 in terms of detection speed and accuracy, and the model size is also smaller. In addition, the present invention uses the residual channel to receive the backbone feature map, and then processes it through the channel attention mechanism, and uses deformable convolution and depth-separable convolution to obtain a better target feature map and reduce the size of the target detection model. While reducing the model's hardware computing power requirements, the model's detection ability in small target detection scenarios is improved and real-time requirements are met.
[0035] The above is a detailed description of the specific implementation of the present invention. It should be understood that individual details are not limited to the above specific implementations. Those skilled in the art can make various variations or modifications within the scope of the claims without affecting the essential content of the present invention.
Claims
1. A real-time small target detection method based on deep learning, the steps of which include: Step 1: Configure the deep learning software environment and the target detection algorithm environment based on deep learning; Step 2: Build a small target image dataset, obtain real small target images, annotate and divide the images, and summarize the annotated labels to obtain a label set; Step 3: Establish an improved YOLOv7 target detection algorithm. First, establish a residual channel to receive the feature map obtained after feature extraction from the backbone network. Then, process the received feature map through deformable convolution and spatial-channel attention mechanism CBAM to enhance the model's attention to small targets. Step 4: Lightweight the model. By lightweighting the network Neck part, the model parameters can be greatly reduced, and the computing resources required for model reasoning can be reduced to meet the real-time requirements of the model. Step 5: Train the model to obtain a real-time small target detection model. Step six: Use the real-time small target detection model to perform real-time detection.
2. The real-time small target detection method based on deep learning according to claim 1, characterized in that: The real small target dataset described in step 2 is a synthetic dataset. First, the real small target images available on the Internet are used and the images taken by drones are merged to form an extended dataset. After collecting the images, the images are annotated and the labels are summarized into corresponding sets. Finally, the obtained dataset is divided into training set, validation set and test set in a fixed ratio of 7:2:
1.
3. The real-time small target detection method based on deep learning according to claim 1 is characterized in that: The improved YOLOv7 target detection algorithm described in step 3 is specifically designed for small target detection scenarios. The CBAM attention mechanism is an attention module that mixes the spatial attention mechanism and the channel attention mechanism. The channel attention mechanism can assign weights to each channel. The larger the weight, the more important the information contained in the channel. The spatial attention mechanism extracts the important information contained in the image by performing maximum pooling and average pooling on the image, thereby extracting all the important information in the feature map.
4. The real-time small target detection method based on deep learning according to claim 3 is characterized in that: The improved YOLOv7 algorithm further includes: The image received by the CBAM attention module is the feature map formed in the backbone network directly transmitted by the residual module, so that weights can be assigned to the channels of feature maps at different scales, and the shallow-scale features and deep-scale features of the object can be fused and complemented; in the Neck part of the network, deformable convolution is used instead of ordinary convolution to more effectively extract the feature information of the object.
5. The real-time small target detection method based on deep learning according to claim 4 is characterized in that: Compared with ordinary convolution, the deformable convolution used in the above method has an additional learnable offset parameter when performing convolution operation, so that the deformable convolution can more effectively highlight the edge contour information of the object.
6. The real-time small target detection method based on deep learning according to claim 1, characterized in that: The lightweight processing of the model described in step 4 is mainly carried out by adjusting the network structure. In the feature pyramid module, the pooling multiples of the maximum pooling layer are reduced from 5, 9, and 12 to 3, 5, and 9 to retain some features of small targets. In the final head part, the ordinary convolution is replaced by the depthwise separable convolution and the clustering algorithm is used to obtain the detection anchor box that is more suitable for small targets.
7. The real-time small target detection method based on deep learning according to claim 1, characterized in that: The model training part described in step 5 needs to first randomly divide the data set obtained after preprocessing into a training set, a validation set and a test set according to a ratio of 7:2:1, and use the training set and the validation set as the training data of the model to train the model. After the model training is completed, the model is first tested using the test set in the data set. After the verification on the test set is completed, the real image is taken using a drone or other equipment, and the captured image is sent to the small target detection model in claim 1 to verify the detection effect under real conditions.