Application of a small target detection algorithm in traffic monitoring
By combining a lightweight image super-resolution reconstruction network with the YOLOv3 algorithm, the problems of low accuracy and poor real-time performance of small target detection in traffic monitoring are solved, and efficient and accurate small target detection is achieved.
Patent Information
- Application Number
- CN202310412581.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-18
- Publication Date
- 2025-09-26
- Estimated Expiration
- 2043-04-18
AI Technical Summary
Existing small target detection algorithms have low detection accuracy and high missed detection rate in traffic monitoring, and the GAN-based super-resolution algorithm is difficult to achieve real-time performance and is difficult to apply to complex traffic environments.
A lightweight image super-resolution reconstruction network structure is adopted, including a shallow feature extraction module, a deep feature extraction module, an attention module and a reconstruction module. The YOLOv3 algorithm is combined for small target detection, and the network is trained through data enhancement and L2 loss function.
It improves the accuracy and real-time performance of small target detection, reduces the number of parameters in the network model, makes it easy to train and deploy, and achieves efficient detection in complex traffic environments.
Smart Images
Figure CN116524432B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traffic monitoring, and in particular to an application of a small target detection algorithm in traffic monitoring. Background Art
[0002] As my country's economy shifts from high-speed growth to high-quality development, the transportation industry has reached a stage where it can basically adapt to the needs of economic and social development. The national road mileage and road traffic volume are growing rapidly. By the end of 2022, the national motor vehicle ownership will reach 348 million, and the number of motor vehicle drivers will reach 435 million. However, due to my country's complex traffic conditions and weak road infrastructure safety assurance capabilities, current traffic management and control capabilities cannot meet the rapidly expanding traffic demand. Traffic conflicts between pedestrians, vehicles, and roads are becoming increasingly prominent, resulting in frequent transportation accidents, which account for a significantly higher proportion of the total national production safety accidents than other industries.
[0003] Using intelligent monitoring systems to detect and predict traffic accidents can significantly reduce the harm caused by them. The core of intelligent monitoring systems lies in object detection algorithms. However, detecting small objects is challenging. Objects such as pedestrians and small vehicles are highly concentrated in images, with sizes below 50 pixels, and their colors, edges, and other features are blurred. These objects are difficult to discern in complex traffic environments with numerous negative examples, such as electric vehicles. Consequently, detection accuracy for small pedestrians and vehicles is low, and the missed detection rate is high. Improving the detection accuracy of these small objects is crucial for traffic safety.
[0004] Existing small object detection methods are generally optimized based on mainstream object detection algorithms, such as enhancing small object samples, optimizing training methods, anchor-free mechanisms, and feature fusion. In 2018, Bai et al. proposed an end-to-end Multi-Task Generative Adversarial Network (MTGAN) to address small object detection accuracy issues. Its method steps are as follows:
[0005] 1) Crop the input image as required;
[0006] 2) Input it into a baseline object detector with Faster RCNN or Mask RCNN as the backbone network to initially identify the object and background;
[0007] 3) The image input generator after preliminary recognition is a super-resolution network that upsamples small blurry images to fine images and restores their details for more accurate detection;
[0008] 4) The super-resolution restored image is input to the discriminator. The discriminator is a multi-task network that describes each super-resolution image patch using a real / fake score, an object category score, and a bounding box regressor. In order to enable the generator to restore more small object details for easier detection, the discriminator will backpropagate the classification and regression losses to the generator during training, thereby improving the generator's generation effect.
[0009] Disadvantages of existing technology:
[0010] 1) Small target detection methods optimized based on mainstream target detection algorithms, such as enhancing small target samples, optimizing training methods, anchor-free mechanisms, and feature fusion, do not fundamentally solve the problem of missing details in small targets. Although the overall detection accuracy of small targets has indeed been improved, the improvement is relatively limited, and many algorithms may produce artifacts in actual applications.
[0011] 2) Introducing a super-resolution algorithm based on a Generative Adversarial Network (GAN) into target detection has effectively improved the detection accuracy of small targets. It can effectively expand the resolution of small targets and increase the amount of feature information. However, because it greatly increases the number of network layers in the algorithm, the training of the GAN network is relatively difficult, making it difficult to achieve real-time performance and apply it to specific scenarios. Summary of the Invention
[0012] In view of the above-mentioned shortcomings, the present invention provides an application of a small target detection algorithm in traffic monitoring.
[0013] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:
[0014] A lightweight image super-resolution reconstruction network structure construction method, which consists of four parts: shallow feature extraction module, deep feature extraction module, attention module and reconstruction module;
[0015] The shallow feature extraction module maps the input image to a high-dimensional feature space and includes a convolution layer with a convolution kernel size of 3*3. The process is expressed as: ;
[0016] The deep feature extraction module consists of several Vast-receptive-field Information distillation Blocks (VIDBs). It performs deep feature extraction on X0 and gradually refines the extracted features by stacking VIDBs. The process is expressed as follows: ;
[0017] The attention module consists of two parts: the Efficient Spatial Attention (ESA) module and the Contrast-based Channel Attention (CCA) module.
[0018] The reconstruction module uses the Pixelshuffle algorithm to complete the reconstruction, and the shape is ( ) is reconstructed into a tensor of shape ( ) tensor.
[0019] In particular, the VIDB block first performs a convolution operation on the input image with a convolution kernel size of 1*1, and then divides the processing into two branches, namely the first branch and the second branch. The processing results of the first branch and the second branch are added and output, and pixel normalization is performed.
[0020] In particular, the first branch is a direct connection path, and the second branch is first activated by an activation function based on a gate function, and then feature weights are distributed through a channel attention module based on information distillation and deep separation convolution operations with large convolution kernels. Then, features between feature maps are fused through a convolution layer with a convolution kernel size of 1*1, and the fused features are added to the direct connection path.
[0021] In particular, the gate function-based activation function first divides the input feature map of size C*H*W into two feature maps of size C / 2*W*H according to the number of channels, and then multiplies them and outputs them; the channel attention module divides the activated feature map into two branches, namely the third branch and the fourth branch, and the processing results of the third branch and the fourth branch are added and output, and a convolution operation with a convolution kernel size of 1*1 is performed.
[0022] In particular, the third branch first performs a convolution operation with a convolution kernel size of 1*1, and then performs a depthwise convolution operation with a convolution kernel size of 9*9, a stride of 1, and a padding of 4; the fourth branch first performs a convolution operation with a convolution kernel size of 1*1, and then activates it through the GELU activation function.
[0023] In particular, the CCA module performs contrast loss calculation and adaptive global pooling on the input image, and then adds them together. The output after addition is sequentially passed through a convolution operation with a convolution kernel size of 1*1, activation with a ReLU activation function, and a convolution operation with a convolution kernel size of 1*1. Finally, it is multiplied with the input image to obtain the output image, completing feature learning based on position information.
[0024] In particular, the ESA module first performs a convolution operation with a convolution kernel size of 1*1 on the input image, and then divides the processing into two branches, namely the fifth branch and the sixth branch. The processing results of the fifth branch and the sixth branch are added together. The feature map after the addition is first subjected to a convolution operation with a convolution kernel size of 1*1 to fuse the features and restore the number of channels. It is then activated by the sigmoid linear activation function and then multiplied with the input image to obtain the output image, thereby learning the cross-channel interaction relationship without reducing the dimension.
[0025] In particular, the fifth branch performs a convolution operation with a convolution kernel size of 1*1; the sixth branch performs a convolution operation with a convolution kernel size of 3*3, a stride of 2, and a padding of 1; a maximum pooling layer with a kernel size of 7*7 and a stride of 7; a depthwise convolution operation with a convolution kernel size of 3*3, GELU activation function activation, and bilinear interpolation processing to scientifically restore the original image size.
[0026] A method for applying a small target detection algorithm in traffic monitoring is based on a lightweight image super-resolution reconstruction network structure construction method, comprising the following steps:
[0027] S1. Perform preliminary processing on the image to be detected. The specific steps are as follows:
[0028] S11, converting the format of the low-resolution image to be processed to obtain a low-resolution YCbCr image;
[0029] S12, cutting the low-resolution YCbCr image into a number of sub-images with equal spacing in rows and columns, wherein the size of the sub-images after cutting is 480*480 pixels;
[0030] S13, randomly rotating the sub-image by 90° or 180° for data augmentation to provide more data samples and reduce the storage space required for feature maps during network transmission;
[0031] S2. Build a lightweight image super-resolution reconstruction network and complete network training. The training loss function adopts L2 loss.
[0032] S3, performing edge sharpening on the image after super-resolution processing;
[0033] S4. Input the edge-sharpened image into the detection module for detection and obtain the small traffic target detection result.
[0034] In particular, the detection module uses the YOLOv3 algorithm to divide the image into multiple regions and predict the bounding box and the probability of each region.
[0035] The beneficial effects of the present invention are:
[0036] 1. The present invention sets up a lightweight image super-resolution reconstruction network structure, and the network structure includes a shallow feature extraction module, a deep feature extraction module (VIDB block), an attention module (ESA module, CCA module), and a reconstruction module. The super-resolution algorithm effectively expands the resolution of small targets and increases the amount of feature information. Compared with traditional methods based on target detection algorithm optimization, the improved accuracy is higher and more effective.
[0037] 2. This invention replaces the GAN-based super-resolution network with a lightweight image super-resolution reconstruction network, which greatly reduces the number of network model parameters while improving the detection accuracy of small target objects, making it easier to train and deploy.
[0038] 3. The present invention uses YOLOv3 as the detection module of the system, which ensures a good compromise between detection accuracy and detection speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 This is a block diagram of the application system of the present invention in traffic monitoring;
[0040] Figure 2 Schematic diagram of the VIDB block structure of the present invention;
[0041] Figure 3 Schematic diagram of the lightweight image super-resolution reconstruction network structure of the present invention;
[0042] Figure 4 Schematic diagram of the CCA module structure of the present invention;
[0043] Figure 5 Schematic diagram of the ESA module structure of the present invention;
[0044] Figure 6 Schematic diagram of the YOLOv3 algorithm structure of the present invention;
[0045] The following is a detailed description of the embodiments of the present invention with reference to the accompanying drawings. DETAILED DESCRIPTION
[0046] The present invention will be further described below in conjunction with embodiment:
[0047] like Figure 3 As shown in the figure, a lightweight image super-resolution reconstruction network structure construction method is composed of four parts: shallow feature extraction module, deep feature extraction module, attention module and reconstruction module;
[0048] The shallow feature extraction module maps the input image to a high-dimensional feature space and includes a convolution layer with a convolution kernel size of 3*3. The process is expressed as: ;
[0049] The deep feature extraction module consists of several Vast-receptive-field Information distillation Blocks (VIDBs). It performs deep feature extraction on X0 and gradually refines the extracted features by stacking VIDBs. The process is expressed as follows: ;
[0050] Specifically, if Figure 2 As shown in the figure, the VIDB block first performs a convolution operation on the input image with a convolution kernel size of 1*1, and then divides the processing into two branches, namely the first branch and the second branch. The processing results of the first branch and the second branch are added and output, and pixel normalization operation is performed.
[0051] The first branch is a direct connection path, and the second branch is first activated by an activation function based on a gate function, and then the feature weights are distributed through a channel attention module based on information distillation and deep separation convolution operation with a large convolution kernel. Then, a convolution layer with a convolution kernel size of 1*1 is used to fuse the features between feature maps, and the fusion is added to the direct connection path.
[0052] The key innovation of this invention lies in proposing an activation function based on a gate function and a channel attention module based on information distillation and deep separation convolution operations with large convolution kernels, namely:
[0053] The gate-based activation function first divides the input feature map of size C*H*W (where C refers to the number of channels, which is 64, and H*W is the size of the cropped image, which is 480*480 pixels) into two feature maps of size C / 2*W*H according to the number of channels, and then multiplies them and outputs them, achieving an effect similar to the traditional activation function but with a greatly reduced number of parameters.
[0054] The channel attention module divides the activated feature map into two branches, namely the third branch and the fourth branch. The processing results of the third branch and the fourth branch are added and output, and a convolution operation with a convolution kernel size of 1*1 is performed.
[0055] The third branch first performs a convolution operation with a convolution kernel size of 1*1, and then performs a depthwise convolution operation with a convolution kernel size of 9*9, a stride of 1, and a padding of 4; the third branch performs depthwise separation convolution with a large convolution kernel. The large 9*9 convolution kernel is beneficial to the extraction of image feature information. The depthwise separation operation divides a convolution layer into pointwise convolution and depthwise convolution with a stride of 1 and padding of 4, which greatly reduces the number of parameters; the fourth branch first performs a convolution operation with a convolution kernel size of 1*1, and then activates it through the GELU activation function; the reduction in parameter amount makes it easier to train and implement deployment, has good real-time performance, and is easy to apply to specific scenarios.
[0056] The attention module consists of two parts: the Efficient Spatial Attention (ESA) module and the Contrast-based Channel Attention (CCA) module.
[0057] Specifically, the present invention adds an attention module after the deep feature extraction module to further extract the performance of the neural network. The ESA module is a lightweight channel attention module that can learn cross-channel interactions through a one-dimensional convolutional layer without reducing the dimension; the CCA module can embed position information into the channel attention and generate an attention map with spatial selectivity.
[0058] like Figure 4 As shown in the figure, the CCA module performs contrast loss calculation and adaptive global pooling on the input image, and then adds them together. After the addition, the output passes through a convolution operation with a convolution kernel size of 1*1, a ReLU activation function activation, and a convolution operation with a convolution kernel size of 1*1. Finally, it is multiplied with the input image to obtain the output image, completing feature learning based on position information.
[0059] like Figure 5 As shown in the figure, the ESA module first performs a convolution operation with a convolution kernel size of 1*1 on the input image, and then divides the processing into two branches, namely the fifth branch and the sixth branch. The processing results of the fifth branch and the sixth branch are added together. The feature map after the addition is first subjected to a convolution operation with a convolution kernel size of 1*1 to fuse the features and restore the number of channels. It is then activated by the sigmoid linear activation function and then multiplied with the input image to obtain the output image. It learns the cross-channel interaction relationship without reducing the dimension.
[0060] Among them, the fifth branch performs a convolution operation with a convolution kernel size of 1*1; the sixth branch performs a convolution operation with a convolution kernel size of 3*3, a step size of 2, and a padding of 1; a maximum pooling layer with a kernel size of 7*7 and a step size of 7; a depthwise convolution operation with a convolution kernel size of 3*3, GELU activation function activation, and bilinear interpolation processing to scientifically restore the original image size.
[0061] The reconstruction module is an upsampling module that uses the Pixelshuffle algorithm to complete the reconstruction. The Pixelshuffle algorithm can achieve efficient sub-pixel convolution with a step size of 1 / r. ) is reconstructed into a tensor of shape ( ) tensor.
[0062] like Figure 1 As shown, a method for applying a small target detection algorithm in traffic monitoring is based on a lightweight image super-resolution reconstruction network structure construction method, including the following steps:
[0063] S1. Perform preliminary processing on the image to be detected. The specific steps are as follows:
[0064] S11. Convert the low-resolution image to be processed into a format to obtain a low-resolution YCbCr (Y represents the luminance component, Cb represents the blue chrominance component, and Cr represents the red chrominance component) image. Compared with RGB images, YCbCr images only require a very small bandwidth during transmission, so format conversion is performed in the present invention.
[0065] S12, cutting the low-resolution YCbCr image into a number of sub-images with equal spacing in rows and columns, wherein the size of the sub-images after cutting is 480*480 pixels;
[0066] S13, randomly rotating the sub-image by 90° or 180° for data augmentation to provide more data samples, which can significantly reduce the storage space required for feature maps during network transmission;
[0067] S2. Build a lightweight image super-resolution reconstruction network and complete network training. The training loss function adopts L2 loss.
[0068] S3, performing edge sharpening on the image after super-resolution processing;
[0069] S4. Input the edge-sharpened image into the detection module for detection and obtain the traffic small target detection result; the detection module of the present invention uses the YOLOv3 algorithm to divide the image into multiple regions and predict the bounding box and the probability of each region.
[0070] Specifically, if Figure 6As shown, this invention uses Yolov3 (You Only Look Once) as the detection module. While not the most accurate algorithm, it offers a good compromise between accuracy and speed, making it suitable for deployment in real-world applications. The YOLOv3 algorithm uses a single neural network to divide an image into multiple regions and predict bounding boxes and probabilities for each region. It utilizes FPN technology and a multi-stage detection approach, resulting in excellent small object detection capabilities.
[0071] YOLOv3 only uses convolutional layers and uses Darknet-53 as the backbone network. It contains 53 convolutional layers, each followed by a batch normalization layer and a leaky ReLU (linear activation layer). The entire framework can be divided into three parts: the Darknet-53 structure, the feature layer fusion structure (concat part), and the classification detection structure. An image x is input into the Darkenet-53 network structure, and a series of convolutions and staggered networks are performed to obtain feature maps of 1 / 8, 1 / 16, and 1 / 32 of the original image (that is, feature map1, 2, and 3 in the figure). This process is the feature extraction process, in which feature fusion of feature maps of different sizes is performed to obtain stronger feature expression. Due to the different sizes, upsampling operations are required in the middle to make the feature maps of the same size, and then stacking, fusion, and corresponding convolution operations are performed. Finally, we get a 255-dimensional feature map, which is then followed by a 3*3 convolution operation and a 1*1 convolution operation to obtain a 75-dimensional feature vector, which contains the target category and position information corresponding to the original image. The detailed structure is 3*(4+1+20), which is expressed as the information of 3 prediction boxes. Each prediction box has 25 parameters, consisting of 4 position coordinate information, 1 category confidence and 20 category prediction values.
[0072] The present invention effectively expands the resolution of small targets and increases the amount of feature information by setting up a lightweight image super-resolution reconstruction network structure. Compared with traditional methods based on target detection algorithm optimization, the improved accuracy is higher and more effective.
[0073] This paper replaces the GAN-based super-resolution network with a lightweight image super-resolution reconstruction network, which greatly reduces the number of parameters of the network model while improving the detection accuracy of small target objects, making it easier to train and deploy.
[0074] The present invention uses YOLOv3 as the detection module of the system, ensuring a good compromise between detection accuracy and detection speed.
[0075] In the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.
[0076] The above is an exemplary description of the present invention. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. As long as various improvements are made using the method concept and technical solution of the present invention, or they are directly applied to other occasions without improvement, they are all within the scope of protection of the present invention.
Claims
1. A lightweight image super-resolution reconstruction network structure construction method, characterized in that: It consists of four parts: shallow feature extraction module, deep feature extraction module, attention module and reconstruction module; The shallow feature extraction module maps the input image to a high-dimensional feature space and includes a convolution layer with a convolution kernel size of 3*3. The process is expressed as: ; The deep feature extraction module consists of several Vast-receptive-field Information distillation Blocks (VIDBs). It performs deep feature extraction on X0 and gradually refines the extracted features by stacking VIDBs. The process is expressed as follows: ; The attention module consists of two parts: the Efficient Spatial Attention (ESA) module and the Contrast-based Channel Attention (CCA) module. The reconstruction module uses the Pixelshuffle algorithm to complete the reconstruction, and the shape is ( ) is reconstructed into a tensor of shape ( ) tensor; The VIDB block first performs a convolution operation with a convolution kernel size of 1*1 on the input image, and then divides it into two branches, namely the first branch and the second branch. The processing results of the first branch and the second branch are added and output, and pixel normalization is performed; the first branch is a direct connection path, and the second branch is first activated by an activation function based on a gate function, and then the feature weights are assigned by a channel attention module based on information distillation and large convolution kernel depth separation convolution operation, and then a convolution layer with a convolution kernel size of 1*1 is used to perform feature fusion between feature maps, and after fusion, it is added to the direct connection path; the activation function based on the gate function first adds the size of C*H*W The input feature map is divided into two feature maps of size C / 2*W*H according to the number of channels, and then they are multiplied and output; the channel attention module divides the activated feature map into two branches, namely the third branch and the fourth branch. The processing results of the third branch and the fourth branch are added and output, and a convolution operation with a convolution kernel size of 1*1 is performed; the third branch first performs a convolution operation with a convolution kernel size of 1*1, and then performs a depth convolution operation with a convolution kernel size of 9*9, a stride of 1, and a padding of 4; the fourth branch first performs a convolution operation with a convolution kernel size of 1*1, and then activates it through the GELU activation function; The CCA module performs contrast loss calculation and adaptive global pooling on the input image, and then adds the two images together. The output of the addition process is sequentially passed through a convolution operation with a convolution kernel size of 1*1, activation with a ReLU activation function, and a convolution operation with a convolution kernel size of 1*1. Finally, it is multiplied with the input image to obtain the output image, completing feature learning based on position information. The ESA module first performs a convolution operation with a convolution kernel size of 1*1 on the input image, and then divides it into two branches, namely the fifth branch and the sixth branch. The processing results of the fifth branch and the sixth branch are added together, and the feature map after addition is first subjected to a convolution operation with a convolution kernel size of 1*1 to fuse the features and restore the number of channels. It is then activated by the sigmoid linear activation function, and then multiplied with the input image to obtain the output image, learning the cross-channel interaction relationship without reducing the dimension; the fifth branch performs a convolution operation with a convolution kernel size of 1*1; the sixth branch performs convolution operations with a convolution kernel size of 3*3, a stride of 2, and a padding of 1; a maximum pooling layer with a kernel size of 7*7 and a stride of 7; a depthwise convolution operation with a convolution kernel size of 3*3, activation with the GELU activation function, and bilinear interpolation processing to scientifically restore the original image size.
2. A method for applying a small target detection algorithm in traffic monitoring, which is based on the lightweight image super-resolution reconstruction network structure construction method according to claim 1, characterized in that: The following steps are involved: S1. Perform preliminary processing on the image to be detected. The specific steps are as follows: S11, converting the format of the low-resolution image to be processed to obtain a low-resolution YCbCr image; S12, cutting the low-resolution YCbCr image into a number of sub-images with equal spacing in rows and columns, wherein the size of the sub-images after cutting is 480*480 pixels; S13, randomly rotating the sub-image by 90° or 180° for data augmentation to provide more data samples and reduce the storage space required for feature maps during network transmission; S2. Build a lightweight image super-resolution reconstruction network and complete network training. The training loss function adopts L2 loss. S3, performing edge sharpening on the image after super-resolution processing; S4. Input the edge-sharpened image into the detection module for detection and obtain the small traffic target detection result.
3. The method for applying a small target detection algorithm in traffic monitoring according to claim 2, characterized in that: The detection module uses the YOLOv3 algorithm to divide the image into multiple regions and predict the bounding box and the probability of each region.