Lightweight truck detection method and system based on improved YOLO model

By introducing GhostConv, C3Ghost modules, MCFP, and SRFEM into the YOLOv5s network, the truck detection model was optimized, solving the problems of insufficient detection accuracy and speed, and realizing efficient multi-scale target recognition and real-time detection.

CN118918545AActive Publication Date: 2024-11-08NANJING UNIV OF POSTS & TELECOMM

Patent Information

Application Number
CN202411036278.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-08
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

Existing YOLO models suffer from low detection accuracy and speed in truck detection, especially under multi-scale targets and complex scenes, and have high computational resource consumption, making it difficult to meet real-time detection requirements.

Method used

The BackBone convolution operation of the YOLOv5s network is replaced with GhostConv. The C3Ghost module and the multi-scale feature context focusing module MCFP are designed and combined with the multi-scale adaptive receptive field enhancement module SRFEM to optimize the network structure and improve detection accuracy and speed.

Benefits of technology

It improves the accuracy and stability of truck detection, reduces computational load, enables real-time detection, and adapts to the identification needs of targets of different scales.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918545A_ABST
    Figure CN118918545A_ABST
Patent Text Reader

Abstract

The invention discloses a lightweight truck detection method and system based on an improved YOLO model. The method comprises the following steps: collecting images of various trucks at different shooting angles to construct a data set and marking the data set; the Ghost is applied to the YOLOv5s backbone network in a convolution mode; a GhostC3 module is constructed to be used for a YOLOv5s trunk feature extraction network; constructing a multi-scale context feature focusing pyramid module MCFP by using target scale diversity characteristics; constructing a multi-scale adaptive receptive field enhancement module SRFEM by using the low-resolution characteristic of the target; and finally, inputting a self-built truck data set into the improved network to obtain a truck detection result. According to the invention, the truck target can be detected in real time with high precision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of target detection and recognition of computer vision, and specifically relates to a lightweight truck detection method based on an improved YOLO model. Background Art

[0002] With the accelerated development of urbanization, the demand for urban logistics is growing. Trucks are an important part of the logistics system. As heavy vehicles, the passage of trucks will aggravate traffic congestion, especially during peak hours, and even cause traffic accidents. In order to reduce the incidence of traffic accidents, it is stipulated that trucks are prohibited from passing at fixed times. However, some truck drivers take advantage of the fact that the images of trucks recorded by surveillance cameras are small and the license plates cannot be recognized, so they adjust their routes and avoid surveillance capture. Therefore, large vehicle target detection is a very meaningful and challenging technology.

[0003] In recent years, with the improvement of computing power and the optimization of algorithms, deep learning has been used more and more widely in vehicle detection. Deep learning target detectors are mainly divided into two-stage target detectors and single-stage target detectors. Foreign scholars proposed the R-CNN algorithm, which was also the first time that deep learning was applied to target detection tasks. Subsequently, improved algorithms such as Faster R-CNN and ContextR-CNN were proposed for traffic vehicle detection. This two-stage method includes two tasks: generating region candidates and locating targets, so that a higher model detection accuracy is obtained, but the reasoning speed is sacrificed. The single-stage method only needs one step to achieve target positioning and classification, so it is the preferred solution for real-time applications such as traffic scenes. For example, the famous YOLO (You Only Look Once) series of algorithms and SSD (Single Shot Multibox Detector) algorithm. The high accuracy of the YOLO algorithm makes it widely used in vehicle detection tasks.

[0004] Although the YOLO series of algorithms are excellent in efficiency, they still have some limitations or problems, such as accuracy issues in certain specific scenarios, detection capabilities for small targets, algorithm complexity, generalization capabilities, etc. Due to the influence of various factors such as the camera shooting angle and environment in traffic scenes, the following problems exist:

[0005] (1) Since the surveillance camera is fixed and the vehicle is in continuous motion, the scale of the same vehicle target varies;

[0006] (2) When the vehicle is far away from the surveillance camera, the image is small and contains less feature information, so the detection accuracy may be low;

[0007] (3) The different installation positions of surveillance cameras will also cause the same vehicle target to have different shapes and features.

[0008] In summary, a truck detection algorithm that can fully integrate multi-scale features and is lightweight enough to be deployed on a camera is of great significance. Summary of the invention

[0009] Technical problem to be solved by the present invention: In order to overcome the shortcomings of the prior art, the detection accuracy and speed of trucks in surveillance images are low, and the application in embedded software is limited. The present invention proposes a lightweight truck detection method based on an improved YOLO model. The method aims to improve the recognition accuracy of multi-scale truck targets, improve the detection speed, and apply it to complex scenarios.

[0010] In order to solve the above technical problems, the technical solutions proposed by the present invention are as follows:

[0011] A lightweight truck detection method based on an improved YOLO model includes the following steps:

[0012] S1. Build a multi-category truck dataset with annotations;

[0013] S2. Replace all convolution operations in BackBone in the YOLOv5s network with GhostConv.

[0014] S3. Design the C3Ghost module to be applied to the backbone feature extraction network of YOLOv5s. Specifically, design the GhostBottleNeck module, which consists of three layers: first, a Ghost module is used as an extension layer to increase the number of input feature map channels through linear transformation; second, a depthwise separable convolution with a stride of 2 is used for downsampling, and finally a GhostModule is used to reduce the number of output channels to match the number of channels of the original input map, and the two are added element by element; the designed GhostBottleNeck module is used to replace the BottleNeck module of the C3 module in the YOLOv5s network to generate the C3Ghost module;

[0015] S4. Construct a multi-scale feature context focusing module MCFP for the diverse target scales. Input the improved YOLOv5s backbone feature extraction network in step S3 into the MCFP module. Construct MCFP specifically as follows: Use the CA attention mechanism to fuse the current feature layer with its adjacent feature layer, and perform weighted multiplication on the original feature map to obtain an output feature map with attention weights in the width and height directions. Then, upsample the high-level semantic features of the output feature map through a top-down path and fuse them with the low-level features. Then, from the bottom up, help the model capture the transition from low-level to high-level detailed information to abstract semantics. For feature maps of the same size in the feature fusion process, use jump connections to directly establish connections.

[0016] S5. Construct a multi-scale adaptive receptive field enhancement module SRFEM for low-resolution targets, and send the SRFEM module to the model detection head to form an improved truck detection model. Construct SRFEM specifically by: using dilated convolutions with different sampling rates to extract feature information in parallel, and then using softmax to calculate weights to assign different weights to each group of features, so that the module can adaptively select different receptive fields according to the scale of the target, and add the output feature maps with different receptive fields in layers, and finally splice each output feature in the channel dimension, and then add it to the original feature map by pixel to obtain the final output feature map.

[0017] S6. Input the data set into the improved truck detection model to detect the trucks in the monitoring image.

[0018] The specific steps of step S1 are as follows:

[0019] Step S11, collecting images of medium and large trucks and tank trucks based on different shooting angles;

[0020] Step S12: Use Labelme to label the image to obtain a label;

[0021] Step S13: randomly divide the images into a training set, a test set, and a validation set in a ratio of 7:2:1 to form a truck-specific dataset.

[0022] The specific steps of step S2 are: replacing the backbone network convolution of YOLOv5s with Ghost convolution, reducing network parameters and calculation amount while ensuring network accuracy, thereby improving calculation speed and reducing delay.

[0023] The step S3 is specifically as follows:

[0024] Step S31, first generate some inherent feature maps through a common convolution operation, assuming that the input feature map is n-dimensional, the number of maps is s, the output feature map is m-dimensional, and the convolution kernel size is k, the feature map Y' can be obtained h'×w'×m .

[0025] Step S32: Apply simple linear operations to Y' to obtain more feature maps, as shown in the formula:

[0026]

[0027] Among them, y' i represents the i-th channel feature map of Y'; Φ ij Represents the jth linear operation used to generate the i-th Ghost feature map.

[0028] Step S33, output the Ghost module as the concatenation result of the common feature map obtained in step 1 and the Ghost feature map obtained in step S32.

[0029] Step S34, for the two Ghost modules before and after, respectively perform channel dimension increase and dimension reduction operations to keep the number of output channels consistent with the number of output channels, obtain the output feature map GhostBottleNeck, and apply it to the original network C3 module to reduce the number of model parameters and calculations.

[0030] The step S4 is specifically as follows:

[0031] In step S41, {P1, P2, P3, P4} is the output of the backbone network, and P5 is the upsampling result of P4. {P1, P2, P3}, {P2, P3, P4}, and {P3, P4, P5} are respectively sent to the CA attention mechanism as input feature groups to fuse the multi-scale features of adjacent layers and capture long-range dependencies, thereby improving the model's ability to transmit hierarchical semantic information and recognize objects of different scales.

[0032] Step S42, CA attention mechanism for the input feature map F∈R C×H×W Global average pooling is performed in the horizontal and vertical directions respectively to obtain feature maps in the width and height directions, and to establish long-range dependencies of features.

[0033] Step S43: The pooled feature maps are concatenated in the channel dimension, and convolution is used to fuse features and reduce the dimension. The feature map F1 processed by batch normalization is sent to the Sigmoid activation function to obtain the feature map f.

[0034] Step S44: Split the feature map f by height and width and perform 1×1 convolution to increase the dimension of the feature map f h and f w After the Sigmoid activation function, the feature maps are obtained at high attention weights g h and the width attention weight g w .

[0035] Step S45, finally, by multiplying the original feature map with weights, an output feature map with attention weights in the width and height directions is obtained. The formula is as follows:

[0036]

[0037] Among them, F' c (i, j) represents the feature response of pixel (i, j) in the feature map F at channel c; Represents the value of the i-th pixel in the horizontal direction of the feature at the c channel; Represents the value of the i-th pixel in the vertical direction of the feature at the c channel.

[0038] Step S46, {P1, P2, P3}, {P2, P3, P4} and {P3, P4, P5} and the three sets of feature maps output by the attention mechanism are upsampled and fused with low-level features through a top-down path to improve the position accuracy of the low-resolution feature maps, and then help the model capture the transition from low-level to high-level detailed information to abstract semantics from bottom to top. For feature maps of the same size in the feature fusion process, jump connections are used to directly establish connections.

[0039] The step S5 is specifically as follows:

[0040] Step S51: For the three sets of feature maps outputted in step S4, dilated convolution combinations with corresponding dilation rates are set, i.e., R1=(1,2), R2=(1,2,5), and R3=(1,2,5,8), and feature information is extracted in parallel using dilated convolutions with different sampling rates;

[0041] Step S52, reallocate the channel weights of the feature branches through the softmax function, so that the network dynamically selects the receptive field according to the target size, so that for targets with larger sizes, larger weights are allocated to branches with larger expansion rates, and for targets with smaller sizes, larger weights are allocated to branches with smaller expansion rates. Taking R1=(1,2 as an example, the specific formula is as follows:

[0042]

[0043]

[0044] Among them, a c represents the weight of the cth element in vector a; b c represents the weight of the cth element in vector b;

[0045]

[0046]

[0047] a′+b′=1

[0048] Step S53, add the feature branches after recalibrating the channel weights pixel by pixel to compensate for the grid effect caused by the hole convolution, and then splice the added feature map with the original feature map. The formula is as follows:

[0049] F3=Cat(F,aF1+bF1)

[0050] Step S54, adjusting the number of output feature channels through 1×1 convolution;

[0051] The above are the processing steps for one set of feature maps. For the remaining two sets of feature output maps, the weights of the corresponding channels are set respectively, and the processing steps are the same as described above.

[0052] The specific steps of step S6 are as follows:

[0053] The hardware configuration of the experimental environment of the present invention is Nvidia GeForce RTX 3050 graphics card, and the software system is Windows 10 and PyTorch 1.2 deep learning framework. The IOU threshold is set to 0.5, the input image is 640×640, the initial learning rate is 0.01, the batch sample size is 16, and the number of training rounds is 300. The evaluation indicators of the detection algorithm include the average accuracy AP (Average Precision) of all categories and the video frame processing rate FPS (Frames Per Second).

[0054] The present invention also proposes a lightweight truck detection system based on an improved YOLO model, comprising:

[0055] A dataset construction unit, used to construct a multi-category truck-specific dataset with annotations;

[0056] The truck detection model building unit is configured to perform the following actions:

[0057] (1) Replace all convolution operations in BackBone in the YOLOv5s network with GhostConv;

[0058] (2) Design the C3Ghost module to be applied to the backbone feature extraction network of YOLOv5s; specifically: design the GhostBottleNeck module, which consists of three layers: first, a Ghost module is used as an extension layer to increase the number of input feature map channels through linear transformation; second, a depthwise separable convolution with a stride of 2 is used for downsampling; and finally, a GhostModule is used to reduce the number of output channels to match the number of channels of the original input map, and the two are added element by element; the designed GhostBottleNeck module replaces the BottleNeck module of the C3 module in the YOLOv5s network to generate the C3Ghost module;

[0059] (3) Construct a multi-scale feature context focusing module MCFP for the diverse target scales. Input the improved YOLOv5s backbone feature extraction network in step (2) into the MCFP module. Construct MCFP specifically as follows: Use the CA attention mechanism to fuse the current feature layer with its adjacent feature layer, and perform weighted multiplication on the original feature map to obtain an output feature map with attention weights in the width and height directions. Then, use a top-down path to upsample the high-level semantic features of the output feature map and fuse them with the low-level features. Then, use a bottom-up path to help the model capture the transition from low-level to high-level detailed information to abstract semantics. For feature maps of the same size in the feature fusion process, use jump connections to directly establish connections.

[0060] (4) A multi-scale adaptive receptive field enhancement module SRFEM is constructed for the low-resolution target, and the SRFEM module is sent to the model detection head to form an improved truck detection model. The construction of SRFEM is specifically as follows: feature information is extracted in parallel using dilated convolutions with different sampling rates, and then different weights are assigned to each group of features using softmax calculation weights, so that the module can adaptively select different receptive fields according to the scale of the target, and the output feature maps with different receptive fields are layered and added. Finally, the output features are spliced ​​in the channel dimension and then added to the original feature map by pixel to obtain the final output feature map.

[0061] The detection unit is used to input the data set into the improved truck detection model to detect the trucks in the monitoring image.

[0062] Furthermore, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the lightweight truck detection method based on the improved YOLO model described above when executing the computer program.

[0063] Furthermore, the present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the lightweight truck detection method based on the improved YOLO model described above are executed.

[0064] Finally, the present invention also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the lightweight truck detection method based on the improved YOLO model described in the present invention.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] (1) To address the problem that the surveillance camera is fixed and the vehicle is in continuous motion, resulting in the variable scale of the same vehicle target, the present invention introduces a multi-scale context feature focusing pyramid module. By establishing contextual interactions between multi-scale features at adjacent levels, the network can more effectively transmit semantic information, and then sample the correlation between features at different levels, thereby improving the generalization and accuracy of the network.

[0067] (2) To address the problem that when the vehicle is far away from the surveillance camera, the image is small, contains less feature information, and the detection accuracy may be low, the present invention introduces a multi-scale adaptive receptive field enhancement module, which extracts features by enhancing the receptive field through a combination of different dilation rates and assigning different weight values ​​to each group of features, so that the network can dynamically select different receptive fields according to the different target scales, thereby improving the recognition accuracy of targets of different scales.

[0068] (3) In order to address the problem that most target detection models occupy high computing resources, have low real-time frame rates, and have poor real-time performance, and cannot meet the needs of actual scenarios, the present invention introduces the Ghost module to reduce the computational complexity of the network, improve the detection speed, and achieve real-time detection while maintaining the detection accuracy of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 The figure is a flow chart of the lightweight truck detection method based on the improved YOLO model.

[0070] Figure 2 This is a schematic diagram of the improved network structure.

[0071] Figure 3 This is a schematic diagram of the C3Ghost module structure.

[0072] Figure 4 Schematic diagram of the MCFP multi-scale context feature focusing pyramid module structure.

[0073] Figure 5 Schematic diagram of the SRFEM multi-scale contextual feature focusing pyramid module structure.

[0074] Figure 6 This is a comparison chart of the detection performance of the present invention and other methods under the same environment. DETAILED DESCRIPTION

[0075] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0076] Embodiment 1: The lightweight truck detection method based on the improved YOLO model of this embodiment is as follows Figure 1 As shown, the method comprises the following steps:

[0077] S1. Build a multi-category truck dataset with annotations. The specific steps are as follows:

[0078] Step S11, collecting images of medium and large trucks and tank trucks based on different shooting angles;

[0079] Step S12: Use Labelme to label the image to obtain a label;

[0080] Step S13: randomly divide the images into a training set, a test set, and a validation set in a ratio of 7:2:1 to form a truck-specific dataset.

[0081] S2. Apply Ghost convolution to the YOLOv5s backbone network; the specific steps are:

[0082] All the ordinary convolution operations in BackBone in the original YOLOv5s network are replaced with GhostConv, which reduces network parameters and calculation amount while ensuring network accuracy, thereby improving calculation speed and reducing latency. The overall network structure is shown in the figure below: Figure 2 shown.

[0083] S3. Design the C3Ghost module for the YOLOv5s backbone network; specifically:

[0084] Step S31, first generate some inherent feature maps through a common convolution operation, assuming that the input feature map is n-dimensional, the height is h, the number of maps is s, the output feature map is m-dimensional, the convolution kernel size is k, and the feature map Y' can be obtained h'×w'×m .

[0085] Step S32: Apply simple linear operations to Y' to obtain more feature maps, as shown in formula (1):

[0086]

[0087] Among them, y' i represents the i-th channel feature map of Y'; Φ ij Represents the jth linear operation used to generate the i-th Ghost feature map.

[0088] Step S33, output the Ghost module as the splicing result of the common feature map obtained in step S31 and the Ghost feature map obtained in step S32.

[0089] Step S34: Output feature map is GhostBottleNeck, whose structure is as follows: Figure 3 As shown in the figure, there are two Ghost modules before and after to perform dimension increase and dimension reduction operations of the channel respectively, so that the number of output channels is consistent with the number of output channels.

[0090] Step S35, finally applying GhostBottleNeck to the original network C3 module to reduce the model parameters and calculation amount;

[0091] S4. Construct a multi-scale feature context focusing module MCFP for the diverse target scales; specifically:

[0092] Step S41: Assume {P1, P2, P3, P4} as the output of the backbone network, and P5 is the upsampling result of P4. Send {P1, P2, P3}, {P2, P3, P4} and {P3, P4, P5} as input feature groups to the CA attention mechanism, fuse the multi-scale features of adjacent layers, and capture long-range dependencies, so as to improve the model's ability to transmit hierarchical semantic information and recognize objects of different scales.

[0093] Step S42: Taking {P2, P3, P4} as an example, the multi-scale context feature interaction pyramid module proposed in the present invention is as follows: Figure 4 As shown. The CA attention mechanism is used for the input feature map F∈R C×H×W Perform global average pooling in the horizontal and vertical directions respectively, obtain feature maps in the width and height directions respectively, and establish the long-range dependency of features. The specific formulas are as follows:

[0094]

[0095]

[0096] in, Represents the value of feature z at h pixel points (c,h) in the horizontal direction, Represents the value of feature w at pixel point (c,w) in the vertical direction.

[0097] Step S43, the pooled feature maps are concatenated in the channel dimension, and convolution is used to fuse features and reduce the dimension. The feature map F1 processed by batch normalization is sent to the Sigmoid activation function to obtain the feature map f, and the calculation is shown in formula (4).

[0098]

[0099] Step S44: Split the feature map f by height and width and perform 1×1 convolution to increase the dimension of the feature map f h and f w After the Sigmoid activation function, the feature maps are obtained at high attention weights g h and the width attention weight g w , the calculation formulas are shown in equations (5) and (6).

[0100] g h=σ(F h (f h )) (5)

[0101] g w =σ(F w (f w )) (6)

[0102] Step S45. Finally, the original feature map is weighted by multiplication to obtain an output feature map with attention weights in the width and height directions. The calculation is shown in formula (7):

[0103]

[0104] Among them, F' c (i, j) represents the feature response of pixel (i, j) in the feature map F at channel c; Represents the value of the i-th pixel in the horizontal direction of the feature at the c channel; Represents the value of the jth pixel in the vertical direction of the feature at the c channel.

[0105] Step S46: The feature map output by the attention mechanism is upsampled through a top-down path to fuse the high-level semantic features with the low-level features to improve the position accuracy of the low-resolution feature map. Then, the model is helped to capture the transition from low-level to high-level detailed information to abstract semantics from bottom to top. For feature maps of the same size in the feature fusion process, jump connections are used to directly establish connections, so that multi-layer features can complement each other and enhance the feature integration capability of the model.

[0106] S5. A multi-scale adaptive receptive field enhancement module SRFEM is proposed for low-resolution targets; specifically:

[0107] Step S51: For the feature graphs of the three branches outputted in step S4, set the dilated convolution combinations with corresponding dilation rates, i.e., R1 = (1, 2), R2 = (1, 2, 5), and R3 = (1, 2, 5, 8). Use dilated convolutions with different sampling rates to extract feature information in parallel. Taking R1 = (1, 2) as an example, dilated convolutions with kernel size 3 and dilation rates 1 and 2 are used to extract features F∈R in parallel. C ×H×W , and the number of channels is reduced to 1 / 2 of the original. Then, the average pooling is used to model the channel information of the two sets of features, and the weight vectors for are calculated as shown in formulas (8) and (9):

[0108]

[0109]

[0110] Among them, a c represents the weight of the cth element in vector a; bc Represents the weight of the cth element in vector b.

[0111] Step S52, reallocate the channel weights of the feature branches through the softmax function, so that the network dynamically selects the receptive field according to the target size, so that for targets with larger sizes, larger weights are allocated to branches with larger expansion rates, and for targets with smaller sizes, larger weights are allocated to branches with smaller expansion rates. The specific formulas are shown in formulas (10) and (11):

[0112]

[0113]

[0114] a+b=1.

[0115] Step S53, add the feature branches after recalibrating the channel weights pixel by pixel to compensate for the grid effect caused by the hole convolution. Then, concatenate the added feature map with the original feature map, as shown in formula (12):

[0116] F3=Cat(F,a'F1+b'F1) (12)

[0117] Step S54, adjust the number of output feature channels through 1×1 convolution. The multi-branch case is analogous to the above description, and the network structure is as follows Figure 5 As shown, 3 weights and 4 weights are taken respectively.

[0118] S6. The data set is input into the improved truck detection model to detect trucks in the surveillance image. The specific steps are as follows:

[0119] The image data set obtained by S1 is input into the network of the present invention, and prediction is performed through the features learned through training, and the position and category of the low-resolution small target in the image are output. The evaluation indicators of the detection algorithm include the average accuracy AP (Average Precision) of all categories and the video frame processing rate FPS (Frames Per Second). The specific contents are:

[0120] In this example, in order to verify the effectiveness of each module of the solution of the present invention, YOLOv5s is used as the baseline network, and an ablation experiment is performed on each key improved module.

[0121] The experimental results are shown in the following table:

[0122] SRFEM MCFP GhostModule FPS AP 26 76.73 √ 24 77.81 √ 22 79.37 √ √ 21 80.15 √ √ √ 28 80.06

[0123] In order to verify the effectiveness of the algorithm proposed in this paper in traffic scenes, this paper selects a real traffic monitoring video and compares the detection performance of the baseline network YOLOv5s and the improved algorithm in this paper at the same time of the video frame. The detection results are as follows: Figure 6 As shown. Figure 6 In (b) t = 6, it can be seen that the overlapping of the passing vehicles leads to missed detection. Figure 6 (c) in the figure shows that the algorithm of the present invention achieves accurate detection. Figure 6 In (b) t = 24, the vehicle target in the distance is small, which causes false detection. Figure 6 In (c), the target is correctly identified. In summary, the network proposed in the present invention effectively improves the problem of false detection and missed detection of the target detector, and improves the accuracy and stability of detection.

[0124] As a second embodiment of the present invention, this embodiment proposes a lightweight truck detection system based on an improved YOLO model, including:

[0125] A dataset construction unit, used to construct a multi-category truck-specific dataset with annotations;

[0126] The truck detection model building unit is configured to perform the following actions:

[0127] (1) Replace all convolution operations in BackBone in the YOLOv5s network with GhostConv;

[0128] (2) Design the C3Ghost module to be applied to the backbone feature extraction network of YOLOv5s; specifically: design the GhostBottleNeck module, which consists of three layers: first, a Ghost module is used as an extension layer to increase the number of input feature map channels through linear transformation; second, a depthwise separable convolution with a stride of 2 is used for downsampling; and finally, a GhostModule is used to reduce the number of output channels to match the number of channels of the original input map, and the two are added element by element; the designed GhostBottleNeck module replaces the BottleNeck module of the C3 module in the YOLOv5s network to generate the C3Ghost module;

[0129] (3) Construct a multi-scale feature context focusing module MCFP for the diverse target scales. Input the improved YOLOv5s backbone feature extraction network in step (2) into the MCFP module. Construct MCFP specifically as follows: Use the CA attention mechanism to fuse the current feature layer with its adjacent feature layer, and perform weighted multiplication on the original feature map to obtain an output feature map with attention weights in the width and height directions. Then, use a top-down path to upsample the high-level semantic features of the output feature map and fuse them with the low-level features. Then, use a bottom-up path to help the model capture the transition from low-level to high-level detailed information to abstract semantics. For feature maps of the same size in the feature fusion process, use jump connections to directly establish connections.

[0130] (4) A multi-scale adaptive receptive field enhancement module SRFEM is constructed for the low-resolution target, and the SRFEM module is sent to the model detection head to form an improved truck detection model. The construction of SRFEM is specifically as follows: feature information is extracted in parallel using dilated convolutions with different sampling rates, and then different weights are assigned to each group of features using softmax calculation weights, so that the module can adaptively select different receptive fields according to the scale of the target, and the output feature maps with different receptive fields are layered and added. Finally, the output features are spliced ​​in the channel dimension and then added to the original feature map by pixel to obtain the final output feature map.

[0131] The detection unit is used to input the data set into the improved truck detection model to detect the trucks in the monitoring image.

[0132] For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0133] Embodiment 3: The embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. It should be noted that when the processor executes the computer program, it corresponds to the specific steps of the method provided in the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0134] Embodiment 4: The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. It should be noted that when the computer program is executed by the processor, it corresponds to the specific steps of the method provided in the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided in the embodiment of the present invention.

[0135] Embodiment 5: Finally, the present invention also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the low-resolution small target detection method based on the improved YOLO model described in the present invention.

[0136] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A lightweight truck detection method based on an improved YOLO model, characterized in that: include: S1. Build a multi-category truck dataset with annotations; S2. Replace all convolution operations in BackBone in the YOLOv5s network with GhostConv. S3. Design the C3Ghost module to be applied to the backbone feature extraction network of YOLOv5s. Specifically, design the GhostBottleNeck module, which consists of three layers: first, a Ghost module is used as an extension layer to increase the number of input feature map channels through linear transformation; second, a depthwise separable convolution with a stride of 2 is used for downsampling, and finally a GhostModule is used to reduce the number of output channels to match the number of channels of the original input map, and the two are added element by element; the designed GhostBottleNeck module is used to replace the BottleNeck module of the C3 module in the YOLOv5s network to generate the C3Ghost module; S4. Construct a multi-scale feature context focusing module MCFP for the diverse target scales. Input the improved YOLOv5s backbone feature extraction network in step S3 into the MCFP module. Construct MCFP specifically as follows: Use the CA attention mechanism to fuse the current feature layer with its adjacent feature layer, and perform weighted multiplication on the original feature map to obtain an output feature map with attention weights in the width and height directions. Then, upsample the high-level semantic features of the output feature map through a top-down path and fuse them with the low-level features. Then, from the bottom up, help the model capture the transition from low-level to high-level detailed information to abstract semantics. For feature maps of the same size in the feature fusion process, use jump connections to directly establish connections. S5. Construct a multi-scale adaptive receptive field enhancement module SRFEM for low-resolution targets, and send the SRFEM module to the model detection head to form an improved truck detection model. Construct SRFEM specifically by: using dilated convolutions with different sampling rates to extract feature information in parallel, and then using softmax to calculate weights to assign different weights to each group of features, so that the module can adaptively select different receptive fields according to the scale of the target, and add the output feature maps with different receptive fields in layers, and finally splice each output feature in the channel dimension, and then add it to the original feature map by pixel to obtain the final output feature map. S6. Input the data set into the improved truck detection model to detect the trucks in the monitoring image.

2. The lightweight truck detection method based on the improved YOLO model according to claim 1 is characterized in that: The specific steps of step S1 are as follows: Step S11, collecting images of medium and large trucks and tank trucks based on different shooting angles; Step S12: Use Labelme to label the image to obtain a label; Step S13: randomly divide the images into a training set, a test set, and a validation set in a ratio of 7:2:1 to form a truck-specific dataset.

3. The lightweight truck detection method based on the improved YOLO model according to claim 1 is characterized in that: The specific steps of step S3 are as follows: Step S31, first generate some inherent feature maps through a convolution operation, assuming that the input feature map is n-dimensional, the number of maps is s, the output feature map is m-dimensional, the convolution kernel size is k, and the feature map Y' is obtained h'×w'×m ; Step S32: for feature graph Y' h'×w'×m Apply linear operations to obtain more feature maps, as shown in the formula: Among them, y' i represents the i-th channel feature map of Y'; Φ ij Represents the jth linear operation used to generate the i-th Ghost feature map; Step S33, outputting the Ghost module as the splicing result of the feature map obtained in step S31 and the Ghost feature map obtained in step S32; Step S34: For the two Ghost modules before and after, perform channel dimension increase and dimension reduction operations respectively, so that the number of output channels of the previous module is consistent with the number of input channels of the latter module, and obtain the output feature map GhostBottleNeck, which is applied to the original network C3 module.

4. The lightweight truck detection method based on the improved YOLO model according to claim 1 is characterized in that: Step S4 is specifically as follows: Step S41, assuming {P1, P2, P3, P4} as the output of the backbone network, P5 is the upsampling result of P4, and PP1, P2, P3}, {P2, P3, P4} and {P3, P4, P5} are respectively sent to the CA attention mechanism as input feature groups to fuse the multi-scale features of adjacent layers and capture long-range dependencies; Step S42: Use CA attention mechanism to input feature map F∈R C×H×W Perform global average pooling in the horizontal and vertical directions respectively to obtain feature maps in the width and height directions, and establish long-range dependencies of features; Step S43, concatenate the pooled feature maps in the channel dimension, use convolution to fuse features and reduce the dimension, and send the batch normalized feature map F1 to the Sigmoid activation function to obtain the feature map f; Step S44: Split the feature map f by height and width and perform 1×1 convolution to obtain feature maps f h and f w , and then after the Sigmoid activation function, the feature maps f are obtained respectively h and f w At the height attention weight g h and the width attention weight g w ; Step S45: multiply and weight the original feature map to obtain an output feature map with attention weights in the width and height directions. The formula is as follows: Among them, F' c (i, j) represents the feature response of pixel (i, j) in the feature map F at channel c; Represents the value of the i-th pixel in the horizontal direction of the feature at the c channel; Represents the value of the i-th pixel in the vertical direction of the feature at the c channel; Step S46, the three sets of feature maps {P1, P2, P3}, {P2, P3, P4} and {P3, P4, P5} respectively output by the attention mechanism are upsampled through a top-down path to high-level semantic features and fused with low-level features, so as to improve the position accuracy of the low-resolution feature maps, and then help the model capture the transition from low-level to high-level detailed information to abstract semantics from bottom to top. For feature maps of the same size in the feature fusion process, jump connections are used to directly establish connections.

5. The lightweight truck detection method based on the improved YOLO model according to claim 3 is characterized in that: Step S5 is specifically as follows: Step S51: For the three sets of feature maps outputted in step S4, dilated convolution combinations with corresponding dilation rates are set, i.e., R1=(1,2), R2=(1,2,5), and R3=(1,2,5,8), and feature information is extracted in parallel using dilated convolutions with different sampling rates; Step S52: reallocate the channel weights of the feature branches through the softmax function, so that the network dynamically selects the receptive field according to the target size, so that for targets with larger sizes, larger weights are allocated to branches with larger expansion rates, and for targets with smaller sizes, larger weights are allocated to branches with smaller expansion rates. R1=(1,2), the specific formula is as follows: Among them, a c represents the weight of the cth element in vector a; b c represents the weight of the cth element in vector b; a′+b′=1 Step S53, add the feature branches after recalibrating the channel weights pixel by pixel to compensate for the grid effect caused by the hole convolution, and then splice the added feature map with the original feature map. The formula is as follows: F′3=Cat(F,a′F1+b′F1) Step S54, adjusting the number of output feature channels through 1×1 convolution; The above are the processing steps for the first set of feature maps. For the remaining two sets of feature output maps, the weights of the corresponding channels are set respectively, and the processing steps are the same as described above.

6. A lightweight truck detection system based on an improved YOLO model, characterized in that: include: A dataset construction unit, used to construct a multi-category truck-specific dataset with annotations; The truck detection model building unit is configured to perform the following actions: (1) Replace all convolution operations in BackBone in the YOLOv5s network with GhostConv; (2) Design the C3Ghost module to be applied to the backbone feature extraction network of YOLOv5s; specifically: design the GhostBottleNeck module, which consists of three layers: first, a Ghost module is used as an extension layer to increase the number of input feature map channels through linear transformation; second, a depthwise separable convolution with a stride of 2 is used for downsampling; and finally, a GhostModule is used to reduce the number of output channels to match the number of channels of the original input map, and the two are added element by element; the designed GhostBottleNeck module replaces the BottleNeck module of the C3 module in the YOLOv5s network to generate the C3Ghost module; (3) Construct a multi-scale feature context focusing module MCFP for the diverse target scales. Input the improved YOLOv5s backbone feature extraction network in step (2) into the MCFP module. Construct MCFP specifically as follows: Use the CA attention mechanism to fuse the current feature layer with its adjacent feature layer, and perform weighted multiplication on the original feature map to obtain an output feature map with attention weights in the width and height directions. Then, use a top-down path to upsample the high-level semantic features of the output feature map and fuse them with the low-level features. Then, use a bottom-up path to help the model capture the transition from low-level to high-level detailed information to abstract semantics. For feature maps of the same size in the feature fusion process, use jump connections to directly establish connections. (4) A multi-scale adaptive receptive field enhancement module SRFEM is constructed for the low-resolution target, and the SRFEM module is sent to the model detection head to form an improved truck detection model. The construction of SRFEM is specifically as follows: feature information is extracted in parallel using dilated convolutions with different sampling rates, and then different weights are assigned to each group of features using softmax calculation weights, so that the module can adaptively select different receptive fields according to the scale of the target, and the output feature maps with different receptive fields are layered and added. Finally, the output features are spliced ​​in the channel dimension and then added to the original feature map by pixel to obtain the final output feature map. The detection unit is used to input the data set into the improved truck detection model to detect the trucks in the monitoring image.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Method and system for detecting artificial small target in SAR (Synthetic Aperture Radar) image

    CN113567984A

  • Lightweight traffic sign detection method, storage medium and system

    CN116453091A

  • YOLO-TGB vehicle detection system and method

    CN116503631A

  • Multi-scale high-precision lightweight target detection method based on large receptive field and attention mechanism

    CN117058542A

  • Light-weighted target detection method and device, and storage medium

    WO2022213395A1

Cited By

  • Fishing boat detection method based on lightweight GSMA-YOLO

    CN121458965A