Lightweight traffic target detection method and device based on LM-YOLO and medium

Abstract

The application discloses a lightweight traffic target detection method and device based on LM-YOLO and a medium, and aims at solving problems that a traditional detection network has poor geometric deformation adaptability, redundant attention mechanism calculation, and a contradiction between lightness and precision in a complex traffic scene. The method introduces a linear deformable convolution (LDConv) module, supports dynamic sampling while reducing the parameter amount through a coordinate generation algorithm and a linear parameter growth mechanism, and effectively captures target deformation features. Based on a mixed local channel attention mechanism (MLCA), a C2f_MLCA module is constructed, a double-channel attention weighting is performed, and the key region focusing capability of a multi-scale target is enhanced. Through hierarchical deployment of the LDConv and MLCA modules, a complementary enhancement mechanism is formed, the detection robustness of vehicles and pedestrians in a complex traffic scene is effectively improved, and the lightweight and high-precision advantages are combined, so that the method is suitable for deployment of a vehicle terminal and an edge computing device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to a lightweight traffic target detection method based on LM-YOLO, equipment and storage medium. BACKGROUND

[0002] The current traffic target detection technology based on deep learning generally uses a fixed structure of convolutional neural network, which has significant limitations in dealing with complex traffic scenes. The standard convolution operation is limited by the fixed sampling grid and square convolution kernel design, and it is difficult to effectively capture the deformation and occlusion features of vehicles and pedestrians. Although deformable convolution improves the deformation adaptation ability by introducing an offset learning mechanism, the number of parameters increases quadratically with the size of the convolution kernel, resulting in a dramatic increase in computing resource consumption, and only supports regular sampling shapes (such as 3x3, 5x5), which cannot flexibly adapt to the irregular contour features of targets in traffic scenes. At the same time, existing attention mechanisms (such as SE, CBAM) mostly use global channel compression or independent spatial weighting strategies, ignoring the relevance of local spatial and channel features, and facing the dual challenges of insufficient feature expression and computational redundancy in real-time traffic monitoring scenarios.

[0003] In terms of lightweight network architecture, the mainstream method reduces the number of parameters through depth separable convolution, but excessive compression of feature dimensions easily causes loss of detailed information of small targets (such as distant pedestrians and traffic signs). For example, although networks such as MobileNetV3 perform well in regular target detection, the SE attention module used by them only performs global channel weighting, which is difficult to capture the edge features of local key areas (such as the occlusion parts of vehicles) in traffic scenes. In addition, existing technologies are insufficiently optimized in the process of multi-scale feature fusion, especially when dealing with high-speed moving targets, which easily causes positioning accuracy to decrease due to feature misplacement. In view of the above problems, it is urgent to develop a new network architecture that takes into account deformation adaptation ability, local feature enhancement and computational efficiency, in order to meet the demand of intelligent transportation systems for high-precision real-time target detection. SUMMARY

[0004] The lightweight traffic target detection method based on LM-YOLO, equipment and storage medium proposed by the present application can at least solve one of the technical problems in the background art, while maintaining low computational overhead, the detection accuracy in complex traffic scenes is improved.

[0005] To achieve the above purpose, the following technical solutions are adopted in the present application:

[0006] A lightweight traffic target detection method based on LM-YOLO, comprising:

[0007] S1, obtaining a public traffic scene dataset and preprocessing it;

[0008] S2, constructing an LM-YOLO network model, the LM-YOLO network taking YOLOv8n as a basic architecture, using an LDConv module to replace a regular convolution to improve the feature extraction capability of the model on complex targets, and constructing a C2f_MLCA module in front of a three-level detection head to strengthen the feature discrimination capability of the model on multi-scale targets;

[0009] S3, configuring a training environment, and using the data set obtained in step S1 to perform LM-YOLO network training and verification analysis;

[0010] S4, inputting a traffic scene image to be detected into the pre-trained LM-YOLO network model for target detection.

[0011] Further, in step S1, the preprocessing of the public data set includes data random extraction, training set and verification set division, and image and label format conversion to adapt to the input of the YOLOv8 model.

[0012] Further, in step S2, the LM-YOLO network only retains the first layer feature extraction using a regular convolution for preliminary feature mapping, and the subsequent down-sampling all uses an LDConv module to replace the regular convolution operation. The LDConv module can break away from the dependence of the network on regular square grids, support any parameter number of convolution kernel sampling shapes, adjust the parameter growth trend from square to linear, and combine dynamic adjustment of the offset to adapt to target changes, so as to reduce the parameter amount while improving the feature extraction capability on complex targets, and provide a lightweight option for hardware resource limited scenarios.

[0013] Further, the LDConv module specifically includes:

[0014] According to the set convolution kernel parameter N, initial sampling shapes and relative coordinates are generated by an initial sampling coordinate generation algorithm;

[0015] According to the resolution and stride of the input feature map, absolute coordinates of the initial sampling points are generated;

[0016] An offset generation module is used to perform convolution operation on the input feature map to generate a dynamic offset of the coordinates;

[0017] The relative coordinates, absolute coordinates and dynamic offset of the initial sampling points are combined to calculate the final coordinates of the sampling points;

[0018] Based on the final coordinates of the sampling points, a bilinear difference method is used to extract feature values of corresponding positions from the input feature map to obtain a resampled feature map after offset;

[0019] The resampled feature map is reshaped so that it can be processed by the convolution kernel of the main convolution module;

[0020] The reshaped feature map is subjected to a convolution operation using a main convolution module to obtain a final output feature map.

[0021] Further, in step S2, the LM-YOLO network introduces an MLCA attention mechanism in a C2f module before a three-level detection head to construct a C2f MLCA module and enhance the feature expression capability of multi-scale traffic targets. The MLCA attention mechanism adopts a double-branch structure, combines local average pooling (LAP) and global average pooling (GAP), models channel, space, local and global information at the same time, and adopts 1D convolution to accelerate, achieve efficient balance of parameters and calculation amount, and make it suitable for lightweight target detection network.

[0022] Further, in step S2, the C2f MLCA module structure specifically includes:

[0023] The input feature map is expanded in channel number through a 1x1 convolution layer;

[0024] The expanded feature map is evenly divided into two parts along the channel dimension;

[0025] One of the feature maps is sequentially subjected to Bottleneck modules, and the output of each module is used as the input of the next module to finally obtain feature maps;

[0026] All the feature maps are spliced along the channel dimension to obtain a spliced feature map;

[0027] The spliced feature map is input into a 1x1 convolution layer to map the channel number to the target output channel number.

[0028] Further, in step S2, the MLCA module specifically includes:

[0029] The input feature map is divided into local regions through LAP, and the average value of each region is calculated to obtain a local feature map;

[0030] In the global branch, the local feature map is subjected to GAP to compress into a global feature map, and the global feature map is input into a 1D convolution operation after reshaping to obtain a global channel attention weight. Finally, the global feature map is expanded to size through uniform neighborhood average pooling (UNAP);

[0031] In the local branch, the local feature map is reshaped, the local channel attention weight is calculated through 1D convolution, and the result is reshaped back to size;

[0032] The local feature map and the global feature map obtained by the two branches are normalized and added, and the input feature map size is adjusted back through UNAP to obtain the final attention weight;

[0033] The obtained attention weight is multiplied element by element with the original input feature map to obtain the final enhanced feature map output.

[0034] In another aspect, the application also discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to make the processor execute the steps of the above method.

[0035] In another aspect, the application also discloses a computer device, which comprises a memory and a processor, and the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the above method.

[0036] From the above technical solution, it can be seen that the lightweight traffic target detection method based on LM-YOLO of the application aims at the problems of poor geometric deformation adaptability, redundant attention mechanism calculation, and conflict between lightweight and precision of the traditional detection network in complex traffic scenes. The method introduces a linear deformable convolution (LDConv) module, which supports dynamic sampling while reducing the parameter amount through a coordinate generation algorithm and a linear parameter growth mechanism, effectively capturing target deformation features. Based on a mixed local channel attention mechanism (MLCA), a C2f MLCA module is constructed, which enhances the key region focusing ability of multi-scale targets through double-channel attention weighting. Through hierarchical deployment of the LDConv and MLCA modules, a complementary enhancement mechanism is formed, which can effectively improve the detection robustness of vehicles and pedestrians in complex traffic scenes, and has the advantages of lightweight and high precision, and is suitable for deployment of vehicle terminals and edge computing devices.

[0037] Specifically, the beneficial effects of the application are as follows:

[0038] 1. The lightweight traffic target detection method based on LM-YOLO of the application introduces linear deformable convolution and mixed local channel attention modules in YOLOv8 algorithm, which has obvious improvement in lightweight, detection precision and hardware adaptability compared with traditional target detection methods.

[0039] 2. The linear deformable convolution of the application can make the convolution kernel sampling points adapt to the target deformation and spatial distribution characteristics through a dynamic offset learning mechanism, effectively improving the feature extraction capability of complex targets such as vehicle deformation and occlusion in traffic scenes; by getting rid of the fixed shape restriction of the traditional convolution kernel, the parameter quantity growth trend is converted from quadratic to linear growth characteristics, while maintaining the hardware computing efficiency, providing flexible feature representation space for traffic target detection of different scales.

[0040] 3. The mixed local channel attention mechanism of the application can overcome the spatial and local information loss problem of the traditional attention mechanism by parallel processing of local and global features; by using a single-dimensional convolution acceleration strategy, the local-global feature collaborative optimization is realized without dimension reduction, the detail perception ability of the traffic target is enhanced, the complex background noise interference is suppressed, and the multi-target feature expression ability is improved while maintaining the model lightweight.

[0041] 4. The LDConv-MLCA joint architecture of the application realizes complementary enhancement between modules through hierarchical deployment, specifically: in the network shallow feature extraction stage, the LDConv module is used, the linear parameter growth strategy and the dynamic sampling mechanism are used to dynamically capture the geometric deformation characteristics of the traffic target, and the details are preserved while reducing the computing overhead; in the deep semantic enhancement stage, the C2f_MLCA module is embedded, the local-global attention collaborative mechanism is used to focus on the key feature area, and the multi-scale target semantic expression ability is strengthened; further, the dynamic sampling mechanism of LDConv and the cross-region attention of MLCA form a spatial-channel two-dimensional adaptive feature, jointly model the deformation characteristics of the traffic target and the context association, and build a complete optimization link from the bottom deformation adaptation to the high-level semantic enhancement, while maintaining the low computational complexity of the model, the detection accuracy and generalization ability of the complex traffic scene are improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A light traffic target detection network structure diagram based on LM-YOLO is provided for the application;

[0043] Figure 2 A LDConv module structure diagram is provided for the application;

[0044] Figure 3 A C2f-MLCA module structure diagram is provided for the application;

[0045] Figure 4 A MLCA module structure diagram is provided for the application. DETAILED DESCRIPTION

[0046] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments of the present application.

[0047] As shown in Figure 1 , the lightweight traffic target detection method based on LM-YOLO described in the present embodiment comprises the following steps: S1, obtaining a public traffic scene dataset and preprocessing;

[0048] S2, constructing an LM-YOLO network model, referring to Figure 1 , the LM-YOLO network takes YOLOv8n as a basic architecture, replaces the conventional convolution with an LDConv module to improve the feature extraction capability of the model on complex targets, and constructs a C2f_MLCA module in front of the three-level detection head to strengthen the feature discrimination capability of the model on multi-scale targets;

[0049] S3, configuring a training environment, and performing LM-YOLO network training and verification analysis by using the dataset obtained in step S1;

[0050] S4, inputting a traffic scene image to be detected into the pre-trained LM-YOLO network model for target detection.

[0051] Preferably, in step S1, the BDD100k dataset is preprocessed, 10000 data are randomly extracted, the training set and the verification set are divided in a ratio of 9:1, and the image and label formats are converted to adapt to the input of the YOLOv8 model.

[0052] Preferably, in step S2, the LM-YOLO network only retains the first layer feature extraction to perform preliminary feature mapping using the conventional convolution, and the subsequent downsampling adopts the LDConv module to replace the conventional convolution operation. The LDConv module can break away from the dependence of the network on the regular square grid, support the sampling shape of the convolution kernel with an arbitrary number of parameters, adjust the parameter growth trend from square to linear, and combine the dynamic adjustment of the offset to adapt to the target changes, so as to reduce the parameter amount while improving the feature extraction capability on complex targets, and provide a lightweight selection for the hardware resource limited scene.

[0053] Preferably, referring to 3, the LDConv module specifically comprises:

[0054] According to the set convolution kernel parameter N, an initial sampling shape and relative coordinates are generated by an initial sampling coordinate generation algorithm , the initial coordinate generation algorithm first generates a parameter amount Generate a square sampling grid of rules, and then make up the remaining points, splice into a complete sampling coordinate matrix;

[0055] According to the resolution and step of the input feature map, generate the initial sampling point absolute coordinates ;

[0056] The offset generation module generates dynamic coordinate offsets by performing convolution operations on the input feature map , the offset generation module is in the form of , wherein indicates that each sampling point has two coordinate offsets (horizontal and vertical directions);

[0057] Combine the relative coordinates, absolute coordinates and dynamic offsets of the initial sampling points to calculate the final sampling point coordinates ;

[0058] Based on the final sampling point coordinates, the feature values of the corresponding positions are extracted from the input feature map using the bilinear difference method to obtain the offset resampling feature map, and the dimension is ;

[0059] Stack the resampling feature map along the direction, reshape it to , so that it can be processed by the convolution kernel of the main convolution module;

[0060] The main convolution module is used to perform convolution operations on the reshaped feature map to obtain the final output feature map. The main convolution module includes a convolution layer, a batch normalization layer and a SiLU activation function. The convolution layer is in the form of to adapt to the feature map after dynamic sampling.

[0061] Preferably, in step S2, the LM-YOLO network introduces an MLCA attention mechanism in the C2f module before the three-level detection head to construct a C2f MLCA module, enhances the feature expression ability of multi-scale traffic targets, and realizes high-precision target detection in complex scenes. The MLCA attention mechanism adopts a double-branch structure, models the channel, space, local and global information simultaneously through the combination of LAP and GAP, and adopts one-dimensional convolution to accelerate, realize efficient balance of parameters and calculation amount, and make it suitable for lightweight target detection network.

[0062] Preferably, in step S2, the structure of the C2f MLCA module is shown in Figure 3 , and specifically includes:

[0063] The input feature map is expanded through a 1x1 convolution layer;

[0064] The expanded feature map is evenly divided into two parts along the channel dimension to obtain two feature maps and ;

[0065] Will Passing in sequence There are several Bottleneck modules, and the output of each module serves as the input to the next module, ultimately resulting in... Each feature map is denoted as _ ... The Bottleneck module contains two convolutional layers and an MLCA module, and the module output uses residual connections.

[0066] All feature maps The feature map is obtained by stitching along the channel dimension.

[0067] The concatenated feature map is input into a 1x1 convolutional layer, so that the number of channels is mapped to the target output channel number.

[0068] Preferably, in step S2, the structure of the MLCA module is as follows: Figure 4 As shown, it specifically includes:

[0069] Input feature map After LAP, it was divided into For each local region, calculate the average value to obtain the local features. This preserves spatial information and avoids the loss of details caused by gaps alone;

[0070] In the global branch, the extracted local feature maps After GAP compression, it is converted into a global feature map. Reshape global features into To adapt to 1D convolution computation, thus obtaining global channel attention weights. Finally, the global feature map is expanded to [a specific value] using unaverage pooling (UNAP). size;

[0071] In local branches, the extracted local feature maps Reshaped into Similarly, local channel attention weights are calculated using 1D convolution, and the results are reconstructed back. size;

[0072] Will and After normalization, the weights are summed and then adjusted back to the input feature map size using UNAP to obtain the final attention weights. ;

[0073] The resulting attention weights Compared with the original input feature map Element-wise multiplication, the final enhanced feature map output is obtained, so that the model can adaptively enhance important features and suppress redundant information.

[0074] Preferably, in step S3, the training environment of the LM-YOLO network is Ubuntu22.04, NVIDIA GeForce RTX 4060 Ti, CUDA11.3, Pytorch1.11, Python3.8, and the experimental parameter configuration is shown in Table 1.

[0075] Table 1 Experimental parameter setting

[0076]

[0077] Preferably, in step S3, after training, the detection performance of the added LDConv and MLCA modules in the LM-YOLO algorithm is verified, YOLOv8n is used as the basic network, and ablation experiments are performed on the BDD100k dataset to evaluate the influence of each module on the performance of the target detection algorithm, and the results are shown in Table 2.

[0078] Table 2 Ablation experiment results

[0079]

[0080] In Table 2, FLOPS represents the floating point operation amount of the model, Paras represents the parameter amount of the model, and Weight represents the size of the model weight file. The smaller the values of the three, the better the real-time performance of the model; mAP@0.5 represents the average precision of all classes when the intersection over union (IOU) threshold is 0.5, and mAP@0.5:0.9 represents the average precision of all classes at multiple IoU thresholds (from 0.5 to 0.9, with a step of 0.05). The larger the values of the two, the better the detection performance of the model. As can be seen from the results, the detection accuracy of the model is improved by adding each module alone. When both are used together, compared with the basic YOLOv8n model, the parameter amount is reduced by 4.8%, and mAP@0.5:0.9 is improved by 3.3%. Therefore, the LM-YOLO model not only meets the lightweight requirement, but also improves the detection accuracy of the model for traffic targets.

[0081] In another aspect, the application also discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to make the processor execute the steps of the above method.

[0082] In another aspect, the application also discloses a computer device, which comprises a memory and a processor, and the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the above method.

[0083] In another embodiment provided in the present application, a computer program product including instructions is provided, which, when executed on a computer, causes the computer to perform the LM-YOLO-based lightweight traffic target detection method in any of the above embodiments.

[0084] It can be understood that the system, device and storage medium provided in the embodiments of the present application correspond to the method provided in the embodiments of the present application, and the explanation, examples and beneficial effects of the related content can refer to the corresponding part in the above method.

[0085] In the above embodiments, the system, device and storage medium provided in the embodiments of the present application can be realized by software, hardware, firmware or any combination thereof, in whole or in part. When realized by software, the computer program product can be realized in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the flow or function described in the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD) or a semiconductor medium (for example, Solid State Disk (SSD)) and the like.

[0086] It should be noted that in this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0087] The various embodiments in the specification are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0088] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A lightweight traffic target detection method based on LM-YOLO, characterized in that, The method comprises the following steps: S1, obtaining and preprocessing a public traffic scene dataset; S2, constructing an LM-YOLO network model, taking YOLOv8n as a basic architecture, using an LDConv module to improve the feature extraction capability of the model for complex targets, and constructing a C2f_MLCA module in front of a three-level detection head to strengthen the feature discrimination capability of the model for multi-scale targets; S3, configuring a training environment, and using the dataset obtained in step S1 to perform LM-YOLO network training and verification analysis; S4, inputting a traffic scene image to be detected into the pre-trained LM-YOLO network model for target detection; In step S2, the LM-YOLO network only retains the first layer feature extraction using conventional convolution for preliminary feature mapping, and the subsequent downsampling all uses the LDConv module to replace the conventional convolution operation, and the LDConv module specifically comprises: According to the set convolution kernel parameter N, the initial sampling shape and relative coordinates are generated by an initial sampling coordinate generation algorithm; According to the resolution and stride of the input feature map, the absolute coordinates of the initial sampling points are generated; The input feature map is convolved by the offset generation module to generate a dynamic offset of the coordinates; The final sampling point coordinates are calculated by combining the relative coordinates, absolute coordinates and dynamic offset of the initial sampling points; Based on the final sampling point coordinates, the feature values of the corresponding positions in the input feature map are extracted using a bilinear difference method to obtain the offset resampled feature map; The resampled feature map is reshaped so that it can be processed by the convolution kernel of the main convolution module; The reshaped feature map is convolved using the main convolution module to obtain the final output feature map.

2. The lightweight traffic target detection method based on LM-YOLO according to claim 1, characterized in that, In step S1, the preprocessing of the public dataset includes data random extraction, training set and validation set division, image and label format conversion to adapt to the YOLOv8 model input.

3. The lightweight traffic target detection method based on LM-YOLO according to claim 1, characterized in that, In step S2, the LM-YOLO network introduces an MLCA attention mechanism in the C2f module in front of the three-level detection head to construct a C2f_MLCA module, and the structure of the C2f_MLCA module specifically comprises: The input feature map is expanded through a 1x1 convolution layer; The expanded feature map is evenly divided into two parts along the channel dimension; The one of the feature maps is sequentially passed through Bottleneck modules, and an output of each module is taken as an input of a next module, and finally a feature map is obtained. All feature maps are spliced along the channel dimension to obtain a spliced feature map; The spliced feature map is input into a 1x1 convolution layer to map the channel number to the target output channel number.

4. The lightweight traffic target detection method based on LM-YOLO according to claim 3, characterized in that, In step S2, the MLCA attention mechanism specifically comprises: The input feature map is divided into local regions by LAP, the average value of each region is calculated, and a local feature map is obtained. In the global branch, GAP is performed on the local feature map to compress it into a global feature map, and the global feature map is input into 1D convolution operation after being reshaped, to obtain a global channel attention weight, and finally the global feature map is expanded to Dimensions; In the local branch, the local feature map is reshaped, and the local channel attention weight is calculated by 1D convolution, and the result is reshaped back Dimensions; The local feature map and the global feature map obtained by the two branches are normalized and added, and the UNAP is adjusted back to the input feature map size to obtain the final attention weight; The obtained attention weight is multiplied element by element with the original input feature map to obtain the final enhanced feature map output.

5. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to perform the steps of the method of any one of claims 1 to 4. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-5 when the computer program is executed by the processor. The computer program is executed by the processor to perform the steps of the method of any one of claims 1 to 4.

Images