Complex road scene target detection method based on improved YOLOv8 model
By introducing DualConv and MLCA modules into the YOLOv8 model, combined with DCNv4 and SEAM attention modules, the problems of low target detection accuracy and slow inference speed in complex road scenarios are solved, and more efficient target detection performance is achieved.
Patent Information
- Application Number
- CN202411875591.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-04-04
AI Technical Summary
In complex road scenarios, the existing YOLOv8 detection algorithm is insufficient in detection accuracy and slow inference speed due to irregular detection target shapes, low detection accuracy and slow inference speed, which affects the safety of the autonomous driving system.
By introducing DualConv convolution, the D-C2f module is constructed, and the MD-C2f module is constructed based on the MLCA module and DCNv4 fusion, the Bottlenect in the C2f module in YOLOv8 is replaced, and the SEAM attention module is added to improve the complex road scene object detection method of the YOLOv8 model.
It improves the accuracy and speed of object detection in complex road contexts, enhances the detection ability of irregular shapes and occlusion targets, and improves the lightweighting and inference speed of the model.
Smart Images
Figure CN119762939A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a target detection method for complex road scenes based on an improved YOLOv8 model. Background Art
[0002] Transportation is a basic industry for national development. While improving living standards, it also brings certain risks. Road object detection algorithms play a key role in autonomous driving and vehicle safety systems. This technology is mainly used to detect traffic signs, pedestrians, vehicles, etc. in road images, providing key information for autonomous driving and intelligent safety systems to ensure traffic safety and efficiency. The YOLO series algorithm is one of the most popular and widely used object detection algorithms. It has been iterated for eight versions, and its speed and accuracy have been greatly improved.
[0003] Some existing target detection algorithms can already perform target detection well in general road backgrounds, but when the road scene is more complex, the distance between the on-board camera and the target changes frequently and dynamically. At the same time, when the road is congested, other vehicles are prone to mutual occlusion, which greatly affects the detection accuracy and speed of the target detection algorithm, and further causes safety issues in the autonomous driving system. Therefore, how to solve the problem of small target detection at a long distance and dense target occlusion in complex road scenes has become an urgent problem to be solved. Summary of the invention
[0004] The purpose of the present invention is to solve the problems of the existing YOLOv8 detection algorithm in complex road scenes, such as irregular shape of detected targets, low detection accuracy of distant targets and slow reasoning speed, and to propose a complex road scene target detection method based on an improved YOLOv8 model.
[0005] A complex road scene target detection method based on an improved YOLOv8 model includes the following steps:
[0006] Step S1: pre-process the data set images and divide them into training set, validation set and test set according to the proportion;
[0007] Step S2: Introduce DualConv convolution to construct the D-C2f module. Construct the MD-C2f module based on the fusion of MLCA module and DCNv4.
[0008] Step S3: Construct an improved YOLOv8 network model, including: Backbone feature extraction module, Head feature fusion module, and detection head module; wherein, in the Backbone feature extraction module, the D-C2f module is used to replace the original C2f module; in the Head feature fusion module, the C2f modules in each detection layer are replaced with MD-C2f modules, and then a layer of SEAM attention module is added after each MD-C2f module.
[0009] Step S4: The training set divided in step S1 is used as the improved YOLOv8 network model constructed in step S3, and feature collection is performed through the Backbone feature extraction module to extract feature graphs Future1, Future2, and Future3 of different scales;
[0010] Step S5: Input Future1, Future2, and Future3 into the detection layers of different scales in the Head feature fusion module to realize the transfer of semantic features and positioning features, and obtain fused features Future6, Future9, and Future12;
[0011] Step S6: Input Future6, Future9, and Future12 into the detection head module, and finally output the position of the detection box in the image and its target category.
[0012] The present invention has the following advantages:
[0013] 1. The present invention uses the MLCA module and two DCNv4 convolutions to construct a new MD-Bottlenect block to replace the Bottlenect in C2f in YOLOv8, forming a new MD-C2f module, so that the model can adapt to targets of irregular shapes and enhance the detection accuracy of small targets.
[0014] 2) In order to solve the problem of missed detection and false detection caused by feature loss and inaccurate positioning when the target to be detected is occluded, the present invention adds a SEAM attention module to the model, aiming to compensate for the feature loss of the occluded target and optimize the detection accuracy of the occluded target.
[0015] 3) The present invention introduces the DualConv convolutional network structure, replaces the two consecutive convolutions of Bottlenect in the C2f module in YOLOv8 with DualConv, and constructs a D-C2f module, which improves the accuracy while lightweighting the model to increase the inference speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute a limitation of the present application. In the drawings:
[0017] Figure 1 A flowchart of a complex road scene target detection method based on an improved YOLOv8 model proposed by the present invention;
[0018] Figure 2 This is a schematic diagram of the improved YOLOv8 network structure proposed in the present invention;
[0019] Figure 3 This is a structural diagram of the MLCA network module introduced in the present invention;
[0020] Figure 4 This is a structural diagram of the MD-C2f network module proposed in the present invention;
[0021] Figure 5 This is the result diagram of the SEAM network module introduced in the present invention;
[0022] Figure 6 This is the DualConv network structure diagram introduced in the present invention; DETAILED DESCRIPTION
[0023] The present invention proposes a target detection method for complex road scenes based on an improved YOLOv8 model, which improves the target detection accuracy and speed under complex road backgrounds and solves the problem of low detection accuracy for targets with irregular shapes and at a distance.
[0024] The present invention is further described below through specific implementations.
[0025] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.
[0026] The words "include" or "comprises" and the like used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms "inside", "outside", "upper", "lower", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, the terms "attachment" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0027] Step S1: pre-process the data set images and divide them into training set, validation set and test set according to the proportion;
[0028] Step S2: Introduce DualConv convolution to construct the D-C2f module. Construct the MD-C2f module based on the fusion of MLCA module and DCNv4.
[0029] Step S3: Construct an improved YOLOv8 network model, including: Backbone feature extraction module, Head feature fusion module, and detection head module; wherein, in the Backbone feature extraction module, the D-C2f module is used to replace the original C2f module; in the Head feature fusion module, the C2f modules in each detection layer are replaced with MD-C2f modules, and then a layer of SEAM attention module is added after each MD-C2f module.
[0030] Step S4: The training set divided in step S1 is used as the improved YOLOv8 network model constructed in step S3, and feature collection is performed through the Backbone feature extraction module to extract feature graphs Future1, Future2, and Future3 of different scales;
[0031] Step S5: Input Future1, Future2, and Future3 into the detection layers of different scales in the Head feature fusion module to realize the transfer of semantic features and positioning features, and obtain fused features Future6, Future9, and Future12;
[0032] Step S6: Input Future6, Future9, and Future12 into the detection head module, and finally output the position of the detection box in the image and its target category.
[0033] In one embodiment, the above step S1: preprocessing the data set images, dividing them into a training set, a validation set and a test set according to the proportion. Specifically includes:
[0034] The SODA10M dataset was selected. The entire dataset was labeled with six main scene categories (pedestrians, bicycles, cars, trucks, trams, and tricycles). The 10,000 labeled images in SODA10M were randomly divided into training set, validation set, and test set in a ratio of 8:1:1.
[0035] In one embodiment, the above step S2: introduces DualConv convolution to construct D-C2f module; and constructs MD-C2f module based on fusion of MLCA module and DCNv4. Specifically includes:
[0036] like Figure 6 As shown in Figure 1, DualConv is an innovative convolutional network structure that uses 3x3 and 1x1 convolution kernels to perform convolution operations on the same input feature at the same time. Replacing the two consecutive convolutions of Bottlenect in the C2f module with DualConv results in D-C2f. Figure 3 As shown in the figure, in order to introduce spatial information, the MLCA module is implemented by dividing the feature map into multiple blocks. The Conv1d in the figure is a one-dimensional convolution, and the convolution kernel size k is proportional to the channel dimension C. The choice of k is determined by the following formula:
[0037]
[0038] Among them, C is the number of channels, k is the convolution kernel size, γ and b are hyperparameters, the default is 2, odd means that k can only be an odd number, if it is an even number, add 1. The principle of MLCA is to convert the input into a vector of 1*C*ks*ks through local pooling to extract local spatial information. Based on this stage, two branches are used to convert the input into a one-dimensional vector, one branch contains global information and the other branch contains local spatial information. After the one-dimensional convolution, the unpooling is used to restore the original resolution of the two vectors, and the information is fused to achieve the goal of mixed attention. Figure 4 As shown, in order to solve the problem of missed detection and false detection caused by different sizes and shapes of the targets to be detected, the present invention introduces DCNv4 to improve the detection capability of the network. For DCNv3, given an input x, the DCNv3 operator with K points is described as follows:
[0039]
[0040] Where m g" represents the spatial aggregation weight, p " represents the predefined sampling point, Δp g" Represents the offset. A key difference between DCNv3 and ordinary convolution is that the spatial aggregation weights are softmax normalized:
[0041]
[0042] If no softmax operation is performed, K T V will be calculated first, which will degenerate into linear projection of all queries in the same attention window, resulting in performance degradation. But in fact, for DCNv3, each point has its own aggregation window and the internal values are different, so there will be no degradation problem. DCNv4 removes softmax normalization and changes the range of the modulation factor from [0,1] to an unbounded dynamic weight, enhancing dynamic properties and expression capabilities. On the other hand, DCNv4 also optimizes memory access to reduce redundant operations, thereby speeding up processing. Figure 4 The right part shows the MD-Bottleneck built based on the fusion of MLCA module and DCNv4. It consists of two DCNv4 connected sequentially, and then a layer of MLCA module is added. After superimposing it with the input feature map, the output result is finally obtained. Figure 4 The left part is the final MD-C2f module.
[0043] In one embodiment, the above step S3: constructs an improved YOLOv8 network model, including: a Backbone feature extraction module, a Head feature fusion module, and a detection head module; wherein, in the Backbone feature extraction module, a D-C2f module is used to replace the original C2f module; in the Head feature fusion module, the C2f modules in each detection layer are replaced with MD-C2f modules, and then a layer of SEAM attention module is added after each MD-C2f module. Specifically, it includes:
[0044] like Figure 2 As shown in the figure, in order to improve the YOLOv8 network structure diagram, in the Backbone feature extraction module, the D-C2f module is used to replace the C2f module, and the model is lightweight while maintaining the network depth and feature extraction capabilities. In the Head feature fusion module, the C2f module in each detection layer is replaced with the MD-C2f module, and then a layer of SEAM module is added after each MD-C2f module to further improve the detection effect of irregularly shaped objects and occluded objects.
[0045] In one embodiment, the above step S4: using the training set divided in step S1 as the improved YOLOv8 network model constructed in step S3, performing feature collection through the Backbone feature extraction module, and extracting feature graphs Future1, Future2, and Future3 of different scales. Specifically includes:
[0046] like Figure 2 ,In the Backbone feature extraction module, there are four D-C2f modules, among which the second and the third D-C2f modules output ,features Future1 and Future2 respectively.,After passing through all modules of the Backbone feature extraction ,module, the last SPPF module outputs feature Future3.
[0047] In one embodiment, the above step S5: input Future1, Future2, and Future3 into the detection layers of different scales in the Head feature fusion module to achieve the transfer of semantic features and positioning features, and obtain fused features Future6, Future9, and Future12. Specifically includes:
[0048] like Figure 2 In the Head feature fusion module, Future1, Future2, and Future3 are fused from bottom to top, and the enhanced feature Future4 is obtained after fusion; Future4 is passed through the MD-C2f module to obtain the enhanced feature Future5; then the feature is transferred from top to bottom: Future5 is input into the SEAM attention module to obtain the enhanced feature Future6; Future6 is input into the Conv module for feature extraction, and Concat operation is performed with the output of the previous C2f module to obtain the feature Future7; Future7 is input into the MD-C2f module to obtain the enhanced feature Future8; Future8 is input into the SEAM attention module to obtain the enhanced feature Future9; similarly, the enhanced features Future10, Future11, and Future12 can be obtained; finally, the enhanced features Future6, Future9, and Future12 of the three scales are used as the output of the Head feature fusion module.
[0049] In one embodiment, the above step S6: input Future6, Future9, and Future12 into the detection head module, and finally output the position of the detection box in the image and its target category.
[0050] Specifically include:
[0051] like Figure 2,In the detection head module, there is a native Detect module of YOLOv8, which calculates the regression loss and classification loss for the input Future6, Future9, and Future12 respectively, and finally inputs the position of the detection box in the image and its target category.
[0052] Therefore, the present invention proposes a complex road scene target detection method based on an improved YOLOv8 model, which improves the target detection accuracy and speed under complex road backgrounds and solves the problem of low detection accuracy for targets with irregular shapes and at a distance.
[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A complex road scene target detection method based on an improved YOLOv8 model, characterized in that: include: Step S1: pre-process the data set images and divide them into training set, validation set and test set according to the proportion; Step S2: Introduce DualConv convolution to construct the D-C2f module. Construct the MD-C2f module based on the fusion of MLCA module and DCNv4. Step S3: Construct an improved YOLOv8 network model, including: Backbone feature extraction module, Head feature fusion module, and detection head module; wherein, in the Backbone feature extraction module, the D-C2f module is used to replace the original C2f module; in the Head feature fusion module, the C2f modules in each detection layer are replaced with MD-C2f modules, and then a layer of SEAM attention module is added after each MD-C2f module. Step S4: The training set divided in step S1 is used as the improved YOLOv8 network model constructed in step S3, and feature collection is performed through the Backbone feature extraction module to extract feature graphs Future1, Future2, and Future3 of different scales; Step S5: Input Future1, Future2, and Future3 into the detection layers of different scales in the Head feature fusion module to realize the transfer of semantic features and positioning features, and obtain fused features Future6, Future9, and Future12; Step S6: Input Future6, Future9, and Future12 into the detection head module, and finally output the position of the detection box in the image and its target category.
2. The complex road scene target detection method based on the improved YOLOv8 model according to claim 1 is characterized in that Step S2: Introduce DualConv convolution to build the D-C2f module. Construct the MD-C2f module based on the fusion of MLCA module and DCNv4. Specifically include: D-C2f is obtained by replacing the two consecutive convolutions of Bottlenect in the C2f module with DualConv. MD-Bottleneck, which is built based on the fusion of MLCA module and DCNv4, is composed of two DCNv4 connected in sequence, and then a layer of MLCA module is added. After superimposing it with the input feature map, the output result is finally obtained. The Bottlenect in the C2f module is replaced with MD-Bottlenect to obtain the MD-C2f module.
3. The complex road scene target detection method based on the improved YOLOv8 model according to claim 1, characterized in that: The step S3: constructs an improved YOLOv8 network model, including: a Backbone feature extraction module, a Head feature fusion module, and a detection head module; wherein, in the Backbone feature extraction module, the D-C2f module is used to replace the original C2f module; in the Head feature fusion module, the C2f modules in each detection layer are replaced with MD-C2f modules, and then a layer of SEAM attention module is added after each MD-C2f module. Specifically, it includes: In the Backbone feature extraction module, the D-C2f module is used to replace the C2f module, and the model is lightweight while maintaining the network depth and feature extraction capabilities. In the Head feature fusion module, the C2f module in each detection layer is replaced with the MD-C2f module, and then a layer of SEAM module is added after each MD-C2f module to further improve the detection effect of irregularly shaped objects and occluded objects.
4. The complex road scene target detection method based on the improved YOLOv8 model according to claim 1, characterized in that: The step S4: using the training set divided in step S1 as the improved YOLOv8 network model constructed in step S3, performing feature collection through the Backbone feature extraction module, and extracting feature graphs Future1, Future2, and Future3 of different scales; specifically includes: In the Backbone feature extraction module, there are 4 D-C2f modules, among which the second and third D-C2f modules output features Future1 and Future2 respectively. After passing through all modules of the BackBone feature extraction module, the final SPPF module outputs feature Future3.
5. The complex road scene target detection method based on the improved YOLOv8 model according to claim 1, characterized in that: The step S5: input Future1, Future2, and Future3 into the detection layers of different scales in the Head feature fusion module to achieve the transfer of semantic features and positioning features, and obtain fused features Future6, Future9, and Future12. Specifically including: In the Head feature fusion module, Future1, Future2, and Future3 are fused from bottom to top to obtain the enhanced feature Future4; Future4 is passed through the MD-C2f module to obtain the enhanced feature Future5; then the feature is transferred from top to bottom: Future5 is input into the SEAM attention module to obtain the enhanced feature Future6; Future6 is input into the Conv module for feature extraction, and Concat operation is performed with the output of the previous C2f module to obtain the feature Future7; Future7 is input into the MD-C2f module to obtain the enhanced feature Future8; Future8 is input into the SEAM attention module to obtain the enhanced feature Future9; similarly, enhanced features Future10, Future11, and Future12 can be obtained; finally, the enhanced features Future6, Future9, and Future12 of three scales are used as the output of the Head feature fusion module.