Improved YOLOv8-based automatic driving vehicle target detection method
By improving in the backbone, neck and head network of YOLOv8, the C2f-Faster module, BiFPN-SDI feature fusion network and DyHead3 module are adopted, and the problem of insufficient target detection capabilities of autonomous vehicles in complex contexts is solved, and the target detection capability of autonomous vehicles is improved. Detection accuracy and stability.
Patent Information
- Application Number
- CN202411183397.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-20
AI Technical Summary
In the prior art, the target detection capability of autonomous driving vehicles in complex backgrounds still needs to be improved, especially in urban traffic environments, it is difficult to meet the detection needs of complex backgrounds.
By replacing the C2f-Faster module, BiFPN-SDI feature fusion network and DyHead3 module in the backbone network, neck network and head network of YOLOv8, an improved YOLOv8 model is formed and trained using the autonomous driving dataset to improve Object detection capability.
It improves the target detection capability of autonomous driving vehicles in complex contexts, reduces the amount of model parameters and calculations, enhances the detection accuracy of small-objective semantic information, and improves the model's processing ability and training stability for complex scenarios.
Smart Images
Figure CN119169260A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of target detection technology, and in particular to a target detection method for an autonomous driving vehicle based on improved YOLOv8. Background Art
[0002] The environmental perception system is the basis of autonomous driving technology, and target detection technology is the basis and prerequisite of the environmental perception system. With the increasing development of urban traffic, the general target detection algorithm can no longer meet the complexity requirements of urban traffic. Therefore, it is necessary to study the target detection method of autonomous driving vehicles.
[0003] In the prior art, Chinese patent CN117351448A discloses a polarization image road target detection method based on improved YOLOv8, including: obtaining a polarization image data set and preprocessing it, dividing the preprocessed polarization image data set into a training set and a test set; constructing a first C2f module through a deformable convolution module DCNv2, constructing a second C2f module through a re-parameter module DBB, and improving YOLOv8 by combining a convolutional attention mechanism module CBAM and a loss function module WiseIoU to obtain an improved network model; inputting the training set into the network model for iterative training to obtain a target network model; inputting the test set into the target network model for testing, and obtaining a road target detection result through the tested target network model.
[0004] Although the above-mentioned existing technologies have improved YOLOv8, their target detection capabilities for autonomous driving vehicles in complex backgrounds, such as urban traffic environments, still need to be improved. Summary of the invention
[0005] This application provides an autonomous driving vehicle target detection method based on improved YOLOv8 to solve the problem that the target detection capability of autonomous driving vehicles in the prior art still needs to be improved.
[0006] On the one hand, the present application provides an autonomous driving vehicle target detection method based on improved YOLOv8, comprising the following steps:
[0007] Step 1: In the backbone network of YOLOv8, the C2f-Faster module is used to replace the C2f module in the original backbone network, the BiFPN-SDI feature fusion network is used to replace the original neck network in the neck network of YOLOv8, and the dynamic detection head DyHead3 module with fusion attention mechanism is used to replace the original head network in the head network of YOLOv8 to obtain the improved YOLOv8 model.
[0008] Step 2: Use the autonomous driving data set to train the improved YOLOv8 model, and obtain the target detection model after the training is completed.
[0009] Step three: Use the target detection model to perform actual autonomous driving vehicle target detection.
[0010] In a possible implementation, in step 1, the C2f-Faster module replaces all Bottleneck units in the C2f module with Fasterblock units.
[0011] In a possible implementation, in step 1, the Fasterblock unit is composed of a partial convolutional layer and two convolutional layers.
[0012] In a possible implementation, in step 1, the BiFPN-SDI feature fusion network is to add a multi-level feature fusion module SDI to the improved BiFPN network.
[0013] In a possible implementation, in step 1, the improved BiFPN network is to integrate the P2 small target detection layer into the original BiFPN network.
[0014] In a possible implementation, in step 1, the DyHead3 module adopts an improved Dynamic Head module.
[0015] The improved Dynamic Head module replaces the DCNv2 mechanism of the original Dynamic Head module with the DCNv3 mechanism.
[0016] In a possible implementation, in step 2, the autonomous driving dataset uses the BDD100k dataset.
[0017] The autonomous driving vehicle target detection method based on improved YOLOv8 in this application has the following advantages:
[0018] By improving the backbone network, neck network and head network of YOLOv8, the C2f-Faster module is used in the backbone network of YOLOv8 to replace the C2f module in the original backbone network, the BiFPN-SDI feature fusion network is used in the neck network of YOLOv8 to replace the original neck network, and the dynamic detection head DyHead3 module with fusion attention mechanism is used in the head network of YOLOv8 to replace the original head network, which improves the target detection capability of the target detection model for autonomous driving vehicles in complex backgrounds.
[0019] The proposed C2f-Faster module replaces all Bottleneck units in the C2f module with Fasterblock units. The Fasterblock unit consists of a partial convolutional layer and two convolutional layers, which reduces the number of model parameters and computational complexity.
[0020] The proposed BiFPN-SDI feature fusion network adds the multi-level feature fusion module SDI to the improved BiFPN network. The improved BiFPN network integrates the P2 small target detection layer into the original BiFPN network, which simplifies the feature fusion process, enhances the semantic information of small targets, and improves the accuracy of the model in detecting small targets.
[0021] The proposed DyHead3 module adopts the improved Dynamic Head module. The improved Dynamic Head module replaces the DCNv2 mechanism of the original Dynamic Head module with the DCNv3 mechanism, so that the model can better capture the details and features of different scales in complex scenes, process complex feature maps faster, better capture the relationship between distant pixels, improve training stability and improve non-local feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 A flowchart of a method for detecting an autonomous driving vehicle target based on improved YOLOv8 provided in an embodiment of the present application;
[0024] Figure 2 A schematic diagram of the network structure of the improved YOLOv8 model provided in an embodiment of the present application;
[0025] Figure 3 A schematic diagram of the structure of the C2f-Faster module provided in an embodiment of the present application;
[0026] Figure 4 A schematic diagram of the structure of a Fasterblock unit provided in an embodiment of the present application;
[0027] Figure 5 A schematic diagram of the structure of an improved BiFPN network provided in an embodiment of the present application;
[0028] Figure 6 A schematic diagram of the structure of a multi-level feature fusion module SDI provided in an embodiment of the present application;
[0029] Figure 7 A schematic diagram of the structure of the Dynamic Head module provided in an embodiment of the present application. DETAILED DESCRIPTION
[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0031] like Figure 1 As shown, the embodiment of the present application provides an autonomous driving vehicle target detection method based on improved YOLOv8, comprising the following steps:
[0032] Step 1: In the backbone network of YOLOv8, the C2f-Faster module is used to replace the C2f module in the original backbone network, the BiFPN-SDI feature fusion network is used to replace the original neck network in the neck network of YOLOv8, and the dynamic detection head DyHead3 module with fusion attention mechanism is used to replace the original head network in the head network of YOLOv8 to obtain the improved YOLOv8 model.
[0033] Step 2: Use the autonomous driving data set to train the improved YOLOv8 model, and obtain the target detection model after the training is completed.
[0034] Step three: Use the target detection model to perform actual autonomous driving vehicle target detection.
[0035] Specifically, Figure 2 The figure shows the network structure of the improved YOLOv8 model. The input image is first pre-processed and enters the backbone network for feature extraction, then enters the neck network for feature fusion, and then enters the head network to generate bounding boxes, class probabilities, and confidence scores. Finally, it is post-processed to output the autonomous driving vehicle target detection results.
[0036] Exemplarily, in step 1, the C2f-Faster module replaces all Bottleneck units in the C2f module with Fasterblock units.
[0037] Specifically, Figure 3 The figure shows the structure diagram of the C2f-Faster module. It can be seen that the C2f-Faster module replaces all the Bottleneck units in the C2f module with Fasterblock units. The C2f module is an existing module structure and will not be described in detail here.
[0038] Exemplarily, in step 1, the Fasterblock unit is composed of a partial convolutional layer and two convolutional layers.
[0039] Specifically, Figure 4 The Fasterblock unit is shown as a schematic diagram of the structure of the Fasterblock unit. In this embodiment, the Fasterblock unit is composed of a 3×3 partial convolution layer (corresponding to Figure 4 Pconv in) and two 1×1 convolutional layers (corresponding to Figure 4 The network consists of two Conv) layers in , where the feature map is calculated through a 3×3 partial convolution layer to reduce the amount of calculation and parameters and enhance the network learning ability. Subsequently, two 1×1 convolution layers are used to extract the effective information of the feature map, and finally the effective information is output for the next step.
[0040] Exemplarily, in step 1, the BiFPN-SDI feature fusion network is to add a multi-level feature fusion module SDI to the improved BiFPN network.
[0041] Specifically, in the BiFPN-SDI feature fusion network, the multi-level feature fusion module SDI is used to replace the Bi_FPN module in the improved BiFPN network to perform splicing and fusion operations on multiple feature maps in the depth dimension. The multi-level feature fusion module SDI integrates the Concat operation of the attention mechanism.
[0042] Exemplarily, in step 1, the improved BiFPN network is to integrate the P2 small target detection layer into the original BiFPN network.
[0043] Specifically, Figure 5 The figure shows the structure diagram of the improved BiFPN network. The improved BiFPN network removes the nodes with only one input in the original BiFPN network. This operation reduces the amount of calculation and the complexity of the model without affecting the overall performance. The original BiFPN network only has three levels of input feature maps, P3, P4, and P5. This application adds an additional P2-level input path, and integrates the P2 small target detection layer into the original BiFPN network, which simplifies the feature fusion process, enhances the semantic information of small targets, and improves the accuracy of the model in detecting small targets.
[0044] Specifically, Figure 6 The figure shows the structure of the multi-level feature fusion module SDI. The SDI module is a component of the UNetV2 model. The UNetV2 model consists of three main modules: encoder, SDI module and decoder. The main idea of the SDI module is to enhance the semantic information and detail information in the image by integrating the hierarchical feature maps generated by the encoder, specifically feature extraction and integration, fusion of high-level features and low-level features, and feature transfer and segmentation. Figure 6In the above, UpSample means upsampling, IdentityMap means identity mapping, DownSample means downsampling, SpatialAttention means spatial attention, Hadamard product means matrix multiplication, feature map means activation value, ChannelAttention means channel attention, SmoothConv means 3×3 convolution of feature smoothing, and l=1 to l=4 means feature maps of 4 different levels. The SDI module processes the features of each level generated by the encoder by upsampling, identity mapping, downsampling, etc., and applying spatial attention and channel attention mechanisms. This process enables the features to integrate local spatial information and global channel information, and finally outputs the feature map through matrix multiplication.
[0045] Exemplarily, in step 1, the DyHead3 module adopts an improved Dynamic Head module.
[0046] The improved Dynamic Head module replaces the DCNv2 mechanism of the original Dynamic Head module with the DCNv3 mechanism.
[0047] Specifically, the DyHead3 module is an upgraded version of the Dynamic Head module, which replaces the DCNv2 mechanism of the original Dynamic Head module with the DCNv3 mechanism. In the Dynamic Head module, DCNv2 is integrated into multiple convolutional layers for feature extraction and feature enhancement, enabling it to more effectively handle multi-scale and multi-shape targets and improve overall detection performance. In this embodiment, replacing DCNv2 with DCNv3 can bring higher detection accuracy and stronger feature expression capabilities, and has better performance in road target detection in complex urban traffic scenarios, and improves computing efficiency without significantly increasing computing costs.
[0048] Specifically, Figure 7 The following is a schematic diagram of the structure of the Dynamic Head module. The Dynamic Head module based on the attention mechanism integrates the three attention mechanisms of scale perception, space perception and task perception into a unified framework, and effectively applies the attention mechanism in the target detection head, thereby improving the detection performance and efficiency.
[0049] Figure 7 In, π L , π S and π CThey are scale-aware attention module, space-aware attention module and task-aware attention module, which are connected in series to form a single Dynamic Head module. After the feature map is input into the Dynamic Head module, it is first L Perform an average pooling operation in , then perform a 1×1 convolution operation, and then activate it using the Hard Sigmoid activation function, and finally multiply the obtained tensor with the input feature to complete scale perception; enter π S In the first step, a 3×3 convolution operation is performed to obtain the offset of the feature map and its offset weight, and the replaced DCNv3 is used to learn sparsity to complete spatial perception. Then, the dynamic linear rectification function is used to adjust the channel value of the feature map according to different tasks to complete task perception.
[0050] Specifically, DCNv3 is extended on the basis of DCNv2, mainly including sharing weights between neurons, introducing multi-group mechanism and normalization processing. After the extension, the characteristic value y(P0) output by DCNv3 is shown as follows:
[0051]
[0052] Among them, G represents the number of convolution groups, w g represents the shared projection weight within each group, m gk represents the normalized modulation factor of the kth sampling point in the gth group, x g represents the input feature map of the slice, Δp gk is the grid sampling position p corresponding to the gth group k The offset p0 represents the correspondence between each position on the output feature map and the center position of the convolution kernel, and then maps it to the position in the input feature map.
[0053] DCNv3 reduces parameter and memory complexity by sharing projection weights while retaining the dependencies of all sampling positions; the multi-group mechanism divides the spatial aggregation process into several groups, each with an independent sampling offset, producing rich feature diversity; the Softmax normalization of each sampling point alleviates the instability of the model, making the model training more stable, and constructs the connection relationship of all sampling points, so that DCNv3 has higher computational efficiency and better training stability while maintaining efficient feature extraction. Therefore, by replacing DCNv2 with DCNv3 in the Dynamic Head module, the model can better capture details and different scale features in complex scenes, process complex feature maps faster, better capture the relationship between distant pixels, improve training stability and improve non-local feature extraction.
[0054] Exemplarily, in step 2, the autonomous driving dataset adopts the BDD100k dataset.
[0055] The embodiment of the present application improves the backbone network, neck network, and head network of YOLOv8. The C2f-Faster module is used in the backbone network of YOLOv8 to replace the C2f module in the original backbone network, the BiFPN-SDI feature fusion network is used in the neck network of YOLOv8 to replace the original neck network, and the dynamic detection head DyHead3 module with a fusion attention mechanism is used in the head network of YOLOv8 to replace the original head network, thereby improving the target detection capability of the target detection model for autonomous driving vehicles under complex backgrounds.
[0056] The proposed C2f-Faster module replaces all Bottleneck units in the C2f module with Fasterblock units. The Fasterblock unit consists of a partial convolutional layer and two convolutional layers, which reduces the number of model parameters and computational complexity.
[0057] The proposed BiFPN-SDI feature fusion network adds the multi-level feature fusion module SDI to the improved BiFPN network. The improved BiFPN network integrates the P2 small target detection layer into the original BiFPN network, which simplifies the feature fusion process, enhances the semantic information of small targets, and improves the accuracy of the model in detecting small targets.
[0058] The proposed DyHead3 module adopts the improved Dynamic Head module. The improved Dynamic Head module replaces the DCNv2 mechanism of the original Dynamic Head module with the DCNv3 mechanism, so that the model can better capture the details and features of different scales in complex scenes, process complex feature maps faster, better capture the relationship between distant pixels, improve training stability and improve non-local feature extraction.
[0059] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0060] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. The autonomous driving vehicle target detection method based on improved YOLOv8 is characterized by: The following steps are involved: Step 1: In the backbone network of YOLOv8, the C2f-Faster module is used to replace the C2f module in the original backbone network, the BiFPN-SDI feature fusion network is used to replace the original neck network in the neck network of YOLOv8, and the dynamic detection head DyHead3 module with fusion attention mechanism is used to replace the original head network in the head network of YOLOv8 to obtain an improved YOLOv8 model; Step 2: Using the autonomous driving data set to train the improved YOLOv8 model, and obtaining a target detection model after the training is completed; Step three: Use the target detection model to perform actual autonomous driving vehicle target detection.
2. The automatic driving vehicle target detection method based on improved YOLOv8 according to claim 1, characterized in that: In step 1, the C2f-Faster module replaces all Bottleneck units in the C2f module with Fasterblock units.
3. The automatic driving vehicle target detection method based on improved YOLOv8 according to claim 2, characterized in that: In step 1, the Fasterblock unit is composed of a partial convolutional layer and two convolutional layers.
4. The automatic driving vehicle target detection method based on improved YOLOv8 according to claim 1, characterized in that: In step 1, the BiFPN-SDI feature fusion network adds the multi-level feature fusion module SDI to the improved BiFPN network.
5. The automatic driving vehicle target detection method based on improved YOLOv8 according to claim 4, characterized in that: In step 1, the improved BiFPN network integrates the P2 small target detection layer into the original BiFPN network.
6. The automatic driving vehicle target detection method based on improved YOLOv8 according to claim 1, characterized in that: In step 1, the DyHead3 module adopts an improved Dynamic Head module; The improved Dynamic Head module replaces the DCNv2 mechanism of the original Dynamic Head module with the DCNv3 mechanism.
7. The automatic driving vehicle target detection method based on improved YOLOv8 according to claim 1, characterized in that: In step 2, the autonomous driving dataset uses the BDD100k dataset.