Vehicle segmentation algorithm research under complex traffic scene based on YOLO

Abstract

The invention relates to the technical field of computer vision and visual perception, aims to solve the technical problem of a vehicle segmentation research algorithm in a complex traffic scene, and provides a vehicle segmentation algorithm improvement based on YOLOV8-seg, and by adding an OfficientViT network architecture, an EMA attention mechanism module and a WIOU LoSS function in the algorithm, the precision in segmentation is improved, and the segmentation precision is improved. By training the deep learning model, the vehicle can identify and understand a complex traffic scene. In daily life, a driver is better assisted in driving, traffic accidents are reduced, and recognition is completed through the optimized YOLOV8-seg algorithm. Therefore, the vehicle segmentation algorithm is better than the existing method in the aspects of accuracy, recall rate and real-time performance, and has better performance.

Description

Technical Field

[0001] This paper proposes a vehicle segmentation perception algorithm in complex traffic scenes based on deep learning, and based on the improved YOLOv8-seg algorithm, simultaneously performs target segmentation of cars, bicycles, buses, trucks, and motorcycles in cloudy, night, sunny and rainy days, and realizes the research on the vehicle segmentation algorithm of YOLOv8-seg in complex traffic scenes. Background Art

[0003] In intelligent transportation systems, segmentation algorithm research plays a vital role. It is the core of autonomous vehicles' understanding of the surrounding environment. This technology enables autonomous vehicles to effectively identify and respond to pedestrians, vehicles, and obstacles around them to ensure driving safety. Autonomous vehicles are usually equipped with a variety of sensors, such as radar, LiDAR, and cameras. These sensors work together to collect accurate data about the vehicle's surroundings, providing an important source of information for segmentation. The image data captured by the camera is deeply analyzed through cutting-edge image recognition and processing technology, allowing the vehicle to accurately identify and distinguish various targets and objects. In the field of autonomous driving, the application of deep learning, especially convolutional neural networks (CNN), has greatly improved the effectiveness of target segmentation. Through the training of deep learning models, autonomous vehicles can deeply understand complex traffic scenes and make corresponding decisions. In addition, the fusion of high-dimensional data from different sensors further enhances the accuracy of segmentation and the robustness of the system. With the continuous innovation and development of technology, segmentation technology is also constantly being optimized and upgraded, thus providing a solid technical guarantee for the safe driving and reliability of autonomous vehicles. Summary of the invention

[0004] The present invention aims to efficiently realize environmental perception in traffic road scenes. Therefore, a vehicle segmentation algorithm in complex traffic scenes is proposed. Based on the improved YOLOv8-seg algorithm, five types of vehicles in road scenes are segmented to realize the research on vehicle segmentation algorithm on traffic roads.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows: The deep learning vehicle segmentation research algorithm in a complex traffic scenario includes the following steps: Step 1: Input image data, introduce a new memory-efficient transformer model family EfficientViT, design a new block with sandwich layout to build the model, and propose a new cascade group attention (CGA) module to improve computational efficiency. The core idea is to enhance the diversity of features of the input attention head. Reduce computational redundancy in multi-head attention, and increase model capacity by increasing network depth.

[0006] Step 2: By adding the EMA attention mechanism module, the information on each channel is retained and the computational overhead is reduced. Some channels are reconstructed into batch dimensions, and the channel dimension is grouped into multiple sub-features so that the spatial semantic features are evenly distributed within each feature group. The output features of the two parallel branches are further aggregated through cross-dimensional interactions to capture pixel-level pairwise relationships.

[0007] Step 3: The WIOU function is added and a reasonable gradient gain distribution is designed. This strategy reduces the large or harmful gradients that appear in extreme samples.

[0008] The specific situation in the step 1 is as follows: The present invention uses a camera mounted on a vehicle to capture images to obtain road traffic information. Improved parts in the YOLOv8-seg model: Use the EfficientViT backbone in all YOLOv8-seg backbones. EfficientViT uses the EfficientViT block as a basic module, each of which consists of a sandwich layout and a cascaded group attention (CGA), and further redistributes parameters to achieve a more efficient quantitative trade-off. This basic module reduces the use of attention and alleviates the problem of memory access time consumption caused by attention calculation. At the same time, a layer of DWConv is added before each FFN as local information interaction and helps introduce inductive bias. The performance of the model is improved by an innovative multi-scale linear attention mechanism while reducing the computational cost. The attention mechanism is optimized to make it more suitable for hardware implementation, and can achieve fast image processing on a variety of hardware platforms, including mobile CPUs, edge GPUs, and cloud GPUs. Compared with traditional high-resolution dense prediction models, EfficientViT greatly improves computational efficiency while maintaining high performance.

[0009] EfficientViT uses a novel multi-scale linear attention mechanism, which is designed to improve the efficiency and effectiveness of the model when processing high-resolution images. Unlike traditional high-resolution dense prediction models, EfficientViT achieves global receptive field and multi-scale learning through lightweight and hardware-efficient operations, which helps reduce computational costs. Global receptive field and multi-scale learning are achieved through simplified operations, which is especially important for high-resolution dense prediction. The lightweight and hardware-efficient operations in EfficientViT mainly refer to the simplified attention mechanism and convolution operations used in the model, which enable EfficientViT to run efficiently on a variety of hardware platforms. The model achieves the goal of maintaining model performance while significantly reducing computational complexity by using a feed-forward network with multi-scale linear attention and deep convolution, and avoiding the computationally expensive Softmax function in the attention module. These operations include using a multi-scale linear attention mechanism to replace the traditional Softmax attention, and using depthwise separable convolution to reduce parameters and computation.

[0010] The specific situation in the step 2 is as follows: A new efficient multi-scale attention EMA without dimension reduction is added to the head of YOLOV8. Only two convolution kernels will be placed in parallel sub-networks respectively. One of the parallel sub-networks is a 1x1 convolution kernel, and the other is a 3x3 convolution kernel. Part of the channel dimension is reshaped into the batch dimension to avoid some form of dimension reduction through universal convolution. In addition to building local cross-channel interactions in each parallel sub-network without channel dimension reduction, the output feature maps of the two parallel sub-networks are fused through a cross-space learning method. For any given input feature map, EMA divides the X cross-channel dimension into G sub-features for learning different semantics. In the parallel sub-network, EMA extracts the attention weight descriptors of the grouped feature maps through three parallel routes. Two 1D global average pooling operations are used to encode the channels along the two spatial directions in the 1x1 branch, and only one 3x3 kernel is stacked in the 3x3 branch to capture the multi-scale feature representation. In order to achieve different cross-channel interaction features between the two parallel paths in the 1x1 branch, we aggregate the attention maps of the two channels by a simple multiplication to get the internal of each group. On the other hand, the 3x3 branch captures local cross-channel interactions through 3x3 convolution to expand the feature space. By using global average pooling to generate channel statistics, the global spatial location information is compressed into the channel descriptor. The parallel substructure helps the network avoid more sequential processing and large depth.

[0011] The specific situation in step three is as follows: The WIOU loss function is added to the YOLOV8 algorithm, and a dynamic non-monotonic focusing mechanism is introduced. The quality is evaluated by the degree of outliers of the anchor box, and the gradient gain is dynamically adjusted. The original bbox_iou is changed, and WIoU v1 constructs an attention-based bounding box loss. WIoU v2 and WIoU v3 are based on this and add a focusing mechanism by constructing a calculation method for the gradient gain (focusing coefficient). The formula is as follows: , , , reducing the large or harmful gradients that appear in extreme samples. Pay more attention to samples of normal quality, thereby improving the generalization ability and overall performance of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 Schematic diagram of the improved YOLOV8-seg model used in the present invention; Figure 2 This is the structure diagram of EfficientViT used in the present invention; Figure 3 This is the structural diagram of the EMA attention mechanism used in the present invention; Figure 4 This is the structure diagram of the WIOU loss function used in the present invention; DETAILED DESCRIPTION

[0013] For those skilled in the art, some well-known structures and their descriptions in the drawings may be omitted. The technical solution of the present invention is further described below in conjunction with the drawings and implementation methods.

[0014] A vehicle segmentation algorithm based on YOLO in complex traffic scenes is proposed. First, the structure is changed to EfficientViT in the original network backbone of YOLOv8-seg, and a single memory-bound MHSA is used between the efficient FFN layers, thereby improving memory efficiency and enhancing channel communication. In addition, a cascaded group attention module is used to feed the attention head with different complete feature segmentations, which not only saves computational costs but also improves attention diversity; on this basis, an EMA attention module is added to reshape some channel dimensions into batch dimensions to avoid some form of dimensionality reduction through general convolution; the use of the WIoU loss function reduces the competitiveness of high-quality anchor boxes while also reducing the harmful gradients generated by low-quality examples. This allows the network to focus on anchor boxes of ordinary quality, improve the overall performance of the detector, better improve the accuracy of the algorithm and make the network more stable, and finally obtain the entire prediction result. Based on the improved YOLOv8-seg algorithm, the segmentation of road vehicles is performed at the same time to realize the study of vehicle segmentation algorithms in complex traffic scenes.

[0015] The specific implementation steps are: Step 1.1 Select a forward-view road target dataset taken by a suitable on-board camera, such as the Cityscapes dataset, and select all the pictures in the dataset that match the vehicle driving to form a new dataset.

[0016] Step 1.2: Add the EfficientViT network structure, use small convolution kernels to aggregate nearby tokens, generate multi-scale tokens, and perform ReLU-based global attention on these multi-scale tokens, reducing the computational complexity from quadratic to linear while retaining the same feature extraction capability, and being able to well combine the global receptive field with multi-scale learning.

[0017] Step 1.3 By using ReLU-based global attention, we can achieve a global receptive field and reduce the computational complexity from quadratic to linear without changing the original feature extraction capability.

[0018] Step 1.4 Enhance multi-scale learning capabilities and aggregate contextual information from nearby Q / K / V tokens to obtain multi-scale tokens. This information aggregation process is independent for each Q, K, and V in each head. In order to avoid affecting hardware efficiency, small convolution kernels are used to perform information aggregation.

[0019] Step 2.1 Use EMA-attention as the 16th, 20th, and 24th layers of the YOLOv8 backbone, retaining all information in the channel dimension to downsample the feature map.

[0020] Step 2.2 Use parallel substructures in the EMA module to help the network avoid more sequential processing and large depth. Effectively establishing short-term and long-term dependencies is conducive to achieving better performance.

[0021] Step 3.1 Calculate the outlier degree to describe the quality of the anchor box, assign a small gradient gain to it, and focus the bounding box regression on the anchor box of normal quality. Assigning a smaller gradient gain to the anchor box with a larger outlier degree will effectively prevent low-quality examples from generating large harmful gradients.

[0022] Step 3.2 The calculation formula is as follows: , , , reducing the large or harmful gradients that appear in extreme samples. Pay more attention to samples of normal quality, thereby improving the generalization ability and overall performance of the network model.

Claims

1. Research on vehicle segmentation algorithm in complex traffic scenarios based on YOLO, including the following steps. 2.Step 1: Input image data. Based on the new technology of EfficientViT, a single memory-bound MHSA is used between efficient FFN layers, thereby improving memory efficiency and enhancing channel communication. Step 2: Add the EMA attention module to the new network structure to reshape part of the channel dimension into the batch dimension to avoid some form of dimensionality reduction through general convolution. In addition to building local cross-channel interactions in each parallel sub-network without channel dimensionality reduction, the output feature maps of the two parallel sub-networks are fused through a cross-space learning method; Step 3: Add the WIoU function to the loss function. The dynamic non-monotonic focusing mechanism uses "outlier degree" instead of IoU to evaluate the quality of anchor boxes and provides a wise gradient gain allocation strategy. This strategy reduces the competitiveness of high-quality anchor boxes while also reducing the harmful gradients generated by low-quality examples.

3. According to the research on vehicle segmentation algorithm in a complex traffic scene based on YOLO in claim 1, the specific process in Step 1 is as follows: Step 1.1 Select a forward-view road target dataset taken by a suitable on-board camera, such as the Cityscapes dataset, and select all the pictures in the dataset that match the vehicle driving to form a new dataset; Step 1.2 Add the EfficientViT network structure, use small convolution kernels to aggregate nearby tokens, generate multi-scale tokens, and perform ReLU-based global attention on these multi-scale tokens, reducing the computational complexity from quadratic to linear while retaining the same feature extraction capabilities, and being able to well combine the global receptive field with multi-scale learning; Step 1.3 By using ReLU-based global attention, we can achieve a global receptive field and reduce the computational complexity from quadratic to linear without changing the original feature extraction capability; Step 1.4 Enhance multi-scale learning capabilities and aggregate contextual information from nearby Q / K / V tokens to obtain multi-scale tokens. This information aggregation process is independent for each Q, K, and V in each head. In order to avoid affecting hardware efficiency, small convolution kernels are used to perform information aggregation.

4. According to the research on vehicle segmentation algorithm in a complex traffic scene based on YOLO in claim 1, the specific process in Step 2 is as follows: Step 2.1 Use EMA-attention as the 16th, 20th, and 24th layers of the YOLOv8 backbone, retaining all information in the channel dimension to downsample the feature map; Step 2.2 Use parallel substructures in the EMA module to help the network avoid more sequential processing and deep requirements. Effectively establishing short-term and long-term dependencies is conducive to better performance.

5. According to the research on vehicle segmentation algorithm in a complex traffic scene based on YOLO in claim 1, the specific process in Step 3 is as follows: Step 3.1 Calculate the outlier degree to describe the quality of the anchor box, assign a small gradient gain to it, and focus the bounding box regression on the anchor box of normal quality. Assigning a smaller gradient gain to the anchor box with a larger outlier degree will effectively prevent low-quality examples from generating large harmful gradients; Step 3.2 The calculation formula is as follows: , , , reducing the large or harmful gradients that appear in extreme samples. Pay more attention to samples of normal quality, thereby improving the generalization ability and overall performance of the network model.

Images