Traffic cone and post recognition algorithm for assisted driving

Abstract

The present application belongs to the technical field of auxiliary driving, and particularly relates to a traffic cone and traffic column recognition algorithm for auxiliary driving, comprising the following steps: acquiring traffic cone and traffic column images; labeling the acquired images to make a data set of traffic cones and traffic columns; constructing a loss function of Sor-Yolo algorithm, using a WIoU loss function to replace a CIoU loss function in a Yolov8n algorithm; building an LGEC2f structure in the Sor-Yolo algorithm; building an MSCSPPF structure in the Sor-Yolo algorithm; setting a Sor-Yolo algorithm training environment and hyperparameters; arranging the trained Sor-Yolo algorithm into an auxiliary driving system to detect traffic cones and traffic columns, and testing the detection effect in complex weather environments. The algorithm can make the auxiliary driving system accurately recognize traffic cones and traffic columns on the road by using only a vehicle-mounted camera to shoot road pictures.

Description

Technical Field

[0001] This invention belongs to the field of driver assistance technology, specifically relating to a traffic cone and traffic post recognition algorithm for driver assistance. Background Technology

[0002] Traffic cones and bollards, as common temporary barriers, not only provide drivers with intuitive and easily understood traffic information through brightly colored warnings and clear markings, reducing the incidence of traffic accidents, but also effectively guide traffic flow, ensuring drivers follow predetermined routes and avoiding accidental entry into construction zones. Furthermore, when detours or traffic flow need to be redirected, the proper placement of traffic cones and bollards can effectively separate traffic flows, preventing congestion or chaos. When merging traffic is necessary, these facilities can also guide vehicles to merge orderly into the main lanes, ensuring smooth and safe traffic flow.

[0003] To date, the accurate identification of traffic cones and traffic bollards, as common traffic safety control facilities on roads, has become a key challenge for automotive driver assistance technology.

[0004] However, current research on traffic cone identification technology is still insufficient. Existing methods for traffic cone identification largely rely on analyzing their shape features, using lidar technology to identify key points of the traffic cone to determine its location, and then performing template matching based on the cone's contour features. However, this identification method encounters significant challenges in practical applications.

[0005] First, the high cost of LiDAR equipment significantly limits its application and widespread use in driver assistance systems. Second, template matching methods are often optimized only for traffic cones of specific shapes in specific scenarios, making it difficult to adapt to changes in the shape and size of traffic cones in reality. Furthermore, existing methods lack generalization ability when faced with complex weather changes, such as rain, snow, and fog, resulting in significantly reduced recognition performance.

[0006] Therefore, developing a traffic cone and traffic post recognition method that can overcome these challenges and has low cost, high generalization and excellent weather adaptability is of great significance for improving road traffic safety. Summary of the Invention

[0007] The purpose of this invention is to provide a traffic cone and traffic post recognition algorithm for assisted driving, so as to solve the problems existing in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a traffic cone and traffic post recognition algorithm for assisted driving, comprising the following steps:

[0009] S01: Use a vehicle-mounted camera to capture video of the road ahead containing traffic cones and traffic posts, and perform frame-by-frame operations on the video to obtain images of the traffic cones and traffic posts;

[0010] S02: Label the acquired images, create a dataset of traffic cones and traffic posts, and divide the dataset into training, validation and test sets;

[0011] S03: Construct the loss function for the Sor-Yolo algorithm, and replace the CIoU loss function in the Yolov8n algorithm with the WIoU loss function to accelerate the convergence speed of the algorithm;

[0012] S04: Construct the LGEC2f structure in the Sor-Yolo algorithm, add the GhostConv module and SGE attention mechanism to the C2f module in the Yolov8n algorithm, and lighten the algorithm without losing its accuracy;

[0013] S05: Construct the MSCSPPF structure in the Sor-Yolo algorithm and add the MSCA mechanism to the SPPF module in the Yolo8n algorithm;

[0014] S06: Set up the Sor-Yolo algorithm training environment and hyperparameters, train the Sor-Yolo algorithm using the training set, and obtain the model file with the best training effect;

[0015] S07: Deploy the trained Sor-Yolo algorithm into the driver assistance system to detect traffic cones and traffic poles, and test the detection performance in complex weather conditions.

[0016] Preferably, in S02, the acquired image data is strictly divided according to a 9:1:1 ratio to form a training set, a validation set, and a test set.

[0017] Preferably, in S03, the Sor-Yolo algorithm uses the third version of the WIoU loss function, which is defined as follows:

[0018] WIoUv3 rL WWTov1 ,

[0019]

[0020] This represents the IoU loss of the current anchor frame. denoted by , represents the mean of the anchor box IoU loss; IoU loss represents the intersection-union ratio (IoU) between the predicted box and the ground truth box; α and δ are the hyperparameters of the IoU loss function; L WIoUv1 The definition is as follows:

[0021] L wIoUv1 =R WIoU L IoU

[0022]

[0023] L IoU =1-IoU

[0024] WIoU loss through R WIoU The shape of the anchor frame is controlled, and β measures the degree of anomaly in the current anchor frame; due to Since it's dynamic, the criteria for defining anchor boxes using WIoU are also dynamic, thus enabling a dynamic measurement of the anomaly level of each predicted box. Furthermore, the WIoU loss at the very beginning... It equals 1.

[0025] Preferably, in S04, the GhostConv module divides the ordinary convolutional layer into two parts. The first part consists of only a limited number of ordinary convolutions, while the second part is a series of inexpensive linear operations. After the feature map enters the GhostConv module, the first part uses ordinary convolutions to form a small number of feature maps, while the second part applies linear operations to these feature maps to generate more feature maps. These two feature maps are then concatenated along the channel dimension to form the final output. After the two GhostConv modules, an SGE attention mechanism is connected to eliminate the accuracy loss caused by the algorithm's lightweight design. The SGE attention mechanism groups the input feature map along the channel dimension and uses a spatial averaging function F within each group. gp (X) Statistical Feature Map: Global features for each channel. SGE uses global features to generate an importance coefficient g to measure the similarity between a channel feature within a group and the global feature. The formula is shown below:

[0026]

[0027] Where, x i This represents the pixel values ​​within the same group and channel, where m is equal to the number of pixels in the feature map. Multiple importance coefficients g are summed with the feature maps of each channel within the group through dot products to form an attention weight map.

[0028] Preferably, in S06, the Sor-Yolo algorithm uses the AdamW optimizer during training, with a base learning rate of 0.01, a learning rate momentum of 0.937, and a weight decay coefficient of 0.0005. The algorithm training batch size is set to 18, and the algorithm training epochs are set to 100. During the algorithm training process, Mosaic data augmentation is used and then turned off in the last 10 epochs.

[0029] The beneficial effects of this invention are as follows: This invention proposes the Sor-Yolo algorithm, which enables assisted driving systems to accurately identify traffic cones and bollards on the road using only images captured by an onboard camera. More importantly, this invention is not limited to specific scenarios but demonstrates broad adaptability to complex and ever-changing road environments, thereby significantly improving the safety and reliability of assisted driving systems.

[0030] This invention trains the proposed Sor-Yolo algorithm for recognizing traffic cones and bollards. The final trained model achieves an mAP50 value of 89.1%, with computational complexity reduced to 7.3 GFLOPs and parameter count reduced to 2.2 MB. Compared to the YOLOv8n algorithm, the mAP50 value is improved by 2.4%; computational complexity is reduced by approximately 11.0%, and parameter count is reduced by approximately 15.3%, accelerating the model's inference speed while balancing the real-time performance and accuracy required by driver assistance technologies. Attached Figure Description

[0031] Figure 1 This is a flowchart of the present invention;

[0032] Figure 2 This is a structural diagram of the Sor-Yolo algorithm in this invention;

[0033] Figure 3 This is a structural diagram of the LGEC2f module in this invention;

[0034] Figure 4 This is a structural diagram of the GhostConv module in this invention;

[0035] Figure 5 This is a structural diagram of the SGE attention mechanism in this invention;

[0036] Figure 6 This is a structural diagram of the MSCSPPF module in this invention;

[0037] Figure 7 This is a structural diagram of the MSCA mechanism in this invention;

[0038] Figure 8 This is a diagram showing the detection results of the Sor-Yolo algorithm in this invention;

[0039] Figure 9 This image shows the detection performance of the Sor-Yolo algorithm in complex weather conditions as described in this invention. Detailed Implementation

[0040] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings and preferred embodiments.

[0041] like Figure 1As shown, this invention discloses a traffic cone and traffic post recognition algorithm for assisted driving, comprising the following steps:

[0042] S01: Use a vehicle-mounted camera to capture video of the road ahead containing traffic cones and traffic posts, and perform frame-by-frame operations on the video to obtain images of the traffic cones and traffic posts.

[0043] Videos containing traffic cones and poles in front of vehicles were captured using a vehicle-mounted camera. These videos were then frame-trimmed, with an average of one image of a traffic cone and pole every 20 frames. These images were then filtered to remove unreasonable or highly distorted images. A total of 2420 images containing traffic cones and poles were ultimately selected.

[0044] S02: Label the acquired images to create a dataset of traffic cones and traffic posts, and divide the dataset into training, validation and test sets.

[0045] The acquired image data was strictly divided in a 9:1:1 ratio to form training, validation, and test sets. First, a series of data augmentation operations were performed on the training set images to enhance the model's generalization ability. These operations included scaling, cropping, rotation, random brightness transformation, and random noise addition. Then, the Labelimg tool was used to annotate the validation set and the data-augmented training set images, generating corresponding .txt annotation files. These files detailed the location and category information of traffic cones and pillars in the images. Finally, the annotated training set, validation set, and unannotated test set were combined to form the dataset used to train the Sor-Yolo algorithm.

[0046] S03: Construct the loss function for the Sor-Yolo algorithm, and replace the CIoU loss function in the Yolov8n algorithm with the WIoU loss function to accelerate the convergence speed of the algorithm.

[0047] The Yolov8n object detection algorithm includes two types of loss functions: classification loss (Cls) and bounding box regression loss. The bounding box regression loss includes CIoU loss and DFL loss. Therefore, the total loss of Yolov8n is defined as:

[0048] L total = 7.5 × CIoU + 1.5 × DFL + 0.5 × Cls

[0049] The overall structure of the Sor-Yolo algorithm proposed in this invention is as follows: Figure 2As shown. In the loss function part, the Sor-Yolo algorithm replaces the CIoU loss function in the Yolov8n algorithm's bounding box loss with the WIoU loss function. There are three versions of the WIoU loss function, all derived from considerations of the static focusing mechanisms in the Focal and EIoU loss functions. The WIoU loss function uses a dynamic focusing mechanism to measure the anomaly degree of each predicted box, making the network focus more on predicted boxes of normal quality rather than low-quality or high-quality predicted boxes.

[0050] The Sor-Yolo algorithm proposed in this invention uses the third version of the WIoU loss function, which is defined as follows:

[0051] WIoUv3 =rL WToUv1 ,

[0052]

[0053] This represents the IoU loss of the current anchor frame. This represents the mean of the anchor box IoU loss. IoU loss represents the intersection-over-union ratio (IoU) between the predicted and ground truth boxes. α and δ are hyperparameters of the IoU loss function. WIoUv1 The definition is as follows:

[0054] L WIoUv1 =R WIoU L IoU

[0055]

[0056] L IoU =1-IoU

[0057] WIoU loss through R WIoU The shape of the anchor frame is controlled, and β measures the degree of anomaly in the current anchor frame. Because... Since it's dynamic, the criteria for defining anchor boxes using WIoU are also dynamic, thus allowing for a dynamic measurement of the anomaly level of each predicted box. Furthermore, the WIoU loss at the beginning... It equals 1.

[0058] S04: Construct the LGEC2f structure in the Sor-Yolo algorithm, add the GhostConv module and SGE attention mechanism to the C2f module in the Yolov8n algorithm, and lighten the algorithm without sacrificing its accuracy.

[0059] In the YOLOv8n algorithm, the C2f module enhances the gradient flow, allowing the algorithm to obtain richer feature representations. Although the C2f module makes significant efforts in pursuing lightweight design, there is still room for improvement.

[0060] Therefore, the Sor-Yolo algorithm proposed in this invention integrates the GhostConv module and the SGE attention mechanism into the C2f module, forming a lightweight grouping enhancement C2f module, namely the LGEC2f module, with the structure as follows: Figure 3 As shown, this module can reduce the computational load and number of parameters in the Yolov8n algorithm without sacrificing accuracy.

[0061] The Sor-Yolo algorithm uses two GhostConv modules and one SGE attention mechanism in the LGEC2f module. The GhostConv module structure is as follows: Figure 4 As shown, the structure of the SGE attention mechanism is as follows: Figure 5 As shown, the GhostConv module divides a regular convolutional layer into two parts. The first part consists of only a limited number of regular convolutions, while the second part uses a series of inexpensive linear operations. After the feature map enters the GhostConv module, the first part uses regular convolutions to form a small number of feature maps, while the second part applies linear operations to these feature maps to generate more feature maps. These two parts of feature maps are then concatenated along the channel dimension to form the final output. By using the GhostConv module, the Sor-Yolo algorithm can reduce the significant computational and parameter requirements of regular convolutional layers, thus achieving algorithm lightweighting.

[0062] A SGE attention mechanism is connected after the two GhostConv modules to mitigate the accuracy loss caused by algorithm lightweighting. The SGE attention mechanism groups the input feature maps along the channel dimension and applies a spatial averaging function F within each group. gp (X) Statistical Feature Map: Global features for each channel. SGE uses global features to generate an importance coefficient g to measure the similarity between a channel feature within a group and the global feature. The formula is shown below:

[0063]

[0064] Among these, x i This represents the pixel values ​​within the same group and channel, where m equals the number of pixels in the feature map. Multiple importance coefficients g are summed with the feature maps of each channel within the group through dot products to form an attention weight map.

[0065] The attention weight map, after normalization and the application of the sigmoid function, is applied to the original group input to obtain the final output. The SGE attention mechanism introduces very few parameters and computational costs by grouping the feature map along the channel dimension, and processes each group in parallel, thereby improving the accuracy of the algorithm in recognizing traffic cones and traffic posts.

[0066] S05: Build the MSCSPPF structure in the Sor-Yolo algorithm and add the MSCA mechanism to the SPPF module in the YOLO8n algorithm to improve the detection accuracy of traffic cones and traffic posts.

[0067] In the YOLOv8n algorithm, the convolution module specifications are mostly 3×3 or 1×1 grid convolutions. However, the shapes of traffic cones and traffic posts are approximately long and thin strips, making it difficult for grid convolutions to accurately capture the shape features of traffic cones and traffic posts.

[0068] The Sor-Yolo algorithm proposed in this invention introduces the MSCA mechanism into the SPPF module of the Yolov8n algorithm as a supplement to standard grid convolution. The MSCA mechanism approximates a standard depthwise separable convolution with a large kernel by using multiple depthwise separable strip convolutions, thereby improving the algorithm's ability to recognize strip-shaped objects. However, since the input feature map size of the SPPF module is 20×20×512, and the depthwise separable strip convolution parameters in the MSCA module are too large, Sor-Yolo modifies the internal parameters of the MSCA mechanism, ultimately forming the MSCSPPF module, the module structure of which is as follows: Figure 6 As shown.

[0069] The structure of the MSCA mechanism is as follows: Figure 7 As shown, in this module, the feature map is processed through a 5×5 depthwise separable grid convolution to aggregate local image information, and then processed through four branches. One branch is left unprocessed, while the other three branches are each processed through two depthwise separable strip convolutions to capture thin strip-like targets in the image. The strip convolution sizes for the three branches are set to 3, 7, and 11, respectively. Finally, the outputs of the four branches are summed and multiplied by the weights formed by a 1×1 grid convolution to obtain the final output.

[0070] S06: Set up the Sor-Yolo algorithm training environment and hyperparameters, train the Sor-Yolo algorithm using the training set, and obtain the model file with the best training effect.

[0071] To train the Sor-Yolo algorithm proposed in this invention, an environment with PyTorch version 2.0.0 and CUDA version 11.8 was built on the Ubuntu 20.04 operating system. The processor of the operating system was an Intel(R) Xeon(R) Platinum 8352V CPU@2.10GHz, and the graphics card was an NVIDIA GeForce RTX 4090.

[0072] During training, the Sor-Yolo algorithm used the AdamW optimizer with a base learning rate of 0.01, a learning rate momentum of 0.937, and a weight decay factor of 0.0005. The training batch size was set to 18, and the number of training epochs was set to 100. Mosaic data augmentation was used during training, but it was disabled in the last 10 epochs.

[0073] S07: Deploy the trained Sor-Yolo algorithm into the driver assistance system to detect traffic cones and traffic poles, and test the detection performance in complex weather conditions.

[0074] After obtaining the Sor-Yolo algorithm model with the best training performance, the model was tested, and the test results are as follows: Figure 8 As shown. To verify the detection performance under complex weather conditions, the Sor-Yolo algorithm was also tested in complex weather environments. The test results are as follows. Figure 9 As shown.

[0075] It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention.

Claims

1. A traffic cone and traffic post recognition algorithm for assisted driving, characterized in that, Includes the following steps: S01: Use a vehicle-mounted camera to capture video of the road ahead containing traffic cones and traffic posts, and perform frame-by-frame operations on the video to obtain images of the traffic cones and traffic posts; S02: Label the acquired images, create a dataset of traffic cones and traffic posts, and divide the dataset into training, validation and test sets; S03: Construct the loss function for the Sor-Yolo algorithm, and replace the CIoU loss function in the YOlOv8n algorithm with the WIoU loss function to accelerate the convergence speed of the algorithm; S04: Construct the LGEC2f structure in the Sor-Yolo algorithm, add the GhostConv module and SGE attention mechanism to the C2f module in the YOlOv8n algorithm, and lighten the algorithm without sacrificing its accuracy; S05: Build the MSCSPPF structure in the Sor-Yolo algorithm and add the MSCA mechanism to the SPPF module in the YOlOv8n algorithm; S06: Set up the Sor-Yolo algorithm training environment and hyperparameters, train the Sor-Yolo algorithm using the training set, and obtain the model file with the best training effect; S07: Deploy the trained Sor-Yolo algorithm into the driver assistance system to detect traffic cones and traffic poles, and test the detection performance in complex weather conditions; In S04, the GhostConv module divides the ordinary convolutional layer into two parts. The first part consists of only a limited number of ordinary convolutions, while the second part uses a series of inexpensive linear operations. After the feature map enters the GhostConv module, the first part uses ordinary convolutions to form a small number of feature maps, while the second part applies linear operations to these feature maps to generate more feature maps. These two feature maps are then concatenated along the channel dimension to form the final output. Following the two GhostConv modules is an SGE attention mechanism to mitigate the accuracy loss caused by the algorithm's lightweight design. The SGE attention mechanism groups the input feature map along the channel dimension and uses a spatial averaging function within each group. The statistical feature map shows the global features of each channel. SGE uses these global features to generate importance coefficients. The formula for measuring the similarity between a channel feature and the global feature within a group is shown below: ； in, This represents the pixel values ​​within the same group and channel. This equals the number of pixels in the feature map, and multiple importance coefficients are considered. The attention weight map is formed by summing the feature maps of each channel within the group through dot products.

2. The traffic cone and traffic post recognition algorithm for assisted driving according to claim 1, characterized in that, In S02, the acquired image data is divided into a training set, a validation set, and a test set in a 9:1:1 ratio.

3. The traffic cone and traffic post recognition algorithm for assisted driving according to claim 1, characterized in that, In S03, the Sor-Yolo algorithm uses the third version of the WIoU loss function, which is defined as follows: ； This represents the IoU loss of the current anchor frame. This represents the mean of the anchor box IoU loss; the IoU loss represents the intersection-over-union ratio of the predicted box and the ground truth box. and These are the hyperparameters of the WIoU loss function; The definition is as follows: ； WIoU loss through Control the shape of the anchor frame. Measure the degree of anomaly in the current anchor frame; due to It is dynamic, so WIoU's criteria for dividing anchor boxes are also dynamic, thus enabling it to dynamically measure the degree of anomaly of each predicted box. In addition, WIoU lost money at the very beginning. It equals 1.

4. The traffic cone and traffic post recognition algorithm for assisted driving according to claim 1, characterized in that, In S06, the Sor-Yolo algorithm uses the AdamW optimizer during training, with a base learning rate of 0.01, a learning rate momentum of 0.937, a weight decay coefficient of 0.0005, a training batch size of 18, and a training epoch size of 100. Mosaic data augmentation is used during training and is turned off in the last 10 epochs.

Images