A traffic target detection method based on PES-PointPillars

Abstract

This invention belongs to the field of intelligent traffic perception and autonomous driving environmental perception technology, and discloses a traffic target detection method based on PES-PointPillars. First, a traffic target LiDAR point cloud dataset is acquired and divided into training, validation, and test sets. Based on the PointPillars 3D target detection framework, a windmill-shaped convolutional structure is introduced to improve the feature extraction capability of the backbone network, and an enhanced inter-layer feature association module is constructed. Multi-layer feature fusion is achieved through grouped feature focusing and multi-level feature reconstruction. During training, a SNWD bounding box regression loss function is designed to improve the localization accuracy of small targets. After training, the point cloud data is input into the model for detection, and the traffic target category and 3D bounding box results are output. This invention can improve the feature representation capability of sparse point clouds and enhance detection accuracy and robustness.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic perception and autonomous driving environment perception technology, and particularly relates to a three-dimensional traffic target detection method based on lidar point cloud data. Specifically, it is a traffic target detection method based on PES-PointPillars, which can be used for automatic identification and positioning of traffic participants such as vehicles, pedestrians and cyclists in autonomous driving systems. Background Technology

[0002] Currently, the PointPillars algorithm is widely used in autonomous driving 3D target detection tasks because it can convert 3D point clouds into 2D pseudo-images and process them using 2D convolutional networks, achieving a good balance between detection speed and accuracy. However, this method still has some shortcomings in complex traffic scenarios: First, LiDAR point clouds have sparse and irregular distribution characteristics. At long distances or under occlusion, small targets contain only a few effective points, which can easily lead to insufficient feature representation. Second, the original network uses a standard convolutional structure, which is difficult to fully adapt to the spatial structure characteristics of sparse point clouds. Third, the multi-layer feature fusion method is relatively simple, and the interaction ability between features at different levels is insufficient, which can easily cause the loss of feature information of small targets. In addition, the traditional bounding box regression loss function is difficult to measure the spatial difference between the predicted box and the ground truth box from the perspective of the overall geometric relationship, thus affecting the target localization accuracy.

[0003] Therefore, it is necessary to design a traffic target detection method that can enhance the feature extraction capability of sparse point clouds, improve the multi-layer feature fusion effect, and optimize the target localization accuracy, so as to improve the accuracy and robustness of 3D target detection in autonomous driving scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a traffic target detection method based on PES-PointPillars to solve the problems of insufficient ability of existing PointPillars algorithms to extract features of small targets in sparse point cloud environments, limited effect of multi-layer feature fusion, and low target localization accuracy, thereby improving the accuracy and robustness of traffic target detection in autonomous driving scenarios.

[0005] To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0006] A traffic target detection method based on PES-PointPillars includes the following steps:

[0007] Step 1) Obtain traffic target point cloud dataset: Obtain LiDAR point cloud dataset for traffic target detection. The dataset contains multiple frames of road scene point cloud data and their corresponding 3D target annotation information. Target categories include vehicles, pedestrians and cyclists.

[0008] Step 2) Divide the dataset: Divide the traffic target point cloud dataset into training set, validation set and test set according to a preset ratio for subsequent model training and performance evaluation.

[0009] Step 3) Based on the PointPillars 3D object detection network, its backbone network structure and feature fusion module are improved to construct the PES-PointPillars traffic object detection model. First, the input 3D point cloud data is divided into columnar units and converted into a 2D pseudo-image representation through a point cloud encoding network. Then, the improved backbone network is used to extract pseudo-image features, and the information interaction between features at different levels is enhanced through a multi-layer feature fusion structure to obtain a richer spatial feature representation of the point cloud.

[0010] Step 3.1) Improvement of the backbone network based on windmill-shaped convolution

[0011] In the PointPillars algorithm, the original point cloud is first converted into a two-dimensional pseudo-image representation through columnar partitioning and point feature encoding, and then feature extraction is performed through a two-dimensional convolutional neural network. However, the pseudo-image generated from the LiDAR point cloud has obvious sparsity, with a large number of grid regions being null values ​​or background information. Traditional square convolutional kernels are easily interfered with by invalid regions when extracting sparse point cloud structural features, thus affecting the detection effect of small targets such as pedestrians and cyclists.

[0012] To enhance the network's ability to model the structural features of sparse point clouds, a windmill-shaped convolutional structure is introduced into the PointPillars backbone network to replace some standard convolutional layers. The windmill-shaped convolution expands the receptive field of features in both the horizontal and vertical directions through multiple directional convolutional branches, enabling the network to better capture directional structural information in sparse point clouds.

[0013] Let the input feature map be The windmill-shaped convolution performs parallel convolution on the input features through four directional convolution branches. Its calculation process can be represented as follows:

[0014]

[0015] in, These represent convolution operations in different directions. Indicates batch normalization. This represents the activation function.

[0016] The features obtained from the four directional convolutional branches are concatenated along the channel dimension and then fused through a convolutional layer to obtain the final output features:

[0017]

[0018] in, , , , These represent the four safe convolutional branches. For convolution operations, For merging and splicing operations.

[0019] Through the above structure, the windmill-shaped convolution decomposes the feature extraction process of traditional convolution into feature perception in multiple directions, enabling the network to expand the feature receptive field while maintaining low computational complexity. This allows it to better adapt to the spatial distribution characteristics of sparse point clouds and improve the network's ability to express features of small targets such as pedestrians and cyclists.

[0020] Step 3.2) Enhance inter-layer feature association module

[0021] In the PointPillars network, the features output from different layers of the backbone network exhibit significant differences in semantic information and spatial details. Deeper features possess stronger semantic expressive power but weaker spatial detail; shallower features, while containing more geometric structural details, also include more background noise. When feature fusion is performed using only deconvolution upsampling and channel concatenation, it is difficult to fully utilize the complementary relationships between features from different layers, thus affecting the detection performance of small targets.

[0022] To enhance the interaction between deep semantic information and shallow geometric information, an enhanced inter-layer feature association module is introduced after deconvolutional upsampling in the backbone network. This module consists of two branches: a grouped feature focusing unit and a multi-level feature reconstruction module. By performing association enhancement and reconstruction processing on features at different levels, it enhances the semantic information expression capability while preserving key geometric details, thereby improving the network's feature representation capability for small targets in sparse point clouds.

[0023] Step 3.2.1) Grouping Feature Focusing Units

[0024] Grouped feature focusing units are used to enhance the correlation between features from different layers and highlight key feature information through spatial attention mechanisms; let the features from two adjacent layers of the backbone network be respectively and First, the two layers of features are added element-wise to obtain the initial fused features:

[0025]

[0026] Spatial weights are then generated through convolution operations and utilized... The function is normalized to obtain the spatial weight map:

[0027]

[0028] in This indicates an element-wise addition operation, used to achieve preliminary fusion of features from different layers;

[0029] Spatial weights are applied to the fused features to obtain features that contain spatial attention information:

[0030]

[0031] in, This indicates element-major multiplication; to further enhance the feature correlation between channels, the features... The feature is divided into multiple feature groups along the channel dimension, and feature interaction processing is performed within each feature group; the features of each group are then reassembled to obtain the aggregated feature. Its expression is as follows:

[0032]

[0033] Finally, the aggregated features are input into the multi-layer original feature fusion and normalization module for spatial normalization processing to obtain the final output features:

[0034]

[0035] in This indicates that the mean of the feature is calculated. This indicates that the standard deviation of the feature is calculated;

[0036] Through the above process, grouped feature focusing units can enhance the feature correlation between different channels and highlight key spatial areas;

[0037] Step 3.2.2) Multi-level feature reconstruction module

[0038] The multi-level feature reconstruction module is used to reduce feature redundancy in deep networks and retain more target-related information during feature fusion; let the input features be:

[0039]

[0040]

[0041] in, and They represent the features of adjacent layers respectively. and The generated weight coefficients are used to characterize the importance of features at different layers; This indicates a batch normalization operation, used to normalize features to stabilize the training process; Used to map the output to the 0-1 range, thereby obtaining the weighting coefficients;

[0042] The two layers of features are then added element-wise to obtain the fused features. An adaptive threshold is generated using global average pooling and learnable bias parameters. :

[0043]

[0044] in, This represents the global average pooling operation, used to extract the overall response information of features; This represents a learnable bias parameter used to adjust the range of threshold distribution.

[0045] Based on the above thresholds, a soft gating mechanism is constructed to generate weight masks for strong and weak features:

[0046]

[0047]

[0048] in, This represents a strong feature weight mask, used to highlight important feature information; This represents a weak feature weight mask, used to preserve auxiliary feature information; This represents the weight coefficients generated from the input features.

[0049] After obtaining the soft mask, different weights are mapped to the fused features. Above, and generate strong features respectively. With weak features :

[0050]

[0051]

[0052] Among them, strong features Feature enhancement is performed through convolutional layers to address weak features. Multi-level feature reconstruction is achieved through feature transformation units.

[0053] Step 3.2.3) Module Output Fusion

[0054] After obtaining strong and weak features, feature transformation is performed on each type of feature. For strong features, convolution operations are used to enhance details, while for weak features, structural optimization is performed through feature transformation units. Finally, the two sets of features are fused to obtain the final output of the multi-level feature reconstruction module. :

[0055]

[0056] in, These are the feature fusion weight coefficients; This represents weak features after feature transformation. This indicates a strong feature after feature enhancement processing.

[0057] Through the above process, the enhanced inter-layer feature association module can retain key target information while fusing features from different layers, reduce redundant feature interference, and improve the network's ability to detect small targets in sparse point cloud environments.

[0058] Step 4.1) Construct the SNWD bounding box regression loss function

[0059] In PointPillars networks, the original bounding box regression typically uses the SmoothL1 loss function to evaluate the 3D bounding box parameters. Supervised learning is performed; to improve the localization capability of small targets, the predicted bounding boxes and the ground truth bounding boxes are represented as two-dimensional Gaussian distributions, and the difference between the two distributions is measured by the Wasserstein distance. Let the distribution corresponding to the predicted box be... The distribution of the real bounding boxes is as follows Its second-order Wasserstein distance is expressed as:

[0060]

[0061] in, and These represent the center coordinates of the predicted bounding box and the actual bounding box, respectively.

[0062] Substituting the bounding box parameters from the 3D detection task into the above formula yields a distance expression suitable for traffic target detection tasks:

[0063]

[0064] in, and Indicates the center coordinates of the predicted bounding box; and Represents the center coordinates of the actual bounding box; and These represent the width and length of the predicted bounding box, respectively. and These represent the width and length of the actual bounding box, respectively.

[0065] To convert this distance metric into a similarity index, it is exponentially normalized to obtain the normalized Wasserstein distance:

[0066]

[0067] in, This is a constant used to control the scale sensitivity of the distance metric. The exponential function is represented by the above metric, and a planar localization loss function is constructed based on this:

[0068]

[0069] This loss function can still provide effective gradients when the predicted bounding box does not overlap with the ground truth bounding box, thus improving the localization accuracy in small object detection tasks.

[0070] Step 4.2) Construct the SNWD bounding box regression loss function

[0071] For height parameter and orientation angle parameters Optimization is performed using the SmoothL1 loss function:

[0072]

[0073] in, Represents the regression loss for height and orientation parameters; Indicates bounding box parameters; This represents the difference between the predicted value and the actual value.

[0074] Subsequently, the planar positioning loss was... With height and angle loss By performing a weighted combination, we obtain the SNWD regression loss function:

[0075]

[0076] in, and These are the weighting coefficients. Based on this, the classification loss is combined... With direction classification loss The overall training loss function of the network is obtained as follows:

[0077]

[0078] in, , and These represent the weight coefficients for regression loss, classification loss, and orientation loss, respectively.

[0079] The network parameters are optimized using the above training method. Average precision (AP), intersection-to-union ratio (IoU), and frame rate (FPS) are used as performance evaluation metrics for the model, as shown in the following expressions:

[0080]

[0081]

[0082] in Indicates accuracy. Indicates recall rate; This indicates that the true class is the i-th class and the predicted result is also the i-th class. The number of samples in each class; Indicates the true category is the first Class but predicted as the first The number of samples in each class; Indicates the true category is the first Class but predicted as the first The number of samples in each class; This represents the total number of categories.

[0083] Step 5) Input the LiDAR point cloud data of the scene to be detected into the trained PES-PointPillars traffic target detection model to obtain the road scene image to be detected, perform feature extraction and target detection on the point cloud data, and output the category information of traffic targets and the 3D bounding box detection results.

[0084] The beneficial effects of this invention are as follows:

[0085] 1. A windmill-shaped convolutional structure is introduced into the PointPillars backbone network. Features are extracted in parallel through convolutional branches in multiple directions, enabling the network to perceive the spatial structure information of point clouds in different directions. While expanding the receptive field of features, it effectively reduces the interference of background noise in sparse point clouds, thereby enhancing the network's feature extraction ability for small targets such as pedestrians and cyclists and improving the accuracy of traffic target detection.

[0086] 2. An enhanced inter-layer feature association module was designed. By using grouped feature focusing units and multi-level feature reconstruction modules to jointly model features at different levels, the deep semantic information and shallow spatial detail information are effectively integrated, the information interaction between features at different levels is strengthened, feature redundancy is reduced and key target information is retained, thereby improving the network's ability to detect small targets in complex traffic scenarios.

[0087] 3. The proposed SNWD bounding box regression loss function represents the predicted bounding box and the true bounding box as a probability distribution, and measures the difference between the two by the Wasserstein distance. It can still provide an effective gradient when the predicted box and the true box do not overlap, and optimizes the bounding box regression process from the perspective of the overall geometric relationship, thereby significantly improving the positioning accuracy of traffic targets. Attached Figure Description

[0088] Figure 1 This is a flowchart illustrating the method of the present invention.

[0089] Figure 2 This is a model structure diagram of the PES-PointPillars method of the present invention.

[0090] Figure 3 This is a diagram of a windmill-shaped convolutional structure.

[0091] Figure 4 To enhance the interlayer feature association module diagram.

[0092] Figure 5 This is a block diagram of the grouped feature focusing unit.

[0093] Figure 6 This is a diagram of a multi-level feature reconstruction module.

[0094] Figure 7 This is a diagram showing the detection results of the present invention. Detailed Implementation

[0095] This invention provides a traffic target detection method based on PES-PointPillars, which specifically includes the following steps:

[0096] Step 1) Obtain traffic target point cloud dataset: Obtain LiDAR point cloud dataset for traffic target detection. The dataset contains multiple frames of road scene point cloud data and their corresponding 3D target annotation information. Target categories include vehicles, pedestrians and cyclists.

[0097] Step 2) Divide the dataset: Divide the traffic target point cloud dataset into training set, validation set and test set according to a preset ratio for subsequent model training and performance evaluation.

[0098] Step 3) Based on the PointPillars 3D object detection network, its backbone network structure and feature fusion module are improved to construct the PES-PointPillars traffic object detection model. First, the input 3D point cloud data is divided into columnar units and converted into a 2D pseudo-image representation through a point cloud encoding network. Then, the improved backbone network is used to extract pseudo-image features, and the information interaction between features at different levels is enhanced through a multi-layer feature fusion structure to obtain a richer spatial feature representation of the point cloud.

[0099] Step 3.1) Improvement of the backbone network based on windmill-shaped convolution

[0100] In the PointPillars algorithm, the original point cloud is first converted into a two-dimensional pseudo-image representation through columnar partitioning and point feature encoding, and then feature extraction is performed through a two-dimensional convolutional neural network. However, the pseudo-image generated from the LiDAR point cloud has obvious sparsity, with a large number of grid regions being null values ​​or background information. Traditional square convolutional kernels are easily interfered with by invalid regions when extracting sparse point cloud structural features, thus affecting the detection effect of small targets such as pedestrians and cyclists.

[0101] To enhance the network's ability to model the structural features of sparse point clouds, a windmill-shaped convolutional structure is introduced into the PointPillars backbone network to replace some standard convolutional layers. The windmill-shaped convolution expands the receptive field of features in both the horizontal and vertical directions through multiple directional convolutional branches, enabling the network to better capture directional structural information in sparse point clouds.

[0102] Let the input feature map be The windmill-shaped convolution performs parallel convolution on the input features through four directional convolution branches. Its calculation process can be represented as follows:

[0103]

[0104] in, These represent convolution operations in different directions. Indicates batch normalization. This represents the activation function.

[0105] The features obtained from the four directional convolutional branches are concatenated along the channel dimension and then fused through a convolutional layer to obtain the final output features:

[0106]

[0107] in, , , , These represent the four safe convolutional branches. For convolution operations, For merging and splicing operations.

[0108] Through the above structure, the windmill-shaped convolution decomposes the feature extraction process of traditional convolution into feature perception in multiple directions, enabling the network to expand the feature receptive field while maintaining low computational complexity. This allows it to better adapt to the spatial distribution characteristics of sparse point clouds and improve the network's ability to express features of small targets such as pedestrians and cyclists.

[0109] Step 3.2) Enhance inter-layer feature association module

[0110] In the PointPillars network, the features output from different layers of the backbone network exhibit significant differences in semantic information and spatial details. Deeper features possess stronger semantic expressive power but weaker spatial detail; shallower features, while containing more geometric structural details, also include more background noise. When feature fusion is performed using only deconvolution upsampling and channel concatenation, it is difficult to fully utilize the complementary relationships between features from different layers, thus affecting the detection performance of small targets.

[0111] To enhance the interaction between deep semantic information and shallow geometric information, an enhanced inter-layer feature association module is introduced after deconvolutional upsampling in the backbone network. This module consists of two branches: a grouped feature focusing unit and a multi-level feature reconstruction module. By performing association enhancement and reconstruction processing on features at different levels, it enhances the semantic information expression capability while preserving key geometric details, thereby improving the network's feature representation capability for small targets in sparse point clouds.

[0112] Step 3.2.1) Grouping Feature Focusing Units

[0113] Grouped feature focusing units are used to enhance the correlation between features from different layers and highlight key feature information through spatial attention mechanisms; let the features from two adjacent layers of the backbone network be respectively and First, the two layers of features are added element-wise to obtain the initial fused features:

[0114]

[0115] Spatial weights are then generated through convolution operations and utilized... The function is normalized to obtain the spatial weight map:

[0116]

[0117] in This indicates an element-wise addition operation, used to achieve preliminary fusion of features from different layers;

[0118] Spatial weights are applied to the fused features to obtain features that contain spatial attention information:

[0119]

[0120] in, This indicates element-major multiplication; to further enhance the feature correlation between channels, the features... The feature is divided into multiple feature groups along the channel dimension, and feature interaction processing is performed within each feature group; the features of each group are then reassembled to obtain the aggregated feature. Its expression is as follows:

[0121]

[0122] Finally, the aggregated features are input into the multi-layer original feature fusion and normalization module for spatial normalization processing to obtain the final output features:

[0123]

[0124] in This indicates that the mean of the feature is calculated. This indicates that the standard deviation of the feature is calculated;

[0125] Through the above process, grouped feature focusing units can enhance the feature correlation between different channels and highlight key spatial areas;

[0126] Step 3.2.2) Multi-level feature reconstruction module

[0127] The multi-level feature reconstruction module is used to reduce feature redundancy in deep networks and retain more target-related information during feature fusion; let the input features be:

[0128]

[0129]

[0130] in, and They represent the features of adjacent layers respectively. and The generated weight coefficients are used to characterize the importance of features at different layers; This indicates a batch normalization operation, used to normalize features to stabilize the training process; Used to map the output to the 0-1 range, thereby obtaining the weighting coefficients;

[0131] The two layers of features are then added element-wise to obtain the fused features. An adaptive threshold is generated using global average pooling and learnable bias parameters. :

[0132]

[0133] in, This represents the global average pooling operation, used to extract the overall response information of features; This represents a learnable bias parameter used to adjust the range of threshold distribution.

[0134] Based on the above thresholds, a soft gating mechanism is constructed to generate weight masks for strong and weak features:

[0135]

[0136]

[0137] in, This represents a strong feature weight mask, used to highlight important feature information; This represents a weak feature weight mask, used to preserve auxiliary feature information; This represents the weight coefficients generated from the input features.

[0138] After obtaining the soft mask, different weights are mapped to the fused features. Above, and generate strong features respectively. With weak features :

[0139]

[0140]

[0141] Among them, strong features Feature enhancement is performed through convolutional layers to address weak features. Multi-level feature reconstruction is achieved through feature transformation units.

[0142] Step 3.2.3) Module Output Fusion

[0143] After obtaining strong and weak features, feature transformation is performed on each type of feature. For strong features, convolution operations are used to enhance details, while for weak features, structural optimization is performed through feature transformation units. Finally, the two sets of features are fused to obtain the final output of the multi-level feature reconstruction module. :

[0144]

[0145] in, These are the feature fusion weight coefficients; This represents weak features after feature transformation. This indicates a strong feature after feature enhancement processing.

[0146] Through the above process, the enhanced inter-layer feature association module can retain key target information while fusing features from different layers, reduce redundant feature interference, and improve the network's ability to detect small targets in sparse point cloud environments.

[0147] Step 4.1) Construct the SNWD bounding box regression loss function

[0148] In PointPillars networks, the original bounding box regression typically uses the SmoothL1 loss function to evaluate the 3D bounding box parameters. Supervised learning is performed; to improve the localization capability of small targets, the predicted bounding boxes and the ground truth bounding boxes are represented as two-dimensional Gaussian distributions, and the difference between the two distributions is measured by the Wasserstein distance. Let the distribution corresponding to the predicted box be... The distribution of the real bounding boxes is as follows Its second-order Wasserstein distance is expressed as:

[0149]

[0150] in, and These represent the center coordinates of the predicted bounding box and the actual bounding box, respectively.

[0151] Substituting the bounding box parameters from the 3D detection task into the above formula yields a distance expression suitable for traffic target detection tasks:

[0152]

[0153] in, and Indicates the center coordinates of the predicted bounding box; and Represents the center coordinates of the actual bounding box; and These represent the width and length of the predicted bounding box, respectively. and These represent the width and length of the actual bounding box, respectively.

[0154] To convert this distance metric into a similarity index, it is exponentially normalized to obtain the normalized Wasserstein distance:

[0155]

[0156] in, This is a constant used to control the scale sensitivity of the distance metric. The exponential function is represented by the above metric, and a planar localization loss function is constructed based on this:

[0157]

[0158] This loss function can still provide effective gradients when the predicted bounding box does not overlap with the ground truth bounding box, thus improving the localization accuracy in small object detection tasks.

[0159] Step 4.2) Construct the SNWD bounding box regression loss function

[0160] For height parameter and orientation angle parameters Optimization is performed using the SmoothL1 loss function:

[0161]

[0162] in, Represents the regression loss for height and orientation parameters; Indicates bounding box parameters; This represents the difference between the predicted value and the actual value.

[0163] Subsequently, the planar positioning loss was... With height and angle loss By performing a weighted combination, we obtain the SNWD regression loss function:

[0164]

[0165] in, and These are the weighting coefficients. Based on this, the classification loss is combined... With direction classification loss The overall training loss function of the network is obtained as follows:

[0166]

[0167] in, , and These represent the weight coefficients for regression loss, classification loss, and orientation loss, respectively.

[0168] The network parameters are optimized using the above training method. Average precision (AP), intersection-to-union ratio (IoU), and frame rate (FPS) are used as performance evaluation metrics for the model, as shown in the following expressions:

[0169]

[0170]

[0171] in Indicates accuracy. Indicates recall rate; This indicates that the true class is the i-th class and the predicted result is also the i-th class. The number of samples in each class; Indicates the true category is the first Class but predicted as the first The number of samples in each class; Indicates the true category is the first Class but predicted as the first The number of samples in each class; This represents the total number of categories.

[0172] Step 5) Input the LiDAR point cloud data of the scene to be detected into the trained PES-PointPillars traffic target detection model to obtain the road scene image to be detected, perform feature extraction and target detection on the point cloud data, and output the category information of traffic targets and the 3D bounding box detection results.

[0173] Example 1:

[0174] The traffic object detection method based on PES-PointPillars proposed in this invention was trained and tested using the publicly available autonomous driving 3D object detection dataset KITTI. The overall process is as follows: Figure 1 As shown.

[0175] Step S1: Import traffic target point cloud data and perform data preprocessing;

[0176] A LiDAR point cloud dataset for traffic target detection is acquired, and each frame of point cloud data and its corresponding 3D annotation information are read. The dataset contains three types of traffic participant targets: vehicles, pedestrians, and cyclists. The raw point cloud data is preprocessed, including point cloud coordinate range cropping, invalid point removal, and data format conversion. The dataset is then divided into training, validation, and test sets according to a preset ratio for model training, parameter tuning, and performance evaluation.

[0177] Step S2: Set up the model running environment and training parameters;

[0178] The software environment for this experiment was deployed as Ubuntu 20.04, Python 3.9.0, PyTorch 2.0.1, and the mmdetection point cloud object detection platform. The hardware environment included an NVIDIA RTX 4090D 24GB GPU and an Intel Xeon (Ice Lake) Platinum 8369B CPU, with CUDA 11.8 used for GPU acceleration. Specific experimental parameter configurations are shown in Table 1.

[0179] Table 1: Experimental Parameter Configuration Table

[0180]

[0181] Step S3: Construct a traffic target detection model based on PES-PointPillars;

[0182] The overall structure of the model is as follows Figure 2As shown. This invention uses the PointPillars 3D target detection network as its basic framework, and constructs the PES-PointPillars traffic target detection model by improving the backbone network structure and feature fusion module.

[0183] First, the input 3D point cloud data is divided into columnar units and converted into a 2D pseudo-image feature representation using a point cloud encoding network. Then, a windmill-shaped convolutional structure is introduced into the backbone network to enhance the network's ability to model the directional structural information of sparse point clouds. Its structure is as follows: Figure 3 As shown, by expanding the feature receptive field through multiple directional convolutional branches, the network can more effectively extract the spatial structure features of point clouds.

[0184] Building upon this, an enhanced inter-layer feature association module is introduced in the feature fusion stage to strengthen the information interaction between features at different levels. Its overall structure is as follows: Figure 4 As shown in the diagram. This module comprises two substructures: a grouped feature focusing unit and a multi-level feature reconstruction module. The structure of the grouped feature focusing unit is as follows: Figure 5 As shown, the structure of the multi-level feature reconstruction module is as follows: Figure 6 As shown, by performing association enhancement and reconstruction processing on multi-layer features, deep semantic information and shallow spatial detail information can be effectively integrated, thereby improving the network's ability to represent features of small targets in sparse point clouds.

[0185] During the training phase, the 3D bounding boxes predicted by the model are compared with the ground truth annotations, and the SNWD loss function is introduced to optimize the bounding box regression. This constrains the spatial difference between the predicted and ground truth boxes from the perspective of overall geometric relationships, thereby improving the localization accuracy of small targets.

[0186] Step S4: Train and test the model;

[0187] The training data obtained in step S1 is input into the constructed PES-PointPillars traffic object detection model for training, and the model hyperparameters are adjusted using the validation set. After training, the model performance is evaluated on the test set. In the experiment, average accuracy (AP), intersection-over-union ratio (IoU), and frame rate (FPS) are used to comprehensively evaluate the model's detection performance.

[0188] Table 2 Performance Comparison of Various Models on KITTI

[0189]

[0190] Experimental results show that the proposed PES-PointPillars performs excellently in detecting various target categories. Compared with the benchmark algorithm PointPillars, PES-PointPillars improves detection accuracy across all categories, especially in detecting small-scale targets such as pedestrians and cyclists. In medium difficulty mode, the AP value for pedestrians is improved by 3.2%, and the AP value for cyclists is improved by 3.8%. Compared with other mainstream algorithms, the benchmark algorithm PointPillars shows a significant advantage in real-time performance, achieving an FPS of 62. The improved PES-PointPillars further increases the FPS by 6.3, reaching 68.3. Although 3DSSD maintains a certain level of real-time performance, its detection accuracy for each category is lower than that of PES-PointPillars. CenterPoint has high detection accuracy for pedestrians, but its accuracy for cyclists needs improvement. The improved PES-PointPillars achieves a better balance between categories and FPS.

[0191] The model's detection performance is as follows Figure 7 As shown in the figure, the method of the present invention can accurately identify traffic targets such as vehicles, pedestrians, and cyclists in complex road scenarios, and can provide relatively accurate three-dimensional bounding box localization results, verifying the effectiveness and practical value of the method of the present invention in autonomous driving environmental perception.

Claims

1. A traffic target detection method based on PES-PointPillars, characterized in that, The steps are as follows: Step 1) Obtain traffic target point cloud dataset: Obtain LiDAR point cloud dataset for traffic target detection. The dataset contains multiple frames of road scene point cloud data and their corresponding 3D target annotation information. Target categories include vehicles, pedestrians, and cyclists. Step 2) Divide the dataset: Divide the traffic target point cloud dataset into training set, validation set and test set according to a preset ratio for subsequent model training and performance evaluation; Step 3) Based on the PointPillars 3D target detection network, the backbone network structure and feature fusion module are improved to construct the PES-PointPillars traffic target detection model. First, the input 3D point cloud data is divided into columnar units and converted into a 2D pseudo-image representation through a point cloud coding network. Then, the improved backbone network is used to extract the pseudo-image features, and the information interaction between features at different levels is enhanced through a multi-layer feature fusion structure to obtain a richer spatial feature representation of the point cloud. Step 4) Input the training data obtained in Step 2) into the PES-PointPillars traffic target detection model constructed in Step 3) for training. Optimize the model parameters by designing an improved SNWD loss function, so that the model can improve the positioning accuracy and detection performance of traffic targets under sparse point cloud conditions. Step 5) Input the LiDAR point cloud data of the scene to be detected into the trained PES-PointPillars traffic target detection model to obtain the road scene image to be detected, perform feature extraction and target detection on the point cloud data, and output the category information of traffic targets and the 3D bounding box detection results.

2. The traffic target detection method based on PES-PointPillars according to claim 1, characterized in that, In step 3), the specific steps are as follows: Step 3.1) Improvement of the backbone network based on windmill-shaped convolution In the PointPillars algorithm, the original point cloud is first converted into a two-dimensional pseudo-image representation through columnar partitioning and point feature encoding, and then feature extraction is performed through a two-dimensional convolutional neural network. To enhance the network's ability to model the structural features of sparse point clouds, a windmill-shaped convolutional structure is introduced into the PointPillars backbone network to replace some standard convolutional layers. The windmill-shaped convolution expands the feature receptive field in the horizontal and vertical directions through multiple directional convolutional branches, enabling the network to capture directional structural information in sparse point clouds. Let the input feature map be The windmill-shaped convolution performs parallel convolution on the input features through four directional convolution branches. Its calculation process is represented as follows: in, These represent convolution operations in different directions. Indicates batch normalization. Indicates the activation function; The features obtained from the four directional convolutional branches are concatenated along the channel dimension and then fused through a convolutional layer to obtain the final output features: in, , , , These represent the four safe convolutional branches. For convolution operations, For merging and splicing operations; Windmill-shaped convolution decomposes the feature extraction process of traditional convolution into feature perception in multiple directions, enabling the network to expand the feature receptive field while maintaining low computational complexity, thereby adapting to the spatial distribution characteristics of sparse point clouds and improving the network's ability to express features of small targets such as pedestrians and cyclists. Step 3.2) Enhance inter-layer feature association module In the PointPillars network, the features output by different layers of the backbone network show significant differences in semantic information and spatial details. To enhance the interaction between deep semantic information and shallow geometric information, an enhanced inter-layer feature association module is introduced after deconvolution upsampling of the backbone network. This module consists of two branches: a grouped feature focusing unit and a multi-level feature reconstruction module. By performing association enhancement and reconstruction processing on features at different levels, it enhances the semantic information expression ability while retaining key geometric details, thereby improving the network's feature expression ability for small targets in sparse point clouds. Step 3.2.1) Grouping Feature Focusing Units Grouped feature focusing units are used to enhance the correlation between features from different layers and highlight key feature information through spatial attention mechanisms; let the features from two adjacent layers of the backbone network be respectively and First, the two layers of features are added element-wise to obtain the initial fused features: Spatial weights are then generated through convolution operations and utilized... The function is normalized to obtain the spatial weight map: in This indicates an element-wise addition operation, used to achieve preliminary fusion of features from different layers; Spatial weights are applied to the fused features to obtain features that contain spatial attention information: in, This indicates element-major multiplication; to further enhance the feature correlation between channels, the features... The feature is divided into multiple feature groups along the channel dimension, and feature interaction processing is performed within each feature group; the features of each group are then reassembled to obtain the aggregated feature. Its expression is as follows: Finally, the aggregated features are input into the multi-layer original feature fusion and normalization module for spatial normalization processing to obtain the final output features: in This indicates that the mean of the feature is calculated. This indicates that the standard deviation of the feature is calculated; Through the above process, grouped feature focusing units can enhance the feature correlation between different channels and highlight key spatial areas; Step 3.2.2) Multi-level feature reconstruction module The multi-level feature reconstruction module is used to reduce feature redundancy in deep networks and retain more target-related information during feature fusion; let the input features be: in, and They represent the features of adjacent layers respectively. and The generated weight coefficients are used to characterize the importance of features at different layers; This indicates a batch normalization operation, used to normalize features to stabilize the training process; Used to map the output to the 0-1 range, thereby obtaining the weighting coefficients; The two layers of features are then added element-wise to obtain the fused features. An adaptive threshold is generated using global average pooling and learnable bias parameters. : in, This represents the global average pooling operation, used to extract the overall response information of features; This represents a learnable bias parameter used to adjust the distribution range of the threshold. Based on the above thresholds, a soft gating mechanism is constructed to generate weight masks for strong and weak features: in, This represents a strong feature weight mask, used to highlight important feature information; This represents a weak feature weight mask, used to preserve auxiliary feature information; These represent the weight coefficients generated from the input features; After obtaining the soft mask, different weights are mapped to the fused features. Above, and generate strong features respectively. With weak features : Among them, strong features Feature enhancement is performed through convolutional layers to address weak features. Multi-level feature reconstruction is achieved through feature transformation units. Step 3.2.3) Module Output Fusion After obtaining strong and weak features, feature transformation is performed on each type of feature. For strong features, convolution operations are used to enhance details, while for weak features, structural optimization is performed through feature transformation units. Finally, the two sets of features are fused to obtain the final output of the multi-level feature reconstruction module. : in, These are the feature fusion weight coefficients; This represents weak features after feature transformation. This indicates a strong feature after feature enhancement processing; Through the above process, the enhanced inter-layer feature association module can retain key target information while fusing features from different layers, reduce redundant feature interference, and improve the network's ability to detect small targets in sparse point cloud environments.

3. The traffic target detection method based on PES-PointPillars according to claim 1, characterized in that, In step 4), the specific steps are as follows: Step 4.1) Construct the SNWD bounding box regression loss function In PointPillars networks, the original bounding box regression typically uses the SmoothL1 loss function to evaluate the 3D bounding box parameters. Supervised learning is performed; to improve the localization capability of small targets, the predicted bounding boxes and the ground truth bounding boxes are represented as two-dimensional Gaussian distributions, and the difference between the two distributions is measured by the Wasserstein distance. Let the distribution corresponding to the predicted box be... The distribution of the real bounding boxes is as follows Its second-order Wasserstein distance is expressed as: in, and These represent the center coordinates of the predicted bounding box and the ground truth bounding box, respectively; Substituting the bounding box parameters from the 3D detection task into the above formula yields a distance expression suitable for traffic target detection tasks: in, and Indicates the center coordinates of the predicted bounding box; and Represents the center coordinates of the actual bounding box; and These represent the width and length of the predicted bounding box, respectively. and These represent the width and length of the actual bounding box, respectively. To convert this distance metric into a similarity index, it is exponentially normalized to obtain the normalized Wasserstein distance: in, This is a constant used to control the scale sensitivity of the distance metric. The exponential function is represented by the above metric, and a planar localization loss function is constructed based on this: This loss function can still provide effective gradients when the predicted bounding box does not overlap with the ground truth bounding box, thus improving the localization accuracy in small object detection tasks. Step 4.2) Construct the SNWD bounding box regression loss function For height parameter and orientation angle parameters Optimization is performed using the SmoothL1 loss function: in, Represents the regression loss for height and orientation parameters; Indicates bounding box parameters; This represents the difference between the predicted value and the actual value. Subsequently, the planar positioning loss was... With height and angle loss By performing a weighted combination, we obtain the SNWD regression loss function: in, and These are the weighting coefficients. Based on this, the classification loss is combined... With direction classification loss The overall training loss function of the network is obtained as follows: in, , and These represent the weight coefficients for regression loss, classification loss, and orientation loss, respectively. The network parameters are optimized using the above training method. Average precision (AP), intersection-to-union ratio (IoU), and frame rate (FPS) are used as performance evaluation metrics for the model, as shown in the following expressions: in Indicates accuracy. Indicates recall rate; This indicates that the true class is the i-th class and the predicted result is also the i-th class. The number of samples in each class; Indicates the true category is the first Class but predicted as the first The number of samples in each class; Indicates the true category is the first Class but predicted as the first The number of samples in each class; This represents the total number of categories.

Images