A perception method for autonomous driving based on triple attention mechanism

Abstract

The present invention belongs to the field of deep learning technology, and specifically discloses an autonomous driving perception algorithm based on a triple attention mechanism. First, the original point cloud data of the scene scanned by the lidar is input into the feature encoding network to be divided into pillars, and the 3D point cloud data is converted into a 2D pseudo image; then the converted 2D pseudo image is input into the 2DBackbone for feature extraction to obtain multi-scale and multi-dimensional features; finally, upsampling and splicing are performed before being sent to the detection head for detection tasks. The DSPE module proposed in the present invention significantly enhances the ability to extract fine-grained information in 3D small target detection by introducing a triple attention mechanism. DSPE not only improves the problem of feature information loss caused by traditional maximum pooling operations, but also effectively captures the spatial correlation between points within each pillar, feature interactions between channels, and contextual dependencies between voxels, greatly improving the detection accuracy and speed of the system.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and in particular relates to an autonomous driving perception method based on a triple attention mechanism. Background Art

[0002] Deep learning technology is increasingly demonstrating its powerful capabilities in various application scenarios. 3D object detection plays a crucial role in the perception phase of autonomous driving, in particular. Selecting and implementing the most appropriate method for the practical needs of autonomous driving scenarios presents a significant challenge. Currently, mainstream 3D object detection methods are categorized as point-based, voxel-based, point-voxel, and pillar-based.

[0003] Point-based and voxel-based methods each possess unique advantages, but also limitations. For example, while the regularized grid structure of voxel representation facilitates fast neighbor finding and simplifies data processing, this approach introduces a trade-off between spatial resolution and computational resources. On the one hand, achieving high-precision representation of spatial detail requires a finer voxel grid, which significantly increases data storage requirements and computational complexity. On the other hand, for sparsely distributed objects or scenes, most voxels may be empty, resulting in a waste of storage and computational resources. Point-based methods operate directly on discrete point sets, flexibly adapting to data distributions of varying densities and avoiding unnecessary spatial partitioning. However, this flexibility also presents challenges. For example, point cloud data often lacks intrinsic topological information, making tasks such as feature extraction and surface reconstruction more difficult. Furthermore, point cloud data is susceptible to noise and outliers, especially when data is incomplete during acquisition due to sensor limitations or environmental factors. These shortcomings further complicate data preprocessing and impact the quality of subsequent analysis and modeling.

[0004] Pillars-based methods are more suitable for real-world autonomous driving scenarios, but these targets typically only occupy a small portion of the entire point cloud scene. Giving equal attention to every point in all regions of the point cloud data inevitably leads to significant waste of computational resources. For example, while pointpillars prioritize information in the xoy plane, the sparsity of the point cloud within that plane prevents them from capturing the important point cloud information corresponding to these targets. This is because the majority of the point cloud covers background or non-critical areas, while the information that truly requires high-precision recognition and processing is concentrated on relatively small targets, resulting in significant computational waste. Furthermore, the MaxPooling method in pointpillars only retains the maximum feature value for each point within each pillar, discarding other information. This can result in the loss of important contextual information or subtle features, which can be detrimental for tasks that require greater detail, such as fine-grained image classification or small object recognition in object detection. These shortcomings can lead to reduced detection accuracy and speed in autonomous driving.

[0005] In actual scenarios of autonomous driving, the point cloud data scanned by the lidar will have sparsity problems due to partial occlusion of the detection target, the diversity of the detection target shape, and the change in distance. Existing models do not have a good way to address these common problems in actual scenarios of autonomous driving. Summary of the Invention

[0006] In response to the problems in the background technology, the present invention proposes an autonomous driving perception method based on a triple attention mechanism, aiming to improve the accuracy and reliability of autonomous driving vehicles' perception of the surrounding small target objects.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A triple attention-based autonomous driving perception method consists of a feature encoding network (VFE), a convolutional neural network (2D backbone) for extracting 2D image features, and a detection head (dense head). The method includes the following steps:

[0009] Step 1: Input the original point cloud data of the scene scanned by the lidar into the feature encoding network to divide it into pillars, and convert the 3D point cloud data into a 2D pseudo image;

[0010] Step 2: Input the converted 2D pseudo-image into 2DBackbone for feature extraction. This is done by downsampling the input feature map into three feature maps of different sizes and channels to obtain multi-scale and multi-dimensional features. Finally, they are concatenated.

[0011] Step 3: Send it to the detection head for detection.

[0012] Furthermore, the detailed working steps of the feature encoding network in step 1 are as follows:

[0013] (1) The 3D point cloud data is evenly divided into grids according to the xoy plane. The cuboid represented by each grid is a pillar. The number of pillars is set to M. The points falling in a grid are considered to be in a pillar. In the KITTI dataset, the feature of a point is represented by a four-dimensional vector, (x, y, z, r) corresponding to the coordinates and reflection intensity of the point on the xyz axis respectively. Each point is represented by a C=10-dimensional vector, namely (x, y, z, r, xc, yc, zc, xp, yp, zp), where x, y, z, r indicate the real coordinate information and reflection intensity of the point cloud, xc, yc, zc indicate the position offset of each point relative to the centroid of the voxel, and xp, yp, zp indicate the coordinate offset of each point relative to the center of the voxel to which it belongs.

[0014] (2) Set a hyperparameter N to limit the number of points in each pillar. If the number of points in a pillar exceeds N, randomly sample N points. If it is less than N, fill it with 0. In this way, the original 3D point cloud data constitutes a tensor (P, N, C), where C = 10, C represents the number of feature channels of each point, P represents the number of pillars, and N represents the number of points. This completes the tensorization of the point cloud data.

[0015] (3) After completing the tensorization of the point cloud in step (2), we obtain data P1∈(P×N×C), which is then passed through a simplified version of PointNet to obtain P1∈(P, N, D), where D=64, where D represents the number of feature channels for each point in the current point cloud.

[0016] (4) Perform attention mechanism between points: First, transform P1∈(P, N, D) into T2∈(N, P, D) through position conversion, apply multi-head attention mechanism to calculate the similarity between points and perform weighted processing, generate weighted feature matrix O∈(N×P×D); then transform the dimension back into O∈(P, N, D) and take the first one to get O1∈(P, 1, D), transform P1∈(P, N, D) into H2∈(D, P, N), apply multi-head attention mechanism to calculate the similarity between channels of each point and perform weighted processing, generate weighted feature matrix K∈(D×P×N); then transform the dimension back into O∈(P, N, D) and take the first one to get O1∈(P, 1, D), transform P1∈(P, N, D) into H2∈(D, P, N), apply multi-head attention mechanism to calculate the similarity between channels of each point and perform weighted processing, generate weighted feature matrix K∈(D×P×N); then transform the dimension back into Convert it back to K∈(P, N, D) and take the first one to get K1∈(P, N, 1). Multiply O1∈(P, 1, D) and K1∈(P, N, 1) element by element to get the fused feature matrix KO1∈(P, N, D). Multiply the original feature matrix P1 with the fused feature matrix KO1 to get the weighted feature matrix KOP1∈(P×N×D). Finally, input the weighted feature matrix KOP1 into the SENet (Squeeze-and-Excitation Network) module to further optimize the feature expression.

[0017] (5) After step (4), the weighted feature matrix KOP1 is subjected to the maximum pooling operation to extract the most representative features and reduce the feature dimension. After this step, the reduced-dimensional feature matrix KOP2∈(P×D) is obtained. At the same time, the original feature matrix P1 is also subjected to the maximum pooling operation to extract the global features G1∈(P×D). The global features G1 and the reduced-dimensional features KOP2 are added point by point to generate the comprehensive feature matrix F∈(P×D). Finally, the point features are dispersed back into their pillars to obtain a (D, H, W) tensor, i.e., a pseudo image. H and W represent the length and width of the pseudo image, respectively. The pseudo image is generated so as to be fed into the next 2D backbone for feature extraction.

[0018] Furthermore, the detailed working steps of the feature extraction performed by 2DBackbone in step 2 are as follows:

[0019] (1) The feature pseudo image size (D, H, W) passes through the first Block. The first layer of the first block has a stride of 2, a convolution kernel size of 3×3, an input fin channel dimension of D, and an output fout channel dimension of D. The second, third, and fourth layers all have the same stride of 1, a convolution kernel size of 3×3, an input fin channel dimension of D, and an output fout channel dimension of D. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (D, H / 2, W / 2), recorded as W1;

[0020] (2) Send W1 to the second Block. The first layer of the second block has a stride of 2, a convolution kernel size of 3×3, an input fin channel dimension of D, and an output fout channel dimension of 2D. The second, third, fourth, fifth, and sixth layers all have the same stride of 1, a convolution kernel size of 3×3, an input fin channel dimension of 2D, and an output fout channel dimension of 2D. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (2D, H / 4, W / 4), recorded as W2;

[0021] (3) W2 is sent to the third Block. The first layer of the third Block consists of a convolution kernel with a stride of 2×2 and a convolution kernel size of 3×3. The input fin channel dimension is 2D and the output fout channel dimension is 4D. The second, third, fourth, fifth and sixth layers all have the same stride of 1 and a convolution kernel size of 3×3. The input fin channel dimension is 4D and the output fout channel dimension is 4D. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (4D, H / 8, W / 8), recorded as W3;

[0022] (4) W1 is passed through the first deblock, which consists of a convolution kernel with a stride of 1 and a kernel size of 1×1. The input fin channel dimension is D and the output fout channel dimension is 2D. It is followed by a Relu and a batch normalization layer BatchNorm2d. The size of the pseudo image is (2D, H / 2, W / 2), which is recorded as H1.

[0023] (5) W2 is passed through the second deblock, which consists of a convolution kernel with a stride of 2 and a kernel size of 2×2. The input fin channel dimension is 2D, and the output fout channel dimension is 2D. It is followed by a Relu and a batch normalization layer BatchNorm2d. The size of the pseudo image is (2D, H / 2, W / 2), which is recorded as H2.

[0024] (6) W3 passes through the last deblock. The third deblock consists of a convolution with a stride of 4 and a kernel size of 4×4. The input fin channel dimension is 4D and the output fout channel dimension is 2D. It is connected to a Relu and a batch normalization layer BatchNorm2d. Finally, a pseudo image with a size of (2D, H / 2, W / 2) is obtained, which is recorded as H3.

[0025] (7) Concatenate H1, H2, and H3 together to obtain the feature map H∈(6D, H / 2, W / 2).

[0026] Preferably, in step 3, SSD is used as the detection head of the model, and the 2D IoU between the candidate box and the ground truth is used for classification and regression.

[0027] Preferably, the detailed steps of step three are:

[0028] (1) In the 3D point cloud, both ground truth boxes and anchors are represented as a seven-dimensional vector (x, y, z, w, l, h, θ), where x, y, z represent the geometric center of the bounding box, w, l, h represent the length, width and height of the bounding box, and the loss function is calculated as follows:

[0029] (1)

[0030] Where: Δx represents the offset of the target in the x-axis direction, Δy represents the offset of the target in the y-axis direction, and Δz represents the offset of the target in the z-axis direction; χ gt 、y gt 、z gt They represent the real position coordinates of the target, χ a 、y a 、 g a Represent the predicted position coordinates, d a Indicates the predicted distance, usually used to normalize the offset;

[0031] Δw represents the offset of the target width, Δl represents the offset of the target length, Δh represents the offset of the target height, and w gt 、 l gt 、 h gt Respectively represent the width, length, and height of the target, w a 、 l a 、 h a Represent the predicted width, length, and height respectively;

[0032] represents the offset of the target attitude angle, represents the real attitude angle of the target, and represents the predicted attitude angle;

[0033] The total position loss function uses the smooth1 function, and the formula is:

[0034] (2)

[0035] For target classification loss, focal loss is used, and the formula is:

[0036] (3)

[0037] Where: α represents the balance factor, represents the predicted probability, and γ represents the focusing parameter; the total loss function calculation formula is: (4)

[0038] in: N POS represents the number of positive samples, β reg , β cls , β dir Represent the weights of regression loss, classification loss and direction loss respectively, L reg 、L cls 、L dirr They represent regression loss, classification loss, and direction loss respectively.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] This paper proposes an innovative pillar encoding module - a feature encoding network that integrates a triple attention mechanism, named Dual Attention and SENet Attention Pillar Encoding (DSPE). Compared with the existing PointPillars method, the DSPE module proposed in this paper significantly enhances the ability to extract fine-grained information in 3D small target detection by introducing a triple attention mechanism. DSPE not only improves the problem of feature information loss caused by traditional maximum pooling operations, but also effectively captures the spatial correlation between points within each pillar, feature interactions between channels, and contextual dependencies between voxels. This makes the model perform better when processing tasks that require more details. At the same time, the DSPE module is efficient and flexible, and can significantly improve the detection accuracy and speed of the system without adding additional computing burden and development costs, making it an ideal choice for real-time embedded applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a network flow chart of the entire present invention.

[0042] Figure 2This is the working principle of the DSPE network proposed in the feature encoding network VFE module of the present invention. DETAILED DESCRIPTION

[0043] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0044] like Figure 1 As shown, the present invention is an autonomous driving perception method based on a triple attention mechanism, which mainly consists of three parts: a feature encoding network VFE, a 2D backbone convolutional neural network, and a dense head. First, the original point cloud data of the scene scanned by the lidar is input into the feature encoding network to be divided into pillars, and then converted into a 2D pseudo-image and input into the 2D backbone for feature extraction. This is mainly done by downsampling the input feature map into three feature maps of different sizes and channels to obtain multi-scale and multi-dimensional features. The features are then upsampled and spliced ​​before being sent to the detection head for detection tasks. The network has good performance in predicting and identifying small target objects in a 3D target perception environment.

[0045] 1. Feature Encoding Network VFE

[0046] The feature extraction network of the present invention converts 3D point cloud data into a 2D pseudo image. First, the 3D point cloud data is evenly divided into grids according to the xoy plane. The rectangular parallelepiped represented by each grid is a pillar. The number of pillars is set to M, and the points falling within a grid are considered to be in a pillar. In the KITTI dataset, the feature of a point is represented by a four-dimensional vector (x, y, z, r), which corresponds to the coordinates and reflection intensity of the point on the xyz axis, respectively. In our method, each point is represented by a C = 10-dimensional vector, namely (x, y, z, r, xc, yc, zc, xp, yp, zp), where x, y, z, r indicate the true coordinate information and reflection intensity of the point cloud, xc, yc, zc indicate the position offset of each point relative to the centroid of the voxel, and xp, yp, zp indicate the coordinate offset of each point relative to the center of the voxel to which it belongs. After the grid is divided and the point features are represented, due to the sparsity of the point cloud data, many pillars contain no points or very few points. To address the issue of the number of points in different pillars, we set a hyperparameter N to limit the number of points in each pillar. If the number of points in a pillar exceeds N, N points are randomly sampled. If the number is less than N, they are padded with zeros. This way, the original 3D point cloud data forms a tensor of (P, N, C), where C represents the number of feature channels per point, P represents the number of pillars, and N represents the number of points. This completes the tensorization of the point cloud data. Existing pointpillars perform maxpooling on the points within each pillar, transforming the shape from (C, P, N) to (C, P). However, this only retains the maximum value of each point within each pillar, discarding other information. This can result in the loss of important contextual information or subtle features, which can be detrimental for tasks that require more detail, such as fine-grained image classification or small object recognition in object detection.

[0047] To further enhance the performance of autonomous driving perception systems, this paper innovatively proposes a feature encoding network called DSPE (Dual Attention and SENet Attention Pillar Encoding). This network incorporates a triple attention mechanism to optimize the relationships between points within each pillar, the information from each point across different channels, and the interactions between pillars in lidar point cloud data. In the DSPE network, attention is primarily focused on the interactions between points within a pillar. Traditional VFE layers typically treat all points within a pillar uniformly, ignoring the varying importance that may exist between points. To address this, DSPE introduces an inter-point attention mechanism that calculates adaptive weights for points within each pillar to highlight those that are more critical for object detection. This weighting strategy not only enhances the model's ability to capture local details but also improves its adaptability to complex scenes. In addition to considering relationships between points, DSPE also emphasizes the feature representation of each point across different channels. Because lidar point cloud data contains rich attribute information (such as coordinates and reflection intensity), the importance of different channels varies. To this end, DSPE implements a channel-level attention mechanism within each pillar. By evaluating the contribution of each channel and dynamically adjusting its weight, DSPE can more accurately extract the most representative features, thereby improving the overall feature expression capability. Finally, DSPE also pays attention to the global correlation between different pillars. Considering that the distribution of objects in real scenes is often non-uniform, pillars in certain areas may contain more valuable information. Therefore, DSPE draws on the idea of ​​SENet (Squeeze-and-Excitation Network) and implements the SENeT attention mechanism between each pillar to evaluate the importance of each pillar in the entire point cloud space and allocate different computing resources accordingly. This method ensures that the model can focus more attention on key areas, reduce unnecessary computational burden, and improve detection efficiency.

[0048] like Figure 2 As shown, the following is a detailed description of the design concept and working principle of the DSPE network:

[0049] After completing the tensorization of the point cloud described above, we obtain data P1∈(P×N×D). This data is then passed through a simplified version of PointNet for dimensionality increase, yielding P1∈(P,N,D), where C=10 and D=64. Next, we apply an attention mechanism between points: First, P1∈(P,N,D) undergoes a positional transformation called Transpose to become T2∈(N,P,D). This transformation ensures that all points within each pillar can be compared and weighted along the same dimension. Multi-head attention is then applied to calculate and weight the similarities between points. This process not only enhances the model's understanding of local details but also improves the efficiency and accuracy of feature extraction. Ultimately, a weighted feature matrix O∈(N×P×D) is generated. This matrix is ​​then transformed back to O∈(P,N,D), and the first one is taken to obtain O1∈(P,1,D). Then, P1∈(P, N, D) is transformed into H2∈(D, P, N) through positional transformation. Then, the multi-head attention mechanism is applied to calculate the similarity between the channels of each point and perform weighted processing. Finally, the weighted feature matrix K∈(D×P×N) is generated. Then, the dimension is transformed back to K∈(P, N, D) and the first one is taken to obtain K1∈(P, N, 1). O1∈(P, 1, D) and K1∈(P, N, 1) are multiplied element by element to obtain the fused feature matrix KO1∈(P, N, D). This process is achieved by combining O1 OThe channel information of 1 is combined with the spatial position information of K1 to generate a new feature representation that contains rich contextual relationships. Next, the original feature matrix P1 is multiplied by the fused feature matrix KO1 to obtain the weighted feature matrix KOP1∈(P×N×D). This operation dynamically adjusts the original features based on the weights calculated in KO1, highlighting those features that are more critical for object detection. Finally, the weighted feature matrix KOP1 is input into the SENet (Squeeze-and-Excitation Network) module to further optimize the feature representation. SENet reassesses the importance of each channel and dynamically adjusts its weights, ensuring that the model more accurately captures the most representative features. The features processed by SENet not only enhance the ability to understand local details but also improve adaptability to complex scenes. Maxpooling is applied to the weighted feature matrix KOP1 to extract the most representative features and reduce the feature dimensionality. After this step, the reduced feature matrix KOP2∈(P×D) is obtained. Max pooling not only simplifies subsequent processing but also preserves the most important local extreme value information, enhancing the robustness of the model. Simultaneously, max pooling is applied to the original feature matrix P1 to extract global features G1∈(P×D), where D represents the global feature dimension of each pillar. This process captures the maximum response of all points within each pillar, providing a global perspective for subsequent feature fusion. Finally, the global features G1 and the dimensionally reduced features KOP2 are added point-by-point (element-wise addition) to generate a comprehensive feature matrix F∈(P×D). This point-by-point addition effectively combines local and global features, enhancing the model's ability to understand features at different scales and improving the quality of the final feature representation. Finally, the point features are dispersed back into their respective pillars, resulting in a (C, H, W) tensor, or a pseudo image, where H and W represent the height and width of the pseudo image, respectively. This pseudo image is then fed into the subsequent 2D backbone for feature extraction.

[0050] 2DBackBone

[0051] The 2D backbone of the present invention consists of two sub-networks. In the first stage, the feature pseudo-image is further fed into a top-down downsampling network. This network consists of three downsampling blocks, each of which contains several convolutional layers, batch normalization layers, and activation functions to progressively extract higher-level semantic information. With each block, the spatial resolution of the feature map decreases, while the number of channels may increase to capture more abstract and complex patterns. The details are as follows: First, the feature pseudo-image is of size (D, H, W). After passing through the first block, the first layer of the first block consists of a stride of 2 and a 3×3 convolution kernel size, with an input fin channel dimension of D and an output fout channel dimension of D. The second, third, and fourth layers all have the same stride of 1 and a 3×3 convolution kernel size, with an input fin channel dimension of D and an output fout channel dimension of D. Each layer is followed by a ReLU and a BatchNorm2d layer, resulting in a pseudo-image of size (D, H / 2, W / 2), denoted as W1. Then W1 is sent to the second Block. The first layer of the second block has a stride of 2, a convolution kernel size of 3×3, an input fin channel dimension of D, and an output fout channel dimension of 2D. The second, third, fourth, fifth, and sixth layers all have the same stride of 1, a convolution kernel size of 3×3, an input fin channel dimension of 2D, and an output fout channel dimension of 2D. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (2D, H / 4, W / 4), recorded as W2. Finally, W2 is sent to the third Block. The first layer of the third Block consists of a convolution kernel with a stride of 2×2 and a convolution kernel size of 3×3. The input fin channel dimension is 2D, and the output fout channel dimension is 4D. The second, third, fourth, fifth, and sixth layers all have the same stride of 1 and a convolution kernel size of 3×3. The input fin channel dimension is 4D, and the output fout channel dimension is 4D. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (4D, H / 8, W / 8), denoted as W3.

[0052] Next comes the second stage, which is an upsampling. First, W1 passes through the first deblock. The first deblock consists of a convolution kernel with a stride of 1 and a convolution kernel size of 1×1. The input fin channel dimension is D, and the output fout channel dimension is 2D. It is followed by a Relu and a batch normalization layer BatchNorm2d. The resulting pseudo image size is (2D, H / 2, W / 2), denoted as H1. Then W2 passes through the second deblock. The second deblock consists of a convolution kernel with a stride of 2 and a convolution kernel size of 2×2. The input fin channel dimension is 2D, and the output fout channel dimension is 2D. It is followed by a Relu and a batch normalization layer BatchNorm2d. The resulting pseudo image size is (2D, H / 2, W / 2), denoted as H2. Next, W3 passes through the final deblock. The third deblock consists of a convolution with a stride of 4 and a kernel size of 4×4. The input fin has a channel dimension of 4D, and the output fout has a channel dimension of 2D. It is followed by a ReLU and a batch normalization layer BatchNorm2d. The resulting pseudo-image is of size (2D, H / 2, W / 2), denoted as H3. Finally, H1, H2, and H3 are concatenated together to obtain a feature map H∈(6D, H / 2, W / 2).

[0053] 3. Detection head dense head

[0054] The present invention uses SSD as the detection head of the model. The difference from other 3D detection heads is that the model of the present invention converts 3D point cloud data into 2D pseudo images for processing. Therefore, the 2D IoU between the candidate box and the groundtruth is used for classification and regression. In the 3D point cloud, we represent the ground truth boxes and anchors as a seven-dimensional vector (x, y, z, w, l, h, θ), where x, y, z represent the geometric center of the bounding box, w, l, h represent the length, width and height of the bounding box, and the loss function calculation formula is as follows (1) (1)

[0055] The total position loss function uses the smooth1 function, and the calculation formula is as follows (2) (2)

[0056] For target classification, use focal loss, calculated as (3) (3)

[0057] The total loss function calculation formula is as follows (4) (4)

[0058] The model of the present invention is embedded in the target detection and segmentation module in the autonomous driving perception link. For the original point cloud data collected by the lidar, after passing through our model, the output is the detection results described by 3D bounding boxes and confidence predictions, which is convenient for decision-making after the perception link.

[0059] 4. Training details

[0060] (1) Data preparation: Extract point cloud information (input) from the data collected by the lidar.

[0061] (2) Point cloud preprocessing: Corresponding to the feature encoding network mentioned above, the point cloud data is divided into pillars to ensure that the point cloud data input into the 2D backbone has a uniform format and size.

[0062] (3) Network training: It is a deep learning network composed of a deep neural network and a target detection head, which has the function of multi-level feature extraction.

[0063] (4) Loss function: The loss function mentioned above is used.

[0064] (5) Training process: The network is trained using the KITTI dataset. The training epoch is set to 120, the x and y dimensions are set to 0.16m respectively, the maximum number of pillars is set to (P) 16000, and the maximum number of points in each pillar is set to (N) 32. If the number exceeds, 100 points are randomly sampled, and if the number is less than 32, the number is padded with zeros. The size of each pillar is set to 0.16x0.16x4, where 0.16x0.16 is the size after voxelization on the xoy plane.

[0065] During training, the x, y, and z axes of the point cloud were set to (0, 69.12), (-39.68, 39.68), and (-3, 1), respectively. Training focused only on the front view of the point cloud scene; the remaining points and ground truth boxes were not used for training. We used the same anchor boxes and matching strategy as [voxelnet] . Each anchor box is represented by its width, length, height, and z-center, and the Intersection over Union (IoU) is used to determine whether a sample is positive or negative. Due to differences in shape and volume, the three classes in the KITTI dataset require different anchor box sizes. We set three different sizes for each class, and each anchor box was oriented at two different angles: 0° and 90°. For the pedestrian and cyclist classes, the anchor box sizes were set to (0.8, 0.6, 1.73) meters and (1.75, 0.6, 1.73) meters, centered at -0.6 meters in both directions. The matching IoU thresholds were set to 0.5 and 0.5, respectively.

[0066] During sample allocation, we used the target allocator [yolo9000]. Specifically, samples with the highest IoU (interference over union) or those exceeding the set positive IoU threshold were considered positive, those with an IoU below the set negative threshold were considered negative, and those with an IoU between the set positive and negative thresholds were discarded. We also used the KITTI test and validation sets to evaluate the model, and uploaded the training results to the KITTI official website.

[0067] (6) Data enhancement: Data enhancement is very helpful for model training. We use some common 3D data enhancement strategies (local translation, local rotation, local scaling) to improve the generalization ability of the model and reduce overfitting.

[0068] In 3D object detection, small objects often lack fine-grained information due to their limited spatial extent and inherent sensor resolution limitations. This deficiency can lead to suboptimal detection performance, as the model may fail to capture the intricate details required for accurate localization and classification. To effectively address the challenge of extracting fine-grained information in 3D small object detection, we propose an innovative pillar encoding module—a pillar encoding approach that combines a triple attention mechanism, named DualAttention and SENet Attention Pillar Encoding (DSPE).

[0069] The DSPE module is designed not only for high efficiency but also for ease of integration. Specifically, it offers a significant plug-and-play advantage, allowing it to be directly embedded into various 3D object detection frameworks without requiring complex adjustments to existing network architectures. By incorporating DSPE, system performance can be significantly improved while avoiding the additional computational burden and development costs associated with redundant modules. This efficient and flexible solution makes DSPE an ideal choice for improving the accuracy and speed of 3D small object detection, particularly for applications requiring high real-time performance and resource efficiency.

[0070] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A perception method for autonomous driving based on a triple attention mechanism, characterized by: It consists of three parts: feature encoding network, 2D backbone and detection head, including the following steps: Step 1: Input the original point cloud data of the scene scanned by the lidar into the feature encoding network and divide it into pillars, thereby converting the 3D point cloud data into a 2D pseudo image. Specifically, (1) The 3D point cloud data is evenly divided into grids according to the xoy plane. The cuboid represented by each grid is a pillar. The number of pillars is set to M. The points falling in a grid are considered to be in a pillar. In the KITTI dataset, the feature of a point is represented by a four-dimensional vector, (x, y, z, r) corresponding to the coordinates and reflection intensity of the point on the xyz axis respectively. Each point is represented by a C=10-dimensional vector, namely (x, y, z, r, xc, yc, zc, xp, yp, zp), where x, y, z, r indicate the real coordinate information and reflection intensity of the point cloud, xc, yc, zc indicate the position offset of each point relative to the centroid of the voxel, and xp, yp, zp indicate the coordinate offset of each point relative to the center of the voxel to which it belongs. (2) Set a hyperparameter N to limit the number of points in each pillar. If the number of points in a pillar exceeds N, randomly sample N points. If it is less than N, fill it with 0. In this way, the original 3D point cloud data constitutes a tensor (P, N, C), where C = 10, representing the number of feature channels of each point, P represents the number of pillars, and N represents the number of points. This completes the tensorization of the point cloud data. (3) After completing the tensorization of the point cloud in step (2), we obtain data P1∈(P×N×C), which is then passed through a simplified version of PointNet to obtain P1∈(P, N, D), where D=64, the number of feature channels for each point in the current point cloud. (4) Perform attention mechanism between points: First, transform P1∈(P, N, D) into T2∈(N, P, D) through position transformation, apply multi-head attention mechanism to calculate the similarity between points and perform weighted processing, and generate weighted feature matrix O∈(N×P×D); then transform the dimension back into O∈(P, N, D) and take the first one to get O1∈(P, 1, D), transform P1∈(P, N, D) into H2∈(D, P, N), apply multi-head attention mechanism to calculate the similarity between channels of each point and perform weighted processing The weighted feature matrix K∈(D×P×N) is generated. The dimension is then transformed back to K∈(P, N, D) and the first one is taken to obtain K1∈(P, N, 1). O1∈(P, 1, D) and K1∈(P, N, 1) are multiplied element by element to obtain the fused feature matrix KO1∈(P, N, D). The original feature matrix P1 is multiplied by the fused feature matrix KO1 to obtain the weighted feature matrix KOP1∈(P×N×D). Finally, the weighted feature matrix KOP1 is input into the SENet module to further optimize the feature expression. (5) After step (4), the weighted feature matrix KOP1 is subjected to the maximum pooling operation to extract the most representative features and reduce the feature dimension. After this step, the reduced-dimensional feature matrix KOP2∈(P×D) is obtained. At the same time, the original feature matrix P1 is also subjected to the maximum pooling operation to extract the global features G1∈(P×D). The global features G1 and the reduced-dimensional features KOP2 are added point by point to generate the comprehensive feature matrix F∈(P×D). Finally, the point features are dispersed back into their pillars to obtain a (D, H, W) tensor, i.e., a pseudo image. H and W represent the length and width of the pseudo image, respectively. The pseudo image is generated so as to be fed into the next 2D backbone for feature extraction. Step 2: Input the transformed 2D pseudo-image into 2DBackbone for feature extraction. This is done by downsampling the input feature map into three feature maps of different sizes and channels, thus obtaining multi-scale and multi-dimensional features, which are then upsampled and spliced. Step 3: Send it to the detection head for prediction task.

2. The autonomous driving perception method based on the triple attention mechanism according to claim 1, characterized in that: The detailed steps of feature extraction in step 2 of 2Dbackbone are as follows: (1) The feature pseudo image size (D, H, W) passes through the first Block. The first layer of the first block has a stride of 2, a convolution kernel size of 3×3, an input fin channel dimension of D, and an output fout channel dimension of D. The second, third, and fourth layers all have the same stride of 1, a convolution kernel size of 3×3, an input fin channel dimension of D, and an output fout channel dimension of D. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (D, H / 2, W / 2), recorded as W1; (2) Send W1 to the second Block. The first layer of the second block has a stride of 2, a convolution kernel size of 3×3, an input fin channel dimension of D, and an output fout channel dimension of 2D. The second, third, fourth, fifth, and sixth layers all have the same stride of 1, a convolution kernel size of 3×3, an input fin channel dimension of 2D, and an output fout channel dimension of 2D. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (2D, H / 4, W / 4), recorded as W2; (3) W2 is sent to the third Block. The first layer of the third Block consists of a convolution kernel with a stride of 2×2 and a convolution kernel size of 3×3. The input fin channel dimension is 2D and the output fout channel dimension is 4D. The second, third, fourth, fifth and sixth layers all have the same stride of 1 and a convolution kernel size of 3×3. The input fin channel dimension is 4D and the output fout channel dimension is 4D. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (4D, H / 8, W / 8), recorded as W3; (4) W1 is passed through the first deblock, which consists of a convolution kernel with a stride of 1 and a kernel size of 1×1. The input fin channel dimension is D and the output fout channel dimension is 2D. It is followed by a Relu and a batch normalization layer BatchNorm2d. The size of the pseudo image is (2D, H / 2, W / 2), which is recorded as H1. (5) W2 is passed through the second deblock, which consists of a convolution kernel with a stride of 2 and a kernel size of 2×2. The input fin channel dimension is 2D, and the output fout channel dimension is 2D. It is followed by a Relu and a batch normalization layer BatchNorm2d. The size of the pseudo image is (2D, H / 2, W / 2), which is recorded as H2. (6) W3 passes through the last deblock. The third deblock consists of a convolution with a stride of 4 and a kernel size of 4×4. The input fin channel dimension is 4D and the output fout channel dimension is 2D. It is connected to a Relu and a batch normalization layer BatchNorm2d. Finally, a pseudo image with a size of (2D, H / 2, W / 2) is obtained, which is recorded as H3. (7) Concatenate H1, H2, and H3 together to obtain the feature map H∈(6D, H / 2, W / 2).

3. The autonomous driving perception method based on the triple attention mechanism according to claim 2, characterized in that: In step 3, SSD is used as the detection head of the model, and the 2D IoU between the candidate box and the ground truth is used for classification and regression.

4. The autonomous driving perception method based on the triple attention mechanism according to claim 3, characterized in that: The detailed steps of step three are: (1) In the 3D point cloud, both ground truth boxes and anchors are represented as a seven-dimensional vector (x, y, z, w, l, h, θ), where x, y, z represent the geometric center of the bounding box, w, l, h represent the length, width and height of the bounding box, and the loss function is calculated as follows: （1） The total position loss function uses the smooth1 function, and the formula is: （2） For target classification, focal loss is used, and the formula is: （3） The total loss function calculation formula is: (4).

Images