Automatic driving perception algorithm based on triple attention mechanism

Abstract

The invention belongs to the technical field of deep learning, and particularly discloses an automatic driving perception algorithm based on a triple attention mechanism, and the algorithm comprises the steps: inputting the original point cloud data of a scene scanned by a laser radar into a feature coding network, dividing the original point cloud data into pillars, and converting the three-dimensional point cloud data into two-dimensional pseudo images; then, inputting the pseudo-graph which is converted into the 2D into a 2DBackbone to carry out feature extraction to obtain a multi-scale and multi-dimensional feature; and finally, carrying out up-sampling splicing and then sending to a detection head to carry out a detection task. According to the DSPE module provided by the invention, a triple attention mechanism is introduced, so that the capability of extracting fine-grained information in 3D small target detection is remarkably enhanced. The DSPE not only improves the problem of feature information loss caused by the traditional maximum pooling operation, but also can effectively capture the spatial correlation between points in each pillar, the feature interaction between channels and the context dependency between voxels, and greatly improves the detection precision and speed of the system.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and in particular relates to an autonomous driving perception algorithm based on a triple attention mechanism. Background Art

[0002] Deep learning technology has gradually demonstrated its powerful capabilities in various application scenarios, especially in the perception link of autonomous driving, where 3D object detection plays a vital role. How to select and use the most appropriate algorithm for the actual needs of autonomous driving scenarios is a major challenge. At present, the mainstream 3D object detection methods are mainly divided into point cloud-based methods, voxel-based methods, point-voxel methods, and pillar-based methods.

[0003] Point-based and voxel-based methods each have their own unique advantages, but they also have certain limitations. For example, when using voxel representation, although its regular grid structure is conducive to fast neighbor search and simplifies the data processing process, this method will introduce a trade-off between spatial resolution and computing resources. On the one hand, in order to achieve high-precision spatial detail expression, a finer voxel grid is required, which will lead to significantly increased data storage requirements and computational complexity; on the other hand, for sparsely distributed objects or scenes, most voxels may be empty, resulting in a waste of storage and computing resources. Point-based methods directly operate on discrete point sets, which can flexibly adapt to data distributions of different densities and avoid unnecessary spatial divisions. However, this flexibility also brings challenges. For example, point cloud data usually lacks intrinsic topological information, making tasks such as feature extraction and surface reconstruction more difficult. In addition, point cloud data is easily affected by noise and outliers, especially when the data is incomplete due to sensor limitations or environmental factors during the acquisition process. These shortcomings further aggravate the difficulty of data preprocessing and affect the quality of subsequent analysis and modeling.

[0004] Pillars-based algorithms are more suitable for actual scenarios of autonomous driving, but these targets usually only occupy a small part of the entire point cloud scene. If the same attention is given to each point in all areas of the point cloud data, it will inevitably lead to a huge waste of computing resources. For example, although Pointpillars pay more attention to the information of the xoy plane, for the sparsity of the point cloud in the xoy plane, Pointpillars does not have a mechanism to capture the important point cloud information corresponding to these targets. This is because most of the point cloud covers the background or non-critical areas, while the information that really needs high-precision recognition and processing is concentrated on relatively small targets, which is bound to cause a lot of computational waste. And the MaxPooling in the pointpillar only retains the maximum value of the feature of each point in each pillar, so other information will be discarded. This may lead to the loss of important contextual information or subtle features. For some tasks that need to retain more details (such as fine-grained image classification or small object recognition in target detection), this information loss may be disadvantageous. Therefore, these shortcomings will cause the problem of reduced detection accuracy in autonomous driving, and the detection speed will also be affected.

[0005] In actual scenarios of autonomous driving, the point cloud data scanned by the lidar will have sparsity problems due to partial occlusion of the detection target, the diversity of the detection target shape and the change in distance. Existing models do not have a good way to address these common problems in actual scenarios of autonomous driving. Summary of the invention

[0006] In response to the problems in the background technology, the present invention proposes an autonomous driving perception algorithm based on a triple attention mechanism, aiming to improve the accuracy and reliability of the autonomous driving vehicle's perception of the surrounding small target objects.

[0007] To achieve the above object, the present invention adopts the following technical solution:

[0008] An autonomous driving perception algorithm based on a triple attention mechanism includes three parts: a feature encoding network (VFE), a convolutional neural network (2D backbone) for extracting 2D image features, and a detection head (dense head), and includes the following steps:

[0009] Step 1: Input the original point cloud data of the scene scanned by the lidar into the feature encoding network to divide it into pillars, and convert the 3D point cloud data into a 2D pseudo image;

[0010] Step 2: Input the converted 2D pseudo-image into 2DBackbone for feature extraction, mainly by downsampling the input feature map into three feature maps of different sizes and channels to obtain multi-scale and multi-dimensional features; finally, splice them together.

[0011] Step 3: Send it to the detection head for detection task.

[0012] Furthermore, the detailed working steps of the feature encoding network in step 1 are as follows:

[0013] (1) The 3D point cloud data is evenly divided into grids according to the xoy plane. The cuboid represented by each grid is a pillar. The number of pillars is set to M, and the points falling in a grid are considered to be in a pillar. In the KITTI dataset, the feature of a point is represented by a four-dimensional vector, (x, y, z, r) corresponds to the coordinates and reflection intensity of the point on the xyz axis, respectively. Each point is represented by a C = 10-dimensional vector, namely (x, y, z, r, xc, yc, zc, xp, yp, zp), where x, y, z, r indicate the real coordinate information and reflection intensity of the point cloud, xc, yc, zc indicate the position offset of each point relative to the centroid of the body, and xp, yp, zp indicate the coordinate offset of each point relative to the center of the voxel to which it belongs.

[0014] (2) Set a hyperparameter N to limit the number of points in each pillar. If the number of points in a pillar exceeds N, N points are randomly sampled. If it is less than N, it is padded with 0. In this way, the original 3D point cloud data constitutes a tensor (P, N, C), where C represents the number of feature channels of the point, P represents the number of pillars, and N represents the number of points. The tensorization of the point cloud data is now completed.

[0015] (3) After completing the tensorization of the point cloud in step (2), the data P1∈(P×N×C) is obtained, and then a simplified version of PointNet is used to upgrade the dimension to obtain P1∈(P, N, D), where C=10, D=64, and D represents the number of feature channels of each point in the point cloud;

[0016] (4) Perform the attention mechanism between points: First, transform P1∈(P, N, D) into T2∈(N, P, D) through position transformation, apply the multi-head attention mechanism to calculate the similarity between points and perform weighted processing to generate a weighted feature matrix O∈(N×P×D); then transform the dimension back to O∈(P, N, D) and take the first one to get O1∈(P, 1, D), transform P1∈(P, N, D) into H2∈(D, P, N), apply the multi-head attention mechanism to calculate the similarity between the channels of each point and perform weighted processing to generate a weighted feature matrix K∈(D×P×N); then transform the dimension back to O∈(P, N, D) and take the first one to get O1∈(P, 1, D), transform P1∈(P, N, D) into H2∈(D, P, N), apply the multi-head attention mechanism to calculate the similarity between the channels of each point and perform weighted processing to generate a weighted feature matrix K∈(D×P×N); Convert it back to K∈(P, N, D) and take the first one to get K1∈(P, N, 1), multiply O1∈(P, 1, D) and K1∈(P, N, 1) element by element to get the fused feature matrix KO1∈(P, N, D), multiply the original feature matrix P1 with the fused feature matrix KO1 to get the weighted feature matrix KOP1∈(P×N×D); finally, input the weighted feature matrix KOP1 into the SENet (Squeeze-and-Excitation Network) module to further optimize the feature expression;

[0017] (5) After step (4), the weighted feature matrix KOP1 is subjected to the maximum pooling operation to extract the most representative features and reduce the feature dimension. After this step, the reduced-dimensional feature matrix KOP2∈(P×D) is obtained. At the same time, the original feature matrix P1 is also subjected to the maximum pooling operation to extract the global features G1∈(P×D). The global features G1 and the reduced-dimensional features KOP2 are added point by point to generate a comprehensive feature matrix F∈(P×D). Finally, the point features are dispersed back into their pillars to obtain a (C, H, W) tensor, i.e., a pseudo image. H and W represent the length and width of the pseudo image, respectively. The pseudo image is generated so as to be sent to the next 2D backbone for feature extraction.

[0018] Furthermore, the detailed working steps of the feature extraction by 2DBackbone in step 2 are as follows:

[0019] (1) The feature pseudo image size (C, H, W) passes through the first block. The first layer of the first block has a stride of 2, a convolution kernel size of 3×3, an input channel of C, and an output channel of C. The second, third, and fourth layers all have the same stride of 1, a convolution kernel size of 3×3, an input channel of C, and an output channel of C. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (C, H / 2, W / 2), denoted as W1;

[0020] (2) W1 is sent to the second Block. The first layer of the second block has a stride of 2, a convolution kernel size of 3×3, an input channel of C, and an output channel of 2C. The second, third, fourth, fifth, and sixth layers all have the same stride of 1, a convolution kernel size of 3×3, an input channel of 2C, and an output channel of 2C. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (2C, H / 4, W / 4), denoted as W2;

[0021] (3) W2 is sent to the third Block. The first layer of the third Block consists of a convolution kernel with a stride of 2×2 and a convolution kernel size of 3×3, an input channel of 2C, and an output channel of 4C. The second, third, fourth, fifth, and sixth layers all have the same stride of 1 and a convolution kernel size of 3×3, an input channel of 4C, and an output channel of 4C. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (4C, H / 8, W / 8), denoted as W3.

[0022] (4) W1 passes through the first deblock, which consists of a convolution kernel with a stride of 1 and a kernel size of 1×1, with an input channel of C and an output channel of 2C, followed by a Relu and a batch normalization layer BatchNorm2d, and the size of the pseudo image is (2C, H / 2, W / 2), denoted as H1;

[0023] (5) W1 is passed through the second deblock, which consists of a convolution kernel with a stride of 2 and a kernel size of 2×2, with 2C input channels and 2C output channels, followed by a Relu and a batch normalization layer BatchNorm2d, resulting in a pseudo-image of size (2C, H / 2, W / 2), denoted as H2;

[0024] (6) After the last deblock, the third deblock consists of a convolution with a stride of 4 and a kernel size of 4×4, with 4C input channels and 2C output channels, followed by a Relu and a batch normalization layer BatchNorm2d. Finally, a pseudo image with a size of (2C, H / 2, W / 2) is obtained, denoted as H3;

[0025] (7) Concatenate H1, H2, and H3 together to obtain the feature map H∈(6C, H / 2, W / 2).

[0026] Preferably, in step three, SSD is used as the detection head of the model, and the 2D IoU between the candidate box and the ground truth is used for classification and regression.

[0027] Preferably, the detailed steps of step three are:

[0028] (1) In the 3D point cloud, both the ground truth boxes and anchors are represented as a seven-dimensional vector (x, y, z, w, l, h, θ), where x, y, z represent the geometric center of the bounding box, and w, l, h represent the length, width, and height of the bounding box. The loss function is calculated as follows:

[0029]

[0030] Where: Δx represents the offset of the target in the x-axis direction, Δy represents the offset of the target in the y-axis direction, and Δz represents the offset of the target in the z-axis direction; χ gt ,y gt 、z gt They represent the real position coordinates of the target, χ a ,y a , g a Respectively represent the predicted position coordinates, d a Indicates the predicted distance, usually used to normalize the offset;

[0031] Δw represents the offset of the target width, Δl represents the offset of the target length, Δh represents the offset of the target height, and w gt , l gt 、h gt Respectively represent the width, length, and height of the target, w a , l a 、h a Represent the predicted width, length, and height respectively;

[0032] Δθ represents the offset of the target attitude angle, θ gt Represents the target's true attitude angle, θ a represents the predicted attitude angle;

[0033] The total position loss function uses the smooth1 function, and the formula is:

[0034]

[0035] For target classification loss, focal loss is used, and the formula is:

[0036]

[0037] Among them: α represents the balance factor, p a represents the predicted probability, γ represents the focusing parameter;

[0038] The total loss function calculation formula is:

[0039]

[0040] Where: N POS represents the number of positive samples, β reg , β cls , β dir Represent the weights of regression loss, classification loss and direction loss respectively, L reg , L cls , L dirr They represent regression loss, classification loss, and direction loss respectively.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention proposes an innovative pillar encoding module - a feature encoding network that integrates a triple attention mechanism, named Dual Attention and SENet Attention Pillar Encoding (DSPE). Compared with the existing PointPillars method, the DSPE module proposed in the present invention significantly enhances the ability to extract fine-grained information in 3D small target detection by introducing a triple attention mechanism. DSPE not only improves the problem of feature information loss caused by traditional maximum pooling operations, but also effectively captures the spatial correlation between points within each pillar, feature interactions between channels, and contextual dependencies between voxels. This makes the model perform better when processing tasks that require more details. At the same time, the DSPE module is efficient and flexible, and can greatly improve the detection accuracy and speed of the system without increasing additional computing burden and development costs, making it an ideal choice for real-time embedded applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The network flow chart of the present invention as a whole.

[0044] Figure 2 This is the working principle of the DSPE network proposed in the feature encoding network VFE module of the present invention. DETAILED DESCRIPTION

[0045] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0046] like Figure 1As shown, the present invention is an autonomous driving perception algorithm based on a triple attention mechanism, which is mainly composed of three parts: feature encoding network VFE, 2Dbackbone convolutional neural network and dense head. First, the original point cloud data of the scene scanned by the laser radar is input into the feature encoding network to be divided into pillars, and then converted into a 2D pseudo-image and input into the 2DBackbone for feature extraction, which is mainly achieved by downsampling the input feature map into three feature maps of different sizes and different channels to obtain multi-scale and multi-dimensional features, and then upsampling and splicing them before sending them to the detection head for detection tasks. The network has good performance in predicting and identifying small target objects in a 3D target perception environment.

[0047] 1. Feature Encoding Network VFE

[0048] The feature extraction network of the present invention converts 3D point cloud data into a 2D pseudo image. First, the 3D point cloud data is evenly divided into grids according to the xoy plane. The cuboid represented by each grid is a pillar. The number of pillars is set to M, and the points falling in a grid are regarded as being in a pillar. In the KITTI dataset, the feature of a point is represented by a four-dimensional vector, (x, y, z, r), which corresponds to the coordinates and reflection intensity of the point on the xyz axis. In our algorithm, we represent the feature of each point with a C=10-dimensional vector, which are (x, y, z, r, xc, yc, zc, xp, yp, zp), respectively, where x, y, z, r indicate the real coordinate information and reflection intensity of the point cloud, xc, yc, zc indicate the position offset of each point relative to the centroid of the body mass, and xp, yp, zp indicate the coordinate offset of each point relative to the center of the voxel to which it belongs. After the grid division and point feature representation are completed, due to the sparsity of point cloud data, many pillars have no points or very few points. In order to solve the problem of the number of points in different pillars, we set a hyperparameter N to limit the number of points in each pillar. If the number of points in a pillar exceeds N, N points are randomly sampled. If it is less than N, 0 is used for padding. In this way, the original 3D point cloud data constitutes a tensor of (P, N, C), where C represents the feature of the point, P represents the number of pillars, and N represents the number of points. The tensorization of the point cloud data is completed. The existing pointpillar performs a feature maximum pooling operation on the points in each pillar, changing the shape from (C, P, N) to (C, P), but this will result in only retaining the maximum value of the feature of each point in each pillar, thereby discarding other information. This may lead to the loss of important contextual information or subtle features. For some tasks that need to retain more details (such as fine-grained image classification or small object recognition in target detection), this information loss may be disadvantageous.

[0049] In order to further improve the performance of the autonomous driving perception system, the present invention innovatively proposes a feature encoding network called DSPE (Dual Attention and SENet Attention Pillar Encoding). The network aims to optimize the relationship between points in each pillar in the lidar point cloud data, the different channel information of each point, and the interaction between each pillar by introducing a triple attention mechanism. In the DSPE network, the interaction between points inside the pillar is first focused on. The traditional VFE layer usually processes all points in each pillar uniformly, ignoring the different importance that may exist between points. To this end, DSPE introduces an inter-point attention mechanism, which calculates adaptive weights for the points in each pillar to highlight those points that are more critical for target detection. This weighting strategy not only enhances the model's ability to capture local details, but also improves its adaptability to complex scenes. In addition to considering the relationship between points, DSPE also places special emphasis on the feature representation of each point in different channels. Since the lidar point cloud data contains rich attribute information (such as coordinates, reflection intensity, etc.), the importance of different channels is also different. To this end, DSPE implements a channel-level attention mechanism in each pillar. By evaluating the contribution of each channel and dynamically adjusting its weight, DSPE can more accurately extract the most representative features, thereby improving the overall feature expression ability. Finally, DSPE also pays attention to the global correlation between different pillars. Considering that the distribution of objects in actual scenes is often non-uniform, pillars in certain areas may contain more valuable information. Therefore, DSPE draws on the idea of ​​SENet (Squeeze-and-Excitation Network) and implements the SENeT attention mechanism between each pillar to evaluate the importance of each pillar in the entire point cloud space and allocate different computing resources accordingly. This method ensures that the model can focus more attention on key areas, reduce unnecessary computational burden, and improve detection efficiency.

[0050] like Figure 2 As shown, the following is a detailed description of the design concept and working principle of the DSPE network:

[0051] After completing the tensorization of the point cloud as described above, we get the data of P1∈(P×N×D), and then pass through a simplified version of PointNet to upgrade the dimension to get P1∈(P, N, D), where C=10 and D=64. Then the attention mechanism between points is performed: first, P1∈(P, N, D) is transformed into T2∈(N, P, D) through position transformation Transpose. This transformation ensures that all points in each pillar can be compared and weighted in the same dimension. Then the multi-head attention mechanism is applied to calculate the similarity between points and perform weighted processing. This process not only enhances the model's understanding of local details, but also improves the efficiency and accuracy of feature extraction. Finally, the weighted feature matrix O∈(N×P×D) is generated. Then the dimension is transformed back to O∈(P, N, D) and the first one is taken to get O1∈(P, 1, D). Then, P1∈(P, N, D) is transformed into H2∈(D, P, N) through position transformation, and then the multi-head attention mechanism is applied to calculate the similarity between the channels of each point and perform weighted processing. Finally, the weighted feature matrix K∈(D×P×N) is generated, and then the dimension is transformed back to K∈(P, N, D) and the first one is taken to obtain K1∈(P, N, 1). O1∈(P, 1, D) and K1∈(P, N, 1) are multiplied element by element to obtain the fused feature matrix KO1∈(P, N, D). This process combines the channel information of O1O1 with the spatial position information of K1 to generate a new feature representation containing rich contextual relationships. Then, the original feature matrix P1 is multiplied with the fused feature matrix KO1 to obtain the weighted feature matrix KOP1∈(P×N×D). This operation aims to dynamically adjust the original features according to the weights calculated in KO1, so as to highlight those features that are more critical to target detection. Finally, the weighted feature matrix KOP1 is input into the SENet (Squeeze-and-Excitation Network) module to further optimize the feature expression. SENet re-evaluates the importance of each channel and dynamically adjusts its weights to ensure that the model can capture the most representative features more accurately. The features processed by SENet not only enhance the ability to understand local details, but also improve the adaptability to complex scenes. The weighted feature matrix KOP1 is subjected to the maximum pooling (MaxPooling) operation to extract the most representative features and reduce the feature dimension. After this step, the reduced-dimensional feature matrix KOP2∈(P×D) is obtained. Maximum pooling not only simplifies the complexity of subsequent processing, but also retains the most important local extreme value information, enhancing the robustness of the model.At the same time, the maximum pooling operation is also applied to the original feature matrix P1 to extract the global feature G1∈(P×D), where D represents the global feature dimension of each pillar. This process captures the maximum response of all points in each pillar and provides a global perspective for subsequent feature fusion. Finally, the global feature G1 and the reduced feature KOP2 are added point by point (Element-wiseAddition) to generate a comprehensive feature matrix F∈(P×D). This point-by-point addition operation enables the effective combination of local and global features, enhances the model's ability to understand features of different scales, and improves the quality of the final feature representation. Finally, the point features are dispersed back to their pillars to obtain a (C, H, W) tensor, that is, a pseudo image, where H and W represent the length and width of the pseudo image, respectively. The pseudo image is generated so that it can be sent to the next 2D backbone for feature extraction.

[0052] 2. 2DBackBone

[0053] The 2D backbone of the present invention is composed of two sub-networks. In the first stage, the feature pseudo-image is further fed into a top-down downsampling network. This network is composed of three downsampling blocks, each of which contains several convolution layers, batch normalization layers and activation functions to gradually extract higher-level semantic information. As each block passes through, the spatial resolution of the feature map will continue to decrease, and the number of channels may increase at the same time, so as to capture more abstract and complex patterns, as follows: first, the feature pseudo-image size (C, H, W), after the first Block, the first layer of the first block is a step size of 2, the convolution kernel size is 3×3, the input channel is C, and the output channel is C. The second, third and fourth layers all have the same step size of 1, the convolution kernel size is 3×3, the input channel is C, and the output channel is C. A Relu and a batch normalization layer BatchNorm2d are added behind each layer, and finally the pseudo-image size is (C, H / 2, W / 2), which is recorded as W1. Then W1 is sent to the second Block. The first layer of the second block has a stride of 2, a convolution kernel size of 3×3, an input channel of C, and an output channel of 2C. The second, third, fourth, fifth, and sixth layers all have the same stride of 1, a convolution kernel size of 3×3, an input channel of 2C, and an output channel of 2C. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (2C, H / 4, W / 4), denoted as W2. Finally, W2 is sent to the third Block. The first layer of the third Block consists of a convolution kernel with a stride of 2×2 and a convolution kernel size of 3×3, an input channel of 2C, and an output channel of 4C. The second, third, fourth, fifth, and sixth layers all have the same stride of 1 and a convolution kernel size of 3×3, an input channel of 4C, and an output channel of 4C. A Relu and a batch normalization layer BatchNorm2d are added after each layer, and the final pseudo image size is (4C, H / 8, W / 8), denoted as W3.

[0054] Then we come to the second stage, which is to perform an upsampling. First, W1 passes through the first deblock. The first deblock consists of a convolution kernel with a stride of 1 and a convolution kernel size of 1×1, with an input channel of C and an output channel of 2C, followed by a Relu and a batch normalization layer BatchNorm2d, and the size of the pseudo image is (2C, H / 2, W / 2), recorded as H1. Then W1 passes through the second deblock. The second deblock consists of a convolution kernel with a stride of 2 and a convolution kernel size of 2×2, with an input channel of 2C and an output channel of 2C, followed by a Relu and a batch normalization layer BatchNorm2d, and the size of the pseudo image is (2C, H / 2, W / 2), recorded as H2. Then after the last deblock, the third deblock consists of a convolution with a stride of 4 and a kernel size of 4×4, with 4C input channels and 2C output channels, followed by a Relu and a batch normalization layer BatchNorm2d, and finally a pseudo image with a size of (2C, H / 2, W / 2) is obtained, denoted as H3. Finally, H1, H2, and H3 are concatenated together to obtain a feature map H∈(6C, H / 2, W / 2).

[0055] 3. Detection head dense head

[0056] The present invention uses SSD as the detection head of the model. The difference from other 3D detection heads is that the model of the present invention converts 3D point cloud data into 2D pseudo images for processing. Therefore, the 2D IoU between the candidate box and the groundtruth is used for classification and regression. In the 3D point cloud, we represent the ground truth boxes and anchors as a seven-dimensional vector (x, y, z, w, l, h, θ), where x, y, z represent the geometric center of the bounding box, w, l, h represent the length, width and height of the bounding box, and the loss function calculation formula is as follows (1):

[0057]

[0058] The total position loss function uses the smooth1 function, and the calculation formula is as follows (2)

[0059]

[0060] For target classification, focal loss is used, and the calculation formula is as follows (3)

[0061]

[0062] The total loss function calculation formula is as follows (4)

[0063]

[0064] The model of the present invention is embedded in the target detection and segmentation module in the perception link of autonomous driving. For the original point cloud data collected by the lidar, the output is the detection result described by 3D bounding boxes and confidence prediction after passing through our model, so as to facilitate the decision-making after the perception link.

[0065] 4. Training details

[0066] (1) Data preparation: Extract point cloud information from data collected by LiDAR (input).

[0067] (2) Point cloud preprocessing: Corresponding to the feature encoding network mentioned above, the point cloud data is divided into pillars to ensure that the point cloud data input into the 2D backbone has a uniform format and size.

[0068] (3) Network training: A deep learning network consisting of a deep neural network and a target detection head has the function of multi-level feature extraction.

[0069] (4) Loss function: The loss function used is the same as above.

[0070] (5) Training process: The KITTI dataset is used to train the network. The training epoch is set to 120, the x and y dimensions are set to 0.16 m respectively, the maximum number of pillars is (P) 16000, and the maximum number of points in each pillar is (N) 32. If the number exceeds the limit, 100 points are randomly sampled, and if the number is less, 0 is used to fill the gap. The size of each pillar is set to 0.16x0.16x4, where 0.16x0.16 is the size after voxelization on the xoy plane.

[0071] During training, the ranges of the point cloud xyz axis are set to (0, 69.12), (-39.68, 39.68), and (-3, 1). In the training process, we only focus on the front view of the point cloud scene, and the rest of the points and gt boxes are not used for training. We use the same anchor boxes and matching strategies as [voxelnet]. Each anchor box is represented by width, length, height, and z center, and sets IoU to determine whether the sample belongs to a positive sample or a negative sample. For the three categories in the KITTI dataset, due to their differences in shape and volume, the sizes of the anchor boxes they require are also different. We set three different sizes for each category, and for each anchor box, two different directions of 0° and 90° are set. For the two categories of pedestrians and bicycles, the sizes of the anchor boxes are set to (0.8, 0.6, 1.73) meters and (1.75, 0.6, 1.73) meters. They are all centered at -0.6 meters in the direction, and the matching IoU thresholds are set to 0.5 and 0.5 respectively.

[0072] In the sample allocation process, the target allocator of [yolo9000] is used. Specifically, the samples with the highest IoU or IoU exceeding the set positive sample IoU threshold are recorded as positive samples, the samples with IoU lower than the set negative sample threshold are recorded as negative samples, and the samples with IoU between the set positive sample threshold and the negative sample threshold are discarded. At the same time, the KITTI test set and validation set are used to evaluate the model, and the training results are uploaded to the KITTI official website.

[0073] (6) Data enhancement: Data enhancement is very helpful for model training. We use some common 3D data enhancement strategies (local translation, local rotation, local scaling) to improve the generalization ability of the model and reduce overfitting.

[0074] In 3D object detection, small objects often lack fine information due to their limited spatial range and inherent resolution limitations of sensors. This deficiency can lead to poor detection performance as the model may fail to capture the complex details required for accurate localization and classification. To effectively address the challenge of fine-grained information extraction in 3D small object detection, we propose an innovative pillar encoding module - a pillar encoding method that combines triple attention mechanisms, named Dual Attention and SENet Attention Pillar Encoding (DSPE).

[0075] The design of the DSPE module not only emphasizes efficiency, but also ensures its easy integration. Specifically, the module has a significant "plug and play" advantage and can be directly embedded in various 3D target detection frameworks without complex adjustments to the existing network architecture. By introducing DSPE, system performance can be greatly improved while avoiding the additional computing burden and development costs brought by redundant modules. This efficient and flexible solution makes DSPE an ideal choice for improving the accuracy and speed of 3D small target detection, especially for application scenarios with high requirements for real-time and resource utilization.

[0076] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An autonomous driving perception algorithm based on a triple attention mechanism, characterized by: It includes three parts: feature encoding network, 2D backbone and detection head, including the following steps: Step 1: Input the original point cloud data of the scene scanned by the lidar into the feature encoding network and divide it into pillars, thereby converting the 3D point cloud data into a 2D pseudo image; Step 2: Input the transformed 2D pseudo-image into 2DBackbone for feature extraction, which is mainly done by downsampling the input feature map into three feature maps of different sizes and channels, so as to obtain multi-scale and multi-dimensional features, and then upsampling and splicing; Step 3: Send it to the detection head for prediction task.

2. The autonomous driving perception algorithm based on triple attention mechanism as claimed in claim 1, characterized in that: The detailed working steps of the feature encoding network in step 1 are as follows: (1) The 3D point cloud data is evenly divided into grids according to the xoy plane. The cuboid represented by each grid is a pillar. The number of pillars is set to M, and the points falling in a grid are considered to be in a pillar. In the KITTI dataset, the feature of a point is represented by a four-dimensional vector, (x, y, z, r) corresponds to the coordinates and reflection intensity of the point on the xyz axis, respectively. Each point is represented by a C = 10-dimensional vector, namely (x, y, z, r, xc, yc, zc, xp, yp, zp), where x, y, z, r indicate the real coordinate information and reflection intensity of the point cloud, xc, yc, zc indicate the position offset of each point relative to the centroid of the body, and xp, yp, zp indicate the coordinate offset of each point relative to the center of the voxel to which it belongs. (2) Set a hyperparameter N to limit the number of points in each pillar. If the number of points in a pillar exceeds N, N points are randomly sampled. If it is less than N, it is filled with 0. In this way, the original 3D point cloud data constitutes a tensor (P, N, C), where C represents the number of feature channels of the point, P represents the number of pillars, and N represents the number of points. The tensorization of the point cloud data is now completed. (3) After completing the tensorization of the point cloud in step (2), the data P1∈(P×N×C) is obtained, and then a simplified version of PointNet is used to upgrade the dimension to obtain P1∈(P, N, D), where C=10, D=64; D represents the number of feature channels of each point in the point cloud; (4) Perform an attention mechanism between points: First, transform P1∈(P, N, D) into T2∈(N, P, D) through position transformation, apply the multi-head attention mechanism to calculate the similarity between points and perform weighted processing to generate a weighted feature matrix O∈(N×P×D); then transform the dimension back to O∈(P, N, D) and take the first one to obtain O1∈(P, 1, D), transform P1∈(P, N, D) into H2∈(D, P, N), apply the multi-head attention mechanism to calculate the similarity between the channels of each point and perform weighted processing The weighted feature matrix K∈(D×P×N) is generated by processing the original feature matrix P1 and the fused feature matrix KO1∈(P×N×D). The weighted feature matrix KOP1∈(P×N×D) is obtained by multiplying O1∈(P, 1, D) and K1∈(P, N, 1) element by element. The weighted feature matrix KOP1∈(P×N×D) is obtained by multiplying the original feature matrix P1 and the fused feature matrix KO1. Finally, the weighted feature matrix KOP1 is input into the SENet module to further optimize the feature expression. (5) After step (4), the weighted feature matrix KOP1 is subjected to the maximum pooling operation to extract the most representative features and reduce the feature dimension. After this step, the reduced-dimensional feature matrix KOP2∈(P×D) is obtained. At the same time, the original feature matrix P1 is also subjected to the maximum pooling operation to extract the global features G1∈(P×D). The global features G1 and the reduced-dimensional features KOP2 are added point by point to generate a comprehensive feature matrix F∈(P×D). Finally, the point features are dispersed back into their pillars to obtain a (C, H, W) tensor, i.e., a pseudo image. H and W represent the length and width of the pseudo image, respectively. The pseudo image is generated so as to be sent to the next 2D backbone for feature extraction.

3. The autonomous driving perception algorithm based on triple attention mechanism as claimed in claim 2, characterized in that: The detailed working steps of feature extraction by 2Dbackbone in step 2 are as follows: (1) The feature pseudo image size (C, H, W) passes through the first block. The first layer of the first block has a stride of 2, a convolution kernel size of 3×3, an input channel of C, and an output channel of C. The second, third, and fourth layers all have the same stride of 1, a convolution kernel size of 3×3, an input channel of C, and an output channel of C. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (C, H / 2, W / 2), denoted as W1; (2) W1 is sent to the second Block. The first layer of the second block has a stride of 2, a convolution kernel size of 3×3, an input channel of C, and an output channel of 2C. The second, third, fourth, fifth, and sixth layers all have the same stride of 1, a convolution kernel size of 3×3, an input channel of 2C, and an output channel of 2C. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (2C, H / 4, W / 4), denoted as W2; (3) W2 is sent to the third Block. The first layer of the third Block consists of a convolution kernel with a stride of 2×2 and a convolution kernel size of 3×3, an input channel of 2C, and an output channel of 4C. The second, third, fourth, fifth, and sixth layers all have the same stride of 1 and a convolution kernel size of 3×3, an input channel of 4C, and an output channel of 4C. A Relu and a batch normalization layer BatchNorm2d are added after each layer. Finally, the pseudo image size is (4C, H / 8, W / 8), denoted as W3. (4) W1 passes through the first deblock, which consists of a convolution kernel with a stride of 1 and a kernel size of 1×1, with an input channel of C and an output channel of 2C, followed by a Relu and a batch normalization layer BatchNorm2d, and the size of the pseudo image is (2C, H / 2, W / 2), denoted as H1; (5) W1 is passed through the second deblock, which consists of a convolution kernel with a stride of 2 and a kernel size of 2×2, with 2C input channels and 2C output channels, followed by a Relu and a batch normalization layer BatchNorm2d, resulting in a pseudo-image of size (2C, H / 2, W / 2), denoted as H2; (6) After the last deblock, the third deblock consists of a convolution with a stride of 4 and a kernel size of 4×4, with 4C input channels and 2C output channels, followed by a Relu and a batch normalization layer BatchNorm2d. Finally, a pseudo image with a size of (2C, H / 2, W / 2) is obtained, denoted as H3; (7) Concatenate H1, H2, and H3 together to obtain the feature map H∈(6C, H / 2, W / 2).

4. The autonomous driving perception algorithm based on triple attention mechanism as claimed in claim 3, characterized in that: In step 3, SSD is used as the detection head of the model, and the 2D IoU between the candidate box and the ground truth is used for classification and regression.

5. The autonomous driving perception algorithm based on triple attention mechanism as claimed in claim 4, characterized in that: The detailed steps of step three are: (1) In the 3D point cloud, both the ground truth boxes and anchors are represented as a seven-dimensional vector (x, y, z, w, l, h, θ), where x, y, z represent the geometric center of the bounding box, and w, l, h represent the length, width, and height of the bounding box. The loss function is calculated as follows: The total position loss function uses the smooth1 function, and the formula is: For target classification, focal loss is used, and the formula is: The total loss function calculation formula is:

Images