Lightweight PointPill automatic driving 3D target detection method based on cylinder self-attention

Abstract

The invention discloses a lightweight PointPillars automatic driving 3D target detection method based on cylinder self-attention, and the method comprises the steps: obtaining a point cloud data set, and cutting a surrounding 360-degree point cloud image into an image visual angle; feature extraction is carried out on the point cloud data by using an encoder network structure of a transformer, and then dimension raising of point cloud features is carried out through a multi-layer perceptron; cutting the point cloud data into columns, and distributing the columns into an original world to form a 2D pseudo image; the method comprises the steps that firstly, a pseudo image is obtained, then the pseudo image is sent to a 2D backbone network for 2D convolution operation, then a 2D feature map is obtained and sent to a detection head based on an SSD, finally, a prediction result is input, and the prediction result comprises 3D center point coordinates, the length, the width, the height and the steering angle of an object; according to the method, the transfomer technology and the depth separable convolution technology are introduced into the feature extraction part, so that the context semantic feature extraction capability of the network is improved, the parameter quantity of the network is greatly reduced, and light weight is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving vehicles, and in particular to a lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention. Background Art

[0002] In recent years, the development trend of technology is to introduce automation and intelligence into life, thereby improving people's quality of life. With the rapid development of social economy and science and technology, cars have become popular and gradually become one of the necessary ways for people to travel. Whether it is commuting to get off work or traveling on holidays, cars are closely related to our lives. Correspondingly, while enjoying the great convenience brought by cars, we will inevitably face some problems. Long-term and high-intensity commuting pressure can easily lead to driving fatigue, making it difficult for drivers to concentrate, and the driving experience will also deteriorate. And it will greatly increase the probability of traffic accidents. Among them, accidents caused by human factors account for a large proportion of accidents, such as driving skills, psychological changes and fatigue, which will increase the risk of traffic accidents. Autonomous driving technology is regarded as one of the most promising remedies to protect humans from serious collisions and has received more and more attention. The key to autonomous driving lies in perception, especially 3D target detection, which is the core foundation of the perception system and is crucial for path planning, motion prediction, collision avoidance and other aspects.

[0003] For self-driving cars to be as intelligent as humans, the core is environmental perception. 3D object detection plays a vital role in Autonomous Vehicles (AVs). LiDAR and color cameras are the main sensors equipped on AVs. Object detection is mainly divided into two categories: 2D object detection and 3D object detection. 2D object detection can only obtain the position information of the target in a two-dimensional image, and the 2D size obtained by 2D object detection alone cannot achieve this goal. In contrast, 3D object detection can obtain information such as the position, size, and heading angle of an object in three-dimensional space, so that the center point of the bottom of the vehicle can be calculated. 3D object detection mainly relies on devices such as LiDAR, binocular cameras, and monocular cameras.

[0004] Currently, most powerful 3D detectors rely on LiDAR because the device can provide accurate depth information based on a 3D point cloud. However, LiDAR-based system equipment also has some limitations. In bad weather (such as rain, snow, and fog), LiDAR's ability to perceive the environment will be reduced, resulting in incomplete or inaccurate data. Secondly, when detecting objects with different surface reflectivity (such as mirrors, glass, water, etc.), LiDAR may miss detections or make false detections. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention, aiming to solve the limitations and defects of lidar detection in the prior art, which leads to inaccurate detected 3D target data and a large number of parameters, making it impossible to lightweight the 3D target detection method.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention, including:

[0008] Step 1: Obtain point cloud data, image data, and equipment parameters of public roads, where the equipment parameters include internal and external parameters of the camera, origin coordinates of the laser radar, and annotated ground truth annotation boxes;

[0009] Step 2: Crop the point cloud data, crop the 360° point cloud around the vehicle body to only the point cloud that appears in the camera view;

[0010] Step 3: Then input the processed original point cloud into the first improved PillarsFeature Net module of PointPillars, add a local transformer structure to the Pillars Feature Net module to improve the feature extraction capability, and finally obtain a 2D pseudo image;

[0011] Step 4: Send the pseudo image obtained in the previous step to the channel attention-based deep separable lightweight backbone network for feature extraction to obtain a 2D feature map;

[0012] Step 5: Send the 2D feature map to the SSD-based object detection head to obtain the detection result;

[0013] Step 6: Mark the detection results into the original point cloud to achieve the 3D object detection task of the self-driving car in the 3D world.

[0014] Furthermore, the local self-attention transformer process of the Pillars Feature Net module in step 3 includes:

[0015] Step 3-1: Get the cropped original point cloud data obtained in step 2, which contains the (X, Y, Z, R) of all points in a scene that appear in the camera's field of view, which represent the 3D world coordinates and surface reflection intensity of a single point respectively. The 3D world coordinates are established in the laser radar coordinate system, with the front of the car as the x-axis, the left side of the car as the y-axis, and the top of the car as the z-axis;

[0016] Step 3-2: Columnarize the point cloud. Send the acquired original point cloud data to the voxelize module for cutting. Cut on the xoy plane and set the size of the cut cylinder to [0.16, 0.16, 4]. Calculate the corresponding coordinates of each cylinder in the original three-dimensional world and return them to the PillarsFeature Net module as the return value.

[0017] Step 3-3: Feature dimension upgrade. After obtaining the segmented cylinders, calculate the cluster center of all points in each cylinder, the center point offset of each cylinder, and the distance from the observer to the point, all of which are considered as the features of the point. Finally, a point with a feature of (4+6)=10 is obtained.

[0018] Step 3-4: Then send it to the local self-attention transformer for local feature extraction and attention calculation; after the previous step, a tensor with a shape such as [batchsize*6542, 32, 10] is obtained, where the first dimension represents the number of cylinders in a sample in this batch, the second dimension represents the number of points contained in each cylinder, and the third dimension represents the specific feature information of each point; send the tensor to the encoder structure of the transformer, first perform position encoding on it, and the encoding formula is:

[0019]

[0020] Among them, PE is the result of position encoding, p is the number of points of the current point, i represents the number of dimensions of the current feature, and d f Represents the overall characteristic dimension;

[0021] The autocorrelation of each bin is then calculated:

[0022]

[0023] In the formula, Q, K, and V represent the query matrix, key matrix, and value matrix respectively. Represents the scaling ratio to prevent the calculation result from exceeding the limit; the final result Atten(Q, K, V) represents the importance of each point relative to other points;

[0024] Using multi-head attention mechanism:

[0025] MultiHead(Q,K,V)=Concat(head1,...,head n )W t ;

[0026]

[0027] In the formula, head1...represent different sub-vectors, They are the weights of the sub-vectors, and then all the sub-vectors are concatenated together and finally passed through a weight W. t The final attention result is obtained; then it goes through a residual structure and forward propagation module and performs multiple iterations;

[0028] Step 3-5: Finally, the point cloud features passed through the encoder are restored into a 2D pseudo image based on the relative coordinates.

[0029] Furthermore, it is characterized in that the 2D feature extraction network in step 4 includes:

[0030] After obtaining the 2D pseudo image in step 3, the lightweight network is redesigned;

[0031] The size of the input image is [64, 496, 432]. After three downsamplings, feature maps of three different scales will be saved, namely [64, 248, 216], [128, 124, 108], and [256, 62, 54]. Then, different transposed convolutions are performed to restore the image to a uniform size of [384, 124, 108]. Finally, the feature maps containing features of different scales are concatenated.

[0032] Among them, all convolutions of the network use depthwise separable convolution to reduce the number of model parameters. The Bneck structure uses a channel-based attention mechanism to improve the feature extraction ability of the model. The Bneck structure contains the SEBlock channel attention module, whose main function is to allow the network to pay attention to those more important feature layers. At the same time, the residual structure is used to prevent gradient disappearance and gradient explosion.

[0033] Furthermore, in step 4, all parameters of the convolutional layer are transformed from h o ·w o ·a o 2 ·d o ·d o ' drops to h o ·w o ·(a o 2 +d o )·d o′; The total parameter volume is 24.3% of the original model.

[0034] Furthermore, the SSD detection head detection in step 5 includes:

[0035] Step 5-1: The 2D feature map obtained in step 4 is first passed through three different prediction heads, namely, the classification head, the regression head, and the direction prediction head; different prediction results will be obtained for different prediction heads. For the classification prediction head, the classification categories are 3 categories, and the cross entropy loss is used for calculation; for the regression prediction head, the position information of the 3D object is mainly predicted, specifically the coordinate information of 7 objects is predicted; for the direction prediction head, the binary cross entropy loss is used for calculation;

[0036] Step 5-2: For the ground truth label, each object has 7 parameters, namely [x, y, z, w, l, h, θ], where the first three represent x, y, z, w, l, h respectively represent width, length, height in the lidar coordinate system, and θ represents the heading angle;

[0037] SmoothL1 regression loss is used between the real ground mark and the predicted box, where the regression loss formula is:

[0038]

[0039] Δθ=sin(θ t -θ a );

[0040] where x t y t z t represents the ground truth label, x a y a z a represents the prediction box, w t l t h t Represents the width, length, and height of the label, w a l a h a Represents the predicted width, length, and height; θ t θ a Represent the labeled steering angle and the predicted steering angle respectively;

[0041] The overall regression loss is:

[0042]

[0043] Considering that in a three-dimensional environment, the positioning loss cannot identify the angles of two opposite positioning frames, the softmax heading angle classification loss is used to learn the heading angle of the three-dimensional object;

[0044] The object classification loss used is focal loss:

[0045] L cls =-α a (1-p a ) γ log p a :

[0046] In the formula, the classification probability score is expressed as p a Indicates that the hyperparameter α a =0.25,γ=2;

[0047] The total loss is expressed as:

[0048]

[0049] Among them, N pos Represents the number of anchor boxes assigned as positive samples, and again adds weights to different losses, because 3D target detection is more concerned with the coordinate information of the object in the 3D world, so β loc =2,β cls =1,β dir =0.2.

[0050] Furthermore, the prediction results obtained in step 5-2 are evaluated:

[0051] Typically, the main performance parameter for 3D object recognition is the Average Precision (AP); it is slightly modified in the KITTI benchmark to use the KITTI standard interpolation API R40 To evaluate the performance of the model:

[0052]

[0053] Among them, R = [0, 1 / 40, 2 / 40, ..., 1] represents different recall rate thresholds, P represents the precision under different recall rates; different IoU thresholds (0.7, 0.5, 0.5) correspond to cars, pedestrians and cyclists respectively, and filter out prediction boxes with low confidence; finally, non-maximum suppression NMS is used to retain the prediction box with the highest confidence.

[0054] Furthermore, in step six, the detection results obtained in step five are marked into the original point cloud, a new point cloud with detection results is generated, and the prediction results are visualized to realize the 3D target detection task of the autonomous driving car in the 3D world.

[0055] The technical solution adopted by the present invention has the following beneficial effects:

[0056] In the present invention, transformer technology and deep separable convolution technology are introduced in the feature extraction part, the Pillars Feature Net module is improved in PointPillars, and a local transformer structure is added to the Pillars Feature Net module, which can enhance the network's ability to extract contextual semantic features and avoid inaccurate 3D target data detected by the lidar; the deep separable convolution technology can greatly reduce the number of network parameters and achieve lightweight, so as to realize 3D target detection in autonomous driving vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is the original PointPillars model diagram;

[0058] Figure 2 It is the overall flow chart of the present invention;

[0059] Figure 3 The detailed structure of the MSA (local point attention) part in the overall flow chart of the present invention;

[0060] Figure 4 It is a 2D backbone network flow chart;

[0061] Figure 5 Schematic diagram of the Bneck structure and channel attention module;

[0062] Figure 6 Schematic diagram of depth-wise separable convolution. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0064] Embodiment 1:

[0065] The present invention provides a lightweight PointPillars autonomous driving 3D target detection method based on column self-attention. The original PointPillars Figure 1 As shown. In the present invention, two modules are added to PointPillars, such as Figure 2As shown in the figure, firstly, a local transformer is added inside each column, located in the Point Cloud MSA, and then the depthwise separable convolution and channel attention mechanism are adopted in Backbone. The accuracy of the model is improved and the number of parameters of the model is reduced respectively. The former is to improve the context feature extraction capability of the model by adding a local transformer encoder structure to each column after the point cloud is columnarized; the latter is achieved by adopting a complete depthwise separable convolution and channel attention mechanism. It includes the following steps:

[0066] Step 1: Obtain point cloud data, image data, and some equipment parameters of public roads.

[0067] First, we obtain the original point cloud data, camera parameters, lidar parameters and annotation information through the KITTI official website. Then we divide the data into training sets and test sets, including 3712 training sets and 3769 test sets.

[0068] Step 2: Crop the point cloud, crop the surrounding 360° point cloud to only the point cloud that appears in the camera view.

[0069] Step 3: The processed original point cloud is then input into the first improved PillarsFeature Net module of PointPillars, in which a local transformer structure is innovatively added to improve the feature extraction capability, and finally a 2D pseudo image is obtained.

[0070] The cropped original point cloud data obtained in step 3. It contains the (X, Y, Z, R) of all points in a scene that appear in the camera's field of view, which represent the three-dimensional world coordinates and surface reflection intensity of a single point respectively. The coordinates are in the laser radar coordinate system, with the front of the car being the x-axis, the left being the y-axis, and the top being the z-axis. Then the point cloud is columnarized, and the acquired original point cloud data is sent to the voxelize module for cutting. The cutting is done on the xoy plane, not in the z-axis direction, and the size of the cut cylinder is specified as [0.16, 0.16, 4], which is the size of a cylinder, and the corresponding coordinates of each cylinder in the original three-dimensional world are calculated and passed back as the return value.

[0071] In step 3, the features are further dimensioned. Specifically, after obtaining the segmented cylinders, the cluster centers of all points, the center point offset of each cylinder, and the distance from the observer to the point are additionally calculated, all of which are considered as the features of the point. Finally, a point with a feature of (4+6)=10 is obtained. Then it enters the transformer structure for feature extraction, such as Figure 3The following is a detailed diagram of the transformer structure, which specifically uses position encoding and self-attention mechanisms. The formula is as follows:

[0072]

[0073] Among them, PE is the result of position encoding, p is the number of points of the current point, i represents the number of dimensions of the current feature, and d f Represents the overall characteristic dimension.

[0074] Then calculate the autocorrelation of each column using the formula:

[0075]

[0076] Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively. Represents the scaling ratio to prevent the calculation result from exceeding the limit. The final result Atten(Q, K, V) represents the importance of each point relative to other points. The multi-head attention mechanism formula is also used:

[0077] MultiHead(Q,K,V)=Concat(head1,...,head n )W t

[0078]

[0079] Where head1...represent different sub-vectors, They are the weights of the sub-vectors, and then all the sub-vectors will be concatenated together through Concat, and finally passed through a weight W t The final attention result is obtained. Then it will go through a residual structure and a forward propagation module, and this process will go through multiple iterations.

[0080] In step 3, the 2D pseudo image is finally mapped to obtain a feature map of [64, 496, 432].

[0081] Step 4: Send the pseudo image obtained in the previous step into the channel attention-based deep separable backbone network for feature extraction to obtain a 2D feature map.

[0082] In step 4, the size of the input image is [64, 496, 432]. After three downsamplings, feature maps of three different scales will be saved, namely [64, 248, 216], [128, 124, 108], and [256, 62, 54]. Then, different transposed convolutions are performed to restore the image to a uniform size of [384, 124, 108]. Finally, the feature maps containing features of different scales are concatenated.

[0083] The convolution of the network all uses depth-separable convolution to reduce the number of model parameters, such as Figure 4 The figure shows the specific structure of the network, in which the Bneck structure also uses a channel-based attention mechanism to improve the model's feature extraction ability. It also includes the SEBlock channel attention module, whose main function is to allow the network to pay attention to more important feature layers, such as Figure 5 Shown is the processing flow of Bneck and SEBlock.

[0084] And the residual structure is also used to prevent the gradient from disappearing and exploding, such as Figure 6 As shown in Figure 1, the residual link edges enable the model to avoid gradient disappearance and gradient explosion.

[0085] Step 5: Send the 2D feature map to the SSD-based object detection head to get the result.

[0086] Specifically, in step 5, the feature map obtained in step 4 is first passed through three different prediction heads, namely, the classification head, the regression head, and the direction prediction head. Different prediction heads will give different prediction results. For the classification prediction head, the classification categories are 3 categories, and the cross entropy loss is used for calculation; for the regression prediction head, the position information of the 3D object is mainly predicted, and the coordinate information of the 7 objects is predicted; for the direction prediction head, the binary cross entropy loss is used for calculation.

[0087] For the ground truth label, each object has 7 parameters, namely [x, y, z, w, l, h, θ], where the first three represent x, y, z, w, l, h respectively represent width, length, and height in the lidar coordinate system, and θ represents the heading angle.

[0088] SmoothL1 regression loss is used between the real ground mark and the predicted box, where the regression loss formula is:

[0089]

[0090] Δθ=sin(θ t -θ a )

[0091] where x t y t z t represents the ground truth label, x a y a z a represents the prediction box, w t l t h t Represents the width, length, and height of the label, w a la h a Represents the predicted width, length, and height. θ t θ a Represent the steering angle of the label and the predicted steering angle respectively. The overall regression loss is:

[0092]

[0093] Considering that in a three-dimensional environment, the positioning loss cannot identify the angles of two opposite positioning frames, the softmax heading angle classification loss is used to learn the heading angle of the three-dimensional object. The object classification loss used is focalloss:

[0094] L cls =-α a (1-p a ) γ log p a

[0095] The classification probability score is expressed as p a Indicates that the hyperparameter α a =0.25, γ=2, the total loss is expressed as:

[0096]

[0097] Where N pos Represents the number of anchor boxes assigned as positive samples, and again adds weights to different losses, because the 3D object detection task is more concerned with locating the coordinate information of the object in the 3D world, so β loc =2,β cls =1,β dir =0.2.

[0098] The evaluation index used in step 5 is the KITTI standard interpolated AP|R40 to evaluate the performance of the model, as shown below:

[0099]

[0100] Where R = [0, 1 / 40, 2 / 40, ..., 1] represents different recall thresholds, and P represents the precision at different R (recall). Different (IoU) thresholds (0.7, 0.5, 0.5) correspond to cars, pedestrians, and cyclists, respectively. Prediction boxes with lower IoU than the defined ones are filtered out. Finally, non-maximum suppression (NMS) is used to retain the prediction box with the highest confidence.

[0101] Step 6: Mark the prediction box obtained in step 5 into the original point cloud, generate a new point cloud with detection results, visualize the prediction results, and realize the 3D target detection task of the autonomous driving car in the 3D world. Then, the data can be passed to the downstream module and handed over to the decision module to make the next decision action based on the predicted 3D target.

[0102] Experimental Results

[0103] Table 1. Comparison results between different models

[0104]

[0105] According to the results in Table 1 above, the model of the present invention greatly reduces the number of parameters required for the model while maintaining accuracy. Only more than 1M parameters are needed to achieve a good mAP effect, which greatly reduces the number of network parameters and achieves lightweight, and has achieved good results in detecting vehicles.

[0106] In the present invention, transformer technology and deep separable convolution technology are introduced in the feature extraction part, the Pillars Feature Net module is improved in PointPillars, and a local transformer structure is added to the Pillars Feature Net module, which can enhance the network's ability to extract contextual semantic features and avoid inaccurate 3D target data detected by the lidar; the deep separable convolution technology can greatly reduce the number of network parameters and achieve lightweight, so as to realize 3D target detection in autonomous driving vehicles.

[0107] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the schemes disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the claims.

Claims

1. A lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention, characterized in that: include: Step 1: Obtain point cloud data, image data, and equipment parameters of public roads, where the equipment parameters include internal and external parameters of the camera, origin coordinates of the laser radar, and annotated ground truth annotation boxes; Step 2: Crop the point cloud data, crop the 360° point cloud around the vehicle body to only the point cloud that appears in the camera view; Step 3: Then input the processed original point cloud into the first improved PillarsFeature Net module of PointPillars, add a local transformer structure to the Pillars Feature Net module to improve the feature extraction capability, and finally obtain a 2D pseudo image; Step 4: Send the pseudo image obtained in the previous step to the channel attention-based deep separable lightweight backbone network for feature extraction to obtain a 2D feature map; Step 5: Send the 2D feature map to the SSD-based object detection head to obtain the detection result; Step 6: Mark the detection results into the original point cloud to achieve the 3D object detection task of the self-driving car in the 3D world.

2. The lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention according to claim 1 is characterized in that: The local self-attention transformer process of the Pillars Feature Net module in step 3 includes: Step 3-1: Get the cropped original point cloud data obtained in step 2, which contains the (X, Y, Z, R) of all points in a scene that appear in the camera's field of view, which represent the 3D world coordinates and surface reflection intensity of a single point respectively. The 3D world coordinates are established in the laser radar coordinate system, with the front of the car as the x-axis, the left side of the car as the y-axis, and the top of the car as the z-axis; Step 3-2: Columnarize the point cloud. Send the acquired original point cloud data to the voxelize module for cutting. Cut on the xoy plane and set the size of the cut cylinder to [0, 16, 0, 16, 4]. Calculate the corresponding coordinates of each cylinder in the original three-dimensional world and return them to the Pillars Feature Net module as the return value. Step 3-3: Feature dimension upgrade. After obtaining the segmented cylinders, calculate the cluster center of all points in each cylinder, the center point offset of each cylinder, and the distance from the observer to the point, all of which are considered as the features of the point. Finally, a point with a feature of (4+6)=10 is obtained. Step 3-4: Then send it to the local self-attention transformer for local feature extraction and attention calculation; after the previous step, a tensor with a shape such as [batchsize*6542, 32, 10] is obtained, where the first dimension represents the number of cylinders in a sample in this batch, the second dimension represents the number of points contained in each cylinder, and the third dimension represents the specific feature information of each point; send the tensor to the encoder structure of the transformer, first perform position encoding on it, and the encoding formula is: Among them, PE is the result of position encoding, p is the number of points of the current point, i represents the number of dimensions of the current feature, and d f Represents the overall characteristic dimension; The autocorrelation of each bin is then calculated: In the formula, Q, K, and V represent the query matrix, key matrix, and value matrix respectively. Represents the scaling ratio to prevent the calculation result from exceeding the limit; the final result Atten(Q, K, V) represents the importance of each point relative to other points; Using multi-head attention mechanism: MultiHead(Q,K,V)=Concat(head1,…,head n )W t ; In the formula, head1...represent different sub-vectors, They are the weights of the sub-vectors, and then all the sub-vectors will be concatenated together through Concat, and finally passed through a weight W t The final attention result is obtained; then it goes through a residual structure and forward propagation module and performs multiple iterations; Step 3-5: Finally, the point cloud features passed through the encoder are restored into a 2D pseudo image based on the relative coordinates.

3. The lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention according to claim 1 is characterized in that: It is characterized in that The 2D feature extraction network in step 4 includes: After obtaining the 2D pseudo image in step 3, the lightweight network is redesigned; The size of the input image is [64, 496, 432]. After three downsamplings, feature maps of three different scales will be saved, namely [64, 248, 216], [128, 124, 108], and [256, 62, 54]. Then, different transposed convolutions are performed to restore the image to a uniform size of [384, 124, 108]. Finally, the feature maps containing features of different scales are concatenated. Among them, all convolutions of the network use depthwise separable convolution to reduce the number of model parameters. The Bneck structure uses a channel-based attention mechanism to improve the feature extraction ability of the model. The Bneck structure contains the SEBlock channel attention module, whose main function is to allow the network to pay attention to those more important feature layers. At the same time, the residual structure is used to prevent gradient disappearance and gradient explosion.

4. The lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention according to claim 3 is characterized in that: In step 4, all parameters of the convolutional layer are transformed from h to o ·w o ·a o 2 ·d o ·d o ' drops to h o ·w o ·(a o 2 +d o )·d o ′; The total parameter volume is 24.3% of the original model.

5. The lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention according to claim 2 is characterized in that: The SSD detection head detection in step 5 includes: Step 5-1: The 2D feature map obtained in step 4 is first passed through three different prediction heads, namely, the classification head, the regression head, and the direction prediction head; different prediction results will be obtained for different prediction heads. For the classification prediction head, the classification categories are 3 categories, and the cross entropy loss is used for calculation; for the regression prediction head, the position information of the 3D object is mainly predicted, specifically the coordinate information of 7 objects is predicted; for the direction prediction head, the binary cross entropy loss is used for calculation; Step 5-2: For the ground truth label, each object has 7 parameters, namely [x, y, z, w, l, h, θ], where the first three represent x, y, z, w, l, h respectively represent width, length, height in the lidar coordinate system, and θ represents the heading angle; SmoothL1 regression loss is used between the real ground mark and the predicted box, where the regression loss formula is: Δθ=sin(θ t -θ a ); where x t y t z t represents the ground truth label, x a y a z a represents the prediction box, w t l t h t Represents the width, length, and height of the label, w a l a h a Represents the predicted width, length, and height; θ t θ a Represent the labeled steering angle and the predicted steering angle respectively; The overall regression loss is: Considering that in a three-dimensional environment, the positioning loss cannot identify the angles of two opposite positioning frames, the softmax heading angle classification loss is used to learn the heading angle of the three-dimensional object; The object classification loss used is focal loss: L cls =-a a (1-p a ) γ log p a : In the formula, the classification probability score is expressed as p a Indicates that the hyperparameter α a =0.25,γ=2; The total loss is expressed as: Among them, N pos Represents the number of anchor boxes assigned as positive samples, and again adds weights to different losses, because 3D target detection is more concerned with the coordinate information of the object in the 3D world, so β loc =2,β cls =1,β dir =0.

2.

6. The lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention according to claim 5 is characterized in that: Evaluate the prediction results obtained in step 5-2: Typically, the main performance parameter for 3D object recognition is the Average Precision (AP); it is slightly modified in the KITTI benchmark, using the interpolated AP of the KITTI standard | R40 To evaluate the performance of the model: Among them, R = [0, 1 / 40, 2 / 40, ..., 1] represents different recall rate thresholds, P represents the precision under different recall rates; different IoU thresholds (0.7, 0.5, 0.5) correspond to cars, pedestrians and cyclists respectively, and filter out prediction boxes with low confidence; finally, non-maximum suppression NMS is used to retain the prediction box with the highest confidence.

7. The lightweight PointPillars autonomous driving 3D target detection method based on pillar self-attention according to claim 1, characterized in that: In step six, the detection results obtained in step five are marked into the original point cloud, a new point cloud with detection results is generated, and the prediction results are visualized to realize the 3D target detection task of the autonomous driving car in the 3D world.

Images