Three-dimensional target detection method based on PointPill

Through the column feature network and two-dimensional backbone network combining self-attention and Transformer module, the problem of insufficient detection accuracy of PointPillars in small objects is solved, the detection accuracy and efficiency are improved, and the feature extraction ability is enhanced.

CN119323779APending Publication Date: 2025-01-17SHANGHAI UNIV OF ENG SCI
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411383620.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-30
Publication Date
2025-01-17

AI Technical Summary

Technical Problem

PointPillars has problems with insufficient detection accuracy and lack of information in three-dimensional object detection, and the detection accuracy needs to be improved.

Method used

The column feature network is used to extract global context information and self-attention mechanisms to form 2D pseudo-images, and detect them through the two-dimensional backbone network and SSD detection head. The Transformer module and ECA-PP module are used to enhance feature extraction and downsampling. Capture complex spatial relationships and reduce irrelevant features.

Benefits of technology

It improves the accuracy of small object detection, enhances feature discrimination capabilities, solves the problem of point cloud space information loss, and maintains the fast running speed and high detection efficiency of the algorithm.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119323779A_ABST
    Figure CN119323779A_ABST
Patent Text Reader

Abstract

The invention relates to a three-dimensional target detection method based on Point Pillars, and the method comprises the following steps: obtaining a 3D point cloud around a vehicle, employing a column feature network to sequentially carry out discretization, extracting global context information based on a self-attention mechanism, and carrying out the re-projection, and forming a 2D pseudo image; inputting the 2D pseudo image into a two-dimensional backbone network, performing down-sampling from top to bottom by adopting an attention mechanism, and then obtaining a splicing feature through deconvolution and self-connection; based on the splicing features, an SSD detection head is adopted for detection, and a target detection result is output. Compared with the prior art, the method has the advantages of improving the small target detection effect and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud detection, and in particular to a three-dimensional target detection method based on PointPillars. Background Art

[0002] Three-dimensional object detection is the core system of autonomous driving perception. It detects and analyzes objects around the vehicle to obtain their size, shape, category and motion characteristics, providing reliable operational decisions for autonomous driving.

[0003] Autonomous driving systems usually combine multiple sensors, such as monocular cameras, which can provide high-resolution RGB images and rich texture information by mapping 3D objects onto a 2D plane, and can achieve more than 80% detection confidence in front-view images even if the object is 50 meters away. However, although cameras provide rich color and texture information, they usually lack depth information and their performance degrades in low lighting conditions.

[0004] To overcome these limitations, LiDAR is widely used. It measures distance by sending and receiving reflected signals, generating sparse 3D point clouds. With the rapid development of deep learning technology, 3D object detection methods based on deep learning have begun to emerge. These methods can learn features directly from raw point clouds, overcoming the cumbersome parameter adjustment and preprocessing steps in traditional methods, thereby improving the accuracy and efficiency of detection.

[0005] PointPillars proposed a columnar feature extraction method that converts 3D point clouds into 2D pseudo images and then uses a 2D detection network on them, eliminating the need for 3D CNN operations and significantly increasing the algorithm's running speed. Although PointPillars remains ahead in detection speed, its detection accuracy still needs to be improved and there is a problem of missing information. Summary of the invention

[0006] The purpose of the present invention is to provide a three-dimensional target detection method based on PointPillars that improves the detection accuracy of small targets.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A three-dimensional target detection method based on PointPillars includes the following steps:

[0009] Obtain the 3D point cloud around the vehicle, discretize it using a column feature network, extract global context information based on a self-attention mechanism, and reproject it to form a 2D pseudo image;

[0010] The 2D pseudo image is input into a two-dimensional backbone network, down-sampled from top to bottom using an attention mechanism, and then a splicing feature is obtained through deconvolution and self-connection;

[0011] Based on the splicing features, an SSD detection head is used to perform detection and output a target detection result.

[0012] Furthermore, the column feature network includes a columnarization module, an enhanced feature extraction module, a customized position encoding module, a Transformer module and a feature reprojection module, and the formation process of the 2D pseudo image includes:

[0013] Inputting the 3D point cloud into the columnarization module, dividing it into a series of regular columnar regions, wherein each point is divided into the nearest columnar region according to its 3D coordinates;

[0014] Based on the columnar region, an enhanced feature extraction module is used to extract key features from the columnar region, and a maximum pooling process is performed to reduce the data dimension to obtain the most representative features of each columnar region;

[0015] Using the customized position coding module to assign position coding to the most representative feature, thereby obtaining a feature containing the position coding, wherein the position coding includes the absolute spatial position information of the feature and the relative position relationship between the features;

[0016] Inputting the feature containing the position code into the Transformer module, and performing feature aggregation using a multi-head attention mechanism to obtain aggregated features;

[0017] The aggregated features are input into a feature reprojection module for reprojection to obtain a 2D pseudo image.

[0018] Furthermore, the execution steps of the Transformer module include:

[0019] Based on the feature containing the position code, the feature is divided into multiple heads, each head performs independent self-attention calculation, and each head outputs weighted features, wherein in the self-attention calculation process, an attention mask is generated to limit the calculation range of the attention weight and different attention weights are assigned through interaction;

[0020] The weighted features of multiple heads are aggregated to obtain aggregated features.

[0021] Furthermore, the column feature network also includes a multi-layer perceptron, which is connected to the Transformer module and the feature reprojection module respectively, and the multi-layer perceptron is used to further process the aggregated features to increase the feature dimension and capture complex spatial relationships in a high-dimensional space.

[0022] Furthermore, the two-dimensional backbone network includes a downsampling subnetwork and a splicing subnetwork.

[0023] The downsampling subnetwork includes m+1 ECA-PP modules and m convolution blocks, the ECA-PP modules and the convolution blocks are alternately connected in a top-down manner, and the downsampling subnetwork is used to perform multiple downsampling on the 2D pseudo image from top to bottom to generate features of multiple different resolutions;

[0024] The splicing subnetwork includes a feature splicing module and m deconvolution blocks connected thereto, the first m-1 deconvolution blocks are connected to the first m-1 deconvolution blocks, and the last deconvolution block is connected to the last ECA-PP module. The splicing subnetwork is used to uniformly amplify the features of different resolutions using the deconvolution blocks, and use the feature splicing module for self-connection to obtain splicing features.

[0025] Furthermore, the ECA-PP module includes a depth-separable convolutional layer, a global average pooling layer, an ECA submodule and an output layer connected in sequence, and the execution steps of the ECA-PP module include:

[0026] Inputting the 2D pseudo image into the depthwise separable convolution layer for depthwise convolution and pointwise convolution, and then inputting the 2D pseudo image into the global average pooling layer for average pooling to extract efficient features;

[0027] The efficient features are input into the ECA submodule, and the attention mechanism is used to adaptively enhance important features and suppress irrelevant features to obtain attention features;

[0028] The attention features are input into the output layer and output.

[0029] Furthermore, the convolution block is used for downsampling, and the convolution block includes a 2D convolution layer and a batch normalization layer and a ReLU activation function layer connected thereto in sequence, wherein the 2D convolution layer is provided with a plurality of output channels.

[0030] Furthermore, the downsampling subnetwork includes 4 ECA-PP modules and 3 convolution blocks, and the splicing subnetwork includes 3 deconvolution blocks and 1 feature splicing module.

[0031] Furthermore, the target detection results include cars, pedestrians and cyclists.

[0032] Furthermore, an SSD detection head is used to perform detection to obtain a target detection result.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The column feature network of the present invention uses the self-attention mechanism to capture the global context information of the point cloud data, and the two-dimensional backbone network uses the attention mechanism to perform downsampling from top to bottom, thereby enhancing the feature discrimination ability and improving the small target detection accuracy.

[0035] (2) The downsampling subnetwork of the present invention improves the design of the two-dimensional backbone network by alternately connecting the ECA-PP module and the convolution block. The ECA-PP module uses the attention mechanism to adaptively enhance important features and suppress irrelevant features in the column channel to compensate for the loss of spatial information, solve the practical challenge of point cloud space loss caused by the column division of PointPillars, and improve the accuracy of target detection. In addition, the ECA-PP module does not impose too much additional burden on the training, operation, calculation and storage characteristics of the algorithm, and the algorithm runs fast and has high detection efficiency.

[0036] (3) The Transformer module of the present invention adopts a multi-head attention mechanism for calculation to capture the global context information of the point cloud data, so that the network can capture the global dependencies in the point cloud, dynamically determine which features are more important through the self-attention mechanism, and use the generated attention mask to perform self-attention calculations, which can effectively reduce the amount of calculation and improve the efficiency of algorithm execution.

[0037] (4) This invention not only provides a new technical solution for environmental perception of autonomous vehicles, but also lays the foundation for lightweight deployment on resource-constrained platforms in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0039] Figure 2 It is a schematic diagram of the overall structure of the present invention;

[0040] Figure 3 It is a schematic diagram of the column characteristic network structure of the present invention;

[0041] Figure 4 A flowchart for executing the column feature network of the present invention;

[0042] Figure 5 Schematic diagram of the comparison between the depthwise separable convolution and the standard convolution of the present invention, wherein (a) is a standard convolution diagram and (b) is a depthwise separable convolution diagram;

[0043] Figure 6 This is a schematic diagram of the ECA-PP module architecture of the present invention;

[0044] Figure 7 It is a schematic diagram of the two-dimensional backbone network structure of the present invention;

[0045] Figure 8 It is a schematic diagram of the results of the ablation experiment of the present invention;

[0046] Fig. 9 This is an example diagram of the visualization result of the present invention;

[0047] Fig.10 This is an example diagram of the comparison effect between the real-time common obstacle detection algorithm of the present invention and this algorithm. DETAILED DESCRIPTION

[0048] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0049] This embodiment provides a three-dimensional target detection method based on PointPillars. Figure 2 The network structure shown in the figure is implemented. The network structure consists of three parts, namely column feature network, two-dimensional backbone network and SSD detection head. Figure 2 and Figure 1 As shown, the method comprises the following steps:

[0050] Step 1: Select the KITTI image dataset as the fusion and training dataset, and use the same dataset segmentation ratio as PointPillars.

[0051] The KITTI dataset is a public dataset widely used in autonomous driving research. It contains a wealth of lidar point clouds and image samples. In the experiment, we set the following parameters: the xy resolution is 0.16m, the maximum number of pillars (P) is 1200, and the maximum number of points per pillar (N) is 100. We define the corresponding 3D object detection frames for car, pedestrian, and cyclist categories, and set the matching positive and negative thresholds. The 3D object detection frame of each object category consists of 0-degree and 90-degree directions. The width, length, and height of the car positioning point are 1.60m, 3.90m, and 1.56m, respectively, the z center is -1.00m, and the matching positive and negative thresholds are 0.60 and 0.45, respectively. The width, length and height of the pedestrian anchor are 0.60m, 0.80m and 1.73m respectively, and the width, length and height of the cyclist anchor are 0.60m, 1.76m, 1.73m respectively. The z center is -0.60m, and the matching positive and negative thresholds are 0.5 and 0.35.

[0052] Step 2: Discretize the original 3D point cloud data and convert it into structured pillars to form an efficient 2D pseudo image.

[0053] This step aims to retain the sparse nature of the point cloud while capturing the essence of its spatial information, providing an optimized data foundation for subsequent processing.

[0054] like Figure 3 As shown, the column feature network (PFN) introduced in this embodiment is responsible for extracting key features from the original three-dimensional point cloud data to facilitate subsequent deep learning tasks. Figure 4 As shown, the specific implementation steps of the column feature network are as follows:

[0055] Step 2.1: The execution process of the pillar feature network (PFN) begins with receiving the original 3D point cloud data. After the point cloud data is input into the PFN, it is then divided into a series of regular columnar regions through the columnarization module. Each point is assigned to the nearest columnar region according to its 3D coordinates. The columnarized point cloud data then enters the enhanced feature extraction module, which is responsible for extracting key features from each column and outputting the feature vector of each pillar. These features are then processed by maximum pooling to reduce the data dimension and retain the most representative feature points. Then, the customized position encoding module assigns unique position codes to these reduced feature points. At this stage, each point is assigned a unique position code, which contains the absolute spatial position information of the point and the relative position relationship between the points.

[0056] Step 2.2: Next, in the attention weighted feature aggregation stage, the feature points with position encoding are fed into the Transformer module and input into the multi-head attention mechanism, which is divided into multiple heads. Each head performs independent self-attention calculations, and each head outputs weighted features. During the calculation process, each element in the input sequence is compared with other elements, interacts and is given different weights according to the correlation between them. This interaction is achieved by calculating the attention weight matrix, and the attention mask is generated to limit the calculation range of the attention weight. The attention mask allows each point to interact only with a few points in its spatial neighborhood during self-attention calculations, while ignoring more distant points, thereby reducing the computational burden while focusing on more relevant local features. This is more in line with the distribution and interaction of objects in the natural environment in many cases, reducing the computational burden and improving efficiency. This module uses the self-attention mechanism to capture the global context information of the point cloud data, enabling the network to capture the global dependencies in the point cloud, dynamically determine which features are more important through the self-attention mechanism, and weight the aggregated features accordingly.

[0057] Step 2.3: The features processed by the Transformer module are reprojected through the feature reprojection module to remap the reduced feature points back to the original 3D space. In addition, further feature processing is performed through MLP (Multi-layer Perceptron), which increases the dimension of the features and provides the model with more information to capture complex spatial relationships.

[0058] 2D pseudo images processed by PFN, which are obtained by converting 3D point cloud data into 2D format, retaining the spatial structure and feature information.

[0059] Step 3: Input the 2D pseudo image into the two-dimensional backbone network, use the attention mechanism to perform downsampling from top to bottom, and then obtain the splicing features through deconvolution and self-connection.

[0060] like Figure 7 As shown, the two-dimensional backbone network (Backbone) includes two subnetworks, namely, a top-down downsampling subnetwork and a splicing subnetwork. The downsampling subnetwork includes m+1 ECA-PP (Efficient Channel Attention-Point Pillars) modules and m convolution blocks. The ECA-PP modules and convolution blocks are alternately connected in a top-down manner. The subnetwork is used to perform downsampling based on the attention mechanism and convolution to generate features of multiple different resolutions. The splicing subnetwork includes a feature splicing module and m deconvolution blocks connected thereto. The first m-1 deconvolution blocks are connected to the first m-1 deconvolution blocks, and the last deconvolution block is connected to the last ECA-PP module. The subnetwork is used to deconvolve features of multiple different resolutions to uniformly amplify the resolution, and then self-connect to form splicing features.

[0061] The processed 2D pseudo image is fed into a deeply customized 2D convolutional neural network. In this stage, not only multiple downsampling is applied to extract feature maps of different resolutions, but also the ECA (Efficient Channel Attention) submodule is introduced to break through the limitations of traditional CNN and significantly enhance the network's ability to learn dynamic dependencies between channels. This design enables the algorithm to process feature maps more carefully and effectively fuse multi-scale information to form a feature set with rich representation. The specific execution process of this step is as follows:

[0062] Step 3.1: Downsampling is performed through the top-down downsampling subnetwork to generate features of various resolutions. Downsampling reduces the spatial dimension of the feature map by convolution or pooling with a stride greater than 1, and generates features in a way of gradually reducing the spatial resolution. The first ECA-PP module (ECA-PP-1 module) is connected to the first convolution block (Block1), and the remaining ECA-PP modules (including ECA-PP-2 module (C, K), ECA-PP-3 module (2C, K) and ECA-PP-4 module (4C, K)) are sequentially connected to the first convolution block Block1 (S, 4, F), the second convolution block Block2 (2S, 6, 2F) and the third convolution block Block3 (4S, 6, 4F) to reconstruct the backbone network. Where S represents the stride, Block1 has 4 3×3 2D convolution layers, and F represents the output channel. Each block is followed by a BatchNorm and ReLU. To ensure that the network layer can retain as much information as possible when processing the input, the first convolution step of this layer is S / Sin, and the subsequent convolution step is 1 to ensure that the network layer still maintains the step size of S after receiving an input with a step size of Sin. This avoids excessive compression of the input signal due to the first convolution step in the network layer being too large, and avoids information loss as much as possible. The upsampling network is followed by BatchNorm and ReLU activation functions, and then all ECA-PP modules with different step sizes are connected to generate the final output features. An improved algorithm is implemented to solve the practical challenge of point cloud space loss caused by column partitioning of PointPillars, which affects the accuracy of target detection. Figure 6 As shown, the ECA-PP module includes a depth-separable convolution layer, a global average pooling layer, an ECA submodule and an output layer connected in sequence. The execution steps of the ECA-PP module include: inputting the 2D pseudo image into the depth-separable convolution layer for depth convolution and point-by-point convolution, and then inputting it into the global average pooling layer for average pooling to extract efficient features; inputting the efficient features into the ECA submodule, using the attention mechanism to adaptively enhance important features and suppress irrelevant features to obtain attention features; inputting the attention features into the output layer and outputting them. Figure 5 is a depth-wise separable convolution ( Figure 5 (b)) and ordinary convolution ( Figure 5In the comparison diagram of (a), this embodiment uses a deep separable convolution to use a convolution kernel for each channel of the input feature map, and performs convolution operations on each channel independently. Subsequently, a 1x1 convolution kernel is used to merge the output of the deep separable convolution, which helps to integrate the information of different channels while keeping the number of channels of the output feature map unchanged. Then, by using a convolution or pooling layer with a step size greater than 1, the spatial dimension of the feature map is reduced, while the semantic depth of the feature is increased. While downsampling, the network continuously extracts and abstracts features through deep convolution layers, and each layer learns a more advanced representation. The ECA-PP module adaptively enhances important features and suppresses irrelevant features within the column channel to compensate for the loss of spatial information. The use of the ECA-PP module can improve the expressiveness of the model by allowing the network to focus on important information in the pseudo-image features and ignore irrelevant information, without imposing too much additional burden on the training, operation, calculation and storage characteristics of the algorithm. The ECA submodule captures global information through a global average pooling layer, and adaptively learns the dependencies between channels through two consecutive fully connected layers. Ultimately, the network outputs a set of feature maps that contain rich spatial and semantic information, providing a basis for subsequent object detection tasks such as bounding box regression and category classification.

[0063] Step 3.2: Use the splicing sub-network to uniformly amplify and self-connect features of different resolutions to obtain splicing features.

[0064] The splicing subnetwork is responsible for upsampling and connecting top-down features. The deconvolution block is used to enlarge the resolution, and then the feature splicing function of the feature splicing module is used to splice and output the spliced ​​features.

[0065] S4. Input the splicing features into the SSD detection head for detection to obtain the target detection result.

[0066] According to the feature map output by the network, operations such as bounding box prediction and non-maximum suppression (NMS) are performed to determine the final detection result.

[0067] In order to verify the effectiveness of the above different modules and the effectiveness of the improved Transformer-based feature encoding network, we conducted a series of ablation experiments on the KITTI validation set.

[0068] The ablation experiment was conducted by observing the impact of changes in Transformer hyperparameters on detection performance. mAP was selected as the evaluation indicator, and each set of parameters was trained five times to reduce the impact of random number seeds on the results. At the same time, for the efficiency of the experiment, each set of parameters was trained 120 times. The results are shown in Figure 8As shown in the figure, we can see that when the number of layers and heads of Transformer increases to a certain extent, the detection performance is better than the original PointPillars. However, if the number of layers and heads continues to increase, the detection performance will decrease. Taking all factors into consideration, the detection performance is optimal when the number of layers is 4 and the number of heads is 2;

[0069] The introduction of the Transformer module and the ECA-PP module has little effect on the real-time performance of the algorithm. The improved algorithm can maintain high computational efficiency while improving detection accuracy. This preliminary result verifies the effectiveness and scalability of the algorithm.

[0070] In order to adapt this method to the application of autonomous driving scenarios, such as Fig. 9 As shown in Figure 1, this method uses 3D and image methods to display the detection effect based on the final output features to complete the final result detection. The final detection result deployed on the actual vehicle is shown in Figure 1. Fig.10 shown.

[0071] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0072] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0073] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0074] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0076] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0077] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A three-dimensional target detection method based on PointPillars, characterized in that: The following steps are involved: Obtain the 3D point cloud around the vehicle, discretize it using a column feature network, extract global context information based on a self-attention mechanism, and reproject it to form a 2D pseudo image; The 2D pseudo image is input into a two-dimensional backbone network, down-sampled from top to bottom using an attention mechanism, and then a splicing feature is obtained through deconvolution and self-connection; Based on the splicing features, detection is performed and the target detection result is output.

2. A three-dimensional target detection method based on PointPillars according to claim 1, characterized in that: The column feature network includes a columnarization module, an enhanced feature extraction module, a customized position encoding module, a Transformer module and a feature reprojection module. The formation process of the 2D pseudo image includes: Inputting the 3D point cloud into the columnarization module, dividing it into a series of regular columnar regions, wherein each point is divided into the nearest columnar region according to its 3D coordinates; Based on the columnar region, an enhanced feature extraction module is used to extract key features from the columnar region, and a maximum pooling process is performed to reduce the data dimension to obtain the most representative features of each columnar region; Using the customized position coding module to assign position coding to the most representative feature, thereby obtaining a feature containing the position coding, wherein the position coding includes the absolute spatial position information of the feature and the relative position relationship between the features; Inputting the feature containing the position code into the Transformer module, and performing feature aggregation using a multi-head attention mechanism to obtain aggregated features; The aggregated features are input into a feature reprojection module for reprojection to obtain a 2D pseudo image.

3. A three-dimensional target detection method based on PointPillars according to claim 2, characterized in that: The execution steps of the Transformer module include: Based on the feature containing the position code, the feature is divided into multiple heads, each head performs independent self-attention calculation, and each head outputs weighted features, wherein in the process of self-attention calculation, an attention mask is generated to limit the calculation range of the attention weight, and different attention weights are assigned through interaction; The weighted features of multiple heads are aggregated to obtain aggregated features.

4. The three-dimensional target detection method based on PointPillars according to claim 2, characterized in that: The column feature network also includes a multi-layer perceptron, which is connected to the Transformer module and the feature reprojection module respectively. The multi-layer perceptron is used to further process the aggregated features to increase the feature dimension and capture complex spatial relationships in a high-dimensional space.

5. The three-dimensional target detection method based on PointPillars according to claim 1, characterized in that: The two-dimensional backbone network includes a downsampling subnetwork and a splicing subnetwork. The downsampling subnetwork includes m+1 ECA-PP modules and m convolution blocks, the ECA-PP modules and the convolution blocks are alternately connected in a top-down manner, and the downsampling subnetwork is used to perform multiple downsampling on the 2D pseudo image from top to bottom to generate features of multiple different resolutions; The splicing subnetwork includes a feature splicing module and m deconvolution blocks connected thereto, the first m-1 deconvolution blocks are connected to the first m-1 deconvolution blocks, and the last deconvolution block is connected to the last ECA-PP module. The splicing subnetwork is used to uniformly amplify the features of different resolutions using the deconvolution blocks, and use the feature splicing module for self-connection to obtain splicing features.

6. A three-dimensional target detection method based on PointPillars according to claim 5, characterized in that: The ECA-PP module includes a depth-separable convolutional layer, a global average pooling layer, an ECA submodule and an output layer connected in sequence, and the execution steps of the ECA-PP module include: Inputting the 2D pseudo image into the depthwise separable convolution layer for depthwise convolution and pointwise convolution, and then inputting the 2D pseudo image into the global average pooling layer for average pooling to extract efficient features; The efficient features are input into the ECA submodule, and the attention mechanism is used to adaptively enhance important features and suppress irrelevant features to obtain attention features; The attention features are input into the output layer and output.

7. The three-dimensional target detection method based on PointPillars according to claim 5, characterized in that: The convolution block is used for downsampling, and the convolution block includes a 2D convolution layer and a batch normalization layer and a ReLU activation function layer connected thereto in sequence, wherein the 2D convolution layer is provided with a plurality of output channels.

8. The method for three-dimensional target detection based on PointPillars according to claim 5, characterized in that: The downsampling subnetwork includes 4 ECA-PP modules and 3 convolution blocks, and the splicing subnetwork includes 3 deconvolution blocks and 1 feature splicing module.

9. The three-dimensional target detection method based on PointPillars according to claim 1, characterized in that: The target detection results include cars, pedestrians and cyclists.

10. The three-dimensional target detection method based on PointPillars according to claim 1, characterized in that: The SSD detection head is used for detection to obtain the target detection result.

Citation Information

Cited By

  • Lightweight PointPill automatic driving 3D target detection method based on cylinder self-attention

    CN119785338A

  • Lightweight pointpillars autonomous driving 3D object detection method based on column self-attention

    CN119785338B