A multi-channel perception 3D point cloud object detection method

Abstract

The present invention belongs to the field of 3D point cloud target detection, and specifically relates to a multi-channel perception 3D point cloud target detection method, including: obtaining original 3D point cloud data, processing the 3D point cloud data using a column method to obtain columns; using an attention perception module to extract the point dimension attention, channel dimension attention and voxel dimension attention of the columns, and optimizing the three attention features through a multi-layer perceptron to obtain three attention scores; constructing a complete attention matrix based on the three attention scores and the columns; inputting the complete attention matrix into a detection head to obtain a target detection result; the present invention uses a multi-dimensional comprehensive attention perception module, which can better retain important information and be extended to various pedestrian representations to overcome the non-rigidity and sparsity of pedestrian data in point clouds. Due to the multi-dimensional attention, the columnar representation can be enhanced, and the context information of all points in the columns can be better obtained through a composite pooling strategy.

Description

Technical Field

[0001] The present invention belongs to the field of 3D point cloud target detection, and specifically relates to a multi-channel perception 3D point cloud target detection method. Background Art

[0002] The automotive industry is currently undergoing a major transformation. Vehicles are gradually transitioning from traditional manual driving to AI-powered autonomous driving. Research on autonomous driving technologies is also growing. However, ensuring reliable, stable, and safe autonomous driving requires sufficient technology and practice. Autonomous driving refers to the ability of vehicles to control their own driving without human intervention. This can be achieved through a variety of approaches, including single-vehicle intelligence, multi-vehicle collaboration, and networked cloud control. Single-vehicle intelligence primarily encompasses three key components: perception, decision-making, and control. Perception involves the collection and processing of vehicle and environmental information, including video, GPS, vehicle posture, acceleration, and point cloud data. In current applications of assisted and autonomous driving systems, the detection of three-dimensional objects such as pedestrians and vehicles is a key technology for establishing vehicle-environment interaction mechanisms. Complementing two-dimensional images, three-dimensional data provides an opportunity to better understand the surrounding environment. However, two-dimensional visual images often suffer from large spatial positioning errors, while object detection based on 3D point clouds provides more accurate depth information and higher detection accuracy. Therefore, autonomous driving technology, driven by 3D point cloud data, has become an indispensable key technology in both autonomous and assisted driving.

[0003] In recent years, object detection algorithms based on 3D point clouds have achieved remarkable results in road scenes. Unlike 2D images, 3D point cloud data processing is characterized by disorder, sparsity, and local sensitivity. Therefore, the models used to extract 3D point cloud data must be permutation- and translation-invariant. Commonly used methods include: those based on raw 3D point cloud data, which use neural networks to directly extract features from unordered points. These methods take the raw point cloud as input and use a multi-layer perceptron (MLP) to map low-dimensional features to a high-dimensional feature space to ensure translation invariance. The high-dimensional features are then fed into symmetric functions such as the average function or the maximum function to ensure permutation invariance to the order of the input point cloud. Other methods also involve hierarchically aggregating point cloud features from different levels to obtain local information. The method then iteratively uses the farthest point sampling (FPS) method to select subsets of the point cloud, attempting to maximize coverage of the entire scene with the fewest sample points before performing feature extraction. It also includes a special voxel with unlimited height to encode 3D point cloud information into a data form that can be processed by 2D convolutional networks and detection heads.

[0004] However, since the posture of the human body is a flexible object, compared to rigid objects such as vehicles, the posture of the human body is not the same shape due to factors such as time, scene, and individual. In addition, the human body is small in size, which is reflected in the obvious technicality and scarcity of the human body point cloud in the point cloud data, resulting in low target detection accuracy. Summary of the Invention

[0005] To solve the above problems in the prior art, the present invention proposes a multi-channel perception 3D point cloud target detection method, which includes:

[0006] S1: Obtain original 3D point cloud data, process the 3D point cloud data using the column method, and obtain columns;

[0007] S2: A composite pooling strategy is used to capture the contextual information of column point attention, point channel attention, and column channel attention respectively;

[0008] S3: Fusion of point attention, point channel attention, and column channel attention based on the captured contextual information to obtain the fused channel attention score;

[0009] S4: Construct an attention matrix based on the fused channel attention scores;

[0010] S5: Generate pseudo 2D image based on attention matrix;

[0011] S6: Input the pseudo 2D image into the Tiny-BiFPN neck module to fuse feature maps of different resolutions;

[0012] S7: Input the fused feature map into the SSD detection head to obtain the detection result.

[0013] Preferably, processing 3D point cloud data using the column method includes: dividing the points along the x-axis and y-axis into individual columns, where the maximum number of points in a column is N, the number of channels for each point is C, and there are a total of P columns in the entire scene.

[0014] Preferably, the composite pooling strategy is used to capture context information, including: performing a maximum pooling operation on all points in each column that generates column information, and using the maximum pooling operation result as the input of the column channel attention module to obtain For F′ v Using maximum pooling and average pooling respectively, we get and The generated column information is input into the point channel attention module and the point attention module respectively, and then the maximum pooling and average pooling are applied to them respectively to obtain and and Among them, F′ vrepresents the result of a maximum pooling operation on all points in each column in the column channel attention, and Respectively represent the attention to F′ in the column channel v The results of the average pooling operation and the maximum pooling operation are: and Represent the results of average pooling and maximum pooling in point attention respectively, and Respectively represent the results of average pooling and maximum pooling operations in point channel attention.

[0015] Preferably, the process of calculating the fusion channel attention score includes: passing the results obtained by the point attention module through a shared multi-layer perceptron structure composed of two fully connected neural networks, a reduction ratio r and an activation function δ, and repeatedly stacking the results P times in the dimension of the number of columns to obtain the attention score A v The results obtained by the point channel attention module and the column channel attention module are passed through a shared multi-layer perceptron structure composed of two fully connected neural networks, a reduction ratio r and an activation function δ to obtain the attention scores A and A respectively. c and A p .

[0016] Preferably, the process of processing the complete attention matrix using Tiny-BiFPN includes: inputting the generated pseudo image into the convolution blocks B1, B2, B3, and B4 to obtain Will The resolution of halved with the superscript, that is, the resolution of the i-th layer image is the resolution of the input pseudo image. Where i∈(1,2,3,4), top-down bidirectional feature fusion is repeatedly performed in the network to aggregate features at different resolutions, and self-trainable weights are added to adaptively adjust the fusion weights to obtain feature maps of different resolutions; feature maps of different resolutions are spliced and fused to obtain the fused feature map.

[0017] Beneficial effects of the present invention:

[0018] The present invention proposes and uses a multi-dimensional integrated attention perception module, which can better retain important information and generalize to various pedestrian representations to overcome the non-rigidity and sparsity of pedestrian data in point clouds. Due to the multi-dimensional attention, the columnar representation can be enhanced, and the contextual information of all points in the column can be better obtained through a composite pooling strategy. The present invention uses the Tiny-BiFPN network, which is a lightweight feature network that can bidirectionally connect and fuse multi-resolution features. It aggregates multi-scale features by repeatedly applying top-down and bottom-up bidirectional feature fusion, and has trainable weights. The fusion weights can be adjusted accordingly, making the model more efficient in performing multi-scale feature fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the overall flow chart of the present invention;

[0020] Figure 2 It is the overall model framework diagram of the present invention. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0022] A multi-channel perception 3D point cloud target detection method, such as Figure 1 As shown, the method includes: obtaining original 3D point cloud data, processing the 3D point cloud data using the pillar method to obtain pillars; using the attention perception module to extract the point dimension attention, channel dimension attention and voxel dimension attention of the pillars, and optimizing the three attention features through a multi-layer perceptron to obtain three attention scores; constructing a complete attention matrix based on the three attention scores and the pillars; inputting the complete attention matrix into the Tiny-BiFPN detection head to obtain the target detection result.

[0023] In this embodiment, a multi-dimensional feature extraction module is designed to fully extract the features of 3D point cloud data. After processing the 3D point cloud data using the pillar method, the point-dimensional attention, channel-dimensional attention, and voxel-dimensional attention are fused to generate a 2D attention map, which is then predicted using a 2D attention model. Due to this multi-dimensional fusion feature, noise points in the original 3D point cloud are effectively suppressed, and due to the fusion of the voxel dimension, more global features are taken into account.

[0024] In this embodiment, a multi-scale, multi-layer, weighted connected detector head, Tiny-BiPFN, was designed to fully exploit the dependencies between points in the feature map. This is because the number of individual object points, as well as the total number of scene points, for small objects such as pedestrians in point cloud data is relatively small. Therefore, a more comprehensive multi-layer connection with multiple resolution scales and adjustable fusion weights was used. Through multiple top-down and bottom-up bidirectional feature aggregations, more closely connected and comprehensive feature information was mined.

[0025] In the present invention, the final result is the position frame of each object, and each frame needs to have c x ,c y ,c z ,w,l,h,θ, where c represents the coordinates of the center point in the three dimensions of space, w,l,h represent the size and length of each frame, and θ is the deflection angle of each frame. In the original point cloud data, each point cloud is defined as T, then:

[0026]

[0027] where x i ,y i ,z i ,r i are the coordinates of the i-th point in three-dimensional space and the reflectivity of each point, and M is the number of points.

[0028] A specific implementation method of a multi-channel perception 3D point cloud target detection method includes the following steps:

[0029] Step 1: Get the original 3D point cloud data.

[0030] Step 2: Use the pillar method to process the original point cloud data into pillars, that is, divide the points into individual pillars along the x-axis and y-axis. Since the points in the point cloud data are not guaranteed to be evenly distributed in the stereo but sparsely distributed, we assume that the maximum number of points in a pillar is N, the number of channels for each point is C, and there are a total of P pillars in the entire scene.

[0031] Step 3: The column data obtained in step 2 is processed in the attention perception module, which performs point, point channel, and column channel attention perception. Both maximum pooling and average pooling operations are applied simultaneously.

[0032] Assume a column network is Performing composite pooling operations at the point level, we can obtain and in Similarly, performing composite pooling operations at the point channel level yields: and in For the column channel level, we first perform a maximum pooling operation on all points in each column to obtain Then perform composite pooling operation on each column to obtain and in

[0033] Step 4: Take the result from step 3 Pass in a shared multi-layer perceptron structure consisting of two fully connected neural networks, a reduction ratio r and an activation function δ, and add the two results to get the attention component Likewise, it will Pass it into the same multi-layer perceptron structure and add the two results to get the attention component And for First, pass them into the same multi-layer perceptron and stack the results P times in the dimension of the number of columns to get the attention score The calculation formula is:

[0034]

[0035] Among them, A p Indicates the point attention score, A c represents the point channel attention score, A v represents the column channel attention score, repeat(,t) means repeating a certain dimension t times, σ is the sigmoid activation function, w i represents the weight of each connection layer, where

[0036] Step 5: Directly multiply the results obtained in step 4 to get the complete attention matrix.

[0037] M j =A p ×A c ×A v

[0038] in, A p Indicates the point attention score, A c represents the point channel attention score, A v Represents the column channel attention score.

[0039] M j With the original column P jBy directly multiplying them, we can obtain the comprehensive attention-weighted features of the three dimensions. The channel attention of the two branches guides the attention mechanism to pay more attention to meaningful local features.

[0040] Step 6: Input the result of step 5 into Tiny-BiFPN, which is a lightweight and improved variant of BiFPN. Its framework diagram is shown in Figure 2 First, input the generated pseudo image into the convolution blocks B1, B2, B3, and B4, and get

[0041]

[0042] And its resolution is halved with the superscript, that is, the resolution of the i-th layer image is the resolution of the input pseudo image Where i∈(1,2,3,4), the top-down bidirectional feature fusion is repeatedly performed in the network, features at different resolutions are aggregated, and self-trainable weights are added to adaptively adjust the fusion weights. The key process is expressed in the formula:

[0043]

[0044] Among them, ε′ 1,…,i ,ε″ 1,…,i ,ε″′ 1,…,i and ε″″ 1,…,i is a trainable parameter, i is the number of features fused in a block, swish() is the swish activation function, and the calculation method of the remaining blocks is similar to the above method. Finally, the output results are concatenated to obtain

[0045]

[0046] Among them, concatenation() is a concatenation operation.

[0047] Eventually Perform concatenation, pass it into SSD and perform classification and regression.

[0048] In this example, the Python programming language is used, which can run on mainstream computer platforms. The operating system used in this example is Ubuntu 18.04.5LTS, the CPU is an Intel Xeon Silver 4214R, the memory is 32GB, the hard disk space requires 20GB or more, and the GPU is an NVIDIA Tesla V100 with 32GB of video memory.

[0049] The dataset used is the KITTI dataset, jointly developed by the Karlsruhe Institute of Technology and Toyota Research Institute America. It is currently the world's largest evaluation dataset for computer vision algorithms in autonomous driving scenarios. This dataset is used to evaluate the performance of computer vision technologies such as stereo, optical flow, visual odometry, 3D object detection, and 3D tracking in vehicle environments. KITTI contains real-world image data collected in urban, rural, and highway scenarios, with up to 15 vehicles and 30 pedestrians per image, and varying degrees of occlusion and truncation. The dataset consists of 389 pairs of stereo and optical flow images, 39.2 km of visual odometry sequences, and over 200,000 images with 3D annotated objects, sampled and synchronized at 10 Hz.

[0050] Evaluation metrics: Use mAP (mean average precision) to evaluate the model, and use IoU (intersection over union) to check whether the detection is correct. The most commonly used threshold is 0.5, that is, if IoU>0.5, it is considered a true detection.

[0051]

[0052] When performing 3D object detection on the public dataset KITTI, the mean average precision can reach 56.82%, 46.79%, and 42.60% in simple, medium, and difficult samples, respectively, and can reach 48.59% overall, which has a good predictive effect on autonomous driving systems.

[0053] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-channel perception 3D point cloud target detection method, characterized in that: include: S1: Obtain original 3D point cloud data, process the 3D point cloud data using the column method, and obtain columns; S2: A composite pooling strategy is used to capture the point attention of the column, the point channel attention, and the contextual information of the column channel attention respectively; specifically, it includes: performing a maximum pooling operation on all points in each column that generates column information, and using the maximum pooling operation result as the input of the column channel attention module to obtain For F′ v Using maximum pooling and average pooling respectively, we get and The generated column information is input into the point channel attention module and the point attention module respectively, and then the maximum pooling and average pooling are applied to them respectively to obtain and and Among them, F′ v represents the result of a maximum pooling operation on all points in each column in the column channel attention, and Respectively represent the attention to F′ in the column channel v The results of average pooling and maximum pooling operations are: and Represent the results of average pooling and maximum pooling in point attention respectively, and Respectively represent the results of average pooling and maximum pooling operations in point channel attention; S3: Fusion of point attention, point channel attention, and column channel attention based on the captured contextual information to obtain the fused channel attention score; S4: Construct an attention matrix based on the fused channel attention scores; S5: Generate pseudo 2D image based on attention matrix; S6: Input the pseudo 2D image into the Tiny-BiFPN neck module to fuse feature maps of different resolutions; specifically, input the generated pseudo image into convolution blocks B1, B2, B3, and B4 to obtain Will The resolution of halved with the superscript, that is, the resolution of the i-th layer image is the resolution of the input pseudo image. Where i∈(1,2,3,4), the top-down bidirectional feature fusion is repeatedly performed in the network to aggregate features at different resolutions, and self-trainable weights are added to adaptively adjust the fusion weights to obtain feature maps of different resolutions; the feature maps of different resolutions are spliced and fused to obtain the fused feature map; S7: Input the fused feature map into the SSD detection head to obtain the detection result.

2. The multi-channel perception 3D point cloud target detection method according to claim 1, characterized in that: The column method is used to process 3D point cloud data, including dividing the points into individual columns along the x-axis and y-axis, where the maximum number of points in a column is N, the number of channels for each point is C, and there are a total of P columns in the entire scene.

3. The multi-channel perception 3D point cloud target detection method according to claim 1, characterized in that: The process of calculating the fusion channel attention score includes: passing the results obtained by the point attention module through a shared multi-layer perceptron structure composed of two fully connected neural networks, a reduction ratio r and an activation function δ, and repeatedly stacking the results P times in the dimension of the number of columns to obtain the attention score A v The results obtained by the point channel attention module and the column channel attention module are passed through a shared multi-layer perceptron structure composed of two fully connected neural networks, a reduction ratio r and an activation function δ to obtain the attention scores A and A respectively. c and A p .

4. The multi-channel perception 3D point cloud target detection method according to claim 1, characterized in that: The calculation formulas for the three attention scores are: Among them, A p Indicates the point attention score, A c represents the point channel attention score, A v represents the column channel attention score, repeat(,t) means repeating a certain dimension t times, σ is the sigmoid activation function, w i represents the weight of each connection layer, where 5. The multi-channel perception 3D point cloud target detection method according to claim 1, characterized in that: The formula for constructing the complete attention matrix is: M j =A p ×A c ×A v Among them, A p Indicates the point attention score, A c represents the point channel attention score, A v Represents the column channel attention score.

6. The multi-channel perception 3D point cloud target detection method according to claim 1, characterized in that: The expression formula for extracting features at different resolutions is: Among them, ε′ 1，...，i ,ε″ 1，...，i ,ε″′ 1，...，i and ε″″ 1，...，i is a trainable parameter, i is the number of fused features, and swish() is the swish activation function.

7. The multi-channel perception 3D point cloud target detection method according to claim 1, characterized in that: The multi-scale fusion results are passed to the SSD detection head for classification and regression to achieve target detection effect.

Images