A three-dimensional target detection method based on point cloud volume columnization

Abstract

The application discloses a kind of three-dimensional target detection methods based on point cloud body column, first the point cloud data in the range taken is body column, the point cloud in each body column is divided into 4 units according to height direction, according to different height, the feature extraction of point cloud data in 4 units is carried out and is dimensioned by full connection network, finally the features after dimensioning are spliced together for the fusion of features to reduce the information loss caused when it is compressed in height direction.The application carries out the extraction and fusion of features on different heights of body column respectively, so that it can retain the features of objects at different heights during generating two-dimensional pseudo image, which is beneficial to the detection of objects at different heights.The application locally uses voxelization to dimension and aggregate features, then generates pseudo image for the aggregated features, so that it can use existing mature architecture for processing two-dimensional image, which reduces the operation speed and also improves the detection accuracy.

Description

Technical Field

[0001] This invention relates to autonomous driving technology for automobiles, and in particular to a three-dimensional target detection method based on point cloud volumetric scaling. Background Technology

[0002] An autonomous driving system is a complex system comprising an environmental perception module, a decision-making module, and a control module. The environmental perception module uses multimodal information provided by onboard sensors to identify and detect key information about the vehicle's environment, such as targets of interest, obstacles, and feasible areas. The decision-making module then plans the driving path and speed based on the environmental perception results and issues path planning commands to the control module. Finally, the control module controls the vehicle's steering wheel, brakes, and accelerator according to the commands, ensuring safe driving on the road. The environmental perception module, as a crucial module in the autonomous driving system, is the first step in autonomous driving. The accuracy, robustness, and computational speed of environmental perception directly affect the rationality of the control commands issued by the decision-making module, and are directly related to the safety of autonomous driving.

[0003] Environmental perception is crucial for autonomous vehicles, enabling 3D understanding of targets of interest within the environment. 3D target detection is the core of 3D scene perception and understanding. 3D target detection involves detecting physical objects from 3D sensor data, estimating 3D bounding boxes, and assigning specific categories. Current 3D target detection primarily utilizes a combination of monocular cameras, binocular cameras, and multi-line LiDAR. LiDAR scans generate point cloud data containing the spatial coordinates and corresponding attribute information of objects, representing the target in the real world. This provides accurate depth information, enabling the identification, localization, and attitude estimation of targets in the environment, thereby obtaining information such as target position, size, and driving direction. This fundamentally improves the accuracy of perception and enhances the safety of autonomous driving.

[0004] With the rapid development of deep learning theory, significant breakthroughs have been achieved in many research areas, such as computer vision, speech recognition, and natural language processing. Deep learning-based LiDAR point cloud target detection algorithms are also attracting increasing attention. Feature extraction is a crucial step in deep learning network models; it can be understood as a data dimensionality reduction process. Feature extraction is extremely useful when it's necessary to reduce the amount of processing resources required without losing important information. Essentially, feature extraction transforms raw data into processable digital features, preserving key information from the original dataset. Feature extraction also reduces redundant data in a given analysis, which can improve the speed of deep learning and generalization steps.

[0005] Point clouds, as a fundamental data format for LiDAR output, preserve the original geometric state in three-dimensional space and provide rich shape and scale information, making them a crucial representation for perception and understanding in autonomous driving scenarios. Currently, deep learning-based LiDAR point cloud data detection algorithms can be categorized into three subclasses: projection-based methods, voxel representation-based methods, and raw point cloud-based methods. Among these, voxel representation-based methods generate a regular voxel grid by equally dividing the point cloud space and then using convolutional neural networks for feature extraction. This method increases data processing efficiency and is widely adopted due to its efficient use of spatial convolution; however, it leads to significant computational complexity and places high demands on both the algorithm and hardware. To address this issue, some scholars have proposed a method of preprocessing point clouds into volumetric columns. Since the height of the volumetric column is the same as the entire point cloud space, point cloud features can be compressed along the height direction to convert the three-dimensional point cloud data into a two-dimensional pseudo-feature map, thus completing the feature extraction of the point cloud. Although this method achieves very high processing efficiency, due to the disordered, sparse, unstructured, and limited information characteristics of point clouds, information loss is inevitable when compressing the entire point cloud volumetric column, especially information loss in the height direction. Summary of the Invention

[0006] To address the issue of information loss in the height direction during point cloud volumetric compression, this invention proposes a three-dimensional target detection method based on point cloud volumetric compression that can reduce information loss during height compression.

[0007] To achieve the above objectives, the basic idea of ​​this invention is as follows: First, the point cloud data within the selected range is columnarized. The point cloud within each column is divided into four units according to the height direction. Based on different heights, features are extracted from the point cloud data within the four units and then upgraded through a fully connected network. Finally, the upgraded features are concatenated in parallel to perform feature fusion, thereby reducing the information loss caused by high compression.

[0008] The technical solution of this invention is as follows: A three-dimensional target detection method based on point cloud volumetric columnarization, comprising the following steps:

[0009] A. Columnar Division

[0010] Let the coordinate system of the point cloud space be xyz, where the origin of the coordinate system is the location of the sensor, the x-axis is a horizontal direction parallel to the ground, the y-axis is the forward detection direction of the sensor in the horizontal plane and is perpendicular to the x-axis, and the z-axis is a vertical direction. The required point cloud space is constructed by taking point clouds within a certain range along the xyz directions.

[0011] The entire point cloud space is uniformly divided into square grids on the xy plane, with each grid having a side length of 'a'. A height 'h' is taken along the z-axis, ensuring that 'h' is equal to the height of the point cloud space. The entire point cloud space is then divided into a certain number of volumetric columns along the predetermined x, y, and z axes. For each volumetric column, a sample is taken every 1 / 4 of the volume along the z-axis, dividing the column into four voxel grids of equal volume. These voxel grids are then combined from top to bottom in sequences of 4 voxel grids, 3 voxel grids, 2 voxel grids, and 1 voxel grid to obtain four volumetric columns of different heights.

[0012] B. Feature Extraction and Pseudo-Image Generation

[0013] The dimensionality of each data point within the volume column is increased and features are extracted. Each data point within the volume column is a 4-dimensional feature consisting of coordinate information x, y, z, and reflectance information r. During dimensionality increase, to enhance the information representation of the data points, the feature dimension is first expanded to 9 dimensions. Then, each data point within the volume column is processed through a fully connected layer to generate a tensor of size A = (C, P, N), where C is the number of channels, P is the number of volume columns, and N is the number of data points within the volume column. Max pooling is then applied along the channel directions to obtain a tensor B = (C, P). Finally, the output tensors are stacked according to the segmentation positions of the volume columns, generating four pseudo-images with different height point cloud information based on the segmentation height coordinates. The size of each pseudo-image is... Where P i ∈R H×W×C This represents a depth image, where i represents the image label, and H and W represent the image height and width.

[0014] C. Fusion of attention mechanism features

[0015] Because the feature information contained in the corresponding pseudo-images differs at different point cloud environments, an attention mechanism is employed for feature fusion to better extract features. This mechanism uses a network to focus on the most important feature information at different altitudes, thereby enhancing the feature representation capability. For the four generated pseudo-images at different altitudes... First, the features are stacked together in the C direction to form a feature map F∈R. 4C×H×W The feature map is fed into two branches: one branch performs no processing, and the other branch passes it through three convolutional layers with kernel size 1. In the first branch, the size of the feature map remains unchanged after the first convolutional layer, still being F'∈R. 4C×H×W After the second convolutional layer, the size of the feature map changes to... Finally, the features are upscaled through a convolutional layer to become F'∈R. 4C×H×W In the second branch, the size of the feature map remains unchanged as F”∈R 4C×H×WFinally, the two branch feature maps are element-wise summed to obtain a new feature map F = F' + F” ∈ R. 4C×H×W .

[0016] D. Object Detection

[0017] The feature map generated after the attention mechanism is fed into the backbone network. The backbone network consists of two sub-networks. The first sub-network... Where γ represents the activation function, δ represents batch normalization, α represents the channel augmentation factor, and λ represents the size reduction factor, the first sub-network gradually reduces the feature map size to half its original size while doubling the number of channels in a top-down sequence, generating features with increasingly smaller spatial resolution. The first sub-network consists of three convolutional modules, each of which uses a 3×3 two-dimensional convolutional layer with a stride of 2 to perform downsampling to extract fine-grained features; the second sub-network... Where F1, F2, and F3 represent the feature maps generated by the first sub-network in the three convolutional modules, and σ represents the concatenation operation. The first sub-network represents deconvolution. The second sub-network performs upsampling and concatenated top-down features to fuse them and generate the final feature map. Finally, the generated feature map is passed through convolutional layers in a regression manner to predict the target class, bounding box, and orientation at each pixel of the feature map, ultimately outputting the network's inferred 3D bounding box.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] 1. Existing feature extraction methods based on volumetric point clouds, such as PointPillars, suffer from reduced accuracy in detecting objects of different heights due to the loss of height information in the Z-axis direction caused by compression during the generation of 2D pseudo-images. This invention extracts and fuses features at different heights of the volumetric column, preserving the features of objects at different heights during the generation of 2D pseudo-images.

[0020] 2. Existing voxelization-based feature extraction methods, such as VoxelNet, require the use of 3D convolution or sparse convolution during point cloud feature extraction, resulting in slow computation speed. This invention employs voxelization locally for feature upscaling and aggregation, followed by pseudo-image generation of the aggregated features. This allows the use of existing mature architectures for processing 2D images, reducing computation speed while improving detection accuracy. Attached Figure Description

[0021] Figure 1 This is a flowchart of the present invention.

[0022] Figure 2 This is a flowchart of the pseudo-image generation process.

[0023] Figure 3 It is an attention mechanism fusion diagram.

[0024] Figure 4 This is the backbone network diagram. Detailed Implementation

[0025] The technical solution of the application will be described in detail below with reference to the accompanying drawings.

[0026] This invention addresses existing problems by proposing a target detection algorithm based on point cloud volumetric scaling. The main approach transforms 3D object detection into a 2D object detection method, performing target detection on a 2D pseudo-image. First, features are extracted at different heights through point cloud volumetric scaling. Then, the volumes are stacked to generate a 2D pseudo-image. Prediction is performed on this 2D image using a mature 2D network architecture. Finally, the final detection bounding box is output by a multi-task detection head prediction model.

[0027] like Figure 1 As shown, a 3D target detection method based on point cloud volumetric scaling includes the following steps:

[0028] Step 1: Body column division

[0029] First, the original point cloud data is used as input point clouds, taking a cubic range of -3 meters to 1 meter along the radar coordinate system in the z-axis direction, -40 meters to 40 meters along the x-axis direction, and 0 meters to 70.4 meters along the y-axis direction. The entire point cloud space is then uniformly divided into equally spaced grids in the xy-plane, with each grid having a length and width of 0.16 meters and a z-axis dimension of 4 meters to maintain the same height as the point cloud space. This results in 440×500 individual cylinders. Figure 2 As shown, for the obtained volume column, samples are taken every 1 / 4 along the z-axis to divide the volume column into 4 voxel grids of the same volume. Then, the obtained voxel grids are combined from top to bottom in the manner of 4 voxel grids, 3 voxel grids, 2 voxel grids and 1 voxel grid to obtain 4 volume columns of different heights.

[0030] Due to the inherently strong sparsity of point clouds, most of the resulting volumetric columns are empty, and the number of points within most non-empty columns is also relatively small. To limit this sparsity, it is necessary to restrict the number of non-empty columns in each sample and the number of points in each column. Specifically, if a sample or column contains too much data, it is randomly sampled to filter the data. Conversely, if a sample or column has too little data to fill the tensor, it is filled with zeros. To maintain a balanced number of data points within each voxel grid, selecting 8 data points in each voxel grid keeps the total number of data points within the entire column at 32.

[0031] Step 2: Feature Extraction and Pseudo-Image Generation

[0032] Before extracting features from each cylinder, the feature dimensions of the point cloud data within the cylinder should be expanded. For a point in the point cloud data, its feature information is a 4-dimensional tensor Q = (x, y, z, r) consisting of coordinate information x, y, z and reflectance information r. To enhance the information representation of the point cloud, the feature dimensions of the data points are expanded to 9 dimensions, which can be represented as W = (x, y, z, r, x...). c ,y c ,z c ,x p ,y p ), where x c ,y c ,z c The distance x represents the coordinates of a data point from the arithmetic mean of all points within the cylinder. p ,y p This represents the deviation of the data point from the center of the cylinder in the xy direction. Its calculation process can be expressed as:

[0033]

[0034] x c =|xx m |,y c =|yy m |,z c =|zz m | (2)

[0035] x p =xx t ,y p =yy t (3)

[0036] Where x i y i z i Let x represent the coordinates of the i-th data point in the column. m y mz m x represents the arithmetic mean of the coordinates of all data points within the column. t y t The x-axis and y-axis coordinates of the center point of the cylinder are represented.

[0037] Subsequently, each data point in the volume pillar is passed through a fully connected layer, a batch normalization layer, and an activation function layer to generate a tensor of size A = (C, P, N), where C is the number of channels, P is the number of volume pillars, and N is the number of data points within the volume pillar. Then, max pooling is applied along the channel directions to obtain an output tensor of size B = (C, P). Finally, the tensors are stacked according to the segmentation positions of the volume pillars, i.e., based on the segmentation height coordinates, four pseudo-images with different height point cloud information are generated. The size of each pseudo-image is... Where P i ∈R H×W×C This represents a two-dimensional depth image, where i represents the image label, and H and W represent the image height and width.

[0038] Step 3: Attention Mechanism Feature Fusion

[0039] To better capture features at different altitudes, this invention employs an attention mechanism for feature fusion. Because the feature information contained in the corresponding pseudo-images differs at different point cloud environments, an attention mechanism can be used for feature fusion to improve feature extraction. This mechanism utilizes the network to focus on the most important feature information at different altitudes, thereby enhancing the feature representation capability. For example... Figure 3 As shown, for the four generated pseudo-images at different heights... First, the pseudo-images are stacked together along the channel direction to form a feature map F∈R. 4C×H×W The feature map is fed into two branches: one branch performs no processing, and the other branch passes it through three convolutional layers with kernel size 1. In the first branch, the size of the feature map remains unchanged after the first convolutional layer, still being F'∈R. 4C×H×W After the second convolutional layer, the size of the feature map changes to... Finally, the features are upscaled through a convolutional layer to become F'∈R. 4C×H×W In the second branch, the size of the feature map remains unchanged as F”∈R 4C×H×W Finally, the two branch feature maps are element-wise summed to obtain a new feature map F = F' + F” ∈ R. 4C×H×W .

[0040] Step 4: Object Detection

[0041] After passing through the attention mechanism, a new feature map F∈R will be generated. 4C×H×W First, it passes through a convolutional network to make its size F∈RC×H×W Then it is sent to the backbone network. For example... Figure 4 As shown, the backbone network consists of two sub-networks, the first sub-network... Where γ represents the activation function, δ represents batch normalization, α represents the channel augmentation factor, and λ represents the size scaling factor. The first sub-network gradually reduces the feature map size to half its original size while doubling the number of channels in a top-down sequence, thus generating features with increasingly smaller spatial resolutions. It consists of three convolutional modules, each of which uses a 3×3 two-dimensional convolutional layer with a stride of 2 to perform downsampling to extract fine-grained features. After passing through the first sub-network, three different sub-feature maps are obtained. Second sub-network Where F1, F2, and F3 represent the feature maps generated by the first sub-network in the three convolutional modules, and σ represents the concatenation operation. In other words, deconvolution occurs when the second sub-network first upsamples the input feature map, transforming F1, F2, and F3 to the same size. Then, a top-down function is concatenated to perform feature fusion and concatenation in the C-direction to generate the final feature map. Finally, the generated feature map is passed through a convolutional layer in a regression manner to predict the target category, bounding box, and orientation at each pixel of the feature map, ultimately outputting the 3D bounding box inferred by the network.

[0042] This invention is not limited to this embodiment. Any equivalent concept or modification within the technical scope disclosed in this invention shall be included within the protection scope of this invention.

Claims

1. A 3D target detection method based on point cloud volumetric scaling, comprising the following steps: A. Columnar Division Let the coordinate system of the point cloud space be xyz, where the origin of the coordinate system is the location of the sensor, the x-axis is the horizontal direction parallel to the ground, the y-axis is the forward detection direction of the sensor in the horizontal plane and is perpendicular to the x-axis, and the z-axis is the vertical direction; the point cloud within a certain range in the xyz direction is used to form the required point cloud space. The entire point cloud space is uniformly divided into square grids on the xy plane, with the side length of each grid set to 'a'. A height 'h' is taken in the z-axis direction, ensuring that the height 'h' is equal to the height of the point cloud space. The entire point cloud space is then divided into a certain number of volume pillars along the set x, y, and z axes. For each volume pillar, a sample is taken every 1 / 4 along the z-axis direction, dividing the volume pillar into four voxel grids of the same volume. The obtained voxel grids are then combined from top to bottom in the following ways: 4 voxel grids, 3 voxel grids, 2 voxel grids, and 1 voxel grid, respectively, to obtain four volume pillars of different heights. B. Feature Extraction and Pseudo-Image Generation Each data point within a volume column is updimensionalized and its features are extracted. Each data point within a volume column is a 4-dimensional feature consisting of coordinate information (x, y, z) and reflectivity information (r). During the updimensionalization process, to enhance the information representation of the data points, the feature dimension is first expanded to 9 dimensions. Then, each data point within a volume column is processed through a fully connected layer to generate a tensor of size A = (C, P, N), where C is the number of channels, P is the number of volume columns, and N is the number of data points within the volume column. Next, max pooling is applied along the channel direction to obtain a tensor B = (C, P). Finally, the output tensors are stacked according to the segmentation positions of the volume columns, i.e., four pseudo-images with different height point cloud information are generated based on the segmentation height coordinates. The size of each pseudo-image is... Where P i ∈R H×W×C This represents a depth image, where i represents the image label, and H and W represent the image height and width. C. Fusion of attention mechanism features Because the feature information contained in the corresponding pseudo-images differs at different point cloud environments, an attention mechanism is employed for feature fusion to better extract features. This mechanism utilizes the network to focus on the most important feature information at different altitudes, thereby enhancing the feature representation capability. For the four generated pseudo-images at different altitudes... First, the features are stacked together in the C direction to form a feature map F∈R. 4C×H×W The feature map is fed into two branches. One branch is left unprocessed, while the other branch goes through three convolutional layers with a kernel size of 1. In the first branch, the size of the feature map remains unchanged after the first convolutional layer, still being F'∈R. 4C×H×W After the second convolutional layer, the size of the feature map changes to... Finally, the features are upscaled through a convolutional layer to become F'∈R. 4C×H×W In the second branch, the size of the feature map remains unchanged as F”∈R 4C×H×W Finally, the two branch feature maps are element-wise summed to obtain a new feature map F = F' + F” ∈ R. 4C×H×W ; D. Object Detection The feature map generated after the attention mechanism is fed into the backbone network; the backbone network consists of two sub-networks; the first sub-network Where γ represents the activation function, δ represents batch normalization, α represents the channel augmentation factor, and λ represents the size reduction factor, the first sub-network gradually reduces the feature map size to half its original size while doubling the number of channels in a top-down sequence, generating features with increasingly smaller spatial resolution. The first sub-network consists of three convolutional modules, each of which uses a 3×3 two-dimensional convolutional layer with a stride of 2 to perform downsampling to extract fine-grained features; the second sub-network... Where F1, F2, and F3 represent the feature maps generated by the first sub-network in the three convolutional modules, and σ represents the concatenation operation. The first sub-network represents deconvolution. The second sub-network performs upsampling and concatenated top-down functions to fuse features and generate the final feature map. Finally, the generated feature map is passed through convolutional layers in a regression manner to predict the target class, bounding box, and orientation at each pixel of the feature map, ultimately outputting the 3D bounding box inferred by the network.

Images