Three-dimensional point cloud target detection method and system based on cross-cylinder feature aggregation

Abstract

The three-dimensional point cloud target detection method and system based on cross-cylinder feature aggregation belong to the field of target detection technology, and solve the problem that point cloud target detection cannot take into account detection accuracy and detection speed at the same time. The method of the present invention includes: a cylinder feature encoding module, which is used to divide the point cloud data into cylinders, and encode the feature information contained in the cylinders and perform cross-cylinder feature aggregation and processing, and then convert the point cloud data into a pseudo image so as to be input into the backbone network for further feature extraction; in the backbone network, spatial features are extracted through multiple designed and optimized residual blocks and upsampling blocks to generate higher-dimensional feature maps; and the detector is used to generate prediction results of target categories, bounding boxes, and target directions. In the cylinder feature encoding module of the model, the point cloud data is divided into a series of cylinders. The cylinders are encoded to convert irregular point cloud data into regular grid structures. The present invention is suitable for three-dimensional target detection in autonomous driving scenarios.

Description

Technical Field

[0001] The present application relates to the technical field of target detection, and in particular to three-dimensional point cloud target detection. Background Art

[0002] LiDAR is one of the most commonly used sensors in the field of autonomous driving and is an indispensable means for machines to understand three-dimensional scenes. In recent years, three-dimensional point cloud object detection has attracted much attention due to its important role in applications such as autonomous driving, robotics, and intelligent transportation. The performance of three-dimensional target detection models has also become stronger with the advancement of deep learning technology. However, many recent studies have focused too much on the model's detection accuracy and ignored the need for real-time computing, which is usually deployed on mobile devices. Therefore, it is necessary to explore methods that balance accuracy and real-time capabilities.

[0003] The current mainstream point cloud target detection technologies are mainly divided into three categories: point-based methods, voxel-based methods, and cylinder-based methods. Point-based methods can make full use of the sparsity and three-dimensional structural characteristics of point cloud data by directly operating on it. However, such methods usually require calculations for each point and complex aggregation operations between global and local features, resulting in high computational costs. Voxel-based methods significantly improve the computational efficiency of the model by dividing the point cloud into regular voxel grids and extracting features using three-dimensional convolution or sparse convolution. However, due to its resolution restrictions, it may be difficult to adapt to scenes with complex working conditions. The cylinder-based method simplifies the point cloud into a cylinder structure representation and combines it with an efficient two-dimensional convolutional network, which not only significantly improves the inference speed, but also becomes the mainstream choice for real-time edge devices. However, this method still has shortcomings in detection accuracy.

[0004] In addition, many point cloud object detection models that pursue high precision have emerged in recent years. These models often further improve detection performance through complex structural design, but due to high computing requirements, they are usually difficult to deploy in practice under current hardware conditions. Therefore, how to improve detection accuracy while taking into account real-time and deployment-friendly performance is still a major challenge in the field of point cloud object detection. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that existing point cloud target detection cannot take into account both detection accuracy and detection speed at the same time, and to provide a three-dimensional point cloud target detection method and system based on cross-cylinder feature aggregation.

[0006] The present invention is implemented by the following technical solutions. On the one hand, the present invention provides a three-dimensional point cloud target detection method based on cross-cylinder feature aggregation, and the method includes:

[0007] Step 1: Acquire point cloud data and perform data enhancement processing on the point cloud data;

[0008] Step 2: Cylinderize the point cloud data enhanced in step 1, and divide the entire point cloud data into a series of regular cylinders;

[0009] Step 3: Encode the cylinder generated in step 2 and add point cloud features. At the same time, remove the empty cylinders that do not contain points, stack the remaining non-empty cylinders, and obtain the cylinder features;

[0010] Step 4: Input the column features of step 3 into the multi-layer perceptron to generate an enhanced feature vector, and use the cross-column fusion module to fuse the features of adjacent columns, including:

[0011] First, the eigenvector of each cylinder Divided into four 16-dimensional sub-features:

[0012]

[0013] in, Represents the index of the cylinder, The first eigenvector of the cylinder arrive dimension;

[0014] By fusing the adjacent previous column And the next column The fusion process of some features is as follows:

[0015]

[0016]

[0017] in, and Represents the weight values ​​from different column features;

[0018] Step 5: Map the new fused cylindrical features generated in step 4 into a pseudo image;

[0019] Step 6: Input the pseudo image in step 5 into the backbone network for further feature extraction to obtain a feature map containing deep semantic information;

[0020] Step 7: The feature map containing deep semantic information obtained in step 6 is used to predict the target information through the detection head to generate a bounding box.

[0021] Furthermore, in step 1, the data enhancement processing of the point cloud data specifically includes:

[0022] The point cloud data is enhanced by using an improved partial perception data enhancement method, wherein the improved point cloud data enhancement method comprises: replacing random noise generation with PointMixup;

[0023] PointMixup enhances the training set by mixing points from different target objects of the same category while preserving the object structure and the relationship between the spaces, specifically:

[0024] Assume that the target area in the dataset For, among them , indicating that there is a total of goals; each goal is divided into Non-overlapping regions are recorded as: ;in, Indicates the target The Regions;

[0025] Get a mixing coefficient from a Beta distribution ,in , Used to control the weights of the two foreground objects in the blending of point cloud and bounding box parameters;

[0026] For targets belonging to the same category and Mix and Represents the two objectives The point set in the region; in each non-empty region, select the first of the two regions Points to mix:

[0027]

[0028] in, is the blended point set; similarly, the bounding box parameters of the target also need to be blended:

[0029] in, , They represent the bounding box parameters such as center, scale and rotation angle respectively.

[0030] Furthermore, in step 3, the point cloud features include: original coordinates ( , , ), reflection intensity r, relative deviation from the point to the mean value within the column ( , , ) and the offset from the point to the center of the column ( , ), variance, position encoding, and the number of points within each bin.

[0031] Furthermore, the variance is obtained by:

[0032] Given a point cloud, divide it into cylinders of the same size, each containing at most For the The first points, whose three-dimensional coordinates are ,in , ;

[0033] Calculate the mean of the three-dimensional coordinates of all points in the cylinder :

[0034]

[0035] in, , , The columns are 3D coordinates of the midpoint , , The mean of

[0036] calculate The three-dimensional coordinate variance of the interior point is For the column middle , , The variance of the coordinates is as follows:

[0037]

[0038] in, , , Respectively , , Variance of the coordinates.

[0039] Furthermore, the method for obtaining the position code is:

[0040] Assume that the dimension after position encoding is ,and is a multiple of 3;

[0041] For each coordinate component , the encoding formula is as follows:

[0042]

[0043]

[0044] in, , the coordinates of the code point are obtained by alternating the sine and cosine functions; , , The encoding vectors are concatenated axially to obtain the point The position encoding vector .

[0045] Furthermore, in step 6, the backbone network is composed of several basic blocks Block ( ) are stacked, and each basic block is in steps relative to the original input feature map. Run on, including 3×3 two-dimensional convolutional layers, each layer outputs channels;

[0046] Each basic block contains two layers of convolution operations and a residual connection path, specifically:

[0047] The convolution operation uses depth-wise separable convolution;

[0048] A channel shuffle operation is added to each basic block to group and rearrange the channels of the feature map to enhance the connection between different channels;

[0049] The residual connection adds the input and output of the basic block.

[0050] Furthermore, the channel shuffling operation is specifically as follows:

[0051]

[0052] Among them, the input feature map is recorded ,in , , and Represent the batch size, number of channels, height and width respectively, represents the feature map after the channels are grouped, Represents the feature map after shuffling the channels, is the number of groups of channel shuffle, is the number of channels per group;

[0053] First, the number of channels Divide Groups, each containing channels; then shuffle the channels of each group, swap the positions of the channel groups, and change the order of their dimensions through permutation operations; finally, merge the shuffled channels into a complete feature map .

[0054] In a second aspect, the present invention provides a three-dimensional point cloud target detection system based on cross-cylinder feature aggregation, the system comprising:

[0055] A data enhancement module, used to acquire point cloud data and perform data enhancement processing on the point cloud data;

[0056] The columnarization module is used to columnarize the enhanced point cloud data and divide the entire point cloud data into a regular series of columns;

[0057] The cylinder feature acquisition module is used to encode the cylinders generated in the cylinderization module and add point cloud features, while removing empty cylinders that do not contain points, stacking the remaining non-empty cylinders, and obtaining cylinder features;

[0058] The cross-column fusion module is used to input the column features of the column feature acquisition module into the multi-layer perceptron to generate an enhanced feature vector, and use the cross-column fusion module to fuse the features of adjacent columns, specifically including:

[0059] First, the eigenvector of each cylinder Divided into four 16-dimensional sub-features:

[0060]

[0061] in, Represents the index of the cylinder, The first eigenvector of the cylinder arrive dimension;

[0062] By fusing the adjacent previous column And the next column The fusion process of some features is as follows:

[0063]

[0064]

[0065] in, and Represents the weight values ​​from different column features;

[0066] A pseudo image acquisition module, used for mapping the generated new fused cylindrical features into a pseudo image;

[0067] The deep semantic information feature map acquisition module inputs the pseudo image into the backbone network for further feature extraction to obtain a feature map containing deep semantic information;

[0068] The bounding box generation module is used to predict the target information through the feature map containing deep semantic information through the detection head and generate a bounding box.

[0069] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the steps of a three-dimensional point cloud target detection method based on cross-cylinder feature aggregation as described above are executed.

[0070] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein a plurality of computer instructions are stored in the computer-readable storage medium, and the plurality of computer instructions are used to enable a computer to execute a three-dimensional point cloud target detection method based on cross-cylinder feature aggregation as described above.

[0071] Beneficial effects of the present invention:

[0072] The method proposed in the present invention maintains a structure similar to PointPillars, such as Figure 1 As shown in the figure, it can be divided into three parts. The first part is the pillar feature encoding module (Pillar Feature Encoder), which is used to divide the point cloud data into pillars, encode the feature information contained in the pillars, and then convert the point cloud data into a pseudo image so that it can be input into the backbone network (Backbone) for further feature extraction. In the backbone network, spatial features are extracted through multiple optimized residual blocks and upsampling blocks to generate higher-dimensional feature maps. Finally, the SSD (SingleShot MultiBox Detector) detector is used to generate prediction results for target categories, bounding boxes, and target directions. In the pillar feature encoding module of the model, the point cloud data is divided into a series of pillars. Subsequently, the pillars are encoded to convert the irregular point cloud data into a regular grid structure for easy input into the subsequent backbone network for processing.

[0073] This paper uses the PointPillars neural network model as the main framework and proposes a simple and efficient cross-pillar feature fusion module. By combining more features, the model achieves considerable performance improvement with very low computing resources. A new backbone network ShuffleResNet is designed to improve detection accuracy without significantly increasing model complexity. A novel data enhancement method is introduced and improved to improve the model's detection ability for more challenging targets.

[0074] The present invention effectively improves the recognition accuracy of the target while maintaining the detection speed by performing feature extraction, fusion and detection on point cloud data.

[0075] The present invention is applicable to a three-dimensional target detection method and system for autonomous driving scenarios. Specifically, it realizes the accurate recognition and positioning of targets such as vehicles, pedestrians and obstacles in autonomous driving scenarios, and can be widely used in the fields of environmental perception, path planning and assisted driving of autonomous driving vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0077] Figure 1 It is the overall framework diagram of the method of the present invention;

[0078] Figure 2 The basic blocks of the backbone network proposed by the method of the present invention;

[0079] Figure 3 This is the visualization result of this method in point cloud target detection;

[0080] Figure 4 It is the projection of the point cloud detection result on the color image. DETAILED DESCRIPTION

[0081] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0082] Embodiment 1: A three-dimensional point cloud target detection method based on cross-cylinder feature aggregation, the method comprising:

[0083] Step 1: Acquire point cloud data and perform data enhancement processing on the point cloud data;

[0084] Step 2: The original point cloud data in step 1 is enhanced and then enters the cylinder encoder, where it is cylinderized to divide the entire point cloud data into a regular series of cylinders.

[0085] Step 3: Encode the cylinder generated in step 2 and add point cloud features. At the same time, remove the empty cylinders that do not contain points, stack the remaining non-empty cylinders, and obtain the cylinder features;

[0086] Step 4: Input the column features from step 3 into the multi-layer perceptron to generate abstract features, and use the cross-column fusion module to fuse the features of adjacent columns, including:

[0087] First, the eigenvector of each cylinder Divided into four 16-dimensional sub-features:

[0088]

[0089] in, Represents the index of the cylinder, The first eigenvector of the cylinder arrive dimension.

[0090] By fusing the adjacent previous column And the next column The fusion process of some features is as follows:

[0091]

[0092]

[0093] in, and Represents the weight values ​​from different column features;

[0094] It should be noted that the abstract feature described in this embodiment is 64-dimensional, and the dimension of the feature is adjustable. The value of 64 is a trade-off between the speed and accuracy of the model. A value that is too large will result in improved accuracy and slower speed, and vice versa. The basis for selecting this value may include empirical design and industry standards.

[0095] Step 5: Map the new fused cylindrical features generated in step 4 to the pseudo image;

[0096] Step 6: Input the pseudo image in step 5 into the backbone network for further feature extraction to obtain a feature map containing deep semantic information; Figure 1 The number of feature map channels is marked as C, and the multiple changes in feature map size are marked in the lower right corner of each feature map;

[0097] Step 7: The feature map containing deep semantic information obtained in step 6 is used to predict the target information through the detection head to generate a bounding box, such as Figure 3 and Figure 4 As shown, Figure 3 is the annotation of the bounding box in the point cloud, Figure 4 for Figure 3 The corresponding image of the point cloud and its bounding box annotations.

[0098] This implementation uses the PointPillars neural network model as the main framework and proposes a simple and efficient cross-pillar feature fusion module. By combining more features, the model achieves considerable performance improvement with very low computing resources. By extracting, fusing and detecting point cloud data, the recognition accuracy of the target is effectively improved while maintaining the detection speed.

[0099] Implementation method 2: This implementation method is a further limitation on the three-dimensional point cloud target detection method based on cross-column feature aggregation as described above. In this implementation method, the data enhancement processing of the point cloud data in step 1 is further limited, specifically including:

[0100] In step 1, the data enhancement processing of the point cloud data specifically includes:

[0101] The point cloud data is enhanced by using an improved partially-aware data enhancement (PA-AUG) method, wherein the improved method for enhancing the point cloud data comprises: replacing random noise generation with PointMixup;

[0102] PointMixup enhances the training set by mixing points from different target objects of the same category while preserving the relationship between the object structure and space. It should be noted that PointMixup was originally proposed for point cloud classification and segmentation tasks, not point cloud target detection tasks. This implementation migrates the application scenario of PointMixup and improves the PA-AUG data enhancement method. The combination of the two achieves new and more effective data enhancement, specifically:

[0103] Assume that the target area in the dataset For, among them , indicating that there is a total of goals; each goal is divided into Non-overlapping regions are recorded as: ;in, Indicates the target The Regions;

[0104] Get a mixing coefficient from a Beta distribution ,in , Used to control the weights of the two foreground objects in the blending of point cloud and bounding box parameters;

[0105] For targets belonging to the same category and Mix and Represents the two objectives The point set in the region; in each non-empty region, select the first of the two regions Points to mix:

[0106]

[0107] in, is the blended point set; similarly, the bounding box parameters of the target also need to be blended:

[0108] in, , They represent the bounding box parameters such as center, scale and rotation angle respectively.

[0109] This embodiment provides an improved PA-AUG data enhancement, which includes applying PointMixup to target division areas; limiting the mixing operation to the same category; and for 3D target detection tasks, mixing bounding box parameters other than the point cloud data itself. This data enhancement method is used to improve the model's detection capability for difficult targets.

[0110] Implementation method 3: This implementation method further limits the three-dimensional point cloud target detection method based on cross-cylinder feature aggregation as described above. In this implementation method, the point cloud features in step 3 are further limited, specifically including:

[0111] In step 3, the point cloud features include: original coordinates ( , , ), reflection intensity r, relative deviation from the point to the mean value within the column ( , , ) and the offset from the point to the center of the column ( , ), variance, position encoding, and the number of points within each bin.

[0112] This implementation provides a plurality of relatively reasonable point cloud features, which can improve the accuracy of point cloud target detection.

[0113] Implementation method 4: This implementation method further limits the three-dimensional point cloud target detection method based on cross-cylinder feature aggregation as described above. In this implementation method, the method for obtaining the variance is further limited, specifically including:

[0114] The method for obtaining the variance is:

[0115] Given a point cloud, divide it into cylinders of the same size, each containing at most For the The first points, whose three-dimensional coordinates are ,in , ;

[0116] Calculate the mean of the three-dimensional coordinates of all points in the cylinder :

[0117]

[0118] in, , , The columns are 3D coordinates of the midpoint , , The mean of

[0119] calculate The three-dimensional coordinate variance of the interior point is For the column middle , , The variance of the coordinates is as follows:

[0120]

[0121] in, , , Respectively , , Variance of the coordinates.

[0122] This embodiment provides a method for calculating variance features, which provides more local geometric structure statistical information at a lower computational cost, helping the model distinguish various types of local structures such as planes, edges, corners, etc. The variance features calculated by this method are used to input into a subsequent neural network model for subsequent processing, specifically, to obtain abstract features through a multi-layer perceptron.

[0123] Implementation method 5: This implementation method further limits the three-dimensional point cloud target detection method based on cross-cylinder feature aggregation as described above. In this implementation method, the method for obtaining the position code is further limited, specifically including:

[0124] The method for obtaining the position code is:

[0125] Assume that the dimension after position encoding is ,and is a multiple of 3;

[0126] For each coordinate component , the encoding formula is as follows:

[0127]

[0128]

[0129] in, , the coordinates of the code point are obtained by alternating the sine and cosine functions; , , The encoding vectors are concatenated axially to obtain the point The position encoding vector .

[0130] This embodiment provides a method for calculating position coding features. Position coding features can introduce richer spatial information by encoding the coordinates of each point, so that the model can better understand the distribution and relative position of the points in three-dimensional space. The position coding calculated by this method aims to map the coordinates of the points to a higher-dimensional feature space.

[0131] Implementation method 6: This implementation method further limits the three-dimensional point cloud target detection method based on cross-column feature aggregation as described above. In this implementation method, the backbone network in step 6 is further limited, specifically including:

[0132] In step 6, the backbone network consists of several basic blocks Block ( ) are stacked, and each basic block is in steps relative to the original input feature map. Run on, including 3×3 two-dimensional convolutional layers, each layer outputs channels;

[0133] Each basic block contains two layers of convolution operations and a residual connection path, specifically:

[0134] The convolution operation uses depth-wise separable convolution;

[0135] A channel shuffle operation is added to each basic block to group and rearrange the channels of the feature map to enhance the connection between different channels;

[0136] The residual connection adds the input and output of the basic block.

[0137] This implementation provides a novel backbone network ShuffleResNet, which aims to improve feature extraction capabilities while maintaining computational efficiency. The backbone network is based on the concept of using deep separable convolution and channel shuffle operations, integrating the feature extraction process into a residual block structure, and improving detection accuracy without significantly increasing model complexity.

[0138] Embodiment 7: This embodiment further limits the three-dimensional point cloud target detection method based on cross-cylinder feature aggregation as described above. In this embodiment, the channel shuffling operation is further limited, specifically including:

[0139] The channel shuffling operation is specifically as follows:

[0140]

[0141] Among them, the input feature map is recorded ,in , , and Represent the batch size, number of channels, height and width respectively, represents the feature map after the channels are grouped, Represents the feature map after shuffling the channels, is the number of groups of channel shuffle, is the number of channels per group;

[0142] First, the number of channels Divide Groups, each containing channels; then shuffle the channels of each group, swap the positions of the channel groups, and change the order of their dimensions through permutation operations; finally, merge the shuffled channels into a complete feature map .

[0143] This embodiment provides a specific implementation of the channel shuffle operation to improve information interaction between different channels.

[0144] Embodiment 8: This embodiment is an example of a three-dimensional point cloud target detection method based on cross-cylinder feature aggregation as described above, and specifically includes:

[0145] Step 1: Get more features, specifically:

[0146] In the PointPillars model, after the point cloud data is divided into cylinders, the original coordinates ( , , ), reflection intensity (r), relative deviation from the point to the mean within the column ( , , ), the offset from the point to the center of the column ( , ) A total of 9 dimensions of features. However, in previous studies, the geometric information and local information provided by these added manual features are relatively limited. In order to solve this problem, the variance feature and position encoding feature of the point cloud are introduced:

[0147] The variance feature provides more local geometric structure statistical information at a lower computational cost, helping the model distinguish various types of local structures such as planes, edges, corners, etc.

[0148] The position encoding feature can introduce richer spatial information by encoding the coordinates of each point, enabling the model to better understand the distribution and relative position of points in three-dimensional space.

[0149] In addition, the number of points in each cylinder is added to the feature list. This feature is simple and intuitive, and can provide information about the point density in a local range; a large number of points in a cylinder usually means a high probability of the target existing. Introducing this feature has almost no effect on the complexity of the model.

[0150] Given a point cloud, divide it into cylinders of the same size, each containing at most For the The first points, whose three-dimensional coordinates are ,in , Without loss of generality, we will examine one of the cylinders below. Provide a detailed description.

[0151] First calculate the mean of the three-dimensional coordinates of all points in the cylinder :

[0152]

[0153] in, , , The columns are 3D coordinates of the midpoint , , The mean of .

[0154] Next calculate The variance of the three-dimensional coordinates of the interior points. For the column middle , , The variance of the coordinates, according to the formula The variance is obtained as follows:

[0155]

[0156] in, , , Respectively , , Variance of the coordinates.

[0157] Position encoding aims to map the coordinates of a point to a higher-dimensional feature space. To achieve this, assume that the dimension of the position encoding is ,and It needs to be a multiple of 3 because it is common to assign the same number of features to each axis. , the encoding formula is as follows:

[0158]

[0159]

[0160] Among them, the formula middle , the coordinates of the code point are obtained by alternating the sine and cosine functions. , , The encoding vectors are concatenated axially to obtain the point The position encoding vector Finally, we get the formula The positional encoded features shown are of shape Tensor of .

[0161] Step 2: Use the cross-column fusion module to fuse the features of adjacent columns, specifically:

[0162] The point cloud features in step 1 are converted into abstract 64-dimensional features through a multi-layer perceptron (MLP). In this part, the feature vector of each column is first converted into Divided into four 16-dimensional sub-features:

[0163]

[0164] in, Represents the index of the cylinder, The first eigenvector of the cylinder arrive dimension.

[0165] The main idea of ​​this method is to merge the adjacent previous columns And the next column To enhance its representation ability, the fusion process is as follows:

[0166]

[0167]

[0168] in, and Represents the weight values ​​from different column features. Specifically, No. dimensional features are retained, and adjacent features are fused for the remaining feature dimensions. dimensional feature as an example, the first Dimensional Features The current column Dimensional Features To obtain more adjacent column features, in this embodiment, you can select and .

[0169] Step 3: Use the backbone network for further feature extraction, specifically:

[0170] A novel backbone network ShuffleResNet is designed to improve feature extraction capabilities while maintaining computational efficiency. The backbone network is based on the idea of ​​using depthwise separable convolution and channel shuffle operations to integrate the feature extraction process into a residual block structure.

[0171] The entire backbone network consists of several basic blocks ( ) are stacked, and the structure of each basic block is as follows Figure 2 As shown. Each basic block has a step size relative to the original input feature map Run on, including 3×3 two-dimensional convolutional layers, each layer outputs Each basic block contains two layers of convolution operations and a residual connection path.

[0172] Furthermore, the convolution operation adopts Depthwise Separable Convolution, which effectively reduces the amount of calculation and parameter scale while retaining the expressive power of feature extraction.

[0173] In order to further improve the information interaction between columns, a channel shuffle operation is added to each basic block to group and rearrange the channels of the feature map to enhance the connection between different channels.

[0174] The residual connection adds the input and output of the basic block, thereby enhancing the gradient propagation and feature preservation capabilities.

[0175] Furthermore, the input feature map is recorded ,in , , and Represents batch size, number of channels, height and width respectively. The operation steps of channel shuffling can be decomposed as follows:

[0176]

[0177] in, represents the feature map after the channels are grouped, Represents the feature map after shuffling the channels, is the number of groups of channel shuffle, is the number of channels in each group. Divide Groups, each containing Then, the channels of each group are shuffled, the positions of the channel groups are swapped, and the order of their dimensions is changed by permutation. Finally, the shuffled channels are merged into a complete feature map. .

[0178] In order to achieve effective multi-scale feature fusion, this embodiment also designs a feature upsampling and fusion module. Figure 1 As shown in the backbone network part, the feature map undergoes a series of convolutions to obtain deep feature information, and then the feature map is upsampled to the same size through deconvolution. In this way, feature maps of different sizes from different scales can be spliced ​​to achieve multi-scale feature fusion. The upsampling operation can be expressed as Up( ), the initial step length The features are upsampled to the final step size This operation restores the feature maps extracted by the basic block to a larger spatial resolution, which is convenient for subsequent detection tasks. After applying batch normalization and ReLU activation after upsampling, multiple feature maps are connected along the channel dimension, and finally feature maps from multiple scales are obtained.

[0179] Furthermore, in step 1, the point cloud data needs to be enhanced, specifically:

[0180] In addition to the model itself, data enhancement is one of the important means to improve the success rate of detection. The present invention introduces and improves an efficient data enhancement method based on point cloud data: Part-Aware Data Augmentation (PA-AUG for short). The core idea of ​​PA-AUG is to use the different features contained in different parts of the target object to divide the target into multiple partitions, and randomly apply five data enhancement methods, namely internal point Dropout, CutMix, CutMixUp, sparse sampling and random noise generation, to each partition. By mixing and applying these different sub-methods, richer data samples can be generated in training. However, studies have shown that random noise may introduce invalid features in some low-density point cloud scenarios, thereby affecting the training effect of the model. Therefore, an improved strategy is proposed to replace random noise generation with PointMixup to more realistically simulate the deformation of the object.

[0181] PointMixup enhances the training set by mixing points from different target objects of the same category while preserving the object structure and spatial relationship. For, among them , indicating that there is a total of goals. Each goal is divided into Non-overlapping regions are recorded as: .in, Indicates the target The regions. First we need a mixing coefficient from a Beta distribution ,in , It is used to control the weight of the two foreground objects in the blending of point cloud and bounding box parameters. Obviously, it is necessary to and Mix. and Represents the two objectives In each non-empty region, select the first of the two regions. Points to mix:

[0182]

[0183] in is the blended point set. Similarly, the bounding box parameters of the target also need to be blended:

[0184] in , represents the bounding box parameters such as center, scale, and rotation angle.

[0185] The present invention is evaluated on the KITTI dataset to verify its effectiveness, which is one of the most popular datasets in the field of autonomous driving. It contains 7481 training samples and 7518 test samples. Each sample includes a color image and a corresponding point cloud. The targets of each category are divided into three levels of difficulty: easy, medium, and difficult according to their size and degree of occlusion. The training samples are divided into a training set (3712 samples) and a test set (3769 samples). The model is trained in the training set and compared with existing methods in the field with a more accurate 40 recall positions in the test set.

[0186] In position encoding, the feature dimension of the encoding determines the length of the encoding vector generated for each point. More feature dimensions can capture richer position information, but it will also increase computational overhead and memory usage. In the method of the present invention, the feature dimension of the encoding is set to 96. In the cross-cylinder fusion module, the weight of the adjacent cylinder is set to 0.6, and the weight of the current cylinder is set to 0.4. The backbone network is configured as three stages, each containing {1, 2, 2} blocks. Each stage starts with a step size of 2, and the feature map is downsampled. The number of filters in each stage is set to {128, 128, 256}. When upsampling, three transposed convolutional layers with step sizes of {1, 2, 4} are used, and each layer outputs 128 channels. In terms of the detection head, this model adopts the SSD framework to directly predict the three-dimensional bounding box and class score from the fused feature map. The loss function uses the same settings as PointPillars and SECOND, using Focal Loss for target classification, Smooth L1 Loss for bounding box regression, and Softmax classification loss to learn the orientation of the target. The parameters involved in PointMixup are set as follows: of Distribution parameters and , both are set to 1, indicating that the point clouds of the two targets are evenly mixed. Indicates the probability of applying the PointMixup enhancement method is set to 0.5, which means that half of the samples will be executed by this method. At the same time, the distance limit is set to 100 to avoid enhancing the uncertainty caused by targets that are too far away. All other parameters not mentioned are set by default. The graphics card used for model training and testing is an NVIDIA RTXA6000 48G.

[0187] Table 1 Comparison of this method with other methods in 3D object detection

[0188]

[0189] According to the KITTI evaluation criteria, the intersection-over-union ratios of the three categories were set to 0.7, 0.5, and 0.5 respectively to test all models. As shown in Table 1, this method achieved the highest mean average precision (mAP) of 67.61, which is better than mainstream methods such as VoxelNeXt and CenterPoint, demonstrating its advantage in comprehensive accuracy.

[0190] For the car category, although this method did not achieve the best performance in this category, it still maintained a stable and competitive performance (87.77%, 76.60%, 73.52%). In the pedestrian category, this method is only behind F-PointNet and VoxelNeXt, which use both point cloud and color image modalities, at the easy difficulty. At the medium and hard difficulty, this method is also ahead of PillarNet, which also uses a pillar-based method. In the cyclist category, this method shows a significant advantage over other methods. As a composite target (cyclist + bicycle), cyclists have the characteristics of variable shapes and rich structures, which reflects that this method has a finer spatial resolution and stronger ability to obtain local context information. In addition, while maintaining excellent overall performance, this model also performs well in detection speed, reaching 65.49 and 60.17 frames per second (FPS), second only to PointPillars, proving its feasibility of deployment on real-time devices.

[0191] Table 2 Comparison of this method with other methods in target detection from a bird's eye view

[0192]

[0193] Table 2 shows the performance of the model under the bird's eye view (BEV) perspective. The method of the present invention is slightly behind the two-stage advanced network PointRCNN in terms of overall mAP, and is close to or even exceeds it in terms of bird's eye view detection accuracy at all difficulty levels, while the detection speed is significantly ahead. Various experiments have proved that the method of the present invention achieves an excellent balance between speed and accuracy.

Claims

1. A three-dimensional point cloud target detection method based on cross-cylinder feature aggregation, characterized in that: The method comprises: Step 1: Acquire point cloud data and perform data enhancement processing on the point cloud data; Step 2: Cylinderize the point cloud data enhanced in step 1, and divide the entire point cloud data into a series of regular cylinders; Step 3: Encode the cylinder generated in step 2 and add point cloud features. At the same time, remove the empty cylinders that do not contain points, stack the remaining non-empty cylinders, and obtain the cylinder features; Step 4: Input the column features of step 3 into the multi-layer perceptron to generate an enhanced feature vector, and use the cross-column fusion module to fuse the features of adjacent columns, including: First, the eigenvector of each cylinder Divided into four 16-dimensional sub-features: in, Represents the index of the cylinder, The first eigenvector of the cylinder arrive dimension; By fusing the adjacent previous column And the next column The fusion process of some features is as follows: in, and Represents the weight values ​​from different column features; Step 5: Map the new fused cylindrical features generated in step 4 into a pseudo image; Step 6: Input the pseudo image in step 5 into the backbone network for further feature extraction to obtain a feature map containing deep semantic information; Step 7: The feature map containing deep semantic information obtained in step 6 is used to predict the target information through the detection head to generate a bounding box.

2. The three-dimensional point cloud target detection method based on cross-cylinder feature aggregation according to claim 1 is characterized in that: In step 1, the data enhancement processing of the point cloud data specifically includes: Performing data enhancement processing on the point cloud data using an improved partial perception data enhancement method, wherein the improved partial perception data enhancement method comprises: replacing random noise generation with PointMixup; PointMixup enhances the training set by mixing points from different target objects of the same category while preserving the object structure and the relationship between the spaces, specifically: Assume that the target area in the dataset For, among them , indicating that there is a total of goals; each goal is divided into Non-overlapping regions are recorded as: ;in, Indicates the target The Regions; Get a mixing coefficient from a Beta distribution ,in , Used to control the weights of the two foreground objects in the blending of point cloud and bounding box parameters; For targets belonging to the same category and Mix and Represents the two objectives The point set in the region; in each non-empty region, select the first of the two regions Points to mix: in, is the blended point set; similarly, the bounding box parameters of the target also need to be blended: in, , denote the center, scale and rotation angle bounding box parameters respectively, and Respectively represent the target and The bounding box parameters of .

3. The three-dimensional point cloud target detection method based on cross-cylinder feature aggregation according to claim 1, characterized in that: In step 3, the point cloud features include: original coordinates ( , , ), reflection intensity r, relative deviation from the point to the mean value within the column ( , , ) and the offset from the point to the center of the column ( , ), variance, position encoding, and the number of points within each bin.

4. The three-dimensional point cloud target detection method based on cross-cylinder feature aggregation according to claim 3 is characterized in that: The method for obtaining the variance is: Given a point cloud, divide it into cylinders of the same size, each containing at most For the The first points, whose three-dimensional coordinates are ,in , ; Calculate the mean of the 3D coordinates of all points in the cylinder : in, , , The columns are 3D coordinates of the midpoint , , The mean of calculate The three-dimensional coordinate variance of the interior point is For the column middle , , The variance of the coordinates is as follows: in, , , Respectively , , Variance of the coordinates.

5. The three-dimensional point cloud target detection method based on cross-cylinder feature aggregation according to claim 4 is characterized in that: The method for obtaining the position code is: Assume that the dimension after position encoding is ,and is a multiple of 3; For each coordinate component , the encoding formula is as follows: in, , the coordinates of the code point are obtained by alternating the sine and cosine functions; , , The encoding vectors are concatenated axially to obtain the point The position encoding vector .

6. The three-dimensional point cloud target detection method based on cross-cylinder feature aggregation according to claim 1, characterized in that: In step 6, the backbone network consists of several basic blocks ( ) are stacked, and each basic block has a step size relative to the original input feature map. Run on, including 3×3 two-dimensional convolutional layers, each layer outputs channels; Each basic block contains two layers of convolution operations and a residual connection path, specifically: The convolution operation uses depth-wise separable convolution; A channel shuffle operation is added to each basic block to group and rearrange the channels of the feature map to enhance the connection between different channels; The residual connection adds the input and output of the basic block.

7. The three-dimensional point cloud target detection method based on cross-cylinder feature aggregation according to claim 6 is characterized in that: The channel shuffling operation is specifically as follows: Among them, the input feature map is recorded ,in , , and Represent the batch size, number of channels, height and width respectively, represents the feature map after the channels are grouped, Represents the feature map after shuffling the channels, is the number of groups of channel shuffle, is the number of channels per group; First, the number of channels Divide Groups, each containing channels; then shuffle the channels of each group, swap the positions of the channel groups, and change the order of their dimensions through permutation operations; finally, merge the shuffled channels into a complete feature map .

8. A three-dimensional point cloud target detection system based on cross-cylinder feature aggregation, characterized in that: The system comprises: A data enhancement module, used to acquire point cloud data and perform data enhancement processing on the point cloud data; The columnarization module is used to columnarize the enhanced point cloud data and divide the entire point cloud data into a regular series of columns; The cylinder feature acquisition module is used to encode the cylinders generated in the cylinderization module and add point cloud features, while removing empty cylinders that do not contain points, stacking the remaining non-empty cylinders, and obtaining cylinder features; The cross-column fusion module is used to input the column features of the column feature acquisition module into the multi-layer perceptron to generate an enhanced feature vector, and use the cross-column fusion module to fuse the features of adjacent columns, specifically including: First, the eigenvector of each cylinder Divided into four 16-dimensional sub-features: in, Represents the index of the cylinder, The first eigenvector of the cylinder arrive dimension; By fusing the adjacent previous column And the next column The fusion process of some features is as follows: in, and Represents the weight values ​​from different column features; A pseudo image acquisition module, used for mapping the generated new fused cylindrical features into a pseudo image; The deep semantic information feature map acquisition module inputs the pseudo image into the backbone network for further feature extraction to obtain a feature map containing deep semantic information; The bounding box generation module is used to predict the target information through the feature map containing deep semantic information through the detection head and generate a bounding box.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor runs the computer program stored in the memory, the steps of the method according to any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of computer instructions, and the plurality of computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images