3D object detection method combining attention mechanism and voxel feature aggregation

Abstract

The present invention discloses a three-dimensional target detection method that combines an attention mechanism with voxel feature aggregation, belonging to the field of target detection technology. The method comprises: obtaining three-dimensional point cloud data of an image to be detected; extracting voxel features of the voxelized three-dimensional point cloud data using a sparse convolutional network; projecting the voxel features onto a bird's-eye view representation to generate a bird's-eye view; extracting enhanced features of the bird's-eye view using a spatial and channel collaborative attention mechanism; downsampling the original three-dimensional point cloud data to obtain key points, and performing secondary optimization on the key points; utilizing an enhanced extended voxel set abstraction module to aggregate the voxel features and the bird's-eye view onto corresponding key points to obtain the final features of the image to be detected; and obtaining a predicted bounding box through 3D proposal refinement. The present invention improves the robustness of the three-dimensional target detection model and increases the generalization capability.

Description

Technical Field

[0001] The present invention relates to a three-dimensional target detection method, specifically a three-dimensional target detection method combining an attention mechanism and voxel feature aggregation, belonging to the field of target detection technology. Background Art

[0002] Three-dimensional object detection is one of the core technologies in the field of autonomous driving and robotic perception. It aims to identify and locate objects in three-dimensional space from point cloud data. The current mainstream methods can be divided into two categories: voxel-based and point-based technologies. Voxel-based methods (such as VoxelNet) convert point clouds into regular voxel grids and use 3D convolutional neural networks to extract features. Although computationally efficient, the voxelization process can lead to the loss of geometric details. Point-based methods (such as PointNet++ and PointRCNN) operate directly on the original point cloud and can better preserve the geometric structure, but the computational complexity is high and it is difficult to handle large-scale scenes. In addition, existing methods still have shortcomings in feature fusion, such as the failure to effectively combine multi-scale features or the lack of an optimized selection mechanism for key points, resulting in limited detection accuracy and robustness.

[0003] In recent years, some research has attempted to improve performance by introducing attention mechanisms or refining sampling strategies (such as farthest point sampling). However, these methods still suffer from insufficient feature representation and computational redundancy in complex scenarios. In particular, the detection accuracy of existing technologies significantly decreases when detecting distant objects and under occlusion. Therefore, there is an urgent need for an improved 3D object detection method that can efficiently fuse multi-scale features, optimize keypoint selection, and enhance feature representation to meet the dual requirements of accuracy and real-time performance for practical applications such as autonomous driving. Summary of the Invention

[0004] Purpose of the invention: In response to the above problems, the purpose of the present invention is to provide a three-dimensional target detection method that combines the attention mechanism with voxel feature aggregation. By combining voxel information and point position information, the quality of 3D target detection is improved while maintaining contextual accuracy.

[0005] Technical solution: The three-dimensional target detection method combining attention mechanism and voxel feature aggregation of the present invention includes the following steps:

[0006] Obtain the three-dimensional point cloud data of the image to be detected;

[0007] The 3D point cloud data is voxelized and the voxel features of the voxelized 3D point cloud data are extracted using a sparse convolutional network.

[0008] Project the voxel features onto the bird's-eye view representation to generate a bird's-eye view image;

[0009] Adopt spatial and channel-wise co-attention mechanism to extract enhanced features of bird’s-eye view;

[0010] Downsample the original 3D point cloud data to obtain key points, perform secondary optimization on the key points, and converge multi-level features to the key points;

[0011] Using the enhanced extended voxel set abstraction module, voxel features and bird's-eye view images are aggregated to corresponding key points to obtain the final features of the image to be detected;

[0012] Through 3D proposal refinement, the predicted bounding box is obtained;

[0013] Using the enhanced extended voxel set abstraction module, the voxel features and the bird's-eye view image are aggregated to the corresponding key points to obtain the final features of the image to be detected. The steps include:

[0014] Step 601: Take each key point Location As the center, a neighborhood search is performed in the voxel feature layer of the mth layer, with a radius of Find the adjacent non-empty voxel features and aggregate the non-empty voxel features;

[0015] Step 602: Use the PointNet deep learning model to identify key points. The neighborhood features of the key points are randomly sampled and nonlinearly transformed, and then the different numbers of neighborhood features are mapped into a fixed-size feature vector through maximum pooling to generate multi-scale features for each key point. , the formula is:

[0016] ,

[0017] Where, It is the maximum pooling in the channel dimension; It is a multi-layer perceptron used to perform nonlinear transformation on voxel features and relative position features; represents random sampling; Indicates different resolution layers; and for Layer The eigenvectors and 3D coordinates of the voxels; It is Layer radius, 、 Respectively represent The coordinates and feature sets of all non-empty voxels in the layer;

[0018] Step 603: fuse all multi-scale features to form 3D multi-scale features ;

[0019] Step 604: combine the bird's-eye view image, the enhanced features of the bird's-eye view image, and the features after context gating. , aggregated to multi-scale features to obtain the final feature F, the formula is:

[0020] ,

[0021] Where, A two-dimensional bird's-eye view. It is an enhanced feature map of the bird's-eye view.

[0022] Furthermore, the steps of voxelizing the three-dimensional point cloud data and extracting voxel features of the voxelized three-dimensional point cloud data using a sparse convolutional network include:

[0023] Divide the point P in the 3D point cloud into voxels with a spatial resolution of L×W×H, where L, W, and H represent the length, width, and height of the voxel respectively;

[0024] 3D sparse convolution is used to downsample the L×W×H voxel features to generate three-dimensional voxel features with a spatial resolution of 1 / n of the original, where n is the downsampling multiple.

[0025] Furthermore, the steps of projecting the voxel features onto the bird's-eye view representation to generate the bird's-eye view include:

[0026] The 3D voxel features obtained after n-fold downsampling are stacked along the height direction, and the stacked 3D voxel features are spliced ​​in the channel dimension to generate a 2D bird's-eye view. , the size of the bird's-eye view is recorded as (b, C′, H, W), where b represents the batch size and C′ represents the number of channels after stacking along the height direction.

[0027] Furthermore, the steps of extracting enhanced features of the bird's-eye view using the spatial and channel collaborative attention mechanism include:

[0028] The bird's-eye view image is decoupled from space and channels, and global average pooling is performed along the height H and width W directions to generate two one-dimensional feature sequences. and , the formulas are:

[0029] ,

[0030] ,

[0031] Where, represents a bird's-eye view of the input, i.e. ; represents the global average pooling operation;

[0032] One-dimensional feature sequence and Divide into K sub-features of equal size and , each sub-feature map contains a different number of channels, and the formulas are:

[0033] ,

[0034] ,

[0035] Where C represents the number of channels, Indicates the The sequence number of the sub-feature map, ;

[0036] Multi-scale depth-separable one-dimensional convolution is used to model different sub-features, extract multi-semantic spatial structure information, and obtain the first Sub-feature graph and , the formulas are:

[0037] ,

[0038] ,

[0039] Where, represents depthwise separable convolution, Indicates that it is used in Convolution kernel of sub-features;

[0040] Each sub-feature map in the height direction and width direction is grouped and normalized respectively, and the spatial attention feature map is generated by the Sigmoid activation function and , the formulas are:

[0041] ,

[0042] ,

[0043] Where, represents the Sigmoid function, and Respectively indicate along and The group normalization operation of the dimension, Represents a splicing operation;

[0044] The spatial attention feature map and Applied to the original bird's-eye view On the top, obtain the spatial enhanced feature map , the formula is:

[0045] ,

[0046] Where, Represents element-wise multiplication;

[0047] The spatially enhanced feature map is pooled using global average pooling. Feature maps compressed to a fixed resolution , the formula is:

[0048] ,

[0049] Where, represents global average pooling, Respectively represent the compressed sizes of H and W;

[0050] The compressed feature map Enter the channel self-attention mechanism to generate query Q, key K and value V. The formulas are:

[0051] ,

[0052] ,

[0053] ,

[0054] Where, Represents depthwise separable convolutions of size 1×1 on Q, K, and V respectively;

[0055] By calculating the dot product of Q and K and using Softmax normalization to obtain channel attention, it is then used for weighted values ​​and Softmax normalization is used to generate similarity weights between channels. , the formula is:

[0056] ,

[0057] The generated similarity weight With the original space characteristics Multiply to obtain enhanced channel features, and use pooling operation to aggregate similarity weights Then apply Sigmoid activation to obtain the enhanced feature map of the final bird's-eye view , the formula is:

[0058] ,

[0059] Where Pool represents the pooling operation.

[0060] Furthermore, the original 3D point cloud data is downsampled to obtain key points, and the key points are optimized twice to converge the multi-level features to the key points. The steps include:

[0061] Step 501: Input point cloud dataset is ,in is the number of input points, is the feature dimension of each point, the goal is to Select a representative subset ,in is the number of points sampled in the subset; a sampling weight is assigned to each input point, and the weight is obtained by the point cloud features and a learnable weight matrix To calculate, the formula is:

[0062] ,

[0063] ,

[0064] in, represents the sampling weight of the j-th point, is a learnable weight matrix, represents the sampling score of the jth point, k represents the kth sampling point, is the output of soft sampling, T represents transpose;

[0065] Step 502: During training, Gumbel-Softmax sampling generates continuous soft sampling outputs, as follows:

[0066] ,

[0067] Where, Represents the sampling result obtained by Gumbel-Softmax; is the sampling probability fraction of the point, is the noise sampled from a Gumbel distribution, is the annealing temperature parameter;

[0068] After soft sampling, key points are obtained , the formula is:

[0069] ,

[0070] Where N represents the number of key point features, Indicates the key point features, which are a feature subset of the original point cloud;

[0071] Step 503: key points Input to the text fusion module, extract point cloud features through the PointNet++ feature extraction network, and obtain feature points , according to the feature points Calculate the modulation characteristic g, the formula is:

[0072] ,

[0073] in, is a learnable weight matrix, is the bias term, σ is the Sigmoid activation function;

[0074] Use modulation feature g to identify feature points Perform gating processing to obtain the features after context gating , the formula is:

[0075] ,

[0076] Where, is another set of learnable weight matrices.

[0077] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0078] 1. This paper proposes a novel two-stage detection architecture specifically designed for 3D object detection in autonomous driving using only LiDAR data. Similar to most two-stage detection algorithms, the first stage uses a region proposal network to generate initial 3D proposals on a bird's-eye view feature map. However, this paper innovatively uses a method that fuses channel and spatial attention to process the bird's-eye view. By working in synergy across the spatial and channel dimensions, it enhances the capture of local details and improves the modeling of global contextual information, thereby achieving better feature representation and performance improvement.

[0079] 2. This invention introduces Gumbel downsampling to replace the traditional FPS for point cloud downsampling to obtain keypoints, making the downsampling process both efficient and robust. This is then integrated with the text fusion module, using a gating system to select the most representative and discriminative keypoint features, laying a solid foundation for subsequent refinement operations. Notably, the enhanced extended voxel set abstraction module fuses features of various scales, specifically aggregating 2D bird's-eye view features and keypoint features obtained through Gumbel downsampling and text fusion. This allows the 3D object detection method to learn a richer feature representation, improving 3D object detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is a flowchart of a 3D object detection method that combines attention mechanism and voxel feature aggregation;

[0081] Figure 2 This is the overall structural principle diagram of the 3D object detection method that combines the attention mechanism with voxel feature aggregation;

[0082] Figure 3 This is a diagram of the working principle of the spatial and channel coordinated attention mechanism;

[0083] Figure 4 This is a diagram of the working principle of Gumbel downsampling;

[0084] Figure 5 This is the working principle diagram of the text fusion module;

[0085] Figure 6 This is an example of the qualitative results of the present invention on the KITTI validation set;

[0086] Figure 7 This is the average precision result of car detection of different models at 40 recall positions on the KITTI validation set;

[0087] Figure 8 This is the average precision result of car detection of different models at 11 recall positions on the KITTI validation set;

[0088] Figure 9 This is a graph showing the average accuracy of vehicle detection during the ablation experiment conducted by the present invention. DETAILED DESCRIPTION

[0089] In order to make the purpose, technical solutions and advantages of this application more clear, this application is further described in detail below with reference to the accompanying drawings and embodiments.

[0090] The three-dimensional target detection method combining attention mechanism and voxel feature aggregation described in this embodiment obtains a predicted bounding box by inputting a point set with color and coordinate information, thereby achieving more accurate 3D target detection. Specifically, Figures 1 to 2 As shown, implementing the method includes the following steps:

[0091] Step 1: Obtain the 3D point cloud data of the image to be detected.

[0092] This example uses the KITTI dataset and 3D point cloud data captured by a Velodyne HDL-64E lidar. It is divided into training, validation, and test sets. Point cloud data preprocessing is performed, including filtering and cropping the sparse, disordered 3D point cloud captured by the original lidar, limiting the processing scope and enhancing data quality. Finally, the point cloud data is read.

[0093] Step 2: voxelize the 3D point cloud data and use a sparse convolutional network to extract the voxel features of the voxelized 3D point cloud data;

[0094] Furthermore, the steps of voxelizing the three-dimensional point cloud data and extracting voxel features of the voxelized three-dimensional point cloud data using a sparse convolutional network include:

[0095] The point P in the 3D point cloud is divided into voxels with a spatial resolution of L×W×H. The features of non-empty voxels are directly calculated as the average of the point-by-point features of all internal points; where L, W, and H represent the length, width, and height of the voxel, respectively.

[0096] The L×W×H voxel features are downsampled using a 3×3×3 3D sparse convolution to generate a 3D voxel feature with a spatial resolution of 1 / n of the original, where n is the downsampling multiple.

[0097] In this example, n takes the values ​​of 1, 2, 4, and 8, which means that the point cloud is gradually converted into 3D voxel features with downsampling sizes of 1×, 2×, 4×, and 8×, respectively.

[0098] Step 3: Project the voxel features onto the bird’s-eye view representation to generate a bird’s-eye view image.

[0099] Furthermore, the steps of projecting the voxel features onto the bird's-eye view representation to generate the bird's-eye view include:

[0100] The 3D voxel features obtained after n-fold downsampling are stacked along the height direction, and the stacked 3D voxel features are spliced ​​in the channel dimension to generate a 2D bird's-eye view. , the size of the bird's-eye view is recorded as (b, C′, H, W), where b represents the batch size and C′ represents the number of channels after stacking along the height direction.

[0101] In step 4, spatial and channel collaborative attention mechanism is used to extract enhanced features of the bird’s-eye view.

[0102] Combine Figure 3 ,Furthermore, the steps of extracting enhanced features of the bird’s-eye ,view image by adopting the spatial and channel collaborative attention mechanism include:

[0103] The bird's-eye view image is decoupled from the space and channel, and global average pooling is performed along the height H and width W directions respectively. Figure 3 With height H as the Y direction and width W as the X direction, two one-dimensional feature sequences will be generated. and , the formulas are:

[0104] ,

[0105] ,

[0106] Where, Represents the input bird's-eye view, that is, the two-dimensional bird's-eye view obtained in step 3 ; represents the global average pooling operation;

[0107] One-dimensional feature sequence and Divide into K sub-features of equal size and , each sub-feature map contains a different number of channels, which effectively separates information at different semantic levels and avoids mutual interference between features. The formulas are:

[0108] ,

[0109] ,

[0110] Where C represents the number of channels, Indicates the The sequence number of the sub-feature map, ;

[0111] The multi-scale depth-separable one-dimensional convolution is used to model different sub-features with different-sized convolution kernels to extract multi-semantic spatial structure information. Small convolution kernels are used to capture local detail information, while large convolution kernels are used to capture global information, thus obtaining the first Sub-feature graph and , the formulas are:

[0112] ,

[0113] ,

[0114] Where, represents depthwise separable convolution, Indicates that it is used in The convolution kernel of each sub-feature is set to 3, 5, 7, and 9. The sub-feature maps in the height direction and width direction are grouped and normalized to prevent the interference of semantic information and ensure that each sub-feature map can independently extract meaningful spatial information. The sub-feature maps are spliced ​​to generate a new feature map, and the spatial attention feature map is generated through the Sigmoid activation function. and , to adjust the input features in a weighted manner, highlight important spatial areas, and suppress redundant information. The formulas are:

[0115] ,

[0116] ,

[0117] Where, represents the Sigmoid function, and Respectively indicate along and The group normalization operation of the dimension, Represents a splicing operation;

[0118] The spatial attention feature map and Applied to the original bird's-eye view On the top, obtain the spatial enhanced feature map , the formula is:

[0119] ,

[0120] Where, Represents element-wise multiplication;

[0121] The feature map is progressively compressed and the spatial enhancement feature map is transformed into the spatial enhancement feature map through global average pooling. Compress to resolution size 8 Feature map of 8 , in order to reduce the computational complexity while retaining the key spatial and channel information, the formula is:

[0122] ,

[0123] Where, represents global average pooling, Respectively represent the compressed sizes of H and W;

[0124] The compressed feature map Enter the channel self-attention mechanism to generate query Q, key K and value V. The formulas are:

[0125] ,

[0126] ,

[0127] ,

[0128] Where, Represents depthwise separable convolutions of size 1×1 on Q, K, and V respectively;

[0129] By calculating the dot product of Q and K and using Softmax normalization to obtain channel attention, it is then used for weighted values ​​and Softmax normalization is used to generate similarity weights between channels. , these weights reflect the degree of correlation between channels, and the formula is:

[0130] ,

[0131] The generated similarity weight With the original space characteristics Multiply to obtain enhanced channel features, and use pooling operation to aggregate similarity weights Then apply Sigmoid activation to obtain the enhanced feature map of the final bird's-eye view , the formula is:

[0132] ,

[0133] Where Pool represents the pooling operation.

[0134] Step 5: Downsample the original 3D point cloud data to obtain key points, perform secondary optimization on the key points, and converge the multi-level features to the key points.

[0135] Gumbel downsampling is used to sample the original points, so that the geometric structure information is maintained and redundant points are reduced while obtaining the key points. Then the text fusion module is used to perform secondary optimization on the key points to converge the multi-level features to the key points.

[0136] Combine Figures 4 and 5 ,Furthermore, the original 3D point cloud data is downsampled to obtain key points, and the key points are ,secondarily optimized to converge the multi-level features to the key points.,The steps include:

[0137] Step 501: convert the point cloud dataset into Input to Gumbel, where is the number of input points, is the feature dimension of each point, the goal is to Select a representative subset ,in is the number of points sampled in the subset; a sampling weight is assigned to each input point, and the weight is obtained by the point cloud features and a learnable weight matrix To calculate, the formula is:

[0138] ,

[0139] ,

[0140] in, represents the sampling weight of the j-th point, is a learnable weight matrix, represents the sampling score of the jth point, k represents the kth sampling point, is the output of soft sampling, T represents transpose;

[0141] Step 502: During training, Gumbel-Softmax sampling generates continuous soft sampling outputs so that the model can be optimized through back propagation. The formula for the sampling process is:

[0142] ,

[0143] Where, Represents the sampling result obtained by Gumbel-Softmax; is the sampling probability fraction of the point, is the noise sampled from a Gumbel distribution, is the annealing temperature parameter;

[0144] Early in training, is larger, making the sampling point distribution smoother, that is, soft sampling; as the training progresses, Gradually decreases, approaching discrete sampling; in the test phase, Gumbel-Softmax sampling degenerates into Gumbel-Max sampling, that is, hard sampling is generated and the point with the maximum weight is selected;

[0145] After soft sampling, key points are obtained , the formula is:

[0146] ,

[0147] Where N represents the number of key point features, Indicates the key point features, which are a feature subset of the original point cloud;

[0148] Step 503: key points Input to the text fusion module, extract point cloud features through the PointNet++ feature extraction network, and obtain feature points , according to the feature points Calculate the modulation characteristic g, the formula is:

[0149] ,

[0150] in, is a learnable weight matrix, is the bias term, σ is the Sigmoid activation function; the calculated modulation feature g will be used to adjust the feature points Perform gating so that the model can select the most representative and discriminative features;

[0151] Use modulation feature g to identify feature points Perform gating processing to obtain the features after context gating , the formula is:

[0152] ,

[0153] Where, is another set of learnable weight matrices.

[0154] In step 6, the enhanced extended voxel set abstraction module is used to aggregate the voxel features and the bird's-eye view image to the corresponding key points to obtain the final features of the image to be detected.

[0155] In step 6, by encoding spatial features from multiple sources, including multi-scale voxel features, key point features, and features from two-dimensional images, the two-dimensional BEV features and the key point features obtained by Gumbel downsampling and text fusion are aggregated.

[0156] Furthermore, the enhanced extended voxel set abstraction module is used to aggregate the voxel features and the bird's-eye view image onto corresponding key points to obtain the final features of the image to be detected. The steps include:

[0157] Step 601: Take each key point Location As the center, a neighborhood search is performed in the voxel feature layer of the mth layer, with a radius of Find the adjacent non-empty voxel features and aggregate the non-empty voxel features;

[0158] Step 602: Use the PointNet deep learning model to identify key points. The neighborhood features of the key points are randomly sampled and nonlinearly transformed, and then the different numbers of neighborhood features are mapped into a fixed-size feature vector through maximum pooling to generate multi-scale features for each key point. , the formula is:

[0159] ,

[0160] Where, It is the maximum pooling in the channel dimension; It is a multi-layer perceptron used to perform nonlinear transformation on voxel features and relative position features; represents random sampling; Indicates different resolution layers, 1×, 2×, 4×, and 8×; and for Layer The eigenvectors and 3D coordinates of the voxels; It is The layer radius determines the size of the neighborhood at different scales. 、 Respectively represent The coordinates and feature sets of all non-empty voxels in the layer;

[0161] Step 603: fuse all multi-scale features to form 3D multi-scale features ;

[0162] Step 604: combine the bird's-eye view image, the enhanced features of the bird's-eye view image, and the features after context gating. , aggregated to multi-scale features to obtain the final feature F, the formula is:

[0163] .

[0164] Step 7: Refine the 3D proposal to obtain the predicted bounding box.

[0165] Through 3D proposal refinement, the features of the region of interest are extracted from the preliminary proposal box, multi-source information is integrated, and regression and classification are performed to finally obtain a predicted 3D bounding box with more accurate location and clearer category.

[0166] To further illustrate the effectiveness and excellence of the three-dimensional target detection method described in the present invention, the following examples are used to illustrate. The KITTI dataset is used as the target data to be detected. The present invention randomly selects four different scenes from the KITTI dataset for experiments. The target detection results are shown in the figure below. Figure 6 As shown, it is shown that the method of the present invention can obtain a predicted three-dimensional bounding box with more accurate position and clearer category.

[0167] Figure 7 The segmentation effect of the model was verified by using the KITTI dataset to train a variety of existing models and verify the model on the official test set, and the average precision of 40 recall positions was calculated. Figure 7 It can be seen that the method described in the present invention shows better performance than other baseline methods, especially in the medium and difficult car categories, achieving accuracies of 82.03% and 77.13% respectively.

[0168] Figure 8 The effects of different models were verified on the car category of the KITTI validation set, and the average precision of 11 recall positions was calculated. Figure 8 It can be seen that the present invention achieved an accuracy rate of 89.14%, 85.74% and 79.01% at the three difficulty levels of easy, medium and difficult, respectively; in addition, compared with the existing methods, the present invention improved by 2.05% and 0.31% at the medium and difficult difficulty levels, respectively.

[0169] Figure 9It is an ablation experiment of the performance of each submodule added. The spatial and channel collaborative attention mechanism in step 4 is recorded as SCSA, the implementation process of using Gumbel downsampling to sample the original points in step 5 and using the text fusion module to perform secondary optimization on the key points is recorded as GSSCF, and the extended voxel set abstraction module in step 6 is recorded as EEVSA. Figure 9 As can be seen, each submodule significantly enhances baseline performance, particularly at medium and hard difficulty levels. For example, with the addition of EEVSA, the mean average precision of our method increased by 0.3%. With the addition of SCSA and GSSCF, EEVSA fuses features from both modules, further enhancing our method's object detection performance. The final mean average precision increased by an additional 0.89% compared to EEVSA.

[0170] In summary, the three-dimensional target detection method combining the attention mechanism and voxel feature aggregation described in the present invention achieves more accurate and efficient detection of three-dimensional targets compared to the original three-dimensional target detection method.

Claims

1. A 3D object detection method combining attention mechanism and voxel feature aggregation, characterized by: The following steps are involved: Obtain the three-dimensional point cloud data of the image to be detected; The 3D point cloud data is voxelized and the voxel features of the voxelized 3D point cloud data are extracted using a sparse convolutional network. Project the voxel features onto the bird's-eye view representation to generate a bird's-eye view image; Adopt spatial and channel-wise co-attention mechanism to extract enhanced features of bird’s-eye view; Downsample the original 3D point cloud data to obtain key points, perform secondary optimization on the key points, and converge multi-level features to the key points; Using the enhanced extended voxel set abstraction module, voxel features and bird's-eye view images are aggregated to corresponding key points to obtain the final features of the image to be detected; Through 3D proposal refinement, the predicted bounding box is obtained; The steps of downsampling the original 3D point cloud data to obtain key points, performing secondary optimization on the key points, and converging the multi-level features to the key points include: Step 501: Input point cloud dataset is ,in is the number of input points, is the feature dimension of each point, the goal is to Select a representative subset ,in is the number of points sampled in the subset; a sampling weight is assigned to each input point, and the weight is obtained by the point cloud features and a learnable weight matrix To calculate, the formula is: ， ， in, represents the sampling weight of the j-th point, is a learnable weight matrix, represents the sampling score of the jth point, k represents the kth sampling point, is the output of soft sampling, T represents transpose; Step 502: During training, Gumbel-Softmax sampling generates continuous soft sampling outputs, as follows: ， Where, Represents the sampling result obtained by Gumbel-Softmax; is the sampling probability fraction of the point, is the noise sampled from a Gumbel distribution, is the annealing temperature parameter; After soft sampling, key points are obtained , the formula is: ， Where N represents the number of key point features, Indicates the key point features, which are a feature subset of the original point cloud; Step 503: key points Input to the text fusion module, extract point cloud features through the PointNet++ feature extraction network, and obtain feature points , according to the feature points Calculate the modulation characteristic g, the formula is: ， in, is a learnable weight matrix, is the bias term, σ is the Sigmoid activation function; Use modulation feature g to identify feature points Perform gating processing to obtain the features after context gating , the formula is: ， Where, is another set of learnable weight matrices; Using the enhanced extended voxel set abstraction module, the voxel features and the bird's-eye view image are aggregated to the corresponding key points to obtain the final features of the image to be detected. The steps include: Step 601: Take each key point Location As the center, a neighborhood search is performed in the voxel feature layer of the mth layer, with a radius of Find the adjacent non-empty voxel features and aggregate the non-empty voxel features; Step 602: Use the PointNet deep learning model to identify key points. The neighborhood features of the key points are randomly sampled and nonlinearly transformed, and then the different numbers of neighborhood features are mapped into a fixed-size feature vector through maximum pooling to generate multi-scale features for each key point. , the formula is: ， Where, It is the maximum pooling in the channel dimension; It is a multi-layer perceptron used to perform nonlinear transformation on voxel features and relative position features; represents random sampling; Indicates different resolution layers; and for Layer The eigenvectors and 3D coordinates of the voxels; It is Layer radius, 、 Respectively represent The coordinates and feature sets of all non-empty voxels in the layer; Step 603: fuse all multi-scale features to form 3D multi-scale features ; Step 604: combine the bird's-eye view image, the enhanced features of the bird's-eye view image, and the features after context gating. , aggregated to multi-scale features to obtain the final feature F, the formula is: ， Where, A two-dimensional bird's-eye view. It is an enhanced feature map of the bird's-eye view.

2. The three-dimensional object detection method combining attention mechanism and voxel feature aggregation according to claim 1 is characterized in that: The steps of voxelizing the 3D point cloud data and extracting the voxel features of the voxelized 3D point cloud data using a sparse convolutional network include: Divide the point P in the 3D point cloud into voxels with a spatial resolution of L×W×H, where L, W, and H represent the length, width, and height of the voxel respectively; 3D sparse convolution is used to downsample the L×W×H voxel features to generate three-dimensional voxel features with a spatial resolution of 1 / n of the original, where n is the downsampling multiple.

3. The three-dimensional object detection method combining attention mechanism and voxel feature aggregation according to claim 2, characterized in that: The steps of projecting voxel features onto the bird's-eye view representation to generate a bird's-eye view include: The 3D voxel features obtained after n-fold downsampling are stacked along the height direction, and the stacked 3D voxel features are spliced ​​in the channel dimension to generate a 2D bird's-eye view. , the size of the bird's-eye view is recorded as (b, C′, H, W), where b represents the batch size and C′ represents the number of channels after stacking along the height direction.

4. The three-dimensional object detection method combining attention mechanism and voxel feature aggregation according to claim 3, characterized in that: The steps of extracting enhanced features of the bird's-eye view using the spatial and channel-wise co-attention mechanism include: The bird's-eye view image is decoupled from space and channels, and global average pooling is performed along the height H and width W directions to generate two one-dimensional feature sequences. and , the formulas are: ， ， Where, represents a bird's-eye view of the input, i.e. ; represents the global average pooling operation; One-dimensional feature sequence and Divide into K sub-features of equal size and , each sub-feature map contains a different number of channels, and the formulas are: ， ， Where C represents the number of channels, Indicates the The sequence number of the sub-feature map, ; Multi-scale depth-separable one-dimensional convolution is used to model different sub-features, extract multi-semantic spatial structure information, and obtain the first Sub-feature graph and , the formulas are: ， ， Where, represents depthwise separable convolution, Indicates that it is used in Convolution kernel of sub-features; Each sub-feature map in the height direction and width direction is grouped and normalized respectively, and the spatial attention feature map is generated by the Sigmoid activation function and , the formulas are: ， ， Where, represents the Sigmoid function, and Respectively indicate along and The group normalization operation of the dimension, Represents a splicing operation; The spatial attention feature map and Applied to the original bird's-eye view On the top, obtain the spatial enhanced feature map , the formula is: ， Where, Represents element-wise multiplication; The spatially enhanced feature map is pooled using global average pooling. Feature maps compressed to a fixed resolution , the formula is: ， Where, represents global average pooling, Respectively represent the compressed sizes of H and W; The compressed feature map Enter the channel self-attention mechanism to generate query Q, key K and value V. The formulas are: ， ， ， Where, Represents depthwise separable convolutions of size 1×1 on Q, K, and V respectively; By calculating the dot product of Q and K and using Softmax normalization to obtain channel attention, it is then used for weighted values ​​and Softmax normalization is used to generate similarity weights between channels. , the formula is: ， The generated similarity weight With the original space characteristics Multiply to obtain enhanced channel features, and use pooling operation to aggregate similarity weights Then apply Sigmoid activation to obtain the enhanced feature map of the final bird's-eye view , the formula is: ， Where Pool represents the pooling operation.

Images