A 3D target detection method, device and equipment using sparse voxel fusion and a storage medium

Abstract

The application relates to a 3D target detection method, device and equipment adopting sparse voxel fusion and a storage medium, wherein the method comprises the following steps: simultaneously performing cubical voxelization and columnar voxelization on collected point clouds to obtain foreground cubical voxels and columnar voxels; calculating columnar voxel centers and foreground cubical voxel centers and extracting voxel centers as voxel features; fusing the features of the foreground cubical voxels and the columnar voxel features to obtain first-stage enhanced cubical voxel features; selecting a cubical voxel of an object center as a center voxel, obtaining a center voxel feature according to the center voxel, and aggregating the center voxel feature and the first-stage enhanced cubical voxel features to obtain second-stage enhanced cubical voxels; and sending the second-stage enhanced cubical voxels into a 3D target detector to realize target detection. Compared with the prior art, the application has the advantages of effectively detecting long-distance and small targets and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision and 3D target detection, and in particular to a 3D target detection method, device and equipment using sparse voxel fusion and a storage medium. BACKGROUND

[0002] 3D target detection is an indispensable core component of the automatic driving environment perception system, and also has wide application in the field of robots. LiDAR point cloud can provide more depth and geometric spatial structure information about the target. The existing technology performs cubic voxelization on the point cloud based on voxel method, and then performs 3D sparse convolution on the voxel to extract the point cloud features. However, most LiDAR-based methods focus on the vehicle or traffic tool category which has a relatively large size and dense point cloud, and have not achieved satisfactory detection performance on difficult-to-detect samples such as small-sized target bicycles and pedestrians which have more sparse point clouds and are far away.

[0003] The existing voxel-based method performs cubic voxelization on the point cloud, and then performs 3D sparse convolution on the voxel to extract the point cloud features. However, due to the inherent sparsity and varying density of LiDAR point cloud, the detector using the traditional cubic voxel division method will generate more empty voxels (for example, the non-empty voxels account for less than 0.1% in a frame of WOD point cloud), which makes the target represented by the point cloud incomplete and lacks object-level information, thereby bringing challenges to 3D target detection. In addition, the uneven distribution of points in voxels caused by cubic voxelization will inevitably introduce additional computational overhead.

[0004] In the prior art, the Chinese patent with publication number CN 114743079 A discloses a 3D target detection method and device using sparse radar and binocular stereo image fusion, which proposes an efficient and fast image detection problem, but does not mention the detection and solution of long-distance and small-size targets. In general, there are two problems in the prior art: 1) the traditional single cubic voxelization has the shortcomings of causing the loss of local information of the object and the limited receptive field of convolution cannot capture rich context information, and the traditional LiDAR detector using only cubic voxels is not accurate in detecting small targets at a long distance and information is lost; 2) the method using multiple point cloud representations for fusion in the prior art also uses an expensive voxel encoder before the 3D backbone network, which increases the time and space overhead, and also introduces background point noise from the cross-view angle, which interferes with the detection. SUMMARY

[0005] The present application aims to overcome the defects of the prior art, and provides a 3D target detection method, device, equipment and storage medium adopting sparse voxel fusion, which fuses columnar voxels and cubic voxels by adopting the sparse voxel fusion method, obtains more information features, and realizes more accurate target detection of long-distance and small targets.

[0006] The object of the present application can be achieved by the following technical solutions:

[0007] In a first aspect, the present application provides a 3D target detection method adopting sparse voxel fusion, comprising the following steps:

[0008] S1, simultaneously performing cubic voxelization and columnar voxelization on the collected point cloud to obtain foreground cubic voxels and columnar voxels;

[0009] S2, calculating the columnar voxel center and the foreground cubic voxel center according to the foreground cubic voxels and the columnar voxels, and extracting the two kinds of voxel centers as two kinds of voxel features;

[0010] S3, selecting part of the foreground cubic voxel centers to query in all columnar voxel centers, and fusing the features of the foreground cubic voxels and the columnar voxel features to obtain the first stage enhanced cubic voxel features and form the first stage enhanced cubic voxels;

[0011] S4, selecting the cubic voxel of the object center as the center voxel, obtaining the center voxel features according to the center voxel, aggregating the center voxel features and the first stage enhanced cubic voxel features to obtain the second stage enhanced cubic voxel features and the second stage enhanced cubic voxels;

[0012] S5, sending the second stage enhanced cubic voxels into the 3D target detector to realize target detection.

[0013] Further, in the step S1, the columnar voxelization is specifically implemented by distance-aware dynamic columnar voxelization, which includes: converting from Cartesian coordinate system representation to cylindrical coordinate system representation, and dividing the voxels in the cylindrical coordinate system in the direction of the column coordinate ρ according to different interval distances.

[0014] Further, in the step S2, the specific steps of calculating the two kinds of voxel centers include: after obtaining the foreground cubic voxels and the columnar voxels, selecting the average spatial coordinates of the points in the voxels to calculate the two kinds of voxel centers.

[0015] Further, the step S3 specifically includes the following steps:

[0016] S31, obtaining a corresponding columnar centroid voxel and a foreground cubic centroid voxel according to the two kinds of voxel centroids, obtaining a foreground prediction score of the foreground cubic centroid voxel through a layer of 3D sparse convolution, ranking according to the foreground prediction score, and selecting the top k foreground cubic centroid voxels as the query source from high to low;

[0017] S32, performing a multi-scale grouping sphere query in the columnar centroid voxel set by taking the two kinds of voxel centroids as query media and the foreground cubic centroid voxel as the query source;

[0018] S33, fusing the columnar centroid voxel features and the foreground cubic centroid voxel features to obtain first-stage fused features and obtain first-stage enhanced cubic voxels.

[0019] Further, the step S4 specifically comprises the following steps:

[0020] S41, generating a 3D sparse heat map by using the foreground cubic voxel centroid;

[0021] S42, selecting a cubic voxel at the center of the object as a center voxel according to the 3D sparse heat map;

[0022] S43, inputting the center voxel into a Transformer module to output the center voxel features;

[0023] S44, mapping the center voxel features back into the first-stage enhanced cubic voxel features to obtain second-stage enhanced cubic voxel features and second-stage enhanced cubic voxels.

[0024] Further, in the step S41, the specific steps of generating the 3D sparse heat map are as follows:

[0025] S411, selecting the foreground cubic voxel centroid, and labeling a bounding box with each foreground cubic voxel as the center of the cubic coordinate system;

[0026] S412, calculating the distance between the center of the bounding box and the foreground cubic voxel centroid;

[0027] S413, based on the distance, applying a 3D Gaussian kernel to limit the numerical range response to the range of 0 to 1 to generate a 3D sparse heat map.

[0028] Further, in the step S42, the cubic voxel at the center of the object is selected as the center voxel by selecting the top k cubic voxels based on the 3D sparse heat map prediction score.

[0029] In a second aspect, the present application provides a 3D object detection device using voxel fusion, which comprises three modules: a dynamic columnar voxelization module, a voxel selection-query-fusion module, and a center voxel Transformer module; the device is used to implement any one of the above-mentioned 3D object detection methods using sparse voxel fusion.

[0030] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the above-mentioned 3D object detection methods using sparse voxel fusion when executing the computer program.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement any one of the above-mentioned 3D object detection methods using sparse voxel fusion.

[0032] Compared with the prior art, the present application has the following beneficial effects:

[0033] 1) The present application proposes to use a uniform and reasonable point distribution pattern in the columnar voxel to voxelize the point cloud in a manner that follows the point cloud rotation scanning rule more closely. In places farther away and with sparser point clouds, the voxel is larger. Therefore, this voxel representation can maintain the spatial structure characteristics of the target in the point cloud, thereby constructing a more compact voxel feature representation, maximizing the use of each point without discarding or supplementing points, minimizing information quantization loss, expanding the detection field of view of the 3D object detector, and effectively detecting distant and small targets.

[0034] 2) The present application uses a uniform and reasonable point distribution pattern in the columnar voxel to selectively locally fuse voxel features of two different representations to form enhanced cubic voxels, and uses the object center as a guide to select voxels near the object center for long-range object-level global context information aggregation. The detection effect of small targets is greatly improved, avoiding other processing calculations related to space and time, without using expensive voxel encoders, and also avoiding the influence of empty voxels on detection. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The figure is a flowchart of the method of the present application;

[0036] Figure 2 The figure is a schematic diagram of the overall architecture of the present application;

[0037] Figure 3 The figure is a schematic diagram of conventional columnar voxelization and distance-aware dynamic columnar voxelization (DDCV) proposed by the present application;

[0038] Figure 4 A flowchart of a foreground-centric voxel selection-query-fusion (FCVSQF) step of the present application;

[0039] Figure 5 A flowchart of an object-centric voxel Transformer (OCVT) step of the present application. DETAILED DESCRIPTION

[0040] The present application will be described in detail below with reference to the drawings and specific embodiments. The embodiments are implemented on the premise of the technical solutions of the present application, and detailed implementation methods and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0041] The present application innovatively proposes a 3D target detection method using voxel fusion, which fully utilizes the complementary information between different representations of LiDAR point clouds. Based on this method, a simple and efficient 3D target detection device can also be made.

[0042] In a first aspect, as Figure 1 shown, the present application discloses a 3D target detection method using sparse voxel fusion, which is realized by the following process:

[0043] Step 1, the collected point cloud is simultaneously cubed and columned to obtain foreground cuboid voxels and column voxels.

[0044] This embodiment is realized by the following way: taking the point cloud scanned by the laser radar of an autonomous vehicle as input, and simultaneously performing columnar and cubic voxelization on the point cloud, wherein the columnar voxelization is realized by dynamic columnar voxelization based on distance perception. Dynamic voxelization does not need to hard-code the number of points in each voxel, thereby maximizing the use of each point without discarding or supplementing points, and minimizing the information quantization loss.

[0045] In order to maintain the three-dimensional geometric structure of the object in the point cloud, adapt to the rotating scanning mode of the laser radar and the change of the sparsity of the point cloud, the present application proposes a dynamic distance perception columnar voxelization, which converts the point cloud from Cartesian coordinates to cylindrical coordinates and divides the voxels along the axis non-uniformly without discarding any points. Dynamic voxelization means that points are assigned to voxel space, and the number of points within the grid is dynamically adjusted according to their spatial coordinates.

[0046] Figure 3 A flowchart of conventional columnar voxelization and distance perception dynamic columnar voxelization (DDCV) proposed by the present application. Referring to Figure 3 , given a frame of point cloud defined in Cartesian coordinates , then its cylindrical coordinate representation is:

[0047]

[0048] where N denotes the number of points in a frame of point cloud.

[0049] As for the cylindrical point set , the voxel set , voxelization can be described as a bidirectional mapping between points and voxels:

[0050]

[0051] where M is the number of non-empty voxels, is the mapping from points to voxels, is the mapping from voxels to points.

[0052] Distance-aware cylindrical voxelization involves uneven division along the rho axis in the cylindrical coordinate system along different rho intervals. The farther away from the origin (O point in LiDAR, the sparser the points, the larger the voxel, which can contain more points, as shown in Figure 3 (b). Figure 3

[0053] Define the voxel size as , discuss the case,

[0054]

[0055] wherein, , the present application regards as a close range, as a medium distance range, as a long distance range.

[0056] Step 2, according to Figure 2 , calculate the cylindrical voxel centroid and the foreground cubic voxel centroid according to the foreground cubic voxel and the cylindrical voxel, and extract the two kinds of voxel centroids as two kinds of voxel features.

[0057] This embodiment is realized in the following way: in order to maintain the original 3D geometry and more accurately represent the voxel features, the present application uses the centroid of the points within each voxel in the two representations as the query source and target through the foreground cubic voxel centroid-based voxel selection-query-fusion (FCVSQF) module, and obtains the voxel index and features corresponding to the centroid.

[0058] In the prior art, the voxel center is selected to represent the position of each voxel feature, but this representation treats voxels with different point distributions equally, inevitably leading to misleading information and ignoring important geometric information. For example, Figure 4 ​As shown, based on the voxel selection-query-fusion (FCVSQF) module of the centroid of the foreground voxel, from the experimental results of the two models of PDV and KPConv, the application selects the voxel centroid as the representative position of the voxel feature to obtain more accurate feature query, in addition, the voxel centroid is also more suitable for the center voxel Transformer (DDCV) module proposed by the application guided by the object center, fully following the distribution law of the midpoint of each voxel.

[0059] Assuming a set of cylindrical voxels , a set of cubic voxels , for each voxel representation, I is the index of the voxel, and F is the corresponding voxel feature, taking the set of cylindrical voxels as an example, the voxel centroid is calculated by taking the average spatial coordinates of the in-point of the voxel. Specifically, for the cylindrical voxel , the centroid is

[0060]

[0061] , is the number of in-points of the voxel . For the set of cubic voxels , the application can calculate the cylindrical voxel centroid in the above manner.

[0062] The steps of centroid voxel feature extraction are as follows: after obtaining the voxel centroids of the two different representations, the centroid and the corresponding voxel (centroid voxel) index of a certain scale s sparse feature map obtained by 3D sparse convolution are searched through scaling and grouping operations, and then indexed to the corresponding voxel feature , which is specifically represented as

[0063]

[0064] , wherein represents scaling (Scale), represents grouping (Group), represents searching (Search); , cub represents cubic voxel, and cyl represents cylindrical voxel; represents the initial voxel index, represents the initial voxel centroid coordinates, represents the scaling multiple of each scale s of the 3D sparse feature map in the downsampling stage, .

[0065] Step 3, select part of the foreground cuboid voxel quality center in all columnar voxel quality center to query, and fuse the features of the foreground cuboid voxel and the columnar voxel features to obtain the first stage enhanced cuboid voxel features and form the first stage enhanced cuboid voxel.

[0066] The embodiment is realized by the following manner: in order to save memory and prevent the introduction of background voxel noise, the application only selects part of important foreground cuboid voxel quality center to query and enhance in all columnar voxel quality center, and the foreground cuboid voxel after feature enhancement in the FCVSQF module, that is, the first stage enhanced cuboid voxel.

[0067] The application obtains the first stage enhanced cuboid voxel, which specifically follows three steps: first, foreground cuboid center voxel selection, second, cross voxel query, and third, fusion of two kinds of voxels.

[0068] As shown in Figure 4 , the application takes the more commonly used cuboid voxel branch as the main branch and the columnar voxel branch as the auxiliary branch.

[0069] First, part of the important foreground cuboid voxel quality center is selected, and the important foreground cuboid voxel quality center is selected by the foreground prediction score. The foreground cuboid center voxel is obtained by one layer of 3D sparse convolution to obtain the foreground prediction score, and the top-k foreground cuboid center voxel is selected as the query source according to the foreground score, wherein the top-k is selected from the front 1024 according to the number of data scale, the data scale can be selected from the front 2048, for example, the KITTI dataset selects the front 1024, and the Waymo Open Dataset (WOD) dataset selects 2048.

[0070] Then, multi-scale group ball query (MSG ball-query) is performed in the columnar center voxel based on the related center; this allows the application to gather columnar voxel features in a local range and provide more fine-grained geometric information.

[0071] Finally, the aggregated columnar center voxel features are fused with the foreground cuboid center voxel features to obtain refined cuboid voxel features, and the fusion process corresponds and correlates the columnar center voxel features and the foreground cuboid center voxel features by the F fusion function.

[0072] To obtain refined cubic centroid voxel features, selecting foreground centroid voxels for feature aggregation is crucial. This invention focuses on important voxels rather than all voxels, selecting a subset of centroid voxels based on prediction scores. This avoids introducing background noise from columnar centroid voxels and also avoids the common practice in existing algorithms like the VoTr model, which aggregates features from all non-empty voxels. Furthermore, this method expands the effective receptive field while maintaining high efficiency. This process can be represented as:

[0073]

[0074] In the above formula These represent the foreground, pooled, and refined elements, respectively. It represents selection, query, and fusion.

[0075] Step 4: Select the cube element at the center of the object as the central voxel, obtain the central voxel feature based on the central voxel, and aggregate the central voxel feature with the cube element feature enhanced in the first stage to obtain the cube element feature enhanced in the second stage, and obtain the cube element enhanced in the second stage.

[0076] This embodiment is implemented in the following way: (e.g.) Figure 5 As shown, to further aggregate global and capture long-range object-level contextual information, this invention proposes a Center Voxel Transformer (OCVT) module guided by the object center. This module focuses on voxels around the object center, selecting center voxels for long-range object-level global contextual information aggregation. The first-stage enhanced cube voxels obtain first-stage enhanced cube voxel features, namely fine-grained foreground cubic centroid voxel features, and are labeled as distinguishable features. These include fine-grained features from another more information-rich columnar voxel representation, partially compensating for the lack of object information in the cube voxel representation. However, due to the independent feature aggregation between voxels, they may lack information interaction. Furthermore, to detect small and distant targets, incorporating global information into the features is crucial. This invention aggregates the center voxels with the first-stage enhanced cube voxels based on the OCVT module to obtain the second-stage enhanced cube voxels.

[0077] Using the foreground cube mass center selected in the FCVSQF module in the previous step Given each Centered bounding box ,calculate The center and Central center of mass The distance between them is calculated as the distance between the center of the bounding box and the center of the foreground cube. Then, a 3D Gaussian kernel is applied to constrain the heatmap response to the range [0,1], generating a 3D sparse heatmap. Formally, this can be represented as:

[0078]

[0079] in It is an adaptive standard deviation of the object's bounding box size. This is the heatmap value generated at centroid i. Taking all centroids yields the final target 3D sparse heatmap. By comparing with the predicted heatmap By calculating the loss, this invention can more effectively focus on the foreground cubic centroid voxel (center voxel) near the center of the object.

[0080] This invention focuses on the subset of foreground cubic centroid voxels closest to the object's center to establish object-level context dependencies, thereby improving efficiency. The selection of the foreground cubic centroid voxel closest to the object's center and... Figure 4 Similar to the selection process, the top k voxels based on the 3D predicted heatmap scores are selected as central voxels. This invention represents the central voxel features as... The central centroid is represented as Then input them into the Transformer encoder block:

[0081]

[0082]

[0083] in Representing Transformer, Q, K, and V represent query, key, and value, This represents the positional encoding based on the centroid. Finally, the centroid voxel features output by the Transformer module are mapped back to the original 3D sparse feature map, i.e., the output features of the 3D sparse convolution are fused with the cube voxel features enhanced in the first stage to obtain the final second-stage enhanced cube voxel features. This second-stage enhanced cube voxel simultaneously possesses local features from the column voxels and global contextual dependencies of the object's length range from the object's centroid voxel.

[0084] Step 5: The enhanced cubic pixels from the second stage are fed into the 3D target detector to achieve target detection. This invention is applicable to most voxel-based 3D target detection devices, whether anchor-frame based or center-point based, and whether it is a one-stage or two-stage 3D target detection device. It can be used as a plug-and-play module to achieve more accurate target detection.

[0085] In another aspect, the present application discloses a 3D target detection device using voxel fusion, which comprises three components: a dynamic columnar voxelization module, a voxel selection-query-fusion module, and a center voxel Transformer module. The dynamic columnar voxelization module is used for distance perception, the voxel selection-query-fusion module is used for selection-query-fusion of foreground centroid voxels, and the center voxel Transformer module focuses on the voxels of the object center. The device is used to implement a 3D target detection method using voxel fusion. Due to the use of sparse voxel fusion, the device can be plug-and-play and can be applied to most voxel-based 3D target detectors. As a method rarely explored in the field of 3D detection, the present application uses columnar voxels with more uniform point cloud distribution to make up for the shortcomings of traditional cubic voxels, providing more abundant information for detection. The query and local fusion based on the centroid cross-voxel representation of the present application to some extent improves the problem of incomplete object information representation of cubic voxels, and fuses fine-grained features. At the same time, the object-level global information learning further refines the feature representation, which is beneficial to subsequent detection.

[0086] In a third aspect, the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the 3D target detection method using sparse voxel fusion when executing the computer program.

[0087] In a fourth aspect, the present application further discloses a computer readable storage medium, which stores a computer program that can be used to calculate the 3D target detection method using sparse voxel fusion when the computer program is executed by a processor.

[0088] The present application obtains the following experimental results by performing experiments on the mainstream automatic driving data sets KITTI and Waymo Open Dataset (WOD) respectively, which show the effectiveness of the method of the present application. The open source code toolbox used in the experiment is OpenPCDet.

[0089] 1) Experimental results on KITTI dataset

[0090]

[0091] Table 1. 3D / BEV detection performance (R40) on KITTI validation set, SVF: Sparse Voxel Fusion

[0092]

[0093] Table 2. Comparison of 3D detection performance on KITTI test set car and bicycle hand class

[0094] To verify the effectiveness of the proposed SVFNeXt, experiments are conducted on KITTI and several widely used models SECOND, CenterPoint, PV RCNN are used as baseline models. The experimental results on the validation set and test set of KITTI are reported in Table 1 and Table 2, respectively. On the validation set, the proposed model has a point rise on both 3D and mAP (as 3D detection performance is improved by 1.17%, 1.67%, 3.02%, respectively), especially in the medium difficulty of pedestrians and cyclists, the performance is greatly improved without reducing the detection performance of cars. On the test set, the proposed method can also have competitive results with advanced methods.

[0095] 2) Experimental results obtained on the WOD dataset

[0096]

[0097] Table 3. Comparison of 3D detection performance on WOD validation set, using 20% of the training set for training

[0098]

[0099] Table 4. Comparison of 3D detection performance of advanced methods on WOD validation set, using all the training set for training

[0100] The present application also conducts experiments on a larger scale dataset Waymo, and reports the results on the validation set in Table 3 and Table 4. In order to save time, the present application trains on 20% of the training set and verifies the effectiveness on 4 baseline models. As shown in Table 3, the present application shows consistent performance improvement on all categories as in KITTI, and leads by a large margin on mAPH(L2) on 4 baseline models (improved by 3.55%, 2.15%, 1.56%, 2.63%, respectively), especially on small objects. On 100% of the training data, the experimental results of the present application are compared with advanced methods in Table 4. Taking the simple and efficient VoxelNeXt as the baseline model, the method of the present application achieves the best performance among these detectors.

[0101] The above describes the preferred embodiments of the present application in detail. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the prior art according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A 3D object detection method using sparse voxel fusion, characterized in that, The method comprises the following steps: S1, the collected point cloud is simultaneously cubed and columnar cubed to obtain foreground cubes and columnar cubes; S2, according to the foreground cubes and columnar cubes, the columnar cube center and the foreground cube center are calculated, and the two kinds of cube centers are extracted as two kinds of cube features; S3, part of the foreground cube center is selected for query in all columnar cube centers, and the foreground cube feature and the columnar cube feature are fused to obtain the first stage enhanced cube feature and form the first stage enhanced cube; S4, the cube at the center of the object is selected as the center cube, the center cube feature is obtained according to the center cube, the center cube feature and the first stage enhanced cube feature are aggregated to obtain the second stage enhanced cube feature and the second stage enhanced cube; S5, the second stage enhanced cube is sent into the 3D target detector to realize target detection; The step S3 specifically comprises the following steps: S31, the corresponding columnar center cube and the foreground cube center cube are obtained according to the two kinds of cube centers, the foreground cube center cube is obtained through a layer of 3D sparse convolution to obtain the foreground prediction score, the foreground cube center cube is ranked from high to low according to the foreground prediction score, and the top k foreground cube center cubes are selected as the query source; S32, the two kinds of cube centers are used as the query medium, the foreground cube center cube is used as the query source, and multi-scale grouping ball query is performed in the columnar center cube set; S33, the columnar center cube feature and the foreground cube center cube feature are fused to obtain the first stage fused feature and the first stage enhanced cube; The step S4 specifically comprises the following steps: S41, a 3D sparse heat map is generated by using the foreground cube center; S42, the cube at the center of the object is selected as the center cube according to the 3D sparse heat map; S43, the center cube is input into the Transformer module to output the obtained center cube feature; S44, the center cube feature is mapped back to the first stage enhanced cube feature to obtain the second stage enhanced cube feature and the second stage enhanced cube. 2.The 3D object detection method with sparse voxel fusion of claim 1, wherein, In the step S1, the columnar cubing is specifically realized by distance-aware dynamic columnar cubing, which comprises converting from Cartesian coordinate system representation to cylindrical coordinate system representation, and dividing the cubes in the cylindrical coordinate system in the direction of the column coordinate ρ according to different distance intervals.

3. The 3D object detection method with sparse voxel fusion according to claim 1, wherein, In the step S2, the specific steps of calculating the two kinds of cube centers comprise: after obtaining the foreground cubes and columnar cubes, the average spatial coordinates of the points in the cubes are selected to calculate the two kinds of cube centers.

4. The 3D object detection method with sparse voxel fusion of claim 1, wherein, In the step S41, the specific steps of generating the 3D sparse heat map are as follows: S411, the foreground cube center is selected, and a bounding box is labeled with each foreground cube as the center of the cubic coordinate system; S412, the distance between the center of the bounding box and the foreground cube center is calculated; S413, based on the distance, a 3D Gaussian kernel is applied to limit the numerical range response to the range of 0 to 1 to generate a 3D sparse heat map.

5. The 3D object detection method with sparse voxel fusion according to claim 1, wherein, In the step S42, the center voxel of the object is selected as the center voxel by selecting the top k voxels based on the 3D sparse heat map prediction score.

6. A 3D object detection apparatus using voxel fusion, characterized by, The device comprises three modules: a dynamic columnar voxelization module, a voxel selection-query-fusion module, and a center voxel Transformer module; and the device is used to implement the 3D target detection method with sparse voxel fusion according to any one of claims 1-5.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the 3D target detection method with sparse voxel fusion according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the 3D target detection method with sparse voxel fusion according to any one of claims 1-5.

Images