Voxel-oriented scale-aware feature extraction architecture system and 3D object detection method
The voxel-oriented scale-aware feature extraction architecture combines CNN and Transformer to efficiently extract global and local features from point clouds, addressing computational and information loss issues, enhancing detection accuracy and adaptability in complex environments.
Patent Information
- Application Number
- CN202410957486.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-17
AI Technical Summary
Existing 3D object detection methods face challenges in efficiently extracting global and local information from point clouds due to computational costs, sparsity, and information loss during voxelization, while traditional CNN models struggle with capturing global information and Transformer models face high computational complexity with large point cloud data.
A voxel-oriented scale-aware feature extraction architecture that combines CNN and Transformer, utilizing submanifold convolution and multi-scale dilated attention to process sparse voxel data, with a hybrid backbone that includes voxel fine-grained perception and dilated scale-aware Transformer modules, adaptively adjusting voxel density and attention for efficient feature extraction.
The architecture effectively captures fine-grained and multi-scale features from complex point cloud data, enhancing detection accuracy and adaptability in diverse environments, improving performance in tasks like object detection and semantic segmentation.
Smart Images

Figure CN118506020B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of object perception in autonomous driving, and specifically relates to a voxel-oriented scale perception feature extraction architecture system and a 3D target detection method. Background Art
[0002] Three-dimensional object detection has received increasing attention in the fields of autonomous driving and robotics. Unlike 2D images from cameras, LiDAR's 3D point cloud contains additional depth information, which helps make accurate and fast decisions in perception tasks such as detection, tracking, and segmentation. Although 2D object detection has been intensively studied under the booming development of deep learning, the lack of depth information limits the understanding of the surrounding environment and the location of objects. LiDAR can easily obtain depth information, but there are still problems such as occlusion, sparsity, and uneven data distribution, especially the difficulty in mining sample features in complex scenes.
[0003] How to fully extract the global and local information of point clouds is a top priority. At present, a large number of studies have shown interest in this goal. Therefore, from a data perspective, 3D object detection methods are mainly divided into three categories: point cloud-based solutions, voxel-based solutions, and point cloud voxel interaction-based solutions. Methods such as PointNet++ and PointRCNN directly cluster on the original point cloud, but the computational cost is too expensive, and the sparsity of the point cloud will also lead to information redundancy or loss. CNN has been widely studied in the field of 2D images, but due to the sparsity and unstructured nature of point clouds, it is not suitable for direct application to 3D detection tasks. Some studies have shown that after the point cloud is converted to different forms, CNN and some variant forms can successfully extract 3D features. For example, methods such as VoxelNet and SECOND convert point clouds into voxel representations, which improves computational efficiency and avoids the unstructured nature of point clouds, but it is easy to lose information during the voxelization process. PVRCNN, VIC-Net, etc. fuse voxels and point clouds to alleviate the problem of information loss caused by voxelization. However, the fusion process not only increases the processing time, but also brings data redundancy. If sufficient feature extraction is performed only on pure voxel data, the effect of point cloud data supplementation can be achieved, which can effectively alleviate the above problems.
[0004] When processing point cloud data, traditional CNN models often have difficulty capturing global information due to their local limitations, while pure Transformer models may face the problem of high computational complexity when processing large amounts of point cloud data. Therefore, combining CNN's local feature extraction capabilities with Transformer's global modeling capabilities has become one of the hot research directions in recent years.
[0005] Therefore, how to solve the problem of sufficient feature extraction on voxel features of large-scale, complex and changeable point cloud data, and provide a method to improve efficiency, enhance detail capture capabilities and model generalization capabilities is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention
[0006] The first object of the present invention is to provide a voxel-oriented scale-aware feature extraction architecture to address the problems in the prior art.
[0007] To this end, the above-mentioned purpose of the present invention is achieved through the following technical solutions:
[0008] The voxel-oriented scale-aware feature extraction architecture system is characterized in that: the architecture uses CNN and Transformer in a mixed way, the convolution module VFPM in CNN is placed before the Transformer module DSAT, after feature extraction by VFPM, a data format converter is executed to convert the sparse convolution tensor required for the convolution operation into the sparse tensor required for the voxel transformer operation, down-sampling of empty voxels is achieved through the sparse voxel module, and after passing through the DSAT module, it is connected to the next VFPM module through the reverse data format converter, so as to achieve effective communication of information flow between Transformer and CNN;
[0009] Among them, VFPM uses submanifold convolution as its basic module and only operates on non-empty voxels. In VFPM, submanifold convolution operations are performed by grouping sparse convolution tensors to process sparse data in three-dimensional space;
[0010] In DSAT, multi-scale dilated attention operates on non-empty voxels and performs local and sparse image patch interactions by adopting different dilation rates to achieve multi-scale interactions while ensuring local voxels.
[0011] While adopting the above technical solutions, the present invention may also adopt or combine the following technical solutions:
[0012] As a preferred technical solution of the present invention: after processing voxelization, the voxels in the sparse convolution tensor format are transformed by a 1×1×1 submanifold convolution operation, and the feature dimension is not changed in this process, including the following steps:
[0013] Data format conversion: Use the data format converter to convert the voxel features into a sparse convolution tensor for subsequent operations; Feature segmentation: Divide the features of the sparse convolution tensor into four equal parts, recorded as x i , where i∈{1, 2, 3, 4};
[0014] Substream convolution operation: for each independent sparse convolution tensor xi Perform submanifold convolution operation and perform 3×3×3 submanifold convolution operation on x1:
[0015] Feature stacking: Add the features of the processed x1 to x2, and then perform the same 3×3×3 submanifold convolution operation, and so on, for each processed X i Perform the same 3×3×3 submanifold convolution operation to obtain features at four different levels;
[0016] Feature splicing: The features of four different levels are spliced and transformed through sub-manifold convolution, and then integrated into a sparse tensor through a data format converter.
[0017] As a preferred technical solution of the present invention: the voxel data format obtained after voxelization is a sparse convolution tensor format, and the voxel features are fully extracted by constructing a voxel scale-aware CNN-Transformer hybrid backbone feature extraction architecture:
[0018] Specifically, the method includes the following steps: first, for voxels in sparse convolution tensor format, three CNN-Transformer hybrid modules are cascaded, each of which includes a voxel fine-grained perception module VFPM and two dilated scale perception Transformer modules DSAT, which are connected in sequence; the sparse convolution tensor obtained after processing by the voxel fine-grained perception module is converted into a sparse tensor format processed by the Transformer module DSAT through a data format converter, and then another data format converter is connected to convert the sparse tensor into a sparse convolution tensor to connect to subsequent VFPM convolution operations; through the cascade of these three hybrid modules, full extraction of voxel features is achieved to support subsequent operations.
[0019] As a preferred technical solution of the present invention: VFPM and DSAT are connected through a sparse voxel module, wherein VFPM and DSAT combine the advantages of local information modeling of 3D sparse convolution and contextual information modeling of Transformer to improve the performance and efficiency of the model when processing complex three-dimensional data.
[0020] As a preferred technical solution of the present invention: in DSAT, the voxel query mechanism dynamically adjusts the index density according to the importance of the data when processing the sparsity of non-empty voxels, and the voxels in important areas are densely indexed, while the voxels in unimportant areas remain sparse;
[0021] At the same time, voxel density is used to simulate attention weighting to increase attention to important areas; the granularity of participating voxels is refined in the intermediate area, and long-range information is integrated to enhance the comprehensive understanding ability of the model.
[0022] As a preferred technical solution of the present invention: the downsampling operation is implemented by a sparse voxel module in VoTr.
[0023] The second object of the present invention is to provide a 3D object detection method to address the problems in the prior art.
[0024] To this end, the above-mentioned purpose of the present invention is achieved through the following technical solutions:
[0025] 3D object detection, characterized in that it comprises the following steps:
[0026] S1, reads the KITTI dataset, divides it into a training set and a validation set, and then performs data enhancement operations such as rotation, scaling, and flipping on the dataset;
[0027] S2, performing voxel feature encoding VFE on the KITTI point cloud data preprocessed in S1, voxelizing the point cloud data to obtain sparse voxels;
[0028] S3, extracting features and downsampling the sparse voxels in step S2 through a voxel-oriented scale-aware feature extraction architecture system, and constructing a voxel-oriented scale-aware feature extraction architecture as a 3D feature extraction network;
[0029] S4, projects the voxels after feature extraction by the 3D feature extraction network in S3 into a 2D bird's-eye view, converting the original sparse data into dense data;
[0030] S5, after conversion into dense data, the 2D bird's-eye view image BEV is input into the 2D feature extraction network for feature extraction;
[0031] S6, input the features obtained in step S5 to the detection head to obtain the position and category of the 3D target;
[0032] S7, draw a 3D target detection box in the point cloud scene according to the position and category parameters of the 3D target obtained in S6.
[0033] Compared with the prior art, the voxel-oriented scale-aware feature extraction architecture system and 3D target detection method of the present invention have the following beneficial effects: the density of the point cloud is reduced by using the powerful spatial downsampling capability of CNN while retaining key feature information. These features are initially extracted through multiple CNN layers to form a high-dimensional feature map. Then, these features are further integrated and optimized through the self-attention mechanism in the Transformer model. The self-attention mechanism can model the global dependency between features at different positions, thereby effectively capturing a wider range of environmental context information. In order to adapt to different point cloud data characteristics and task requirements, the architecture also introduces a voxel scale-aware module, which can adaptively adjust the scale of feature extraction to better process point cloud data of different resolutions and densities. The voxel scale-aware module divides the point cloud data into non-overlapping voxel units and performs feature extraction and conversion operations within each voxel, so that the network can learn local and global feature representations more carefully. In addition, the overall architecture also includes a hybrid attention mechanism, which not only includes traditional self-attention, but also introduces local attention and global attention to adapt to point cloud data processing requirements at different levels from fine-grained to coarse-grained. This multi-scale attention mechanism further enhances the processing capability of the model, enabling it to accurately detect and segment objects in complex driving environments. The voxel-oriented scale-aware feature extraction architecture of the present invention significantly improves the performance of point cloud data processing by comprehensively utilizing the advantages of CNN and Transformer, which not only improves the adaptability of the model to complex environments, but also enhances its application potential in various computer vision tasks, such as target detection, semantic segmentation, etc.
[0034] The present invention provides a complex network architecture that combines CNN, Transformer and voxel-scale perception module for processing point cloud data. It combines the local feature extraction capability of CNN, the global modeling advantage of Transformer and the adaptive characteristics of voxel-scale perception module to form a powerful and flexible point cloud processing network. The introduction of the hybrid attention mechanism further enhances the performance of the model in multi-scale environments, enabling it to achieve excellent performance in tasks such as object detection and scene understanding. It provides an effective solution for processing large-scale, complex and changeable point cloud data, and has great application prospects in intelligent system fields such as autonomous driving and high-precision map construction, robotics and automation, remote sensing and geographic information systems. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is an overall block diagram of the voxel-oriented scale-aware feature extraction architecture system of the present invention;
[0036] Figure 2 This is the overall framework diagram of DSAT;
[0037] Figure 3 This is a comparison diagram of the voxel space search granularity of the present invention, (a1)-(a2) are the expanded attention in VoTr, the light blue squares and the yellow squares represent the voxel distribution of the point cloud voxelization and the voxels searched by the expanded attention of VoTr, respectively, (a1) is the distribution diagram of the original point cloud after voxelization, (a2) is the voxel searched by the expanded attention in VoTr; (b1)-(b3) are the voxels searched by the extended scale-aware attention at different scales in DSAT, and the blue squares represent the participating voxels at different expansion rates. DETAILED DESCRIPTION
[0038] The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.
[0039] A voxel-oriented scale-aware feature extraction architecture system. The effect of point cloud-based 3D object detection fully demonstrates the effectiveness of the architecture feature extraction. The voxel-based 3D object detection method has the following main steps:
[0040] S1: Read the KITTI dataset, divide it into a training set and a validation set according to a certain ratio, and then perform data enhancement operations such as rotation, scaling, and flipping on the dataset.
[0041] S2: Perform voxel feature encoding (VFE) on the KITTI point cloud data preprocessed in S1 to voxelize the point cloud data.
[0042] S3: Construct a 3D feature extraction network, this step is the core of the present invention. The voxel data format obtained after S2 voxelization is a sparse convolution tensor format. By constructing a voxel scale-aware CNN-Transformer hybrid backbone feature extraction architecture, the voxel features are fully extracted for downstream tasks. First, for the voxels in the sparse convolution tensor format, the present invention cascades three CNN-Transformer hybrid modules. Each hybrid module is mainly composed of a voxel fine-grained perception module (VFPM) and two expanded scale-aware Transformer modules (DSAT) connected in sequence. The sparse convolution tensor obtained after processing by the convolution module (VFPM) needs to be converted into a sparse tensor format that can be processed by the Transformer module (DSAT) through the data format converter designed in the invention, and then connected to a data format converter, and then the sparse tensor is converted into a sparse convolution tensor to connect the subsequent convolution operation (VFPM). Through the cascade of three hybrid modules, the voxel features can be fully extracted to support subsequent operations. The VFPM designed by the 3D convolution part and the DSAT designed by the Transformer part are connected by a downsampling module (sparse voxel module), wherein the voxel fine-grained perception module (VFPM) and the dilated scale perception transformer module (DSAT) effectively combine the advantages of 3D sparse convolution (local information modeling) and Transformer (context information modeling). In VFPM, sparse voxel tensors are grouped to perform submanifold convolution operations, and more fine-grained voxel multi-scale information is obtained, while also improving the anti-interference ability and robustness of the 3D object detector. In DSAT, multi-scale dilation attention is used to operate on non-empty voxels. Under the premise of ensuring local voxels, different dilation rates are used for multi-scale interaction. Due to the sparsity of non-empty voxels, the voxel query mechanism can make the voxels indexed in important areas dense, while the voxels indexed in unimportant areas are sparse. In the present invention, the density of the voxels involved in the calculation is used to simulate the attention weighting, so that the module pays more attention to the important foreground area. In addition, in the middle area, it refines the granularity of the participating voxels. It also considers the modeling of telematics.
[0043] S4: Project the voxels after the 3D backbone feature extraction network feature extraction in S3 into a 2D bird's-eye view (BEV). This step is to convert the original sparse data into dense data.
[0044] S5: After conversion into dense data, the 2D bird's-eye view (BEV) is input into the 2D feature extraction network for feature extraction.
[0045] S6: The final features are input to the detection head to obtain the position and category of the 3D target. In order to prove the effectiveness of the architecture proposed in this method, this method controls the variables, maintains the 3D backbone feature extraction architecture unchanged, and uses the SECOND and PV-RCNN detection heads on the one-stage and two-stage detection heads respectively.
[0046] S7: Finally, the 3D target detection box can be drawn in the point cloud scene using the position and category parameters of the 3D target obtained in S6.
[0047] In this invention, the convolution module is redesigned to extract local prior information before performing the voxel transformer operation, which can effectively obtain more refined and multi-level features. In the voxel transformer stage, a dilated scale-aware transformer module (DSAT) is designed, which uses three different scale-aware branches combined with a fast voxel query mechanism. Under the premise of a fixed voxel query amount, it is possible to adaptively obtain an expanded attention range and a finer granularity.
[0048] On the KITTI dataset, the detection accuracy of cars is generally higher than VoTr, especially for the recognition of foreground areas and difficult samples. It also exceeds PV-RCNN and other point cloud voxel interactive 3D object detection solutions. Specifically:
[0049] First, the Voxel Fine-Grained Perception Module (VFPM) proposed in the present invention performs sub-manifold convolution grouping operations on sparse convolution tensors, which can obtain more fine-grained voxel-level multi-scale information;
[0050] Second, the Dilated Voxel-Aware Transformer module (DSAT) proposed in this paper is based on the special attention mechanism and fast voxel query mechanism in VOTR, and introduces the idea of multi-scale interaction into voxels. It can be scale-aware and adaptively search voxels to ensure the complementation of search granularity; Third, this paper proposes the first voxel-based CNN-Transformer hybrid module for 3D object detection tasks.
[0051] In this application, CNN refers to convolutional neural network, VFPM refers to convolution module; DSAT is Transformer module.
[0052] Example 1
[0053] like Figure 1As shown, the overall block diagram of the feature extraction architecture proposed in the present invention, in the voxel-scale-aware CNN-Transformer hybrid backbone feature extraction architecture of the present invention, the downsampling operation is implemented by a sparse voxel module similar to that in VoTr. Before this step, the present invention designs a VFPM module, mainly to obtain local prior features. Inspired by Res2Net, the present invention applies the idea of ordinary 2D convolution to the 3D task of the present invention, so as to obtain finer-grained features in the convolution operation. The present invention designs a VFPM with submanifold convolution as the basic module, which makes it possible to learn richer multi-level information in a single module, and makes the 3D local space receptive field more flexible and adaptive.
[0054] The high efficiency of the method of the present invention in extracting and mining fine-grained features enables the present invention to compete with solutions that rely on point clouds rather than voxels as information supplements. Compared with other voxel-based methods, the method of the present invention not only has advantages in extracting foreground information, but also has obvious advantages in mining features from difficult samples.
[0055] First, the voxels in the sparse convolution tensor format obtained after voxelization are transformed by the 1×1×1 submanifold convolution operation. In this process, the present invention does not change its feature dimension. Therefore, the present invention divides the features of the sparse convolution tensor into four equal parts, denoted as x i , where i∈{1, 2, 3, 4}. The present invention uses a data format converter to convert them into separate sparse convolution tensors for subsequent operations. First, the present invention performs a 3×3×3 submanifold convolution operation on x1, adds the processed features to x2, and then performs the same 3×3×3 submanifold convolution operation, and so on. Finally, the features of the four branches are concatenated and converted into a whole through a data format converter. The output part y of each VFPM in the method of the present invention is i There is a similar statement in Res2Net, which can be written as:
[0056]
[0057] Compared with ordinary submanifold convolution, the method of the present invention introduces almost no additional computation, but it allows the present invention to mine deep features at multiple scales and levels. Due to the characteristic that submanifold convolution only operates on non-empty voxels, the present invention can better extract features from dense voxels in the foreground. Therefore, the VFPM of the present invention brings a greater contribution.
[0058] The entire architecture is a hybrid design of CNN and Transformer, with the convolution module (VFPM) placed before the Transformer module (DSAT). After feature extraction through VFPM, a data format converter is executed to convert the sparse convolution tensor required for the convolution operation into the sparse tensor required for the voxel transformer operation. The empty voxels are downsampled through the sparse voxel module, and after passing through the DSAT module, they need to be connected to the next VFPM module through the reverse data format converter. This allows the Transformer to communicate effectively with the information flow of the CNN.
[0059] like Figure 2 As shown, the overall framework diagram of the DSAT of the present invention is
[0060] In DSAT, the present invention uses three branches with different expansion rates to search for non-empty voxels involved in the calculation. Each branch uses a different expansion rate to search for non-empty voxels, while ensuring local voxels with a step size of 1. Due to the sparsity of non-empty voxels and the fast voxel query mechanism, the present invention can adaptively and quickly expand the search range according to the real environment. There is granular complementarity between the scales of the branches, which enhances the search granularity of the present invention. After fusing the features obtained by the self-attention of the three branches, the complementarity of the search granularity is effectively achieved, and important voxels of long distances and short distances can be considered at the same time, making it more versatile and adaptable to different scenarios.
[0061] (1) Self-attention mechanism on non-empty voxels: In the non-empty voxels of the present invention, N sparse ×3 integer index array ν is used to index and maintain the feature array corresponding to the non-empty voxels to improve computational efficiency. In each DSAT module, multi-head self-attention is used to establish long-range relations between non-empty voxels. When a query voxel i is given, for each head, the present invention calculates the query embedding Q with reference to VoTr i , key embedding K j The sum value is embedded in V j , as shown below:
[0062]
[0063] Where BF represents the branch feature. Therefore, the self-attention of voxels on the three branches can be expressed as:
[0064]
[0065] where Ω(i) represents the attention span of each branch and Softmax(·) is the softmax normalization function.
[0066] Next, three branches are added for feature fusion.
[0067] (2) Expanded scale-aware attention: In DSAT, the present invention designs three branches with expansion rates of 1, 2, and 4. The attention range Ω designed by the present invention in each branch is Scale-Aware(i) It is expressed by the following formula:
[0068] Ω Scale-Aware (i) = φ(v i -R local , v i +R local , (1, 1, 1))∪
[0069] φ(v i -R end , v i +R end , R stride )\
[0070] φ(v i -R local , v i +R local , R stride ), (4)
[0072] where Φ(start, end, stride) represents a function that inputs non-empty indices between (start, end) sampled by stride. The function \ represents a set subtraction operator. U takes the union of all non-empty voxel sets within the search range. R local is the local index range. The local attention fixes the stride to (1, 1, 1) to exploit the local range R local Each non-empty voxel in , so that the local attention can retain enough important foreground information and the corresponding fine-grained structure. It can be noted that in the present invention, only a configurable step ratio is needed to search R local and R end This enables the present invention to adaptively retain more nearby voxels of interest while still considering some other voxels of interest. The present invention only needs to give R end The attention range of the present invention can be adaptively expanded according to the size of the local attention voxel. It should be noted that in the specific implementation process, the local attention mainly refers to the attention voxels with a step size of 1.
[0073] The above-mentioned specific implementation methods are used to explain the present invention and are only preferred embodiments of the present invention, rather than limiting the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.
Claims
1. A voxel-oriented scale-aware feature extraction architecture system, characterized by: A data format converter is used to convert voxel features into sparse convolution tensors. First, the voxels in the sparse convolution tensor format are cascaded through three CNN-Transformer hybrid modules. Each CNN-Transformer hybrid module includes a voxel fine-grained perception module VFPM and two dilated scale perception Transformer modules DSAT, which are connected in sequence. The sparse convolution tensor obtained after processing by the voxel fine-grained perception module is converted into a sparse tensor format processed by the Transformer module DSAT through a data format converter, and then connected to another data format converter to convert the sparse tensor into a sparse convolution tensor to connect the subsequent VFPM convolution operation; through the cascade of these three hybrid modules, the voxel features are fully extracted to support subsequent operations. Among them, VFPM uses submanifold convolution as its basic module and only operates on non-empty voxels. In VFPM, submanifold convolution operations are performed by grouping sparse convolution tensors to process sparse data in three-dimensional space, which specifically includes the following steps: First, the feature transformation is performed through a 1×1×1 submanifold convolution operation; the feature dimension is not changed in this process. Feature segmentation: The features of the sparse convolution tensor are divided into four equal parts, denoted as x i , where i∈{1, 2, 3, 4}; Sub-population convolution operation: for each independent sparse convolution tensor x i Perform submanifold convolution operation and perform 3×3×3 submanifold convolution operation on x1: Feature stacking: Add the features of the processed x1 to x2, and then perform the same 3×3×3 submanifold convolution operation, and so on, for each processed X i Perform the same 3×3×3 submanifold convolution operation to obtain features at four different levels; Feature concatenation: concatenate the features of the four different levels and transform them through a 1×1×1 submanifold convolution operation, and then integrate them into a sparse tensor through a data format converter; In DSAT, multi-scale dilated attention operates on non-empty voxels. Local and sparse image block interactions are performed by adopting different dilation rates to achieve multi-scale interactions under the premise of ensuring local voxels. Specifically, three branches with different expansion rates are used to search for non-empty voxels involved in the calculation, while ensuring local voxels with a step size of 1.
2. The voxel-oriented scale-aware feature extraction architecture system according to claim 1, characterized in that: VFPM and DSAT are connected through a sparse voxel module, where VFPM and DSAT combine the advantages of local information modeling of 3D sparse convolution and contextual information modeling of Transformer to improve the performance and efficiency of the model when processing complex three-dimensional data.
3. The voxel-oriented scale-aware feature extraction architecture system according to claim 2, characterized in that: In DSAT, the voxel query mechanism dynamically adjusts the index density according to the importance of the data when dealing with the sparsity of non-empty voxels. Voxels in important areas are densely indexed, while voxels in unimportant areas remain sparse. At the same time, voxel density is used to simulate attention weighting to increase attention to important areas; the granularity of participating voxels is refined in the intermediate area, and long-range information is integrated to enhance the comprehensive understanding ability of the model.
4. The voxel-oriented scale-aware feature extraction architecture system according to claim 1, characterized in that: The downsampling operation is implemented through the sparse voxel module in VoTr. 5.3D object detection method, characterized in that The voxel-oriented scale-aware feature extraction architecture system according to any one of claims 1 to 4 comprises the following steps: S1, read the KITTI dataset, divide it into a training set and a validation set, and then perform data enhancement operations such as rotation, scaling, and flipping on the dataset; S2, performing voxel feature encoding VFE on the KITTI point cloud data preprocessed in S1, voxelizing the point cloud data to obtain sparse voxels; S3, extracting features and downsampling the sparse voxels in step S2, and constructing a voxel-oriented scale-aware feature extraction architecture system as a 3D feature extraction network; S4, projects the voxels after feature extraction by the 3D feature extraction network in S3 into a 2D bird's-eye view, converting the original sparse data into dense data; S5, after conversion into dense data, the 2D bird's-eye view image BEV is input into the 2D feature extraction network for feature extraction; S6, input the features obtained in step S5 to the detection head to obtain the position and category of the 3D target; S7, draw a 3D target detection box in the point cloud scene according to the position and category parameters of the 3D target obtained in S6.
Citation Information
Patent Citations
Three-dimensional target detection method and system based on substream sparse convolution
CN112270332A
Transform-based time sequence point cloud three-dimensional target detection
CN116740424A
Multi-task environment sensing method based on point cloud fusion of camera and laser radar
CN117036895A
Double-path fusion road point cloud segmentation method and device based on weighted sub-manifold convolution constraint
CN118097655A
Analyzing Spatially-Sparse Data Based on Submanifold Sparse Convolutional Neural Networks
US20190156206A1
