Target detection method and system based on multi-scale three-dimensional voxel feature unified modeling

Abstract

The invention provides a target detection method and system based on multi-scale three-dimensional voxel feature unified modeling, and relates to the technical field of three-dimensional perception and computer vision. The method comprises the following steps: acquiring original three-dimensional point cloud data and performing sparse voxelization; performing feature extraction on the obtained sparse voxel representation by adopting a sparse convolutional coding network to obtain multi-scale voxel feature representations with different spatial resolutions; constructing and training a multi-scale unified modeling model, and inputting the multi-scale voxel feature representation into the trained multi-scale unified modeling model to obtain enhanced voxel features; the enhanced multi-scale voxel features are mapped to the same spatial resolution, and final unified feature representation is obtained; and inputting the final uniform feature representation into a detection head network based on Transform to obtain a three-dimensional target detection result. According to the method, linear calculation complexity is maintained, and collaborative modeling of global semantic information and local space structure information is realized.

Description

Technical Field

[0001] This invention relates to the fields of 3D perception and computer vision technology, and in particular to a target detection method and system based on unified modeling of multi-scale 3D voxel features. Background Technology

[0002] 3D target detection is one of the key foundational technologies in the field of 3D perception, playing a crucial role in applications such as autonomous driving, mobile robots, intelligent transportation, and virtual reality. LiDAR, as a sensor capable of directly acquiring high-precision 3D spatial information, offers irreplaceable advantages in complex environment perception through its point cloud data. Therefore, 3D target detection methods based on LiDAR point clouds have been extensively researched and rapidly developed in recent years.

[0003] However, lidar point cloud data inherently possesses characteristics such as sparsity, high dimensionality, and irregular spatial distribution. How to extract discriminative 3D spatial features from large-scale point clouds while ensuring computational efficiency remains one of the core technical challenges in this field. To address these issues, existing 3D target detection methods can be broadly categorized into two types: point-based methods and voxel-based methods.

[0004] Point-based methods typically process the raw point cloud directly, constructing a permutation-invariant point feature extraction network and combining strategies such as neighborhood search, local feature aggregation, and layer-by-layer downsampling to gradually build multi-scale contextual information. These methods can preserve the geometric details of the point cloud relatively well and have certain advantages in local structure modeling. However, because they require neighborhood construction and feature computation at the point level, the computation process is highly dependent on point-by-point operations and multi-level geometric searches, resulting in significant computational overhead and limited inference efficiency in large-scale scenarios, making it difficult to meet the needs of applications with high real-time and scalability requirements.

[0005] In contrast, voxel-based methods discretize irregular point clouds into regular voxel structures and then perform feature extraction and modeling based on voxels, offering significant advantages in computational efficiency and parallelism. Depending on the voxel feature modeling method, this class of methods can be further divided into voxel methods based on sparse convolution and voxel modeling methods based on attention mechanisms.

[0006] Voxel-based methods using sparse convolution utilize sparse convolution operators to compute only non-empty voxels, effectively avoiding ineffective computation on a large number of empty voxels and thus significantly improving overall computational efficiency. These methods typically extract local voxel features by stacking multiple layers of sparse convolutional networks and gradually expand the receptive field using downsampling structures. However, since convolution operations are inherently limited by fixed-size kernels, their receptive field expansion mainly relies on network depth or additional downsampling operations, making it difficult to efficiently model long-distance dependencies between voxels. Furthermore, their ability to perceive global contextual information in complex scenes remains limited.

[0007] To compensate for the shortcomings of sparse convolution in global modeling, voxel modeling methods based on attention mechanisms have gained increasing attention in recent years. These methods, by introducing self-attention or its variants, explicitly model the correlations between voxel features, possessing a natural advantage in capturing long-range dependencies. However, the computational complexity of standard attention mechanisms increases quadratically with the number of voxels, resulting in significant computational and storage overhead when directly applied to large-scale 3D point cloud scenes. Therefore, existing methods typically employ strategies such as window partitioning, group computation, or local attention to reduce computational complexity. However, these strategies limit the receptive field to some extent, thus restricting the ability to model global dependencies.

[0008] Furthermore, to fuse voxel features at different scales, existing 3D object detection methods generally employ multi-scale structures similar to U-Net or feature pyramids, aligning and fusing features at different scales point-by-point in the spatial dimension through upsampling and downsampling operations. However, due to the high computational cost of 3D point cloud upsampling operations, existing methods typically use a static, linear fusion approach, simply superimposing or stitching features at different scales. This point-by-point alignment fusion strategy struggles to model dynamic dependencies across scales, and when spatial resolution and semantic granularity differ across scales, it easily introduces semantic interference between scales, thus affecting the stability and consistency of feature representation.

[0009] In summary, existing 3D object detection methods still face challenges in multi-scale voxel feature fusion, including insufficient computational efficiency, inadequate long-distance dependency modeling capabilities, and difficulties in feature alignment. Therefore, a novel technical solution is urgently needed that can achieve unified modeling of sparse 3D voxel multi-scale features while maintaining linear computational complexity, thereby improving the overall performance and robustness of 3D object detection in complex scenarios. Summary of the Invention

[0010] To address the problems of high computational cost, strong point-to-point spatial alignment dependence, and difficulty in effectively modeling long-distance cross-scale dependencies in existing LiDAR-based 3D target detection methods during multi-scale voxel feature fusion, this invention proposes a target detection method and system based on unified modeling of sparse voxel features at different scales. This method introduces a serialization modeling mechanism and a local spatial feature enhancement strategy, aiming to achieve collaborative modeling of global semantic information and local spatial structure information while maintaining linear computational complexity. This provides an efficient and stable feature modeling scheme for 3D point cloud target detection.

[0011] On the one hand, this invention proposes a target detection method based on unified modeling of multi-scale three-dimensional voxel features, which includes the following process:

[0012] The original 3D point cloud data is acquired and sparsed voxelized to obtain a sparse voxel representation.

[0013] A sparse convolutional coding network is used to extract features from the sparse voxel representation, and stepwise downsampling is performed during the feature extraction process to obtain multi-scale voxel feature representations with different spatial resolutions.

[0014] Construct and train a multi-scale unified modeling model to obtain a well-trained multi-scale unified modeling model.

[0015] The multi-scale voxel feature representation is input into the trained multi-scale unified modeling model to obtain the enhanced multi-scale voxel features.

[0016] The enhanced multi-scale voxel features are mapped to the same spatial resolution to obtain the final unified feature representation;

[0017] The final unified feature representation is input into the Transformer-based detection head network to obtain the 3D target detection results.

[0018] Furthermore, the multi-scale unified modeling model includes:

[0019] The local spatial location embedding module is used to perform local spatial location embedding on voxel feature representations at different scales in multi-scale voxel feature representations. For each scale, the obtained local spatial location embedding is added to the voxel feature representation at that scale to obtain the position-enhanced voxel feature representation at that scale.

[0020] The unified spatial representation module is used to map the location-enhanced voxel feature representations at different scales to the spatial coordinate system corresponding to the highest resolution, thereby obtaining a unified set of spatial coordinates. It then performs feature fusion by concatenating all the mapped voxel feature representations to obtain a joint feature representation.

[0021] The multi-curve serialization input module is used to encode a unified set of spatial coordinates by using different space-filling curves, map the joint feature representation to a one-dimensional sequence space, and generate several one-dimensional voxel sequences.

[0022] The dual-end state space modeling module is used to perform bidirectional state space modeling on each one-dimensional voxel sequence using a state space model, and to concatenate and fuse the obtained positive and negative output results to obtain a unified feature representation corresponding to each one-dimensional voxel sequence.

[0023] The feature separation module is used to splice together the unified feature representations corresponding to all one-dimensional voxel sequences. By performing the inverse mapping in the multi-curve serialization input module, the spliced ​​unified feature representations are restored to a unified spatial coordinate system. Then, by performing the inverse process of the unified spatial representation module on the restored unified feature representations, multi-scale features that correspond one-to-one with the scales in the multi-scale voxel feature representations are obtained.

[0024] The multi-kernel local feature enhancement module is used to divide multi-scale features into multiple sub-feature groups, model each sub-feature group using several parallel sub-manifold convolution branches, and concatenate and fuse the local features output by each branch to obtain enhanced multi-scale voxel features.

[0025] Furthermore, the process of embedding the local spatial location is as follows:

[0026] For arbitrary scales in multi-scale voxel feature representation The voxel feature representation is used to extract the non-empty set of voxel coordinates from the voxel feature representation. ;

[0027] Based on the non-empty voxel coordinate set Generate the coordinate feature representation of the voxel feature representation;

[0028] Using the coordinate feature representation as input, spatial neighborhood encoding of non-empty voxels in the voxel feature representation is performed using three-dimensional submanifold convolution to generate the local spatial location embedding of the voxel feature representation.

[0029] Furthermore, the specific content of encoding a unified set of spatial coordinates using different space-filling curves, mapping the joint feature representation to a one-dimensional sequence space, and generating several one-dimensional voxel sequences is as follows:

[0030] The unified set of spatial coordinates is encoded in parallel using at least two different space-filling curves.

[0031] Where for any space-filling curve Using space-filling curves Encode each non-empty voxel coordinate in a unified set of spatial coordinates to obtain a sequence of values ​​for all non-empty voxel coordinates;

[0032] Sort the sequence values ​​of all non-empty voxel coordinates, and assign a serialization index to each non-empty voxel in the joint feature representation based on the sorting result;

[0033] Based on the serialized index of all non-empty voxels in the joint feature representation, all non-empty voxels in the joint feature representation are arranged to generate a one-dimensional voxel sequence.

[0034] Furthermore, the dual-end state space modeling module includes two parallel and structurally identical state space model branches, namely: a forward state space model branch and a reverse state space model branch;

[0035] The forward state space model branch is used to update the state of the one-dimensional voxel sequence in the order of the one-dimensional voxel sequence and generate a forward output result.

[0036] The reverse state space model branch is used to update the state of the one-dimensional voxel sequence in reverse order to generate the reverse output result.

[0037] The forward and backward outputs are concatenated along the feature dimension, and the concatenated outputs are then subjected to linear projection, normalization, and random path discarding to obtain the global features.

[0038] Then, through residual connection, the global features are concatenated to the one-dimensional voxel sequence to obtain the unified feature representation corresponding to the one-dimensional voxel sequence.

[0039] Furthermore, the process of dividing the multi-scale features into multiple sub-feature groups, modeling each sub-feature group using several parallel sub-manifold convolutional branches, and concatenating and fusing the local features output by each branch to obtain the enhanced multi-scale voxel features is as follows:

[0040] For arbitrary scales in multi-scale features Features , for features Perform batch normalization, and then divide the batch-normalized features into channels. Sub-feature groups;

[0041] use Three-dimensional convolution of submanifolds with different kernel sizes, respectively. Local modeling is performed on each sub-feature group to obtain... Each sub-feature group corresponds to a convolutional kernel size.

[0042] Will Local features are concatenated along the channel dimension and then processed by a convolution kernel. Sub-manifold convolution performs cross-channel fusion of the concatenated local features to obtain the fused features;

[0043] A feedforward neural network (FFN) is used to perform a nonlinear transformation on the fused features, and the features obtained from the nonlinear transformation are then connected to the feature vector through residual connections. Add them together to obtain the features. Enhanced voxel features.

[0044] Furthermore, the submanifold 3D convolution is implemented using depth-separable submanifold convolution.

[0045] Furthermore, the training method for the multi-scale unified modeling model is as follows:

[0046] Acquire several sets of original 3D point cloud data, and perform sparse voxelization on each set of original 3D point cloud data to obtain sparse voxel representation.

[0047] A sparse convolutional coding network is used to extract features from the sparse voxel representation, and downsampling is performed stepwise during the feature extraction process to obtain multi-scale voxel feature representations with different spatial resolutions, thereby constructing a training dataset.

[0048] Set the initial learning rate Training batch size and training rounds The Adam optimizer is used to train the multi-scale unified model by inputting the training dataset into it.

[0049] For the former In any training epoch, within the multi-curve serialization input module, a dynamic shuffle mechanism is used to randomly arrange the order of all space-filling curves, and the unified set of spatial coordinates is encoded according to the generated random order, thereby generating several one-dimensional voxel sequences; for the subsequent... In any batch of training batches, the unified set of spatial coordinates is encoded in the multi-curve serialization input module according to the fixed order preset by all space-filling curves, thereby generating several one-dimensional voxel sequences.

[0050] After training is completed, a trained multi-scale unified modeling model is obtained.

[0051] Furthermore, the specific content of mapping the enhanced multi-scale voxel features to the same spatial resolution to obtain the final unified feature representation is as follows: the enhanced multi-scale voxel features are uniformly mapped to the highest spatial resolution, and the mapped unified voxel features are fused to obtain the final unified feature representation.

[0052] On the other hand, this invention proposes a target detection system based on unified modeling of multi-scale three-dimensional voxel features, the system comprising:

[0053] The data acquisition module is used to acquire the raw 3D point cloud data to be detected;

[0054] The sparse voxelization module is used to sparse voxelize the original 3D point cloud data to obtain a sparse voxel representation.

[0055] The feature extraction module is used to extract features from the sparse voxel representation using a sparse convolutional coding network, and to perform stepwise downsampling during the feature extraction process to obtain multi-scale voxel feature representations with different spatial resolutions.

[0056] The unified modeling module is used to input multi-scale voxel feature representations into a trained multi-scale unified modeling model to obtain enhanced multi-scale voxel features.

[0057] The feature mapping module is used to map the enhanced multi-scale voxel features to the same spatial resolution to obtain the final unified feature representation;

[0058] The target detection module is used to input the final unified feature representation into the Transformer-based detection head network to obtain the 3D target detection results.

[0059] The beneficial effects of adopting the above technical solution are as follows:

[0060] This invention improves the global expressive power and robustness of voxel features by uniformly modeling sparse voxel features at different scales and achieving efficient interaction between features at multiple scales without requiring point-by-point spatial alignment. By introducing a serialization modeling mechanism and a local spatial feature enhancement strategy, it achieves collaborative modeling of global semantic information and local spatial structural information while maintaining linear computational complexity, providing an efficient and stable feature modeling scheme for 3D object detection.

[0061] In summary, this invention achieves efficient and unified modeling of multi-scale voxel features in sparse 3D point clouds through multi-curve serialization, multi-scale bidirectional state-space modeling, and local feature enhancement. This method can capture long-distance dependencies across scales and spaces without requiring point-by-point spatial alignment, while maintaining the integrity of local geometric and semantic information. Therefore, this method is suitable for LiDAR-based 3D target detection tasks, especially for applications in sparse, large-scale 3D scenes that require modeling and fusing long-distance dependencies across scales while ensuring computational efficiency. Attached Figure Description

[0062] Figure 1This is a flowchart of the target detection method based on multi-scale three-dimensional voxel feature unified modeling in this embodiment;

[0063] Figure 2 This is an overall schematic diagram of the target detection method based on unified modeling of multi-scale three-dimensional voxel features in this embodiment;

[0064] Figure 3 This is a structural diagram of the multi-scale unified modeling model in this embodiment;

[0065] Figure 4 This is a structural diagram of the multi-curve serialization input module in this embodiment;

[0066] Figure 5 This is a structural diagram of the dual-end state space modeling module in this embodiment;

[0067] Figure 6 This is a structural diagram of the multi-core local feature enhancement module in this embodiment;

[0068] Figure 7 The images show the final results of different methods on the test dataset; where (a) is the baseline image and (b) is the image of this embodiment.

[0069] Figure 8 This is a structural diagram of the target detection system based on multi-scale three-dimensional voxel feature unified modeling in this embodiment. Detailed Implementation

[0070] To facilitate understanding of this application, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following embodiments are illustrative of the invention but are not intended to limit its scope. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0071] Example 1:

[0072] This embodiment presents a target detection method based on unified modeling of multi-scale three-dimensional voxel features, such as... Figure 1 As shown, the method includes the following steps:

[0073] The original 3D point cloud data is acquired and sparsed voxelized to obtain a sparse voxel representation.

[0074] In this embodiment, the original 3D point cloud data is acquired and sparse voxelization is performed to map the unstructured original 3D point cloud data into a regular sparse voxel representation.

[0075] A sparse convolutional coding network is used to extract features from the sparse voxel representation, and stepwise downsampling is performed during the feature extraction process to obtain multi-scale voxel feature representations with different spatial resolutions.

[0076] In this embodiment, as Figure 2 As shown, the sparse convolutional coding network uses a 3D sparse backbone network composed of several layers of sparse convolutions to extract features from the input sparse voxel representation. During the feature extraction process, it downsamples layer by layer to obtain multi-scale voxel feature representations with different spatial resolutions and semantic levels, i.e., multi-scale feature maps.

[0077] Construct and train a multi-scale unified model to obtain a well-trained multi-scale unified model.

[0078] Based on this, this embodiment does not adopt the multi-scale fusion method that relies on point-by-point spatial alignment in traditional U-Net or feature pyramid networks. Instead, it introduces a unified multi-scale voxel modeling mechanism to model sparse voxel features at different scales as a whole, so as to establish long-distance dependencies across scales. After completing the unified modeling, the local spatial structure information of the voxels is restored and enhanced. Finally, the fused voxel features are input into the detection head network to achieve the classification and localization of 3D targets. The multi-scale unified modeling and local structure restoration constitute the core innovation of this embodiment.

[0079] Specifically, such as Figure 3 As shown, the core objective of the multi-scale unified modeling model is to perform unified global modeling of feature maps from different resolution levels. This embodiment uses a cross-scale simplified spatial alignment mechanism to map and align the spatial coordinates corresponding to feature maps at different scales to the same resolution space, thereby achieving the fusion of multi-scale information within a unified feature representation domain.

[0080] The multi-scale unified modeling model includes:

[0081] The local spatial location embedding module is used to perform local spatial location embedding on voxel feature representations at different scales in multi-scale voxel feature representations. For each scale, the obtained local spatial location embedding is added to the voxel feature representation at that scale to obtain the position-enhanced voxel feature representation at that scale.

[0082] In this embodiment, after the 3D sparse backbone network obtains multi-scale feature maps, it is first input into the local spatial location embedding module to introduce location embedding representations related to the three-dimensional spatial coordinates of the voxel features at different scales, so as to make up for the problem of missing spatial location information that may be introduced in the subsequent voxel serialization and state space modeling process.

[0083] The process of embedding the local spatial location is as follows: for any scale in the multi-scale voxel feature representation The voxel feature representation is used to extract the non-empty set of voxel coordinates from the voxel feature representation. .

[0084] Based on the non-empty voxel coordinate set Generate the coordinate feature representation of the voxel feature representation.

[0085] Using the coordinate feature representation as input, spatial neighborhood encoding of non-empty voxels in the voxel feature representation is performed using three-dimensional submanifold convolution to generate the local spatial location embedding of the voxel feature representation.

[0086] In the unified modeling process, considering that multi-scale sparse voxels may lose their original spatial location information when directly mapped to the sequence modeling space, this embodiment first introduces a local spatial location embedding mechanism to explicitly encode the local geometric location of the voxels. Specifically, let the first voxel in the multi-scale feature map... The set of non-empty voxel coordinates at each scale is The local spatial location embedding at the corresponding scale is achieved through convolution of a 3D submanifold that operates only on non-empty voxel locations, and the computational form is as follows:

[0087]

[0088] in, express Local spatial location embedding; The total number of multi-scale feature maps; This represents the 3D convolution operation on submanifolds. This operation encodes the local spatial neighborhood of voxels while preserving the sparse structure and computational efficiency, thus providing spatial constraint information at each scale for subsequent sequential modeling. By applying LPE to feature maps at all scales, the local geometry of voxels across multiple scales can be fully preserved during the unified modeling stage.

[0089] The unified spatial representation module is used to map the location-enhanced voxel feature representations at different scales to the spatial coordinate system corresponding to the highest resolution, thereby obtaining a unified set of spatial coordinates. By splicing together all the mapped voxel feature representations, feature fusion is performed to obtain a joint feature representation.

[0090] After completing the local spatial location embedding, this embodiment uses a simple spatial alignment operation to achieve spatial alignment and feature fusion processing of multi-scale features. This operation does not rely on complex interpolation operations, but rather unifies the spatial locations corresponding to features from different resolution scales to the same resolution, providing the basic conditions for learning cross-scale dependencies.

[0091] The specific process is as follows: A cross-scale coordinate mapping strategy is adopted to uniformly map voxel coordinates from different resolution levels to the same reference resolution space. This applies to multi-scale feature maps. Feature maps of different scales have a fixed downsampling ratio relationship in terms of spatial resolution. Let the first... The set of non-empty voxel coordinates corresponding to each scale is: Its corresponding spatial resolution is relative to the highest resolution. The downsampling ratio is To achieve spatial alignment of multi-scale features, scale mapping is performed on the voxel coordinates at each scale, uniformly mapping them to the spatial coordinate system corresponding to the highest resolution. The mapping process is as follows:

[0092]

[0093] in, express The mapped non-empty set of voxel coordinates; when Sometimes, ; express Any non-empty voxel in; Indicates non-empty voxels The three-dimensional coordinates.

[0094] Perform a union operation on the set of voxel coordinates after all scales are aligned to obtain a unified set of spatial coordinates. , is represented as:

[0095]

[0096] Based on this, the voxel features at each scale are spliced ​​together after spatial alignment to form a joint feature representation. .

[0097]

[0098] Through the above cross-scale spatial alignment and feature fusion operations, voxel features at different resolution levels can establish clear spatial correspondences in a unified sparse voxel space, enabling the model to simultaneously perceive multi-scale semantic information and spatial structural information at a unified scale, laying the foundation for subsequent multi-curve serialization modeling and dual-end state space modeling.

[0099] The multi-curve serialization input module is used to encode a unified set of spatial coordinates by using different space-filling curves, map the joint feature representation to a one-dimensional sequence space, and generate several one-dimensional voxel sequences.

[0100] In this embodiment, after obtaining a unified spatial representation, the joint feature representation is fed into a multi-curve serialization input module. The three-dimensional voxel space is serialized and mapped using various spatial filling curves, generating multiple one-dimensional feature sequences from different spatial traversal perspectives. The multi-curve serialization method can introduce complementary global structural perspectives while maintaining local spatial adjacency relationships, thereby alleviating the limitations of a single serialization strategy in preserving spatial topology.

[0101] The specific content of encoding a unified set of spatial coordinates using different space-filling curves, mapping the joint feature representation to a one-dimensional sequence space, and generating several one-dimensional voxel sequences is as follows:

[0102] The unified set of spatial coordinates is encoded in parallel using at least two different space-filling curves.

[0103] In this embodiment, multiple sequences are generated in parallel using various space-filling curves, including but not limited to Z-order curves, Hilbert curves, and their transposes. This encodes the spatial relationships of voxels from different spatial traversal perspectives, alleviating the limitations of a single serialization strategy in preserving spatial topology. It should be noted that the space-filling curves used in this embodiment are all existing, and the length of the resulting one-dimensional voxel sequences obtained using different space-filling curves is guaranteed to be consistent; there is no need to limit the number of generated sequences.

[0104] For any space-filling curve Using space-filling curves Encode each non-empty voxel coordinate in a unified set of spatial coordinates to obtain a sequence of values ​​for all non-empty voxel coordinates.

[0105] To adapt to the modeling paradigm of state-space models for one-dimensional sequences, such as Figure 4 As shown. This embodiment proposes a multi-curve serialization input module for mapping three-dimensional sparse voxels to a one-dimensional sequence space according to corresponding curves. A unified set of spatial coordinates is used. Represented as:

[0106]

[0107] in, The number of non-empty voxels in a unified set of spatial coordinates; Represents any non-empty voxel in a unified set of spatial coordinates. The coordinates.

[0108] Using space-filling curves Encoding the coordinates of non-empty voxels yields the corresponding sequence values, represented as follows:

[0109]

[0110] in, Represents coordinates The sequence value obtained by encoding.

[0111] Sort the sequence values ​​of all non-empty voxel coordinates and assign a serialization index to each non-empty voxel in the joint feature representation based on the sorting results.

[0112]

[0113] in, Represents a set of serialization indexes; This indicates a sorting operation.

[0114] Based on the serialized index of all non-empty voxels in the joint feature representation, all non-empty voxels in the joint feature representation are arranged to generate a one-dimensional voxel sequence.

[0115]

[0116] in, This represents a sorted one-dimensional voxel sequence. This represents the first voxel in the sorted one-dimensional voxel sequence. Non-empty voxels at each position.

[0117] The dual-end state space modeling module is used to perform bidirectional state space modeling on each one-dimensional voxel sequence using a state space model, and to concatenate and fuse the obtained forward and reverse output results to obtain a unified feature representation corresponding to each one-dimensional voxel sequence.

[0118] In this embodiment, the serialized one-dimensional voxel sequence is input into a two-ended state-space modeling module. This module constructs forward and backward state-space branches, where the forward branch performs state recursion along the forward direction of the sequence to model direction-dependent long-range dependencies, while the backward branch performs isomorphic state updates on the reverse sequence to supplement geometric and structural information in the reverse direction. Through the two-ended state propagation mechanism, the model can form complementary global receptive fields within the feature domain, thereby improving the overall modeling capability for complex spatial structures.

[0119] The dual-end state space modeling module includes two parallel and identical state space model branches: a forward state space model branch and a reverse state space model branch.

[0120] The forward state space model branch is used to update the state of the one-dimensional voxel sequence in the order of the one-dimensional voxel sequence to generate a forward output result.

[0121] The reverse state-space model branch is used to update the state of the one-dimensional voxel sequence in reverse order to generate the reverse output result.

[0122] In this embodiment, as Figure 5 As shown, a unified multi-scale global model is performed on the serialized voxel features. To enhance the model's ability to express complex dependencies, an input-dependent, time-variable state-parameter state-space model (SSM) is needed, allowing the state transition matrix to adaptively adjust according to the sequence features. In this embodiment, Mamba is used to construct the state-space model branches. Furthermore, to improve the directional robustness of sequence modeling, state-space branches are constructed in both the forward and reverse directions of the sequence, yielding the forward output results respectively. and reverse output results Specifically, the state-space model models the dependencies between elements in the sequence through state recursion, and its state update process is represented as follows:

[0123]

[0124]

[0125]

[0126]

[0127] in, Indicates the time step index; For time step The hidden state vector, and ; For time step The hidden state vector; For a one-dimensional voxel sequence at time step The input features, and , express The dimension; For time step The output features, and , express The dimension; This represents the step size or time scale parameter related to the input. Its initial value is given through random initialization and is adaptively learned by the neural network during training to adjust the mapping process from continuous-time system to discrete-time system. The continuous-time state transition matrix is ​​represented by the HiPPO (Highly Optimized Projection Operator) construction method to ensure effective modeling of historical information. ; This represents the discretization function of the continuous-to-discrete state transition matrix, based on the step size parameter. The continuous-time state transition matrix Mapped to discrete-time state transition matrix ; Represents a continuous-time input matrix, and ; express The corresponding discretization function, given , and step size parameters Generate discrete-time input matrix under the condition ; This represents the output projection matrix, used to project the hidden states. Mapping to the output space yields the final model output. .

[0128] The forward and backward outputs are concatenated along the feature dimension, and the concatenated outputs are then subjected to linear projection, normalization, and random path discarding to obtain the global features.

[0129] Then, through residual connection, the global features are concatenated to the one-dimensional voxel sequence to obtain the unified feature representation corresponding to the one-dimensional voxel sequence.

[0130] In this embodiment, the forward and reverse output results are concatenated and feature-fused, as shown below:

[0131]

[0132] in, This indicates that the result will be output in the positive direction. The reverse output result Concatenate along the feature dimension; This is a linear projection operation; Used to normalize features to stabilize training; This involves a random path drop-off operation to enhance the model's regularization and generalization capabilities; finally, the original input features are combined through residual connections. The processed global features are fused together to obtain the output features at each time step. That is, the unified feature representation corresponding to a one-dimensional voxel sequence.

[0133] The feature separation module is used to splice together the unified feature representations corresponding to all one-dimensional voxel sequences. By performing the inverse mapping in the multi-curve serialization input module, the spliced ​​unified feature representations are restored to a unified spatial coordinate system. Then, by performing the inverse process of the unified spatial representation module on the restored unified feature representations, multi-scale features corresponding one-to-one with the scales in the multi-scale voxel feature representations are obtained.

[0134] After completing the two-end state space modeling, this embodiment obtains a unified feature representation. The data is then reprojected and restored to their respective spatial scales and resolution levels. This ensures the spatial consistency and scale compatibility of multi-scale features in subsequent network stages, providing fundamental support for local structure modeling and fine geometric information recovery.

[0135] The multi-kernel local feature enhancement module is used to divide multi-scale features into multiple sub-feature groups, model each sub-feature group using several parallel sub-manifold convolution branches, and concatenate and fuse the local features output by each branch to obtain enhanced multi-scale voxel features.

[0136] While mapping three-dimensional voxels to one-dimensional sequences and modeling the global state space can effectively establish cross-scale, long-distance dependencies, it may still weaken the voxel's geometric structure and fine-grained semantic representation capabilities within its local neighborhood. For example... Figure 6 As shown, this embodiment further introduces a multi-kernel local feature enhancement module after unified modeling. This module restores and strengthens the local spatial structure information of voxels while maintaining global semantic consistency, achieving fine-grained local structure compensation for the restored multi-scale features. By employing multi-kernel local operators with different receptive fields, local geometric details are enhanced on the globally modeled features to restore neighborhood structural relationships that may have been weakened during serialization and global modeling in the state space. Ultimately, this achieves a synergistic enhancement of global semantic modeling capabilities and local geometric representation capabilities.

[0137] The process involves dividing multi-scale features into multiple sub-feature groups, modeling each sub-feature group using several parallel sub-manifold convolutional branches, and concatenating and fusing the local features output by each branch to obtain the enhanced multi-scale voxel features.

[0138] For arbitrary scales in multi-scale features Features , for features Perform batch normalization, and then divide the batch-normalized features into channels. Sub-feature groups; among which .

[0139] In this embodiment, multi-scale features are first processed by batch normalization. Then, it is divided equally along the channel dimension into Each sub-feature group is represented as:

[0140]

[0141] in, Indicates batch normalization; They represent the 1st, 2nd, and 3rd respectively. Sub-feature groups.

[0142] use Three-dimensional convolution of submanifolds with different kernel sizes, respectively. Local modeling is performed on each sub-feature group to obtain... Each sub-feature group corresponds to a convolutional kernel size.

[0143] In this embodiment, during actual use, a total of [number] are set. Each sub-feature group is modeled using convolutional kernels of different sizes: 1, 3, 5, and 7. For each sub-feature group, local modeling is performed using 3D convolution with sub-manifolds of different kernel sizes, as shown below:

[0144]

[0145] in, Indicates the first Local features; Indicates the first Three-dimensional convolution of sub-manifolds, and different Corresponding to different spatial receptive fields, it is used to characterize the geometric structural changes of voxels in small-scale, medium-scale and large-scale neighborhoods; Indicates the first Sub-feature groups.

[0146] Will Local features are concatenated along the channel dimension and then processed by a convolution kernel. Sub-manifold convolution performs cross-channel fusion of the spliced ​​local features to obtain the fused features.

[0147]

[0148] in, Indicates the characteristics after fusion; Indicates that the convolution kernel is Convolution of submanifolds; They represent the 1st and the 2nd respectively. Local features.

[0149] A feedforward neural network (FFN) is used to perform a nonlinear transformation on the fused features, and the features obtained from the nonlinear transformation are then connected to the feature vector through residual connections. Add them together to obtain the features. Enhanced voxel features.

[0150]

[0151] in, Indicates the enhanced voxel characteristics; This describes the processing procedure of the feedforward neural network (FFN).

[0152] Unlike traditional methods that rely solely on single-scale submanifold convolution for local compensation, this embodiment divides the input multi-scale reconstructed features into multiple sub-feature groups along the channel dimension. These sub-feature groups are then modeled in parallel using submanifold convolution branches with different receptive field sizes, thus explicitly capturing the local geometric changes of voxels at different spatial scales. This design effectively supplements the local geometric information of voxels at different spatial scales without compromising the global state space modeling results, enhancing the model's ability to perceive changes in edges, contours, and local structures.

[0153] In practical applications, to improve computational efficiency, based on the aforementioned multi-kernel local feature enhancement module, a further implementation based on depthwise separable submanifold convolution is proposed to reduce computational complexity and parameter size while maintaining modeling capabilities. Specifically, for each submanifold convolution branch, depthwise separable submanifold 3D convolution is first used to perform channel-by-channel spatial modeling of voxel features; that is, each input channel only performs convolution operations with its corresponding kernel to efficiently capture local geometric structures. Then, the voxel features are further... Submanifold convolution accomplishes cross-channel information fusion. This design significantly reduces redundant computation in sparse voxel scenes while maintaining multi-scale local modeling capabilities. Mathematically, it can be expressed as:

[0154]

[0155]

[0156] in, This represents a channel-wise submanifold 3D convolution operation, where the convolution kernels are independent of each other in the channel dimension and only perform local receptive field modeling in the spatial dimension. This represents a pointwise flow 3D convolution operation, through... Convolution performs weighted fusion of multi-branch features along the channel dimension, thereby achieving cross-channel information integration;

[0157] The training method for the multi-scale unified modeling model is as follows:

[0158] Several sets of original 3D point cloud data are acquired, and each set of original 3D point cloud data is sparsed voxelized to obtain a sparse voxel representation.

[0159] A sparse convolutional coding network is used to extract features from the sparse voxel representation. During the feature extraction process, downsampling is performed stepwise to obtain multi-scale voxel feature representations with different spatial resolutions, thereby constructing a training dataset.

[0160] Set the initial learning rate Training batch size and training rounds The Adam optimizer is used to train the multi-scale unified model by inputting the training dataset into the model.

[0161] In this embodiment, the model is trained on the nuScenes dataset, and the voxel size is set to... The dataset was trained from scratch using the Adam optimizer with an initial learning rate of 0.003. Training was performed using eight NVIDIA RTX 4090 GPUs with a batch size of 16. Training was conducted on the nuScenes dataset for 32 epochs.

[0162] For the former In any training epoch, within the multi-curve serialization input module, a dynamic shuffle mechanism is used to randomly arrange the order of all space-filling curves, and the unified set of spatial coordinates is encoded according to the generated random order, thereby generating several one-dimensional voxel sequences; for the subsequent... In any batch of training batches, the unified set of spatial coordinates is encoded in the multi-curve serialization input module according to the fixed order preset by all space-filling curves, thereby generating several one-dimensional voxel sequences.

[0163] To further enhance the robustness and generalization ability of the modeling, this embodiment introduces a dynamic shuffle mechanism. In each of the first 32 rounds of training, the input order of multiple space-filling curves is randomly shuffled, thereby avoiding the model's over-reliance on a fixed sequence order, as shown below:

[0164]

[0165]

[0166] in, Indicates the number of parallel space-filling curves; Indicates the random rearrangement strategy; Indicates the random order of generation; They represent the 1st, 2nd, and 3rd respectively. A space-filling curve.

[0167] Then, according to Multiple space-filling curves are sequentially input into the multi-curve serialization input module. Dynamic shuffle is disabled after 32 training epochs or during the evaluation phase. This makes the data distribution of the input and evaluation data more similar, while ensuring the consistency and reproducibility of model inference.

[0168] The multi-scale voxel feature representation is input into the trained multi-scale unified modeling model to obtain the enhanced multi-scale voxel features.

[0169] The enhanced multi-scale voxel features are mapped to the same spatial resolution to obtain the final unified feature representation.

[0170] The specific content of mapping the enhanced multi-scale voxel features to the same spatial resolution to obtain the final unified feature representation is as follows: the enhanced multi-scale voxel features are uniformly mapped to the highest spatial resolution, and the mapped unified voxel features are fused to obtain the final unified feature representation.

[0171] The final unified feature representation is input into the Transformer-based detection head network to obtain the 3D target detection results.

[0172] The three-dimensional target detection results include: target category and target's three-dimensional bounding box.

[0173] In this embodiment, after completing the unified modeling and local feature enhancement, the fused voxel features are restored to their original multi-scale resolution through the corresponding downsampling or feature mapping path, and then input into the sparse detection head network. The detection head performs target classification and 3D bounding box regression through a lightweight sparse convolutional structure, thereby achieving efficient and accurate 3D target detection.

[0174] As can be seen from the above technical solution, the present invention achieves unified modeling of multi-scale features of sparse three-dimensional voxels without the need for point-by-point spatial alignment, and takes into account both long-distance dependency modeling capability and local spatial structure preservation capability, making it suitable for complex, large-scale sparse three-dimensional point cloud scenarios.

[0175] To verify the effectiveness of the proposed method, this embodiment conducted model training and testing on the widely used large-scale autonomous driving dataset nuScenes. The experimental environment was based on a computing platform configured with eight NVIDIA RTX 4090 GPUs, using the Adam optimizer for parameter updates, with an initial learning rate of 0.003 and a weight decay coefficient of 0.01. The model training cycle was 36 epochs, and the Balanced Group Sampling (CBGS) strategy was not used during training to directly evaluate the feature extraction and generalization capabilities of the model architecture itself.

[0176] In the comparative experiments, this embodiment selected several mainstream object detection algorithms as baselines, covering existing technologies based on pure convolution (CNN-based), Transformer-based, and State-space model (SSM). The evaluation metrics adopted were the average accuracy (mAP) and detection score (NDS) of the nuScenes dataset. mAP, calculated based on the ground plane center distance thresholds (0.5m, 1m, 2m, 4m) between the predicted bounding box and the ground truth bounding box, is used to evaluate localization accuracy. NDS is a weighted combination of mAP and five true positive metrics (translation, scale, orientation, velocity, and attributes), used to comprehensively quantify the model's detection quality of the target's 3D state.

[0177] Table 1. Comparison with state-of-the-art methods on the nuScenes validation set.

[0178]

[0179] To verify the effectiveness of each core module proposed in this embodiment, detailed ablation experiments were conducted on the nuScenes validation set, using the base network without any additional enhancement modules as the baseline model, as shown in Table 2.

[0180] Table 2 Ablation experiment results of different components on the nuScenes validation set.

[0181]

[0182] In Table 2, "multi-scale" represents the multi-scale unified modeling module, and "multi-kernel" represents the multi-kernel local feature enhancement module.

[0183] Furthermore, to explore the specific impact of spatial location embedding methods on model performance in the multi-scale unified modeling module, this embodiment compares five different embedding strategies, using non-positional embedding (None) as the baseline. The comparative experiments include: absolute positional encoding (abs), linear mapping (Linear), standard convolution (Conv), and the 3D submanifold convolution (SubConv3D) used in this embodiment. The experimental results are shown in Table 3.

[0184] Table 3 Ablation experiments with different local spatial embedding methods

[0185]

[0186] As shown in Table 3, while introducing simple absolute position encoding (abs) or linear mapping (Linear) brings some performance gains (mAP increases to 67.04% and 67.28% respectively), the improvement is relatively limited, indicating that simple numerical mapping alone is insufficient to fully capture complex 3D geometric information. It is worth noting that the standard convolution method (Conv) performs worse than linear mapping here, possibly due to different initialization strategies for default parameters. In contrast, SubConv3D can preserve the positional relationships between voxels to the maximum extent while maintaining the voxel space topology. It achieved the best experimental results, with mAP reaching 67.83% and NDS reaching 71.60%. Compared to the baseline model (None), mAP and NDS are significantly improved by 1.00% and 1.06%, respectively.

[0187] To determine the optimal internal structure of the local feature enhancement module, a comparative experiment was conducted on three different design variants of the module: the ordinary convolutional module (SBB), the multi-kernel local feature enhancement module (MKSB) proposed in this embodiment, and its lightweight variant based on depthwise separable convolution (DWMKSB). The results are shown in Table 4.

[0188] Table 4 Ablation experiments of multi-core local feature enhancement modules

[0189]

[0190] As shown in Table 4, although the baseline module SBB has 25.91M parameters, its performance is the lowest among the three (mAP is 67.10%). This indicates that simply stacking standard convolutions results in low parameter utilization when processing sparse 3D features, making it difficult to efficiently extract key geometric information.

[0191] The DWMKSB, which incorporates depthwise separable convolutions, demonstrates excellent performance. Compared to the baseline SBB, it reduces the number of parameters by approximately 16.7% (from 25.91M to 21.58M), while simultaneously improving mAP and NDS by 0.43% and 0.47%, respectively. This strongly demonstrates the rationale behind the multi-kernel selectivity mechanism design—even the lightweight version can surpass traditional convolutional baselines through superior feature interactions.

[0192] In pursuit of optimal detection accuracy, the full version of MKSB moderately increased the number of parameters by approximately 6.8% (27.67M) compared to the baseline, ultimately achieving the most significant performance gain (mAP improvement of 0.79%).

[0193] Furthermore, this embodiment also explores the impact of the number of multi-scale features on the model's detection performance. In the multi-scale unified modeling module, ablation experiments were conducted with different numbers of feature scales, and the results are shown in Table 5.

[0194] Table 5 Ablation experiments with the number of scales in the multi-scale unified modeling module.

[0195]

[0196] The results show that as the number of scales involved in the fusion increases, all detection metrics of the model exhibit a steady improvement trend. When using only a single-scale feature, the model's mAP is 67.39%. Although it possesses certain detection capabilities, it struggles to simultaneously capture long-range semantics and local geometric details. When the number of scales increases to 3, the model is able to aggregate richer contextual information from feature maps of different resolutions. Ultimately, the mAP improves to 67.83%, and the NDS improves to 71.60%.

[0197] Because the SSM model has the characteristic of causal sequence modeling, flattening 2D / 3D feature maps into 1D sequences (i.e., serialization strategy) significantly affects the model's ability to capture spatial neighborhood information. To mitigate the spatial information loss caused by a single scan order, this embodiment explores the effects of different scan curve combinations and scan curve rearrangement strategies. The experimental results are shown in Table 6.

[0198] Table 6 Ablation experiments with serialized curves and curve rearrangements

[0199]

[0200] By comparing combinations of Hilbert curves (H), transposed Hilbert curves (HT), Z-order curves (Z), and their transposes (ZT), Table 6 shows that when using only a single Hilbert curve (H), the mAP is 65.90%. Even with the introduction of the transposed variant (HT) of the same architecture, the metric does not show a significant improvement. However, by introducing scan curves with more directions (such as H+HT+Z), the model can capture spatial dependencies from different angles, and the mAP gradually increases to 66.81%. This demonstrates that multi-view scans can effectively compensate for feature discontinuities under a single viewpoint.

[0201] Comparing the last two rows in Table 6, we can see that by introducing the Shuffle mechanism and then turning it off at the end of training, the mAP can eventually reach 67.83% and the NDS can reach 71.60%.

[0202] like Figure 7 The image shows the effect of this method on the nuScenes dataset. The left side is the baseline effect, and the right side is the method proposed in this embodiment. Red represents the ground truth bounding boxes, and other colors represent the detection bounding boxes. It is clear that this method can effectively reduce false positives during the detection process. Figure 7 The area marked with a red circle.

[0203] Example 2:

[0204] This embodiment presents a target detection system based on unified modeling of multi-scale three-dimensional voxel features, such as... Figure 8 As shown, the system includes:

[0205] The data acquisition module is used to acquire the raw 3D point cloud data to be inspected.

[0206] The sparse voxelization module is used to sparse voxelize the original 3D point cloud data to obtain a sparse voxel representation.

[0207] The feature extraction module is used to extract features from the sparse voxel representation using a sparse convolutional coding network, and performs stepwise downsampling during the feature extraction process to obtain multi-scale voxel feature representations with different spatial resolutions.

[0208] The unified modeling module is used to input multi-scale voxel feature representations into a trained multi-scale unified modeling model to obtain enhanced multi-scale voxel features.

[0209] The feature mapping module is used to map the enhanced multi-scale voxel features to the same spatial resolution to obtain the final unified feature representation.

[0210] The target detection module is used to input the final unified feature representation into the Transformer-based detection head network to obtain the 3D target detection results.

[0211] Example 3:

[0212] This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the target detection method based on multi-scale three-dimensional voxel feature unified modeling.

[0213] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the target detection method based on unified modeling of multi-scale three-dimensional voxel features as described in the embodiments. It is understood that the electronic device may also include an input / output (I / O) interface and communication components.

[0214] The processor is used to execute all or part of the steps in the target detection method based on multi-scale three-dimensional voxel feature unified modeling as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in an electronic device, as well as application-related data.

[0215] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the target detection method based on unified modeling of multi-scale three-dimensional voxel features described in the above embodiments.

[0216] Example 4:

[0217] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0218] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the target detection method based on multi-scale three-dimensional voxel feature unified modeling described in the various embodiments of this application.

[0219] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) application stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the target detection method based on multi-scale three-dimensional voxel feature unified modeling described above.

[0220] Example 5:

[0221] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the target detection method based on multi-scale three-dimensional voxel feature unified modeling.

[0222] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0223] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0224] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A target detection method based on unified modeling of multi-scale three-dimensional voxel features, characterized in that, This method includes the following steps: The original 3D point cloud data is acquired and sparsed voxelized to obtain a sparse voxel representation. A sparse convolutional coding network is used to extract features from the sparse voxel representation, and stepwise downsampling is performed during the feature extraction process to obtain multi-scale voxel feature representations with different spatial resolutions. Construct and train a multi-scale unified modeling model to obtain a well-trained multi-scale unified modeling model. The multi-scale voxel feature representation is input into the trained multi-scale unified modeling model to obtain the enhanced multi-scale voxel features. The enhanced multi-scale voxel features are mapped to the same spatial resolution to obtain the final unified feature representation; The final unified feature representation is input into the Transformer-based detection head network to obtain the 3D target detection results.

2. The target detection method based on multi-scale three-dimensional voxel feature unified modeling according to claim 1, characterized in that, The multi-scale unified modeling model includes: The local spatial location embedding module is used to perform local spatial location embedding on voxel feature representations at different scales in multi-scale voxel feature representations. For each scale, the obtained local spatial location embedding is added to the voxel feature representation at that scale to obtain the position-enhanced voxel feature representation at that scale. The unified spatial representation module is used to map the location-enhanced voxel feature representations at different scales to the spatial coordinate system corresponding to the highest resolution, thereby obtaining a unified set of spatial coordinates. It then performs feature fusion by concatenating all the mapped voxel feature representations to obtain a joint feature representation. The multi-curve serialization input module is used to encode a unified set of spatial coordinates by using different space-filling curves, map the joint feature representation to a one-dimensional sequence space, and generate several one-dimensional voxel sequences. The dual-end state space modeling module is used to perform bidirectional state space modeling on each one-dimensional voxel sequence using a state space model, and to concatenate and fuse the obtained positive and negative output results to obtain a unified feature representation corresponding to each one-dimensional voxel sequence. The feature separation module is used to splice together the unified feature representations corresponding to all one-dimensional voxel sequences. By performing the inverse mapping in the multi-curve serialization input module, the spliced ​​unified feature representations are restored to a unified spatial coordinate system. Then, by performing the inverse process of the unified spatial representation module on the restored unified feature representations, multi-scale features that correspond one-to-one with the scales in the multi-scale voxel feature representations are obtained. The multi-kernel local feature enhancement module is used to divide multi-scale features into multiple sub-feature groups, model each sub-feature group using several parallel sub-manifold convolution branches, and concatenate and fuse the local features output by each branch to obtain enhanced multi-scale voxel features.

3. The target detection method based on multi-scale three-dimensional voxel feature unified modeling according to claim 2, characterized in that, The process of embedding the local spatial location is as follows: For arbitrary scales in multi-scale voxel feature representation The voxel feature representation is used to extract the non-empty set of voxel coordinates from the voxel feature representation. ; Based on the non-empty voxel coordinate set Generate the coordinate feature representation of the voxel feature representation; Using the coordinate feature representation as input, spatial neighborhood encoding of non-empty voxels in the voxel feature representation is performed using three-dimensional submanifold convolution to generate the local spatial location embedding of the voxel feature representation.

4. The target detection method based on multi-scale three-dimensional voxel feature unified modeling according to claim 2, characterized in that, The specific content of encoding a unified set of spatial coordinates using different space-filling curves, mapping the joint feature representation to a one-dimensional sequence space, and generating several one-dimensional voxel sequences is as follows: The unified set of spatial coordinates is encoded in parallel using at least two different space-filling curves. Where for any space-filling curve Using space-filling curves Encode each non-empty voxel coordinate in a unified set of spatial coordinates to obtain a sequence of values ​​for all non-empty voxel coordinates; Sort the sequence values ​​of all non-empty voxel coordinates, and assign a serialization index to each non-empty voxel in the joint feature representation based on the sorting result; Based on the serialized index of all non-empty voxels in the joint feature representation, all non-empty voxels in the joint feature representation are arranged to generate a one-dimensional voxel sequence.

5. The target detection method based on multi-scale three-dimensional voxel feature unified modeling according to claim 2, characterized in that, The dual-end state space modeling module includes two parallel and identical state space model branches: a forward state space model branch and a reverse state space model branch. The forward state space model branch is used to update the state of the one-dimensional voxel sequence in the order of the one-dimensional voxel sequence and generate a forward output result. The reverse state space model branch is used to update the state of the one-dimensional voxel sequence in reverse order to generate the reverse output result. The forward and backward outputs are concatenated along the feature dimension, and the concatenated outputs are then subjected to linear projection, normalization, and random path discarding to obtain the global features. Then, through residual connection, the global features are concatenated to the one-dimensional voxel sequence to obtain the unified feature representation corresponding to the one-dimensional voxel sequence.

6. The target detection method based on multi-scale three-dimensional voxel feature unified modeling according to claim 2, characterized in that, The process involves dividing multi-scale features into multiple sub-feature groups, modeling each sub-feature group using several parallel sub-manifold convolutional branches, and concatenating and fusing the local features output by each branch to obtain the enhanced multi-scale voxel features. For arbitrary scales in multi-scale features Features , for features Perform batch normalization, and then divide the batch-normalized features into channels. Sub-feature groups; use Three-dimensional convolution of submanifolds with different kernel sizes, respectively. Local modeling is performed on each sub-feature group to obtain Each sub-feature group corresponds to a convolutional kernel size. Will Local features are concatenated along the channel dimension and then processed by a convolution kernel. Sub-manifold convolution performs cross-channel fusion of the concatenated local features to obtain the fused features; A feedforward neural network (FFN) is used to perform a nonlinear transformation on the fused features, and the features obtained from the nonlinear transformation are then connected to the feature vector through residual connections. Add them together to obtain the features. Enhanced voxel features.

7. The target detection method based on multi-scale three-dimensional voxel feature unified modeling according to claim 6, characterized in that, The submanifold 3D convolution is implemented using depth-separable submanifold convolution.

8. The target detection method based on multi-scale three-dimensional voxel feature unified modeling according to claim 2, characterized in that, The training method for the multi-scale unified modeling model is as follows: Acquire several sets of original 3D point cloud data, and perform sparse voxelization on each set of original 3D point cloud data to obtain sparse voxel representation. A sparse convolutional coding network is used to extract features from the sparse voxel representation, and downsampling is performed stepwise during the feature extraction process to obtain multi-scale voxel feature representations with different spatial resolutions, thereby constructing a training dataset. Set the initial learning rate Training batch size and training rounds The Adam optimizer is used to train the multi-scale unified model by inputting the training dataset into it. For the former In any training epoch, within the multi-curve serialization input module, a dynamic shuffle mechanism is used to randomly arrange the order of all space-filling curves, and the unified set of spatial coordinates is encoded according to the generated random order, thereby generating several one-dimensional voxel sequences; for the subsequent... In any batch of training batches, the unified set of spatial coordinates is encoded in the multi-curve serialization input module according to the fixed order preset by all space-filling curves, thereby generating several one-dimensional voxel sequences. After training is completed, a trained multi-scale unified modeling model is obtained.

9. The target detection method based on multi-scale three-dimensional voxel feature unified modeling according to claim 1, characterized in that, The specific content of mapping the enhanced multi-scale voxel features to the same spatial resolution to obtain the final unified feature representation is as follows: the enhanced multi-scale voxel features are uniformly mapped to the highest spatial resolution, and the mapped unified voxel features are fused to obtain the final unified feature representation.

10. A target detection system based on unified modeling of multi-scale three-dimensional voxel features, used to implement the target detection method based on unified modeling of multi-scale three-dimensional voxel features as described in any one of claims 1-9, characterized in that, The system includes: The data acquisition module is used to acquire the raw 3D point cloud data to be detected; The sparse voxelization module is used to sparse voxelize the original 3D point cloud data to obtain a sparse voxel representation. The feature extraction module is used to extract features from the sparse voxel representation using a sparse convolutional coding network, and to perform stepwise downsampling during the feature extraction process to obtain multi-scale voxel feature representations with different spatial resolutions. The unified modeling module is used to input multi-scale voxel feature representations into a trained multi-scale unified modeling model to obtain enhanced multi-scale voxel features. The feature mapping module is used to map the enhanced multi-scale voxel features to the same spatial resolution to obtain the final unified feature representation; The object detection module is used to input the final unified feature representation into the Transformer-based detection head network to obtain the 3D object detection results.

Images