Lidar three-dimensional target detection method based on sparse feature refinement and alignment

Abstract

The application discloses a laser radar three-dimensional target detection method based on sparse feature refinement and alignment. The method voxelizes point clouds, extracts step-4 and step-8 sparse features; at step-8 activated coordinates, performs gated bounded residual refinement according to neighborhood support rate, input features and context features to obtain refined sparse features; aligns and multi-scale aggregates multi-step deep sparse features, performs height dimension compression and resolution recovery to obtain step-4 bird's eye view semantic candidate features; constructs observation fine-grained evidence according to step-4 sparse features and queries at candidate coordinates, performs bounded gated fusion according to compatibility when the evidence exists, and retains original semantic candidate features when the evidence does not exist, obtains evidence-constrained step-4 bird's eye view fusion features and takes the step-4 fusion features as step-4 detection features; inputs the step-4 detection features into a detection prediction layer and outputs target categories and three-dimensional bounding boxes. The method improves detection stability in sparse, long-distance and small target scenes.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving environment perception, LiDAR point cloud processing, and 3D target detection technology, specifically to a LiDAR 3D target detection method based on sparse feature refinement and alignment. Background Technology

[0002] Autonomous vehicles typically use LiDAR to acquire 3D point clouds of the surrounding road environment and identify the categories, locations, sizes, and orientations of traffic participants such as vehicles, pedestrians, and cyclists from the point clouds. Compared to 2D images, LiDAR point clouds can directly provide measurements of depth and 3D geometric information; therefore, LiDAR-based 3D target detection is an important component of autonomous driving perception systems.

[0003] Point clouds exhibit a highly sparse and irregular distribution in 3D space. Voxelization detection methods typically first quantize the coordinates of continuous points to a regular voxel grid, and then perform sparse convolution only on valid voxels containing points to avoid unnecessary computation on a large amount of blank space. Fully sparse detection networks further extend the sparsity processing to the bird's-eye view detection stage, achieving high computational efficiency.

[0004] However, lidar echo density decreases with increasing measurement distance, and distant targets often retain only a few reflection points; occlusion, viewing angle, and reflection characteristics can also cause incomplete local structures of the target. This degradation propagates to intermediate sparse voxel features, making the feature representation of existing activated voxels unreliable. Directly using a fixed convolutional response for correction can easily introduce overly strong or unstable contextual information at locations with insufficient neighborhood support.

[0005] On the other hand, after continuous downsampling by a sparse convolutional backbone network, small targets such as pedestrians and cyclists may only occupy a few grid points in a bird's-eye view. The coarser detection stride weakens the spatial resolution required for center localization and bounding box regression of small targets. Simply upsampling deep features densely increases the computational cost for large blank areas; directly generating new 3D voxels may misinterpret semantic inference as real geometric observations.

[0006] Shallow step size 4 features offer fine spatial resolution, but their semantic level is low and they only exist at locations supported by actual observations. Deep step size 8 and coarser-scale features have strong semantics but lack fine-grained localization evidence. A key challenge in improving the robustness of detection in scenes with long distances, small targets, and incomplete local structures is how to establish a limited number of 2D candidate grid points using deep semantics without synthesizing unobserved 3D geometry, and inject shallow, fine-grained evidence only at locations with matching coordinates and confirmed observations.

[0007] Therefore, there is a need for a lidar 3D target detection method that retains the advantages of full sparse processing, can refine supported and constrained features on existing 3D active coordinates, establishes finer bird's-eye view semantic candidate grids, and integrates fine-grained evidence from real observations through coordinate matching. Summary of the Invention

[0008] The present invention aims to solve the problem of unstable detection of distant targets, small targets, and some observed targets in the three-dimensional target detection of existing sparse lidar, which is caused by sparse range correlation, incomplete local structure, and insufficient spatial evidence of small targets due to continuous downsampling.

[0009] This invention further addresses the problems of how to avoid overly strong corrections at unsupported locations when enhancing sparse features, how to distinguish between two-dimensional semantic candidates and real three-dimensional observations, and how to maintain the original semantics at candidate locations without fine-grained observational evidence without synthesizing false evidence.

[0010] To address the aforementioned technical problems, this invention provides a lidar 3D target detection method based on sparse feature refinement and alignment, comprising the following steps:

[0011] S1, acquire the lidar point cloud, perform voxelization on the lidar point cloud, and extract three-dimensional sparse features with different spatial resolutions through a sparse convolutional backbone network to obtain sparse features with stride of 4 and stride of 8 and their effective voxel coordinates.

[0012] S2, on the existing active voxel coordinates of the sparse feature tensor with a step size of 8, generate spatial thinning gate, channel thinning gate and thinning residual based on the neighborhood support rate of the active voxel, input features and context features, perform bounded residual mapping on the thinning residual, and update the input features at the existing active voxel coordinates using the mapped thinning residual, to obtain thinning sparse features with the coordinate set unchanged.

[0013] S3, continue to extract layered features from the refined sparse features, align deep sparse features of different step lengths to a uniform spatial resolution and perform multi-scale aggregation; perform height-dimensional compression and resolution restoration on the aggregated three-dimensional sparse features to obtain bird's-eye view semantic candidate features with a step size of 4.

[0014] S4, perform high-level aggregation and support-preserving encoding on the sparse feature tensor of step 4 to obtain observation fine-grained evidence; query the observation fine-grained evidence at the candidate coordinates of the semantic candidate features of the step 4 bird's-eye view, and when the observation fine-grained evidence is found, generate fusion gating and fusion residual based on the semantic candidate features and the observation fine-grained evidence, perform bounded residual mapping on the fusion residual and fuse it to the semantic candidate features to obtain the step 4 detection features;

[0015] S5, input the detection features of step size 4 into the detection prediction layer, perform target category prediction and three-dimensional bounding box parameter regression, and obtain the three-dimensional target detection result.

[0016] Further, in step S2, the context features are obtained by encoding the stride 8 sparse feature tensor by one or more sub-manifold sparse convolutional layers, and the input activation coordinate set and output activation coordinate set of the sub-manifold sparse convolutional layers are the same.

[0017] Further, in step S2, for the i-th active voxel, the ratio of the number of coordinates belonging to the set of active coordinates of the step size 8 sparse feature tensor in the three-dimensional 26-neighborhood centered on the active voxel to 26 is determined as the neighborhood support rate.

[0018] Further, in step S2, the normalized result of the input feature, the normalized result of the context feature, the normalized result of the difference between the input feature and the context feature, and the neighborhood support rate are concatenated to obtain a refined descriptor; based on the refined descriptor, the spatial refined gate, the channel refined gate, and the refined residual are obtained respectively, wherein the spatial refined gate is obtained by multiplying the neighborhood support rate with the Sigmoid function value of the refined descriptor after a learnable transformation, so as to suppress the context refined intensity when the neighborhood support rate decreases.

[0019] Furthermore, the bounded residual mapping in steps S2 and S4 preserves the direction of the residual vector and uses the residual radius and hyperbolic tangent function to smooth and saturate the L2 norm of the residual vector so that the L2 norm of the feature update amount does not exceed the corresponding residual radius.

[0020] Further, in step S3, the sparse features of step 16 and step 32 are aligned to the reference coordinates of step 8, and sparse feature union and high aggregation are performed with the refined sparse features to obtain deep fusion semantic features; the two planar coordinate components of the bird's-eye view coordinates of step 8 are multiplied by 2 respectively to map to the bird's-eye view space of step 4, and local sparse expansion is performed in the two-dimensional candidate space through regular sparse bird's-eye view convolution.

[0021] Further, in step S4, the height aggregation aggregates three-dimensional sparse features with the same batch index and planar coordinates along the height direction; the support-preserving encoding adopts sub-manifold sparse bird's-eye view convolution, so that the observation fine-grained evidence exists only in the bird's-eye view coordinates supported by the stride 4 sparse feature tensor.

[0022] Further, in step S4, the semantic candidate features and the observational fine-grained evidence are projected onto a common compatibility space, and the cosine compatibility between the two is calculated. The semantic candidate features, the observational fine-grained evidence, the difference between the two, the cosine compatibility, and the binary variable indicating whether the corresponding candidate coordinates have observational fine-grained evidence are concatenated to form a fusion descriptor.

[0023] Further, a channel fusion gate and a fusion residual are generated based on the fusion descriptor, and the channel fusion gate is multiplied by the binary variable; when the binary variable is 0, the channel fusion gate is set to zero and the semantic candidate feature is retained; when the binary variable is 1, the semantic candidate feature is updated using the fusion residual mapped by the bounded residual.

[0024] Furthermore, during the training phase, effective coordinates are selected within the bird's-eye view of the target's 3D bounding box using 3D object detection annotations. Semantic embeddings and evidence embeddings at the same coordinates are used as positive sample pairs, while embeddings belonging to other target instances are used as negative samples. The bidirectional instance-aware cross-branch contrast loss is calculated, and the bidirectional instance-aware cross-branch contrast loss and the target detection loss are weighted and summed to train the network.

[0025] Compared with the prior art, the present invention has at least the following beneficial effects:

[0026] (1) Supports the refinement of the perceptual context to update only the features of existing 3D active voxels, and uses the neighborhood support rate to suppress the spatial gating of low support locations, which can reduce the risk of introducing unreliable context correction in structurally incomplete or extremely sparse regions.

[0027] (2) Bounded residual mapping restricts the L2 norm of refinement update and fusion update in a smooth saturation manner, which avoids excessive updates caused by individual abnormal features while maintaining the residual direction, thus improving the stability of feature processing.

[0028] (3) By integrating the bird’s-eye view sparse upsampling, the deep multi-scale semantics are transferred to the step size 4 two-dimensional candidate space, which improves the resolution of the bird’s-eye view used for small target center localization and three-dimensional bounding box regression, without creating new three-dimensional activation voxels, thus avoiding mistaking two-dimensional semantic candidates as real three-dimensional observations.

[0029] (4) Semantic-guided fine-grained evidence fusion uses candidate coordinate precise query and evidence presence markers to inject shallow fine-grained evidence only at locations supported by actual observations; semantic candidate features are maintained at locations without evidence to reduce the interference of pseudo-fine-grained evidence on detection.

[0030] (5) The compatibility of semantics and evidence embedding participates in the fusion gating, and the bidirectional instance perception cross-branch comparison supervision further utilizes the existing 3D detection annotation to constrain the cross-branch correspondence of the same target position without the need for additional point cloud completion labels or pseudo-geometric labels.

[0031] (6) The final detection features maintain a sparse representation and reuse the sparse prediction layer, which can output the categories and three-dimensional bounding boxes of targets such as vehicles, pedestrians and cyclists, and is suitable for autonomous driving environment perception. Attached Figure Description

[0032] Figure 1 This is a flowchart of the lidar 3D target detection method based on sparse feature refinement and alignment according to the present invention;

[0033] Figure 2 This is a schematic diagram of the overall architecture for sparse feature refinement and resolution alignment in this invention;

[0034] Figure 3 This is a schematic diagram of the perceptual context refinement module of the present invention;

[0035] Figure 4 This is a schematic diagram illustrating the fusion of sparse upsampling of bird's-eye view and fine-grained evidence in this invention;

[0036] Figure 5 This is a schematic diagram of the network training process of the present invention;

[0037] Figure 6 The figure shows the experimental results of a robust dataset based on real vehicle data. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings. It should be noted that the following specific embodiments are used to illustrate the present invention, and not to limit the scope of protection of the present invention; equivalent processing methods that can achieve the same technical effect can be adopted without changing the inputs, outputs and interdependencies of each step.

[0039] Reference Figure 1 As shown, a LiDAR 3D target detection method based on sparse feature refinement and alignment includes the following steps:

[0040] S1, acquire the lidar point cloud, perform voxelization on the lidar point cloud, and extract three-dimensional sparse features with different spatial resolutions through a sparse convolutional backbone network to obtain sparse features with stride of 4 and stride of 8 and their effective voxel coordinates.

[0041] S2, on the existing active voxel coordinates of the sparse feature tensor with a step size of 8, generate spatial thinning gate, channel thinning gate and thinning residual based on the neighborhood support rate of the active voxel, input features and context features, perform bounded residual mapping on the thinning residual, and update the input features at the existing active voxel coordinates using the mapped thinning residual, to obtain thinning sparse features with the coordinate set unchanged.

[0042] S3, continue to extract layered features from the refined sparse features, align deep sparse features of different step lengths to a uniform spatial resolution and perform multi-scale aggregation; perform height-dimensional compression and resolution restoration on the aggregated three-dimensional sparse features to obtain bird's-eye view semantic candidate features with a step size of 4.

[0043] S4, perform high-level aggregation and support-preserving encoding on the sparse feature tensor of step 4 to obtain observation fine-grained evidence; query the observation fine-grained evidence at the candidate coordinates of the semantic candidate features of the step 4 bird's-eye view, and when the observation fine-grained evidence is found, generate fusion gating and fusion residual based on the semantic candidate features and the observation fine-grained evidence, perform bounded residual mapping on the fusion residual and fuse it to the semantic candidate features to obtain the step 4 detection features;

[0044] S5, input the detection features of step size 4 into the detection prediction layer, perform target category prediction and three-dimensional bounding box parameter regression, and obtain the three-dimensional target detection result.

[0045] The following is a detailed explanation of each step.

[0046] Step S1 is as follows:

[0047] like Figure 2 As shown, the point coordinates are transformed to a preset reference coordinate system according to the lidar calibration parameters, points outside the preset sensing range are filtered out, and the three-dimensional coordinates, reflection intensity, echo information, or a combination thereof of the points are determined as point features. The point cloud is spatially quantized according to a preset voxel size, and the point features falling into the same voxel are aggregated to form an input sparse tensor composed of the active voxel coordinates and voxel features, the expression of which is:

[0048]

[0049] in, Step size The corresponding sparse feature tensor; The cumulative step size relative to the input voxel grid, which can be 4, 8, 16, or 32; For the first The coordinates of each activated voxel; For the first The feature vector corresponding to each activation voxel; Step size The number of activated voxels; To activate the voxel index. The activated voxel coordinates include the batch index, the height index, and two planar indices.

[0050] The input sparse tensor is fed into a sparse convolutional backbone network to obtain sparse features with stride 4, stride 8, and deeper layers. The stride 4 sparse features are used to form fine-grained evidence of observation, and the stride 8 sparse features are used to support the refinement of perceptual context.

[0051] Step S2 is as follows:

[0052] As shown in Figure 3, support-aware context refinement is performed on the existing activation voxel coordinates of the sparse features with a stride of 8. Submanifold sparse convolution is used to obtain context features with the same set of activation coordinates as the input features; for the ... For each active voxel, the neighborhood support rate is determined based on the number of active coordinates in its three-dimensional neighborhood. The expression for this rate is:

[0053]

[0054] in, For the first Neighborhood support of each activated voxel; For coordinates A 3D 26-neighborhood centered on the center and excluding the center location; The set of activation coordinates for a sparse feature tensor with a step size of 8; symbol The intersection of sets is represented by the symbol. This represents the number of elements in the set, where 26 is the total number of positions in the three-dimensional neighborhood. Neighborhood support rate. The value ranges from 0 to 1 and is used to characterize the degree of local observational support for the current voxel.

[0055] The input features, context features, their differences, and neighborhood support are combined to form a refined descriptor, the expression of which is:

[0056]

[0057] in, For the first A refined descriptor for each active voxel; For layer normalization operation; For the first Input features of each activation voxel; Context features that have the same activation coordinates as the input features; The value represents the neighborhood support rate; the semicolon indicates that the features are concatenated. The refinement descriptor is used to simultaneously characterize the input features, context features, the difference between the two, and the degree of support from local observations.

[0058] Spatial refinement gate, channel refinement gate, and refinement residual are generated based on the refinement descriptor. The spatial refinement gate is modulated by the neighborhood support rate, and its expression is as follows:

[0059]

[0060] in, For the first Spatial refinement gating of each active voxel; σ(·) is the Sigmoid function; Learnable local transformations for processing refinement descriptors; To refine the descriptor, neighborhood support is used. Modulating the spatial refinement gate allows for smaller context correction strengths at locations with insufficient local observation support.

[0061] To prevent excessively large residuals from corrupting existing observation characteristics, a bounded residual mapping that preserves the residual direction is performed on the refined residuals. The expression for the bounded residual mapping is:

[0062]

[0063] in, The residual radius is Bounded residual mapping; Let be the residual vector to be mapped; The residual radius is used to limit the residual amplitude; tanh(·) is the hyperbolic tangent function; It is a norm 2. To prevent positive numbers with a denominator of zero, bounded residual mapping preserves the direction of the residual vector and ensures that the L2 norm of the mapping result is no greater than the residual radius. .

[0064] By combining bounded residuals with spatial refinement gating, channel refinement gating, and global refinement scale, the input features at existing active voxels are updated. The expression for the refined voxel features is as follows:

[0065]

[0066] in, For the first The refined features of each activated voxel; To refine the scale globally; To refine the gate control for the channel, For learnable channel gating transformation; This represents element-wise multiplication; For the refined residuals that have not been mapped to bounded residuals, This is a learnable residual prediction transformation; To refine the residual radius, this formula uses spatial gating, channel gating, and the residual radius to jointly constrain feature updates while maintaining the set of active coordinates unchanged before and after refinement.

[0067] Therefore, step S2 only changes the feature values ​​at existing active voxels and does not generate new three-dimensional voxel coordinates; when the neighborhood support is low, the spatial refinement gate is reduced accordingly to limit feature updates at locations with insufficient observation support.

[0068] Step S3 is as follows:

[0069] like Figure 4 As shown, the refined stride 8 sparse features are further subjected to sparse convolution processing. Stride 16, stride 32, or other deep sparse features are aligned to the stride 8 reference coordinates and then sparsely joined and highly aggregated with the refined stride 8 features to obtain deep fused semantic features, the expression of which is:

[0070]

[0071] in, Deep fusion semantic features with a step size of 8; For semantic mapping; A bird's-eye view projection that converges along the height direction; This represents the refined step size 8 sparse feature; The scale alignment operator aligns features with a step size of 16 to reference coordinates with a step size of 8. A scale alignment operator for aligning features with a step size of 32 to reference coordinates with a step size of 8; This represents a sparse union on the aligned coordinates; For a sparse feature tensor with a step size of 16, Let be a sparse feature tensor with a step size of 32. This formula is used to fuse deep semantic information at different scales under a unified reference coordinate system.

[0072] The planar coordinates of the deep fusion semantic features are mapped to the stride 4 bird's-eye view space, and then a finite local expansion is performed around the mapped location through regular sparse bird's-eye view convolution to form the stride 4 bird's-eye view semantic candidate features, the expression of which is:

[0073]

[0074] in, For the semantic candidate features of the bird's-eye view with a step size of 4; It is a linear rectified function; Convolution of a regular sparse bird's-eye view with a kernel size of 3×3; This is the semantic feature obtained by mapping the deep fusion semantic features to the bird's-eye view space with a stride of 4. This formula only performs a limited local expansion in the two-dimensional bird's-eye view candidate space and does not add any new three-dimensional activation voxels.

[0075] Local expansion occurs only in the two-dimensional semantic candidate space and is used to improve the candidate sampling resolution of the detection center without changing the three-dimensional set of activation voxel coordinates.

[0076] Step S4 is as follows:

[0077] like Figure 4 As shown, the 3D sparse features with a step size of 4 in step S1 are aggregated along the height direction, and observational fine-grained evidence is obtained by an evidence encoder that keeps the set of activated coordinates unchanged. For each semantic candidate coordinate, a precise query is performed in the observational fine-grained evidence, and the evidence features and evidence presence markers are determined based on the query results. The expressions are as follows:

[0078]

[0079] in, For the first Evidence features obtained by querying at each semantic candidate coordinate. This serves as a marker of the existence of evidence; To provide fine-grained evidence for step size 4; For the first Four semantic candidate coordinates with a step size of 4; The set of activation coordinates for observing fine-grained evidence; This is a zero vector with the same dimension as the evidence features. When the candidate coordinates belong to the set of activation coordinates... Select 1 and read the corresponding evidence feature; otherwise... Take 0 and return the zero vector.

[0080] The semantic candidate features and the evidence features obtained from the query are mapped to a common compatibility space, and the compatibility degree between the two is calculated. The expression for the compatibility degree is as follows:

[0081]

[0082] in, For the first Cross-branch cosine compatibility at each semantic candidate coordinate; The semantic embedding is obtained by mapping semantic candidate features. The evidence embedding obtained by evidence feature mapping, Represents the dot product of vectors; This is used to prevent positive numbers with a denominator of zero. Compatibility. It is used to characterize the degree of consistency between semantic candidate features and actual observational evidence.

[0083] A fusion gate is generated based on semantic candidate features, evidence features, their differences, compatibility, and evidence presence indicators. The expression for the fusion gate is:

[0084]

[0085] in, For the first Channel fusion gating at semantic candidate coordinates For learnable gating transformations, This is a fusion descriptor composed of semantic candidate features, evidential features, their differences, and compatibility. When evidence is absent... Set to 0, thereby merging the channel gating. Set to zero.

[0086] The fused residuals obtained through bounded residual mapping are superimposed onto the semantic candidate features to obtain the step-size 4 detection features, the expression of which is:

[0087]

[0088] in, For the first The step size 4 detection features are fused at each semantic candidate coordinate. For the first Semantic candidate features at each semantic candidate coordinate; For global fusion scale; The fused residual is the one that has not been mapped to a bounded residual. To fuse residual radii, this formula only performs amplitude-limited residual updates on semantic candidate features at locations where actual observational evidence exists.

[0089] When evidence is absent, the semantic candidate features remain unchanged; when evidence is present, the fusion update amount is jointly constrained by the fusion scale and the fusion residual radius, and its expression is:

[0090]

[0091] Among them, when When the value is 0, the fused detection features Preserve as original semantic candidate features ;when When the value is 1, the L2 norm of the merged update is no greater than 1. .

[0092] Step S5 is as follows:

[0093] The detection features with a step size of 4 are input into the detection prediction layer. The classification branch outputs the target category response, and the bounding box regression branch outputs the target center offset, target height, target size, and target orientation. The decoded 3D bounding boxes are filtered and duplicate boxes are suppressed to obtain the target category and the target 3D bounding box detection results.

[0094] like Figure 5As shown, during network training, valid coordinates are selected within the internal region after the target's 3D bounding box is projected onto the bird's-eye view. Semantic embeddings and evidence embeddings at the same coordinates are used as positive sample pairs, and embeddings of other target instances are used as negative samples. The bidirectional instance-aware cross-branch contrastive loss is calculated and weighted and summed with the target detection loss. The expression for the total network training loss is:

[0095]

[0096] in, This represents the total loss during network training. The target detection loss includes target center classification loss and 3D bounding box regression loss; Cross-branch contrast loss for bidirectional instance awareness. This formula represents the weighting coefficients for the bidirectional instance-aware cross-branch contrastive loss. It is used to maintain cross-branch consistency constraints between semantic embeddings and evidence embeddings at the same target location while preserving the constraints of the target detection task.

[0097] Specifically, valid coordinates are selected within the bird's-eye view region corresponding to the target's 3D bounding box. Normalized semantic embeddings and normalized evidence embeddings at the same coordinates are used as positive sample pairs, while embeddings belonging to other target instances are used as negative samples. The contrast loss in both directions is calculated using semantic embeddings and evidence embeddings as anchor points, and the bidirectional instance-aware cross-branch contrast loss is determined according to the following formula:

[0098]

[0099] in, Cross-branch contrast loss for bidirectional instance perception; For the set of valid coordinate indices, The number of valid coordinates; Valid coordinate index; The contrast loss is used to calculate the difference between the semantic branch and the evidence branch. This is the contrastive loss from the evidence branch to the semantic branch. Through bidirectional contrast constraints, semantic embeddings and evidence embeddings at the same coordinate are matched, and embeddings belonging to different target instances are distinguished. The bidirectional instance-aware cross-branch contrastive loss is only used during the network training phase.

[0100] After training, the network parameters are fixed. During the inference phase, the point cloud to be detected is processed according to steps S1 to S5. There is no need to read the 3D target annotation or calculate the bidirectional instance perception cross-branch contrast loss.

[0101] To verify the performance of the method of this invention in 3D target detection in a real vehicle multi-LiDAR scenario, a multi-LiDAR dataset based on real vehicle data was constructed. The dataset was collected by multiple LiDARs mounted on the test vehicle during road travel. After time synchronization, extrinsic parameter calibration, and vehicle-wide coordinate system transformation, a multi-LiDAR point cloud was formed, and 3D bounding boxes were labeled for vehicles, pedestrians, and cyclists in the point cloud.

[0102] Under the same data partitioning, training conditions, and evaluation rules, the VoxelNeXt method and the method of this invention were used to train and test multiple LiDAR datasets, respectively. Vehicles, pedestrians, and cyclists were used as detection categories, and the average precision (AP) for each target category and the mean of the average precision for the three target categories were calculated. The average precision (AP) is used to comprehensively characterize the 3D target detection performance.

[0103] Experimental results are as follows Figure 6 As shown, the average accuracy of the method of the present invention on three types of targets—vehicles, pedestrians, and cyclists—and the mean of the average accuracy of the three types of targets are all higher than that of the VoxelNeXt method, indicating that the method of the present invention has better three-dimensional target detection performance under the condition of multiple lidar point clouds collected from real vehicles.

Claims

1. A lidar 3D target detection method based on sparse feature refinement and alignment, characterized in that, Includes the following steps: S1, acquire the lidar point cloud, perform voxelization on the lidar point cloud, and extract three-dimensional sparse features with different spatial resolutions through a sparse convolutional backbone network to obtain sparse features with stride of 4 and stride of 8 and their effective voxel coordinates. S2, on the existing active voxel coordinates of the sparse feature tensor with a step size of 8, generate spatial thinning gate, channel thinning gate and thinning residual based on the neighborhood support rate of the active voxel, input features and context features, perform bounded residual mapping on the thinning residual, and update the input features at the existing active voxel coordinates using the mapped thinning residual, to obtain thinning sparse features with the coordinate set unchanged. S3, continue to extract layered features from the refined sparse features, align deep sparse features of different step lengths to a uniform spatial resolution and perform multi-scale aggregation; perform height-dimensional compression and resolution restoration on the aggregated three-dimensional sparse features to obtain bird's-eye view semantic candidate features with a step size of 4. S4, perform high-level aggregation and support-preserving encoding on the sparse feature tensor of step 4 to obtain observation fine-grained evidence; query the observation fine-grained evidence at the candidate coordinates of the semantic candidate features of the step 4 bird's-eye view, and when the observation fine-grained evidence is found, generate fusion gating and fusion residual based on the semantic candidate features and the observation fine-grained evidence, perform bounded residual mapping on the fusion residual and fuse it to the semantic candidate features to obtain the step 4 detection features; S5, input the detection features of step size 4 into the detection prediction layer, perform target category prediction and three-dimensional bounding box parameter regression, and obtain the three-dimensional target detection result.

2. The lidar 3D target detection method based on sparse feature refinement and alignment according to claim 1, characterized in that, In step S2, the context features are obtained by encoding the stride 8 sparse feature tensor by one or more sub-manifold sparse convolutional layers, and the input activation coordinate set and output activation coordinate set of the sub-manifold sparse convolutional layers are the same.

3. The lidar 3D target detection method based on sparse feature refinement and alignment according to claim 1, characterized in that, In step S2, for the i-th activation voxel, the ratio of the number of coordinates belonging to the set of activation coordinates of the step size 8 sparse feature tensor in the three-dimensional 26-neighborhood centered on the activation voxel to 26 is determined as the neighborhood support rate.

4. The lidar 3D target detection method based on sparse feature refinement and alignment according to claim 1, characterized in that, In step S2, the normalized result of the input feature, the normalized result of the context feature, the normalized result of the difference between the input feature and the context feature, and the neighborhood support rate are concatenated to obtain a refined descriptor. The spatial refinement gate, the channel refinement gate, and the refinement residual are obtained based on the refinement descriptor, wherein the spatial refinement gate is obtained by multiplying the neighborhood support rate with the Sigmoid function value of the refinement descriptor after a learnable transformation, so as to suppress the context refinement intensity when the neighborhood support rate decreases.

5. The lidar 3D target detection method based on sparse feature refinement and alignment according to claim 1, characterized in that, In steps S2 and S4, the bounded residual mapping preserves the direction of the residual vector and uses the residual radius and hyperbolic tangent function to smooth and saturate the L2 norm of the residual vector so that the L2 norm of the feature update does not exceed the corresponding residual radius.

6. The lidar 3D target detection method based on sparse feature refinement and alignment according to claim 1, characterized in that, In step S3, the sparse features of step 16 and step 32 are aligned to the reference coordinates of step 8, and sparse feature union and high aggregation are performed with the refined sparse features to obtain deep fusion semantic features; the two planar coordinate components of the bird's-eye view coordinates of step 8 are multiplied by 2 to map to the bird's-eye view space of step 4, and local sparse expansion is performed in the two-dimensional candidate space through regular sparse bird's-eye view convolution.

7. The lidar 3D target detection method based on sparse feature refinement and alignment according to claim 1, characterized in that, In step S4, the height aggregation aggregates three-dimensional sparse features with the same batch index and planar coordinates along the height direction; the support-preserving encoding adopts sub-manifold sparse bird's-eye view convolution, so that the observation fine-grained evidence exists only in the bird's-eye view coordinates supported by the stride 4 sparse feature tensor.

8. The lidar 3D target detection method based on sparse feature refinement and alignment according to claim 1, characterized in that, In step S4, the semantic candidate features and the observational fine-grained evidence are projected onto a common compatibility space, and their cosine compatibility is calculated. The semantic candidate features, the observational fine-grained evidence, the difference between the two, the cosine compatibility, and the binary variable indicating whether the corresponding candidate coordinates have observational fine-grained evidence are concatenated to form a fusion descriptor.

9. The lidar three-dimensional target detection method based on sparse feature refinement and alignment according to claim 8, characterized in that, The channel fusion gate and fusion residual are generated based on the fusion descriptor, and the channel fusion gate is multiplied by the binary variable. When the binary variable is 0, the channel fusion gate is set to zero and the semantic candidate feature is retained. When the binary variable is 1, the semantic candidate feature is updated using the fusion residual mapped by the bounded residual.

10. The lidar three-dimensional target detection method based on sparse feature refinement and alignment according to claim 8, characterized in that, During the training phase, effective coordinates are selected in the bird's-eye view region of the target's 3D bounding box using 3D object detection annotation. Semantic embeddings and evidence embeddings at the same coordinates are used as positive sample pairs, and embeddings belonging to other target instances are used as negative samples. The bidirectional instance perception cross-branch contrast loss is calculated, and the bidirectional instance perception cross-branch contrast loss and the target detection loss are weighted and summed to train the network.

Images