A LiDAR 3D Target Detection Method Integrating Hierarchical Vertical Prior and Multi-Attention Adaptive Fusion

By optimizing the hierarchical vertical prior detection framework through adaptive non-uniform voxel partitioning, lightweight residual dual-branch backbone network, and multi-attention SIF fusion module, the problems of vertical feature loss and high computational overhead in LiDAR 3D target detection are solved, achieving high-precision and low-latency target detection for autonomous driving.

CN123071968APending Publication Date: 2026-11-10ANHUI NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611378157.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-07
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing LiDAR 3D target detection algorithms have shortcomings in areas such as loss of vertical dimension information compression, target adhesion in dense scenes, high computational overhead, and poor adaptability to rotating targets, making it difficult to meet the real-time and accuracy requirements of autonomous driving.

Method used

We employ adaptive non-uniform voxel partitioning, lightweight residual dual-branch backbone network, improved multi-attention SIF fusion module, and multi-scale neck aggregation network, combined with rotational blur NMS post-processing, to optimize the hierarchical vertical prior detection framework, thereby enhancing feature fusion capability and detection accuracy.

Benefits of technology

While ensuring real-time performance, it significantly improves the accuracy and adaptability of 3D target detection, especially in the detection of pedestrians, small targets at long distances, and dense crowds. It reduces the false negative rate and the false positive rate, and is suitable for low-computing-power vehicle edge devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure FT_1
    Figure FT_1
  • Figure FT_2
    Figure FT_2
  • Figure FT_3
    Figure FT_3
Patent Text Reader

Abstract

This invention discloses a LiDAR 3D target detection method that integrates hierarchical vertical prior and multi-attention adaptive fusion, belonging to the field of autonomous driving LiDAR point cloud perception. It aims to solve problems such as loss of vertical information in PointPillars and the original SVP columnar frame, missed detection of dense pedestrians, insufficient feature fusion capability, and easy missed detection in post-processing of rotated targets. This method constructs a dual-branch preprocessing link, uses adaptive non-uniform voxels to generate hierarchical height maps, and combines the PointPillars branch to generate BEV pseudo-images. It employs a lightweight residual parallel backbone to extract multi-scale vertical and horizontal features; proposes an improved SIF fusion module embedded with CBAM to achieve noise suppression and dual-branch feature differentiation enhancement; aggregates features through multi-scale upsampling of the neck network, and uses a composite CIoU multi-task loss with fused height and segmentation auxiliary terms for training; finally, candidate boxes are selected through rotational fuzzy-NMS. This invention also provides a corresponding detection device and a computer-readable storage medium.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving environmental perception and LiDAR point cloud 3D target detection technology, and is particularly applicable to the optimization of real-time LiDAR 3D object detection algorithms based on columnar structures. It can be deployed on vehicle computing platforms and intelligent robot perception systems. Background Technology

[0002] Autonomous driving relies on LiDAR point clouds to detect 3D targets such as vehicles, pedestrians, and non-motorized vehicles. The mainstream lightweight solution PointPillars (a real-time LiDAR 3D target detection algorithm proposed at CVPR 2019) uses vertical columnar voxels to convert point clouds into bird's-eye view pseudo-images and relies on 2D convolution to ensure inference speed. However, it has a core defect of losing vertical dimension information due to compression. The height of an object is a key feature that distinguishes pedestrians, vehicles, and cyclists. A uniform 4m height voxel will smooth out vertical details, resulting in missed detections and false detections of small targets at a distance and dense pedestrians.

[0003] The existing SVP (Hierarchical Vertical Prior) framework adds an independent heightmap branch to extract vertical features and designs a SIF fusion module to fuse vertical / horizontal features, which improves detection accuracy to some extent, but still has four major drawbacks:

[0004] The original SIF (Spatial Information Fusion) simply fuses max / average pooling + single convolutional attention, which cannot adaptively model channel and spatial correlation, and has limited feature fusion capabilities.

[0005] With uniform voxel partitioning, adjacent target point clouds in dense pedestrian scenes are connected, making it easy for the model to identify multiple pedestrians as a single target.

[0006] Parallel dual-branch networks have a large number of parameters and floating-point operations, resulting in high inference latency, making them difficult to adapt to low-computing-power vehicle edge devices.

[0007] The axis-aligned NMS post-processing cannot adapt to 3D targets with arbitrary orientations, and in dense scenes, it is easy to over-suppress effective detection boxes.

[0008] To address the shortcomings of the existing technology, this invention proposes a multi-dimensional modular improvement scheme, which optimizes the hierarchical vertical prior detection framework from multiple dimensions, including fusion module, backbone network, voxel partitioning, loss function, post-processing, lightweighting, and multi-dimensional optimization. This significantly improves the accuracy of 3D target detection while ensuring real-time performance, especially optimizing the detection effect of pedestrians, small targets at a distance, and dense crowds. Summary of the Invention

[0009] Purpose of the Invention: The purpose of this invention is to provide a LiDAR 3D target detection method that integrates hierarchical vertical prior and multi-attention adaptive fusion, solving the problem of vertical feature loss in traditional columnar detection networks and enhancing the adaptive fusion capability of hierarchical height information; improving the defect of missed detection due to target adhesion in dense pedestrian scenes by adopting adaptive non-uniform voxel partitioning to retain individual distinguishing features; reducing the computational overhead of dual-branch networks and providing a lightweight, edge-deployable 3D detection solution; optimizing bounding box regression and class balancing capabilities, and improving the detection accuracy of rotated 3D targets and difficult samples; thereby improving the overall performance of 3D target detection.

[0010] Technical solution: To achieve the above objectives, the present invention adopts the following technical solution.

[0011] A LiDAR 3D target detection method that integrates hierarchical vertical prior and multi-attention adaptive fusion includes the following steps:

[0012] Step 1: Preprocessing of dual-branch point clouds

[0013] Height map branch (vertical prior branch): Abandoning the original uniform 3D voxel partitioning, an adaptive non-uniform voxel strategy is adopted to mesh the point cloud space and dynamically adjust the Z-axis voxel size based on the local point cloud density; the Z-axis voxel size is reduced in dense pedestrian areas and near target areas, while the Z-axis voxel size is increased in open and distant areas; the maximum Z coordinate within each voxel is extracted to generate a layered height map, preserving the fine vertical contour.

[0014] Pseudo-image branch (horizontal BEV branch): The PointPillars columnar partitioning is adopted, and the granularity of the XY plane segmentation is aligned with the height map branch. The BEV pseudo-image is generated by stacking the intra-column point features extracted by MLP (Multilayer Perceptron). A point cloud data augmentation module is added, which includes random translation / rotation / occlusion, GT sample copy and paste, hard sample oversampling, and balanced distribution of pedestrian and engineering vehicle classes with few samples.

[0015] Step 2: Lightweight residual bi-branch backbone network

[0016] Using the layered height map F respectively m BEV pseudo-image F p For input, two independent lightweight convolutional backbones, Backbone_H and Backbone_P, are constructed; all convolutional modules uniformly use depthwise separable convolutions + residual short connections to replace the original standard 3×3 convolutions.

[0017] The heightmap backbone (Backbone_H) extracts 3-scale vertical features. This backbone consists of concatenated convolutional blocks B1, B2, and B3, each with a fixed structure: 2 depthwise separable 3×3 convolutions + BN (batch normalization) + ReLU activation + residual short-circuit connections + max pooling with a stride of 2. The network performs spatial downsampling step by step. Although the basic architecture of the convolutional blocks is consistent, the channel upsampling strategy is differentiated: the B1 convolutional block only performs spatial downsampling, and the channel dimension remains unchanged; starting from B2, upsampling convolutions are added within the module, and the number of channels doubles step by step.

[0018] The pseudo-image backbone (Backbone_P) extracts horizontal features at three scales. The backbone consists of concatenated convolutional blocks B4, B5, and B6. B4 contains one pooling layer and three residual depthwise separable convolutions; B5 and B6 each contain one pooling layer and five residual depthwise separable convolutions. The stride is uniformly set to 2, and the network progressively performs spatial downsampling. Although the basic architecture of the convolutional blocks is consistent, the channel upsampling strategy is differentiated: the B4 convolutional block only performs spatial downsampling, keeping the channel dimension unchanged; starting from B5, upsampling convolutions are added within the module, doubling the number of channels progressively. The backbone network outputs three sets of vertical features f at different scales. h1 ,f h2 ,f h3 With 3 groups of level features f p1 ,f p2 ,f p3 The three sets of features are respectively fed into an independent improved SIF fusion module.

[0019] Step 3: Improved Multi-Attention SIF Shared Information Fusion Module

[0020] To address the limitations of the single attention mechanism in the original SIF (Self-Attention Module), an adaptive fusion module is designed that integrates channel-space joint attention from CBAM (Convolutional Block Attention Module) and learnable scale weights. The workflow is as follows:

[0021] Vertical and horizontal features of the same scale are compressed into compressed vertical features after passing through 3×3 lightweight convolution channels. Horizontal features after compression The original fused feature map f is obtained by stitching along the channel. cat:

[0022]

[0023] For f cat A CBAM attention module is introduced, and channel dependency and spatial pixel association are modeled to generate global joint attention weights.

[0024] Global average pooling and global max pooling are performed on fcat respectively to obtain two sets of one-dimensional channel description vectors; the two sets of vectors share a two-layer MLP network as input, and are activated by Sigmoid to output a channel attention weight map Mc; the weights are multiplied with the original fused features channel by channel: For the channel-weighted fcat, the maximum and average values ​​are taken along the channel dimension and concatenated to form a 2-channel spatial feature; a spatial attention weight map Ms is generated by 3×3 convolution + Sigmoid, and weighted pixel by pixel. This step allows the network to learn the global scene semantics in advance, and filters redundant noise within the fusion features from both the channel dimension and the spatial dimension.

[0025] Then return to compressed features , Perform a 1×1 convolution to generate a vertical attention map g. h Horizontal attention map g p。

[0026] Set the learnable scaling factors λ and γ (both initialized to 0.8) and construct the residual enhancement formula:

[0027]

[0028]

[0029] In equations (2) and (3) This represents a pixel-wise multiplication operation that retains the original basic features while adaptively amplifying the key target features.

[0030] Then, differentiated weighting coefficients are set for the three scales, and a multi-scale adaptive weighting factor is introduced to assign higher fusion weights to shallow, small target scale features, thereby strengthening the feature representation of distant objects. After multiplying the corresponding scale-enhanced features by the weighting coefficients, the features are concatenated along the channels to output the final fusion-enhanced feature f at that scale. out The enhanced fusion features are then fed into the neck aggregation network.

[0031] Step 4: Multi-scale neck aggregation network and multi-task detection head

[0032] The neck network employs deconvolutional upsampling to unify the 3-scale fused features to the same resolution, and channel concatenation aggregates global multi-scale information. This network includes three upsampling operations, Up1, Up2, and Up3, implemented using deconvolution operations with strides of 1, 2, and 4, respectively, corresponding to kernel sizes of 1, 2, and 4. The output channel dimension from Up1 to Up3 is 2C. Concatenating the three output features along the channel dimension yields the global aggregated feature tensor. The global aggregated feature Fcat Input a multi-task detection head to perform parallel prediction of multiple parameters of three-dimensional targets.

[0033] Classification header cls_head: Input 1×1 convolution output N cls For the number of classes in the dataset, the 3D regression head reg_head outputs F. reg The 7-dimensional parameters correspond to the 3D bounding box center (x, y, z), length, width, and height (l, w, h), and height offset. The orientation head dir_head outputs F. dir The binary classification determines whether the target is facing positive or negative.

[0034] Step 5: Multi-task composite loss function

[0035] Obtain various prediction tensors output by the detection head, combine them with the 3D labeled ground truth values ​​inherent in the point cloud data, construct a weighted total loss function, and synchronously constrain the network parameter updates for all branch losses. The total loss function is:

[0036]

[0037] Calculation methods for each item:

[0038] Balanced Focal Loss, γ=2, α adaptively weighted according to the number of samples of each category, to alleviate the problem of insufficient pedestrian samples;

[0039] CIoU Loss replaces the original SmoothL1, calculating 3D-CIoU for a 3D revolved bounding box. The formula is:

[0040]

[0041] In equation (5), ρ is the Euclidean distance between the center of the predicted box and the center of the ground truth box, c is the diagonal of the minimum bounding cube, and v is the length, width and height similarity penalty term.

[0042] Binary cross-entropy loss, weight 0.5;

[0043] : Height reconstruction using L1 loss, constraining the height map branch output to fit the real object height distribution;

[0044] The binary cross-entropy segmentation loss applies additional weights to dense pedestrian samples during training for weighted constraints.

[0045] Step 6: Rotational Blur Fuzzy-NMS Post-processing

[0046] Load the trained model weights and perform inference to filter and optimize the massive candidate boxes output by the detection head. Steps:

[0047] Filter low-confidence boxes with a confidence threshold of 0.25; boxes below the threshold are discarded. Sort the remaining 3D rotated boxes by prediction category confidence from high to low.

[0048] Calculate the 3D rotation IoU for each box, traverse the high-confidence reference box, and calculate the 3D rotation intersection-union ratio of the remaining candidate boxes with the reference box in turn;

[0049] The fuzzy weighted suppression strategy is as follows: IoU > 0.7: high overlap is determined, and low-confidence duplicate boxes are directly removed; 0.5 < IoU ≤ 0.7: no direct deletion is performed, and the 3D box coordinates are fused based on the confidence of the two boxes; IoU ≤ 0.5: no overlap is determined, and all target information after filtering is output, including object category, detection confidence, 3D rotated bounding box, and object orientation angle, as the autonomous driving perception output result.

[0050] Furthermore, the 3D target detection device can be mounted on an in-vehicle perception hardware platform, which includes an in-vehicle LiDAR, an embedded processor, and a computer-readable storage medium. Accordingly, this embodiment of the invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by the processor, it implements the LiDAR 3D target detection method of this invention, which integrates hierarchical vertical prior and multi-attention adaptive fusion. The computer-readable storage medium includes, but is not limited to, any carrier capable of storing a computer program, such as a read-only memory, random access memory, solid-state drive, flash memory chip, in-vehicle embedded Flash, or hard disk.

[0051] The above-mentioned technical solutions adopted in this invention can achieve the following beneficial effects: In the LiDAR 3D target detection method that integrates hierarchical vertical prior and multi-attention adaptive fusion provided by this invention, the adaptive non-uniform voxel partitioning strategy refines the vertical granularity for dense pedestrian areas, significantly reducing the false detection rate in dense scenes and solving the problem of original uniform voxel adhesion; the CBAM multi-attention SIF module synchronously models channel and spatial dependency, and the vertical and horizontal feature fusion is more complete, improving both AP and overall mAP in pedestrian medium difficulty on the KITTI validation set; 3D-CIoU loss + rotation blur NMS significantly reduces the localization error of rotated 3D boxes, and dense candidate boxes are not over-suppressed; the lightweight residual dual-branch backbone and depthwise separable convolution reduce the floating-point operation and inference latency of the model; the lightweight dynamic inference scheme can be deployed on low-computing-power automotive edge chips such as Orin X, balancing accuracy and real-time performance; the overall module is a plug-in design, requiring no modification to mainstream columnar network backbones such as PointPillars, CenterPoint, and BEVFusion, with strong versatility and low engineering implementation cost. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0053] Figure 1 A flowchart illustrating a lidar 3D target detection method that integrates hierarchical vertical prior and multi-attention adaptive fusion, provided by the present invention.

[0054] Figure 2 This is a schematic diagram of the internal structure of the improved multi-attention SIF module in this invention;

[0055] Figure 3 This is a schematic diagram of the multi-scale neck aggregation network and multi-task detection head in this invention; Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in the specification without creative effort are within the scope of protection of this application.

[0057] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0058] This embodiment uses the PointPillars algorithm as the baseline model. The baseline model only uses a single columnar pseudo-image branch. This invention adds an adaptive non-uniform height map vertical branch, a lightweight residual dual backbone, a CBAM-improved SIF fusion module, replaces the CIoU loss, and uses rotational blur NMS post-processing on the basis of PointPillars.

[0059] Figure 1 This is a schematic diagram of the process of a lidar 3D target detection method that integrates hierarchical vertical prior and multi-attention adaptive fusion in this invention. The complete execution link includes 6 core sub-steps, and the parameters, operators and data transformation logic are described in detail for each step.

[0060] Step 1: Preprocessing of dual-branch point clouds

[0061] First, the LiDAR point cloud undergoes dual-branch adaptive preprocessing. The input raw point cloud format is as follows:

[0062]

[0063] In equation (6), x / y are planar coordinates, z is height, and r is laser reflection intensity. The original point cloud spatial range is uniformly constrained, KITTI: X[0,69.12], Y[-39.68,39.68], Z[-3,1]. The three-dimensional coordinates of x, y, and z of the point cloud are normalized and mapped to the interval [0,1]. The reflection intensity r is normalized separately to [0,1]. The preprocessing is divided into two parallel branches: the vertical branch of the height map and the horizontal branch of the columnar pseudo-image.

[0064] To obtain layered height information, an adaptive non-uniform height map was designed. An adaptive non-uniform voxel strategy was employed to mesh the point cloud space, with a fixed mesh size of 0.16m along the X and Y axes, dividing the point cloud into fixed X and Y mesh units. Local point density was determined, and the number of points N within a single X and Y mesh was counted. pts Set the density threshold T=30 and dynamically switch the Z-axis voxel size; if N pts ≥30 (nearby, densely populated pedestrian areas), Z-axis fine-grained voxel size Z voxel =0.25m; if N pts <30 (long-distance open area), Z-axis coarse-grained voxel size Z voxel =4m, perform 3D voxel segmentation on each XY grid according to adaptive Z granularity, traverse all points within the voxel, take the maximum z coordinate within the voxel as the height feature, and fill empty voxels with 0 values; stitch together the height values ​​of all voxels to output a layered height map. C represents the number of layers along the Z-axis, and W / H represents the width and height of the XY grid.

[0065] To extract horizontal features from the point cloud and convert them into pseudo-images, we constructed a pseudo-image horizontal extraction branch. Following the PointPillars column partitioning rule, the XY grid granularity is strictly aligned with the heightmap branch to ensure a one-to-one correspondence between the two feature spaces. A maximum of 32 points are retained within each column; any points exceeding this limit are randomly sampled and discarded. A multilayer perceptron (MLP) is used to learn the features of all points within each column structure, outputting a C-dimensional feature vector. Finally, the point features are stacked according to the original column structure positions to generate the pseudo-image. .

[0066] A point cloud data augmentation module has been added, which is enabled only during the training phase and disabled during the inference phase. It includes three types of augmentation strategies:

[0067] Geometric transformations: global random rotation of the point cloud [-π / 4, π / 4], translation ±0.5m, global scaling [0.95, 1.05];

[0068] Oversampling of hard samples: Within the training batch, pedestrian, cyclist, and construction vehicle GT samples are repeated twice to balance the category ratio;

[0069] GT Copy and Paste: Randomly extracts the labeled target point cloud and overlays it onto the blank road area to expand the small target sample.

[0070] Step 2: Lightweight residual bi-branch backbone network

[0071] Two inputs F m (Height map), F p (Pseudo-images) are fed into two independent lightweight convolutional backbones, Backbone_H and Backbone_P, respectively. All convolutional blocks are uniformly replaced with depthwise separable convolutions and residual connections instead of the original standard convolutions.

[0072] The heightmap backbone, Backbone_H, consists of concatenated convolutional blocks B1, B2, and B3. Each convolutional block has a fixed structure: two depthwise separable 3×3 convolutions + Batch Normalization (BN) + ReLU activation + residual connections + max pooling with a stride of 2. The network performs spatial downsampling step by step. Although the basic architecture of the convolutional blocks is consistent, the channel upsampling strategy is differentiated: the B1 convolutional block only performs spatial downsampling, and the channel dimension remains unchanged; starting from B2, upsampling convolutions are added inside the module, and the number of channels doubles step by step. B1 convolutional block: Input Output shallow vertical features Preserves fine object height details, adapting to small targets at a distance; B2 convolution block: input f h1 Output mid-level vertical features Balancing high detail with local semantics; B3 convolutional block: input f h2 Output deep vertical features This captures the vertical semantics of the global scene. The final output is three sets of vertical multi-scale feature sets F. h ={f h1 ,f h2 ,f h3}

[0073] The pseudo-image backbone `Backbone_P` consists of concatenated convolutional blocks B4, B5, and B6. B4 contains one pooling layer and three residual depthwise separable convolutions; B5 and B6 each contain one pooling layer and five residual depthwise separable convolutions. The stride is uniformly set to 2, and the network progressively performs spatial downsampling. Although the basic architecture of the convolutional blocks is consistent, the channel upsampling strategy is differentiated: the B4 convolutional block only performs spatial downsampling, keeping the channel dimension unchanged; starting from B5, upsampling convolutions are added within the module, doubling the number of channels progressively. B4 convolutional block: Input Output shallow horizontal features B5 convolutional block: Input f p1 Output mid-level features B6 convolutional block: Input f p2 Output deep horizontal features The two sets of features correspond one-to-one according to scale: (f h1 ,f p1 ), (f h2 ,f p2 ), (f h3 ,f p3 The three sets of features are respectively sent to the independent improved SIF fusion module.

[0074] Step 3: Improved Multi-Attention SIF Shared Information Fusion Module

[0075] See Figure 2 For each set of vertical features f of the same scale h Horizontal features f p The complete fusion process is executed with identical logic across all three scale operations. The single-scale subdivision operation is as follows:

[0076] f h f p Each feature channel is compressed using a lightweight 3×3 convolutional layer: shallow features (64 channels) are compressed to 32 channels; mid-layer features (128 channels) are compressed to 64 channels; and deep features (256 channels) are compressed to 128 channels. The compressed vertical features are then output. Horizontal features after compression The original fused feature map f is obtained by stitching along the channel. cat: As shown in formula (1).

[0077] For f catGlobal average pooling and global max pooling are performed respectively to obtain two sets of one-dimensional channel description vectors. The two sets of vectors share a two-layer MLP network as input, and are activated by Sigmoid to output a channel attention weight map M. c Multiply the weights by the original fused features channel by channel: It automatically suppresses invalid channel features and enhances valid channels. The channel-weighted f... cat The maximum and average values ​​along each channel dimension are taken and concatenated to form a 2-channel spatial feature; a spatial attention weight map M is generated by 3×3 convolution + Sigmoid. s Pixel-wise weighted: This step highlights foreground target pixels and filters ground noise. It involves generating CBAM channel-spatial joint attention weights, completing global scene prior learning, and solidifying the discrimination capabilities of foreground, noise, and effective feature channels into the network weight parameters. This constrains the dual-branch independent attention generation process in subsequent steps, ensuring that the vertical and horizontal branches only adaptively enhance features in real obstacle areas, thus preventing noise pixels from being incorrectly weighted.

[0078] Then, the independent attention residuals of the vertical and horizontal branches are weighted to compress the features. , Perform a 1×1 convolution to generate a vertical attention map g. h Horizontal attention map g p .

[0079] Set the learnable scaling factors λ and γ (both initialized to 0.8) and construct residual enhancement formulas, such as formula (2) and formula (3).

[0080] Then, differentiated weighting coefficients are set for the three scales, and a multi-scale adaptive weighting factor is introduced: the weighting coefficient for shallow scales is α1=1.2, for medium scales it is α2=1.0, and for deep scales it is α3=0.7. Higher fusion weights are assigned to shallow, small target scale features to enhance the representation of distant object features. After multiplying the corresponding scale-enhanced features by the weighting coefficients, they are concatenated along the channels to output the final fused enhanced feature f at that scale. out The enhanced fusion features are then fed into the neck aggregation network.

[0081] Step 4: Multi-scale neck aggregation network and multi-task detection head

[0082] See Figure 3The neck network employs deconvolutional upsampling to unify the fused features across three scales to the same resolution, and channel concatenation aggregates global multi-scale information. This network includes three upsampling operations, Up1, Up2, and Up3, implemented using deconvolution operations with strides of 1, 2, and 4, respectively, corresponding to kernel sizes of 1, 2, and 4. The output channel dimension from Up1 to Up3 is 2C. Concatenating the three output features along the channel dimension yields the global aggregated feature tensor. The global aggregated feature F cat Input a multi-task detection head to perform parallel prediction of multiple parameters of three-dimensional targets.

[0083] Classification header cls_head: Input 1×1 convolution output N cls For the number of classes in the dataset, the 3D regression head reg_head outputs F. reg The 7-dimensional parameters correspond to the 3D bounding box center (x, y, z), length, width, and height (l, w, h), and height offset. The orientation head dir_head outputs F. dir The binary classification determines whether the target is facing positive or negative.

[0084] Step 5: Multi-task composite loss function

[0085] Obtain the various prediction tensors output by the detection head, combine them with the three-dimensional annotation ground truth of the point cloud data, construct a weighted total loss function, and synchronize the update of network parameters for all branch losses. The total loss function is as shown in formula (4).

[0086] Calculation methods for each item:

[0087] Balanced Focal Loss, γ=2, α adaptively weighted according to the number of samples of each category, to alleviate the problem of insufficient pedestrian samples;

[0088] CIoU Loss replaces the original SmoothL1 and calculates 3D-CIoU for 3D rotated boxes, as shown in Equation (5). In Equation (5), ρ is the Euclidean distance between the center of the predicted box and the center of the ground truth box, and c is the diagonal of the minimum bounding cube. is the length, width, and height similarity balance coefficient, and v is the length, width, and height similarity penalty term;

[0089] Binary cross-entropy loss, weight 0.5;

[0090] : Height reconstruction using L1 loss, constraining the height map branch output to fit the real object height distribution;

[0091] The binary cross-entropy segmentation loss applies additional weights to dense pedestrian samples during training for weighted constraints.

[0092] Step 6: Rotational Blur Fuzzy-NMS Post-processing

[0093] Load the trained model weights and perform inference to filter and optimize the massive candidate boxes output by the detection head. Steps:

[0094] Filter low-confidence boxes with a confidence threshold of 0.25; boxes below the threshold are discarded. Sort the remaining 3D rotated boxes by prediction category confidence from high to low.

[0095] Calculate the 3D rotation IoU for each box, traverse the high-confidence reference box, and calculate the 3D rotation intersection-union ratio of the remaining candidate boxes with the reference box in turn;

[0096] The fuzzy weighted suppression strategy is as follows: IoU > 0.7: high overlap is determined, and low-confidence duplicate boxes are directly removed; 0.5 < IoU ≤ 0.7: no direct deletion is performed, and the 3D box coordinates are fused based on the confidence of the two boxes; IoU ≤ 0.5: no overlap is determined, and all target information after filtering is output, including object category, detection confidence, 3D rotated bounding box, and object orientation angle, as the autonomous driving perception output result.

[0097] Based on the above technical solutions, this invention addresses the shortcomings of existing SVP frameworks, such as insufficient feature fusion capabilities, target adhesion in dense scenes, high inference overhead, and difficulty in adapting post-processing to rotating targets. It achieves adaptive collaborative optimization of hierarchical vertical prior features and horizontal BEV features. Experimental verification shows that this solution can be seamlessly embedded as a plug-in into mainstream columnar LiDAR 3D detection networks such as PointPillars. It exhibits excellent detection performance on the KITTI dataset, with particularly outstanding improvement effects on pedestrian, distant sparse small targets, and dense crowd scenes. It can reduce missed detections and false detections while meeting the real-time inference requirements of vehicles, and has high engineering application value for autonomous driving.

[0098] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention; any equivalent substitutions, improvements or changes made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lidar 3D target detection method that integrates hierarchical vertical prior and multi-attention adaptive fusion, characterized in that, comprising the following steps: Step 1: Dual-branch point cloud preprocessing, including a height map branch and a pseudo-image branch; The height map branch employs an adaptive non-uniform voxel strategy to mesh the point cloud space, dynamically adjust the Z-axis voxel size based on local point cloud density, and extract the maximum Z-coordinate within each voxel to generate a layered height map F. m The pseudo-image branches follow a columnar division, and the BEV pseudo-image F is generated by stacking the intra-column point features extracted by MLP. p ; Step 2: Construct a lightweight residual dual-branch backbone network, using hierarchical height maps F. m and BEV pseudo-image F p As input, a multi-scale vertical feature set F is extracted through two independent lightweight convolutional backbones, Backbone_H and Backbone_P. h ={f h1 ,f h2 , f h3 } and the horizontal feature set F p ={f p1 , f p2 , f p3 The convolutional modules in the backbone network employ depthwise separable convolution combined with residual short connections. Step 3: Input the vertical features and horizontal features of the same scale obtained in Step 2 into the improved multi-attention SIF shared information fusion module respectively for fusion; The fusion module comprises: performing channel compression on input features and then splicing the compressed features along channels, sequentially performing CBAM channel attention weighting and spatial attention weighting to generate a global joint attention weight, then generating independent attention maps for the vertical branch and the horizontal branch respectively and introducing learnable scaling coefficients for residual enhancement, finally setting differentiated weighting coefficients for different scales and performing channel-wise concatenation to output fused enhanced features; Step 4: Input the multi-scale fusion enhancement features output from Step 3 into the neck aggregation network, upsample them through deconvolution to unify them to the same resolution, and then concatenate them along the channels to obtain the global aggregation feature tensor F. cat ; F cat Input a multi-task detection head to predict 3D target category, 3D bounding box parameters, and orientation in parallel; Step 5: Construct a multi-task composite loss function to calculate the weighted total loss between the various prediction tensors output by the detection head and the 3D labeled ground truth, and simultaneously constrain the network parameter updates; the loss function includes classification loss. Regression loss Directional loss High reconstruction losses and segmentation loss ; Step 6: Using rotation-blurred Fuzzy-NMS post-processing to screen and optimize the candidate boxes output by the detection head, comprising: filtering out low-confidence boxes, sorting the boxes by confidence, calculating the 3D rotation intersection over union, executing a fuzzy weighted suppression strategy according to the IoU threshold, and outputting the screened target information.

2. The method according to claim 1, characterized in that, The adaptive non-uniform voxel strategy described in step 1 is specifically: fix the grid size on the XY axes, and count the number N of point clouds in a single XY grid pts , and set a density threshold T; if N pts ≥ T, it is determined as a dense region, and the voxel size on the Z axis is set to a fine-grained value; if N pts < T, it is determined as a sparse region, and the voxel size on the Z axis is set to a coarse-grained value; the fine-grained value is smaller than the coarse-grained value; wherein the value range of the density threshold T is 20 to 50, the value range of the fine-grained value is 0.1m to 0.5m, and the value range of the coarse-grained value is 2m to 6m.

3. The method according to claim 1, characterized in that, The lightweight convolutional backbone Backbone_H described in Step 2 is composed of convolutional blocks B1, B2 and B3 connected in series, each convolutional block comprises 2 layers of depthwise separable 3×3 convolutions, batch normalization, ReLU activation, residual short connections and max pooling with a stride of 2, which performs downsampling step by step, and the number of channels doubles step by step starting from B2; the lightweight convolutional backbone Backbone_P is composed of convolutional blocks B4, B5 and B6 connected in series, B4 comprises 1 layer of pooling and 3 layers of residual depthwise separable convolutions, B5 and B6 each comprise 1 layer of pooling and 5 layers of residual depthwise separable convolutions, the stride is uniformly set to 2, downsampling is performed step by step, and the number of channels doubles step by step starting from B5.

4. The method according to claim 1, characterized in that, The improved multi-attention SIF shared information fusion module described in Step 3 specifically performs the following operations: Vertical features f of the same scale h and level features f p After being compressed into 3×3 lightweight convolution channels, the following results were obtained: and By piecing together along the channel, we obtain: For f cat Perform global average pooling and global max pooling, input the two pooling results into a shared two-layer MLP network, and activate the output channel attention weight map M via Sigmoid. c M c with f cat Multiply by channel; weight f after channel addition cat The maximum and average values ​​along the channel dimension are taken and concatenated to form a 2-channel spatial feature. This feature is then processed by a 3×3 convolution and a sigmoid function to generate a spatial attention weight map M. s Pixel-wise weighting; respectively for and Perform a 1×1 convolution to generate a vertical attention map g. h and horizontal attention map g p ; By setting learnable scaling factors λ and γ, a residual enhancement formula is constructed: in This represents a pixel-by-pixel multiplication operation; different weighting coefficients are set for different scales, and the enhanced features of the corresponding scale are multiplied by the weighting coefficients and then concatenated along the channels to output the final fused enhanced features of that scale.

5. The method according to claim 4, characterized in that, The differentiated weighting coefficients described in Step 3 are: the shallow scale weight coefficient α1=1.2, the middle layer α2=1.0, and the deep layer α3=0.

7.

6. The method according to claim 1, characterized in that, The neck aggregation network described in step 4 includes three upsampling operations, Up1, Up2, and Up3, implemented using deconvolution operations with strides of 1, 2, and 4, respectively, with corresponding kernel sizes of 1, 2, and 4. The output channel dimension of Up1 to Up3 is 2C. The output features of Up1, Up2, and Up3 are concatenated along the channel dimension to obtain the global aggregation feature tensor. .

7. The method according to claim 1, characterized in that, The total loss function mentioned in step 5 is: in, A balanced Focal Loss was used, with γ=2 and α adaptively weighted according to the number of samples in each category. 3D-CIoU loss is used; The cross-entropy loss is used for binary classification. For high-resolution reconstruction of L1 loss; The loss is the binary cross-entropy segmentation loss.

8. The method according to claim 1, characterized in that, The fuzzy weighted suppression strategy described in Step 6 is specifically: for overlapping candidate boxes with IoU>0.7, directly eliminate low-confidence duplicate boxes; for candidate boxes with 0.5<IoU≤0.7, perform weighted fusion of 3D box coordinates based on the confidence of the two boxes; for candidate boxes with IoU≤0.5, determine that there is no overlap and retain them.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which when executed by a processor, implements the method according to any one of the preceding claims 1 to 8.

10. A computer device, characterized in that, Comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor implements the method according to any one of the preceding claims 1 to 8 when executing the program.