3D multi-modal target detection method based on false point cloud

Abstract

The invention provides a 3D multi-mode target detection method based on false point cloud, which is suitable for automatic driving and unmanned system environment perception. According to the method, through fusion of the monocular camera and the LiDAR, the problems of difficult feature alignment, fusion strategy staticization and inaccurate detection frame positioning in the prior art are solved. The method comprises the following steps: firstly, designing an adaptive modal weighted fusion device to dynamically generate a fusion weight of multi-modal data, and adaptively balancing contribution of point cloud and image features; secondly, providing a bidirectional feature mutual attention calibrator, and enhancing feature interaction and alignment at the ROI level; and finally, introducing a geometric parameter aggregation optimizer, and optimizing a final detection result. Experiments on a KITTI data set show that AP values of 92.22%, 85.03% and 82.25% are respectively achieved under the Easy difficulty, the Moderate difficulty and the Hard difficulty, and the detection precision and the robustness are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and autonomous driving technology, specifically relating to a multimodal 3D target detection method based on pseudo point clouds, which is particularly suitable for environmental perception systems of unmanned ground vehicles (UGVs). Background Technology

[0002] 3D object detection is a core technology for environmental perception in autonomous driving and unmanned systems, crucial for ensuring the safe operation of vehicles or drones in complex and dynamic environments. With the development of deep learning, multimodal data fusion has become a key approach to improving detection accuracy, among which the fusion of monocular cameras and LiDAR has attracted much attention due to its low cost and strong applicability. However, existing methods have significant limitations in dynamic weight allocation, feature alignment, and decision optimization, requiring urgent improvement. LiDAR-based 3D object detection methods are mainly divided into three categories: point-based methods (such as PointNet++ directly processing the raw point cloud, but with low computational efficiency), voxel-based methods (such as VoxelNet dividing the point cloud into a grid, but with high memory consumption), and projection-view-based methods (such as PIXOR projecting the point cloud onto a 2D plane, but potentially losing 3D information). Pure vision methods use monocular or stereo cameras, but depth estimation is inaccurate, resulting in limited detection accuracy. Multimodal fusion methods aim to combine the precise geometric information of LiDAR with the rich texture information of cameras, but achieving depth complementarity remains a challenge. Existing fusion methods can be categorized into projection-based fusion, model-based fusion, and decision-level fusion. For example, Frustum-PointNet generates 3D candidate regions from 2D bounding boxes, but this method relies on the accuracy of the 2D detectors, has a limited fusion range, and is prone to losing contextual information outside the bounding boxes. BEVFusion unifies and fuses features in the bird's-eye view (BEV) space, but requires a complex feature alignment network, which is computationally expensive and does not address the feature bias problem at the ROI level. CLOCs encode 2D and 3D bounding boxes as sparse tensors and learn geometric consistency, but this method is insufficient in feature-level information fusion, which may lead to loss of detail. Existing methods have three core problems: most methods use fixed weights (such as addition or concatenation), and cannot dynamically adjust modal contributions according to the scene. For example, in sparse regions of LiDAR point clouds, the model should rely more on image semantics; while in regions with weakened texture, the point cloud geometry should be trusted. Due to noise and encoding errors in deep completion networks (such as PENet), the feature distribution of pseudo point clouds and real point clouds at the ROI level differs significantly. Direct fusion can lead to modal misalignment, weakening the advantages of multimodal complementarity. Traditional non-maximum suppression (NMS) suppresses overlapping boxes through IoU thresholding, but in dense scenes, it may cause false suppression due to small offsets and discard the complementary information of multimodal predictions, affecting localization accuracy and recall. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes an adaptive fusion network that achieves more accurate and robust 3D detection through dynamic weight allocation, deep feature alignment, and decision-level optimization. This method is particularly suitable for resource-constrained UGV and UAV platforms, providing a reliable perception foundation for autonomous driving.

[0004] The technical solution adopted by this invention to achieve the above objectives is: a 3D multimodal target detection method based on pseudo-point clouds, comprising the following steps:

[0005] S1. Convert the monocular visible light RGB image into a pseudo point cloud through a deep completion network, establish a spatial correspondence with the LiDAR point cloud, and construct a unified multimodal data input channel.

[0006] S2. Establish an adaptive modality weighted fusion unit, which includes a global context descriptor extraction unit and a meta-network dynamic weight generation unit, and automatically balances the contributions of different modalities based on scene features.

[0007] S3. A bidirectional feature mutual attention calibrator is introduced after the region proposal network to achieve deep feature interaction and alignment through LiDAR-to-Image and Image-to-LiDAR dual-path attention mechanisms.

[0008] S4. The initial detection results are post-processed using a geometric parameter aggregation optimizer, and a more accurate 3D detection box is generated through cluster weighted fusion.

[0009] The DepthCompletion network employs a dual-branch coding structure, including a depth prediction branch dominated by the color image and an upsampling branch guided by the sparse depth map. It performs weighted fusion using a learnable confidence map to ultimately generate a confidence-based depth map. .

[0010] The global context descriptor extraction unit includes:

[0011] First, global feature extraction is performed on the two input modal features to obtain a three-dimensional feature tensor. Two-dimensional feature tensor ;

[0012] For the LiDAR point cloud branch, the 3D feature tensor Global average pooling is performed in the spatial dimension to obtain a compact global feature vector that contains the geometric structure information of the entire scene.

[0013] For the pseudo-point cloud branch of the image, the same pooling operation is used for extraction. Global semantic feature vector;

[0014] The two feature vectors are then concatenated to form a comprehensive scene context descriptor. It is used to comprehensively characterize the multimodal characteristics of the current scene.

[0015] The meta-network dynamic weight generation unit includes:

[0016] With context descriptor As input;

[0017] The meta-network consists of two fully connected Linear layers. The first Linear layer maps high-dimensional features to the hidden space and performs a non-linear transformation using the ReLU activation function. The second Linear layer maps the hidden features to a three-dimensional weight space and finally performs normalization using the Softmax function. The three weights correspond to the contribution of three basic fusion strategies, and the network can automatically adjust the weight allocation according to different scene features.

[0018] In the actual fusion process, the system executes three basic fusion operations in parallel: the first is LiDAR feature-driven fusion, the second is image feature-driven fusion, and the third is an average fusion strategy. Then, based on dynamically generated weight values, the three fusion results are weighted and combined to obtain the final fused feature. .

[0019] The specific implementation of the bidirectional feature mutual attention calibrator includes:

[0020] First, after the region proposal network generates candidate regions, LiDAR modal features are extracted respectively. and image modal features For each candidate region, LiDAR point cloud features and image pseudo-point cloud features are simultaneously input into the bidirectional attention mechanism.

[0021] a. In the attention path from LiDAR to the image direction, using LiDAR modal features Using the query vector Q, and image features as the key vector K and value vector V, a multi-head attention mechanism enables LiDAR features to query and fuse semantic information from image features to compute LiDAR enhanced features. This is used to enhance the representational ability of geometric features;

[0022] b. In the attention path from the image to LiDAR, using image modality features As the query vector, LiDAR features serve as the key vector K and value vector V. Through the same multi-head attention mechanism, image features can be queried and the geometric structural information in LiDAR features can be fused to calculate image enhancement features. This is used to improve the accuracy of semantic feature localization;

[0023] c. Subsequently, the attention outputs for both directions are processed. , Perform gating fusion processing separately and calculate gating weights. ,in Original features With enhanced features The splicing; among them, for , , for , ; to original features , With attention enhancement features , The data is then stitched together, and the channel weights are calculated using a multilayer perceptron (MLP) network. ,pass Achieve adaptive feature selection, yes and ;

[0024] Finally, the enhanced features of the two modalities are fused to output the aligned multimodal feature representation.

[0025] The operation process of the geometric parameter aggregation optimizer includes: firstly, sorting the initial detection box set output by the 3D detection head in descending order of confidence;

[0026] The process begins with the detection box with the highest confidence level, which is used as the center of the first cluster. For each detection box to be processed, the 3D intersection-union ratio with the existing cluster centers is calculated.

[0027] When the IoU value of a detection box with an existing cluster exceeds a preset threshold, the detection box is assigned to that cluster; if the IoU with all existing clusters is below the threshold, a new cluster is created with the detection box as the center.

[0028] Within each cluster, a confidence-based weighted fusion algorithm is used to average the geometric parameters of all bounding boxes in the cluster according to their respective confidence scores; the geometric parameters include the center point coordinates, bounding box size, and rotation angle.

[0029] After cluster fusion, the average confidence level is calculated;

[0030] After clustering and merging all detection boxes, each cluster outputs an optimized detection box and its confidence score.

[0031] The present invention has the following beneficial effects and advantages:

[0032] 1. By dynamically analyzing global scene features through a lightweight meta-network, the module automatically generates optimal fusion weights, overcoming the limitations of traditional static fusion strategies. This module can intelligently adjust the contribution of different modalities based on scene characteristics such as point cloud density and image texture complexity. For example, it enhances image feature weights in sparse LiDAR point cloud regions and strengthens point cloud geometric information in weakly textured regions, significantly improving feature representation capabilities in complex environments.

[0033] 2. A bidirectional cross-attention mechanism is established at the ROI level, enabling deep feature interaction through LiDAR-to-Image and Image-to-LiDAR dual-path queries. This module, combined with an adaptive gating fusion mechanism, effectively addresses the feature misalignment problem caused by depth completion noise, enhancing the complementarity of multimodal features. Experiments show that this module improves detection accuracy by 2.03% on the KITTI dataset at Hard difficulty.

[0034] 3. This method innovatively employs a weighted fusion strategy to replace traditional NMS, treating highly overlapping bounding boxes as multiple observations of the same target and aggregating geometric parameters through confidence-weighted aggregation. This approach preserves fine-grained localization information while improving bounding box stability, significantly enhancing recall and localization accuracy in dense scenes.

[0035] 4. The three modules form a collaborative optimization system: the adaptive fusion unit implements dynamic weight allocation at the front end, the attention calibrator solves feature alignment in the middle, and the parameter aggregator optimizes the detection output at the end. This hierarchical optimization mechanism enables the present invention to achieve leading performance of 92.22% (Easy), 85.03% (Moderate), and 82.25% (Hard) on the KITTI dataset. Attached Figure Description

[0036] Figure 1 The overall network architecture of this invention includes an adaptive modality weighted fusion unit, a bidirectional feature mutual attention calibrator, and a geometric parameter aggregation optimizer.

[0037] Figure 2 This is a schematic diagram of the adaptive modal weighted fusion converter.

[0038] Figure 3 The detailed process for a bidirectional feature mutual attention calibrator.

[0039] Figure 4 This is a schematic diagram of the geometric parameter aggregation optimizer.

[0040] Figure 5 A visual diagram illustrating the input of multimodal data.

[0041] Figure 6 This is a comparison chart of the computational efficiency of the models.

[0042] Figure 7 This is a graph showing the sensitivity analysis of hyperparameters.

[0043] Figure 8 A visualization comparison of the detection results for the KITTI dataset. Detailed Implementation

[0044] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0046] like Figure 1 As shown, the proposed 3D multimodal target detection algorithm based on pseudo-point clouds consists of four main parts. Our proposed modality weighted fusion unit, bidirectional feature mutual attention calibrator, and geometric parameter aggregation optimizer correspond to the Weighted Fusion, BFA, and GPA modules in the figure, respectively. Their specific functions are as follows:

[0047] S1. The monocular visible light RGB image is converted into a pseudo point cloud using the DepthCompletion network. Specifically, PENet is used, and its dual-branch structure processing flow is as follows:

[0048] (1) Color-dominant branch: Input RGB image captured by visual camera, extract features through ResNet-50 backbone network, use skip connections to preserve details, and output dense depth estimation map based on visible light image.

[0049] (2) Depth-dominant branch: Input the sparse LiDAR depth map collected by the LiDAR, gradually restore the resolution through the upsampling module, and output the depth completion map based on the radar image.

[0050] (3) Confidence-weighted fusion: The final confidence depth map is generated through learnable parameters. The calculation formula is:

[0051]

[0052] in, Represents confidence level. and Represents a confidence plot. and Represents the color-dominated branch and the depth-dominated branch. Represents the pixel coordinates in the depth map.

[0053] Finally, the depth map is converted into a pseudo point cloud using the camera intrinsic parameter matrix.

[0054]

[0055] This represents the 3D coordinates of the pseudo-point cloud in the camera coordinate system. and This represents the camera's effective focal length. and Z is the principal point coordinate, representing the position of the intersection of the camera's optical axis and the image plane in the pixel coordinate system. Z is the depth of the 3D point, that is, the distance from the point to the camera's optical center along the optical axis (Z-axis).

[0056] S2, such as Figure 2 As shown, the adaptive modal weighted fusion module Weighted Fusion is established. The specific implementation of this module includes three core steps:

[0057] S2.1) Global Context Descriptor Extraction: First, global feature extraction is performed on the two input modal features to obtain a three-dimensional feature tensor. Two-dimensional feature tensor .

[0058] For the LiDAR point cloud branch, the 3D feature tensor Global average pooling is performed in the spatial dimension to obtain a compact global feature vector that contains the geometric structure information of the entire scene.

[0059] For the pseudo-point cloud branch of the image, the same pooling operation is used for extraction. The global semantic feature vector.

[0060] The pooling operation formula is as follows:

[0061]

[0062] The two feature vectors are then concatenated to form a comprehensive scene context descriptor. = This descriptor can comprehensively characterize the multimodal characteristics of the current scene.

[0063] S2.2) Meta-network weight generation: A lightweight meta-network structure is adopted, using the aforementioned scene context descriptor. As input, the meta-network consists of two fully connected layers. The first layer maps high-dimensional features to the hidden space using a ReLU activation function for non-linear transformation. The second layer maps the hidden features to a three-dimensional weight space, and finally normalizes them using a Softmax function to ensure that the sum of the three weight values ​​is 1. These three weights correspond to the contributions of three basic fusion strategies, and the network can automatically adjust the weight allocation according to different scene features. The specific calculation process is as follows:

[0064]

[0065] in , For learnable parameters, output normalized weights. .

[0066] Feature-weighted fusion: In the actual fusion process, the system executes three basic fusion operations in parallel. The first is fusion dominated by radar LiDAR features, the second is fusion dominated by image features, and the third is an average fusion strategy. Then, based on dynamically generated weight values, the three fusion results are weighted and combined to obtain the final fused features. .

[0067]

[0068] in This indicates an element-wise addition operation.

[0069] S3, such as Figure 3 As shown, after ROI alignment, bidirectional feature mutual attention calibration is performed on each candidate region. A bidirectional feature mutual attention calibrator (BFA) is deployed on the ROI regions generated by the region proposal network to achieve cross-modal feature alignment. Specifically:

[0070] S3.1) Utilize the Region Proposal Network (RPN) to generate confidence regression boxes, locate the Region of Interest (ROI) regions, and use them for alignment. and

[0071] S3.2) For each ROI region, extract LiDAR modal features respectively. and image modal features , used to compute bidirectional attention enhancement features.

[0072] a. In the attention path from LiDAR to the image direction, using LiDAR modal features The query vector Q is used, and the image features are used as the key vector K and value vector V. A multi-head attention mechanism enables LiDAR features to query and fuse semantic information from image features, enhancing the representational power of geometric features. Specifically, the LiDAR-to-image attention calculation process involves calculating the query vector... key vector value vector ,in The projection matrix is ​​used. Enhanced features are calculated using a multi-head attention mechanism. The calculation process is as follows:

[0073]

[0074] A single attention head is calculated as follows: , The dimension of the key vector.

[0075] b. In the attention path from the image to LiDAR, using image modality features As the query vector Q, LiDAR features serve as the key vector K and value vector V. Through the same multi-head attention mechanism, image features can query and fuse geometric structural information from LiDAR features, improving the localization accuracy Z of semantic features, thus obtaining... .

[0076] c. Subsequently, the attention outputs for both directions are processed. , Perform gating fusion processing separately and calculate gating weights. ,in Original features With enhanced features The splicing. Among them, for , , for , .

[0077] S3.3) The original features , With attention enhancement features , The data is then stitched together, and the channel weights are calculated using a multilayer perceptron (MLP) network. ,pass Achieve adaptive feature selection, yes and .

[0078] S4, such as Figure 4As shown, the initial set of detection boxes output by the 3D detection head is first sorted in descending order of confidence level to ensure that detection boxes with high confidence are processed first. The system maintains a dynamically updated cluster set, which is initially empty. Processing begins with the detection box with the highest confidence level, which is used as the center of the first cluster. For each detection box to be processed, its 3D intersection-union ratio with the existing cluster centers is calculated.

[0079]

[0080] in Represents three-dimensional volume. The parameters representing the bounding box include coordinates, dimensions, and orientation. This represents the j-th cluster center.

[0081] In the specific calculation, the center coordinates, dimensions (length, width, and height), and rotation angle of the detection box are considered to accurately assess the spatial overlap between the two 3D bounding boxes. When the IoU value of the detection box with an existing cluster exceeds a preset threshold (usually set to 0.6), the detection box is assigned to that cluster; if the IoU with all existing clusters is below the threshold, a new cluster is created with the detection box as the center.

[0082] Within each cluster, a confidence-based weighted fusion algorithm is employed. For all bounding boxes in a cluster, their geometric parameters (including center point coordinates, bounding box size, and rotation angle) are weighted and averaged according to their respective confidence levels. Bounding boxes with higher confidence levels have greater weights during the fusion process. This approach preserves the accuracy of high-quality detection results while improving localization accuracy by fusing complementary information from multiple bounding boxes.

[0083] For each cluster The geometric parameters of the detection box within the bounding box are weighted by confidence, and the fusion formula for the center coordinates, dimensions, and rotation angle is as follows:

[0084]

[0085] in This represents the geometric parameter vector of the i-th detection box. This corresponds to the confidence level. The average confidence level after cluster fusion is taken. The final output is an optimized set of 3D bounding boxes.

[0086] like Figure 5The figure illustrates the four data modalities used in the method of this invention and their corresponding relationships. From left to right, the figure presents: the original color image captured by the RGB camera, the 3D point cloud data acquired by the LiDAR sensor, the alignment effect of the point cloud projected onto the image plane, and the virtual point cloud generated by the depth completion algorithm (marked in red). This illustration intuitively demonstrates the spatial correspondence of multimodal data, providing a foundation for subsequent feature fusion.

[0087] like Figure 6 As shown, the computational efficiency of the models is compared, and the computational overhead of each module is systematically evaluated through three dimensions: number of parameters, inference time, and FPS. As shown in the figure, the baseline model (A) has 42.75 MB of parameters, an inference time of 71.02 seconds, and an FPS of 7.04. After adding the adaptive modal weighted fusion unit (B), the number of parameters increases to 44.30 MB, and the inference time rises to 81.14 seconds. The complete model (E) still maintains a real-time processing capability of 6.93 FPS despite an increase of 6.56 MB of parameters, demonstrating the computational efficiency advantage of this invention.

[0088] like Figure 7 As shown, this paper presents a hyperparameter sensitivity analysis, systematically studying the impact of key parameters on model performance. The left figure illustrates the trend of the number of attention heads on detection accuracy, with optimal results achieved at 8 heads in both Moderate and Hard difficulties. The right figure shows the relationship between the dimension of the meta-network hidden layers and accuracy, with 64 dimensions yielding the best overall performance. This analysis provides experimental basis for model hyperparameter tuning.

[0089] like Figure 8 As shown, the detection results on the KITTI dataset are presented, visually demonstrating the performance difference between the present invention and the baseline method. From left to right, the results are: ground truth annotations, detection results from the VirConv baseline method, and detection results from the present invention. The green bounding boxes indicate that the present invention has more accurate detection capabilities in complex scenes (such as occluded vehicles or distant targets), and exhibits stronger robustness, especially in areas with strong background interference.

[0090] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should be considered within the scope of protection of the present invention.

Claims

1. A 3D multimodal target detection method based on pseudo-point clouds, characterized in that, Includes the following steps: By using a deep completion network, the monocular visible light RGB image is converted into a pseudo point cloud, establishing a spatial correspondence with the LiDAR point cloud and constructing a unified multimodal data input channel. An adaptive modality weighted fusion fusion unit is established, which includes a global context descriptor extraction unit and a meta-network dynamic weight generation unit, and automatically balances the contributions of different modalities based on scene features; A bidirectional feature mutual attention calibrator is introduced after the region proposal network to achieve deep feature interaction and alignment through LiDAR-to-Image and Image-to-LiDAR dual-path attention mechanisms. A geometric parameter aggregation optimizer is used to post-process the initial detection results, and a more accurate 3D detection box is generated through cluster weighted fusion.

2. The 3D multimodal target detection method based on pseudo-point clouds according to claim 1, characterized in that, The DepthCompletion network employs a dual-branch coding structure, including a depth prediction branch dominated by the color image and an upsampling branch guided by the sparse depth map. It performs weighted fusion using a learnable confidence map to ultimately generate a confidence-based depth map. .

3. The 3D multimodal target detection method based on pseudo-point clouds according to claim 1, characterized in that, The global context descriptor extraction unit includes: First, global feature extraction is performed on the two input modal features to obtain a three-dimensional feature tensor. Two-dimensional feature tensor ; For the LiDAR point cloud branch, the 3D feature tensor Global average pooling is performed in the spatial dimension to obtain a compact global feature vector that contains the geometric structure information of the entire scene. For the pseudo-point cloud branch of the image, the same pooling operation is used for extraction. Global semantic feature vector; The two feature vectors are then concatenated to form a comprehensive scene context descriptor. It is used to comprehensively characterize the multimodal characteristics of the current scene.

4. The 3D multimodal target detection method based on pseudo-point clouds according to claim 1, characterized in that, The meta-network dynamic weight generation unit includes: With context descriptor As input; The meta-network consists of two fully connected Linear layers. The first Linear layer maps high-dimensional features to the hidden space and performs a non-linear transformation using the ReLU activation function. The second Linear layer maps the hidden features to a three-dimensional weight space and finally performs normalization using the Softmax function. The three weights correspond to the contribution of three basic fusion strategies, and the network can automatically adjust the weight allocation according to different scene features. In the actual fusion process, the system executes three basic fusion operations in parallel: the first is LiDAR feature-driven fusion, the second is image feature-driven fusion, and the third is an average fusion strategy. Then, based on dynamically generated weight values, the three fusion results are weighted and combined to obtain the final fused feature. .

5. The 3D multimodal target detection method based on pseudo-point clouds according to claim 1, characterized in that, The specific implementation of the bidirectional feature mutual attention calibrator includes: First, after the region proposal network generates candidate regions, LiDAR modal features are extracted respectively. and image modal features For each candidate region, LiDAR point cloud features and image pseudo-point cloud features are simultaneously input into the bidirectional attention mechanism. a. In the attention path from LiDAR to the image direction, using LiDAR modal features Using the query vector Q and image features as the key vector K and value vector V, a multi-head attention mechanism enables LiDAR features to query and fuse semantic information from image features to compute LiDAR enhanced features. This is used to enhance the representational ability of geometric features; b. In the attention path from the image to LiDAR, using image modality features As the query vector, LiDAR features serve as the key vector K and value vector V. Through the same multi-head attention mechanism, image features can be queried and the geometric structural information in LiDAR features can be fused to calculate image enhancement features. This is used to improve the accuracy of semantic feature localization; c. Subsequently, the attention outputs for both directions are processed. , Perform gating fusion processing separately and calculate gating weights. ,in Original features With enhanced features splicing; among them, for , , for , ; to original features , With attention enhancement features , The data is then stitched together, and the channel weights are calculated using a multilayer perceptron (MLP) network. ,pass Achieve adaptive feature selection, yes and ; Finally, the enhanced features of the two modalities are fused to output the aligned multimodal feature representation.

6. The 3D multimodal target detection method based on pseudo-point clouds according to claim 1, characterized in that, The operation process of the geometric parameter aggregation optimizer includes: firstly, sorting the initial detection box set output by the 3D detection head in descending order of confidence; The process begins with the detection box with the highest confidence level, which is used as the center of the first cluster. For each detection box to be processed, the 3D intersection-union ratio with the existing cluster centers is calculated. When the IoU value of a detection box with an existing cluster exceeds a preset threshold, the detection box is assigned to that cluster; if the IoU with all existing clusters is below the threshold, a new cluster is created with the detection box as the center. Within each cluster, a confidence-based weighted fusion algorithm is used to average the geometric parameters of all bounding boxes in the cluster according to their respective confidence scores; the geometric parameters include the center point coordinates, bounding box size, and rotation angle. After cluster fusion, the average confidence level is calculated; After clustering and merging all detection boxes, each cluster outputs an optimized detection box and its confidence score.

Images