Fast target detection algorithm based on multi-modal fusion

Abstract

The invention belongs to the technical field of target detection, and particularly relates to a fast target detection algorithm based on multi-modal fusion, which comprises the steps of data acquisition and calibration, synchronous acquisition of images and point clouds and completion of internal and external parameter calibration; performing data enhancement operation such as zooming on the image, performing enhancement such as Pill-processing on the point cloud, and sharing geometric transformation to keep bimodal alignment; obtaining a two-dimensional bounding box and a confidence coefficient by using a YOLOv5 network; obtaining a three-dimensional bounding box by using a PointPillars network, and outputting a projection frame and a confidence coefficient; projecting the three-dimensional bounding box to a pixel plane according to the calibration matrix to form a bounding box; when the intersection-to-union ratio is greater than 0.5, generating a matching pair; each matching pair is coded into a 14-dimensional vector; vectors are input into a multi-layer perceptron, single-mode false detection is effectively inhibited through geometric and semantic consistency screening, 14-dimensional input features learn a nonlinear relation of a multi-mode bounding box through a three-hidden-layer MLP, and fusion precision is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of target detection technology, and specifically refers to a fast target detection algorithm based on multimodal fusion. Background Art

[0002] Single-modal target detection methods often have problems with unclear or inaccurate detection in scenes with poor lighting, rain, fog, or sparse point clouds, while multimodal fusion strategies can significantly improve stability and recall rate.

[0003] The multimodal fusion target detection algorithm combines the texture and color information provided by the visual camera with the precise three-dimensional geometry output by the LiDAR in the same coordinate system, superimposing the complementary advantages of the two types of sensors to obtain a target frame with both high resolution and high depth accuracy in complex environments; its core process usually includes independent feature extraction of each modality → coordinate alignment → matching / fusion → subsequent regression or classification.

[0004] According to the timing of fusion, it can be divided into pre-fusion, deep / feature-level fusion and post-fusion. Among them, post-fusion is most widely implemented in scenarios such as autonomous driving, mobile robots and mixed reality because it has the shortest link and can easily meet the ≤100ms vehicle real-time budget.

[0005] Existing technology (publication number CN113985445B) discloses a 3D object detection algorithm based on camera and lidar data fusion, employing a minimalist post-fusion chain consisting of camera YOLOv5 2D detection, point cloud ground segmentation / cropping, projection, and Euclidean clustering. First, a 2D bounding box is quickly obtained in the image domain. The ground-segmented foreground point cloud is then projected onto the imaging plane using the camera-LiDAR calibration matrix. Only those point clouds that fall within the corresponding 2D bounding box are subjected to Euclidean clustering to regress the 3D bounding box. Finally, the fused bounding box is directly output using a weighted average. Unlike common deep feature-level fusion algorithms, this method omits cross-modal interaction layers such as convolution and Transformer layers, retaining only three lightweight steps: sensor calibration, coordinate projection, and Euclidean clustering. This significantly reduces model parameters and graphics memory usage, while also compressing end-to-end inference latency to a level that can be easily supported by embedded platforms.

[0006] However, the above process of projection, clustering, and averaging still has three problems:

[0007] 1. Cross-class mismatching: Using only the IoU threshold can easily mismatch adjacent boxes of different categories, leading to false detection and missed detection;

[0008] 2. Missed detection of distant and sparse point clouds: Relying on Euclidean clustering requires a sufficiently dense foreground point cloud. When the target scale is small or the point cloud is diluted by rain or fog, clustering fails, causing the recall rate to drop sharply.

[0009] 3. Manual threshold links have poor portability: Parameters such as the ground segmentation threshold and clustering radius need to be repeatedly adjusted according to sensor height and road conditions, making it difficult to adapt to multiple vehicle models and complex working conditions.

[0010] The fundamental reason is that its path still remains at the geometric heuristic level. Although the network depth is reduced, all error tolerance is placed on fixed thresholds and clustering rules, thus exposing the defect of insufficient accuracy in extreme scenarios. Summary of the Invention

[0011] The object of the present invention is to provide a fast target detection algorithm based on multimodal fusion, comprising the following steps:

[0012] S101 data acquisition and calibration, synchronously acquires RGB images and LiDAR point clouds and completes internal and external parameter calibration;

[0013] S102 multimodal data preprocessing and simultaneous enhancement: random scaling, horizontal flipping, color gamut dithering, blurring, and occlusion enhancement are performed on images, while point cloud enhancements such as pillarization, rotation, translation, and resampling are performed, while sharing geometric transformations to maintain bimodal alignment.

[0014] S103 uses the YOLOv5 network to obtain a two-dimensional bounding box and confidence ;

[0015] S104 uses the PointPillars network to obtain a 3D bounding box and outputs the projection box and confidence level. ;

[0016] S105 projects the three-dimensional bounding box onto the pixel plane according to the calibration matrix to form ;

[0017] S106 semantic-geometric consistency screening, when Generate matching pairs ;

[0018] S107 puts each Encoded as a 14-dimensional vector

[0019] ;

[0020] S108 will The input structure is The multi-layer perceptron outputs .

[0021] Furthermore, the YOLOv5 network adopts CSPDarknet backbone, PANet neck and Anchor-based Head, and the bounding box regression loss is generalized intersection-over-union loss. .

[0022] Furthermore, the PointPillars network output parameter And the classification branch uses Focal Loss

[0023] .

[0024] Furthermore, the 14-dimensional fusion vector is defined as

[0025] .

[0026] Furthermore, the total loss function of MLP is

[0027] .

[0028] Furthermore, MC Dropout is enabled in the inference phase, and the same frame is repeated Sub-sampling inference to obtain classification entropy

[0029] ,

[0030] Positioning variance

[0031] ,

[0032] and the combined uncertainty

[0033] .

[0034] Further, Normal fusion Dynamically adjust weights; Trigger sensor bypass degradation.

[0035] Furthermore, the number of MC Dropout sampling , the additional delay of a single frame is ≤0.2ms.

[0036] The beneficial effects achieved by the present invention using the above structure are as follows:

[0037] (1) Through geometric (IoU>0.5) and semantic consistency (category matching) screening, single-modal false detection (such as LiDAR misclassification of low obstacles) is effectively suppressed.

[0038] (2) The 14-dimensional input features (including confidence, coordinates, and minimum bounding rectangle) are used to learn the nonlinear relationship of multimodal bounding boxes through a three-hidden layer MLP (56→128→128 nodes), which significantly improves the fusion accuracy.

[0039] (3) The classification entropy and positioning variance are obtained through 10 Monte Carlo samplings, which can identify high-confidence-high-uncertainty pseudo targets. When the uncertainty of a certain mode is too high, its weight is dynamically reduced; if it exceeds the high-risk threshold, the failed sensor is automatically bypassed and the radar enters the degradation mode to ensure that the minimum perception capability is not met. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The sensor layout of the data collection vehicle of the present invention;

[0041] Figure 2 A map of the collection route of the data set of the present invention;

[0042] Figure 3 Flowchart of the algorithm proposed by the present invention;

[0043] Figure 4 Schematic diagram of the post-fusion model of the present invention;

[0044] Figure 5 A comparison chart of the detection results of the present invention and the detection results of the backbone model; DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0046] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0047] The present invention Figure 3 The multimodal fusion target detection algorithm architecture proposed in this paper is demonstrated; detection performance is improved by fusing RGB images and lidar point cloud data.

[0048] The system first processes data from different modalities through two independent backbone networks. In the image detection branch, the RGB image is passed through the YOLOv5 network to generate n detection results, including class scores and 2D bounding boxes. In the point cloud detection branch, the lidar point cloud is processed by the PointPillars network, outputting k projections of 3D detection results, including class scores and 3D bounding boxes. These preliminary detection results from both modalities carry the scene's visual features and geometric depth information, respectively, laying the foundation for subsequent fusion.

[0049] The multimodal post-fusion system then processes the preliminary results. First, it screens detection pairs with high spatial matching through consistency checks to ensure the correlation between the modalities. Then, the matching results are unified into feature representations through the encoder, and feature fusion and optimization are performed using a multi-layer perceptron (MLP). Finally, the decoder generates the final k fused bounding boxes and outputs detection results that combine the advantages of both modalities.

[0050] Example 1

[0051] This embodiment includes step S101 data collection and calibration;

[0052] This embodiment takes the KITTI public autonomous driving platform as an example to collect and calibrate the required raw data. The specific process is as follows.

[0053] like Figure 1 As shown, the vehicle's roof is equipped with two Point Gray Flea2 grayscale cameras (1.4 megapixels), two Point Gray Flea2 color cameras (1.4 megapixels), and a Velodyne HDL-64E 3D lidar (64-line, 10 Hz, 120 m range, 0.09° angular resolution). This is supplemented by four Edmund optical lenses with a 4 mm focal length and a 90° horizontal field of view, and an OXTS TR3003 GPS / IMU (100 Hz, 0.02 m positioning accuracy, 0.1° attitude accuracy). All sensors are synchronized to the millisecond level using PPS pulses and NMEA timestamps, ensuring a cross-modal inter-frame error of ≤1 ms.

[0054] like Figure 2 As shown in the figure, data collection was completed over a six-hour period of continuous driving through urban areas, residential areas, campuses, roads, and pedestrian-dense areas in Karlsruhe, Germany. Approximately 25% of the valid footage is available on the official website, with 7,481 frames used for object detection. The pedestrian samples were divided into 4,489:1,496:1,496 frames in a training: validation:test ratio of 60:20:20, and synchronized RGB image and LiDAR point cloud pairs were constructed using a unified timestamp index (e.g., 000001).

[0055] The original data is stored in the mmdetection3d / data / kitti directory:

[0056] image_2 / Save 1242 × 375 resolution RGB image (*.png)

[0057] velodyne / Save XYZR point cloud (binary *.bin, single frame ≈ 1.8 MB)

[0058] label_2 / stores 3D ground truth annotations (*.txt, including fields such as category, bbox size, rotation angle, etc.)

[0059] calib / records the internal and external distortion coefficients of each sensor (.txt). Files with the same serial number (such as 000001.) in the directory represent multimodal samples at the same time.

[0060] Read the calib text and 4×4 homogeneous matrix, first transform the three-dimensional point in the LiDAR coordinate system L Convert to the world coordinate system and then map it to the camera coordinate system C:

[0061]

[0062] The external parameter is automatically calibrated by a vehicle-mounted laser flat target in conjunction with ICP, with a calibration residual of ≤2 cm.

[0063] Camera intrinsic parameter matrix Including focal length With the main point To reduce the influence of barrel and pincushion distortion on projection accuracy, the radial and tangential Distortion coefficient, inverse distortion preprocessing is performed on the two-dimensional points in the camera coordinate system. After distortion correction, the projection error is reduced to 0.3 px.

[0064] After completing the loading of internal and external parameters, the pinhole imaging model is used to project the three-dimensional points to the pixel plane:

[0065]

[0066] in 、 The experiment randomly samples 1000 frames to calculate the projection mean square error (MSE) and verifies the geometric alignment accuracy ≤ 0.5 pixel using the Monte Carlo method.

[0067] Through the above step S101, RGB-LiDAR multimodal basic data that is time-synchronized, spatially aligned, and distortion-corrected can be obtained, providing high-precision input for subsequent feature extraction and fusion detection.

[0068] S102 Multimodal data preprocessing;

[0069] After completing the calibration and data sorting in S101, this embodiment designs a preprocessing process of first independent enhancement within the branch and then cross-modal synchronous transformation to address the data differences between the RGB image and the LiDAR point cloud;

[0070] The visual branch performs the following operations on the RGB image:

[0071] (1) Random 0.5-2.0 times scaling; (2) 0.5 probability horizontal flip; (3) HSV ±20% jitter; (4) 5 × 5 σ = 1.5 Gaussian blur; (5) ≤30% random occlusion;

[0072] All enhanced images are uniformly resized to 640 × 384 and normalized to [0, 1] to provide YOLOv5 with training samples that are robust to deformation, lighting, blur, and occlusion.

[0073] The point cloud branch first performs local copy-paste on the pedestrian box to improve the density of sparse objects;

[0074] Then, geometric perturbations were applied, including a Z-axis rotation of ±9°, an XYZ Gaussian translation (0, 0.25 / 0.25 / 0.1), and a global resampling, mirroring, and scale consistency transformation of 0.8-1.2 times;

[0075] Finally, the point cloud is Pillared, and 20 point codes (x, y, z, r, xc, yc, zc, xp, yp) are taken from each column to generate a pseudo 2D feature map input PointPillars.

[0076] The two branches share rotation / scaling parameters, synchronously update extrinsic parameters, and inspect the projection error to be ≤0.5 px; dynamically oversample and reweight categories for difficult occluded examples; and load the enhanced data in parallel using batch 8 and worker 8 multi-threading to ensure real-time inference.

[0077] S103 Image branch object detection;

[0078] This example uses the YOLOv5s model as the image branch backbone. Its architecture consists of a CSPDarknet backbone, a PANet neck, and an anchor head. The backbone leverages CSPNet to reduce the number of parameters while ensuring gradient flow; PANet fuses multi-scale features to improve detection of pedestrians of varying sizes; and the head presets multiple anchor boxes at each grid location and simultaneously outputs bounding box coordinates, object confidence, and category probability.

[0079] The model takes a 640 × 384 normalized image as input and generates n two-dimensional detection boxes per frame. and the corresponding confidence In the training phase, an IoU threshold of 0.5 is used to distinguish positive and negative samples; in the inference phase, candidate boxes with confidence levels lower than 0.25 are first filtered out, and then NMS is performed with IoU = 0.45, outputting the final result with confidence levels ≥ 0.4.

[0080] The bounding box regression loss uses the generalized intersection-over-union loss

[0081]

[0082] in Introducing the minimum circumscribed convex hull based on IoU , strengthen scale symmetry; Represents the positive sample indicator.

[0083] Binary cross entropy is used for pedestrians and backgrounds. To alleviate category imbalance, Distribution FocalLoss is introduced to increase the weight of low-confidence samples and improve the accuracy of small target detection.

[0084] Using PyTorch training, the initial learning rate , Adam optimizer, and cosine annealing scheduling. The measured single-frame inference time is approximately 5 ms, which can be run in parallel with the point cloud branch to meet the 20 ms end-to-end real-time requirement.

[0085] S104 point cloud branch target detection;

[0086] Divide the lidar point cloud into uniform grids in the XY plane to form Pillar. Each point encodes a vector

[0087]

[0088] in are the column center coordinates, is the point-to-center offset; each column is sampled or zero-filled to a fixed N=20 points to obtain a pseudo 2D feature map and input it into the lightweight convolution backbone network.

[0089] The backbone extracts multi-scale features by downsampling and FPN structure; the detection head presets 3D anchor boxes at each position in the feature map and outputs parameterized bounding boxes As well as category probability and direction classification score, spatial positioning and direction estimation are achieved.

[0090] Positioning loss design Use Smooth L1 loss

[0091]

[0092] in is the difference between the predicted and true value, and the angle term is Avoid periodic ambiguity.

[0093] The classification loss design uses the focal loss function Focal Loss

[0094]

[0095]

[0096] in, Represents the category prediction probability of the anchor box, and Represent the matching labels of the predicted bounding box and the true annotation box respectively.

[0097] The hyperparameters α and γ are set to 0.25 and 2 respectively to dynamically adjust the weights of positive and negative samples and focus on difficult samples.

[0098] The optimized PointPillars branch performs single-frame inference in approximately 14 ms. Combined with the image branch and post-fusion module (2 ms), the end-to-end latency of the entire system is controlled at 20 ms, meeting the requirements of real-time vehicle detection.

[0099] Through the above steps S103 and S104, high-precision, low-latency preliminary detection results can be obtained in the RGB image and LiDAR point cloud domains respectively, laying the foundation for subsequent semantic-geometric consistency screening and fusion regression.

[0100] S105 3D bounding box projection;

[0101] First read the 4 × 4 external parameter matrix in the calib directory and , the three-dimensional bounding box in the lidar coordinate system Convert to the world coordinate system W and the camera coordinate system C in turn:

[0102]

[0103] The above matrix is ​​accurately calibrated by laser flat target + ICP, with residual error ≤ 2 cm.

[0104] Will Substitute eight vertices into the pinhole model

[0105]

[0106] in is the camera internal parameter, is the attitude matrix, and the pixel coordinates are obtained after radial / tangential distortion inverse compensation . Take the circumscribed rectangle of all projection points , and record the projection confidence .

[0107] In the same frame, traverse the two-dimensional detection frame output by the image branch S103 ;like

[0108]

[0109] The two are combined into a candidate pair , and save the IoU value and respective confidence.

[0110] All candidate pairs are written into a shared memory queue using a hash key consisting of frame number + box index, facilitating parallel reading by the subsequent semantic-geometric consistency screening module. An average of 20–30 sets of cross-modal candidate pairs are generated per frame, with a cache occupancy of < 0.4MB, which can meet 10 Hz real-time requirements.

[0111] S106 Semantic-geometric consistency screening;

[0112] For the candidate pairs generated in S105 , check whether the category labels at both ends match. In this embodiment, only the “person”↔“pedestrian” correspondence is accepted, and the rest are directly discarded.

[0113] Calculate the projection box-two-dimensional box overlap for semantically consistent candidate pairs:

[0114]

[0115] when When it is judged as a positive match, it is recorded as Otherwise, it is regarded as a negative sample and added to the difficult example mining pool for category balance in the training phase.

[0116] To further reduce the mismatching rate, two additional constraints can be added: the center distance

[0117]

[0118] And the scale ratio

[0119]

[0120] Those who meet the above conditions will continue to be retained.

[0121] After semantic and geometric double screening, the average number of frames retained High-confidence cross-modal matching The result is directly used as the input of the 14-dimensional fusion encoder (S107), providing high-quality positive samples for subsequent multi-layer perceptron regression and providing clear positive / negative label boundaries for the training stage.

[0122] Through steps S105-S106, accurate projection of 3D boxes to 2D boxes and semantic-geometric consistency screening can be completed in milliseconds, laying a reliable data foundation for multimodal fusion detection.

[0123] S107 fusion encoder constructs 14-dimensional feature vector;

[0124] High confidence cross-modal matching pairs obtained in semantic-geometric consistency screening (S106) On this basis, this embodiment constructs a 14-dimensional fusion feature vector for each pair , its dimensions have the following meanings:

[0125] Dimensions meaning illustrate 1 #timg# Image detection confidence 2 #timg# Point cloud detection confidence 3–6 #timg# Image frame #timg# upper left-lower right coordinates 7–10 #timg# Projection frame#timg# Upper left-lower right coordinates 11–12 #timg# Center offset between two frames (pixels) 13 #timg# Overlap between two frames 14 #timg# Area ratio, used to measure scale consistency

[0126] All coordinates are normalized to 0-1 according to the input resolution of 640×384 before constructing vectors; offsets and area ratios are used Coordinates are normalized to compress the dynamic range, making it easier for subsequent networks to converge.

[0127] S108 Multilayer Perceptron (MLP) Fusion Regression;

[0128] The network structure includes an input layer, which is 14-dimensional ;

[0129] This example uses an MLP with three hidden layers:

[0130] 14 → 56 → 128 → 128 → 5 (output), the activation function of each hidden layer is ReLU to prevent gradient disappearance and produce sparse activation.

[0131] Output vector They correspond to center offset, width and height scaling, and fusion confidence, respectively.

[0132] Let the minimum bounding rectangle of the matching box be The upper left corner is , width and height are .

[0133] The MLP output is converted to the final fused frame by the decoder :

[0134]

[0135] Sigmoid Limit the offset to , the exponential mapping guarantees positive size and supports large-scale scaling.

[0136] The loss function design of this embodiment includes

[0137] Regression loss :when A positive sample is recorded when the IoU with the ground-truth box is ≥ 0.5, otherwise it is recorded as a negative sample. The IoU loss is calculated to align the position and scale;

[0138] Confidence loss : Confidence of positive and negative samples Adopt binary cross entropy (BCE) to give greater penalty to false positives;

[0139] The total loss .

[0140] Through the above S108 step, MLP imposes physical feasibility constraints on the output coordinates while learning the joint distribution of multimodal bounding boxes, significantly improving the positioning accuracy and confidence robustness of the fused box.

[0141] Figure 5 Qualitative results comparing the proposed method with the backbone model are shown.

[0142] The first row shows the original RGB image in the KITTI dataset. The second and third rows are the detection results obtained by the original backbone and the post-fusion strategy mentioned in this paper, respectively. The red and blue bounding boxes represent the detection results of YOLOv5 and PointPillars, with confidence scores exceeding 0.25 and 0.4 respectively. The green bounding box shows the detection result of the method proposed in this paper, with a confidence score higher than 0.4. Figure 5 In a, the fusion method proposed in this invention corrects the missed detection of a single branch, while Figure 5 In b, false positives are effectively filtered out.

[0143] Furthermore, the present invention improves the bounding box to increase accuracy.

[0144] Example 2;

[0145] This embodiment introduces an improvement mechanism based on uncertainty based on the above fusion detection to enhance the robustness of the system to abnormal situations.

[0146] The core of this mechanism is to not only consider the confidence level of the model output, but also quantitatively evaluate the uncertainty of the prediction results. When sensor data is abnormal or model judgment is biased, the corresponding response strategy is triggered by uncertainty, thereby reducing the risk of high-confidence false detection.

[0147] The difference from Example 1 is that this embodiment uses the Monte Carlo sampling (MCDropout) method in step S108 to obtain the uncertainty of the model prediction;

[0148] S108-1 Uncertainty Sampling Reasoning;

[0149] MC Dropout was enabled to maintain “training state” random dropout in all hidden layers of the MLP (Dropout p = 0.3);

[0150] Repeat forward reasoning to fused 14-dimensional feature vectors of the same frame Perform N = 10 forward samplings and obtain .

[0151] S108-2 Uncertainty Calculation;

[0152] Categorical entropy

[0153]

[0154] Positioning variance

[0155]

[0156] Comprehensive uncertainty

[0157]

[0158] S108-3 Adaptive Decision Making;

[0159] Uncertainty determination Treatment strategy illustrate #timg# (safe zone, default #timg#) Normal fusion Use the fusion frame #timg# of Example 1 to output #timg# (controllable area, default #timg#) Dynamic weight adjustment Press #timg#, #timg# to recalculate the fusion confidence and reduce the influence of unstable channels #timg# (High-risk area) Sensor Bypass Degradation Temporarily shield high uncertainty branches and retain only robust data such as lidar / millimeter wave for detection

[0160] Offline training still uses the regression-confidence joint loss of Example 1; after going online, when real-time monitoring finds that the number of high-risk area triggers exceeds 3% in the sliding window (500 frames), these false positive samples are automatically added to the difficult example pool, and then fine-tuned for 3 epochs using MC Dropout to achieve closed-loop adaptive iteration.

[0161] Through the high-confidence-high-uncertainty dual discrimination and graded response strategy of Example 2, the system significantly improves robustness and safety in complex working conditions, including simulated camera glare, sparse LiDAR point clouds, and rain and fog occlusion.

[0162] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A fast target detection algorithm based on multimodal fusion, characterized by: The following steps are involved: S101 data acquisition and calibration, synchronously acquires RGB images and LiDAR point clouds and completes internal and external parameter calibration; S102 multimodal data preprocessing and simultaneous enhancement: random scaling, horizontal flipping, color gamut dithering, blurring, and occlusion enhancement are performed on images, while point cloud enhancements such as pillarization, rotation, translation, and resampling are performed, while sharing geometric transformations to maintain bimodal alignment. S103 uses the YOLOv5 network to obtain a two-dimensional bounding box and confidence ; S104 uses the PointPillars network to obtain a 3D bounding box and outputs the projection box and confidence level. ; S105 projects the three-dimensional bounding box onto the pixel plane according to the calibration matrix to form ; S106 semantic-geometric consistency screening, when Generate matching pairs ; S107 puts each Encoded as a 14-dimensional vector ； S108 will The input structure is The multi-layer perceptron outputs .

2. The fast target detection algorithm based on multimodal fusion according to claim 1 is characterized in that: The YOLOv5 network uses CSPDarknet backbone, PANet neck and Anchor-based Head, and the bounding box regression loss is generalized intersection-over-union loss. .

3. The fast target detection algorithm based on multimodal fusion according to claim 1 is characterized in that: The PointPillars network output parameters And the classification branch uses Focal Loss 。 4. The fast target detection algorithm based on multimodal fusion according to claim 1, characterized in that: The 14-dimensional fusion vector is defined as 。 5. The fast target detection algorithm based on multimodal fusion according to claim 1 is characterized in that: The total loss function of MLP is .

6. The fast target detection algorithm based on multimodal fusion according to claim 1, characterized in that: Enable MC Dropout in the inference phase and repeat for the same frame Sub-sampling inference to obtain classification entropy ， Positioning variance ， and the combined uncertainty 。 7. The fast target detection algorithm based on multimodal fusion according to claim 6 is characterized in that: Normal fusion Dynamically adjust weights; Trigger sensor bypass degradation.

8. The fast target detection algorithm based on multimodal fusion according to claim 6, characterized in that: MCDropout sampling times , additional delay per frame .

Images