High-performance loosely-coupled multi-modal data fusion system for smart driving environmental perception system and vehicle-mounted device

By employing modal-independent feature interaction and motion-appearance feature cascade coupling data association strategies, the problem of insufficient information complementarity in multimodal data fusion is solved, thereby improving the accuracy and reliability of the intelligent driving environment perception system and enabling it to adapt to complex traffic scenarios and severe weather.

WO2025241308A1 Publication Date: 2025-11-27JIANGSU UNIV
6 Cites 38 Cited by

Patent Information

Application Number
PCT/CN2024/108816
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-05-24
Filing Date
2024-07-31
Publication Date
2025-11-27

AI Technical Summary

Technical Problem

Existing multimodal data fusion methods suffer from insufficient information complementarity in intelligent driving environment perception, resulting in low accuracy and reliability of detection and tracking tasks, especially in complex traffic scenarios and in adverse weather conditions.

Method used

A modal-independent feature interaction strategy and a motion-appearance feature cascade coupling data association strategy are adopted for multimodal data fusion detection and tracking tasks, respectively. The modal-independent feature interaction strategy improves detection performance, and the motion-appearance feature cascade coupling data association strategy improves tracking performance.

Benefits of technology

It significantly improves the accuracy and robustness of multimodal data fusion systems in complex traffic scenarios and in adverse weather conditions, ensuring the safe operation of intelligent connected vehicles under all working conditions.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.
Need to check novelty before this filing date? Find Prior Art