Multi-modal data fusion sensing method based on adaptive calibration and physical guidance

By employing an adaptive calibration and physically guided multimodal data fusion method, the problems of sensor extrinsic drift and dynamic feature lag in autonomous BEV perception were solved, achieving accurate fusion and stable detection of multimodal data in a unified space.

CN122090404APending Publication Date: 2026-05-26WUXI UNIV
0 Cites 1 Cited by

Patent Information

Application Number
CN202610151781.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-02-03
Publication Date
2026-05-26

AI Technical Summary

Technical Problem

Existing autonomous BEV perception solutions suffer from decreased multimodal data fusion accuracy due to sensor extrinsic drift and dynamic feature lag during long-term operation or high-speed turning, making it impossible to achieve accurate target detection and trajectory prediction.

Method used

An adaptive calibration and physically guided multimodal data fusion method is adopted. By introducing the nuscenes dataset, dual-stream feature extraction and BEV spatial projection, online calibration based on BEV consistency and physical prior-driven cross-modal Transformer fusion, sensor extrinsic parameters are calibrated in real time and attention mechanism is guided for feature fusion.

Benefits of technology

It achieves continuous adaptive calibration and pixel-level alignment of multimodal data in a unified space, improving the detection accuracy and robustness of autonomous driving systems in complex environments, and solving the problems of sensor extrinsic drift and dynamic feature lag.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a multi-modal data fusion sensing method and system based on adaptive calibration and physical guidance, belongs to the technical field of computer vision and automatic driving, and aims to solve the core problems of projection dislocation caused by extrinsic parameter drift of a long-time operation sensor in existing BEV sensing and target detection lag in a high-speed dynamic scene. According to the method, a sensing framework comprising four modules, namely a multi-view image feature projection transformation module, a LiDAR point cloud feature coding module, a BEV consistency-based multi-sensor online adaptive calibration module and a physical prior-driven cross-modal deformable Transform fusion module is constructed; a closed-loop feedback mechanism is formed through perception and calibration, external parameters of the sensor are corrected in real time in combination with visual reprojection, IMU pre-integration and BEV feature alignment constraint, and projection ghosting is eliminated; vehicle kinematics priori is introduced to construct dynamics Query, attention sampling points are driven to adaptively deviate along a target motion trail, and the problem of detection lag is solved.
Need to check novelty before this filing date? Find Prior Art