Bird's eye view feature generation method based on multi-modal fusion

By employing a multimodal fusion generation method and utilizing bird's-eye view query vectors and cross-attention mechanisms, the compatibility issue of different modal inputs is resolved, generating high-quality bird's-eye view features and improving the perception accuracy and robustness of autonomous driving.

CN115578705BActive Publication Date: 2026-06-09CHANGCHUN YIHANG INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Patent Type
Patents(China)
Current Assignee / Owner
CHANGCHUN YIHANG INTELLIGENT TECH CO LTD
Filing Date
2022-10-21
Publication Date
2026-06-09

AI Technical Summary

Technical Problem

Existing bird's-eye view feature generation methods are difficult to effectively accommodate inputs from different modalities and fail to construct a unified feature representation, which affects the perception accuracy of downstream autonomous driving tasks.

Method used

By extracting modal features from multiple vehicle sensors, utilizing bird's-eye view query vectors and cross-attention mechanisms, multimodal information is fused to generate high-quality bird's-eye view features, supporting arbitrary modal inputs, including vehicle cameras, LiDAR, and millimeter-wave radar.

Benefits of technology

It achieves effective interaction and fusion of multimodal information, generates high-quality bird's-eye view features, improves the perception accuracy and robustness of downstream autonomous driving tasks, and adapts to different scenarios and sensor changes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115578705B_ABST
    Figure CN115578705B_ABST
Patent Text Reader

Abstract

The disclosure provides a bird's eye view feature generation method based on multi-modal fusion, comprising: extracting all vehicle-mounted sensor collection information of the current time of the vehicle to obtain at least first modal collection information and second modal collection information; using a first modal feature extraction network to extract first modal features of the first modal collection information, and using a second modal feature extraction network to extract second modal features of the second modal collection information; obtaining the bird's eye view feature after initialization of the current time based on the historical time bird's eye view feature; obtaining the bird's eye view feature with first modal feature information based on the first modal features and the bird's eye view feature after initialization of the current time; obtaining the bird's eye view feature with second modal feature information based on the bird's eye view feature with first modal feature information and the second modal features; and generating the bird's eye view feature with multi-modal fusion information. The disclosure also provides an electronic device, a readable storage medium and a program product.
Need to check novelty before this filing date? Find Prior Art