Multi-target tracking method and system based on multi-sensor and spatio-temporal graph self-attention

By fusing lidar point cloud and camera image features, and combining them with a spatiotemporal graph self-attention mechanism, the challenge of quickly and accurately tracking multiple targets in autonomous driving is solved, achieving stable multi-target tracking in complex environments, and is applicable to autonomous driving systems.

CN120411167BActive Publication Date: 2026-06-26CHONGQING UNIV +1
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
CHONGQING UNIV
Filing Date
2025-04-22
Publication Date
2026-06-26

AI Technical Summary

Technical Problem

In the face of ever-changing and complex road environments, how to quickly and accurately track multiple targets remains a significant challenge for autonomous driving perception subsystems.

Method used

A multi-target tracking method based on multi-sensor and spatiotemporal graph self-attention is adopted. By fusing the features of LiDAR point cloud and camera image, and combining the spatiotemporal graph self-attention mechanism, 2D-3D target matching and feature fusion are achieved. CUDA is used to accelerate point cloud processing, and the Kuhn-Munkres algorithm is used for efficient matching to establish the target offset and trajectory correlation matrix.

Benefits of technology

It improves target matching accuracy, enhances the system's robustness in complex environments such as occlusion, lighting changes, and viewing angle changes, and has real-time multi-target tracking capabilities, making it suitable for applications with high timeliness requirements such as autonomous driving.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120411167B_ABST
    Figure CN120411167B_ABST
Patent Text Reader

Abstract

The application relates to a multi-target tracking method and system based on multi-sensor and space-time graph self-attention, which comprises the following steps: S1, obtaining point cloud detection targets in a laser radar coordinate system and image detection targets in a camera coordinate system during vehicle driving; S2, obtaining 2D-3D target matching pairs, unmatched point cloud detection targets and unmatched image detection targets according to the point cloud detection targets and the image detection targets, and forming a 2D-3D target matching result; S3, extracting 2D image features and 3D point cloud features according to the 2D-3D target matching result, connecting the 2D image features and the 3D point cloud features, and obtaining image and point cloud fusion features of a detection frame; and S4, obtaining target offset and target and trajectory association matrix according to the image and point cloud fusion features of the detection frame and the image and point cloud fusion features of a historical frame target, and realizing real-time tracking of the multi-target. The application improves tracking precision, reduces errors and efficiently realizes cross-frame tracking.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Real-time target detection method based on laser radar and vision fusion

    CN114782729A

  • 3D multi-target tracking method based on space-time adaptive attention

    CN115731267A