基于注视点定位与长短时建模的第一视角多人脸跟踪方法及装置

By employing a gaze localization and long-short-term modeling approach, and utilizing 3D convolutional kernels and gaze probability maps for feature extraction and tracking, combined with a self-attention mechanism and an IoU loss function, the accuracy and robustness issues of multi-face tracking in first-person view video scenarios are resolved, achieving more efficient multi-face tracking.

CN120219436BActive Publication Date: 2026-07-17CHINA UNIV OF MINING & TECH
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
CHINA UNIV OF MINING & TECH
Filing Date
2025-03-18
Publication Date
2026-07-17

Smart Images

  • Figure CN120219436B_ABST
    Figure CN120219436B_ABST
Patent Text Reader

Abstract

本发明公开了一种基于注视点定位与长短时建模的第一视角多人脸跟踪方法及装置，首先利用第一视角视频中的连续图像帧作为输入，构建人脸轨迹核生成器生成动态3D卷积核，再通过构建显著性预测模块与注视点概率图融合模块生成高精度的注视点概率图，在多人脸跟踪阶段，引入注视点概率图，构建短时关联建模模块和长时关联建模模块不断更新轨迹活跃集和轨迹丢失集，来进行第一视角多人脸跟踪。本发明采用端到端的深度学习框架学习第一视角多人脸跟踪，充分利用注视点位置及变化信息，使注视点定位与多人脸跟踪两个任务相互促进，有效解决了遮挡、移动及视角突变等复杂场景的跟踪难题，显著提升了多人交互场景中的人脸跟踪精度与鲁棒性。
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Multi-face tracking method based on depth apparent characteristics and a self-adaptive aggregation network

    CN109829436A

  • Twin network tracking method based on long and short time correlation attention mechanism

    CN118628957A