Multi-modal spatio-temporal alignment safe driving emotion recognition method based on sensitive word guidance

By using a mechanism that proactively guides responses to sensitive words, the analysis of driver voice and video data solves the problem of spatiotemporal misalignment of multimodal data in driving scenarios, enabling accurate identification of driver emotions and improving the real-time performance and accuracy of safe driving.

CN121188535BActive Publication Date: 2026-05-22HEFEI UNIV OF TECH
5 Cites 0 Cited by

Patent Information

Application Number
CN202511179955.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-08-22
Publication Date
2026-05-22
Estimated Expiration
2045-08-22

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively address the spatiotemporal misalignment of multimodal data in driving scenarios, resulting in insufficient accuracy and real-time performance in emotion recognition. In particular, the asynchronous nature of voice-sensitive words and visual facial emotion expression in time and space has not been fully utilized.

Method used

By using a mechanism that actively guides sensitive words, this method extracts sensitive word features from driving monitoring video and voice data, analyzes the temporal attention between sensitive words and voice text, determines multiple candidate time intervals, and analyzes the alignment relationship between multi-spatial region features of facial components and voice sensitive words within key time frames, thereby achieving spatiotemporally aligned multimodal emotion recognition.

Benefits of technology

It improves the accuracy and reliability of driver emotion recognition, meets the real-time and accurate requirements of safe driving for emotion recognition, and reduces misjudgments caused by spatiotemporal misalignment.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application discloses a kind of multi-modal space-time alignment safe driving emotion recognition methods based on sensitive word guide.The application extracts sensitive word features by driving monitoring video and voice data.With the analysis sensitive word-voice text time attention, determine the multiple candidate time intervals that possibly exist emotion in video.With the sensitive word-visual time cross attention, adaptively determine the key time frame that exists driving emotion.In key time frame, extract component features in the multiple space regions of human face component, analyze its alignment relationship with voice sensitive word, with the sensitive word-visual space cross attention, adaptively determine the key space region that exists driving emotion.The application fully considers the space-time asynchronization of sensitive word in multi-modal data, sequentially analyzes candidate time interval, key time frame, key space region, gradually finds the key clue of driving emotion, and can effectively improve the accuracy of safe driving emotion recognition.
Need to check novelty before this filing date? Find Prior Art