Combined Video Audio Summary Interface for Event Detection

The patent merges video summary generation and audio summary generation into a unified system that processes both video and audio data independently yet complementarily. The video summary module detects video events while the audio summary module detects audio events, and both summaries are integrated to provide comprehensive event detection that captures both visual and auditory information without relying solely on one data type.

The patent introduces a combined summary generation module that acts as an intermediary between raw audio data and user comprehension. This module processes audio events in chronological order to generate a structured audio summary that maintains temporal context, while also integrating with video summary to provide multi-modal context. The intermediary structure allows users to grasp audio content meaningfully even when not listening to the original chronological audio stream.

The patent segments the audio data into discrete audio events based on detected audio patterns, and organizes these segmented events chronologically in the audio summary. This segmentation allows the system to process and present audio information in manageable, time-ordered units rather than overwhelming users with all audio data simultaneously. The segmented structure maintains recognition accuracy by preserving temporal relationships while reducing information density.

The patent applies preliminary action by generating video and audio summaries independently in separate processing modules before the synchronization step. The video summary module processes video data to identify video events, while the audio summary module processes audio data to identify audio events. Both summaries are prepared with metadata including timestamps and event characteristics, enabling efficient synchronization in the subsequent integration phase without requiring complex real-time coordination during generation.