Multimedia Time Point Extraction via Audio-Video Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices struggle to accurately identify the time point at which a designated motion is captured in multimedia content, such as sports games, due to the complexity of synchronizing audio and video signals.
Innovation Solution
An electronic device equipped with at least one processor and memory, configured to receive requests for detecting sound time points in multimedia content. The device obtains a distribution of probabilities in the time domain for designated motions based on audio signals, using a neural network to identify peak values corresponding to specific time points. By synchronizing video and audio signals, the device accurately determines the sound time point when the designated motion is captured.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If audio signal analysis is used to identify sound time points, then the ability to detect designated motions is improved, but the accuracy is reduced due to multiple peak values and synchronization complexity
Solution Approach 1:
The patent introduces a video signal as an intermediary to resolve the ambiguity in audio signal peak detection. When multiple peak values are detected in the audio signal, the video signal serves as a mediator to determine which peak corresponds to the actual designated motion, thereby improving time point accuracy while maintaining detection capability
Solution Approach 2:
The patent transitions from one-dimensional audio signal analysis to two-dimensional analysis by incorporating video signal data. This dimensional expansion allows the system to disambiguate between multiple audio peaks by cross-referencing visual information, thus resolving the accuracy issue without sacrificing detection sensitivity
2Adaptability or versatility
If multiple peak values are considered in audio signal, then more potential time points are identified, but the precision of identifying the correct sound time point deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the video signal provides verification information back to the audio peak selection process. The system detects multiple audio peaks, then uses video frame analysis to feedback which peak truly corresponds to the designated motion, ensuring both comprehensive coverage and high precision in time point identification
Data Source
AI summary
An electronic device may include memory for storing instructions, and at least one processor operably coupled to the memory. The at least one processor, when the instructions are executed, is configured to receive a request for detecting a sound time point in a multimedia content when a designated motion is captured; obtain a distribution of probabilities in a time domain that the designated motion is performed based on an audio signal in the multimedia content, wherein the distribution of probabilities comprises a plurality of peak values corresponding to respective time points in the multimedia content; and obtain the sound time point when the designated motion is captured from among the respective time points corresponding to the plurality of peak values, using a video signal synchronized to the audio signal, in the multimedia content.