Video Recording Audio Metadata for Deferred Cinematic Remixing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing portable devices struggle to capture high-quality audio recordings due to microphone placement away from the audio source, leading to degraded dialogue clarity and excessive background noise, with existing audio processing techniques failing to achieve cinematic audio quality in diverse environments.
Innovation Solution
Concurrently process audio signals during video recording to calculate statistics and generate metadata, which is stored in a video container along with the original audio and video signals, enabling deferred audio rendering with reduced power consumption and improved audio quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If microphones are co-located on the recording device, then device portability is improved, but audio recording quality deteriorates due to distance from audio source and background noise
Solution Approach 1:
The patent segments the audio processing into multiple stages: initial capture by co-located microphones, statistical analysis to identify sound classes and features, and deferred rendering with intelligent remixing. This segmentation allows the system to maintain portability while achieving high audio quality through multi-stage processing.
Solution Approach 2:
The patent performs preliminary actions by calculating audio statistics and generating metadata during the video recording capture phase. This preliminary processing prepares the audio data for subsequent high-quality rendering without requiring additional hardware, thus maintaining portability while improving audio quality.
2Manufacturing precision
If audio processing is performed in real-time during video recording, then audio quality is improved, but power consumption increases
Solution Approach 1:
The patent performs preliminary statistical analysis and metadata generation during video recording, but defers the computationally intensive audio rendering process to occur after recording is complete. This approach improves audio quality while managing power consumption by performing heavy processing only when needed, not continuously during recording.
Solution Approach 2:
The system uses the captured video recording and its associated audio statistics to automatically generate the mixed audio signal without requiring additional real-time processing resources. The deferred rendering leverages the already-captured data and metadata, making the system self-sufficient and reducing ongoing power requirements.
3Manufacturing precision
If complex audio processing techniques are applied, then audio fidelity is improved, but device complexity increases
Solution Approach 1:
The patent segments complex audio processing into manageable components: statistical analysis during recording to identify sound classes and features, metadata generation, and deferred intelligent remixing. This segmentation reduces immediate device complexity while enabling sophisticated audio processing capabilities.
Solution Approach 2:
The patent introduces metadata as an intermediary that bridges the captured audio signal and the final rendered output. This metadata contains statistical information and processing parameters that guide the deferred rendering process, simplifying the overall system architecture while enabling complex audio processing.
4Loss of time
If original audio signal is preserved without processing, then processing time is reduced, but audio quality deteriorates due to background noise and interference
Solution Approach 1:
The patent performs preliminary statistical analysis and metadata generation during video recording without altering the original audio signal. This preliminary action preserves the original audio for later use while preparing processing parameters in advance, thus maintaining both processing efficiency and audio quality.
Solution Approach 2:
The patent creates a copy of the audio signal statistics and metadata during recording, which then guides the deferred rendering process. The original audio signal remains unprocessed and intact, while the copied statistical information enables quality improvement during the rendering phase without adding real-time processing complexity.
Data Source
AI summary
A device may include a camera, one or more microphones, and one or more processors. The device can receive a video signal of a scene being produced by a camera and an audio signal of the scene being produced by the one or more microphones. While capturing a video recording, the device can digitally process the audio signal to determine that a first segment of the audio signal is in a first sound class and a second segment of the audio signal is in a second sound class and determine a plurality of features of the audio signal based on the segments and sound classes. After capturing the video recording, the device can generate metadata based on the plurality of features and store a video container comprising the video signal, the audio signal, and the metadata. Other aspects are also described and claimed.