Dual-View Audio Processing for Synchronized Spatial Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video technologies provide a stereoscopic visual experience but lack synchronization between auditory and visual perception, resulting in a suboptimal user experience.
Innovation Solution
An audio processing method that adjusts audio based on image interface information, including displaying multiple camera views and processing sounds according to picture weights and focal lengths to achieve synchronized audio-visual three-dimensional experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional single-channel audio is used for video playback, then device complexity is low, but auditory perception cannot match visual perception to create synchronous spatial experience
Solution Approach 1:
The patent segments the audio processing into multiple independent channels corresponding to different camera views. Each audio channel is processed separately with its own spatial parameters, allowing the system to create multi-dimensional spatial audio experience while maintaining manageable processing complexity through modular architecture
Solution Approach 2:
The patent transitions from traditional mono or stereo audio to multi-channel spatial audio by adding dimensional elements. Audio signals are rendered in three-dimensional space with parameters including horizontal angle, vertical angle, and distance, creating a spatial audio field that matches the multi-view video's visual dimensionality
2Reliability
If audio is processed based on multiple picture weights and focal lengths, then synchronized audio-visual three-dimensional experience is achieved, but processing time and computational load increase
Solution Approach 1:
The patent performs preliminary calculations of picture weights and focal length parameters during the video shooting and editing phases. These pre-computed parameters are stored with the video data, so during playback the audio rendering system can directly use them without real-time computation, significantly reducing processing time while maintaining synchronization accuracy
Solution Approach 2:
The patent replaces complex real-time mechanical computation with pre-computed mathematical models. By substituting runtime processing with offline calculations and lookup tables, the system achieves reliable audio-visual synchronization without the computational burden of real-time processing
3Loss of information
If dual-view video is displayed with different picture weights, then visual information is optimized, but audio rendering complexity increases
Solution Approach 1:
The patent applies local quality by assigning different audio rendering parameters to different spatial regions corresponding to each camera view. Each picture weight region has its own optimized audio characteristics including directionality, volume, and spatial positioning, allowing audio rendering to match the local visual importance without requiring complete re-rendering of all audio channels
Data Source
AI summary
An audio processing method and an electronic device are provided. In a dual-view recording mode, the electronic device perform filtering, azimuth virtualization, and remixing on collected audio based on focal lengths of pictures in two display regions, relative positions of the two display regions, and values of areas of the two display regions, so that audio-picture presentation synchronization of the picture and the sound is achieved, and a user has synchronized three-dimensional experience in terms of hearing and vision.