Panorama Audio Counter-Rotation for Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating panorama images with audio data face challenges in synchronizing and aligning audio sources with visual images, leading to rotating sound sources when played back, as audio is captured in different directions during image capture.
Innovation Solution
The solution involves capturing audio data with defined focus elements and modifying it to align with the visual panorama, using techniques like ambisonics and DirAC to counter-rotate audio based on device movement, ensuring sound sources appear to originate from consistent directions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If audio data is captured during panorama image capture, then audio-visual synchronization is achieved, but sound sources rotate and appear to originate from incorrect directions during playback
Solution Approach 1:
The system applies counter-rotation to the audio data before playback to pre-compensate for the rotation that will occur during panorama display. By rotating the audio in the opposite direction of the camera movement, the sound sources remain stationary relative to the viewer while the visual panorama rotates, resolving the directional mismatch between audio and visual elements
Solution Approach 2:
The system dynamically adjusts the audio rotation angle based on the captured camera movement trajectory. By continuously tracking the device orientation changes during panorama capture and applying corresponding dynamic counter-rotation to the audio data, the system maintains accurate sound source positioning throughout the entire playback sequence
2Quantity of substance
If audio capturing time frame is extended to cover the entire panorama capture duration, then complete audio coverage is achieved, but audio quality deteriorates due to including periods of device movement
Solution Approach 1:
The audio capture process is segmented into multiple quality tiers based on device movement characteristics. The system divides the capture period into high-quality segments (when device is stationary or moving smoothly) and lower-quality segments (during rapid or unstable movement), allowing selective processing or filtering of audio data to maintain overall quality while preserving complete temporal coverage
Solution Approach 2:
Different quality standards are applied to different portions of the audio capture based on local device stability conditions. Audio captured during stable periods is processed with higher fidelity settings, while audio from unstable periods may be processed differently or marked for selective playback, ensuring that each segment's quality matches the capture conditions
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
An apparatus, method and computer program is described comprising: capturing a plurality of visual images from an image capturing start time to an image capturing end time, for use in generating a panorama image; and capturing audio data relating to said visual images from an audio capturing start time to an audio capturing end time.