Audio Gain Control Linked to Video Screen Zoom
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MPEG-H 3D Audio technologies fail to effectively link audio signals with video screen zooming, leading to a loss of immersive experience in VR environments due to lack of gain control and interaction between video and audio signals.
Innovation Solution
A method and apparatus that generate and render audio signals by adjusting object and channel gain values based on video screen zooming, using metadata processing and binaural rendering to maintain an immersive audio experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If video and audio are encoded independently without interaction, then encoding complexity is reduced, but immersive experience is degraded
Solution Approach 1:
The patent introduces screen information (zoom level, pan position, focus area) as an intermediary that mediates between video and audio encoding. This screen information is embedded in the bitstream and used by the audio decoder to adjust audio rendering parameters, enabling interaction between video and audio without complex direct coupling while maintaining immersive experience.
Solution Approach 2:
The patent applies preliminary action by pre-calculating and embedding screen information (zoom level, pan position, focus area) into the video bitstream during encoding. This allows the audio decoder to retrieve and use this information later for gain adjustment and audio object positioning, enabling synchronized audio-video interaction without real-time complex processing.
2Device complexity
If audio objects are not adjusted during screen zooming, then processing complexity is reduced, but audio realism is degraded
Solution Approach 1:
The patent applies dynamics by making audio rendering adaptive to dynamic screen zooming operations. Audio object gains and positions are dynamically adjusted based on the current zoom level and focus area retrieved from screen information, allowing audio realism to match the visual experience while using efficient parameter adjustment rather than complex re-rendering.
Solution Approach 2:
The patent changes audio parameters (gain values, position coordinates) based on screen zooming parameters. Specifically, audio object gains are adjusted according to zoom level, and audio object positions are shifted according to pan position, creating a simple yet effective mapping from visual parameters to audio parameters that maintains realism without complex processing.
3Device complexity
If gain control is not applied during zooming, then system complexity is reduced, but audio-visual synchronization is degraded
Solution Approach 1:
The patent applies local quality by adjusting audio gains locally for different audio objects based on their relevance to the current focus area. Audio objects within the zoomed focus area receive different gain adjustments compared to those outside, creating localized audio-visual synchronization that enhances realism without requiring global system complexity.
Data Source
AI summary
Disclosed are method and device for outputting audio linked with a video screen zoom. The method of the present invention comprises the steps of: receiving an audio signal linked with a video and generating a decoded audio signal and object metadata; linking with an execution of a video screen zoom so as to change respective object gain values for objects which are present in a zoom area; and rendering the decoded audio signal by using the changed object gain values. In addition, the method for outputting audio, of the present invention, further comprises a step for changing a channel signal-specific gain value which is linked with the video screen zoom. An audio can be linked, through the method and device for outputting audio, according to an embodiment of the present invention, in accordance with the video screen zoom such that the audio which is more realistic can be provided.