Audiovisual Remapping for VR Aspect Ratio Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing approaches for processing audiovisual data in virtual reality (VR) and augmented reality (AR) applications often result in image distortion due to aspect ratio mismatches between capture devices and display devices, leading to an inconsistent user experience, and intelligent upscaling algorithms require significant computational resources and large learning databases, making them unsuitable for real-time end-user devices.
Innovation Solution
An apparatus and method that includes a receiver for audiovisual data, an image remapper for content-dependent non-uniform mapping of images to match the aspect ratio of the display, and an audio remapper that modifies spatial properties of audio elements based on the mapping data to ensure consistent and immersive experiences across different devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If intelligent upscaling algorithms are used to adapt image aspect ratios, then image quality is improved, but computational complexity and resource requirements increase significantly
Solution Approach 1:
The patent extracts the computationally intensive intelligent upscaling algorithm from the end-user device and relocates it to a server environment. The device only receives pre-processed audiovisual data and performs simple rendering, while the server handles the complex image adaptation tasks using AI models trained offline.
Solution Approach 2:
The patent applies intelligent upscaling and aspect ratio adaptation in advance during content encoding and preparation, before distribution to end devices. The server pre-processes the audiovisual content with computationally expensive AI algorithms, so that client devices receive ready-to-display content requiring minimal local processing.
2Adaptability or versatility
If aspect ratio adaptation is applied to match display devices, then compatibility is improved, but image distortion occurs
Solution Approach 1:
The patent applies different processing strategies to different regions of the image based on content analysis. Important visual elements are preserved with minimal distortion while less critical areas are adapted to fill the display aspect ratio. The system identifies and protects key visual information during the remapping process.
Solution Approach 2:
The patent implements dynamic aspect ratio adaptation that adjusts processing parameters based on the specific content being displayed. The system continuously analyzes the audiovisual stream and adapts the remapping strategy in real-time to maintain image quality while ensuring compatibility across different display devices with varying aspect ratios.
3Stability of the object's composition
If spatial properties of audio elements are modified to match image remapping, then audio-visual consistency is improved, but processing complexity increases
Solution Approach 1:
The patent merges the audio remapping process with the image remapping operation by using the same mapping data generated during visual processing. The audio spatial properties are adjusted using the identical transformation parameters derived from the image aspect ratio adaptation, ensuring synchronized audio-visual output without requiring separate complex audio processing pipelines.
Solution Approach 2:
The patent uses mapping data as an intermediary that connects and coordinates both audio and video processing. This mapping data, generated during image remapping, serves as a shared reference that simultaneously guides visual transformation and audio spatial adjustment, simplifying the overall processing architecture while maintaining audio-visual consistency.
Data Source
AI summary
An apparatus for processing audiovisual data for a scene comprises a receiver (201) for receiving audiovisual data for the scene. The audiovisual data comprises audio data for the scene comprising a plurality of audio elements and image data for at least a first image of the scene where the first image has a first aspect ratio. An image remapper (203) performs a content dependent non-uniform mapping of the first image to a second image which has a different aspect ratio. The image remapper (207) is arranged to generate mapping data describing the content dependent non-uniform mapping. An audio remapper (207) replaces a first audio element of the plurality of audio elements by a second audio element generated by modifying a spatial property for the first audio element in response to the mapping data. The spatial property being modified may be a position and/or spatial spread of the first audio element.