Multi-View Video Codec for Smooth Foveated Region Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Foveated imaging systems face challenges in managing changing areas of interest, leading to edge errors and inefficiencies due to the need to decode multiple image streams in parallel.
Innovation Solution
A multi-view video codec system encodes multiple copies of an image stream with varying resolutions, focusing on specific areas of interest, using warping techniques and selective blurring to smooth transitions as the area of interest changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple image streams with different foveations are decoded in parallel to handle changing areas of interest, then the system can adapt to different regions, but edge errors are introduced during transitions between image streams
Solution Approach 1:
The patent implements dynamic area of interest tracking by monitoring motion vectors and disparity maps to automatically identify and switch between different foveated regions. The system adapts the high-resolution area location in real-time based on scene content analysis, eliminating the need for manual configuration and enabling seamless transitions without edge errors
Solution Approach 2:
The system uses feedback from motion estimation and depth map analysis to continuously adjust the foveated imaging parameters. By analyzing optical flow and disparity information, the system determines the current area of interest and adjusts the high-resolution region accordingly, ensuring accurate tracking and preventing transition errors
2Adaptability or versatility
If multiple image streams are decoded in parallel to support foveated imaging, then changing areas of interest can be managed, but computational complexity and processing overhead increase
Solution Approach 1:
The patent segments the image processing into distinct functional modules: motion estimation unit, disparity map generation unit, area of interest determination unit, and foveated decoding unit. Each module handles a specific aspect of the processing pipeline, allowing independent optimization and reducing overall system complexity while maintaining parallel processing capabilities
Solution Approach 2:
The system performs preliminary analysis of motion vectors and disparity maps before initiating full decoding of multiple image streams. By pre-identifying the area of interest using low-complexity motion estimation, the system avoids the need to fully decode multiple high-resolution streams simultaneously, significantly reducing computational overhead
3Loss of energy
If foveated imaging is used to optimize bandwidth usage, then transmission efficiency improves, but transition smoothness between different foveated regions deteriorates
Solution Approach 1:
The patent applies preliminary anti-action by using selective blurring to pre-compensate for potential transition discontinuities. The blur filter is applied to the periphery of the high-resolution area before transitions occur, creating a smooth gradient that prevents visible edge artifacts when the foveated region shifts, thus maintaining transition smoothness while preserving bandwidth efficiency
Data Source
AI summary
Encoding an image stream may include receiving an image stream with an original image resolution; generating a plurality of copies of the image stream with the original image resolution; encoding, for each copy of the plurality of copies of the image stream, the copy of the image stream to generate an encoded copy of the image stream, wherein the encoded copy of the image stream comprises a first region having a first image resolution and a second region having a second image resolution, wherein each encoded copy of the plurality of encoded copies of the image stream has a different first region, and providing, to a playback device, at least one encoded copy of the plurality of encoded copies of the image stream.