Multi-View Video Codec for Smooth Foveated Stream Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Foveated imaging systems face challenges in managing changing areas of interest, leading to edge errors and inefficiencies due to the need for parallel decoding of multiple image streams with different foveations.
Innovation Solution
A multi-view video codec system encodes multiple copies of an image stream with varying resolutions, focusing on specific areas of interest, using warping techniques and selective blurring to smooth transitions as the area of interest changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple image streams with different foveations are decoded in parallel to handle changing areas of interest, then the system can maintain high resolution in the area of interest, but edge errors are introduced during transitions between image streams
Solution Approach 1:
The patent implements dynamic foveation by allowing the area of interest to change over time and automatically adjusting which image stream is decoded based on the current area of interest location. This dynamic adaptation eliminates the need for parallel decoding of multiple fixed foveation streams while maintaining high resolution where needed.
Solution Approach 2:
The patent introduces blending as an intermediary technique during transitions between different foveated image streams. By blending the outgoing and incoming image streams during transition periods, edge errors are smoothed out and visual artifacts are eliminated, providing seamless transitions.
2Measurement precision
If multiple image streams are decoded in parallel to handle changing areas of interest, then the system can maintain high resolution, but computational complexity and processing overhead increase
Solution Approach 1:
The patent extracts only the currently needed foveated image stream for decoding based on the present area of interest, rather than maintaining multiple parallel decoding streams. This extraction approach significantly reduces computational complexity while preserving high resolution in the active area of interest.
Solution Approach 2:
The system performs preliminary encoding of multiple foveated image streams during the encoding phase, so that during playback only one stream needs to be decoded at a time. This preliminary preparation enables fast switching between areas of interest without requiring parallel decoding.
3Loss of energy
If foveated imaging is used to optimize bandwidth usage, then transmission efficiency improves, but transitions between different areas of interest may exhibit visual artifacts
Solution Approach 1:
The patent uses blending as an intermediary process during transitions between foveated image streams. The blending operation combines the outgoing and incoming streams with time-varying weights, smoothly transitioning between them and eliminating visual artifacts while maintaining bandwidth efficiency.
Solution Approach 2:
The system prepares for transitions by initiating blending operations before the area of interest fully changes, cushioning against potential visual artifacts. This proactive blending approach ensures smooth transitions without abrupt changes or artifacts.
Data Source
AI summary
Encoding an image stream may include receiving an image stream with an original image resolution; generating a plurality of copies of the image stream with the original image resolution; encoding, for each copy of the plurality of copies of the image stream, the copy of the image stream to generate an encoded copy of the image stream, wherein the encoded copy of the image stream comprises a first region having a first image resolution and a second region having a second image resolution, wherein each encoded copy of the plurality of encoded copies of the image stream has a different first region, and providing, to a playback device, at least one encoded copy of the plurality of encoded copies of the image stream.