Multi-directional Image Viewport Shift for Coding Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding techniques for multi-directional imaging applications result in unnecessary processing latencies when generating viewport data for display, as they decode the entire image before rendering, leading to delayed rendering of the desired viewport data.
Innovation Solution
The proposed solution involves shifting the content of multi-directional image data based on viewport location data provided by the decoder, using predictive coding to accelerate the coding process, and transmitting the coded data along with shift information, allowing the decoder to extract and display the viewport data efficiently without decoding the entire image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the decoder decodes the entire multi-directional image before rendering, then complete image data is available for display, but processing latency increases and rendering is delayed
Solution Approach 1:
The patent segments the multi-directional image decoding process by identifying and prioritizing viewport-related data regions. Instead of uniformly decoding the entire image, the system divides the image into relevant viewport regions and decodes them with higher priority, reducing overall processing latency while ensuring complete viewport data availability.
Solution Approach 2:
The patent employs preliminary action by performing early identification of viewport parameters and pre-processing of metadata related to viewport extraction. This allows the decoder to prepare extraction pathways in advance, so when viewport data is needed, the processing pipeline is already optimized and ready, significantly reducing rendering latency.
2Loss of energy
If video compression techniques are used on the entire multi-directional image, then data transmission efficiency improves, but decoding time increases and delays viewport rendering
Solution Approach 1:
The patent applies local quality by using different compression strategies for different regions of the multi-directional image. Viewport-related regions are encoded with higher quality and priority, while non-viewport regions use more aggressive compression. This localized approach maintains transmission efficiency while reducing decoding time for critical viewport data.
Solution Approach 2:
The patent implements partial action by decoding only the necessary portion of the compressed image data required for viewport rendering, rather than fully decoding the entire compressed image. This allows the system to achieve viewport rendering with a fraction of the total decoding time, while still maintaining efficient data transmission for the complete image.
3Loss of substance
If the display device extracts viewport data from the multi-directional image, then only necessary data is displayed, but the entire image must still be decoded first
Solution Approach 1:
The patent uses preliminary action by pre-identifying viewport parameters and preparing extraction metadata before the actual viewport rendering occurs. This preliminary setup enables the decoder to directly extract viewport data from the compressed stream without waiting for complete image decoding, significantly reducing rendering delay while maintaining efficient data processing.
Solution Approach 2:
The patent introduces an intermediary layer of viewport parameter metadata that mediates between the compressed image data and the final viewport extraction. This intermediary structure allows the system to navigate directly to viewport regions in the compressed domain, avoiding the need to fully decode the entire image first, thus reducing both processing volume and time.
Data Source
AI summary
Image processing techniques may accelerate coding of viewport data contained within multi-view image data. According to such techniques, an encoder may shifting content of a multi-directional image data according to the viewport location data provided by a decoder. The encoder may code the shifted multi-directional image data by predictive coding, and transmit to the decoder, the coded multi-directional image data and data identifying an amount of the shift. Doing so may move the viewport location to positions in the image data that are coded earlier than the positions that the viewport location naturally occupies and, thereby, may accelerate coding. On decode, a decoder may compare its present viewport location with viewport location data provided by the encoder with coded video data. The decoder may decode the coded video data and extract a portion of the decoded video data corresponding to a present viewport location for display.