Immersive Video Frame Processing via Predicted Pose Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Displaying immersive video efficiently is challenging due to the computational expense of processing entire frames, especially when the device pose changes between processing and display times, leading to time-consuming processing and potential inaccuracies in frame presentation.
Innovation Solution
A method is implemented where a predicted pose of a device is used to select a processing spatial portion of the frame, which is then processed, and subsequently, a display spatial portion is determined based on the current pose for display, optimizing processing power and ensuring accurate frame presentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire frame is processed for immersive video, then the video quality is maintained, but the computational burden and processing time increase significantly
Solution Approach 1:
The patent divides the frame into multiple spatial portions and processes only the selected processing spatial portion based on predicted pose, rather than processing the entire frame. This segmentation approach maintains video quality for the relevant region while significantly reducing computational burden on the entire frame.
Solution Approach 2:
The patent applies image processing selectively to specific local regions (processing spatial portion) of the frame that are predicted to be displayed, rather than uniformly processing the entire frame. This ensures high quality where needed while reducing overall processing requirements.
2Loss of information
If image processing is performed on the entire frame, then complete video information is preserved, but processing time increases and may cause delays
Solution Approach 1:
The patent performs pose prediction in advance before selecting the processing spatial portion for image processing. This preliminary action allows the system to pre-determine which frame portion will be displayed, enabling selective processing that reduces time loss while ensuring the correct information is processed and displayed.
Solution Approach 2:
By segmenting the frame and processing only the predicted relevant portion, the patent minimizes information loss in the displayed region while dramatically reducing processing time compared to full-frame processing.
3Adaptability or versatility
If the device pose changes between processing and display times, then dynamic viewing is enabled, but processing inaccuracies and presentation errors occur
Solution Approach 1:
The patent performs pose prediction before selecting the processing spatial portion, anticipating the device pose at display time. This preliminary pose estimation allows the system to pre-process the correct frame region, reducing inaccuracies caused by pose changes between processing and display.
Solution Approach 2:
The patent dynamically adjusts the processing spatial portion selection based on predicted pose, allowing the system to adapt to changing device orientations while maintaining processing accuracy through predictive compensation for pose changes.
Data Source
AI summary
In one implementation, a method of displaying immersive video is performed at a device having a display, one or more processors, and non-transitory memory. The method includes obtaining an image. The method includes selecting a processing spatial portion of the image based on a predicted pose of the device. The method includes performing image processing on the processing spatial portion. The method includes, after performing image processing on the processing spatial portion, selecting a display spatial portion of the processing spatial portion based on a current pose of the device. The method includes displaying, on the display, the display spatial portion.