Server-Side View Rendering Gap Filling for Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based streaming platforms face issues with high latency due to network congestion and distance, leading to pauses and delays in content streaming, and post-render image warping methods result in undesirable visual artifacts by revealing occluded pixels and failing to account for connected neighboring pixels.
Innovation Solution
The system employs multiple server-rendered views to accurately render gaps formed by post-render image warping, using pixel information from different perspectives to fill in gaps and reduce visual artifacts, leveraging server processing power to compensate for latency and improve image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If post-render image warping is used to compensate for latency, then responsiveness is improved, but visual artifacts increase due to revealed occluded pixels and broken neighboring pixel connections
Solution Approach 1:
The patent introduces an intermediary gap rendering process between the server-rendered current view and the final displayed image. This intermediary process identifies gaps caused by warping and fills them using pixel information from alternative sources (predicted view, prior views, or server-rendered views), thereby mediating between the need for rapid warping and the need for image quality
Solution Approach 2:
The system performs preliminary actions by pre-rendering predicted views and storing prior views before they are needed for gap filling. This allows the gap rendering process to quickly access pre-prepared pixel information without adding significant processing delay, thus maintaining responsiveness while improving image quality
2Manufacturing precision
If server processing power is used to render multiple views, then image quality is improved, but device complexity increases
Solution Approach 1:
The patent segments the rendering task into distinct components: the server renders multiple views (current view, predicted view, and historical views), while the client device performs only the gap identification and selection process. This segmentation allows complex multi-view rendering to be performed on the server without proportionally increasing client device complexity
Solution Approach 2:
The system performs partial rendering actions by having the server render only the specific views needed for gap filling (predicted view and selected historical views) rather than rendering all possible views. This reduces unnecessary processing complexity while maintaining image quality
Data Source
AI summary
A first user input is received when a client program executed by a client computing device is in a first state. The first user input is sent to a server computing device to render a view of a virtual scene. A state change from the first state in the client program due to a second user input or a program event is identified. One or more gaps in a server-rendered current view due to the state change are determined. A rendering of the one or more gaps is selected from among the server-rendered current view, a server-rendered predicted view and one or more prior-rendered views. A current view is rendered using a simplified model of the virtual scene by rendering the one or more gaps from the selected rendering. The current rendered view is visually presented via a display of the client computing device.