Second Eye Viewport Synthesis via Interleaving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based gaming platforms face high latency issues due to network congestion and distance, which can lead to frustrating delays and lag in real-time gameplay, exacerbated by buffering mechanisms that increase perceived latency in gaming applications.
Innovation Solution
A client device and method that utilize interleaving to synthesize a second eye viewport by obtaining current and previously rendered rendering surfaces, interpolating pixel data, and displaying the synthesized viewport, predicting future user navigation inputs to render extra content, and using subsampling and time-shifting to reduce bandwidth and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If buffering is used to reduce perceived delays in streaming content, then playback continuity is improved, but communication time lag between user input and content response increases
Solution Approach 1:
The system performs preliminary rendering of multiple future frames at the server before they are requested by the client. The server anticipates user navigation inputs and pre-renders content that may be needed, storing it in a content buffer. This allows the client to immediately display pre-rendered frames without waiting for rendering time, significantly reducing the communication time lag while maintaining playback continuity.
Solution Approach 2:
The system dynamically adjusts the rendering and buffering strategy based on predicted user navigation inputs. When user input indicating a navigation action is detected, the system adapts by pre-rendering frames corresponding to the anticipated navigation destination. This dynamic adaptation allows the system to optimize between playback continuity and response time based on actual user behavior patterns.
2Manufacturing precision
If server-side rendering is used to provide high-end graphics on less powerful client devices, then graphical performance is improved, but network bandwidth consumption increases
Solution Approach 1:
The system renders more frames than immediately necessary at the server (excessive action), creating a content buffer with multiple pre-rendered frames. This allows the client to receive and display frames more efficiently, reducing the total network bandwidth consumption over time. By rendering frames in advance, the system avoids the need for continuous high-bandwidth streaming of individually rendered frames.
Solution Approach 2:
The server performs preliminary rendering of multiple future frames before they are needed by the client. This pre-rendering approach allows the system to optimize bandwidth usage by transmitting frames that were already rendered, rather than continuously requesting and receiving individually rendered frames, thereby reducing overall network bandwidth consumption while maintaining high graphical performance.
3Stability of the object's composition
If content is buffered to absorb latency delays, then playback smoothness is improved, but user perception of latency in interactive gaming increases
Solution Approach 1:
The system pre-renders multiple frames at the server and buffers them before they are needed. When a frame is requested by the client, it can be immediately displayed from the pre-rendered buffer without waiting for the rendering process. This eliminates the perceived latency for the user while maintaining smooth playback, as the content is already prepared and ready for immediate display.
Solution Approach 2:
The system creates copies of rendered frames in advance and stores them in a content buffer. Instead of rendering frames on-demand at the client device (which would cause latency), the server creates and buffers multiple frame copies beforehand. The client then retrieves and displays these pre-copied frames, achieving both smooth playback and minimal perceived latency.
Data Source
AI summary
A client device and method are provided for use in synthesizing a second eye viewport using interleaving in order to reduce bandwidth costs. The client device may comprise a user input device that receives a stream of user input, a stereoscopic display device, a processor, and a storage device holding instructions for a client application program, executable by the processor to obtain, at a view interpolation module of the client application program, a current rendered rendering surface representing a current view of a scene for a first eye of a user and a previously rendered rendering surface representing a past view of the scene for a second eye of the user, synthesize and display a current second eye viewport representing a current view of the scene for the second eye of the user based on the current rendered rendering surface and the previously rendered rendering surface.