3D Motion Mapping for Cloud Gaming Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based gaming systems face significant latency issues due to processing delays and network traffic, primarily caused by computationally intensive motion search in video encoding, which is GPU-unfriendly and affects the gaming experience.
Innovation Solution
Implementing a 3D motion mapping technique that uses geometric information to construct motion vectors quickly, reducing processing delays by selectively performing reverse 3D geometric transformations and occlusion detection on anchor pixels, thereby simplifying the encoding process and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional motion search is used in video encoding, then motion estimation accuracy is improved, but processing time and latency increase significantly
Solution Approach 1:
The patent performs reverse 3D geometric transformation on anchor pixels before the main encoding process to pre-calculate their positions in the reference frame. This preliminary action provides ready-to-use position information that eliminates the need for computationally intensive motion search during encoding, thus reducing processing time while maintaining accuracy for anchor pixels.
Solution Approach 2:
The patent extracts and processes only anchor pixels (a subset of pixels at specific locations) through reverse 3D transformation, rather than performing motion search on all pixels. This selective extraction reduces the computational burden significantly while still providing sufficient motion information for efficient encoding.
2Measurement precision
If comprehensive motion search is performed on all pixels, then motion vector accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies different processing methods to different parts of the image: anchor pixels undergo reverse 3D transformation to obtain precise motion information, while other pixels use simplified motion compensation. This local differentiation maintains high accuracy where needed while reducing overall computational complexity.
Solution Approach 2:
The patent performs reverse 3D transformation on only a partial set of pixels (anchor pixels) rather than all pixels. This partial action provides sufficient motion information for efficient encoding without the excessive computational cost of processing every pixel, achieving a practical balance between accuracy and complexity.
3Productivity
If GPU-friendly encoding methods are used, then processing speed is improved, but motion search capability is reduced
Solution Approach 1:
The patent replaces the traditional CPU-based motion search algorithm with a GPU-friendly approach using reverse 3D geometric transformation. The reverse transformation leverages GPU's strength in parallel geometric calculations, substituting the iterative motion search mechanism with a direct geometric computation that is highly efficient on GPU architectures.
Data Source
AI summary
Three-dimensional motion mapping is disclosed. A reverse three-dimensional transform for a set of anchor pixels in the current frame is performed using a model-view matrix, a projection matrix, and view parameters associated with the computer game. Reverse transformed anchor pixels are indicated as occluded if the estimated depth for the pixel in the previous frame is greater than a real depth for the pixel in the previous frame or otherwise indicated as visible. A motion vector map for a subset of the reverse transformed pixels is constructed. A best target motion vector for a first set of sections in the current frame is derived or motion estimation is performed for sections in a second set that contain occluded pixels. Target motion vectors for groups of two or more sections of the current frame comprising two or more adjoining sections each with best target motion vectors are derived.