Server-Side Rendering Block Classification for Video Streaming Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing moving image coding techniques for multiuser online network games are inefficient in performing real-time rendering, particularly in determining whether to perform intra or inter-frame coding, which hinders high-speed response to user input and limits the number of users due to high rendering performance requirements.
Innovation Solution
A moving image distribution server that acquires rendered screens, viewpoint information, and depth buffers, divides the screen into blocks, sets evaluation pixels, compares depth values to determine coding methods, and transmits coding details before generating coded data, allowing for efficient intra or inter-frame coding based on block analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If inter-frame coding with motion compensation is performed to achieve high compression efficiency, then the compression ratio is improved, but the processing time increases due to analysis of similarity and distance calculation
Solution Approach 1:
The patent performs preliminary classification of blocks into intra-coding candidate blocks and inter-coding candidate blocks based on depth buffer comparison before actual coding. This preliminary action avoids time-consuming motion compensation analysis for blocks that will use intra-coding, thereby reducing overall processing time while maintaining compression efficiency for suitable blocks.
Solution Approach 2:
The patent divides the screen into multiple blocks and performs independent classification for each block by comparing depth values. This segmentation allows parallel processing of different blocks and enables selective application of coding methods, reducing the total processing time compared to uniform inter-frame coding of the entire frame.
2Manufacturing precision
If high-performance client devices are required to execute rendering processing, then the rendering quality is improved, but the number of users is limited due to device cost
Solution Approach 1:
The patent replaces the mechanical rendering system on client devices with a server-based rendering system. The server performs all rendering processing and transmits coded images to client devices, which only need to decode and display. This substitution allows high-quality rendering on the server while enabling user access on low-performance devices.
Solution Approach 2:
The patent creates coded copies of rendered images on the server and transmits these copies to client devices. The client devices do not need to perform rendering themselves; they only need to decode the transmitted coded image data. This copying approach maintains rendering quality while reducing client device requirements.
3Ease of operation
If real-time rendering with high-speed response to user input is required, then the interactivity is improved, but the coding processing time must be reduced to meet response requirements
Solution Approach 1:
The patent performs preliminary classification of blocks using depth buffer comparison before actual image coding. This preliminary action quickly identifies which blocks are suitable for inter-frame coding without requiring time-consuming motion compensation analysis, thereby reducing overall coding processing time and enabling faster response to user input.
Solution Approach 2:
The patent applies different coding methods to different blocks based on their characteristics determined by depth value comparison. By locally optimizing each block's coding method rather than applying a uniform approach, the system achieves faster processing while maintaining overall image quality, thus improving response speed.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A moving image distribution server acquires in series a rendered screen, viewpoint information used for rendering the screen, and a depth buffer corresponding to the screen. It sets a evaluation pixel for each of the blocks that it divided a first screen into, and specifies a screen coordinate and a depth value, in a second screen acquired before the first screen, for a rendering object rendered on the evaluation pixel. Also, it compares a depth value of the screen coordinate and the depth value of the rendering object rendered on the evaluation pixel, and determines that inter-frame coding will be performed with the second screen for a block for which the 2 depth values are considered to be the same. Then, it transmits, to a external device, coding details information including the viewpoint information used for rendering of the first screen, the depth buffer corresponding to the first screen, and information indicating whether or not blocks of the first screen are to be inter-frame coded.