Device-Cloud 3D Rendering With Intermediate-Result Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing rendering technologies face high interaction delays and inefficient bit rate overheads due to the reliance on cloud-side servers for image/video processing, which is exacerbated by increasing image quality demands and device computational limitations.
Innovation Solution
A device-cloud collaboration system where the terminal device performs initial rendering and encoding based on intermediate rendering results, reducing the need for cloud-side transmission of intermediate results, and utilizing optimized encoders and decoders to enhance encoding efficiency and reduce bit rate overheads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If cloud-side server performs rendering and transmits compressed rendered image/video to device-side device, then image quality and resolution can be improved, but bit rate overheads increase and network bandwidth occupation increases
Solution Approach 1:
The patent segments the rendering and encoding process into two parts: cloud-side server performs rendering and generates intermediate rendering results, while device-side device performs encoding based on these intermediate results. This segmentation allows the cloud to handle computationally intensive rendering while the device handles the encoding, reducing the bit rate overheads that would result from transmitting complete high-resolution rendered images.
Solution Approach 2:
The cloud-side server performs preliminary rendering to generate intermediate rendering results before transmission. These intermediate results serve as a foundation for subsequent encoding at the device side, allowing the system to achieve high image quality while minimizing the amount of data that needs to be transmitted.
2Quantity of substance
If cloud-side server transmits intermediate rendering result to device-side device for upsampling, then bit rate overheads can be reduced, but encoding efficiency remains low
Solution Approach 1:
Instead of the conventional approach where the device simply upsamples received low-resolution images, this patent inverts the process by having the device perform encoding operations on intermediate rendering results. This inversion transforms the device from a passive display unit to an active encoding participant, significantly improving encoding efficiency while maintaining reduced bit rate overheads.
Solution Approach 2:
The device-side device performs encoding operations autonomously based on intermediate rendering results received from the cloud. This self-service approach eliminates the need for complex cloud-based encoding operations and reduces the bit rate overheads while improving encoding efficiency through local processing capabilities.
3Quantity of substance
If device-side device performs upsampling on rendered low-resolution image/video to generate high-resolution image, then transmission data can be reduced, but interaction delay increases
Solution Approach 1:
The cloud-side server performs preliminary rendering to generate intermediate rendering results that contain sufficient information for high-quality reconstruction. By preparing these intermediate results in advance with appropriate structure and detail, the system reduces the need for extensive post-processing upsampling operations, thereby reducing interaction delay while maintaining reduced transmission data requirements.
Data Source
Figure 1a
Figure 1b
Figure 2
AI summary
Embodiments of this application provide a device-cloud collaboration system, an encoding and decoding method, and an electronic device. The encoding method includes: performing rendering processing on a three-dimensional scene based on a rendering parameter, to obtain a rendered image; selecting a first intermediate rendering result based on an intermediate rendering result generated in a rendering processing process; encoding the rendered image based on the first intermediate rendering result, to obtain encoded data of a residual block, and encode the encoded data of the residual block into a bitstream. The first intermediate rendering result acts on at least one type of the following processing in an encoding process: partitioning, prediction, or filtering, the bitstream does not include encoded data of the first intermediate rendering result, and a second intermediate rendering result is a part of the first intermediate rendering result. In this way, a server may not send an intermediate rendering result to the terminal device, to reduce an interaction delay while it is ensured that overheads of a data stream transmitted by the server to the terminal device are effectively reduced.