Graphics API Call Streaming for Low-Latency Remote Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graphics processing methods in client-server architectures require frequent bidirectional data transmission and high network latency, especially in wide area networks, leading to poor user experience due to the need for serial graphics API calls and waiting for processing notifications from the client.
Innovation Solution
Implement unidirectional transmission of graphics API calls from the server to the client, allowing the server to independently execute graphics processing without relying on client feedback, reducing network latency and enabling seamless operation even in disconnected networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If bidirectional data transmission is used for graphics API calls, then the server can receive processing notifications from the client, but the network latency increases significantly in wide area networks
Solution Approach 1:
The server pre-records the calling information of graphics APIs in a queue before sending them to the client. This preliminary action allows the server to continue executing subsequent graphics API calls without waiting for processing notifications from the client, thereby reducing network latency while maintaining processing reliability.
Solution Approach 2:
The graphics processing is segmented into two independent parts: the server records and sends calling information, while the client executes the graphics APIs locally. This segmentation eliminates the need for the server to wait for client feedback, reducing network latency while maintaining processing reliability through independent operation.
2Reliability
If the server waits for processing notifications from the client, then bidirectional communication is established, but the application program execution is interrupted
Solution Approach 1:
The server pre-records multiple graphics API calling information in a queue before sending to the client. This preliminary recording allows the server to continue executing subsequent graphics APIs without interruption, improving productivity while maintaining communication reliability through batch verification.
Solution Approach 2:
The server continuously executes graphics API calls and records their information without interruption, even while waiting for client processing notifications. This continuous execution maintains high productivity while ensuring communication reliability through asynchronous processing.
3Stability of the object's composition
If serial graphics API calls are made, then processing order is maintained, but the number of network round trips increases
Solution Approach 1:
The server pre-records multiple graphics API calling information in sequence in a queue before sending them to the client in a single batch. This preliminary recording maintains the processing sequence while reducing network communication overhead by eliminating multiple round trips.
Solution Approach 2:
Multiple graphics API calling information that would otherwise be sent in separate serial communications are merged into a single batch transmission to the client. This merging maintains the processing sequence through proper ordering while significantly reducing network communication overhead.
4Reliability
If the server relies on client feedback for each graphics API call, then processing accuracy is ensured, but the system cannot operate in disconnected networks
Solution Approach 1:
The server pre-records all necessary graphics API calling information locally in a queue before network transmission. This preliminary local recording enables the server to continue executing graphics processing accurately even when disconnected from the network, while maintaining processing accuracy through proper recording and playback of calling information.
Solution Approach 2:
The server creates a local copy of the graphics API calling information in its queue, which can be executed independently without real-time client feedback. This copying mechanism enables disconnected network operation while maintaining processing accuracy through faithful reproduction of the original calling sequence.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
This application discloses a graphics processing method and related apparatus, and a device. The method includes: obtaining a first drawing instruction that is initiated by an application program and that corresponds to a first graphics API; calling a first graphics API in a first graphics library according to the first drawing instruction to execute the first drawing instruction, and sending a first processing notification to the application program; generating calling information of the first graphics API, where the calling information of the first graphics API is used to instruct a client to call a first graphics API in a second graphics library, and the calling information of the first graphics API includes a parameter related to the first graphics API and an identifier of the first graphics API; and sending the calling information of the first graphics API to the client. According to the foregoing solutions, the server unidirectionally transmits calling information of a graphics API to the client, so that a requirement for a network latency can be reduced, and proper running of the application program on the server can be ensured in a disconnected network, thereby improving user experience.