Optimizing Remote Procedure Calls in Distributed GPU Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed systems, the inefficiency caused by network latency and potential failures hinders the effective use of remote computing nodes, particularly GPUs, for tasks like deep learning and machine learning, as existing solutions fail to optimize the interaction between clients and hosts in these systems.
Innovation Solution
A method and apparatus that optimize the execution of remote procedure calls by determining the type of command and adjusting it to reduce latency, using techniques such as caching query results, creating virtual handlers, and optimizing data copy operations within the same device, thereby minimizing network interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If remote procedure calls are used to access computing nodes in a distributed system, then computing tasks can be distributed across multiple nodes, but network latency and potential failures cause the application to operate inefficiently
Solution Approach 1:
The patent caches query results and creates virtual handlers in advance before they are actually needed. By pre-computing and storing results of frequently accessed computing nodes, the system avoids repeated network round-trips, thereby reducing latency and improving application operation efficiency.
Solution Approach 2:
The patent introduces virtual handlers as intermediary objects that localise access to remote computing nodes. These virtual handlers act as proxies, allowing the application to interact with remote resources through local interfaces, thereby reducing the impact of network latency and potential failures.
2Productivity
If multiple remote procedure calls are made to computing nodes, then distributed computing capabilities are utilized, but frequent network interactions increase latency and reduce performance
Solution Approach 1:
The system pre-creates virtual handlers for multiple computing nodes and caches their query results beforehand. This preliminary preparation allows subsequent operations to proceed locally without repeated network interactions, significantly reducing network interaction time while maintaining distributed computing capabilities.
Solution Approach 2:
The patent creates virtual copies (virtual handlers) of remote computing node interfaces locally. These copies replicate the essential functionality of remote nodes, allowing the application to perform operations locally and only synchronize with remote nodes when absolutely necessary, thereby reducing network interaction time.
3Adaptability or versatility
If the application directly calls remote computing nodes, then remote resources are accessed, but network failures and latency make the application operate inefficiently
Solution Approach 1:
The patent introduces virtual handlers as intermediary objects between the application and remote computing nodes. These virtual handlers provide a stable local interface that abstracts away network uncertainties, allowing the application to operate reliably even when remote nodes are inaccessible or experiencing latency.
Solution Approach 2:
The system caches query results and prepares virtual handlers in advance to cushion against potential network failures. By having local copies and cached data ready beforehand, the application can continue operating even when remote nodes become unavailable, thereby improving reliability and operational stability.
Data Source
AI summary
According to one example embodiment of the present disclosure, there is provided a method for optimization in a distributed system, where the distributed system comprises a client and multiple hosts among which a host comprises a computing node. The method comprises: receiving a first command requesting to use the computing node from an application at the client; determining the type of the first command; and adjusting the first command on the basis of the type of the first command to optimize the execution of the first command in the distributed system, where the computing node is a graphics processing unit, and the first command is a remote procedure call of the graphics processing unit.