Distributed Accelerator Pairing for Low-Latency Task Shunting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The parallel processing efficiency of distributed accelerator clusters is hindered by the need for a controller-side server to assign and schedule computing tasks, manage interactions, and perform context configuration, leading to high latency and low data processing rates.
Innovation Solution
A distributed computing method that establishes an accelerator direct-connection pair within the cluster, allowing idle accelerators with matching computing logic to execute tasks and shunt tasks to direct or indirect-connected accelerators via a controller, utilizing a computer express link protocol and extended memory for improved efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a controller-side server is used to assign and schedule computing tasks and manage interactions in a distributed accelerator cluster, then the system can achieve centralized control and coordination, but the parallel processing efficiency deteriorates due to high latency and low data processing rates
Solution Approach 1:
The system segments the cluster into direct-connection pairs of accelerators that can operate independently without continuous controller intervention. Each pair forms a self-contained unit that can execute tasks autonomously, reducing the bottleneck effect of centralized control while maintaining system coordination through the controller only when necessary.
Solution Approach 2:
The patent introduces a peer-to-peer communication mechanism as an intermediary between accelerators, allowing them to exchange data and coordinate directly without routing through the controller-side server. This intermediary channel eliminates the latency caused by centralized management while preserving the controller's ultimate authority over task assignment.
2Ease of operation
If accelerators are connected through the controller for task shunting, then centralized management is maintained, but the latency increases and data processing rate decreases
Solution Approach 1:
The network topology is segmented into direct-connection pairs, creating short-lived communication paths that bypass the controller for local operations. This segmentation allows accelerators within a pair to exchange data directly, eliminating the time penalty of controller-mediated communication while the controller retains management capabilities for task assignment and monitoring.
3Reliability
If traditional distributed cluster architecture is used with controller-mediated communication, then system coordination is achieved, but the computing performance is limited by controller bottlenecks
Solution Approach 1:
The system dynamically switches between controller-mediated coordination and direct peer-to-peer execution modes. For routine operations within a direct-connection pair, accelerators operate autonomously without controller intervention. The controller dynamically re-engages only when task assignment, scheduling decisions, or inter-pair coordination is required, optimizing the balance between coordination and performance.
Solution Approach 2:
The system performs preliminary setup by establishing direct-connection pairs and pre-configuring communication channels between accelerators before task execution begins. This preliminary action creates ready-to-use high-speed pathways that eliminate the need for real-time controller mediation during actual computing operations, thereby maximizing computing performance while maintaining coordination capability.
Data Source
AI summary
The present application discloses a distributed computing method, apparatus, device and system and a readable storage medium. The distributed computing method is applied to a controller of a distributed accelerator cluster and includes: acquiring information of accelerators in the distributed accelerator cluster; establishing an accelerator direct-connection pair according to the information of the accelerators; and in response to receiving a service task, dividing the service task into computing tasks and distributing the computing tasks to an idle target accelerator having an application computing logic matching the type of the corresponding computing tasks, wherein the accelerator direct-connection pair includes two accelerators that are directly connected to each other, and at least one of the two accelerators is a first accelerator that supports a computer express link protocol and has an extended memory, and/or the two accelerators have the same application computing logic.