CXL Shared Memory Switching for Accelerator Data Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing solution of using coprocessors to enhance computing capability introduces a data transmission bottleneck due to the need for importing and exporting data through a host memory, which hampers computational efficiency.
Innovation Solution
Implementing a shared memory system managed by a host controller using Compute Express Link (CXL) protocol and Remote Direct Memory Access (RDMA) to directly write and read data between accelerators and the host, bypassing the host memory for data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If coprocessors are used to enhance computing capability, then computational power is improved, but data transmission efficiency deteriorates due to the need to import/export data through host memory
Solution Approach 1:
The patent extracts the data transmission bottleneck from the traditional host-coprocessor architecture by introducing a shared memory system that coprocessors can access directly. The shared memory is separated from host memory, allowing coprocessors to read/write data without going through the host memory interface, thus resolving the data transmission efficiency problem while maintaining enhanced computing capability.
Solution Approach 2:
The shared memory acts as an intermediary between the host and coprocessors. Instead of data flowing directly between host memory and coprocessor memory (which creates the bottleneck), the shared memory serves as a middle ground that coprocessors can access directly, eliminating the need for data to traverse the host memory interface and thereby improving transmission speed.
2Ease of operation
If data is imported/exported through host memory, then data access is simplified, but computational efficiency deteriorates due to the added data transmission path
Solution Approach 1:
The memory system is segmented into host memory and shared memory. Host memory remains managed by the host for control purposes, while shared memory is dedicated for coprocessor data access. This segmentation allows coprocessors to access data directly in shared memory without involving host memory operations, thereby maintaining ease of operation for host-controlled data while dramatically improving computational efficiency for coprocessor workloads.
3Speed
If shared memory is introduced for direct accelerator access, then data transmission speed is improved, but system complexity increases due to memory control switching
Solution Approach 1:
The host controller implements a feedback mechanism to manage shared memory access. The host controller receives feedback about which coprocessor needs access to the shared memory and dynamically switches control rights accordingly. This feedback-based control allows the system to maintain simple access patterns for each coprocessor while the host controller handles the complexity of switching control rights, thus improving data transmission speed without requiring each coprocessor to have complex built-in control logic.
Data Source
AI summary
The present application relates to the technical field of computers, and specifically discloses a data processing method, apparatus, and device, a storage medium, and a distributed cluster. Based on a compute express link (CXL) protocol, a memory of a first accelerator installed on a host becomes a shared memory, whereby when receiving a computing task, a host controller first switches a control right of the shared memory to a sender device to cause the sender device to write to-be-processed data of the computing task into the shared memory, then switches the control right to a second accelerator installed on the host to cause the second accelerator to read the to-be-processed data from the shared memory to complete the computing task, and switches the control right of the shared memory to the sender device or the host controller to read a computation result of the computing task from the shared memory.