Cached Multi-Device Commands Using Coherent Host Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing Compute Express Link (CXL.io) protocol for managing peripheral devices in high-performance computing environments is inefficient, requiring one-to-one communication and DMA transfers, which are time-consuming and limited by kernel space memory, especially when managing multiple identical devices.
Innovation Solution
Implementing a modified CXL.io protocol with a cached buffers flag that allows devices to access coherent host memory using cache coherency, enabling multicast and broadcast commands, and eliminating the need for DMA transfers by using coherent caches for data exchange.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If one-to-one communication protocol is used for managing peripheral devices, then device management is simple and reliable, but management time increases and productivity decreases when handling multiple devices
Solution Approach 1:
The patent combines multiple device management operations into a single multicast command that can be sent to multiple devices simultaneously. The host CPU issues one command that is received by multiple peripheral devices, each processing it independently. This merging of operations maintains reliability through individual device processing while dramatically improving productivity by eliminating the need for sequential one-to-one communication.
Solution Approach 2:
The patent segments the command processing at the device level. While the host sends a single multicast command, each peripheral device independently parses and processes the command according to its own state and capabilities. This segmentation allows parallel processing across multiple devices, resolving the contradiction between simple individual management and efficient batch management.
2Reliability
If DMA transfers are used for data exchange with peripheral devices, then data transfer is reliable, but kernel space memory is limited and constrains the number of concurrent devices
Solution Approach 1:
The patent introduces coherent cache as an intermediary between host memory and peripheral device memory. Instead of direct DMA transfers that require kernel space management, devices access host memory through coherent cache lines that maintain cache coherency. This intermediary approach preserves data transfer reliability while eliminating the kernel space memory constraint, allowing unlimited concurrent device access to host memory.
Solution Approach 2:
The patent replaces the mechanical DMA transfer mechanism with a cache-based memory access mechanism. Rather than using DMA engines that require pre-allocated kernel buffers, devices perform load/store operations through coherent cache, substituting the complex DMA infrastructure with simpler cache-managed memory access that scales to unlimited devices.
3Ease of operation
If separate command processing is performed for each peripheral device, then individual device control is precise, but management time increases and efficiency decreases
Solution Approach 1:
The patent merges command issuance into a single multicast operation that reaches multiple devices simultaneously. The command structure includes device-specific identifiers that allow each device to determine whether it should process the command. This combining of command distribution maintains precise individual control through device-specific processing while eliminating the time loss associated with sequential command issuance.
Solution Approach 2:
The patent implements preliminary filtering at the device level where each device checks the command against its own identifier or capability set before processing. This preliminary action allows devices to quickly determine relevance and skip unnecessary processing, maintaining precise individual control while reducing overall management time by avoiding redundant command processing across all devices.
Data Source
Figure 1
Figure 2
Figure 3~4A
AI summary
An example method of processing a command at a device sent by a host in a computing system, the host comprising a central processing unit (CPU) and host memory, is described. The method includes: receiving, at the device, the command from the host; parsing, by the device, the command to identify a flag in the command that is set and to obtain a first address that references a source buffer in the host memory; and reading, by the device in response to the flag being set, source data from the source buffer by reading from a coherent cache on the device using the first address, a cache coherency manager in the host managing the coherent cache.