Cache Coherent Interconnect Dedicated Register Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current CPU performance enhancement techniques, such as hardware accelerators, face challenges in achieving low latency and high bandwidth communication between CPUs and accelerators, which limits the efficient allocation and utilization of computing resources.
Innovation Solution
A method and system that utilize a cache coherent interconnect (CCI) and a dedicated cache register (DCR) to manage shared data, where the CPU indicates outdated data, requests and updates it, and marks the data as valid, ensuring continuous up-to-date data is provided to accelerators, thereby reducing latency and increasing bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If hardware accelerators are used to improve CPU performance, then processing speed is improved, but communication latency between CPU and accelerator increases
Solution Approach 1:
The patent segments the data path by introducing a dedicated cache register (DCR) as an intermediate storage structure between the CPU and hardware accelerator. This segmentation allows data to be pre-loaded and staged in the DCR, enabling the accelerator to access data without waiting for the CPU, thereby reducing communication latency while maintaining processing speed improvements.
Solution Approach 2:
The dedicated cache register (DCR) acts as an intermediary between the CPU and hardware accelerator. It receives data from the CPU via the cache coherent interconnect and makes it available to the accelerator, mediating the data transfer to reduce latency. The DCR includes validity bits and status flags that manage the data flow and synchronization between CPU and accelerator.
2Reliability
If data is continuously updated in the dedicated cache register, then data freshness is improved, but bandwidth consumption increases
Solution Approach 1:
The system performs preliminary actions by pre-loading data into the dedicated cache register (DCR) before the hardware accelerator needs it. The CPU can update data in the DCR in advance, and the accelerator can begin processing as soon as the data is ready, indicated by status flags. This preliminary action ensures data freshness without requiring continuous high-bandwidth updates during processing.
Solution Approach 2:
The DCR includes validity bits and status flags that provide feedback about data readiness and freshness. This feedback mechanism allows the accelerator to know when updated data is available without requiring constant polling or continuous data streaming, thereby maintaining data freshness while controlling bandwidth consumption through event-driven updates.
Data Source
AI summary
A system and method for low latency and higher bandwidth communication between a central processing unit (CPU) and an accelerator is disclosed. When the CPU updates a copy of data stored at a shared memory, the CPU also sends an “invalidate” command to a cache coherent interconnect (CCI). The CCI forwards the invalidate command to a dedicated cache register (DCR). The DCR marks its copy of the data as “out-of-date” and requests an up-to-date copy of the data from the CCI. The CCI then retrieves up-to-date data for the DCR. When the DCR receives the up-to-date data from the CCI, the DCR replaces the out-of-date data with the up-to-date data, and marks the up-to-date data with the status of “valid.” The DCR can then provide data to an accelerator with a status of “out-of-date” or “valid.”