CGRA Compute Fabric Token Buffers for Inter-Thread Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CGRA processors face inefficiencies in transferring data between threads, leading to increased memory access and latency, particularly when data needs to be used multiple times across different threads.
Innovation Solution
Implementing a compute fabric with token buffers and interconnects that allow direct data transfer between threads within the fabric, bypassing system memory, using token buffers to store and route data based on thread identifiers, and cascading buffers for larger data transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transferred between threads through system memory, then data can be shared across threads, but memory access time and latency increase
Solution Approach 1:
The patent introduces on-chip buffers as intermediary storage structures between compute nodes and system memory. These buffers cache data locally within the compute fabric, allowing threads to access shared data without repeatedly accessing system memory. The buffer acts as a mediator that holds data temporarily, enabling fast local access while maintaining data sharing capability across multiple threads.
Solution Approach 2:
The system pre-loads data into on-chip buffers before threads need to access it. By anticipating data access patterns and loading data into buffers in advance, the system prepares data in a readily accessible state, eliminating the need for threads to wait for slow system memory access when they need to share or access the same data.
2Reliability
If data is fetched from system memory for each thread access, then data availability is ensured, but computational throughput decreases due to repeated memory access
Solution Approach 1:
On-chip buffers serve as intermediary storage that maintains data availability for multiple threads simultaneously. Instead of threads directly accessing system memory, they access the buffer which holds copies of the data. This maintains data availability while dramatically improving throughput by eliminating repeated system memory access cycles.
Solution Approach 2:
Data is pre-fetched from system memory into on-chip buffers before threads require it for computation. This preliminary loading action ensures data is immediately available when threads need it, maintaining data availability while avoiding the throughput penalty of on-demand memory access during computation phases.
3Productivity
If on-chip buffers are used for inter-thread data transfer, then memory access is reduced, but buffer management complexity increases
Solution Approach 1:
The buffer management system operates autonomously using hardware-controlled allocation and deallocation mechanisms. Buffer memory is automatically assigned to threads based on their access patterns and requirements, and automatically freed when no longer needed. This self-managing approach reduces the burden on software and simplifies the overall system while maintaining high memory access efficiency.
Solution Approach 2:
The on-chip buffer structure is designed as a universal resource that can serve multiple threads and multiple data types simultaneously. The same buffer infrastructure handles inter-thread data transfer, temporary storage, and caching operations for various compute nodes. This multi-functional design consolidates what could be multiple separate management systems into a single unified buffer management mechanism.
Data Source
AI summary
A processor includes a compute fabric and a controller. The compute fabric includes an array of compute nodes and interconnects that configurably connect the compute nodes. The controller is configured to configure at least some of the compute nodes and interconnects in the compute fabric to execute specified code instructions, and to send to the compute fabric multiple threads that each executes the specified code instructions. A compute node among the compute nodes is configured to execute a code instruction for a first thread, and to transfer a result of the code instruction within the fabric, for use as an operand by a second thread, different from the first thread.