Coarse Grained Reconfigurable Architecture With Iteration Level Commits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current coarse-grained reconfigurable architectures (CGRAs) face limitations such as lack of realistic load-store units, memory latency hiding mechanisms, and support for loop-related hardware features like data-dependent exits, which hinder performance and efficiency in accelerating loops with varying conditions.
Innovation Solution
A novel CGRA architecture employing Simultaneous Multiple Iterations (SMI) and Iteration Level Commits (ILC) paradigms, featuring a runtime mechanism for executing multiple concurrent iterations of a loop across processing elements, with a synchronized execution control unit and advanced load/store unit capabilities for handling concurrent operations and dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional modulo scheduling is used to achieve instruction-level parallelism in CGRAs, then PEs can be kept occupied, but runtime performance becomes suboptimal due to mismatch between compiler-assumed latencies and actual runtime latencies
Solution Approach 1:
The patent implements dynamic scheduling that adapts to actual runtime latencies of load and store instructions. The system uses a scheduler that can dynamically adjust the placement and execution timing of instructions based on observed latency variations, transitioning from static compile-time scheduling to dynamic runtime scheduling. This allows the system to maintain high PE utilization while adapting to actual memory access patterns and latency characteristics.
2Productivity
If multiple loop iterations are executed concurrently in CGRAs, then productivity increases, but complexity of managing dependencies and synchronization increases
Solution Approach 1:
The patent segments the loop execution into multiple independent iteration instances that can be concurrently executed on different PEs. Each iteration is treated as an independent executable unit with its own state, allowing parallel execution while reducing inter-iteration dependencies. The system uses iteration-level commit mechanisms that allow individual iterations to be committed or rolled back independently, simplifying the management of concurrent executions.
Solution Approach 2:
The patent introduces an execution control unit as an intermediary that manages synchronization between concurrently executing iterations. This control unit coordinates data flow between PEs, manages load-store operations, and handles dependency resolution without requiring complex direct communication between all PE pairs. The intermediary abstracts the synchronization complexity from the individual PE execution logic.
3Productivity
If CGRA architectures lack memory latency hiding mechanisms, then device complexity remains low, but productivity suffers due to severe performance penalty from data cache misses
Solution Approach 1:
The patent implements memory latency hiding through concurrent execution of multiple loop iterations while memory operations are in progress. The system maintains continuous useful computation by switching between different iteration instances when memory latency occurs, ensuring that PEs remain productive during memory wait states. This is achieved through the SMI mechanism that allows seamless transition between iterations.
Solution Approach 2:
The patent uses load-store buffering and prediction mechanisms that prepare memory operations in advance. The load-store unit predicts future memory access patterns and pre-fetches data, or buffers store operations to be executed when memory is available. This preliminary action reduces the impact of memory latency by ensuring data is ready before needed or delaying writes until memory bandwidth is available.
4Ease of operation
If compiler-assisted static placement is used for loop instructions in CGRAs, then ease of operation is improved, but adaptability to runtime conditions deteriorates
Solution Approach 1:
The patent combines static compiler-assisted placement with dynamic runtime adaptation. The compiler provides initial placement guidance and high-level optimization, while the runtime system observes actual execution characteristics and dynamically adjusts scheduling decisions, iteration concurrency levels, and resource allocation. This hybrid approach maintains programming simplicity while adapting to runtime conditions.
Data Source
AI summary
An apparatus and method for supporting simultaneous multiple iterations (SMI) in a course grained reconfigurable architecture (CGRA). In support of SMI, the apparatus includes: Hardware structures that connect all of multiple processing engines (PEs) to a load-store unit (LSU) configured to keep track of which compiled program code iterations have completed, which ones are in flight and which are yet to begin, and a control unit including hardware structures that are used to maintain synchronization and initiate and terminate loops within the PEs. SMI permits execution of the next instruction within any iteration (in flight). If instructions from multiple iterations are ready for execution (and are pre-decoded), then the hardware selects the lowest iteration number ready for execution. If in a particular clock cycle, a loop iteration with a lower iteration number is stalled (i.e., is waiting for data), the instruction from the next highest iteration number that is ready thereby will be automatically executed automatically allowing the CGRA to have high ILP by overlapping concurrent loop iterations.