Cooperative Instruction Prefetch Multicore System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity and cost-effectiveness of full cache coherent solutions in central processing units (CPUs) make it impractical for cross-lane processing units (XPUs) to provide each core with a private instruction memory, especially in single program multiple data (SPMD) scenarios, where a common instruction memory shared across all tiles is a more feasible but less efficient alternative.
Innovation Solution
An instruction prefetch pipeline architecture is implemented, comprising a hardware circuit with multiple tiles, each containing a processing core, prefetch unit, and instruction buffer, along with a shared task instruction memory and instruction router, which optimizes instruction prefetching and reduces redundant requests through filtering and arbitration, allowing for efficient instruction delivery without the complexity of a full cache coherent solution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If each core is provided with a private instruction memory, then instruction access efficiency is improved, but device complexity and cost increase significantly
Solution Approach 1:
Multiple cores share a single common instruction memory (TiMem) instead of each core having its own private instruction memory. This merging approach reduces device complexity and cost while maintaining instruction access efficiency through the prefetch pipeline architecture that proactively loads instructions into individual core buffers before they are needed.
2Device complexity
If a common instruction memory is shared across all tiles, then device complexity is reduced, but instruction access efficiency deteriorates due to concurrent access conflicts
Solution Approach 1:
The prefetch unit proactively fetches instructions from the shared common instruction memory into the core's private instruction buffer before the instructions are actually needed by the core. This preliminary action eliminates concurrent access conflicts by having each core work from its own pre-loaded buffer, thereby maintaining high instruction access efficiency while allowing multiple cores to share the same physical memory resource.
Data Source
AI summary
Aspects of the disclosure are directed to methods, systems, and apparatuses using an instruction prefetch pipeline architecture that provides good performance without the complexity of a full cache coherent solution deployed in conventional CPUs. The architecture can include components which can be used to construct an instruction prefetch pipeline, including instruction memory (TiMem), instruction buffer (iBuf), a prefetch unit, and an instruction router.