Repeat Cache for Multi-Thread Code Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-threaded processors, especially those designed for machine intelligence applications, there is a need to efficiently manage code reuse among threads performing similar computations, as fetching the same code from memory multiple times leads to increased power consumption and inefficiency.
Innovation Solution
A processor with a barrel-threaded execution unit and a repeat cache mechanism, where the repeat instruction allows caching of code and automatic state management to enable multiple threads to reuse cached code without re-fetching from memory, optimizing power consumption and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If multiple threads fetch the same code from memory independently, then each thread can execute its instructions, but power consumption increases due to repeated memory access
Solution Approach 1:
The patent merges code fetching operations across multiple threads by implementing a shared repeat cache that stores code portions commonly executed by multiple threads. When the first thread fetches code and detects a repeat instruction, it caches the code portion in the repeat cache. Subsequent threads that need to execute the same code portion retrieve it from the cache instead of fetching from memory again, thereby combining multiple fetch operations into one and reducing power consumption.
Solution Approach 2:
The patent creates a copy of the code portion in the repeat cache when the first thread encounters a repeat instruction. This copied version of the code is then reused by subsequent threads that need to execute the same instructions, eliminating the need for repeated memory fetches and reducing energy consumption while maintaining execution efficiency.
2Loss of energy
If a shared cache is implemented for code reuse, then power consumption is reduced, but cache management complexity increases
Solution Approach 1:
The repeat cache implementation uses self-service mechanisms where the first thread to encounter a repeat instruction automatically caches the code portion without requiring external intervention. The cache state (claimed or unclaimed) is automatically managed through hardware flags that are set and cleared based on thread execution, eliminating the need for complex software-based cache management protocols.
Solution Approach 2:
The patent uses parameter changes in the form of state flags (claimed/unclaimed) to simplify cache management. Instead of implementing complex arbitration and allocation algorithms, the system changes the state parameter of the cache to indicate availability, allowing threads to autonomously determine whether to use the cached code or fetch new code, thereby reducing management complexity.
3Productivity
If threads share the same execution pipeline through time-slot interleaving, then resource utilization improves, but threads cannot simultaneously execute the same code portion
Solution Approach 1:
The patent adds another dimension to code execution by introducing a shared cache layer between memory and the execution pipeline. This allows code to exist in multiple states: in memory for all threads, in the repeat cache for shared access, and being executed in the time-slot interleaved pipeline. This dimensional addition enables threads to simultaneously access the same code portion through the cache while maintaining sequential execution in the pipeline.
Solution Approach 2:
The repeat cache acts as an intermediary between memory and the execution pipeline. It mediates code access by storing portions of code that are commonly executed by multiple threads, allowing threads to retrieve code from this intermediate layer rather than directly from memory or requiring simultaneous pipeline access. This intermediary structure enables both time-slot interleaving and code reuse.
Data Source
AI summary
A processor is disclosed including: a barrel-threaded execution unit for executing concurrent threads, and a repeat cache shared between the concurrent threads. The processor's instruction set includes a repeat instruction which takes a repeat count operand. When the repeat cache is not claimed and the repeat instruction is executed in a first thread, a portion of code is cached from the first thread into the repeat cache, the state of the repeat cache is changed to record it as claimed, and the cached code is executed a number of times. When the repeat instruction is then executed in a further thread, then the already-cached portion of code is again executed a respective number of times, each time from the repeat cache. For each of the first and further instructions, the repeat count operand in the respective instruction specifies the number of times to execute the cached code.