Simultaneous Multithreading with Loop Support for Bulk Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CPU architectures consume significant power due to unrolling every instruction through all stages of the pipeline, particularly in deep neural network (DNN) inference tasks, despite the structured and repetitive nature of these operations.
Innovation Solution
Implement a Loop Support ISA Extension (LSX) that encodes metadata for loop iterations, allowing multiple instructions to be fetched, decoded, and allocated only once, thereby bypassing power-hungry stages of the CPU pipeline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If every instruction is unrolled through all stages of the CPU pipeline, then complete instruction execution is achieved, but power consumption increases significantly
Solution Approach 1:
The patent segments the CPU pipeline into distinct stages (fetch, decode, allocate, dispatch, execute) and selectively applies full unrolling only to critical stages while bypassing others. The loop support extension allows instructions to be marked for selective unrolling, dividing the pipeline processing into mandatory and optional segments based on power consumption characteristics of each stage.
Solution Approach 2:
The patent applies different processing qualities to different pipeline stages. Power-hungry stages like fetch and decode are selectively bypassed for loop instructions using the loop support extension, while essential execution stages maintain full processing. This creates local quality variations in instruction handling, applying full unrolling only where necessary for correctness while reducing it elsewhere to save power.
2Productivity
If loop instructions are fully unrolled through all pipeline stages, then correct execution is ensured, but efficiency is reduced due to repetitive processing
Solution Approach 1:
The patent performs preliminary actions by setting up loop metadata and control structures before entering the loop body. The loop support extension allows the compiler to pre-calculate loop bounds, strides, and iteration counts, storing this information in special register fields. This preliminary setup enables the loop instructions to be executed efficiently with reduced pipeline processing while maintaining correctness through the pre-established control parameters.
Solution Approach 2:
The patent uses copying by duplicating loop control information into specialized fields rather than repeatedly processing full instructions. The loop support extension creates compact representations of loop parameters that are copied and reused across iterations, avoiding the need to fetch and decode identical loop control instructions multiple times while ensuring correct execution through the copied control data.
3Loss of energy
If all instructions go through fetch, decode, allocation and dispatch stages, then thorough processing is achieved, but power-hungry operations are repeated unnecessarily
Solution Approach 1:
The patent introduces an intermediary mechanism - the loop support extension with its special register fields and metadata structures - that sits between the standard pipeline stages. This intermediary captures loop control information once and makes it available to execution stages without requiring repeated passage through fetch, decode, and dispatch. The intermediary structure mediates between full processing requirements and power savings by providing efficient access to loop parameters.
Data Source
AI summary
Apparatus and method for leveraging simultaneous multithreading for bulk compute operations. For example, one embodiment of a processor comprises: a plurality of cores including a first core to simultaneously process instructions of a plurality of threads; a cache hierarchy coupled to the first core and the memory, the cache hierarchy comprising a Level 1 (L1) cache, a Level 2 (L2) cache, and a Level 3 (L3) cache; and a plurality of compute units coupled to the first core including a first compute unit associated with the L1 cache, a second compute unit associated with the L2 cache, and a third compute unit associated with the L3 cache, wherein the first core is to offload instructions for execution by the compute units, the first core to offload instructions from a first thread to the first compute unit, instructions from a second thread to the second compute unit, and instructions from a third thread to the third compute unit.