Loop Support ISA Extension for DNN Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current CPUs consume significant power due to the need for all instructions in deep neural network (DNN) kernels to go through power-hungry stages of the CPU pipeline, such as fetch, decode, and dispatch, with existing optimizations only partially addressing the issue of unrolling multiple nested fixed iteration count loops.
Innovation Solution
The introduction of a Loop Support ISA Extension (LSX) that encodes metadata about loop iterations, base addresses, and register strides, allowing for the bypassing of power-hungry stages by pre-decoding and scheduling instructions closer to execution units, thereby reducing the need for repeated fetch and decode operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If all instructions go through fetch, decode, and dispatch stages of CPU pipeline, then instructions can be executed with standard CPU architecture, but power consumption increases significantly
Solution Approach 1:
The patent segments the CPU pipeline into two distinct paths: a standard front-end pipeline for fetch and decode operations, and a specialized back-end pipeline for unrolled instruction execution. This segmentation allows instructions to bypass the power-hungry front-end stages by being directly loaded into execution units after decoding, thereby reducing overall power consumption while maintaining architectural complexity through the dual-path design
Solution Approach 2:
The patent implements preliminary decoding of loop instructions before execution, where the decoder prepares multiple unrolled instruction instances in advance and stores them in a dedicated buffer. This preliminary action eliminates the need for repeated fetch and decode operations during loop execution, significantly reducing power consumption while introducing complexity in the decoding and buffer management mechanisms
2Productivity
If multiple nested fixed iteration count loops are unrolled into all stages of CPU pipeline, then loop execution can be optimized, but power consumption increases due to repeated fetch and decode operations
Solution Approach 1:
The patent performs preliminary decoding of loop instructions and stores unrolled instruction instances in a dedicated buffer before execution. This allows the CPU to bypass repeated fetch and decode operations during loop execution, significantly reducing power consumption while maintaining high loop execution efficiency through the pre-prepared instruction instances
Solution Approach 2:
The patent creates multiple copies of loop instructions by unrolling them into explicit iteration steps and storing these copies in a dedicated buffer. This copying mechanism allows the execution units to directly load and execute instruction copies without repeated fetch and decode operations, thereby reducing power consumption while maintaining productivity through efficient loop execution
3Use of energy by moving object
If instructions are unrolled and executed without going through all pipeline stages, then power consumption is reduced, but the CPU architecture becomes more complex
Solution Approach 1:
The patent divides the CPU architecture into two segmented pipelines: a standard front-end for fetch and decode operations, and a specialized back-end for unrolled instruction execution. This segmentation enables power-saving by bypassing the front-end for unrolled instructions while managing architectural complexity through clear functional separation and dedicated hardware components
Solution Approach 2:
The patent introduces an intermediary buffer that stores unrolled instruction instances between the decoder and execution units. This intermediary component simplifies the overall architecture by providing a dedicated storage layer that bridges the gap between decoding and execution, making the complex unrolling mechanism more manageable and integrable into the existing CPU design
Data Source
AI summary
Techniques for processing loops are described. An exemplary apparatus at least includes decoder circuitry to decode a single instruction, the single instruction to include a field for an opcode, the opcode to indicate execution circuitry is to perform an operation to configure execution of one or more loops, wherein the one or more loops are to include a plurality of configuration instructions and instructions that are to use metadata generated by ones of the plurality of configuration instructions; and execution circuitry to perform the operation as indicated by the opcode.