Vector Processing Unit Micro-Op Cache for Out-of-Order Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vector processing units (VPUs) face inefficiencies due to the need for operation caches (OCs) that track micro-ops, leading to increased power consumption, area requirements, and scaling issues as the number of micro-ops increases, and limitations in executing instructions out of order.
Innovation Solution
Introduce a micro-op cache (μOC) separate from the operation cache (OC) to track the state and control logic of micro-ops, allowing instructions to be sent whole and split into micro-ops by the OC, enabling out-of-order execution and reducing the number of micro-ops per instruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If instructions are split into multiple micro-ops before reaching the operation cache, then the VPU can execute micro-ops separately with proper tracking, but the operation cache requires more entries and storage time, increasing power consumption and area
Solution Approach 1:
The patent divides the tracking responsibility into two separate caches: the operation cache (OC) tracks only the instruction-level state, while a new micro-op cache (μOC) tracks the micro-op level state. This segmentation allows the OC to have fewer entries since it doesn't need to track every micro-op individually, thereby reducing power consumption while maintaining reliable instruction tracking.
Solution Approach 2:
The patent extracts the micro-op tracking function from the operation cache and places it in a separate micro-op cache. This extraction allows the OC to focus only on instruction-level tracking, reducing the number of entries needed in the OC and consequently reducing its power consumption and area requirements.
2Reliability
If instructions are split into multiple micro-ops before reaching the operation cache, then micro-ops can be tracked individually, but the operation cache area increases due to more entries being required
Solution Approach 1:
The patent segments the caching system into two distinct caches with different responsibilities: the OC handles instruction-level tracking with fewer entries, while the μOC handles micro-op-level tracking. This segmentation reduces the area of the OC by eliminating the need for it to store entries for every micro-op, while still maintaining reliable micro-op state tracking through the μOC.
Solution Approach 2:
The patent extracts the micro-op tracking functionality from the OC and places it in a separate μOC. This extraction reduces the OC area by removing the need for numerous micro-op entries, while the μOC is optimized specifically for micro-op tracking requirements.
3Reliability
If the operation cache tracks all micro-ops, then complete control state is maintained, but the cache size scales poorly as the number of micro-ops per instruction increases
Solution Approach 1:
The patent segments the control state tracking between two caches: the OC maintains instruction-level control state, and the μOC maintains micro-op-level control state. This segmentation allows the system to maintain complete control state information while avoiding the poor scaling behavior of a single large cache, as each cache is optimized for its specific level of abstraction.
Solution Approach 2:
The patent extracts the micro-op control state tracking from the OC and places it in a separate μOC. This extraction reduces the scaling complexity of the OC by removing the burden of tracking every micro-op, while the μOC is specifically designed to handle micro-op tracking requirements efficiently.
4Reliability
If instructions must be executed in order, then correctness is maintained, but performance is limited due to inability to execute ready micro-ops out of order
Solution Approach 1:
The patent segments the execution control between the OC, which maintains instruction-order correctness, and the μOC, which enables out-of-order micro-op execution. This segmentation allows the system to maintain correctness at the instruction level while achieving higher throughput through out-of-order execution of micro-ops, resolving the contradiction between reliability and productivity.
Data Source
AI summary
A vector processing unit contains an operation cache and a separate micro-op cache. The operation cache tracks state and logic of instructions, and is responsible for splitting instructions into micro-ops. The micro-op cache tracks state and logic of micro-ops. Having a separate micro-op cache provides power and area benefits, as well as allowing instructions to be split out of order.