Systolic Array Throttling Arbiter for Power Stability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrated circuits, particularly those used in machine learning applications like systolic arrays, face challenges in managing power usage effectively due to rapid changes in workload and input data patterns, leading to potential power-related issues such as temporary peaking of current and high sustained activity that can exceed the capacity of thermal cooling systems.
Innovation Solution
Implementing on-circuit activity monitoring and power management techniques that utilize sensors to detect and respond to changes in power demand and utilization, allowing for preventative throttling to prevent problematic power conditions by controlling the rate of processing through external and internal power management systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the systolic array processes data at high speed to improve productivity, then processing performance increases, but power consumption and current peaking increase causing thermal overload and reliability issues
Solution Approach 1:
The patent implements on-circuit activity monitoring that detects data activity patterns before they cause problematic power conditions. The system proactively identifies high-activity scenarios and triggers throttling recommendations in advance, preventing current peaking and thermal overload before they occur, thus maintaining both high processing speed and power stability.
Solution Approach 2:
The patent establishes a feedback loop where on-circuit sensors continuously monitor data activity, utilization metrics, and power conditions. This real-time feedback is processed by an arbiter that generates throttling recommendations, creating a closed-loop control system that dynamically adjusts processing rates based on actual power conditions, enabling the system to maintain reliability while operating at high productivity levels.
2Reliability
If external power management systems are used to monitor and control power, then power management capability is provided, but response time is slow (milliseconds) compared to the nanosecond-scale operations of the systolic array
Solution Approach 1:
The patent segments the power management function into multiple components: on-circuit activity monitoring for real-time detection, an arbiter for decision-making, and external power management systems for execution. This segmentation allows the sensitive timing-critical detection and decision functions to be implemented on-circuit at nanosecond scales, while the external systems provide the necessary power control capability without being constrained by their slower response times.
Solution Approach 2:
The patent introduces an intermediary arbiter component that sits between the on-circuit monitoring system and external power management. This arbiter receives real-time activity data from on-circuit sensors and generates throttling recommendations that can be executed by external power management systems, serving as a bridge that translates nanosecond-scale detection into actionable power control commands.
3Loss of time
If on-circuit activity monitoring is implemented to enable rapid detection, then response time improves to nanoseconds, but device complexity increases due to additional sensors and monitoring circuitry
Solution Approach 1:
The patent designs the on-circuit monitoring system to serve multiple functions: detecting data activity patterns, measuring processing utilization, identifying thermal conditions, and triggering throttling recommendations. By making the monitoring infrastructure multi-functional, the patent reduces the need for separate dedicated circuits for each function, thereby limiting the increase in device complexity while maintaining nanosecond-scale detection capability.
Data Source
AI summary
Throttling recommendations for a systolic array may be arbitrated. Throttling recommendations may be received at an arbiter for a systolic array from different sources, such as one or more monitors implemented in an integrated circuit along with the systolic array or sources external to the integrated circuit with the systolic array. A strongest throttling recommendation may be selected. The rate at which data enters the systolic array may be modified according to the strongest throttling recommendation.