On-Die Telemetry Power Balancing Between GPU and Switch

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Data centers waste power due to overprovisioning at peak bandwidth, leading to inefficiencies as devices like GPUs and switches enter idle modes, with power management latencies making real-time adjustments infeasible.

Innovation Solution

Implementing on-die telemetry to estimate switch power usage, allowing a power management unit (PMU) to adjust GPU power accordingly, balancing power between GPUs and switches to optimize total system power usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If power is provisioned for peak bandwidth, then system reliability is improved, but energy efficiency deteriorates due to overprovisioning when devices are idle

Engineering Contradiction:

Improvesystem reliabilityVSAvoidenergy efficiency

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent implements dynamic power provisioning that adjusts power allocation between GPUs and switches based on real-time operational states. The system monitors switch idle modes and dynamically reallocates power from underutilized switches to active GPUs, transforming the static peak-power-provisioning model into a dynamic adaptation model that maintains reliability while reducing energy waste during partial utilization scenarios

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system employs feedback mechanisms through on-die telemetry data that continuously monitors switch power consumption and operational states. This feedback loop enables the power management unit to detect when switches enter idle modes and automatically adjust power allocation accordingly, creating a closed-loop control system that prevents energy overprovisioning while maintaining system reliability

Inventive Principle:

Principle #23Feedback

2Loss of energy

If real-time power adjustments are implemented, then energy efficiency is improved, but system complexity increases due to management latencies

Engineering Contradiction:

Improveenergy efficiencyVSAvoidsystem complexity

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The patent enables self-service power management where the system autonomously monitors its own power consumption through on-die telemetry and automatically adjusts power allocation without external intervention. The power management unit uses built-in monitoring capabilities and predefined adjustment rules to self-regulate power distribution, reducing the need for complex external power management infrastructure while improving energy efficiency

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system replaces complex mechanical power management infrastructure with software-based power adjustment mechanisms. By using on-die telemetry data and software-controlled power allocation, the system eliminates the need for complex hardware power management layers, reducing overall system complexity while enabling real-time power adjustments through firmware or software logic

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Reliability

If power is allocated to idle switches, then system performance is maintained, but productivity decreases due to wasted power resources

Engineering Contradiction:

Improvesystem performanceVSAvoidsystem efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent dynamically changes power allocation parameters based on switch utilization states. When switches enter idle modes, the system automatically reduces power allocation to those switches and redirects the freed power resources to active GPUs, thereby maintaining system performance where needed while eliminating power waste in idle components, directly improving overall system productivity

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12554305B2System power balancing via on-die telemetry data

Publication Date: 2026.02.17 NVIDIA CORP

AI summary

A system includes a processing unit coupled with one or more switches via one or more links. The processing unit is to determine a total threshold power value associated with the processing unit and the one or more links and estimate a power consumption value associated with a switch of the one or more switches. The processing unit can also determine the power consumption value of the switch and a second power consumption value of the processing unit fail to satisfy the total power threshold value and responsive to determining the power consumption value and the second power consumption value fail to satisfy the total power threshold value, increase an amount of power supplied to the processing unit to satisfy the total power threshold value.