Dynamic Processor Frequency Adjustment for Neural Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural processing units (NPUs) face inefficiencies in data transfer and processing due to fixed operating frequencies and lack of dynamic adjustment of processing rates and data transfer rates, leading to suboptimal performance and energy consumption during workload processing.
Innovation Solution
An information processing apparatus and method that estimates access and processing times to dynamically adjust the processing rate and data transfer rate between a processor and memory, calculating quality of service values for memory requests to optimize data transfer and processing, allowing for efficient workload processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed operating frequencies are used for processor and memory, then system stability is maintained, but processing efficiency deteriorates due to inability to adapt to varying workload requirements
Solution Approach 1:
The patent implements dynamic frequency adjustment by allowing the processor and memory to operate at variable frequencies based on workload characteristics. The system estimates access times and processing times, then dynamically sets operating frequencies to match the slower of the two operations, ensuring optimal utilization of both components without fixed frequency constraints.
Solution Approach 2:
The system changes operational parameters (frequencies) based on workload analysis. By estimating access times and processing times beforehand, the system determines appropriate frequency settings that adapt to specific workload requirements, transforming fixed parameters into variable ones that optimize performance for different computational tasks.
2Speed
If high constant frequency is maintained for data transfer, then data transfer speed is improved, but energy consumption increases due to inability to scale down during low-demand periods
Solution Approach 1:
The data transfer rate is made dynamic by adjusting the memory operating frequency based on estimated access times relative to processing times. The system scales the transfer rate up when data access is the bottleneck and scales it down when processing is slower, eliminating the need to maintain high constant frequency and reducing energy consumption during low-demand periods.
Solution Approach 2:
The system performs preliminary estimation of access times and processing times before actual data transfer begins. This advance planning allows the system to pre-determine the optimal data transfer rate for the upcoming workload, avoiding unnecessary high-frequency operation and enabling energy-efficient transfer rate selection from the outset.
3Speed
If processor operates at high frequency, then processing speed is improved, but synchronization with memory access deteriorates when access times are longer
Solution Approach 1:
The system estimates both access times and processing times before execution, allowing it to pre-calculate the appropriate processor frequency that will complete processing刚好 when data becomes available from memory. This preliminary timing analysis prevents processing speed mismatches and eliminates synchronization delays by ensuring the processor operates at the optimal frequency for the specific workload's memory access characteristics.
Solution Approach 2:
The system uses feedback from timing estimates to adjust processor frequency selection. By continuously analyzing the relationship between access times and processing times, the system selects processor frequencies that maintain synchronization with memory operations, preventing both idle waiting and premature execution that would cause synchronization issues.
Data Source
AI summary
An information processing apparatus is described for processing a workload. The information processing apparatus comprises a processor and a memory element connected to the processor via a data link. In advance of processing a workload, the information processing apparatus estimates an access time required to transfer an amount of the workload that is to be transferred from the external memory element to the processor, and estimates a processing time for the processor to process the workload. A processing rate characteristic of the processor and/or a data transfer rate between the memory and the processor is set in dependence upon the estimated processing time and estimated access time. Methods for varying a quality of service (QoS) value of requests to the external memory element are also described.