IR Controller Circuit Selecting Hardware Compute Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hardware designs optimized for machine learning tasks, such as neural networks, become outdated quickly due to constant changes in software primitives, leading to inefficient processing and power consumption.
Innovation Solution
An intermediate representation (IR) controller is implemented to select the most suitable hardware compute units based on collected data and profiles, adapting in real-time to changes in both hardware and software primitives, by leveraging a writeable control store for microcode updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware is optimized for specific machine learning functions, then processing performance is improved, but adaptability to new software primitives deteriorates
Solution Approach 1:
The patent implements a dynamic controller that selects from multiple microcode implementations based on runtime conditions. The system maintains multiple versions of microcode for the same IR primitive, each optimized for different hardware configurations or software requirements. The controller dynamically chooses which microcode implementation to execute based on current operational context, enabling the hardware to adapt to new software primitives without physical reconfiguration.
Solution Approach 2:
The system changes the parameter of microcode version selection based on identified IR primitives. By maintaining multiple microcode implementations with different optimization parameters and selecting the appropriate one at runtime, the hardware can effectively change its behavior to match new software requirements while maintaining optimized performance for known operations.
2Use of energy by moving object
If hardware compute units are specialized for specific tasks, then energy efficiency is improved, but flexibility to handle varying workloads deteriorates
Solution Approach 1:
The patent creates a universal control mechanism that manages multiple specialized microcode implementations. The controller itself is designed to be universal, capable of selecting and executing different microcode versions for various IR primitives. This allows the system to maintain specialized optimized code paths for different task types while using a single unified controller to manage them all, achieving both energy efficiency through specialization and flexibility through universality.
Solution Approach 2:
The controller acts as an intermediary between the hardware compute units and the diverse software primitives. It translates high-level IR primitives into appropriate microcode implementations, mediating between the need for specialized hardware optimization and the requirement to handle varying workloads. This intermediary layer enables specialized hardware to efficiently execute diverse workloads by selecting the most appropriate optimized microcode path for each task.
Data Source
AI summary
An intermediate representation (IR) controller is described that, for a given intermediate representation (IR) primitive, selects a hardware compute unit of a plurality of hardware compute units. In a non-limiting example, the IR controller receives an input that specifies an IR primitive, a device mask indicating a type of hardware circuitry to be used to process the primitive, and a goal vector specifying a goal in the processing of the primitive. The IR controller also collects data describing power consumption by respective hardware compute units and completion times for processing respective IR primitives. This data is maintained as implementation profiles that describe operation of respective hardware compute units in processing respective IR primitives, e.g., as histograms. The implementation profiles are then leveraged by the IR controller to select hardware compute units for execution of subsequent IR primitives.