Hijack Control Circuit for Idle AI MAC Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In AI processors, inactive multiply and accumulate (MAC) units lead to reduced throughput and increased power consumption due to idle computational units during neural network processing.
Innovation Solution
A hijack control circuit is introduced to selectively toggle the inputs of idle MAC units, re-utilizing them by switching to alternative input vectors managed by a processing element control circuit, which retrieves these vectors from a cloud platform.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If MAC units are left idle during neural network processing, then processing accuracy is maintained, but throughput is reduced and power consumption increases
Solution Approach 1:
The patent applies multi-functionality by enabling idle MAC units to perform alternative computations (such as processing different input vectors or performing preprocessing operations) while maintaining their primary neural network processing function. This allows the same hardware resources to serve multiple purposes, increasing overall system utilization without compromising the primary computational task.
Solution Approach 2:
The patent implements continuity of useful action by keeping MAC units continuously active through the hijack control circuit. When a MAC unit would normally be idle during neural network processing, the circuit redirects alternative input vectors to it, ensuring continuous computational work. This eliminates idle cycles and maintains productive operation of all processing elements throughout the computation pipeline.
2Productivity
If MAC units are re-utilized with alternative input vectors, then throughput is improved, but processing element control complexity increases
Solution Approach 1:
The patent introduces a hijack control circuit as an intermediary component that manages the redirection of input vectors to MAC units. This intermediary layer handles the complexity of determining when and how to re-utilize idle MAC units, isolating the control logic from the main neural network processing flow. The circuit acts as a mediator between the neural network processor and alternative computational tasks, simplifying the overall system architecture while enabling enhanced utilization.
Data Source
AI summary
The disclosed embodiments are directed toward improved control circuitry for artificial intelligence processors. In one embodiment, a device is disclosed comprising a processing element, the processing element including a processing device configured to receive a first set of vectors; a hijack control circuit, the hijack control circuit configured to replace the first set of vectors with a second set of vectors in response to detecting that the processing element is idle; and a processing element control circuit (PECC), the PECC storing a set of values representing the second set of vectors, the set of values retrieved from a remote data source.