AI Accelerator Command Dispatcher Reduces Transmission Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI accelerators face inefficiencies in data transmission and command control due to increased complexity and overhead from additional processing units and channels, with traditional RISC instruction sets exacerbating command transmission overhead.
Innovation Solution
The proposed AI accelerator incorporates an external command dispatcher, data access units, and a data/command switch to manage data and command transmission efficiently, reducing overhead by distinguishing between data and command types and optimizing their routing through a global buffer and internal command dispatcher.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of processing units and transmission channels is increased to improve computing speed and data transmission speed, then computing performance is improved, but control command complexity increases and transmission overhead increases
Solution Approach 1:
The patent divides the data access functionality into separate data access units (first data access unit, second data access unit) that can independently obtain data from storage devices. This segmentation allows parallel data access operations while keeping control command processing simplified, as the external command dispatcher only needs to send access information to appropriate data access units based on address routing.
Solution Approach 2:
The patent introduces a data/command switch as an intermediary component that receives data from data access units and routes it to either the global buffer or internal command dispatcher based on address information. This intermediary simplifies the control flow by automatically directing data packets without requiring complex control commands to manage multiple processing units.
2Adaptability or versatility
If traditional RISC instruction set is used to implement control commands, then compatibility is maintained, but command transmission overhead increases due to multiple commands needed for multiple control registers
Solution Approach 1:
The patent merges multiple control register operations into a unified control flow managed by the external command dispatcher. Instead of sending separate commands to multiple control registers in multiple sequencers, the system uses a single external command dispatcher that routes access information to appropriate data access units, effectively combining multiple control operations into one streamlined process.
Solution Approach 2:
The external command dispatcher serves multiple functions: it receives addresses and access information, determines which data access unit should handle the request, sends appropriate access information, and works in conjunction with the data/command switch to route data to the correct destination. This multi-functional component reduces the need for multiple specialized control registers and sequencers.
3Adaptability or versatility
If multiple sequencers with multiple control registers are used to control multiple processing units, then control flexibility is improved, but the overhead of sending multiple commands increases
Solution Approach 1:
The patent extracts the complex control register and sequencer architecture from the traditional design and replaces it with a simplified external command dispatcher that directly manages data access units. This extraction removes the overhead of multiple control registers while maintaining control flexibility through the dispatcher's ability to route access information to appropriate units based on address information.
Data Source
AI summary
An artificial intelligence accelerator includes an external command dispatcher, a first data access unit, a second data access unit, a global buffer, an internal command dispatcher, and a data/command switch. The external command dispatcher receives an address and access information. The external command dispatcher sends the access information to one of the first data access unit and the second data access unit, the first data access unit receives first data from a storage device according to the access information, and sends the first data to the global buffer. The second data access unit receives second data from the storage device according to the access information, and sends the second data. The data/command switch receives the address and the second data from the second data access unit, and sends the second data to one of the global buffer and the internal command dispatcher.