Streaming Compute Unit Reducing AI Chip Power via Kernel Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence chips face high power consumption during convolution operations due to the repeated acquisition of convolution kernel data for multiple feature map data in different convolution windows, which is inefficient and energy-intensive.
Innovation Solution
A streaming-based compute unit is introduced, capable of performing multiple convolutions on N convolution windows using a single set of convolution kernel data, where each convolution involves sequential multiplication operations on feature map data from the same positions across all windows, reducing the need for repeated kernel data acquisition and minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the compute unit acquires convolution kernel data repeatedly for multiple feature map data in different convolution windows, then the convolution operation can be performed on all feature map data, but the power consumption increases significantly
Solution Approach 1:
The patent loads the convolution kernel data into a register before performing multiple convolution operations. By preparing the kernel data in advance and storing it in a register, the system avoids repeated acquisition of the same kernel data from external memory during the convolution process, thereby reducing power consumption while maintaining the ability to process multiple feature map data
Solution Approach 2:
The patent creates a copy of the convolution kernel data by loading it into a register. This register-based copy allows the kernel data to be reused multiple times without repeatedly accessing the original data source, enabling efficient convolution operations on multiple feature map data while minimizing energy consumption from repeated data acquisition
2Use of energy by moving object
If the compute unit uses a single convolution kernel data set for multiple feature map data at the same positions, then the power consumption is reduced, but the device complexity increases
Solution Approach 1:
The patent implements a universal register-based storage mechanism that can hold convolution kernel data and serve multiple convolution operations. This single register structure performs multiple functions: storing kernel data, enabling repeated access without external memory, and supporting different feature map data at the same positions, thereby reducing power consumption without proportionally increasing device complexity
Data Source
AI summary
A streaming-based compute unit and method, and an artificial intelligence chip, relating to artificial intelligence field. The compute unit includes N registers configured to perform N convolutions on N convolution windows and a convolution kernel. A jth convolution includes performing M multiplications on M data in a jth convolution window and M data in the convolution kernel, to obtain M first computation results. The N convolutions include N multiplications sequentially and consecutively performed on at least one set of feature map data and convolution kernel data. Each feature map data set includes N data from N convolution windows at the same position. A jth register is configured to store a second computation result of the jth convolution window. After an ith multiplication in the jth convolution, the second computation result is updated into a sum of i first computation results in the jth convolution.