AI Chip Storage Control System for Efficient Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI chips face inefficiencies in performing general matrix multiplication due to the need for external module data rearrangement of weight matrices, especially when processing large matrices, which results in prolonged time consumption and low processing efficiency.
Innovation Solution
The introduction of a storage control system in the AI chip that splits weight data into multiple pieces and distributes them across multiple convolution cores, eliminating the need for external data rearrangement by reading and processing weight data in the form of 3DTiles, thereby reducing the number of GEMM operations and enhancing convolution operation efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If weight matrix data is transmitted to an external module for matrix data rearrangement, then the convolution operation can be performed with correct GEMM multiplication-addition order, but the time consumption is long and the efficiency is very low
Solution Approach 1:
The patent divides the weight matrix into multiple weight blocks, where each weight block contains contiguous elements in memory. This segmentation allows the storage control system to read and distribute weight data in parallel to multiple convolution cores without requiring external module intervention for rearrangement, thus maintaining data correctness while reducing time consumption
Solution Approach 2:
The storage control system within the AI chip performs the data distribution and arrangement functions that previously required external modules. By integrating this functionality into the chip's own storage control system, the patent eliminates the need for time-consuming external data rearrangement while ensuring correct GEMM operation order
2Productivity
If weight data is read and processed in the form of 3DTiles, then the number of GEMM operations is reduced and convolution operation efficiency is improved, but the device complexity increases due to the need for storage control system
Solution Approach 1:
The storage control system is designed to perform multiple functions: reading weight blocks from memory, splitting them into N pieces of weight data, distributing data to N convolution cores, and managing the 3DTile format. This multi-functional design consolidates what would otherwise require separate external modules, improving efficiency while controlling complexity through integration
Data Source
Figure 1~2
Figure 3~4
Figure 5~6b
AI summary
The present disclosure relates to an AI chip, an electronic device and a convolution operation method, and belongs to the field of artificial intelligence. The AI chip includes: N convolution cores and a storage control system, where the storage control system is electrically connected to the N convolution cores. The storage control system is configured to read input image data from a memory, distribute the input image data to each convolution core, read each weight block from the memory, split each weight block into N pieces of weight data, and distribute the N pieces of weight data to the N convolution cores, each convolution core corresponds to a piece of weight data, and each weight block is a part of complete weight; each convolution core is configured to perform convolution operation on the received weight data and the input image data, where the convolution operation results of each convolution core for the same weight block are added to obtain a convolution operation result of each weight block, and the convolution operation results of each weight block are added to obtain a final convolution operation result.