Convolution Core Multiplexing for Higher NPU Resource Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing neural network processing unit (NPU) architecture faces a memory wall problem due to limited bandwidth in physical memory, hindering its ability to meet the increasing computing power requirements of advanced artificial intelligence technologies, particularly with algorithms like ChatGPT and transformer, and is inefficient in resource utilization and hardware costs.
Innovation Solution
A computing core with multiple convolution circuits and multiplexers, allowing flexible resource allocation for network depth, input and output channels, and feature map size, coupled with a hardware multi-thread control circuit for time-division execution, enhancing resource utilization and reducing hardware costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a centralized computing and centralized storage architecture is used, then the system structure is simple, but the memory bandwidth is limited and computing power improvement is restricted
Solution Approach 1:
The patent segments the centralized computing architecture into multiple distributed computing units (convolution circuits), each capable of independent computation. This segmentation allows parallel processing of data blocks, thereby increasing overall computing power while maintaining manageable system complexity through modular design.
Solution Approach 2:
The patent introduces a new dimension of computation by implementing in-memory computing within each convolution circuit. This allows computation to occur at the memory location rather than requiring data to be moved to a separate processing unit, effectively adding a spatial dimension to the computing architecture that bypasses the memory wall limitation.
2Productivity
If more computing resources are allocated to meet increasing AI computing requirements, then computing power increases, but hardware costs and circuit area increase
Solution Approach 1:
The patent merges the storage function and computing function within each convolution circuit by implementing in-memory computing. This combination eliminates the need for separate memory modules and data transmission pathways, significantly reducing circuit area while maintaining high computing power through efficient local processing.
Solution Approach 2:
Each convolution circuit is designed as a universal computing unit that can process different types of convolution operations and data blocks. This multi-functionality allows the same hardware structure to be reused across multiple computing tasks, reducing the total circuit area needed compared to having dedicated hardware for each function.
3Adaptability or versatility
If the network depth and feature map size are increased to improve AI model performance, then algorithm capability improves, but resource utilization becomes inefficient
Solution Approach 1:
The patent implements dynamic resource allocation where each convolution circuit can be dynamically activated or deactivated based on the current computational requirements. This dynamic control allows the system to efficiently handle varying network depths and feature map sizes by activating only the necessary computing units, thereby maintaining high resource utilization across different operating conditions.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Embodiments of this application relate to the field of chip technologies, and provide a computing core and a data processing method, to resolve problems of low computing resource utilization and high hardware costs of an existing in-memory computing apparatus. A specific solution is as follows: The computing core includes a plurality of convolution circuits (81) and a plurality of multiplexers (82). The multiplexer (82) includes an output end and at least two input ends. The output end of the multiplexer (82) is in a one-to-one correspondence with an input end of the convolution circuit (81). An output end of each of the plurality of convolution circuits (81) is coupled to input ends of L multiplexers (82), where L is an integer greater than or equal to 2. At least one of the L multiplexers (82) further includes an external data output end. At least one of L convolution circuits (81) corresponding to the L multiplexers (82) further includes an external data input end. The multiplexer (82) is configured to connect an output end of at least one convolution circuit (81) coupled to the multiplexer (82) to the output end of the multiplexer (82).