Neural Network Processing Modules with On-Chip Interconnection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for executing multicore and multi-layer neural network operations, such as using general-purpose processors and GPUs, face performance bottlenecks due to low CPU efficiency, high power consumption, and limited on-chip buffer capacity, leading to inefficient data communication and memory access overheads.
Innovation Solution
A device with on-chip interconnection modules and neural network processing modules that enable efficient data exchange and operation result sharing among multiple processing units, utilizing dedicated instructions and high-speed storage to accelerate multicore and multi-layer neural network operations, reducing memory access and communication overheads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple general-purpose processors execute neural network operations in parallel, then operation performance is improved, but communication overhead and power consumption become bottlenecks
Solution Approach 1:
The system segments neural network operations into distinct layers, with each layer processed by dedicated processing modules. This segmentation allows parallel execution of different layers while reducing inter-processor communication overhead, as each module handles specific layer computations independently.
Solution Approach 2:
An on-chip interconnection module serves as an intermediary between processing modules, enabling efficient data exchange without requiring direct processor-to-processor communication. This mediator reduces communication overhead and power consumption by handling data routing at the interconnection level rather than requiring full processor engagement.
2Adaptability or versatility
If general-purpose processors decode neural network operations, then operation flexibility is maintained, but front-end decoding overhead increases power consumption
Solution Approach 1:
The system extracts and separates the decoding function from general-purpose processors, implementing dedicated decoding logic within the neural network processing device. This extraction eliminates the need for complex front-end decoding on general processors, significantly reducing power consumption while maintaining operation flexibility through hardware-accelerated decoding.
Solution Approach 2:
The patent replaces the mechanical/software-based decoding process of general-purpose processors with dedicated hardware decoding circuits. This substitution eliminates the sequential instruction decoding overhead, reducing power consumption while maintaining the ability to handle various neural network operations through configurable hardware logic.
3Productivity
If GPU executes neural network operations using general SIMD instructions, then graphic operation capability is utilized, but front-end decoding overhead and on-chip buffer limitations reduce efficiency
Solution Approach 1:
The system segments the neural network processing pipeline into dedicated modules for different operations (convolution, activation, pooling). Each module is optimized for its specific function, eliminating the need for complex front-end decoding of general SIMD instructions and reducing on-chip buffer requirements through specialized hardware implementations.
Solution Approach 2:
The patent changes the operational parameters from general-purpose SIMD instruction execution to dedicated neural network operation codes. This parameter change simplifies the control logic and decoding requirements, as the hardware is designed to execute specific neural network operations directly without requiring complex instruction translation and buffer management.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
The present disclosure relates to a device and a method for executing neural network operation. The device comprises an on-chip interconnection module and a plurality of neural network processing modules which are communicatively connected to the on-chip interconnection module, wherein the neural network processing module can read/write data from/to other neural network processing modules through the on-chip interconnection module. In multicore and multi-layer artificial neural network operations, the neural network operation of each layer is divided, and then is performed by a plurality of neural network processing modules to obtain their respective operation result data, and a plurality of neural network processing units perform data exchange of their respective operation result data.