Hierarchical Hybrid NoC for Compute-in-Memory ML Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory accesses in neural networks are a major contributor to energy consumption, and as neural networks grow larger, the energy required to move data during processing increases.
Innovation Solution
A hierarchical hybrid network-on-chip architecture that combines analog and digital components, featuring an analog router with a Gaussian distribution circuit and stochastic rounding, which reduces data conversions and enhances accuracy and training speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is moved during neural network processing, then computation can be performed, but energy consumption increases
Solution Approach 1:
The patent combines analog and digital computing components into a hybrid architecture where analog circuits perform multiply-accumulate operations directly in memory, eliminating the need to move data between separate storage and processing units. This merging of computation and memory operations reduces energy consumption while maintaining computational capability.
Solution Approach 2:
The patent replaces traditional digital data movement and processing mechanisms with analog computational mechanisms. Instead of moving digital data through multiple stages of processing, analog voltages and currents directly represent and compute neural network parameters, significantly reducing the energy required for data movement and processing.
2Use of energy by moving object
If analog components are used to reduce data conversions, then power consumption decreases, but system complexity increases
Solution Approach 1:
The patent segments the neural network processing system into distinct analog and digital domains, with analog circuits handling compute-in-memory operations and digital circuits handling control and data movement. This segmentation allows each domain to be optimized independently, reducing overall system complexity while maintaining power efficiency benefits of analog processing.
Solution Approach 2:
The patent designs the hybrid architecture to provide universal functionality by enabling the same system to handle both analog compute-in-memory operations and digital data movement and control. This multi-functionality allows the system to leverage the power efficiency of analog processing while maintaining the flexibility and precision of digital processing, without requiring separate dedicated systems.
Data Source
AI summary
Systems, methods, apparatuses, and computer-readable media. An analog router of a first supertile of a plurality of supertiles of a network on a chip (NoC) may receive a first analog output from a first compute-in-memory tile of a plurality of compute-in-memory tiles of the first supertile. The analog router may determine, based on a configuration of a neural network executing on the NoC, that a destination of the first analog output includes a second supertile of the plurality of supertiles. An analog-to-digital converter (ADC) of the analog router may convert the first analog output to a first digital output and transmit the first digital output to the second supertile via a communications bus of the NoC.