Distributed Neural Network Storage via Model Network-on-Chip
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural inference processors face performance limitations due to the challenge of delivering neural network weights and parameters to computation cores in a timely manner, leading to congestion and stalls in the weight delivery network-on-chip, which affects the efficiency of neural network computations.
Innovation Solution
A modular neural inference processor architecture with a Model Network-on-Chip (MNoC) that distributes neural network models, including weights and instructions, across multiple neural inference cores, eliminating the need for separate memory units and allowing for efficient broadcasting of weights and instructions to all cores, thereby reducing congestion and improving computation efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network weights and parameters are delivered to computation cores through a centralized weight delivery network-on-chip, then the neural network computations can be performed, but congestion and stalls occur in the weight delivery network leading to performance limitations
Solution Approach 1:
The patent segments the centralized weight delivery function into distributed storage across multiple neural inference cores. Each core stores portions of the neural network model locally, eliminating the bottleneck of centralized weight delivery and enabling parallel access without congestion or stalls.
2Quantity of substance
If separate memory units are provided for storing neural network models, then the models can be stored, but the device complexity increases and bandwidth efficiency decreases
Solution Approach 1:
The patent merges the functions of computation cores and memory units by enabling each neural inference core to store neural network models locally. This eliminates separate memory units, reduces device complexity, and improves bandwidth efficiency by allowing cores to access their local model data without external memory access delays.
3Use of energy by moving object
If neural network models are distributed across multiple cores, then bandwidth and energy efficiency improve, but the complexity of weight distribution management increases
Solution Approach 1:
The patent implements self-service through automatic weight distribution mechanisms where each neural inference core autonomously manages its local model storage and retrieval. The system automatically partitions and distributes weight data across cores based on computational requirements, eliminating the need for complex external management while improving energy efficiency through localized access.
Data Source
AI summary
Modular neural network computing apparatus are provided with distributed neural network storage. In various embodiments, a neural inference processor comprises a plurality of neural inference cores, at least one model network interconnecting the plurality of neural inference cores, and at least one activation network interconnecting the plurality of neural inference cores. Each of the plurality of neural inference cores comprises memory adapted to store input activations, output activations, and a neural network model. The neural network model comprises synaptic weights, neuron parameters, and neural network instructions. The at least one model network is configured to distribute the neural network model among the plurality of neural inference cores. Each of the plurality of neural inference cores is configured to apply the synaptic weights to input activations from its memory to produce a plurality of output activations to its memory. The at least one activation network is configured to provide input activations to each of the plurality of neural inference cores and to obtain output.