Neural Network Processor On-Chip Parameter Memory Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network processors face inefficiencies due to the need to repeatedly read large weight parameters from external memory during computations, leading to wasted time and reduced speed.
Innovation Solution
A computing device with distributed arithmetic parameter storage, where each computing unit has an arithmetic parameter memory and an address controller to manage the storage and output of parameters, allowing for reduced memory access and improved computation speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If weight parameters are stored in external memory, then device complexity is reduced, but computation speed deteriorates due to repeated reading
Solution Approach 1:
The patent divides the weight parameter storage into two segments: external memory for bulk storage and on-chip arithmetic parameter memories in each computing unit for frequently accessed parameters. This segmentation allows the system to maintain low overall complexity while speeding up computation by keeping critical parameters locally available.
Solution Approach 2:
The patent implements local quality by placing arithmetic parameter memories specifically within computing units that require frequent access to weight parameters. This localized storage approach ensures that computation-critical parameters are readily available locally, improving computation speed without requiring all parameters to be stored locally.
2Speed
If all weight parameters are loaded into the processor, then computation speed improves, but device complexity increases
Solution Approach 1:
The patent segments the parameter storage hierarchy into external memory for complete weight parameter storage and distributed on-chip arithmetic parameter memories for active computation parameters. This segmentation enables fast computation with a subset of parameters while maintaining manageable device complexity through the external memory backup.
Solution Approach 2:
The patent applies partial action by loading only the necessary arithmetic parameters into on-chip memories rather than all weight parameters. This partial loading approach achieves sufficient computation speed improvement while avoiding the complexity overhead of loading and managing the entire parameter set locally.
3Device complexity
If weight parameters are repeatedly read from external memory, then device complexity remains low, but time consumption increases
Solution Approach 1:
The patent implements preliminary action by pre-loading arithmetic parameters into on-chip memories before computation begins. This preliminary loading eliminates the need for repeated reading during computation, significantly reducing time consumption while maintaining simple external memory structure for overall storage.
Data Source
AI summary
The present application discloses a computing device and a neural network processor including the computing device. The computing device includes one or more columns of computing units arranged in an array, wherein at least one computing unit in each column comprises: an arithmetic parameter memory for storing one or more arithmetic parameters; an arithmetic logical unit (ALU) for receiving input data and performing computation on the input data using the one or more arithmetic parameters stored in the arithmetic parameter memory; and an address controller for providing an address control signal to the arithmetic parameter memory to control the storage and output of the one or more arithmetic parameters.