Calculating device and method for a sparsely connected artificial neural network
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional general-purpose processors and GPUs face performance bottlenecks and high power consumption when calculating sparse artificial neural networks due to inefficient memory access and decoding overhead, especially in multi-layer networks.
Innovation Solution
An MNN acceleration processor with a data modifier that receives input data, weight values, and connection data to modify and store them efficiently, reducing the need for index array retrieval by implementing pruning or compensation modes to eliminate or add zero values based on connection values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose processors are used to calculate sparse artificial neural networks, then the system can handle various computational tasks, but the calculation time and power consumption increase significantly
Solution Approach 1:
The processor is divided into multiple computing units, each capable of performing neural network calculations independently. This segmentation allows parallel processing of different portions of the neural network computation, significantly reducing overall calculation time while maintaining the versatility of handling various neural network architectures.
Solution Approach 2:
An index array is introduced as an intermediary data structure to store the mapping relationships between input neurons and output neurons in sparse neural networks. This index array enables the computing units to quickly locate and access only the relevant weight values and connections, avoiding unnecessary calculations and reducing calculation time.
2Adaptability or versatility
If general-purpose processors are used to calculate sparse artificial neural networks, then the system can handle various computational tasks, but power consumption increases significantly
Solution Approach 1:
The design extracts and optimizes specifically for neural network workloads by implementing dedicated computing units with specialized instruction sets and data formats. This extraction of neural network processing from general-purpose computation reduces power consumption by eliminating unnecessary general-purpose processing overhead while maintaining the ability to handle various neural network configurations.
Solution Approach 2:
Connection data and index arrays are pre-computed and stored before the actual neural network inference or training. This preliminary action organizes the sparse connection patterns in advance, allowing the computing units to directly access pre-organized data during execution without performing additional searches or computations, thereby reducing real-time power consumption.
3Quantity of substance
If index arrays are used to store connection relationships in sparse neural networks, then the system can represent sparse connections efficiently, but additional calculation overhead is required to search for weight values
Solution Approach 1:
The index array is designed to be self-service by directly encoding the positions of non-zero weight values and their corresponding input-output neuron mappings. This design allows the computing units to automatically retrieve the necessary weight values without requiring additional search algorithms or complex lookup procedures, thereby eliminating calculation overhead while maintaining efficient memory storage.
Solution Approach 2:
The index array stores transformed parameters that directly map input neuron indices to output neuron indices and weight value positions. By changing the parameter representation from a general graph structure to a specialized index format optimized for neural network operations, the system achieves both efficient storage and direct access without additional computational overhead.
4Productivity
If multiple general-purpose processors work concurrently to calculate neural networks, then the calculation speed increases, but inter-processor communication becomes a performance bottleneck
Solution Approach 1:
Multiple computing units are merged into a unified processor architecture with shared memory and coordinated control. This merging allows the computing units to work concurrently on different portions of the neural network while sharing the index array and connection data, eliminating the need for complex inter-processor communication protocols and reducing communication overhead.
Solution Approach 2:
Each computing unit has access to copies or views of the shared index array and connection data structures. This copying mechanism allows parallel computing units to independently access the same sparse connection information without requiring inter-processor communication, thereby maintaining high calculation speed while avoiding communication bottlenecks.
Data Source
AI summary
Aspects for modifying data in a multi-layer neural network (MNN) acceleration processor for neural networks are described herein. As an example, the aspects may include receiving a predetermined weight value array and connection data. Further, the aspects may include modifying the weight values included in the predetermined weight value array based on the connection data. Further still, the aspects may include calculating one or more groups of output data based on the modified weight values.