Neural Network Computation Fabric Parallel Dot Product Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network integrated circuits face inefficiencies in computation due to the need for numerous clock cycles and resource usage for memory operations in performing dot products, particularly in massively parallel computations.
Innovation Solution
The integrated circuit employs a neural network computation fabric with multiple cores that compute partial dot products in parallel, a global channel for aggregating these partial dot products, and post-processing units that apply bias, scale, and activation functions to produce the final outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If MAC circuits are used to compute dot products sequentially, then computation accuracy is maintained, but the number of clock cycles required increases significantly
Solution Approach 1:
The patent divides the dot product computation into multiple segments, each handled by a separate MAC circuit. Instead of one MAC circuit computing all N terms sequentially, multiple MAC circuits compute different segments of the dot product in parallel, reducing the total computation time while maintaining accuracy through proper aggregation of partial results.
Solution Approach 2:
The patent transitions from sequential computation in time to parallel computation across multiple spatial dimensions (multiple MAC circuits). By adding spatial parallelism through multiple computation units, the system reduces time consumption without sacrificing computational accuracy.
2Productivity
If parallel computation is implemented to reduce clock cycles, then computation speed improves, but circuit surface area increases significantly
Solution Approach 1:
The patent implements parallel computation with a number of MAC circuits that is less than the full N required for complete parallelism. Each MAC circuit computes a portion of the dot product, and the results are aggregated. This partial parallelism achieves significant speedup while keeping the circuit area manageable by not over-provisioning the number of parallel units.
Solution Approach 2:
The patent combines multiple partial dot product results from parallel MAC circuits into a final result through aggregation circuits. This merging of partial computations achieves the benefits of parallel speedup while using fewer individual MAC circuits than full parallelism would require, thus controlling the overall circuit surface area.
3Measurement precision
If intermediate terms are stored in memory during computation, then computation accuracy is maintained, but resource usage for read/write operations increases
Solution Approach 1:
The patent extracts only the essential intermediate results (partial dot products from each MAC circuit) that need to be stored, rather than storing all intermediate computation terms. By taking out only the necessary partial results for aggregation, the system maintains computational accuracy while minimizing memory resource usage and read/write operations.
4Loss of time
If more MAC circuits are used to compute multiple terms in parallel, then computation time decreases, but device complexity increases
Solution Approach 1:
The patent uses a partial number of MAC circuits (less than N) to compute a portion of the dot product terms in parallel. This partial parallelism achieves significant computation time reduction without the full device complexity that would result from using N MAC circuits for complete parallelism, balancing performance and complexity.
Data Source
AI summary
Some embodiments provide a method for a circuit that executes a neural network including multiple nodes. The method loads a set of weight values for a node into a set of weight value buffers, a first set of bits of each input value of a set of input values for the node into a first set of input value buffers, and a second set of bits of each of the input values into a second set of input value buffers. The method computes a first dot product of the weight values and the first set of bits of each input value and a second dot product of the weight values and the second set of bits of each input value. The method shifts the second dot product by a particular number of bits and adds the first dot product with the bit-shifted second dot product to compute a dot product for the node.