Neural Network Read Controller for Variable-Block Address Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network computation methods in integrated circuits require numerous clock cycles and significant memory usage due to the sequential nature of dot product computations and storage of intermediate terms, leading to inefficiencies in resource utilization and computation speed.
Innovation Solution
An integrated circuit design that includes a neural network computation fabric with cores that compute dot products in parallel, a global channel for aggregation, and post-processing units, optimized for convolutional neural networks with sparse weight values and efficient data storage and retrieval mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If MAC circuits are used to compute dot products sequentially, then computation accuracy is maintained, but computation speed deteriorates requiring numerous clock cycles
Solution Approach 1:
The patent segments the weight values into multiple groups (first group, second group, third group, fourth group) and computes dot products for each group in parallel using separate MAC circuits. This segmentation enables simultaneous computation of multiple terms in the dot product, dramatically increasing computation speed while maintaining accuracy through proper aggregation of all segment results
Solution Approach 2:
The patent merges multiple partial dot product results from different MAC circuit groups into a final aggregated result. The adder circuits combine the partial sums from all parallel MAC operations to produce the complete dot product, ensuring computational accuracy is maintained while benefiting from parallel processing speed
2Reliability
If intermediate terms are stored in memory during dot product computation, then computation completeness is ensured, but resource usage deteriorates with significant memory consumption
Solution Approach 1:
The patent implements continuous computation by processing weight values in parallel groups without storing intermediate partial results in memory. Multiple MAC circuits compute different groups of dot product terms simultaneously and continuously, feeding their results directly to adder circuits for immediate aggregation, thereby eliminating the need for memory storage of intermediate terms
Solution Approach 2:
The patent introduces adder circuits as intermediary components between parallel MAC circuits and the final output. These adder circuits serve as mediators that aggregate partial dot product results in real-time, replacing the need for memory storage by providing a direct computational path from parallel MAC operations to the final aggregated result
3Productivity
If parallel MAC circuits are used to compute multiple dot product terms simultaneously, then computation speed is improved, but device complexity increases with more circuit surface area
Solution Approach 1:
The patent transitions from sequential single-dimensional computation to parallel multi-dimensional computation by organizing MAC circuits into multiple groups that operate simultaneously. Each group processes a specific dimension or segment of the dot product computation, enabling parallel execution without requiring a single massive circuit, thus improving speed while managing complexity through structured organization
4Ease of manufacture
If weight values are loaded from off-chip storage at startup, then initialization is completed, but power consumption increases during data transfer
Solution Approach 1:
The patent performs preliminary action by loading all weight values from off-chip storage into on-chip memory during system initialization or startup phase. This one-time data transfer prepares the system for subsequent neural network computations, eliminating the need for repeated off-chip memory accesses during inference and thereby reducing power consumption during actual computation operations
Data Source
AI summary
Some embodiments provide a neural network inference circuit (NNIC) for executing a neural network (NN) that includes computation nodes at multiple layers. The NNIC includes a set of processing circuits for executing the computation nodes of the NN, a set of memories for storing data used by the processing circuits to execute the NN layers, and a read controller for retrieving the data from the memories for use by the processing circuits. The data is stored in the memories as multiple varying-size blocks. The read controller receives read instructions for a requested block of data to be used by the processing circuits for one or more computation nodes. The read instructions include a base memory address for multiple blocks of data, a size of the requested block of data, and a location of the requested block of data within the multiple blocks of data.