Lookup-Table Neural Network Circuit for Low-Power Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network architectures, which rely on graphical processing units (GPUs) for high-speed arithmetic operations, face challenges due to high power consumption and significant chip space requirements, limiting their deployment in computing devices.
Innovation Solution
The implementation of lookup table techniques in neural network processing nodes to approximate scaling operations, using reduced bit representation and probability-based encoding to minimize the size and power consumption of lookup tables, replacing floating-point multipliers with more power-efficient solutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If graphical processing units (GPUs) are used for high-speed arithmetic operations in neural networks, then processing speed is improved, but power consumption and chip space requirements increase significantly
Solution Approach 1:
The patent segments the multiplication operation into two phases: (1) pre-computation phase where lookup tables are generated offline storing pre-computed products of weight values and input ranges, and (2) runtime phase where only table lookup and addition operations are performed. This segmentation moves the computationally intensive multiplication to the pre-computation phase, allowing the runtime processing to use simpler, lower-power operations.
Solution Approach 2:
The patent performs preliminary action by pre-computing and storing multiplication results in lookup tables before actual neural network inference. The lookup tables contain pre-calculated values of weight × input combinations, which are generated offline and then used during runtime without performing actual multiplication operations, thereby reducing real-time power consumption.
2Speed
If graphical processing units (GPUs) are used for high-speed arithmetic operations in neural networks, then processing speed is improved, but chip space requirements increase significantly
Solution Approach 1:
The patent segments the computational workload between pre-computation (offline) and runtime (online) phases. The complex multiplication operations are segmented into the pre-computation phase where lookup tables are generated, while the runtime phase uses only simple addition and table lookup operations, reducing the need for complex hardware circuits on the chip.
Solution Approach 2:
The patent creates a copy of multiplication results in the form of lookup tables that store pre-computed values. Instead of implementing full multiplication circuits on the chip, the system copies the essential computational results into lookup tables, allowing the hardware to use simpler structures that occupy less chip space while maintaining processing capability.
3Use of energy by moving object
If lookup tables are used to approximate scaling operations, then power consumption is reduced, but lookup table size increases
Solution Approach 1:
The patent applies local quality by using reduced bit representation specifically for the lookup table indices and entries. Instead of using full precision floating-point numbers throughout, the system uses reduced bit representations (e.g., 8-bit or 16-bit integers) for lookup table operations where high precision is not critical, thereby reducing lookup table size while maintaining acceptable accuracy for the approximation.
Solution Approach 2:
The patent changes parameters by using reduced bit representation and probability-based encoding schemes. The lookup tables store values with reduced precision (fewer bits) compared to full floating-point representation, and probability-based encoding further compresses the table size by storing only essential information. These parameter changes reduce lookup table size while the approximation error remains acceptable for neural network inference.
4Quantity of substance
If reduced bit representation is used in lookup tables, then lookup table size is reduced, but approximation accuracy decreases
Solution Approach 1:
The patent applies partial action by using reduced bit representation that provides sufficient (but not excessive) precision for neural network inference. Instead of using full floating-point precision, the system uses reduced bit formats that provide the necessary accuracy for the application while accepting that some precision is lost. This partial precision approach achieves a balance between table size and accuracy suitable for approximate computing in neural networks.
Data Source
AI summary
In one example, an integrated circuit includes a first circuit, a second circuit, a third circuit, and a fourth circuit. The first circuit is configured to receive an input value and generate a first intermediate value based on a first probability density distribution associated with the input value. The second circuit comprises a set of multiplexer circuits configured to select, from a first set of candidate values and based on the first intermediate value, a first product of the first intermediate value and a weight value. The third circuit is configured to generate a second intermediate value based on a sum of the first product and a second product received from another circuit. The fourth circuit is configured to generate an output value based on the second intermediate value and a second probability density distribution associated with the second intermediate value.