NAND Memory Arrays for Ternary Neural Network Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks are computationally intensive and require significant memory and data transfer for weight operations, leading to high energy dissipation and inefficiency in inference processes.
Innovation Solution
The use of Binary Neural Networks (BNNs) and ternary-weight neural networks, where weights and inputs are truncated into binary or ternary values, and stored and processed using NAND memory arrays, simplifying operations to XNOR and bit-count operations, and utilizing multi-bit sense amplifiers for increased parallelism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full-precision floating-point operations are used in neural networks, then high accuracy is achieved, but computational complexity and energy dissipation increase significantly
Solution Approach 1:
The patent applies parameter changes by quantizing neural network weights from full-precision floating-point values to lower-precision representations (binary, ternary, or low-bit integer values). This reduction in precision parameters significantly decreases computational complexity and energy consumption while maintaining acceptable accuracy through optimized quantization algorithms and calibration techniques.
Solution Approach 2:
The patent extracts and eliminates unnecessary computational operations by removing redundant calculations in neural network inference. By using quantized weights and optimized algorithms, the system extracts only the essential computational steps needed for accurate inference, reducing overall computational load and energy dissipation.
2Measurement precision
If full-precision floating-point weights are stored and transferred, then high accuracy is maintained, but memory requirements and data transfer intensity increase
Solution Approach 1:
The patent changes the precision parameters of weight storage from full-precision floating-point to lower-precision formats (binary, ternary, or low-bit integer). This parameter reduction dramatically decreases memory requirements and data transfer bandwidth needs while maintaining sufficient accuracy for practical neural network applications through careful quantization design.
Solution Approach 2:
The patent segments the weight data into quantized representations that can be stored more efficiently. By dividing the weight space into discrete quantization levels, the system reduces the amount of data that needs to be stored and transferred, while still capturing the essential information needed for accurate neural network inference.
3Device complexity
If binary or ternary quantization is applied, then computational complexity is reduced, but accuracy may deteriorate
Solution Approach 1:
The patent applies parameter changes by using mixed-precision quantization schemes where different parts of the neural network use different quantization levels. Critical layers or neurons that require higher accuracy can use finer quantization (e.g., 4-bit or 8-bit), while less critical parts use coarser quantization (e.g., binary or ternary), optimizing the balance between complexity reduction and accuracy maintenance.
Solution Approach 2:
The patent applies local quality by allowing different quantization precisions in different regions of the neural network. Instead of uniform quantization across all layers, the system applies higher precision where needed and lower precision where acceptable, thereby maintaining overall accuracy while maximizing computational efficiency gains from quantization.
Data Source
AI summary
Use of a NAND array architecture to realize a binary neural network (BNN) allows for matrix multiplication and accumulation to be performed within the memory array. A unit synapse for storing a weight of a BNN is stored in a pair of series connected memory cells. A binary input is applied on a pair of word lines connected to the unit synapse to perform the multiplication of the input with the weight. The results of such multiplications are determined by a sense amplifier, with the results accumulated by a counter. The arrangement extends to ternary inputs to realize a ternary-binary network (TBN) by adding a circuit to detect 0 input values and adjust the accumulated count accordingly. The arrangement further extends to a ternary-ternary network (TTN) by allowing 0 weight values in a unit synapse, maintaining the number of 0 weights in a register, and adjusting the count accordingly.