Binary Neural Network Inference in Non-Volatile Memory Latch Structure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks are computationally intensive and require significant data transfer for weight propagation, leading to high power consumption and memory requirements during inference operations.
Innovation Solution
Implementing Binary Neural Networks (BNNs) and performing computations within the latch structure of non-volatile memory devices, where weights and inputs are truncated into binary values, allowing for XNOR and bit-count operations, reducing the need for external data transfer and enhancing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If traditional neural network inference is performed with external data transfer, then computational accuracy is maintained, but power consumption and data transfer requirements increase significantly
Solution Approach 1:
The patent merges the neural network weight storage function with the inference computation function by implementing both within the same non-volatile memory device. The weights are stored in the memory array and the inference operations are performed using the memory cell currents directly, eliminating the need for separate data transfer between storage and processing units.
Solution Approach 2:
The memory device performs inference operations using its own internal resources - the memory cell currents representing weights are directly utilized for computation without requiring external processing units. The device serves both as storage medium and computation engine, reducing dependency on external systems.
2Productivity
If full-precision weights are transferred for inference, then computational accuracy is maintained, but computation time and bandwidth requirements increase
Solution Approach 1:
The patent changes the representation parameter of weights from multi-bit digital values to analog-like current magnitudes in memory cells. This parameter change enables direct utilization of memory cell properties for computation, allowing inference operations to be performed using the physical state of the memory cells themselves rather than requiring digital data transfer.
3Loss of energy
If neural network weights are stored in external memory, then flexibility is maintained, but memory bandwidth consumption increases during inference
Solution Approach 1:
The non-volatile memory device is designed to serve multiple functions: it acts as both the storage medium for neural network weights and the computation engine for inference operations. The same memory array and circuitry are utilized for both data retention and mathematical operations, eliminating the need for separate specialized hardware components.
Data Source
AI summary
A non-volatile memory device includes an array of non-volatile memory cells that are configured to store weights of a neural network. Associated with the array is a data latch structure that includes a page buffer, which can store weights for a layer of the neural network that is read out of the array, and a transfer buffer, that can store inputs for the neural network. The memory device can perform multiply and accumulate operations between inputs and weight of the neural network within the latch structure, avoiding the need to transfer data out of the array and associated latch structure for portions of an inference operation. By using binary weights and inputs, multiplication can be performed by bit-wise XNOR operations. The results can then be summed and activation applied, all within the latch structure.