Mixed Memory Synaptic Layers for Neural Network Weight Stability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In-memory computing devices using reprogrammable non-volatile memory cells, such as floating gate memories and resistive RAMs, experience fluctuations in stored weights due to device variability and non-ideal operations, leading to less accurate output data from neural networks.
Innovation Solution
An integrated circuit with a neural network implementation in-memory computing device featuring multiple types of synaptic layers, where the first type of memory cells are configured for more accurate and stable data storage and operations compared to the second type, reducing weight fluctuations and improving inference accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If reprogrammable non-volatile memory cells are used for in-memory computing, then device versatility and reconfigurability are improved, but weight stability and inference accuracy deteriorate due to device variability and operation fluctuations
Solution Approach 1:
The patent applies local quality by using different memory cell types for different synaptic layers. Specifically, volatile memory cells (e.g., DRAM) are used in synaptic layers closer to the input layer where weight stability is less critical, while non-volatile memory cells (e.g., resistive RAM) are used in synaptic layers closer to the output layer where weight stability is more important for accurate inference. This spatial differentiation of memory cell properties resolves the contradiction between reconfigurability and weight stability.
2Duration of action of stationary object
If non-volatile memory cells are used for weight storage, then data retention and non-volatility are improved, but measurement precision and weight accuracy deteriorate due to read/write operation inaccuracies
Solution Approach 1:
The patent segments the synaptic layers into different groups based on their requirements for data retention versus measurement precision. Synaptic layers that benefit more from non-volatility are assigned non-volatile memory cells, while layers that require higher precision are assigned volatile memory cells with more accurate read/write operations. This segmentation allows the system to optimize for different priorities in different parts of the neural network.
Data Source
AI summary
An in-memory computing device includes a plurality of synaptic layers including a first type of synaptic layer and a second type of synaptic layer. The first type of synaptic layer comprises memory cells of a first type of memory cell and the second type of synaptic layer comprises memory cells of a second type, the first type of memory cell being different than the second type of memory cell. The first and second types of memory cells can be different types of memories, have different structures, different memory materials, and/or different read/write algorithms, any one of which can result in variations in the stability or accuracy of the data stored in the memory cells.