Universal Memory Cell for AI Training and Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory technologies struggle to simultaneously provide high endurance for AI training and high retention for AI inference, necessitating separate memory types for each mode.
Innovation Solution
A universal memory semiconductor circuit is designed to operate in both DRAM-like mode for high endurance during AI training and NVM-like mode for high retention during AI inference, utilizing a two-transistor structure with a charge trap layer that can be altered by specific write voltages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of moving object
If a memory is designed for high endurance to support AI training operations, then the memory can be repeatedly programmed and erased, but the retention capability deteriorates making it unsuitable for AI inference
Solution Approach 1:
The memory device dynamically switches between two operational modes: DRAM mode for high-endurance training operations and NVM mode for high-retention inference operations. The mode selection is controlled by voltage signals applied to the control gate, allowing the same physical structure to exhibit different functional characteristics based on operational requirements.
Solution Approach 2:
The memory device changes its operational parameters by adjusting the voltage state of the control gate. When the control gate is at a first voltage state, the device operates in DRAM mode with high endurance. When the control gate is at a second voltage state, the device operates in NVM mode with high retention. This parameter change allows the same device to satisfy contradictory requirements for different AI workloads.
2Reliability
If separate memory types are used for AI training and AI inference, then each memory can be optimized for its specific function, but the system complexity increases
Solution Approach 1:
The memory device is designed as a universal memory that can perform both DRAM-like and NVM-like operations within the same physical structure. By incorporating a control gate that can be switched between different voltage states, the device achieves multi-functionality, eliminating the need for separate memory types for training and inference operations.
Solution Approach 2:
The invention merges the functionality of DRAM and NVM into a single memory device structure. The control gate mechanism allows the same physical components to serve dual purposes: enabling high-endurance operations when needed and providing high-retention operations when needed, thereby simplifying the overall system architecture.
3Reliability
If multiple memory types are deployed for different AI modes, then performance requirements are met, but power consumption and latency increase
Solution Approach 1:
The universal memory device eliminates the need for data transfer between separate DRAM and NVM components by performing both training and inference operations within the same device. This reduces the energy associated with data movement and eliminates the latency of interfacing with multiple memory types.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The universal memory achieves efficient operation in both training and inference modes, reducing the need for multiple memory types and minimizing power consumption and latency, thereby enhancing overall AI system performance.
Implementation Method 1
The charge trap layer is configured to: be unalterable when a first write voltage is applied at the first terminal of the first transistor, and be alterable when a second write voltage is applied at the first terminal of the first transistor to change a threshold voltage of the second transistor
Implementation Method 2
where the second write voltage is a voltage high enough to realize Fowler-Nordheim tunneling and hot carrier injection in the charge trap layer of the second transistor
Implementation Method 3
where the second write voltage is a voltage high enough to realize Fowler-Nordheim tunneling and hot carrier injection in the charge trap layer of the second transistor
Data Source
AI summary
A universal memory device includes an array of universal memory cells. Each universal memory cell includes a write transistor and a read transistor. The write transistor has a gate terminal configured to receive a gate voltage to turn on or off the write transistor, a first terminal configured to receive a write voltage, and a second terminal coupled to a gate terminal of the read transistor. The read transistor includes a charge trap layer at the gate terminal of the read transistor. The charge trap layer is configured to: be unalterable when the first write voltage is applied at the first terminal of the write transistor, and be alterable when the second write voltage is applied at the first terminal of the write transistor to change a threshold voltage of the read transistor. The second write voltage is greater than the first write voltage.