3D Hybrid Memory Stack Using NVM Layers for Thermal Buffering

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current memory technologies face challenges such as yield issues, non-linear power increase, thermal management, and scalability limitations in achieving high memory density and performance, particularly due to the limitations of traditional DRAM and the inefficiencies in hybrid memory systems that do not fully leverage the unique properties of volatile and non-volatile memory technologies.

Innovation Solution

The use of stacked memory dies that combine high and low operational temperature memory technologies, including non-volatile memory technologies like FeRAM and volatile memory technologies like DRAM, to create hybrid memory stacks that optimize performance and power efficiency by exploiting the specific properties of each memory type, with non-volatile memory dies functioning as temperature buffers and tolerating higher temperatures without increased refresh rates.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If traditional DRAM is used to increase memory density, then memory capacity increases, but power consumption increases non-linearly and thermal management becomes difficult

Engineering Contradiction:

Improvememory densityVSAvoidpower consumption

Core Design Contradiction:

Quantity of substanceVSUse of energy by stationary object

Solution Approach 1:

The memory system is segmented into multiple types (DRAM, FeRAM, MRAM, PCM) with different characteristics, organized in a hybrid architecture where each type serves specific functions. This segmentation allows the system to leverage the high speed of DRAM for frequently accessed data while using non-volatile memories for less frequently accessed data, thereby reducing overall power consumption while maintaining high memory density.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different memory types are strategically placed in the memory hierarchy based on their local qualities: DRAM is positioned for high-speed access to hot data, while FeRAM, MRAM, and PCM are used for cold data storage. This local quality assignment optimizes the trade-off between speed, power, and density for different data sets, reducing non-linear power increase while achieving high memory capacity.

Inventive Principle:

Principle #3Local quality

2Speed

If stacked DRAM is used to increase memory bandwidth, then memory bandwidth increases, but thermal challenges negatively impact DRAM retention

Engineering Contradiction:

Improvememory bandwidthVSAvoidDRAM retention

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

Non-volatile memory types (FeRAM, MRAM, PCM) are introduced as intermediary layers in the memory stack between the CPU and DRAM. These intermediary memories act as thermal buffers, absorbing heat away from the DRAM layers while maintaining data retention capabilities at elevated temperatures, thus protecting DRAM retention reliability while preserving high bandwidth through the stacked architecture.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Use of energy by stationary object

If non-volatile memory is used to avoid refresh requirements, then power efficiency improves, but scalability and large capacity are limited

Engineering Contradiction:

Improvepower efficiencyVSAvoidmemory capacity

Core Design Contradiction:

Use of energy by stationary objectVSQuantity of substance

Solution Approach 1:

The patent merges multiple memory types (DRAM, FeRAM, MRAM, PCM) into a unified hybrid memory system. Non-volatile memories provide refresh-free operation for power efficiency, while their capacities are combined and complemented by DRAM layers to achieve large overall system capacity. This merging allows the system to overcome the individual capacity limitations of each non-volatile memory type while maintaining their power efficiency advantages.

Inventive Principle:

Principle #5Merging (Combining)

4Loss of time

If HBM stack is used to reduce off-chip access latency, then access latency decreases, but yield challenges and complexity increase

Engineering Contradiction:

Improveaccess latencyVSAvoidsystem complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The hybrid memory stack is designed with multi-functionality where different memory types serve multiple purposes: DRAM provides high-speed caching, non-volatile memories provide persistent storage and thermal management, and the stacked architecture itself provides both high bandwidth and reduced latency. This universal design reduces the need for separate specialized components, thereby managing complexity while achieving low access latency through integrated multi-functional memory layers.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20240088098A1Hybrid memory architecture for advanced 3D systems

Publication Date: 2024.03.14 ADVANCED MICRO DEVICES INC

AI summary

Disclosed wherein stacked memory dies that utilize a mix of high and low operational temperature memory and non-volatile based memory dies, and chip packages containing the same. High temperature memory dies, such as those using non-volatile memory (NVM) technologies are in a memory stack with low temperature memory dies, such as those having volatile memory technologies. In some cases, the high temperature memory technologies could be used together, in some cases, on the same IC die as logic circuitry. In one example, a memory stack is provided that include a first memory IC die having high temperature memory circuitry, such as non-volatile memory, stacked below a second memory IC die. The second memory IC die has high temperature memory circuitry, such as volatile memory circuitry.