Memory devices and systems including memory devices

Abstract

A memory device and a system including the memory device are provided. The memory device can include a memory die stack including a plurality of memory dies stacked on the memory die stack, and a base die below the memory die stack. The base die can include a via interface circuit connected to the memory die stack, a die-to-die (2D) interface circuit connected to a processor, a memory interface circuit connected to an external memory device, and a memory management circuit configured to control data migration between the memory die stack and the external memory device, determine whether to migrate first data stored in the memory die stack to the external memory device based on access information related to the first data, and provide second data stored in the external memory device to the processor based on an access request from the processor to access the second data.

Description

[0001] This application is based on and claims priority to Korean Patent Application No. 10-2024-0179699, filed on December 5, 2024, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference. Technical Field

[0002] One or more embodiments of this disclosure relate to memory devices and systems including memory devices, and more specifically, to memory devices having a hierarchical memory structure and systems including memory devices. Background Technology

[0003] As artificial intelligence (AI) systems continue to develop, the demand for semiconductor chips with high-performance and high-capacity data processing capabilities is increasing. Because massive amounts of data need to be processed to enable AI systems, ensuring high computing performance, as well as the memory bandwidth and capacity to support it, is crucial.

[0004] High-bandwidth memory (HBM) is a memory technology developed to meet the ever-growing demand for high data transfer rates and low power consumption. HBM is implemented by vertically stacking multiple memory dies using three-dimensional (3D) stacking technology and connecting the dies using through-hole vias (such as through-silicon vias (TSVs)). HBM can be utilized in graphics processing units (GPUs), AI accelerators, and more to perform well in compute-intensive and data center systems. However, compared to memories used in traditional general-purpose systems, HBM has relatively limited capacity and a higher cost per unit of storage.

[0005] In general-purpose systems, for example, Double Data Rate (DDR) Dynamic Random Access Memory (RAM) (DRAM) (DDR DRAM) can be used. DDR DRAM is widely used in systems such as personal computers (PCs), servers, and workstations, and is characterized by lower bandwidth, but offers higher capacity and higher cost-effectiveness than HBM. Summary of the Invention

[0006] One or more embodiments provide methods for increasing data transfer bandwidth in memory devices and systems while simultaneously achieving high memory capacity and cost-effectiveness.

[0007] According to one aspect of the disclosure, a memory device may include: a memory die stack including a plurality of memory dies stacked on the memory die stack; and a base die below the memory die stack. The base die may include: via interface circuitry including a plurality of through vias and connected to the memory die stack; die-to-die (D2D) interface circuitry connected to a processor; memory interface circuitry connected to an external memory device; and memory management circuitry configured to: control data migration between the memory die stack and the external memory device; determine whether to migrate the first data from the memory die stack to the external memory device based on access information relating to the first data stored in the memory die stack; and provide the second data to the processor based on an access request from the processor for accessing the second data stored on the external memory device.

[0008] According to one aspect of the disclosure, a system may include: a first memory device including a first memory die stack and a first base die, the first memory die stack being configured to store first data; a second memory device including a second memory die stack and a second base die, the second memory die stack being configured to store second data; a third memory device configured to store third data and connected to the first base die; and a logic die including a host processor connected to the first base die and configured to generate access requests related to the first data, the second data, and the third data. The first base die may include: via interface circuitry connected to the first memory die stack; a first die-to-die (D2D) interface circuitry connected to the logic die; a second D2D interface circuitry connected to the second base die; a memory interface circuitry connected to the third memory device; and memory management circuitry configured to control data migration between the first memory die stack, the second memory die stack, and the third memory device.

[0009] According to one aspect of the disclosure, a system may include: a logic die including a host processor; a first memory device including a memory die stack and a base die, the memory die stack including a plurality of memory dies stacked on the memory die stack, the base die being below the memory die stack and connected to the logic die; and a second memory device connected to the base die. The base die may include: via interface circuitry connected to the memory die stack; die-to-die (D2D) interface circuitry connected to the logic die; memory interface circuitry connected to the second memory device; and memory management circuitry configured to: control data migration between the memory die stack and an external memory device, migrate first data stored in the memory die stack to the second memory device based on access information relating to first data stored in the memory die stack, and migrate second data to the memory die stack based on an access request from the logic die for accessing second data stored in the second memory device. Attached Figure Description

[0010] The embodiments will be more clearly understood through the following detailed description taken in conjunction with the accompanying drawings.

[0011] Figure 1 This is a diagram of a system according to one or more embodiments.

[0012] Figure 2 This is a diagram of a first memory device according to one or more embodiments.

[0013] Figure 3 This is a diagram of a first memory device according to one or more embodiments.

[0014] Figure 4 This is a diagram of a system according to one or more embodiments.

[0015] Figure 5 This is an illustration of a base die according to one or more embodiments.

[0016] Figure 6 This is an illustration of a base die according to one or more embodiments.

[0017] Figure 7 This is an illustration of a base die according to one or more embodiments.

[0018] Figure 8 This is a diagram of a system according to one or more embodiments.

[0019] Figure 9 This is a diagram of a system according to one or more embodiments.

[0020] Figure 10This is a diagram of a system according to one or more embodiments.

[0021] Figure 11 This is a diagram of a system according to one or more embodiments. Detailed Implementation

[0022] In the following description, embodiments of the inventive concept are described in detail with reference to the accompanying drawings. When the description is given with reference to the drawings, the same or corresponding components may be given the same reference numerals, and repeated descriptions are omitted.

[0023] Figure 1 This is a diagram of system 10 according to one or more embodiments.

[0024] Reference Figure 1 The system 10 may include a first memory device 100, a second memory device 200, and a logic die 300.

[0025] The first memory device 100 may include a memory die stack 110 and a base die 120. In embodiments, the first memory device 100 may include a three-dimensional (3D) stacked memory or a 3D stacked dynamic random access memory (DRAM), which includes multiple memory dies stacked vertically and connected by one or more through-silicon vias (such as through-silicon vias, or TSVs). For example, the first memory device 100 may include a high-bandwidth memory (HBM) comprising multiple stacked memory dies. In some embodiments, the first memory device 100 may be referred to as a 3D stacked memory, a 3D stacked DRAM, or a custom HBM.

[0026] The memory die stack 110 may include first memory dies MD_1 to Nth memory dies MD_N (N is a natural number equal to or greater than 2). The memory die stack 110 may include multiple channels containing independent interface connections to each other (e.g., Figure 3 The first memory device 100 may include components for 1024-bit data input / output when each of the first channel CH1 to the sixteenth channel CH16 has a bandwidth of 64 bits. In some embodiments, each of the plurality of memory dies may be referred to as a core die.

[0027] The base die 120 may include memory management circuitry 123, die-to-die (D2D) interface circuitry 124, and memory interface circuitry 126. In some embodiments, the base die 120 may be referred to as a buffer die or a bottom die. References are made below. Figure 2 Describe the other components included in the base die 120.

[0028] Memory management circuitry 123 controls data migration between a first memory device 100 and a second memory device 200. The second memory device 200 may include two-dimensional (2D) memory in terms of physical layout, such as Double Data Rate (DDR) DRAM, Low Power Double Data Rate (LPDDR) DRAM, or Graphics Double Data Rate (GDDR) DRAM. In these types of memory, memory chips are horizontally arranged on a substrate, and data communication occurs across the surface via pins or solder balls and in a vertically stacked manner without memory dies. For example, memory management circuitry 123 can control data migration between 3D stacked memory and 2D memory.

[0029] By providing a request signal to the first memory device 100 via a D2D interface circuit 320, the logic die 300 can access the first memory device 100 to write or read data. When the request signal is received by the first memory device 100 from the logic die 300, the memory management circuit 123 can store access information corresponding to the request signal. In this disclosure, "memory access" or "memory access operation" refers to an operation that includes reading data from memory or writing data to memory.

[0030] In one or more embodiments, the logic die 300 may be located outside the first memory device 100 and may include components distinct from the base die 120. That is, the logic die 300 and the base die 120 may be manufactured using different dies. The logic die 300 may be referred to as a semiconductor die containing logic circuitry that performs data-intensive logical operations (e.g., training an artificial intelligence (AI) model, inference using an AI model, etc.) by writing data to or reading data stored in the first memory device 100 and the second memory device 200. In some embodiments, the logic die 300 may be referred to as a computing device, a processor die (e.g., including a general-purpose processor), or an application-specific integrated circuit (ASIC) die.

[0031] In one or more embodiments, the request signal may be referred to as a signal that includes at least one of a command signal, an address signal, and a data signal for write operations to the first memory device 100 and the second memory device 200 and read operations from the first memory device 100 and the second memory device 200.

[0032] In an embodiment, the access information may include information indicating the time point at which a request signal provided by the logic die 300 is received by the first memory device 100.

[0033] In an embodiment, when logic die 300 provides a request signal to first memory device 100 for accessing first data stored in memory die stack 110 of first memory device 100, base die 120 of first memory device 100 may perform an operation (e.g., write or read) on memory die stack 110 in response to the request signal. In response to the access, memory management circuitry 123 may record the access time associated with the first data accessed by logic die 300. Memory management circuitry 123 may continuously monitor access past time, which represents the duration since the most recent access to the first data by logic die 300. When access past time is equal to or greater than a migration reference value (e.g., a predetermined migration threshold), memory management circuitry 123 may determine that the first data will be migrated from first memory device 100 to second memory device 200. First memory device 100, implemented using high-bandwidth memory (HBM), is suitable for storing data with a high access frequency (e.g., frequently used intermediate results or model parameters generated by logic die 300 in AI computations). A second memory device 200, which can be implemented using a two-dimensional memory architecture (such as DDR, LPDDR, or GDDR), can store data with a lower access frequency due to its larger capacity and lower cost-per-bit.

[0034] In an embodiment, the migration reference value may include a value that has been pre-set and stored in the memory management circuit 123.

[0035] In an embodiment, when logic die 300 provides a request signal to first memory device 100 for accessing second data stored in memory 210 of second memory device 200, the base die 120 of first memory device 100 may perform an operation (e.g., write or read) indicated by the request signal on memory 210 of second memory device 200 in response to the request signal. In response to the access, memory management circuitry 123 may analyze the access pattern and determine whether the second data should be migrated from second memory device 200 to first memory device 100. The migration decision may be based on factors such as increased access frequency for computationally intensive tasks being performed by logic die 300, latency sensitivity, or performance optimization. Based on the migration decision, memory management circuitry 123 may determine that the second data will be migrated to first memory device 100.

[0036] The first memory device 100 and the logic die 300 can be interconnected via a D2D interface and can perform communication. A D2D interface circuit 124 can be electrically connected to a D2D interface circuit 320 of the logic die 300. The D2D interface circuit 124 and the D2D interface circuit 320 may include a physical or electronic layer and a logic layer, which are provided with the signals, frequencies, timings, drives, detailed operating parameters, and functions required for efficient communication between the first memory device 100 and the logic die 300.

[0037] In an embodiment, the D2D interface circuit 320 of the logic die 300 and the D2D interface circuit 124 of the base die 120 can operate based on the Peripheral Component Interconnect High Speed ​​(PCIe) or Universal Chip Interconnect High Speed ​​(UCIe) standard specifications in performing communication between the logic die 300 and the base die 120.

[0038] The first memory device 100 and the second memory device 200 can be interconnected via an interface based on the Joint Electronic Devices Engineering Committee (JEDEC) standard specification and can perform communication. Memory interface circuitry 126 can be electrically connected to memory interface circuitry 220 of the second memory device 200. Memory interface circuitry 126 and memory interface circuitry 220 may include a physical or electrical layer and a logic layer, which are provided with the signals, frequencies, timing, drives, detailed operating parameters, and functions required for efficient communication between the first memory device 100 and the second memory device 200.

[0039] In an embodiment, the memory interface circuit 220 of the second memory device 200 and the memory interface circuit 126 of the base die 120 may operate via an interface that meets JEDEC standard specifications such as Double Data Rate (DDR), Low Power (LP) DDR (LPDDR), and Graphics (G) DDR (GDDR) in performing communication between the second memory device 200 and the base die 120.

[0040] In this embodiment, the interface between the first memory device 100 and the logic die 300 may include a non-JEDEC interface that does not meet the JEDEC standard specification, and the interface between the first memory device 100 and the second memory device 200 may include an interface that meets the JEDEC standard specification.

[0041] The logic die 300 may include a processor 310 and a D2D interface circuit 320. In some embodiments, the logic die 300 may be referred to as a host or a system-on-a-chip (SoC).

[0042] Processor 310 can execute applications supported by system 10 by using first memory device 100 and second memory device 200. In order to execute the application, processor 310 can provide request signals to first memory device 100 and second memory device 200 to access first memory device 100 and second memory device 200.

[0043] Processor 310 may be configured to execute one or more machine-executable instructions, or a piece of software, firmware, or a combination thereof. For example, processor 310 may include a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a neural processing unit (NPU), an application processor (AP), a communication processor (CP), a cryptographic processor, a physical processor, a machine learning processor, etc.

[0044] The first memory device 100 may have a relatively larger bandwidth than the second memory device 200 in sending and receiving data to and from the logic die 300. The memory capacity of the first memory device 100 may be smaller than that of the second memory device 200. In one or more embodiments, the first memory device 100 may be configured as a first-level memory, and the second memory device 200 may be configured as a second-level memory. The system 10 may have a hierarchical memory architecture in which memory devices are organized into hierarchical levels based on performance characteristics such as bandwidth, latency, and capacity. In this hierarchical structure, data frequently accessed by the logic die 300 (e.g., model weights, intermediate feature maps, time-critical parameters) may be stored in the first memory device 100, and data not frequently accessed by the logic die 300 may be stored in the second memory device 200. The system according to the embodiments can ensure high memory capacity while maintaining high bandwidth related to data transmission by using a hierarchical memory structure.

[0045] Figure 2 This is a diagram of a first memory device 100 according to one or more embodiments. Figure 2 For reference Figure 1 The description is used to describe it, and its repeated descriptions are omitted.

[0046] Reference Figure 2 The first memory device 100 may include a memory die stack 110 and a base die 120.

[0047] The base die 120 may include a through-silicon via (TSV) interface circuit 121, a first memory controller 122, a memory management circuit 123, a D2D interface circuit 124, a second memory controller 125, and a memory interface circuit 126.

[0048] TSV interface circuit 121 can be electrically connected to multiple TSVs that penetrate the memory die stack 110 in the vertical direction. The base die 120 can perform input / output operations on the data stored in the memory die stack 110 by using TSV interface circuit 121.

[0049] By accessing the memory die stack 110 via the TSV interface circuitry 121, the first memory controller 122 can write data to or read data stored in the memory die stack 110 based on a request signal provided by the logic die 300. In some embodiments, the first memory controller 122 may be referred to as an HBM controller.

[0050] By accessing the second memory device 200 via memory interface circuitry 220, the second memory controller 125 can write data to or read data stored in memory 210 of the second memory device 200 based on a request signal provided by logic die 300. In some embodiments, the second memory controller 125 may be referred to as a 2D memory controller.

[0051] Figure 3 This is a diagram of a first memory device 100 according to one or more embodiments.

[0052] Reference Figure 3 The first memory device 100 may include an HBM, which includes first channels CH1 to sixteenth channels CH16 having interfaces that are independently accessible to each other. The first memory device 100 may include a plurality of memory dies, and may include, for example, a memory die stack 110 and a base die 120. The memory die stack 110 may be stacked on the base die 120.

[0053] The memory die stack 110 may include four memory dies MD1 to MD4, and the memory dies MD1 to MD4 may support sixteen first channels CH1 to sixteenth channels CH16. In some embodiments, the memory dies constituting the memory die stack 110 may be referred to as DRAM dies. In this embodiment, although the first memory device 100 is shown as including four memory dies MD_1 to MD_4, it is not limited thereto, and eight, twelve, sixteen, or more memory dies may be stacked.

[0054] The first memory die MD_1 may include channels CH1 to CH4, the second memory die MD_2 may include channels CH5 to CH8, the third memory die MD_3 may include channels CH9 to CH12, and the fourth memory die MD_4 may include channels CH13 to CH16. When each of channels CH1 to CH16 supports 64 data transfer paths (i.e., when there are 64 data signal pins corresponding to each of channels CH1 to CH16), the first memory device 100, including 16 channels (i.e., channels CH1 to CH16), can support 1024 data transfer paths (i.e., a bandwidth of 1024 bits). However, embodiments are not limited to this; the first memory device 100 may support 1024 or more data transfer paths and supports various numbers of channels (e.g., 8 channels). For example, when the first memory device 100 supports 8 channels and each channel supports 128 data transfer paths, the first memory device 100 can support 1024 data transfer paths.

[0055] Each of the first channel CH1 to the sixteenth channel CH16 may include multiple memory banks (or memory banks) MBK. Each memory bank MBK may include memory cells connected to word lines and bit lines.

[0056] In one or more embodiments, a channel may be divided into two pseudo-channels that operate independently. For example, pseudo-channels may share the channel's commands but may decode and execute commands independently. For example, when a channel supports 64 data transfer paths, each pseudo-channel may support 32 data transfer paths. For example, when a channel includes 32 memory banks (MBKs), each pseudo-channel may include 16 memory banks (MBKs).

[0057] The base die 120 and memory dies MD_1 to MD_4 may include TSV regions TAR. In the TSV region TAR, TSVs are configured to penetrate the memory dies MD_1 to MD_4 in the vertical direction. The base die 120 can transmit and receive various signals via electrical connections to the memory dies MD_1 to MD_4 via TSVs. Each of the memory dies MD_1 to MD_4 can transmit and receive signals to and from the base die 120 and other memory dies via TSVs. In this case, signals can be transmitted and received independently via the TSV corresponding to each channel. For example, when the logic die 300 sends a data signal to the first channel CH1 to store data in the memory cell of the first channel CH1, the base die 120 can send the data signal to the first memory die MD_1 via the TSV corresponding to the first channel CH1. Therefore, data can be stored in the memory cell of the first channel CH1.

[0058] The base die 120 can communicate with conductive components (e.g., bumps or solder balls) formed outside the first memory device 100. The base die 120 can receive request signals from the logic die 300 and can perform operations indicated by the request signals by accessing channels constituting the memory die stack 110 based on commands, addresses, and data representing the received request signals.

[0059] Figure 4 This is a diagram of system 10 according to one or more embodiments. Figure 4 For reference Figures 1 to 3 It is used to describe, and its repeated descriptions can be omitted.

[0060] Reference Figure 4 System 10 may include a first memory device 100, a second memory device 200, a logic die 300, a first substrate 402, a second substrate 404, and an intermediary 406. The first memory device 100 may include a memory die stack 110 and a base die 120. In one or more embodiments, a direction perpendicular to the first substrate 402, the second substrate 404, and the intermediary 406 may be referred to as a first direction D1, a direction perpendicular to the first direction D1 may be referred to as a second direction D2 or a first horizontal direction, and a direction perpendicular to the first direction D1 and the second direction D2 may be referred to as a third direction D3 or a second horizontal direction.

[0061] The memory die stack 110 may include first memory dies MD_1 to fourth memory dies MD_4. The first memory dies MD_1 to fourth memory dies MD_4 may include TSVs 130 penetrating in a first direction D1, which is perpendicular to the first substrate 402. Between each of the first memory dies MD_1 to fourth memory dies MD_4, bumps 140, acting as conductive swellings, are capable of electrically connecting the memory dies.

[0062] Bump 405 may be attached to the upper portion of the second substrate 404, and solder ball 403 may be attached to the lower portion of the second substrate 404. For example, bump 405 may include flip chip bumps. Intermediate 406 may be stacked on the second substrate 404 by using bump 405.

[0063] In one embodiment, the first substrate 402 may include a printed circuit board (PCB). The second substrate 404 may include a package substrate.

[0064] In an embodiment, the second substrate 404 and the components disposed on the second substrate 404 may form a semiconductor package. The semiconductor package may transmit and receive signals to and from other external packages or external semiconductor devices.

[0065] Interposer 406 buffers the linewidth difference between the base die 120 and the second substrate 404. Interposer 406 may include an electrical interface that routes connections to one socket or another to extend electrical wiring to a wider pitch or reroute electrical wiring to a different pitch. Interposer 406 physically connects the base die 120 and the logic die 300 to the second substrate 404. The first memory device 100 can send and receive signals to and from the logic die 300 via wiring arranged in interposer 406.

[0066] Figure 5 This is a diagram of the base die 120a according to one or more embodiments. Figure 5 For reference Figure 1 and Figure 2 It is used to describe, and its repeated descriptions can be omitted.

[0067] Reference Figure 5 , Figure 5 The basic bare die 120a can correspond to Figure 1 The base nude film in the middle is 120.

[0068] The basic die 120a may include a TSV interface circuit 121, a first memory controller 122, a memory management circuit 123a, a D2D interface circuit 124, a second memory controller 125, a memory interface circuit 126, and an interconnect circuit 127.

[0069] exist Figure 5 In the illustration, the base die 120a is shown as including one of each of the TSV interface circuit 121, the first memory controller 122, the second memory controller 125, and the memory interface circuit 126, but this is only an example for illustration, and more circuits (e.g., four memory controllers 122 and four memory controllers 125) may be included.

[0070] Figure 5 The D2D interface circuit 124 shown includes a first D2D interface circuit 124_1 and a second D2D interface circuit 124_2, but this is only an example for description, and the D2D interface circuit 124 may include more circuits (e.g., two first D2D interface circuits 124_1 and two second D2D interface circuits 124_2).

[0071] Figure 5 The memory management circuit 123a shown includes a direct memory access (DMA) controller 123_2a, but this is only an example for illustration, and the memory management circuit 123a may include more circuits (e.g., four DMA controllers 123_2a).

[0072] The memory management circuit 123a may include a microcontroller 123_1a, a DMA controller 123_2a, and an access monitor 123_3a.

[0073] The memory management circuit 123a can process requests received by the base die 120a via the microcontroller 123_1a. Furthermore, the memory management circuit 123a can control the DMA controller 123_2a using the microcontroller 123_1a. By controlling the DMA controller 123_2a based on access information AINF stored in the access monitor 123_3a, the memory management circuit 123a can control the DMA controller 123_2a to perform data migration operations and data prefetch operations between the first memory device 100 and the second memory device 200.

[0074] In an embodiment, the microcontroller 123_1a may include a microcontroller unit (MCU), a central processing unit (CPU), or a graphics processing unit (GPU).

[0075] In one or more embodiments, the DMA controller 123_2a may be referred to as hardware that independently performs data transfers for the memory die stack 110 or the memory 210.

[0076] Access monitor 123_3a can store access information AINF related to memory operations performed by logic die 300. Access monitor 123_3a can monitor data signals input and output via TSV interface circuit 121 and memory interface circuit 126.

[0077] In an embodiment, when logic die 300 accesses first data stored in the memory die stack 110 of the first memory device 100, access monitor 123_3a can record or store access information AINF indicating the memory address and access time of the first data. Furthermore, access monitor 123_3a can monitor or track past time since the last access to the first data by logic die 300. When the past access time indicating the past time since the last access to the first data is equal to or greater than a migration reference value, access monitor 123_3a can determine the migration of the first data. When access monitor 123_3a determines the migration of the first data, memory management circuitry 123a can migrate the first data from the memory die stack 110 of the first memory device 100 (e.g., first layer memory) to the memory 210 of the second memory device 200 (e.g., second layer memory).

[0078] In an embodiment, when the second data stored in memory 210 has been accessed by logic die 300, access monitor 123_3a can determine the migration of the second data. When access monitor 123_3a determines the migration of the second data, memory management circuit 123a can migrate the second data from memory 210 of the second memory device 200 to memory die stack 110 of the first memory device 100.

[0079] In one embodiment, the memory management circuit 123a can perform a prefetch operation on data that is expected to be used in the future.

[0080] In embodiments, the access monitor 123_3a can be implemented in various forms and disposed within the memory management circuitry 123a. For example, the access monitor 123_3a can be implemented as hardware (HW) such as logic circuitry. In this case, the access monitor 123_3a may include a separate memory for storing access information AINF. Alternatively, the access monitor 123_3a can be implemented as software (SW) including programs, or the access monitor 123_3a can be implemented as a combination of HW and SW.

[0081] D2D interface circuit 124 may include a first D2D interface circuit 124_1 and a second D2D interface circuit 124_2. The first D2D interface circuit 124_1 and the second D2D interface circuit 124_2 may be electrically connected to interconnect circuit 127.

[0082] The first D2D interface circuit 124_1 may include circuitry for performing communication between the logic die 300 and the base die 120 by electrically connecting the logic die 300 to the base die 120.

[0083] By electrically connecting the base die 120 to another base die, the second D2D interface circuit 124_2 may include circuitry for performing communication between the first memory device 100 and a memory device having the same structure. (See below for further details.) Figure 8 Describe the second D2D interface circuit 124_2.

[0084] Interconnect circuit 127 may be electrically connected to first memory controller 122, second memory controller 125, D2D interface circuit 124, and memory management circuit 123a. Interconnect circuit 127 may include circuitry for providing communication between first memory controller 122, second memory controller 125, D2D interface circuit 124, and memory management circuit 123a.

[0085] In an embodiment, interconnect circuitry 127 may include circuitry with a network-on-chip (NoC) architecture. Since many IP blocks (such as processor cores, memory, and peripheral devices) are designed in a block-to-block interconnect manner within an integrated system-on-chip (SoC), NoC can be described as an interconnect circuitry architecture that supports communication between various components.

[0086] In this embodiment, the interconnect circuit 127 may operate based on one of a variety of bus protocols. These bus protocols may include at least one of the following: Advanced Microcontroller Bus Architecture (AMBA) protocol, Universal Serial Bus (USB) protocol, Multimedia Card (MMC) protocol, Peripheral Component Interconnect (PCI) protocol, PCI-Fast (E)PCI-E protocol, Advanced Technology Attachment (ATA) protocol, Serial (S)ATA (SATA) protocol, Parallel (P)ATA (PATA) protocol, Small Computer System Interface (SCSI) protocol, Enhanced Small Disk Interface (ESDI) protocol, Integrated Drive Electronics (IDE) protocol, Mobile Industrial Processor Interface (MIPI) protocol, Universal Flash Memory (UFS) protocol, etc.

[0087] Figure 6 This is a diagram of the base die 120b according to one or more embodiments. Figure 6This may be a diagram of one or more embodiments in which the memory management circuit 123b is arranged between the interconnect circuit 127 and the first memory controller 122. Figure 6 For reference Figure 1 , Figure 2 and Figure 5 It is used to describe, and its repeated descriptions can be omitted.

[0088] Reference Figure 6 The base die 120b can correspond to Figure 1 The base nude film in the middle is 120.

[0089] The basic die 120b may include a TSV interface circuit 121, a first memory controller 122, a memory management circuit 123b, a D2D interface circuit 124, a second memory controller 125, a memory interface circuit 126, interconnect circuitry 127, and a microcontroller 128. The following description primarily focuses on... Figure 5 The differences.

[0090] Microcontroller 128 may correspond to microcontroller 123_1a. Memory management circuitry 123b may include DMA controller 123_2b and access monitor 123_3b. DMA controller 123_2b may correspond to... Figure 5 The DMA controller 123_2a and access monitor 123_3b can correspond to Figure 5 Access monitor 123_3a. Memory management circuitry 123b is electrically connected to the first memory controller 122.

[0091] Interconnect circuit 127 can be electrically connected to memory management circuit 123b, D2D interface circuit 124, second memory controller 125 and microcontroller 128.

[0092] Figure 7 This is a diagram of the base die 120c according to one or more embodiments. Figure 7 This is a schematic diagram showing the circuit layout that makes up the basic die 120c. Figure 7 For reference Figures 1 to 6 It is used to describe, and its repeated descriptions can be omitted.

[0093] Reference Figure 7 The base die 120c can correspond to Figure 1 The basic die 120c may include a TSV interface circuit 121, an HBM controller 122c, a microcontroller 123_1, a DMA controller 123_2, an access monitor 123_3, a first D2D interface circuit 124_1, a second D2D interface circuit 124_2, a 2D memory controller (e.g., a conventional memory controller) 125c, a memory interface circuit 126, and an interconnect circuit 127.

[0094] Figure 7 Four of each of the TSV interface circuit 121, HBM controller 122c, and DMA controller 123_2 are shown, but this is an example for illustration and the number may be greater or less than four.

[0095] Figure 7 Two of each of the first D2D interface circuit 124_1, the second D2D interface circuit 124_2, and the 2D memory controller 125c are shown, but this is an example for description and the number may be greater than or less than 2.

[0096] HBM controller 122c can correspond to Figure 2 The first memory controller 122 in the 2D memory controller 125c can correspond to Figure 2 The second memory controller 125 in the middle.

[0097] Microcontroller 123_1 can correspond to Figure 5 The microcontroller 123_1a or Figure 6 The microcontroller 128 in the DMA controller 123_2 can correspond to... Figure 5 DMA controller 123_2a or Figure 6 The DMA controller 123_2b in the middle. Access monitor 123_3 can correspond to Figure 5 Access monitor 123_3a or Figure 6 Access monitor 123_3b in the system.

[0098] The components included in the base die 120c can be electrically connected to each other via interconnect circuitry 127.

[0099] The first D2D interface circuit 124_1 may include a logic die electrically connected to ( Figure 1 The 300 in the middle) is an interface circuit that performs communication with the logic die 300.

[0100] The second D2D interface circuit 124_2 may include interface circuitry that performs communication with another base die via electrical connection to that base die. In an embodiment, a first memory device 100 including base die 120c may communicate with a third memory device having the same structure as the first memory device 100. In this case, the first memory device 100 may communicate with the third memory device via the second D2D interface circuit 124_2 of base die 120c. This communication refers to... Figure 8 Described below.

[0101] Memory interface circuit 126 may include a second memory device electrically connected to it. Figure 1The 200 in the middle) is an interface circuit that performs communication with the second memory device 200.

[0102] In an embodiment, the base die 120c may further include test logic circuitry containing logic for testing defects in the memory die stack 110.

[0103] Figure 8 This is a diagram of system 10a according to one or more embodiments. Figure 8 For reference Figures 1 to 7 It is used to describe, and its repeated descriptions can be omitted.

[0104] Reference Figure 8 System 10a may include a first logic die 300_1a, a second logic die 300_2a, a first memory device 100_1a, a second memory device 200_1a, a third memory device 100_2a, and a fourth memory device 200_2a.

[0105] Each of the first logic die 300_1a and the second logic die 300_2a can correspond to Figure 1 The logic die 300 in the first memory device 100_1a and the third memory device 100_2a may correspond to the logic die 300 in the third memory device 100_2a. Figure 1 The first memory device 100. Each of the second memory device 200_1a and the fourth memory device 200_2a may correspond to the first memory device 100. Figure 1 The second memory device 200 in.

[0106] The interface circuit for communication between the first logic die 300_1a and the first memory device 100_1a may be a first D2D interface circuit 124_1. In one or more embodiments, the first D2D interface circuit 124_1 may be a D2D interface circuit for communication between the logic die and the memory device.

[0107] In this embodiment, when the first logic die 300_1a needs to access the first memory device 100_1a to write or read data, the first logic die 300_1a can access the first memory device 100_1a via the first D2D interface circuit 124_1. The first path PATH1 can indicate the path taken by the first logic die 300_1a to access the first memory device 100_1a.

[0108] The interface circuit for communication between the first memory device 100_1a and the third memory device 100_2a may be a second D2D interface circuit 124_2. In one or more embodiments, the second D2D interface circuit 124_2 may represent a D2D interface circuit for communication between memory devices.

[0109] In this embodiment, when the first logic die 300_1a needs to access the third memory device 100_2a to write or read data, the first logic die 300_1a can access the third memory device 100_2a via the first D2D interface circuit 124_1 and the second D2D interface circuit 124_2. The second path PATH2 can indicate the path taken by the first logic die 300_1a to access the third memory device 100_2a.

[0110] Figure 9 This is a diagram of system 10b according to one or more embodiments. Figure 9 For reference Figures 1 to 8 It is used to describe, and its repeated descriptions can be omitted.

[0111] Reference Figure 9 The system 10b may include multiple logic dies 300b, multiple first memory devices 100b and multiple second memory devices 200b.

[0112] As Figure 9 The components of system 10b, including six logic dies 300b, six first memory devices 100b, and six second memory devices 200b, are shown, but this is only an example, and the number of components may be less or greater than these.

[0113] Each of the multiple logic dies 300b can correspond to Figure 1 The logic die 300 in the middle. Each of the plurality of first memory devices 100b may correspond to Figure 1 The first memory device 100. Each of the plurality of second memory devices 200b may correspond to Figure 1 The second memory device 200 in.

[0114] In an embodiment, the plurality of first memory devices 100b may include HBM, and the plurality of second memory devices 200b may include DDR DRAM, LPDDR DRAM, or GDDR DRAM.

[0115] In an embodiment, the plurality of first memory devices 100b may include HBM, and the plurality of second memory devices 200b may also include HBM. In this case, the plurality of first memory devices 100b may include components Figure 1 The base die 120 may include a custom HBM that does not meet the JEDEC standard specification in performing communication with multiple logic dies 300b respectively. Multiple second memory devices 200b may include HBMs that communicate with multiple first memory devices 100b respectively according to the JEDEC standard specification.

[0116] Figure 10This is a diagram of system 10c according to one or more embodiments. Figure 10 For reference Figures 1 to 9 It is used to describe, and its repeated descriptions can be omitted.

[0117] Reference Figure 10 The system 10c may include a logic die 300c, a plurality of first memory devices 100c and a plurality of second memory devices 200c.

[0118] Logic die 300c can correspond to Figure 1 The logic die 300 in the middle. Each of the plurality of first memory devices 100c may correspond to Figure 1 The first memory device 100. Each of the plurality of second memory devices 200c may correspond to Figure 1 The second memory device 200 in.

[0119] Figure 10 System 10c can be used with Figure 9 The system is similar to 10b, but there are differences. Figure 9 System 10b includes multiple logic dies 300b. Figure 10 The system 10c may include a logic die 300c. Figure 10 The logic die 300c of the system 10c can be designed to communicate with a plurality of first memory devices 100c and a plurality of second memory devices 200c.

[0120] Figure 11 This is a diagram of a system 1000 according to one or more embodiments.

[0121] Figure 11 This is a block diagram of a system 1000 including a heterogeneous memory system according to an embodiment.

[0122] Reference Figure 11 System 1000 may include a camera 1100, a display 1200, an audio processor 1300, a modem 1400, HBM 1510a and 1510b, DRAM 1520a and 1520b, flash memory devices 1600a and 1600b, input / output (I / O) devices 1700a and 1700b, and an access point (AP) 1800. System 1000 may be implemented as a laptop computer, mobile phone, smartphone, tablet PC, wearable device, healthcare device, or Internet of Things (IoT) device. Furthermore, system 1000 may be implemented as a server or PC.

[0123] Camera 1100 can capture still or moving images under user control and can store the captured images / image data or send the captured images / image data to display 1200. Audio processor 1300 can process audio data included in network content or in flash memory devices 1600a and 1600b. Modem 1400 can modulate and transmit signals for transmitting and receiving wired / wireless data and demodulate the modulated signals to recover the original signal at the receiving side. I / O devices 1700a and 1700b may include devices that provide digital input and / or output functions (such as Universal Serial Bus (USB) or storage devices), digital cameras, Secure Digital (SD) cards, Digital Universal Discs (DVDs), network adapters, and touchscreens.

[0124] AP 1800 may include a controller 1810 configured to control all operations of system 1000, and an interface 1830 configured to send data and signals to or receive data and signals from external components. AP 1800 may control display 1200 such that a portion of the contents stored in flash memory devices 1600a and 1600b is displayed on display 1200. When user input is received via I / O devices 1700a and 1700b, AP 1800 may perform control operations corresponding to the user input. AP 1800 may include an accelerator block as dedicated circuitry for artificial intelligence (AI) data computation, or may include an accelerator chip 1820 (or accelerator 1820) separate from AP 1800. DRAM may be additionally mounted on the accelerator block or accelerator chip 1820. Accelerators can be function blocks that specialize in performing specific functions of the AP 1800, and can include GPUs as function blocks that specialize in processing graphics data, NPUs as function blocks that specialize in AI computing and inference, and data processors (DPUs) as function blocks that specialize in data transfer.

[0125] System 1000 may include multiple HBM 1510a and 1520a and multiple DRAM 1520a and 1520b. AP 1800 may correspond to Figure 1 The logic die 300 in the middle, each of the multiple HBM 1510a and 1510b can correspond to Figure 1 The first memory device 100, and each of the plurality of DRAMs 1520a and 1520b may correspond to Figure 1 The second memory device 200 in.

[0126] Multiple HBM 1510a and 1510b modules and multiple DRAM 1520a and 1520b modules offer comparable I / O devices 1700a and 1700b and flash memory devices 1600a and 1600b with relatively low latency and relatively high bandwidth. Multiple HBM 1510a and 1510b modules and multiple DRAM 1520a and 1520b modules can be initialized at the time of system 1000's power-on and, after the operating system and application data (or AP data) are loaded, can be used as arbitrary storage locations for the operating system and AP data or as execution space for various software codes.

[0127] Addition / subtraction / multiplication / division (the four basic arithmetic operations), vector computation, address computation, or Fast Fourier Transform (FFT) computation can be performed in multiple HBM 1510a and 1510b and multiple DRAM 1520a and 1520b. Furthermore, functions for performing inference can be performed in multiple HBM 1510a and 1510b and multiple DRAM 1520a and 1520b. In this case, inference can be performed using a deep learning algorithm that utilizes an artificial neural network. The deep learning algorithm may include training operations that learn a model using various data and inference operations that identify data using the learned model. As one or more embodiments, an image captured by a user using camera 1100 can be signal processed for storage in HBM 1510b or DRAM 1520b, and the accelerator block or accelerator chip 1820 can perform AI data computation for identifying the data using the data stored in HBM 1510b or DRAM 1520b and functions for inference operations.

[0128] System 1000 may include multiple storage devices or multiple flash memory devices 1600a and 1600b, which have a larger capacity than multiple HBM 1510a and 1510b and multiple DRAM 1520a and 1520b. An accelerator block or accelerator chip 1820 can perform training operations and AI data operations using the flash memory devices 1600a and 1600b. Each of the flash memory devices 1600a and 1600b may include a memory controller 1610 and flash memory 1620.

[0129] Although the inventive concept has been specifically shown and described with reference to embodiments thereof, it will be understood that various changes in form and detail may be made therein without departing from the spirit and scope of the appended claims.

Claims

1. A memory device, comprising: A memory die stack, comprising multiple stacked memory dies; as well as The base die is located below the memory die stack. The basic nude film includes: Via interface circuitry connects to the memory die stack; The die-to-die interface circuit connects to the processor. Memory interface circuitry, connecting to an external memory device; and The memory management circuit is configured as follows: Controls data migration between memory die stacks and external memory devices; Based on access information relating to first data stored in the memory die stack, determine whether to migrate the first data from the memory die stack to an external memory device; and The second data is provided to the processor based on the access request from the processor for accessing the second data stored in the external memory device.

2. The memory device according to claim 1, wherein, The memory management circuitry includes: First controller; The second controller is configured to, under the control of the first controller, access the memory die stack and external memory devices; and The access monitor is configured to store access information related to the first data stored in the memory die stack.

3. The memory device according to claim 2, wherein, The memory management circuitry is also configured as follows: Monitor the access patterns used to access the first data stored in the memory die stack, and The memory address storing the first data and the last access time of the first data are used as access information.

4. The memory device according to claim 1, wherein, The via interface circuit is connected to the plurality of memory dies by passing through a plurality of vias in the vertical direction.

5. The memory device according to claim 1, further comprising: Interconnect circuits connect to die-to-die interface circuits, memory interface circuits, and memory management circuits.

6. The memory device according to claim 1, further comprising: The first memory controller is configured to write data to or read data stored in the memory die stack based on a request from the processor. The first memory controller is also configured to be connected to the via interface circuit.

7. The memory device according to claim 1, further comprising: The second memory controller is configured to write data to or read data stored in an external memory device based on a request from the processor. The second memory controller is connected to the memory interface circuit.

8. The memory device according to claim 1, in, The die-to-die interface circuitry is configured to communicate with the processor based on general-purpose chip interconnect high-speed standards, and The memory interface circuit is configured to communicate with external memory devices based on the Joint Electronic Devices Engineering Committee (JEDC) standard specifications.

9. The memory device according to claim 1, in, Access information includes access past time, which indicates the time elapsed since the last access by the processor to the first data stored in the memory die stack, and The memory management circuitry is also configured to migrate first data from the memory die stack to an external memory device based on the access past time being equal to or greater than a migration reference value.

10. The memory device according to any one of claims 1 to 9, in, The memory management circuitry is also configured to determine, based on a request from the processor for accessing second data stored in an external memory device, whether to migrate the second data from the external memory device to the memory die stack.

11. A system including a memory device, comprising: A first memory device includes a first memory die stack and a first base die, the first memory die stack being configured to store first data; A second memory device includes a second memory die stack and a second base die, the second memory die stack being configured to store second data; A third memory device is configured to store third data and is connected to the first base die; as well as A logical die, connected to the first underlying die, is configured to generate access requests related to the first, second, and third data. The first basic nude film includes: Via interface circuitry connects to the first memory die stack; The first die-to-die interface circuit is connected to the logic die; The second die-to-die interface circuit is connected to the second base die. Memory interface circuitry, connected to a third memory device; and The memory management circuit is configured to control data migration between the first memory die stack, the second memory die stack, and the third memory device.

12. The system according to claim 11, wherein, The first base bare metal is configured as follows: Based on an access request from the logic die for accessing the first data, the first data is accessed via a via interface circuit. Based on the access request from the logic die for accessing the second data, the second data is accessed via the second die-to-die interface circuit, and The third data is accessed via the memory interface circuitry based on an access request from the logic die for accessing the third data.

13. The system according to claim 11, wherein, The memory management circuitry is also configured as follows: Based on access information relating to first data stored in the first memory die stack, the first data is migrated to the third memory device, and Based on the access request from the logic die for accessing the third data, determine whether to migrate the third data to the first memory die stack.

14. The system according to claim 11, in, The first die-to-die interface circuit and the second die-to-die interface circuit are configured to communicate with logic dies based on general-purpose chip interconnect high-speed standards, and The memory interface circuit is configured to communicate with the third memory device based on the Joint Electronic Devices Engineering Committee (JEDC) standard specifications.

15. The system according to any one of claims 11 to 14, wherein, Each of the first and second memory die stacks includes a plurality of memory dies stacked in a vertical direction.

16. A system including a memory device, comprising: Logic dies; A first memory device includes a memory die stack and a base die, the memory die stack including a plurality of stacked memory dies, and the base die being below the memory die stack and connected to a logic die; as well as A second memory device is connected to the base die. The basic nude film includes: Via interface circuitry connects to the memory die stack; Die-to-die interface circuitry connects to logic dies; Memory interface circuitry, connected to a second memory device; and The memory management circuit is configured as follows: Controls data migration between the memory die stack and the second memory device. Based on access information related to the first data stored in the memory die stack, the first data stored in the memory die stack is migrated to the second memory device, and Based on an access request from the logic die for accessing second data stored in the second memory device, the second data is migrated to the memory die stack.

17. The system according to claim 16, in, The system also includes: First base; The second substrate, on the first substrate; and Intermediate body, on the second base, Among them, the logic dies and the basic dies are on the intermediate body, and The second memory device is located on the first substrate and is separated from the second substrate in the horizontal direction.

18. The system according to claim 16, in, The die-to-die interface circuit is configured to communicate with logic dies based on general-purpose chip interconnect high-speed standards, and The memory interface circuit communicates with the second memory device based on the Joint Electronic Devices Engineering Committee (JEDC) standard specifications.

19. The system according to claim 16, in, Access information includes access past time, which indicates the time elapsed since the last access by the logical die pair to the first data stored in the memory die stack. The memory management circuit is further configured to migrate first data from the memory die stack to the second memory device based on the access past time being equal to or greater than a migration reference value.

20. The system according to any one of claims 16 to 19, wherein, The memory management circuitry is also configured to determine, based on a request from the logic die for accessing second data stored in the second memory device, whether to migrate the second data from the second memory device to the memory die stack.

Images