Technology for non-deterministic operation of stacked memory systems

Abstract

Techniques for non-deterministic operation of a stacked memory system are provided. In one example, a method for operating a memory package may include receiving a plurality of memory access requests for a channel on a logic die; returning a first data to a host in response to a first memory access request among the plurality of memory access requests; returning a second memory access request among the plurality of memory access requests for second data to a host indicating that the data is not ready; returning a first index to a host along with an indication that the data is not ready; returning a third data along with an indication that the data is ready in response to a third memory access request among the plurality of memory access requests; and returning a first index along with an indication that the data is ready.

Description

Technology Field

[0001] Priority and related applications

[0002] This application claims the benefit of priority to U.S. provisional patent application No. 62 / 953,821 of Pawlowski, filed on December 26, 2019, titled "TECHNIQUES FOR NON-DETERMINISTIC OPERATION OF A STACKED MEMORY SYSTEM," the entirety of which is incorporated by reference into this application.

[0003] Technology field

[0004] The following concerns operating a memory array in general, and more specifically, increasing the bandwidth of a stacked memory device. Background Technology

[0005] Memory devices are widely used to store information in various electronic devices, such as computers, wireless communication devices, cameras, and digital displays. Information is stored by programming various states of the memory device. For example, binary devices often have two states, represented by logic "1" or logic "0". In other systems, more than two states may be stored. To access stored information, components of the electronic device can read or detect the states stored in the memory device. To store information, components of the electronic device can write or program states to the memory device.

[0006] There are various types of memory devices, including magnetic hard disks, random access memory (RAM), read-only memory (ROM), DRAM, synchronous dynamic RAM (SDRAM), ferroelectric RAM (FeRAM), magnetic RAM (MRAM), resistive RAM (RRAM), flash memory, and phase change memory (PCM). Memory devices can be volatile or non-volatile.

[0007] Improving memory devices generally involves increasing memory cell density, read / write speeds, reliability, data retention, power consumption, or manufacturing costs, among other metrics. While advancements in memory technology have enabled improvements across many of these metrics, memory bandwidth can become a bottleneck in improving overall system performance as processing speeds improve. Brief explanation of the drawing

[0008] In drawings that are not necessarily drawn to scale, similar numbers may describe similar components in different drawings. Similar numbers with different letter suffixes may represent different instances of similar components. The drawings generally illustrate the various embodiments described in this document as examples rather than limitations. FIG. 1 illustrates an example of a memory die that supports features and operations according to an example of the present disclosure. FIG. 2 schematically illustrates an example of a device that supports features and operations according to an example of the present disclosure. FIG. 3 schematically illustrates an exemplary storage system including a host device capable of requesting and receiving information from a storage system according to the present subject. Figure 4 illustrates an overall and exemplary timeline of the flow of information between a channel and a host of an exemplary memory package according to the present subject. FIG. 5 illustrates an overall and exemplary method of operation of a memory device according to the present subject. FIG. 6 illustrates an exemplary method of operation of a host according to the present subject. FIG. 7 schematically illustrates a drawing of a system including a device that supports a storage system including a stacked DRAM device according to an embodiment disclosed in the present application. Specific details for implementing the invention

[0009] Techniques for non-deterministic operation of a stacked memory system are provided. In one example, a method for operating a memory package may include receiving a plurality of memory access requests for a channel on a logic die; returning a first data to a host in response to a first memory access request among the plurality of memory access requests; returning a second memory access request among the plurality of memory access requests for second data to a host indicating that the data is not ready; returning a first index to a host along with an indication that the data is not ready; returning a third data along with an indication that the data is ready in response to a third memory access request among the plurality of memory access requests; and returning a first index along with an indication that the data is ready.

[0010] FIG. 1 is a schematic diagram of an exemplary memory package (110) comprising an interface (IF) chip (106) or circuit and a plurality of core chips (100). For example, the memory package (100) may be a 3D memory device such as high bandwidth memory (HBM), hypermemory cube (HMC), Wide-IO DRAM, etc. The memory package (110) is formed by stacking chips vertically as shown in FIG. 1. The stacked chips may include two stacks (12 and 13) each assigned stack IDs "0" and "1". Each stack (12, 13) may individually include core chips (12a to 12d, 13a to 13d). In certain examples, each stack may have a plurality of bit channels per chip. In certain examples, each plurality of bit channels may include at least 128 bits over a width of 1024 bits or more across eight channels. The interface (IF) chip (11) of the memory package (110) may provide an interface with a plurality of input / output channels. In a particular example, each of the channels may function independently of one another between the core chips (12a to 12d and 13a to 13d) and a host device which may be a memory controller (not shown). The IF chip (106) may connect each channel to the host device through a plurality of data queues (DQ). Each channel may include a plurality of memory cells and circuitry for accessing the memory cells. For example, the memory cells may be DRAM memory cells.

[0011] FIG. 2 illustrates a device or system (290) that supports channel routing for memory packages according to various examples disclosed herein. The system (290) may include a host device (205) and a plurality of memory packages (210). In a conventional system, the plurality of memory devices are of the same type, such as DRAM memory devices. In certain examples, the memory devices may include a mixture of capacitive-based memory devices, such as DRAM memory devices, and cross-linked inverter memory devices, such as SRAM memory devices. The inventors recognized that bandwidth improvement can be realized when the host has access to a second, faster type of memory, such as SRAM memory.

[0012] The host device (205) may be an example of a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU)) or a system-on-chip (SoC). In some cases, the host device (205) may be a separate component from the memory device so that the host device (205) can be manufactured separately from the memory device. The host device (205) may be located outside the memory device (210) (e.g., a laptop, a server, a personal computing device, a smartphone, a personal computer). In the system (290), the memory package (210) may be configured to store data for the host device (205).

[0013] A host device (205) can exchange information with a memory package (210) using a signal communicated through a signal path. The signal path may be a path through which a message or transmission can move from a transmitting component to a receiving component. In some cases, the signal path may be a conductor coupled with at least two components, wherein the conductor may optionally allow electrons to flow between at least two components. The signal path may be formed in a wireless medium, as in the case of wireless communication (e.g., radio frequency (RF) or optical). The signal path may at least partially include a first substrate, such as an organic substrate of a memory device, and / or a second substrate, such as a package substrate (e.g., a second organic substrate), which can be coupled with at least one of the memory package (210) and the host device (205), even if not both. In some cases, the memory package (210) may function as a slave device to a host device (205) that can function as a master device.

[0014] In some applications, the system (290) can benefit from a high-speed connection between the host device (205) and the memory device (210). As such, some memory packages (210) support applications, processes, host devices, or processors that have bandwidth requirements of multiple terabytes per second (TB / s). Within an acceptable energy budget, meeting these bandwidth constraints can pose a problem in certain situations.

[0015] The memory die (200) of the memory package (210) may be configured to work with a number of types of communication media (211) (e.g., substrates such as organic substrates and / or high-density interposers such as silicon interposers). The host device (205) may be configured with an interface or ball-out including a design of terminals (e.g., a matrix or pattern) in some cases.

[0016] In some cases, a buffer layer may be located between the memory die (200) and the communication medium (211). The buffer layer may be configured to drive (e.g., re-drive) signals to and from the memory die (200). In some cases, the stack of the memory die (200) may be bufferless, which means that there is no buffer layer or the base layer does not include a re-driver among other components. In a specific example of a bufferless memory, a routing layer or a logic die (206) may be located between the memory die (200) or the stack of the memory die (200) and the communication medium (211). In a specific example, the logic die (206) may form a lower layer of the memory die (200). In a specific example, the bufferless memory package (210) may include a lower memory die (200) having a logic die layer (206).

[0017] FIG. 3 schematically illustrates an exemplary storage system (391) comprising a host device (305) and a memory package (310) according to the subject. The host (305) may request and receive information from the memory package (310) according to the subject using a bus outside the memory package. The host device (305) may be, but is not limited to, a CPU, a graphics processing unit (GPU), an acceleration processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and any other component of a larger system communicating with the storage system (310). In some embodiments, the device (305) may be multiple devices accessing the same storage system (310). The memory package (310) may include a logic die (306) integrated with a memory stack (320), such as a stack of dynamic random access memory (DRAM) devices.

[0018] The logic die (306) may include a host interface (331) connected to a stacked DRAM control (332) and prefetch and cache logic (333). The stacked DRAM control (332) is connected to and interfaces with the memory stack (320). The prefetch and cache logic (333) may be connected to a prefetcher, a prefetch buffer, and a cache array (334). The prefetcher may be a hardware prefetcher. The prefetch buffer and cache array (334) may be an SRAM array, or any other memory array technology, or a register having a faster access speed than the memory type used in the memory stack (320), but are not limited thereto.

[0019] The host interface (331) may include an instruction decoder (335) and an interface register (336). The host interface (331), more specifically, the instruction decoder (335), may receive all incoming memory requests for the memory stack (320) from the host (305). Requests may be sent to the prefetch and cache logic (333) (e.g., next-line, stride, etc.). The prefetch and cache logic (333) may monitor incoming memory requests. Prefetched data may be placed in the prefetch buffer and cache array (334). The prefetch and cache logic (333) may also check for any incoming memory requests for data in the prefetch buffer and cache array (334). Any hit may be served directly from the prefetch buffer and cache array (334) without proceeding to the stacked DRAM control (332). This can not only reduce service latency for these requests but also reduce contention for any remaining requests (i.e., requests not hit in the prefetch buffer and cache array (334)) in stacked DRAM control (332).

[0020] A prefetcher may include any prefetching algorithm / method or a combination of algorithms / methods. Due to the row buffer-based organization of most memory technologies (e.g., DRAM), prefetch algorithms that leverage spatial locality (e.g., next-line, small stride, etc.) have relatively low overhead because prefetch requests will (likely) hit the row buffer in memory. Implementations may issue prefetch requests for large data blocks (i.e., data with a value of two or more 64B cache lines), such as prefetching the entire row buffer, half of the row buffer, or other granularities.

[0021] The prefetch buffer and cache array (334) may be implemented as a direct mapping, set associative, or fully associative cache structure. In an embodiment, the prefetch buffer and cache array (334) may be used to service only read requests (i.e., a write would cause invalidation of the prefetch buffer entry or a write-through policy would be used). In another embodiment, the prefetch buffer and cache array (334) may use alternative policies such as LRU (Least Recently Used), LFU (Least Frequency Used), or FIFO (First In First Out). If the prefetch unit generates a request for a data size larger than the cache line, the prefetch buffer and cache array (334) may also need to be configured with a corresponding wider data block size. In some embodiments, sub-blocking may be used.

[0022] Although described in this application as being used in a memory configuration organized with one logic chip and one or more memory chips, other physical manifestations exist. Although described as a vertical stack of logic dies having one or more memory chips, other embodiments may have some or all of the logic arranged horizontally on separate chips on an interposer or packaged together in a multi-chip module (MCM). Two or more logic chips may be included in the entire stack or system.

[0023] As discussed above, the prefetch and cache logic (333) can also check for any incoming memory requests for data within the prefetch buffers and cache array (334). Any hit can be served directly from the prefetch buffers and cache array (334) without going to the stacked DRAM control (332). This can reduce service latency for these requests and reduce contention for any remaining requests in the stacked DRAM control (332). However, prefetch or cache misses can still occur. Conventional architectures provide a Data Error (DERR) indication at the host interface for each channel of the memory stack when a request cannot be served. For example, when a host read request fails to hit data associated with a prefetch or catch, the host interface indicates a cache miss by setting a Data Error (DERR) indication for the corresponding channel at an individual logic level. If a data error is detected, the host typically moves on to the next access request and then reissues the failed request later.

[0024] The inventors have recognized that a logic die having a memory-side SRAM cache as described above can operate to assist and improve accessing memory data after a missed access request. In this context, a "missed access request" is received by the memory interface but cannot be processed from the cache at that time. In certain examples, the system (391) may utilize the burst length of the memory system to encode additional information associated with the missed access requests. Generally, the burst length is the number of clock cycles used by a channel to exchange information across a data queue (DQ). In the example discussed above in relation to FIG. 1, a channel width of 32 may require 8 clock cycles to provide a 128-bit channel length word, and thus the burst length is 8. In certain examples, each cycle of the burst length may use the Physical Data Error (DERR) bit of each channel I / O to convey 8 bits of information, for example, regarding actual data errors, whether the previous request was ready, and an index associated with a missed request. The 8 bits may be transmitted serially during the burst using the Physical Data Error bit of each channel I / O. As an example, the serially communicated bits of the Physical Data Error I / O point (DERR) may be assigned as follows:

[0025] Bit function

[0026] 0 data errors

[0027] 1 Not Ready

[0028] 2 Warnings

[0029] 3-7 Index

[0030] It is understood that other sequences of bit allocations are possible without departing from the scope of the subject matter of the present invention. The "data error" function differs slightly from conventional methods in that the data error bit can be activated when a request cannot be serviced and is not serviced. In such a scenario, if a "data error" indication is received after making a memory access request, the host will need to reissue the memory access request at least once.

[0031] The "not ready" feature or bit can be enabled when a request is a missed request. For example, the "not ready" feature / bit can be enabled when a request is received, and although the request cannot be serviced by the cache, the interface plans to attempt to read the data associated with the missed request from the cache in the near future. In addition to enabling the "not ready" feature / bit, index bits (e.g., 3-7) can provide an index number associated with the missed request on the corresponding channel. The index number can be used by the host to track and later capture the corresponding data associated with previous missed requests.

[0032] The "alert" function or bit may be enabled when data associated with a previous missed request is available in the cache. Additionally, when the alert function / bit is enabled, index bits may provide the index number corresponding to the missed request. In some examples, the host interface may provide data within an interval immediately after the corresponding "alert" function is enabled. In some examples, the "alert" function may resume the missed read access request associated with the index to obtain the corresponding data. In certain examples, the cache may contain a list of index numbers associated with pending requests.

[0033] In the above example, the burst length allows 5 index bits. As such, each channel has up to 32 pending (e.g., 2 5 There may be (number) of missed requests. In some examples, some of the index numbers may be assigned to convey other information, and thus, the number of pending missed requests may be smaller. Additionally, exemplary memory systems may have burst lengths or channel widths different from the above examples without departing from the scope of the present invention.

[0034] FIG. 4 illustrates an overall and exemplary timeline (400) of information flow between a channel of an exemplary memory package (310) according to the present invention and a host (305). The memory package (310) may include a logic die (306) and a stack of memory (320), such as a stack of DRAM. The logic die (306) may include a host interface (331) and a cache (337). In certain examples, the cache (337) may include SRAM. In certain examples, the cache (337) may include buffers and tag maps associated with conventional stacked memory devices. In (401), the host (305) may request data from the memory system, and the request may be received at the host interface (331). In (402), the host interface (331) may request data from the cache (337). At (403), after determining that the data is in the cache (337), the data can be received at the host interface (331), and at (404), the data can be delivered to the host (305). At (405), the host (305) can request the second data from the memory system, and the request can be received at the host interface (331). At (406), the host interface (331) can request the data from the cache (337). At (407), after determining that the data is not in the cache (337), the status of the cache request can be received at the host interface (331). At (408), the host interface (331) can request the second data from the memory stack (320), and at (409), the missed request can be reported to the host (305).

[0035] In certain examples, at (409), when reporting a missed request, the host interface (331) may use a physical data error (DERR) output point for the channel to indicate that the request is not ready and also to provide an index number for the request. A compatible host (305) may use the index number to retrieve the requested data later, as discussed below. In certain examples, because accessing the memory stack (320) is more time-consuming than servicing data requests from the cache (337), the host interface (331) may make the channel available for additional data requests while the second data is being retrieved. Thus, at (410), the host (305) may request a third data from the memory system, and the request may be received at the host interface (331). At (411), the host interface (331) may request data from the cache (337). In (412), after determining that data is in the cache (337), the data may be received from the host interface (331), and in (413), the data may be transferred to the host (305). In (414), the second data may be transferred from the memory stack (320) to the cache (337). Without going out of the scope of the subject, it is understood that the host interface (331) may receive one or more data requests from the host (305) before receiving a request for the missed data.

[0036] In (416), the host (305) may request fourth data from the memory system, and the request may be received at the host interface (331). In (417), the host interface (331) may request data from the cache (337). In (418), after determining that the data is not in the cache (337), the status of the cache request may be received at the host interface (331). In (419), the host interface (331) may request fourth data from the memory stack (320), and in (420), the missed request may be reported to the host (305). As before, in certain examples, when reporting a missed request in (420), the host interface (331) may use a physical data error (DERR) output point for the channel to indicate that the request is not ready and to provide an index number for the request. The index number may differ from the index number associated with reporting the missing request for the second data in (409).

[0037] In (421), the host (305) may request the fifth data from the memory system, and the request may be received at the host interface (331). In (422), the host interface (331) may request data from the cache (337). In (423), after determining that the data is in the cache (337), the data may be received at the host interface (331), and in (424), the data may be delivered to the host (305). Additionally, when the fifth data is delivered to the host (305), the host interface (331) may activate the aforementioned "data" ready bit or "warning" bit during the burst, and may also provide a corresponding index number for a request for the second data when the second data is now available in the cache (337). In a particular example, in (425), the cache (337) may deliver the second data to the host (305) via the host interface (331). In some examples, the host interface (331) may require the host (305) to resend a request for the second data before delivering the second data. In some examples, delivering the second data through the host interface (331) may involve separate transactions between the cache (337) and the host interface (331) and between the host (305) interface and the host (305), each transaction may be separated at a different time than illustrated without going out of the scope of the subject. In (426), the fourth data may be delivered from the memory stack (320) to the cache (337).

[0038] In (427), the host (305) may request the sixth data from the memory system, and the request may be received at the host interface (331). In (428), the host interface (331) may request the sixth data from the cache (337). In (429), after determining that the sixth data is in the cache (337), the sixth data may be received at the host interface (331), and in (430), the sixth data may be delivered to the host (305). Additionally, when the sixth data is delivered to the host (305), the host interface (331) may enable a data ready indicator, such as a “data ready” bit, during a DERR burst, and may also provide a corresponding index number for a request for the fourth data when the fourth data is now available in the cache (337). In certain examples, in (431), the cache (337) may deliver the fourth data to the host (305) through the host interface (331). In some examples, the host interface (331) may require the host (305) to resend a request for the fourth data before delivering the fourth data from the cache (337). In some examples, delivering the fourth data through the host interface (331) may involve separate transactions between the cache (337) and the host interface (331) and between the host interface (331) and the host (305), each transaction may be separated at a different time than illustrated without going out of the scope of the subject matter.

[0039] FIG. 5 illustrates an overall and exemplary method (500) for operating a memory device according to the present invention. In (501), a plurality of memory access requests may be received from a host in a memory system comprising a stack of DRAM memory devices. In (503), a first data may be returned to the host in response to a first memory access request among a plurality of memory access requests. In (505), a first indication that data is not ready may be returned to the host in response to a second memory access request among a plurality of memory access requests. In (507), an index may be returned with a first indication that data is not ready. In (509), a third indication that data is ready may be returned with third data in response to a third memory access request among a plurality of memory access requests. In (511), a first index may be returned with an indication that data is ready. In (513), the second data can be returned to the host in response to the first index and the indication that the data is ready, received along with the third memory access request.

[0040] FIG. 6 illustrates an exemplary method (600) for operating a host according to the present invention. In (601), a plurality of memory access requests may be sent to a memory system having a stack of memory devices. In (603), a first data may be received in response to the first request among the plurality of memory access requests. In (605), an indication that data is not ready may be received in response to the second request among the plurality of memory access requests, and the second request is a request for the second data. In (607), an index may be received along with an indication that data is not ready. In (609), a third data may be received in response to the third request among the plurality of memory access requests. In (611), an index and an indication that data is ready may be received along with the third data. In (613), a second data may be received in response to the indication that data is ready and the index.

[0041] FIG. 7 schematically illustrates a diagram of a system (700) including a device (705) that supports a storage system including a stacked DRAM device according to an embodiment disclosed in the present application. The device (705) may include components for bidirectional voice and data communication, including a memory controller (715), a memory cell (720), a basic input / output system (BIOS) component (725), a processor (730), an I / O controller (735), a peripheral component (740), a memory chip (755), a system memory controller (760), an encoder (765), a decoder (770), and a multiplexer (775). These components may communicate electronically through one or more buses (e.g., bus (710)). For example, the bus (710) may have a bus width of 16 data lines ("DQ" lines). The bus (710) can electronically communicate with 32 banks of memory cells.

[0042] The memory controller (715 or 760) may operate one or more memory cells as described in the present application. Specifically, the memory controller may be configured to support flexible multi-channel memory. In some cases, the memory controller (715 or 760) may operate a row decoder, a column decoder, or both, as described with reference to FIG. 1. The memory controller (715 or 760) may be configured to communicate electronically with a host and to transmit data during the rising edge and falling edge, respectively, of the clock signal of the memory controller (715 or 760).

[0043] The memory cell (720) can store information as described in the present application (i.e., in the form of a logical state). The memory cell (720) may represent, for example, the memory cell (105) described with reference to FIG. 1. The memory cell (720) may communicate electronically with a memory controller (715 or 760), and the memory cell (720) and the memory controller (715 or 760) may be located on a chip (755), which may be one or more planar memory devices as described in the present application. The chip (755) may be managed, for example, by a system memory controller (715 or 760).

[0044] A memory cell (720) may represent a first array of memory cells having a plurality of zones coupled to a substrate. Each of the plurality of zones may include a plurality of memory cell banks and a plurality of channels crossing the first array of memory cells. At least one of the plurality of channels may be coupled to at least one zone. A memory controller (715 or 760) may be configured to transfer data between the coupled zone and the memory controller (715 or 760).

[0045] The BIOS component (725) may be a software component that includes a BIOS operating as firmware capable of initializing and executing various hardware components. The BIOS component (725) may also manage data flow between the processor and various other components, such as peripheral components, input / output control components, etc. The BIOS component (725) may include a program or software stored in read-only memory (ROM), flash memory, or any other non-volatile memory.

[0046] The processor (730) may include intelligent hardware devices (e.g., a general-purpose processor, a digital signal processor (DSP), a central processing unit (CPU), a microcontroller, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof). In some cases, the processor (730) may be configured to operate a memory array using a memory controller (715 or 760). In other cases, the memory controller (715 or 760) may be integrated into the processor (730). The processor (730) may be configured to execute computer-readable instructions stored in memory to perform various functions (e.g., functions or tasks that support flexible multi-channel memory).

[0047] The I / O controller (735) can manage input and output signals for the device (705). The I / O controller (735) can also manage peripherals that are not integrated into the device (705). In some cases, the I / O controller (735) may represent a physical connection or port to an external peripheral. The I / O controller (735) may use an operating system such as iOS®, ANDROID®, MS-DOS®, MS-WINDOWS®, OS / 2®, UNIX®, LINUX®, or other known operating systems. In other cases, the I / O controller (735) may represent or interact with a modem, keyboard, mouse, touchscreen, or similar device. In some cases, the I / O controller (735) may be implemented as part of a processor. The user can interact with the device (705) through the I / O controller (735) or through a hardware component controlled by the I / O controller (735).

[0048] The peripheral component (740) may include any input or output device, or an interface to such device. Examples may include a disk controller, a sound controller, a graphics controller, an Ethernet controller, a modem, a USB (universal serial bus) controller, a serial or parallel port, a peripheral component interconnect (PCI) or accelerated graphics port (AGP) slot, or a peripheral card slot.

[0049] The input (745) may represent a signal or device outside the device (705) that provides input to the device (705) or its component. This may include a user interface or an interface with or between other devices. In some cases, the input (745) may be managed by an I / O controller (735) and may interact with the device (705) through a peripheral component (740).

[0050] The output (750) may also represent a signal or device outside the device (705) configured to receive an output from the device (705) or any of its components. Examples of the output (750) may include a graphic display, an audio speaker, a printing device, another processor, or a printed circuit board. In some cases, the output (750) may be a peripheral element that interfaces with the device (705) through peripheral component(s) (740). The output (750) may be managed by an I / O controller (735).

[0051] The system memory controller (715 or 760) may communicate electronically with a first array of memory cells (e.g., memory cells (720)). The host may be a component or device that controls or directs the operation of the memory controller (715 or 760) and the corresponding memory array, which are part of the device. The host may be a component such as a computer or a mobile device. Or the device (705) may be referred to as the host. In some examples, the system memory controller (715 or 760) is a GPU.

[0052] The encoder (765) may represent a signal or device outside the device (705) that provides for the performance of error correction encoding for data to be stored in the device (705) or its component. The encoder (765) may write encoded data to at least one selected memory through at least one channel and may also encode data through error correction coding.

[0053] The decoder (770) may represent a signal or device outside the device (705) that sequences command signals and addressing signals to the device (705) or its components. In some examples, a memory controller (715 or 760) may be located together within the decoder (770).

[0054] The multiplexer (775) may represent a signal or device outside the device (705) that multiplexes data to the device (705) or its components. The multiplexer (775) may multiplex data to be transmitted to the encoder (765) and demultiplex data received from the encoder (765). The multiplexer (775) may communicate electronically with the decoder (770). In some examples, the multiplexer (775) may communicate electronically with a controller such as a system memory controller (715 or 760).

[0055] A component of the device (705) may include a circuit designed to perform its function. This may include various circuit elements, such as conductive lines, transistors, capacitors, inductors, resistors, amplifiers, or other active or inactive elements configured to perform the function described in this application. The device (705) may be a computer, server, laptop computer, notebook computer, tablet computer, mobile phone, wearable electronic device, personal electronic device, etc. Or the device (705) may be a part or aspect of such a device. In some examples, the device (705) is an aspect of a computer having high reliability, mission-critical, or low-latency constraints or parameters, such as a vehicle (e.g., autonomous vehicle, airplane, spacecraft, etc.). The device (705) may be or may include logic for artificial intelligence (AI), augmented reality (AR), or virtual reality (VR) applications.

[0056] In one example, the memory device may include an array of memory cells having a plurality of zones, each of which may include a plurality of banks of memory cells and a plurality of channels crossing the array of memory cells. Each channel may be coupled to a zone of the array of memory cells and may be configured to communicate signals between a plurality of banks of memory cells within the zone together with a host device.

[0057] In some examples, the memory device may further include an I / O region extending across an array of memory cells, and the I / O region occupies a region of the array of memory cells where there may be no memory cells. In some examples of the memory device, the I / O region may include a TSV configured to connect the array of memory cells to a power node or a ground node.

[0058] In some examples, the memory device may further include a plurality of channel interfaces distributed in an array of memory cells. In some examples of the memory device, the plurality of channel interfaces may be bump-out. In some examples of the memory device, the channel interfaces of the plurality of channel interfaces may be located in each quadrant of the array of memory cells.

[0059] In some examples, the memory device may further include multiple signal paths extending between the memory cells of a region and the channel interface associated with the region. In some examples of the memory device, the channel interface may be located in an array of memory cells to minimize the length of the signal path.

[0060] In some examples, the memory device may further include a second array of memory cells stacked on top of an array of memory cells. In some examples of the memory device, the second array of memory cells may have a zone that may each include a plurality of banks of memory cells. In some examples, the memory device may further include a second plurality of channels traversing the second array of memory cells. In some examples of the memory device, each of the second plurality of channels may be coupled to a second zone of the second array of memory cells and may be configured to communicate signals between a plurality of banks of memory cells in the second zone where a host device is located.

[0061] In some examples, the memory device may further include a TSV extending through the array of memory cells to combine a second array of memory cells with a second plurality of channels. In some examples of the memory device, the channel may establish a point-to-point connection between a zone and a host device. In some examples of the memory device, each channel may include four or eight data pins. In some examples of the memory device, a zone of the array of memory cells may include eight or more memory cell banks.

[0062] In some examples, the memory device may further include an interface configured for bidirectional communication with a host device. In some examples of the memory device, the interface may be configured to communicate a modulated signal using at least one of the NRZ modulation scheme or the PAM4 scheme, or both.

[0063] In one example, the memory device may include a zone each comprising a plurality of banks of memory cells, an I / O area extending across an array of memory cells—the I / O area may include a plurality of terminals configured to route signals to the array of memory cells—and an array of memory cells having a plurality of channels located in the I / O area of ​​the array of memory cells, each channel may be coupled to a zone of the array of memory cells and configured to communicate signals between a host device and a plurality of banks of memory cells in the zone.

[0064] In some examples, the memory device may further include a plurality of channel interfaces located in the I / O region of an array of memory cells, and a signal path combines the region with the plurality of channel interfaces. In some examples of the memory device, the I / O region may include a TSV configured to combine a second array of memory cells stacked on top of the array of memory cells with the channel interfaces.

[0065] In some examples of memory devices, a channel interface of a zone may be located within an I / O region that divides the zone served by the channel interface. In some examples of memory devices, the I / O region may include a TSV configured to connect an array of memory cells to a power node or a ground node. In some examples of memory devices, the I / O region may occupy a region of an array of memory cells where there may be no memory cells. In some examples of memory devices, an array of memory cells may be divided by two I / O regions. In some examples of memory devices, an array of memory cells may be divided by four I / O regions.

[0066] In one example, the system may include a host device, a memory device including a memory die having a plurality of zones each having a plurality of memory cell banks, and a plurality of channels configured to communicately combine the host device and the memory device, each channel may be combined with a zone of the memory die and configured to communicate signals between the host device and a plurality of memory cell banks within the zone.

[0067] In some examples, the system may include an interface configured for bidirectional communication with a host device. In some examples of the system, the interface may be configured to communicate a modulated signal using at least one of NRZ modulation or PAM4, or both. In some examples of the system, the host device may be an example of a GPU. In some examples of the system, the memory device may be located in the same package as the host device.

[0068] In one example, the memory device may include an array of memory cells having a plurality of zones each comprising a plurality of banks of memory cells and a plurality of channels crossing the array of memory cells, each channel may be coupled to at least one zone of the array of memory cells, and each channel may include two or more data pins and one or more command / address pins.

[0069] In some examples of memory devices, each channel may include two data pins. In some examples of memory devices, each channel may include one command / address pin. In some examples of memory devices, each section of the array may include four memory cell banks. In some examples of memory devices, each channel may include four data pins. In some examples of memory devices, each channel may include two command / address pins. In some examples of memory devices, each section of the array may include eight memory cell banks. In some examples of memory devices, each bank of memory cells may be adjacent to a channel.

[0070] In some examples of memory devices, a first set of each plurality of banks may be adjacent to a channel, and a second set of each plurality of banks may be adjacent to another bank and not adjacent to a channel. In some examples, the memory device may include 128 data pins and may be configured at a ratio of 2, 4, or 8 data pins per channel.

[0071] In some examples, the memory device may include 1, 2, 3, 4, or 6 command / address pins per channel. In some examples, the memory device may include 256 data pins and may be configured in a ratio of 2, 4, or 8 data pins per channel. In some examples, the memory device may include 1, 2, 3, 4, or 6 command / address pins per channel. In some examples of the memory device, the array may include multiple memory dies, each of which may include multiple channels.

[0072] In some examples of memory devices, each of a plurality of memory dies may be coupled to a different channel among a plurality of channels. In some examples, the memory device may include a buffer layer coupled to an array. In some examples, the memory device may include an array-substrate organic substrate.

[0073] In some examples of memory devices, the array may be configured for a pin rate of 10, 16, 20, or 24 Gbps. In some examples, the memory device may include an interface configured for bidirectional communication with a host device. In some examples of memory devices, the interface may be configured for at least one of binary modulation signaling or pulse amplitude modulation, or both.

[0074] In one example, the system may include at least one memory die having a plurality of zones, each having a plurality of banks of memory cells, one or more channels associated with each memory die—each channel may be coupled to at least one zone of the memory cell die and each channel may have at least two or more data pins in at least one zone—and an organic substrate below the memory die.

[0075] In some examples, the system may include a host device and an interface configured for bidirectional communication with the host device, and the interface supports at least one of NRZ signaling or PAM4, or both. In some examples of the system, the host device may include a GPU.

[0076] In some examples, the system may include a plurality of memory arrays, each of which may include 128 or 256 data pins and configured at a ratio of 2, 4, or 8 data pins per channel. In some examples, the system may include a buffer layer located between at least one memory die and an organic substrate.

[0077] The information and signals described in this application may be represented using any of the various different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented as voltage, current, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof. While some drawings may illustrate a signal as a single signal; however, those skilled in the art will understand that a signal may represent a bus of signals, wherein the bus may have various bit widths.

[0078] As may be used in this application, the term “virtual ground” refers to a node in an electrical circuit that is maintained at a voltage of approximately 0V (0V) but is not directly connected to ground. Thus, the voltage of the virtual ground may fluctuate temporarily and then return to approximately 0V in a steady state. A virtual ground can be implemented using various electronic circuit elements, such as a voltage divider composed of an operational amplifier and a resistor. Other implementations are also possible. “Virtually grounded” or “virtually grounded” means connected to approximately 0V.

[0079] The terms “electronic communication” and “coupled” as used in this application refer to a relationship between components that supports the flow of electrons between components. This may include a direct connection between components or may include an intermediate component. The electronic communication or coupled components may actively exchange electrons or signals (e.g., in an active circuit), or may not actively exchange electrons or signals (e.g., in an inactive circuit), but may be configured and operated to exchange electrons or signals when the circuit is active. For example, two components physically connected via a switch (e.g., a transistor) may electronically communicate or be coupled regardless of the state of the switch (i.e., open or closed).

[0080] As used in this application, the term “layer” refers to a stratum or sheet of geometric structure. Each layer may have three dimensions (e.g., height, width, and depth) and may cover part or all of the surface. For example, a layer may be a three-dimensional structure in which two dimensions are greater than the third dimension, e.g., a thin film. A layer may include other elements, components, and / or materials. In some cases, a single layer may consist of two or more sublayers. In some of the accompanying drawings, two of the three dimensions of a layer are illustrated for illustrative purposes. However, those skilled in the art will recognize that a layer is inherently three-dimensional.

[0081] As used in this application, the term "electrode" may refer to an electrical conductor and, in some cases, may be used as an electrical contact to a memory cell or other component of a memory array. The electrode may include traces, wires, conductive lines, conductive layers, etc., that provide a conductive path between elements or components of a memory array.

[0082] The term "isolated" refers to a relationship between components where electrons cannot currently flow between them; if there is an open circuit between the components, the components are isolated from each other. For example, two components physically connected by a switch can be isolated from each other when the switch is open.

[0083] The device described in this application, comprising a memory array, may be formed on a semiconductor substrate such as silicon, germanium, silicon-germanium alloy, gallium arsenide, gallium nitride, etc. In some cases, the substrate is a semiconductor wafer. In other cases, the substrate may be a silicon-on-insulator (SOI) substrate such as silicon-on-glass (SOG) or silicon-on-sapphire (SOP), or an epitaxial layer of a semiconductor material on another substrate. In some examples, the substrate may be an organic build-up substrate formed from a material such as ABF or BT. The conductivity of the substrate, or the subregions of the substrate, may be controlled by doping using various chemical species, including but not limited to phosphorus, boron, or arsenic. Doping may be performed by ion implantation or by any other doping means during the initial formation or growth of the substrate.

[0084] The transistor or transistors described in this application may represent a field-effect transistor (FET) and may include a three-terminal device comprising a source, a drain, and a gate. The terminals may be connected to other electronic components through a conductive material, such as metal, for example, metal. The source and drain may be conductive and may include a high-concentration doped, for example, degenerate semiconductor region. The source and drain may be separated by a low-concentration doped semiconductor region or a channel. If the channel is n-type (i.e., the majority carriers are electrons), the FET may be referred to as an n-type FET. If the channel is p-type (i.e., the majority carriers are holes), the FET may be referred to as a p-type FET. The channel may be covered with an insulating gate oxide. Channel conductivity may be controlled by applying a voltage to the gate. For example, applying a positive voltage or a negative voltage to an n-type FET or a p-type FET, respectively, may make the channel conductive. When a voltage greater than or equal to the transistor's threshold voltage is applied to the transistor gate, the transistor can be turned "on" or "activated". When a voltage less than the transistor's threshold voltage is applied to the transistor gate, the transistor can be turned "off" or "deactivated".

[0085] The various exemplary blocks and modules described in connection with the disclosure of this application may be implemented or performed by a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described in this application. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any processor, controller, microcontroller, or state machine.

[0086] The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors with a DSP core, or any other such configuration).

[0087] In Example 1, the first example, the storage system may include a stack of first memory devices configured to store data, and the stack includes a plurality of memory dies and logic dies of a first storage type. The logic die may include a second memory of a second storage type and an interface circuit. The interface circuit may be configured to receive a plurality of memory requests from an external host using an external bus, relay data between the external host and a plurality of channels of the stack of first memory devices through the second memory, provide an indication of a data error on a first output bit of a corresponding channel during a single clock cycle among a plurality of clock cycles used to exchange data with a corresponding channel of the stack of first memory devices in response to a first individual memory request among the plurality of memory requests, provide an indication of data readiness associated with a second individual memory request among the plurality of memory requests, and provide a first index configured to identify data associated with the second individual memory request.

[0088] In Example 2, the second memory of Example 1 optionally includes a list configured to store multiple indices including the first index.

[0089] In Example 3, any one or more of the data preparation indications from Examples 1-2 are optionally encoded on the first output bit for a number of clock cycles.

[0090] In Example 4, any one or more indications of data preparation in Examples 1-3 optionally include an indication that data is not prepared.

[0091] In Example 5, any one or more of the indications of data preparation in Examples 1-4 optionally include an indication that data is prepared.

[0092] In Example 6, any one or more first indices from Examples 1-5 are optionally encoded on the first output bit for a number of clock cycles.

[0093] In Example 7, any one or more of the first storage types in Examples 1-6 are optionally dynamic random access memory (DRAM).

[0094] In Example 8, any one or more of the second storage types in Examples 1-7 are optionally static random access memory (SRAM).

[0095] In Example 9, a method for operating a memory package having a logic die and a stack of memory devices is provided, wherein the memory package may be configured to communicate with a host using a plurality of independent channels. The method may include receiving a plurality of memory access requests for a channel on the logic die; returning a first data to the host in response to a first memory access request among the plurality of memory access requests; returning a second memory access request among the plurality of memory access requests for a second data to the host indicating that the data is not ready; returning a first index to the host along with a third data in response to a third memory access request among the plurality of memory access requests; returning a first index along with a data ready; and returning the second data to the host in response to the first index and the data ready returned along with the third memory access request.

[0096] In Example 10, the step of returning an indication that data is not ready for any one or more of Examples 1-9 optionally includes the step of encoding an indication that data is not ready on a single output bit of the channel during a first single cycle of a burst of cycles used to exchange data of each of the multiple memory access requests with the host.

[0097] In Example 11, the step of returning an indication that any one or more of the data in Examples 1-10 is ready includes optionally encoding an indication that the data is ready on a single output bit of the channel during a second single cycle of the burst of cycles.

[0098] In Example 12, the step of returning any one or more first indices of Examples 1-11 optionally includes the step of encoding the first index on a single output bit of the channel during multiple cycles of a burst of cycles.

[0099] In Example 13, any one or more of the cycles in Examples 1-12 do not optionally include a first single cycle or a second single cycle.

[0100] In Example 14, any one or more of the methods of Examples 1-13 optionally include the step of returning a data error (DERR) indication when the memory package is unable to service a memory access request among a number of memory access requests, and said DERR indication is encoded on a single output bit.

[0101] In Example 15, a method for operating a host configured to exchange information using a plurality of independent channels of a memory package including a stack of memory devices may include: sending a plurality of memory access requests to the memory package using a single channel; receiving a first data through the single channel in response to a first memory access request among the plurality of memory access requests; receiving an indication that data is not ready through the single channel in response to a second memory access request among the plurality of memory access requests for second data; receiving a first index along with an indication that data is not ready; receiving an indication that data is ready through the single channel along with third data in response to a third memory access request among the plurality of memory access requests; receiving the first index along with an indication that data is ready; and receiving the second data through the single channel in response to the first index and the indication that data is ready received in response to the third memory access request.

[0102] In Example 16, the step of receiving an indication that any one or more of the data in Examples 1-15 is not ready optionally includes the step of decoding the indication that the data in the data in the data on a single output bit of a single channel during a first single cycle of a burst of cycles, and the burst of cycles is used to exchange data of each of a plurality of memory access requests between a host and a memory package.

[0103] In Example 17, the step of receiving an indication that any one or more of the data in Examples 1-16 is ready includes optionally decoding an indication that the data on a single output bit of a single channel is ready during a second single cycle of the burst of cycles.

[0104] In Example 18, the step of receiving the first index of any one or more of Examples 1-17 optionally includes the step of decoding the first index on a single output bit of a single channel during multiple cycles of a burst of cycles.

[0105] In Example 19, any one or more of the cycles in Examples 1-18 do not optionally include a first single cycle or a second single cycle.

[0106] In Example 20, any one or more of the methods in Examples 1-11 optionally include the step of receiving a data error (DERR) indication when the memory package is unable to service a memory access request among a plurality of memory access requests, and the DERR indication is encoded on a single output bit.

[0107] Example 21 may include or use any part or combination of any part of any one or more of Examples 1 through 20, or optionally may be combined with, for the purpose of including or using a machine-readable medium that includes a subject that may include means for performing any one or more of the functions of Examples 1 through 20, or instructions that cause a machine to perform any one or more of the functions of Examples 1 through 20 when performed by a machine.

Claims

Claim 1 In a storage system, a stack of first memory devices configured to store data, wherein the stack comprises a plurality of memory dies of a first storage type; and a logic die comprising a second memory of a second storage type; A storage system comprising an interface circuit, wherein the interface circuit receives a plurality of memory requests from an external host using an external bus, relays data between the external host and a plurality of channels of a stack of the first memory devices through the second memory, provides an indication of a data error on a first output data line of a corresponding channel during a single clock cycle among a plurality of clock cycles used to exchange data with a corresponding channel of the stack of the first memory devices in response to a first individual memory request among the plurality of memory requests, provides a first index on the first output data line during other clock cycles among the plurality of clock cycles used to exchange data, provides an indication of readiness of the data requested by the first individual memory request together with the data requested by the second individual memory request among the plurality of memory requests, provides the first index, and is configured to provide the data requested by the first individual memory request. Claim 2 A storage system according to claim 1, wherein the second memory includes a list that stores a plurality of indices including the first index. Claim 3 A storage system according to claim 1, wherein the indication of the data preparation is encoded on the first output data line during the plurality of clock cycles. Claim 4 In paragraph 3, the storage system wherein the indication of data preparation includes an indication that data is not prepared. Claim 5 In paragraph 3, the above-mentioned indication of data preparation includes an indication that data is prepared, a storage system. Claim 6 delete Claim 7 A storage system according to claim 1, wherein the first storage type is a DRAM (Dynamic Random Access Memory). Claim 8 A storage system according to claim 1, wherein the second storage type is SRAM (static random access memory). Claim 9 A method for operating a memory package having a stack of logic dies and memory devices, wherein the memory package is configured to communicate with a host using a plurality of independent channels, and the method comprises: receiving a plurality of memory access requests for a channel on the logic die; returning a first data to the host in response to a first memory access request among the plurality of memory access requests; returning a second memory access request among the plurality of memory access requests for a second data to the host indicating that the data is not ready; returning a first index to the host along with a third data in response to a third memory access request among the plurality of memory access requests; returning a first index along with a data ready; and returning the second data to the host in response to the first index returned along with the third data and the second data ready. Claim 10 In claim 9, the step of returning an indication that data is not ready comprises encoding an indication that data is not ready on a single output bit of the channel during a first single cycle of a burst of cycles used to exchange data of an individual memory access request among the plurality of memory access requests with the host. Claim 11 In claim 10, the step of returning an indication that data is ready comprises encoding the indication that data is ready on the single output bit of the channel during a second single cycle of the burst of the cycles. Claim 12 A method according to claim 11, wherein the step of returning the first index comprises encoding the first index on the single output bit of the channel during a plurality of cycles of the burst of the cycles. Claim 13 A method according to claim 12, wherein the plurality of cycles does not include the first single cycle or the second single cycle. Claim 14 A method according to claim 10, comprising the step of returning a data error (DERR) indication when the memory package is unable to service a memory access request among the plurality of memory access requests, wherein the DERR indication is encoded on the single output bit. Claim 15 A method of operation of a host configured to exchange information using a plurality of independent channels of a memory package including a stack of memory devices, comprising: sending a plurality of memory access requests to the memory package using a single channel; receiving a first data through the single channel in response to a first memory access request among the plurality of memory access requests; receiving an indication that data is not ready through the single channel in response to a second memory access request among the plurality of memory access requests for second data; receiving a first index along with an indication that data is not ready; receiving an indication that the second data is ready along with third data through the single channel in response to a third memory access request among the plurality of memory access requests; receiving the first index along with an indication that data is ready; and receiving the second data through the single channel in response to the first index received along with the third data and the indication that the second data is ready. Claim 16 In paragraph 15, the step of receiving an indication that data is not ready comprises decoding the indication that data is not ready on a single output bit of the single channel during a first single cycle of a burst of cycles, and the burst of cycles is used to exchange data of individual memory access requests among the plurality of memory access requests between the host and the memory package. Claim 17 In claim 16, the step of receiving an indication that data is ready comprises decoding an indication that data is ready on the single output bit of the single channel during a second single cycle of the burst of the cycles. Claim 18 In claim 17, the step of receiving the first index comprises decoding the first index on the single output bit of the single channel during a plurality of cycles of the burst of the cycles. Claim 19 A method according to paragraph 18, wherein the plurality of cycles does not include the first single cycle or the second single cycle. Claim 20 A method according to claim 16, comprising the step of receiving a data error (DERR) indication when the memory package is unable to service a memory access request among the plurality of memory access requests, wherein the DERR indication is encoded on the single output bit.

Images