Techniques for non-deterministic operations for stacked memory systems

Abstract

Techniques are provided for non-deterministic operations for stacked memory systems. In an example, a method of operating a memory package can include receiving, at a logic die, a plurality of memory access requests for a channel, returning first data to a host in response to a first memory access request of the plurality of memory access requests, returning an indication of data not ready to the host in response to a second memory access request of the plurality of memory access requests for second data, returning a first index and the indication of data not ready to the host, returning an indication of data ready and third data in response to a third memory access request of the plurality of memory access requests, and returning the first index and the indication of data ready.

Description

[0001] Priority and Related Applications

[0002] This application claims priority to Pawlowski U.S. Provisional Patent Application No. 62 / 953,821, filed December 26, 2019, entitled “TECHNIQUES FOR NON-DETERMINISTIC OPERATION OF A STACKED MEMORY SYSTEM,” which is hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003] The following relates generally to operating memory arrays, and more specifically to increasing bandwidth of stacked memory devices. BACKGROUND

[0004] Memory devices are widely used to store information in various electronic devices such as computers, wireless communication devices, cameras, digital displays, and the like. Information is stored by programming different states of the memory device. For example, binary devices have two states typically indicated by a logical “1” or a logical “0.” In other systems, more than two states can be stored. To access stored information, components of an electronic device can read or sense the stored states in the memory device. To store information, components of an electronic device can write or program states in the memory device.

[0005] There are a variety of types of memory devices, including magnetic hard disks, random access memory (RAM), read-only memory (ROM), DRAM, synchronous dynamic RAM (SDRAM), ferroelectric RAM (FeRAM), magnetic RAM (MRAM), resistive RAM (RRAM), flash memory, phase change memory (PCM), and the like. Memory devices can be volatile or non-volatile.

[0006] Generally, improving memory devices can include increasing memory cell density, improving read / write speeds, enhancing reliability, enhancing data retention, reducing power consumption, or reducing manufacturing costs, among other metrics. Improved memory technology has achieved improvements in many of these metrics, however, as processing speeds increase, memory bandwidth can become a bottleneck to overall system performance improvements. BRIEF DESCRIPTION OF DRAWINGS

[0007] In the drawings, which are not necessarily drawn to scale, like numerals describe similar components throughout the several views. Like numerals having different letter suffixes can represent different instances of the like components. The drawings illustrate generally, by way of example, various embodiments discussed in the present document.

[0008] Figure 1 An example of a memory die that supports features and operations according to examples of the disclosure is shown.

[0009] Figure 2 An example of an apparatus that supports features and operations according to examples of the disclosure is shown generally.

[0010] Figure 3 An example storage system including a host device that can request and receive information from the storage system according to the inventive subject matter is shown generally.

[0011] Figure 4 An example timeline of information flow between a host and a channel of an example memory package according to the inventive subject matter is shown generally.

[0012] Figure 5 An example method of operating a memory device according to the inventive subject matter is shown generally.

[0013] Figure 6 An example method of operating a host according to the inventive subject matter is shown generally.

[0014] Figure 7 A diagram of a system including an apparatus that supports a storage system including stacked DRAM devices according to aspects disclosed herein is shown generally. DETAILED DESCRIPTION

[0015] Techniques for non-deterministic operations for stacked memory systems are provided. In an example, a method of operating a memory package can include receiving, at a logic die, a plurality or a number of memory access requests for a channel; returning, to a host, first data in response to a first memory access request of the plurality of memory access requests; returning, to the host, an indication that data is not ready in response to a second memory access request of the plurality of memory access requests for second data; returning, to the host, a first index and the indication that data is not ready; returning, to the host, an indication that data is ready and third data in response to a third memory access request of the plurality of memory access requests; and returning, to the host, the first index and the indication that data is ready.

[0016] Figure 1 is a schematic diagram of an example memory package 110 that includes an interface (IF) chip 106 or circuit and a plurality of core chips 100. For example, the memory package 100 can be a 3D memory device such as a high bandwidth memory (HBM), a hybrid memory cube (HMC), a Wide-IO DRAM, etc. The memory package 110 is formed by stacking the chips vertically, as shown in Figure 1The illustrated stacked chip can include two stacks 12 and 13, each assigned a stack ID of "0" and "1," respectively. Each stack 12 and 13 can include core chips 12a-d and 13a-d, respectively. In certain examples, each stack can have a number of multi-lane channels per chip. In certain examples, each multi-lane channel can include at least 128 bits for a width of 1024 bits or more across eight lanes. An interface (IF) chip 11 of the memory package 110 can provide an interface to the multi-lane input / output channels. In certain examples, each lane can function independently of each other between the core chips 12a-d and 13a-d and a host device (not shown), which can be a memory controller. The IF chip 106 can couple each lane to the host device via a number of data queues (DQs). Each lane can include a number of memory cells and circuitry to access the memory cells. For example, the memory cells can be DRAM memory cells.

[0017] Figure 2 An apparatus or system 290 that supports memory package signal routing is shown in accordance with various examples disclosed herein. The system 290 can include a host device 205 and a plurality of memory packages 210. In conventional systems, the plurality of memory devices are of the same type, such as DRAM memory devices. In certain examples, the memory devices can include a mix of capacitive-based memory devices (e.g., DRAM memory devices) and cross-linked inverting memory devices (e.g., SRAM memory devices). The inventors have recognized that bandwidth improvements can be realized if the host has access to a second, faster type of memory, such as SRAM memory.

[0018] The host device 205 can be an example of a processor (e.g., central processing unit (CPU), graphics processing unit (GPU)) or system on a chip (SoC). In some cases, the host device 205 can be a component separate from the memory devices, such that the host device 205 can be fabricated separately from the memory devices. The host device 205 can be external to the memory devices 210 (e.g., laptop computer, server, personal computing device, smartphone, personal computer). In the system 290, the memory packages 210 can be configured to store data for the host device 205.

[0019] The host device 205 can exchange information with the memory package 210 using signals transmitted on a signal path. The signal path can be a path from a transmitting component to a receiving component where messages or content can be obtained. In some cases, the signal path can be a conductor coupled to at least two components, wherein the conductor may selectively allow electrons to flow between the at least two components. In the case of wireless communication (e.g., radio frequency (RF) or optical), the signal path may be formed in a wireless medium. The signal path may at least partially comprise a first substrate, such as an organic substrate of the memory device; and / or a second substrate, such as a packaging substrate (e.g., a second organic substrate), which may be coupled to at least one of the memory device 210 and the host device 205 (in the case of not being coupled to both the memory device and the host device). In some cases, the memory package 210 may be used as a slave device to the host device 205, which may be used as a master device.

[0020] In some applications, system 290 can benefit from a high-speed connection between host device 205 and memory device 210. Therefore, some memory packages 210 support applications, processors, host devices, or processors requiring bandwidth of several megabytes per second (TB / s). Meeting such bandwidth constraints within an acceptable energy budget can be challenging in some cases.

[0021] The memory die 200 of the memory package 210 can be configured to work with various types of communication media 211 (e.g., substrates such as organic substrates and / or high-density interposers such as silicon interposers). In some cases, the host device 205 may be configured with an interface or ball-out that includes a termination design (e.g., a matrix or pattern).

[0022] In some cases, a buffer layer may be located between memory die 200 and communication medium 211. The buffer layer may be configured to drive (e.g., redrive) signals to and from memory die 200. In some cases, the stack of memory dies 200 may be unbuffered, meaning there is no buffer layer or the base layer does not contain redrives or other components. In some instances of unbuffered memory, a routing layer or logic die 206 may be located between memory die 200 or the stack of memory dies 200 and communication medium 211. In some instances, logic die 206 may form the lower layer of memory die 200. In some instances, unbuffered memory package 210 may include a lowermost memory die 200 with logic die layer 206.

[0023] Figure 3An example storage system 391 comprising a host device 305 and a memory package 310 is generally illustrated. The host 305 can request and receive information from the memory package 310 according to the subject matter of the invention using a bus external to the memory package. The host device 305 may be, but is not limited to, a CPU, a graphics processing unit (GPU), an accelerated processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and any other component of a larger system communicating with the storage system 310. In some embodiments, the device 305 may be multiple devices accessing the same storage system 310. The memory package 310 may include a logic die 306 integrated with a memory stack 320, such as a stack of dynamic random access memory (DRAM) devices.

[0024] The logic die 306 may include a host interface 331 connected to the stacked DRAM controller 332 and the prefetch and cache logic 333. The stacked DRAM controller 332 is connected to and interfaces with the memory stack 320. The prefetch and cache logic 333 may be connected to a prefetcher, a prefetch buffer, and a cache array 334. The prefetcher may be a hardware prefetcher. The prefetch buffer and cache array 334 may be, but is not limited to, an SRAM array or any other memory array technique, or a register with a faster access speed than the type of memory used in the memory stack 320.

[0025] Host interface 331 may include command decoder 335 and interface register 336. Host interface 331, and more specifically command decoder 335, may receive all incoming memory requests from memory stack 320 from host 305. Requests may be sent to prefetch and cache logic 333 (e.g., next line, step, etc.). Prefetch and cache logic 333 may monitor incoming memory requests. Prefetched data may be placed in prefetch buffer and cache array 334. Prefetch and cache logic 333 may also check any incoming memory requests against the data in prefetch buffer and cache array 334. Any hits may be provided directly from prefetch buffer and cache array 334 without going to stacked DRAM controller 332. This can reduce the service latency of these requests, as well as reduce contention for any remaining requests in stacked DRAM controller 332 (i.e., requests that do not hit prefetch buffer and cache array 334).

[0026] A prefetcher can contain any prefetch algorithm / method or combination of algorithms / methods. Due to the row-buffer-based organization of most memory technologies (e.g., DRAM), prefetch algorithms utilizing spatial positioning (e.g., next row, small stride, etc.) have relatively low overhead because prefetch requests (likely) will hit the memory's row buffer. Implementations may issue prefetch requests for larger data blocks (i.e., more than one 64B cache line), such as prefetching the entire row buffer, half of the row buffer, or other granularities.

[0027] The prefetch buffer and cache array 334 can be implemented as a directly mapped, set-associative, or fully associative cache class structure. In one embodiment, the prefetch buffer and cache array 334 can be used to serve only read requests (i.e., writes that invalidate prefetch buffer entries or require a write-through policy). In another embodiment, the prefetch buffer and cache array 334 can employ replacement policies such as Least Recently Used (LRU), Least Frequent Used (LFU), or First-In-First-Out (FIFO). If a prefetch unit generates a request for data larger than the cache line size, the prefetch buffer and cache array 334 may also need to be organized with a correspondingly wider block size. In some embodiments, sub-blocks can be used.

[0028] Although described herein as being used in a memory organization consisting of a logic chip and one or more memory chips, other physical manifestations exist. While described as a vertical stack of logic dies with one or more memory chips, another embodiment may place some or all of the logic on a single chip, which is horizontally positioned on an insert or packaged together in a multi-chip module (MCM). More than one logic chip may be included in the overall stack or system.

[0029] As discussed above, the prefetch and cache logic 333 can also inspect any incoming memory requests for data in the prefetch buffer and cache array 334. Any hit can be served directly from the prefetch buffer and cache array 334 without going to the stacked DRAM controller 332. This can reduce the service latency of these requests, as well as reduce contention for any remaining requests in the stacked DRAM controller 332. However, prefetch or cache misses can still occur. When a request cannot be served, a conventional architecture provides a Data Error (DERR) indication at the host interface for each channel of the memory stack. When, for example, a host read request fails to hit data associated with the prefetch or cache, the host interface indicates a cache miss by setting the Data Error (DERR) indication for the corresponding channel to the appropriate logic level. After seeing the data error indication, the host typically moves on to the next access request and then reissues the failed request at a later time.

[0030] The inventors have recognized that a logical die having a memory-side SRAM cache as described above can be used to assist and improve access to memory data after a miss access request. In this context, a "miss access request" is a request received by the memory interface but not yet processed from the cache. In some instances, system 391 can utilize the cluster length of the memory system to encode additional information associated with the miss access request. Generally, the cluster length is the number of clock cycles the channel uses to exchange information across data queues (DQs). (The above text regarding...) Figure 1 In the examples discussed, a channel width of 32 might require eight clock cycles to provide a 128-bit channel length word, thus the burst length is eight. In some instances, each cycle of the burst length can use the Physical Data Error (DERR) bit of each channel I / O to convey 8 bits of information, such as actual data errors, whether a previous request is ready, and an index associated with a missed request. Eight bits can be serially transmitted during bursting using the Physical Data Error I / O bit of each channel I / O. For example, the serial transmission bits of the Physical Data Error I / O point (DERR) can be allocated as follows:

[0031] Bit function

[0032] 0 data errors

[0033] 1 Not ready

[0034] 2 Warnings

[0035] 3-7 Index

[0036] It should be understood that other bit allocation sequences are possible without departing from the scope of the subject matter of this invention. The "data error" function differs slightly from conventional methods in that the data error bit can be activated when a request cannot be served and will not be served. In this case, when the host receives a "data error" indication after issuing a memory access request, it needs to reissue the memory access request at least once.

[0037] When a request is a miss request, a "Not Ready" feature or bit can be activated. For example, when a request is received, the "Not Ready" feature / bit can be activated, indicating that the request cannot be serviced by the cache, but the interface plans to attempt to read the data associated with the miss request into the cache in the near future. In addition to activating the "Not Ready" feature / bit, index bits (e.g., 3-7) can provide an index number associated with the miss request on the channel. The host can use the index number to track and later capture the corresponding data associated with the previous miss request.

[0038] The "Warning" function or bit can be activated when the data associated with a previous missed request becomes available in the cache. Additionally, when the Warning function / bit is activated, the index bit provides the index number corresponding to the missed request. In some instances, the host interface can provide data in the interval immediately following the activation of the corresponding Warning function. In one instance, the Warning function can re-initiate the missed read access request associated with the index to obtain the corresponding data. In some instances, the cache may contain a list of index numbers associated with pending requests.

[0039] In the example above, the cluster length allows for five index bits. Therefore, each channel can have at most 32 (e.g., 2) at any given time. 5 There are 10 pending missed requests. In some instances, index numbers can be assigned to convey other information, thus reducing the number of pending missed requests. Additionally, without departing from the scope of the invention, the example memory system may have a different cluster length or channel width than the examples described above.

[0040] Figure 4 An example timeline 400 of the information flow between a host 305 and an example memory package 310, according to the subject matter of the invention, is generally shown. The memory package 310 may include a logic die 306 and a memory stack 320, such as a DRAM stack. The logic die 306 may include a host interface 331 and a cache memory 337. In some instances, the cache memory 337 may include SRAM. In some instances, the cache memory 337 may include buffers and tag mappings associated with conventional stacked memory devices. At 401, the host 305 may request data from the memory system, and the request may be received at the host interface 331. At 402, the host interface 331 may request data from the cache memory 337. At 403, after determining that data is in the cache memory 337, data may be received at the host interface 331, and at 404, the data may be passed to the host 305. At 405, host 305 may request second data from the memory system, and the request may be received at host interface 331. At 406, host interface 331 may request data from cache memory 337. At 407, after determining that the data is not in cache memory 337, the status of the cache request may be received at host interface 331. At 408, host interface 331 may request second data from memory stack 320, and at 409, a miss request may be reported to host 305.

[0041] In some instances, at 409, when a request is reported as not ready, host interface 331 can use the channel's Physical Data Error (DERR) output point to indicate that the request is not ready, and also provides the index number of the request. Compatible host 305 can use the index number to retrieve the requested data later, as discussed below. In some instances, because accessing memory stack 320 is more time-consuming than servicing a data request at cache memory 337, host interface 331 can make the channel available for further data requests while second data is being retrieved. Therefore, at 410, host 305 can request third data from the memory system, and this request can be received at host interface 331. At 411, host interface 331 can request data from cache memory 337. At 412, after determining that data is in cache memory 337, data can be received at host interface 331, and at 413, the data can be passed to host 305. At 414, second data can be passed from memory stack 320 to cache memory 337. It should be understood that, without departing from the scope of the subject matter of this invention, host interface 331 is capable of receiving more than one data request from host 305 before receiving data for a missed request.

[0042] At 416, host 305 may request fourth data from the memory system, and the request may be received at host interface 331. At 417, host interface 331 may request data from cache memory 337. At 418, after determining that the data is not in cache memory 337, the status of the cache request may be received at host interface 331. At 419, host interface 331 may request fourth data from memory stack 320, and at 420, a miss request may be reported to host 305. As previously described, in some instances, at 420, when a miss request is reported, host interface 331 may use the channel's Physical Data Error (DERR) output point to indicate that the request is not ready, and also provide the index number of the request. The index number may be different from the index number associated with the miss request that reports second data at 409.

[0043] At 421, host 305 may request fifth data from the memory system, and the request may be received at host interface 331. At 422, host interface 331 may request data from cache memory 337. At 423, after determining that the data is in cache memory 337, the data may be received at host interface 331, and at 424, the data may be delivered to host 305. Additionally, when the fifth data is delivered to host 305, host interface 331 may activate a "data" ready bit or the "alert" bit discussed above during transmission, and may also provide a corresponding index number for the request for second data, since the second data is now available in cache memory 337. In some instances, at 425, cache memory 337 may deliver the second data to host 305 via host interface 331. In some instances, host interface 331 may request host 305 to resend the request for the second data before delivering it. In some instances, the transfer of second data via host interface 331 may involve separate transactions between cache memory 337 and host interface 331, and between the interface of host 305 and host 305, and each transaction may be separated at a time different from the time shown without departing from the scope of the subject matter of the invention. At 426, fourth data may be transferred from memory stack 320 to cache memory 337.

[0044] At 427, host 305 may request sixth data from the memory system, and the request may be received at host interface 331. At 428, host interface 331 may request sixth data from cache memory 337. At 429, after determining that the sixth data is in cache memory 337, the sixth data may be received at host interface 331, and at 430, the sixth data may be passed to host 305. Additionally, when the sixth data is passed to host 305, host interface 331 may activate a data ready indication, such as a "data ready" bit, during DERR generation, and may also provide a corresponding index number for the request for fourth data, since the fourth data is now available in cache memory 337. In some instances, at 431, cache memory 337 may pass the fourth data to host 305 via host interface 331. In some instances, host interface 331 may request host 305 to resend the request for fourth data before passing the fourth data from cache memory 337. In some instances, the transmission of fourth data via host interface 331 may involve separate transactions between cache memory 337 and host interface 331, and between host interface 331 and host 305, and each transaction may be separated at a time different from the time shown without departing from the scope of the subject matter of this invention.

[0045] Figure 5An example method 500 for operating a memory device according to the subject matter of the present invention is generally illustrated. At 501, a plurality of memory access requests may be received from a host at a memory system comprising a stack of DRAM memory devices. At 503, first data may be returned to the host in response to a first memory access request among the plurality of memory access requests. At 505, a first indication that data is not ready may be returned to the host in response to a second memory access request among the plurality of memory access requests. At 507, an index and the first indication that data is not ready may be returned. At 509, an indication that data is ready and third data may be returned in response to a third memory access request among the plurality of memory access requests. At 511, a first index and the indication that data is ready may be returned. At 513, second data may be returned to the host in response to the indication that data is ready received using the third memory access request and the first index.

[0046] Figure 6 An example method 600 of an operating host according to the subject matter of the present invention is illustrated. At 601, a plurality of memory access requests may be sent to a memory system having a stack of memory devices. At 603, first data may be received in response to a first request among the plurality of memory access requests. At 605, an indication that data is not ready may be received in response to a second request among the plurality of memory access requests, wherein the second request is a request for second data. At 607, an index and the indication that data is not ready may be received. At 609, third data may be received in response to a third request among the plurality of memory access requests. At 611, an indication that data is ready and an index, as well as the third data, may be received. At 613, second data may be received in response to the indication that data is ready and the index.

[0047] Figure 7 A schematic diagram of a system 700 including a device 705 supporting a memory system including stacked DRAM devices is generally shown according to aspects disclosed herein. Device 705 may include components for bidirectional voice and data communication, components for transmitting and receiving communication, and includes a memory controller 715, memory cells 720, a basic input / output system (BIOS) component 725, a processor 730, an I / O controller 735, peripheral components 740, memory chips 755, a system memory controller 760, an encoder 765, a decoder 770, and a multiplexer 775. These components may communicate electronically via one or more buses (e.g., bus 710). For example, bus 710 may have a bus width of 16 data lines (“DQ” lines). Bus 710 may communicate electronically with 32 banks of memory cells.

[0048] Memory controller 715 or 760 can operate one or more memory cells as described herein. Specifically, the memory controller can be configured to support flexible multichannel memory. In some cases, memory controller 715 or 760 can operate a row decoder, a column decoder, or both, as described in the references. Figure 1 As described, the memory controller 715 or 760 can communicate electronically with a host and can be configured to transmit data during each of the rising and falling edges of the clock signal of the memory controller 715 or 760.

[0049] Memory cell 720 can store information (i.e., in the form of logical states), as described herein. Memory cell 720 can represent, for example, references... Figure 1 The memory cell 105 is described. The memory cell 720 can electronically communicate with a memory controller 715 or 760, and the memory cell 720 and the memory controller 715 or 760 can be located on a chip 755, which can be one or more planar memory devices as described herein. The chip 755 can be managed, for example, by the system memory controller 715 or 760.

[0050] Memory cell 720 may represent a first memory cell array having a plurality of regions coupled to a substrate. Each of the plurality of regions may contain a plurality of memory banks of memory cells and a plurality of channels traversing the first memory cell array. At least one of the plurality of channels may be coupled to at least one region. Memory controller 715 or 760 may be configured to transfer data between the coupled regions and memory controller 715 or 760.

[0051] BIOS component 725 is a software component that includes a BIOS operating as firmware, which can initialize and run various hardware components. BIOS component 725 controls the data flow between the processor and various other components, such as peripheral device components, input / output control components, etc. BIOS component 725 may contain programs or software stored in read-only memory (ROM), flash memory, or any other non-volatile memory.

[0052] Processor 730 may include intelligent hardware devices (e.g., general-purpose processors, digital signal processors (DSPs), central processing units (CPUs), microcontrollers, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices, discrete gate or transistor logic components, discrete hardware components, or any combination thereof). In some cases, processor 730 may be configured to operate a memory array using memory controller 715 or 760. In other cases, memory controller 715 or 760 may be integrated into processor 730. Processor 730 may be configured to execute computer-readable instructions stored in memory to perform various functions (e.g., functions or tasks supporting flexible multi-channel memory).

[0053] The I / O controller 735 manages the input and output signals of the device 705. The I / O controller 735 can also manage peripheral devices not integrated into the device 705. In some cases, the I / O controller 735 may represent a physical connection or port to an external peripheral device. The I / O controller 735 may utilize, for example... The I / O controller 735 may represent, or interact with, a modem, keyboard, mouse, touchscreen, or similar device. In some cases, the I / O controller 735 may be implemented as part of a processor. Users may interact with device 705 via the I / O controller 735 or via hardware components controlled by the I / O controller 735.

[0054] Peripheral component 740 may include any input or output device, or an interface for such devices. Examples may include a disk controller, sound controller, graphics controller, Ethernet controller, modem, Universal Serial Bus (USB) controller, serial or parallel port, or peripheral card slot, such as a Peripheral Component Interconnect (PCI) or Accelerated Graphics Port (AGP) slot.

[0055] Input 745 may represent a device or signal external to device 705 that provides input to device 705 or its components. This may include a user interface or an interface with or between other devices. In some cases, input 745 may be managed by I / O controller 735 and may interact with device 705 via peripheral component 740.

[0056] Output 750 may also represent a device or signal external to device 705, configured to receive output from device 705 or any of its components. Examples of output 750 may include a graphic display, audio speaker, printing device, another processor, or printed circuit board, etc. In some cases, output 750 may be a peripheral element that interfaces with device 705 via peripheral component 740. Output 750 may be managed by I / O controller 735.

[0057] System memory controller 715 or 760 may electronically communicate with a first memory cell array (such as memory cell 720). A host may be a component or device that controls or directs the operation of the memory controller 715 or 760 and the corresponding memory array being part of it. The host may be a component of a computer, mobile device, etc. Alternatively, device 705 may be referred to as a host. In some instances, system memory controller 715 or 760 is a GPU.

[0058] Encoder 765 may represent a device or signal external to device 705, providing error correction encoding for data to be stored in device 705 or its components. Encoder 765 may write encoded data to at least one selected memory via at least one channel, and may also encode the data via error correction encoding.

[0059] Decoder 770 can represent devices or signals external to device 705, and it sequences command and address signals for device 705 or its components. In some instances, memory controller 715 or 760 may be co-located within decoder 770.

[0060] Multiplexer 775 may represent a device or signal external to device 705, which multiplexes data to device 705 or its components. Multiplexer 775 can multiplex data to be transmitted to encoder 765 and demultiplex data received from encoder 765. Multiplexer 775 can communicate electronically with decoder 770. In some instances, multiplexer 775 can communicate electronically with controllers such as system memory controller 715 or 760.

[0061] The components of device 705 may include circuitry designed to perform its functions. This may include various circuit elements configured to perform the functions described herein, such as conductive lines, transistors, capacitors, inductors, resistors, amplifiers, or other active or passive components. Device 705 may be a computer, server, laptop computer, notebook computer, tablet computer, mobile phone, wearable electronic device, personal electronic device, etc. Alternatively, device 705 may be a part or aspect of such a device. In some instances, device 705 is an aspect of a computer with high reliability, mission criticality, or low latency constraints or parameters, such as a vehicle (e.g., autonomous vehicle, aircraft, spacecraft, etc.). Device 705 may be or include logic for artificial intelligence (AI), augmented reality (AR), or virtual reality (VR) applications.

[0062] In one example, the memory device may include an array of memory cells having multiple regions having multiple banks of memory that may each contain memory cells, and multiple channels traversing the array of memory cells. Each of the channels may be coupled to a region of the memory cell array and may be configured to transmit signals between multiple banks of memory cells in the region by a host device.

[0063] In some instances, the memory device may further include an I / O region extending across the memory cell array, the I / O region occupying a region of the memory cell array that may not contain memory cells. In some instances of the memory device, the I / O region may include a TSV configured to couple the memory cell array to a power node or a ground node.

[0064] In some instances, the memory device may further include multiple channel interfaces distributed across the memory cell array. In some instances of the memory device, the multiple channel interfaces may be bump-out pins. In some instances of the memory device, the channel interfaces among the multiple channel interfaces may be located in each quadrant of the memory cell array.

[0065] In some instances, the memory device may further include multiple signal paths extending between the memory cells of the region and the channel interface associated with the region. In some instances of the memory device, the channel interface may be located within the memory cell array to minimize the length of the signal paths.

[0066] In some instances, the memory device may further include a second memory cell array stacked on top of the memory cell array. In some instances of the memory device, the second memory cell array may have a region comprising a plurality of memory banks, each containing a memory cell. In some instances, the memory device may further include a second plurality of channels traversing the second memory cell array. In some instances of the memory device, each of the second plurality of channels may be coupled to a second region of the second memory cell array and may be configured to transmit signals between a host device and a plurality of memory banks of memory cells in the second region.

[0067] In some instances, the memory device may further include a TSV extending through the memory cell array to couple a second memory cell array to a second plurality of channels. In some instances of the memory device, the channels may establish a point-to-point connection between the region and a host device. In some instances of the memory device, each channel may include four or eight data pins. In some instances of the memory device, a region of the memory cell array may contain eight or more memory banks of memory cells.

[0068] In some instances, the memory device may further include an interface configured for bidirectional communication with a host device. In some instances of the memory device, the interface may be configured to transmit signals modulated using at least one of an NRZ modulation scheme or a PAM4 modulation scheme, or both.

[0069] In one example, the memory device may include: a memory cell array having regions of multiple banks, each containing memory cells; an I / O region extending across the memory cell array, the I / O region including multiple terminals configured to route signals to and from the memory cell array; and multiple channels located in the I / O region of the memory cell array, each channel being coupled to a region of the memory cell array and configured to transmit signals between a host device and multiple banks of memory cells in the region.

[0070] In some instances, the memory device may further include multiple channel interfaces located in an I / O region of the memory cell array, with signal paths coupling the region to the multiple channel interfaces. In some instances of the memory device, the I / O region may include a TSV configured to couple a second memory cell array stacked on top of the memory cell array to the channel interfaces.

[0071] In some instances of the memory device, the channel interface of the region may be located within an I / O region equally divided by the channel interface. In some instances of the memory device, the I / O region may contain a TSV configured to couple the memory cell array to a power node or ground node. In some instances of the memory device, the I / O region may occupy a region of the memory cell array that may not contain memory cells. In some instances of the memory device, the memory cell array may be equally divided by two I / O regions. In some instances of the memory device, the memory cell array may be equally divided by four I / O regions.

[0072] In one example, the system may include: a host device; a memory device comprising a memory die comprising multiple regions of a plurality of memory banks, each of which may each contain memory cells; and multiple channels configured to communicatively couple the host device and the memory device, each channel being coupled to a region of the memory die and configured to transmit signals between the host device and the plurality of memory banks of memory cells in said region.

[0073] In some instances, the system may include an interface configured for bidirectional communication with a host device. In some instances, the interface may be configured to transmit signals modulated using at least one or both of the NRZ modulation scheme and the PAM4 modulation scheme. In some instances, the host device may be an instance of a GPU. In some instances, the memory device may be located in the same package as the host device.

[0074] In one example, the memory device may include an array of memory cells having multiple regions of multiple banks of memory, each containing memory cells, and multiple channels traversing the memory cell array, each channel being coupled to at least one region of the memory cell array and each channel including two or more data pins and one or more command / address pins.

[0075] In some instances of a memory device, each channel may contain two data pins. In some instances of a memory device, each channel may contain one command / address pin. In some instances of a memory device, each region of the array may contain four banks of memory cells. In some instances of a memory device, each channel may contain four data pins. In some instances of a memory device, each channel may contain two command / address pins. In some instances of a memory device, each region of the array may contain eight banks of memory cells. In some instances of a memory device, each bank of a memory cell may be adjacent to a channel.

[0076] In some instances of the memory device, a first set of memory banks for each plurality of memory cells may be adjacent to a channel, and a second set of memory banks for each plurality of memory cells may be adjacent to another set of memory banks but not adjacent to a channel. In some instances, the memory device may include 128 data pins, configured at a ratio of two, four, or eight data pins per channel.

[0077] In some instances, the memory device may include one, two, three, four, or six command / address pins per channel. In some instances, the memory device may include 256 data pins, configured in a ratio of two, four, or eight data pins per channel. In some instances, the memory device may include one, two, three, four, or six command / address pins per channel. In some instances of the memory device, the array may comprise multiple memory dies, each capable of containing multiple channels.

[0078] In some instances of the memory device, each of the plurality of memory dies may be coupled to a different channel of the plurality of channels. In some instances, the memory device may include a buffer layer coupled to the array. In some instances, the memory device may include an organic substrate located beneath the array.

[0079] In some instances of the memory device, the array may be configured for pin rates of 10Gbps, 16Gbps, 20Gbps, or 24Gbps. In some instances, the memory device may include an interface configured for bidirectional communication with a host device. In some instances of the memory device, the interface may be configured for at least one or both of binary modulation signaling or pulse amplitude modulation.

[0080] In one instance, the system may include: at least one memory die, which may include multiple regions of a plurality of memory banks, each of which may contain memory cells; one or more channels associated with each memory die, each channel being coupled to at least one region of the die containing memory cells and each channel including two or more data pins; and an organic substrate located beneath the memory die.

[0081] In some instances, the system may include a host device and an interface configured for bidirectional communication with the host device, the interface supporting at least one or both of NRZ signaling and PAM4. In some instances of the system, the host device may include a GPU.

[0082] In some instances, the system may include multiple memory arrays, each containing 128 or 256 data pins, configured at a ratio of two, four, or eight data pins per channel. In some instances, the system may include a buffer layer positioned between at least one memory die and an organic substrate.

[0083] The information and signals described herein can be represented using any of a variety of different techniques and methods. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the foregoing description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or light particles, or any combination thereof. Some diagrams may show signals as single signals; however, those skilled in the art will understand that the signals may represent signal buses, where the buses may have various bit widths.

[0084] As may be used herein, the term "virtual ground" refers to a circuit node that is maintained at approximately zero volts (0V) but is not directly connected to ground. Therefore, the voltage of a virtual ground may fluctuate temporarily and return to approximately 0V in a steady state. Virtual grounding can be implemented using various electronic circuit elements, such as a voltage divider consisting of operational amplifiers and resistors. Other implementations are also possible. "Virtual ground" or "virtual ground connection" implies a connection to approximately 0V.

[0085] As may be used herein, the terms “electronic communication” and “coupling” refer to a relationship between components that supports the flow of electrons between them. This may include direct connections between components or may include intermediate components. Components that are electronically communicating or coupled to each other may actively exchange electrons or signals (e.g., in an energized circuit) or may not actively exchange electrons or signals (e.g., in an unenergized circuit), but may be configured and available to exchange electrons or signals after the circuit is energized. As an example, two components physically connected via a switch (e.g., a transistor) may electronically communicate or be coupled regardless of the state of the switch (i.e., open or closed).

[0086] As used herein, the term "layer" refers to a layer or sheet of geometry. Each layer may have three dimensions (e.g., height, width, and depth) and may cover part or all of a surface. For example, a layer may be a three-dimensional structure in which two dimensions are greater than the third, such as a thin film. Layers may contain different elements, components, and / or materials. In some cases, a layer may consist of two or more sublayers. In some figures, two dimensions of a three-dimensional layer are depicted for illustrative purposes. However, those skilled in the art will recognize that layers are inherently three-dimensional.

[0087] As used herein, the term "electrode" can refer to an electrical conductor and, in some cases, can serve as an electrical contact to a memory cell or other component of a memory array. An electrode may comprise traces, wires, conductive lines, conductive layers, etc., that provide a conductive path between elements or components of the memory array.

[0088] The term "isolation" refers to the relationship between components in which electrons are currently unable to flow between them; if there is an open circuit between the components, the components are isolated from each other. For example, two components physically connected by a switch can be isolated from each other when the switch is open.

[0089] The devices comprising memory arrays discussed herein can be formed on semiconductor substrates such as silicon, germanium, silicon-germanium alloys, gallium arsenide, and gallium nitride. In some cases, the substrate is a semiconductor wafer. In others, the substrate can be a silicon-on-insulator (SOI) substrate, such as silicon-on-glass (SOG) or silicon-on-sapphire (SOP), or an epitaxial layer of semiconductor material on another substrate. In some instances, the substrate can be an organic multilayer substrate formed of materials such as ABF or BT. The conductivity of the substrate or subregions of the substrate can be controlled by using doping with various chemical species including, but not limited to, phosphorus, boron, or arsenic. Doping can be performed during the initial formation or growth of the substrate, by ion implantation or by any other doping method.

[0090] The transistor discussed herein may refer to a field-effect transistor (FET) and includes a three-terminal device comprising a source, a drain, and a gate. These terminals can be connected to other electronic components via a conductive material, such as a metal. The source and drain can be conductive and may comprise heavily doped (e.g., degenerate) semiconductor regions. The source and drain can be separated by lightly doped semiconductor regions or a channel. If the channel is n-type (i.e., the majority of charge carriers are electrons), the FET can be called an n-type FET. If the channel is p-type (i.e., the majority of charge carriers are holes), the FET can be called a p-type FET. The channel can be capped by an insulating gate oxide. The channel conductivity can be controlled by applying a voltage to the gate. For example, applying a positive or negative voltage to an n-type FET or a p-type FET, respectively, can make the channel conductive. When a voltage greater than or equal to the transistor's threshold voltage is applied to the transistor's gate, the transistor can be "turned on" or "activated." When a voltage less than the transistor's threshold voltage is applied to the transistor's gate, the transistor can be "turned off" or "deactivated."

[0091] The various illustrative blocks and modules described in connection with this disclosure may be implemented or performed using a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware component, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any processor, controller, microcontroller, or state machine.

[0092] The processor can also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration).

[0093] In a first example, namely Example 1, a storage system may include: a first memory device stack configured to store data, the stack including a plurality of memory dies of a first memory type; and a logic die. The logic die may include a second memory of a second memory type and interface circuitry. The interface circuitry may be configured to: receive a plurality of memory requests from an external host using an external bus; relay data between the external host and a plurality of channels of the first memory device stack via the second memory; in response to a first corresponding memory request among the plurality of memory requests; provide a data error indication on a first output bit of the corresponding channel during a single clock cycle of a plurality of clock cycles for exchanging the data with the corresponding channel of the first memory device stack; provide a data ready indication associated with a second corresponding memory request among the plurality of memory requests; and provide a first index configured to identify data associated with the second corresponding memory request.

[0094] In Example 2, the second memory of Example 1 optionally includes a list configured to store a plurality of indices containing the first index.

[0095] In Example 3, the data ready indication described in any one or more of Examples 1 to 2 is optionally encoded on the first output bit during the plurality of clock cycles.

[0096] In Example 4, any one or more of the data ready indications described in Examples 1 to 3 may optionally include an indication that the data is not ready.

[0097] In Example 5, any one or more of the data ready indications described in Examples 1 to 4 may optionally include an indication that data is ready.

[0098] In Example 6, the first index described in any one or more of Examples 1 to 5 is optionally encoded on the first output bit during the plurality of clock cycles.

[0099] In Example 7, the first memory type described in any one or more of Examples 1 to 6 is optionally Dynamic Random Access Memory (DRAM).

[0100] In Example 8, the second storage type described in any one or more of Examples 1 to 7 is optionally static random access memory (SRAM).

[0101] In Example 9, a method for operating a memory package having a stack of logical dies and memory devices is described, the memory package being configured to communicate with a host using multiple independent channels. The method may include: receiving a plurality of memory access requests for channels at the logical die; transmitting first data back to the host in response to a first memory access request among the plurality of memory access requests; transmitting an indication that data is not ready back to the host in response to a second memory access request for second data among the plurality of memory access requests; transmitting a first index and the indication that data is not ready back to the host; transmitting an indication that data is ready and third data back in response to a third memory access request among the plurality of memory access requests; transmitting the first index and the indication that data is ready back; and transmitting second data back to the host in response to transmitting the indication that data is ready and the first index back using the third memory access request.

[0102] In Example 10, the indication that the returned data is not ready is optionally included in a first single cycle of a burst cycle for exchanging data of the respective memory access requests among the plurality of memory access requests with the host, and the indication that the data is not ready is encoded on a single output bit of the channel.

[0103] In Example 11, the indication of data readiness for return as described in any one or more of Examples 1 to 10 is optionally included in encoding the data readiness indication on the single output bit of the channel within a second single cycle of the burst cycle.

[0104] In Example 12, the return of the first index described in any one or more of Examples 1 to 11 is optionally included in encoding the first index on the single output bit of the channel within multiple cycles of the bursting cycle.

[0105] In Example 13, any one or more of the multiple cycles described in Examples 1 to 12 may optionally not include the first single cycle or the second single cycle.

[0106] In Example 14, any one or more of the methods described in Examples 1 to 13 optionally include returning a Data Error (DERR) indication when the memory package cannot serve a memory access request among the plurality of memory access requests, wherein the DERR indication is encoded on the single output bit.

[0107] In Example 15, a method of operating a host configured to exchange information using multiple independent channels of a memory package comprising a stack of memory devices, the method comprising: sending multiple memory access requests to the memory package using a single channel; receiving first data via the single channel in response to a first memory access request among the multiple memory access requests; receiving an indication that data is not ready via the single channel in response to a second memory access request for second data among the multiple memory access requests; receiving a first index and the indication that data is not ready; receiving an indication that data is ready and third data via the single channel in response to a third memory access request among the multiple memory access requests; receiving the first index and the indication that data is ready; and receiving the second data via the single channel in response to the indication that data is ready and the first index received in response to the third memory access request.

[0108] In Example 16, the indication that data is not ready to be received as described in any one or more of Examples 1 to 15 is optionally included in the decoding of the indication that data is not ready on a single output bit of the single channel within a first single cycle of a burst cycle, the burst cycle being used to exchange data of the corresponding memory access requests among the plurality of memory access requests between the host and the memory package.

[0109] In Example 17, the indication of data readiness to receive data described in any one or more of Examples 1 to 16 is optionally included in the decoding of the data readiness indication on the single output bit of the single channel within the second single cycle of the burst cycle.

[0110] In Example 18, the reception of the first index described in any one or more of Examples 1 to 17 is optionally included in the decoding of the first index on the single output bit of the single channel within multiple cycles of the bursting cycle.

[0111] In Example 19, any one or more of the multiple cycles described in Examples 1 to 18 may optionally not include the first single cycle or the second single cycle.

[0112] In Example 20, any one or more of the methods in Examples 1 to 11 optionally include receiving a Data Error (DERR) indication when the memory package cannot serve a memory access request among the plurality of memory access requests, wherein the DERR indication is encoded on the single output bit.

[0113] Example 21 may include or use any part or combination of any one or more of the features described in Examples 1 to 20, or optionally combine with them to include or use a subject that may contain components for performing any one or more of the functions described in Examples 1 to 20, or a machine-readable medium containing instructions that, when executed by a machine, cause the machine to perform any one or more of the functions described in Examples 1 to 20.

Claims

1. A storage system comprising: A first memory device stack configured to store data, the stack comprising a plurality of memory dies of a first memory type; as well as Logic bare chip, which includes: Second memory of the second storage type; as well as The interface circuit, which is configured to, Receive multiple memory requests from an external host using an external bus. Data is relayed between the external host and multiple channels stacked with the first memory device via the second memory. In response to a first corresponding memory request among the plurality of memory requests, an indication of a data error is provided on a first output bit of the corresponding channel during a single clock cycle of a plurality of clock cycles used to exchange the data with the corresponding channel stacked with the first memory device. Provides a data ready indication associated with a second corresponding memory request among the plurality of memory requests, and A first index is provided, configured to identify the data associated with the second corresponding memory request.

2. The storage system of claim 1, wherein the second memory includes a list configured to store a plurality of indexes containing the first index.

3. The storage system of claim 1, wherein the data ready indication is encoded on the first output bit during the plurality of clock cycles.

4. The storage system of claim 3, wherein the data ready indication includes an indication that data is not ready.

5. The storage system of claim 3, wherein the data ready indication includes an indication that data is ready.

6. The storage system of claim 1, wherein the first index is encoded on the first output bit during the plurality of clock cycles.

7. The storage system of claim 1, wherein the first storage type is dynamic random access memory (DRAM).

8. The storage system of claim 1, wherein the second storage type is static random access memory (SRAM).

9. A method of operating a memory package having a stack of logic dies and memory devices, the memory package being configured to communicate with a host using multiple independent channels, the method comprising: Multiple memory access requests for the channel are received at the logic die; In response to the first memory access request among the plurality of memory access requests, the first data is transmitted back to the host; In response to a second memory access request for second data among the plurality of memory access requests, an indication that the data is not ready is sent back to the host; The first index and the indication that the data is not ready are transmitted back to the host. In response to a third memory access request among the plurality of memory access requests, an indication that data is ready and third data are returned. Return the first index and an indication that the data is ready; as well as The second data is returned to the host in response to the indication that the data is ready and the first index being returned using the third memory access request.

10. The method of claim 9, wherein the indication that the data is not ready to be returned is included in encoding the indication that the data is not ready on a single output bit of the channel within a first single cycle of a burst cycle for exchanging data of the respective memory access requests among the plurality of memory access requests with the host.

11. The method of claim 10, wherein the data-ready indication is included in the encoding of the data-ready indication on the single output bit of the channel within a second single period of the transmission period.

12. The method of claim 11, wherein returning the first index comprises encoding the first index on the single output bit of the channel within a plurality of cycles in the bursting cycle.

13. The method of claim 12, wherein the plurality of cycles does not include the first single cycle or the second single cycle.

14. The method of claim 10, further comprising returning a Data Error (DERR) indication when the memory package is unable to serve a memory access request among the plurality of memory access requests, wherein the DERR indication is encoded on the single output bit.

15. A method of operating a host, the host being configured to exchange information using a plurality of independent channels comprising a memory package containing a stack of memory devices, the method comprising: Multiple memory access requests can be sent to the memory encapsulation using a single channel; First data is received via the single channel in response to a first memory access request among the plurality of memory access requests; An indication that data is not ready to be received via the single channel in response to a second memory access request for second data among the plurality of memory access requests; Receive the first index and an indication that the data is not ready; In response to a third memory access request among the plurality of memory access requests, an indication of data readiness and third data are received via the single channel. Receive the first index and the indication that the data is ready; as well as The second data is received via the single channel in response to an indication that the data is ready received in response to the third memory access request and the first index.

16. The method of claim 15, wherein the indication that data is not ready to be received is included in decoding the indication that data is not ready to be received on a single output bit of the single channel within a first single cycle of a burst cycle, the burst cycle being used to exchange data of the respective memory access requests among the plurality of memory access requests between the host and the memory package.

17. The method of claim 16, wherein the indication of data readiness is included in the decoding of the indication of data readiness on the single output bit of the single channel within a second single cycle of the transmission cycle.

18. The method of claim 17, wherein receiving the first index includes decoding the first index on the single output bit of the single channel within a plurality of cycles in the bursting cycle.

19. The method of claim 18, wherein the plurality of cycles does not include the first single cycle or the second single cycle.

20. The method of claim 16, further comprising receiving a data error (DERR) indication when the memory package is unable to serve a memory access request among the plurality of memory access requests, wherein the DERR indication is encoded on the single output bit.

Images