Two-stage hybrid memory buffer for multiple streams

Abstract

Embodiments related to a two-level hybrid memory buffer for multiple streams in a memory subsystem are described herein. The processing device determines that first write data of a first stream stored in the host buffer component satisfies a threshold for programming the first programming unit. The processing device transfers the first write data from the host buffer component to a hierarchical buffer component, and writes the first write data from the hierarchical buffer component as the first programming unit to a first die of a plurality of non-volatile memory (NVM) dies. The processing device determines that second write data of a second stream satisfies a threshold for programming a second programming unit, transfers the second write data from the host buffer component to the hierarchical buffer component, and write the second write data from the hierarchical buffer component as the second programming unit to a second die of the plurality of NVM dies.

Description

[0001] Relevant information on divisional applications

[0002] This is a divisional application. The parent case of this divisional application is a patent application for invention with the application date of October 23, 2019, the invention title of "Two-Level Hybrid Memory Buffer for Multiple Streams", and the application number of 201980076853.4. Technical Field

[0003] Embodiments of the present invention generally relate to memory subsystems, and more specifically, to a two-level hybrid memory buffer for multiple streams in a memory subsystem. Background Art

[0004] The memory subsystem may be a storage system such as a solid state drive (SSD), and may include one or more memory components that store data. The memory components may be, for example, non-volatile memory components and volatile memory components. Generally, a host system may utilize the memory subsystem to store data at and retrieve data from the memory components. Summary of the Invention

[0005] One embodiment of the present application provides a system, which includes: one or more external dynamic random access memory (DRAM) devices having a plurality of programming unit buffers including a first programming unit buffer and a second programming unit buffer; an internal static random access memory (SRAM) device; one or more non-volatile memory (NVM) devices; and a processing device operably coupled to the one or more external DRAM devices, the internal SRAM device, and the one or more NVM devices to perform operations including: in response to first write data satisfying a first programming unit (PU) threshold related to the PU size of a first NVM device among the one or more NVM devices, transferring the first write data from the first programming unit buffer to the internal SRAM device; writing the first write data from the internal SRAM device to the one or more NVM devices as a first programming unit; in response to second write data satisfying a second PU threshold related to the PU size of a second NVM device among the one or more NVM devices, transferring the second write data from the second programming unit buffer to the internal SRAM device, where the second PU threshold is different from the first PU threshold; and writing the second write data from the internal SRAM device to the one or more NVM devices as a second programming unit.

[0006] Another embodiment of the present application provides a method, which includes: determining whether first write data in a first programming unit buffer in one or more external dynamic random access memory (DRAM) devices meets a first programming unit (PU) threshold, where the first PU threshold is related to a first PU size of a first non-volatile memory (NVM) device in the one or more NVM devices; in response to the first write data meeting the first PU threshold, transferring the first write data from the first programming unit buffer to an internal static random access memory (SRAM) device; writing the first write data from the internal SRAM device as a first programming unit to the first NVM device in the one or more NVM devices; determining whether second write data in a second programming unit buffer in the one or more external DRAM devices meets a second PU threshold, where the second PU threshold is related to a second PU size of a second NVM device in the one or more NVM devices, and the second PU threshold is different from the first PU threshold; in response to the second write data meeting the second PU threshold, transferring the second write data from the second programming unit buffer to the internal SRAM device; and writing the second write data from the internal SRAM device as a second programming unit to the second NVM device in the one or more NVM devices.

[0007] Another embodiment of the present application provides a non-transitory computer-readable medium including instructions that, when executed by a processing device, cause the processing device to perform operations including: determining whether first write data in a first programming unit buffer in one or more external dynamic random access memory (DRAM) devices meets a first programming unit (PU) threshold, where the first PU threshold is related to a first PU size of a first non-volatile memory (NVM) device in the one or more NVM devices; in response to the first write data meeting the first PU threshold, transferring the first write data from the first programming unit buffer to an internal static random access memory (SRAM) device; writing the first write data from the internal SRAM device as a first programming unit to the first NVM device in the one or more NVM devices; determining whether second write data in a second programming unit buffer in the one or more external DRAM devices meets a second PU threshold, where the second PU threshold is related to a second PU size of a second NVM device in the one or more NVM devices, and the second PU threshold is different from the first PU threshold; in response to the second write data meeting the second PU threshold, transferring the second write data from the second programming unit buffer to the internal SRAM device; and writing the second write data from the internal SRAM device as a second programming unit to the second NVM device in the one or more NVM devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present invention will be more fully understood from the following detailed description and the accompanying drawings of various embodiments of the invention.

[0009] Figure 1 Illustrate an example computing environment that includes a memory subsystem in accordance with some embodiments of the present invention.

[0010] Figure 2 Is a flowchart of an example method for buffering write data from multiple streams from a host system in a two - level memory buffer in accordance with some embodiments of the present invention.

[0011] Figure 3 Is a flowchart of an example method for buffering write data and independent NAND redundant array (RAIN) data in a two - level memory buffer in accordance with some embodiments of the present invention.

[0012] Figure 4 Is a flowchart of write data, discard item collection data, and RAIN data from multiple streams using a two - level memory buffer in accordance with some embodiments of the present invention.

[0013] Figure 5 Is a diagram illustrating eight streams striped across eight channels to free dies in accordance with some embodiments of the present invention.

[0014] Figure 6 Is a diagram illustrating a hierarchical buffer for programming four streams on a single channel in accordance with some embodiments of the present invention.

[0015] Figure 7 Is a diagram illustrating a sequence of program operations with simultaneous completion across all eight channels in accordance with some embodiments of the present invention.

[0016] Figure 8 Is a block diagram of an example computer system in which embodiments of the present invention may operate. DETAILED DESCRIPTION

[0017] Aspects of the present invention relate to a two - level hybrid memory buffer for multiple streams in a memory subsystem. The memory subsystem is also referred to hereinafter as a "memory device" or a "memory system". An example of a memory subsystem is a storage system such as a solid - state drive (SSD). The memory subsystem may be a hybrid memory / storage subsystem. Generally, a host system may utilize a memory subsystem that includes one or more memory components. The memory device may include non - volatile memory devices such as, for example, NAND (negative - AND). The host system may provide data stored at the memory subsystem and may request data retrieved from the memory subsystem.

[0018] Conventional memory subsystems communicate with a host system using a single write data stream. This conventional memory subsystem uses a buffer for data going through the NAND controller to the NAND flash memory. The buffer is typically an external dynamic random access memory (DRAM) or an on-chip static RAM (SRAM or internal SRAM). The NAND controller obtains write data from the single stream received from the host and accumulates the write data in the external DRAM or accumulates the write data in the internal SRAM until there is enough data to send to the NAND flash for programming. There is enough data when the stored data meets (reaches or exceeds) a programming unit (PU) threshold, where the PU threshold corresponds to the programming unit size of one or more NVM devices. The internal SRAM must support the full bandwidth of the back-end and the host. SRAM provides the highest possible bandwidth at high cost for performance, and DRAM significantly reduces cost but also reduces performance. When using multiple streams, problems arise with conventional memory subsystems. In some cases, the memory subsystem is required to support 32 streams and may increase to even 1024 streams. However, the memory subsystem needs to support all these different streams being open and operating simultaneously, regardless of whether the host system is performing sequential writes (e.g., sequential access) or using random writes to random access different streams. A conventional memory subsystem with a single buffer (external DRAM or internal SRAM) cannot support a high number of streams with high performance (e.g., sequential writes or random writes). The size of the internal SRAM in these conventional memory subsystems must be large enough to store the data for all streams. Although SRAM has a higher bandwidth than DRAM, adding a larger internal SRAM to an integrated circuit is prohibitively expensive in terms of cost, die area, and power. Using DRAM would be cheaper and provide a large memory capacity, but the performance would be limited by the bandwidth of the DRAM. Although a wider DRAM interface can increase the DRAM bandwidth, increasing the DRAM interface would increase the cost and power of the integrated circuit and make it more difficult to fit into small form factors such as M.2 or EDSFF 1UShort.

[0019] Aspects of the present invention address the above and other drawbacks by providing a two - level hybrid memory buffer for supporting multiple streams. The two - level memory buffer may include a host buffer component (e.g., external DRAM) and a hierarchical buffer component (e.g., internal SRAM). The host buffer component may be the primary buffer for all data, while the hierarchical buffer component, which may be on - chip SRAM, may buffer - stage its data before streaming it to the NVM memory component of the memory subsystem. Data from a host programming stream may remain separated into individual programming units (PUs) in the host buffer component (primary buffer) until there is enough data to program an NVM device (e.g., an NVM die, flash device, or the like). When the data volume meets a threshold, there is enough data or sufficient data. At that time, the data in the PUs may flow to the hierarchical buffer component (e.g., internal SRAM), where the data will be processed with error protection and other features in preparation for writing to the NVM memory. The data in the PUs will have a much shorter lifespan in the hierarchical buffer component than in the host buffer component. Then, the NVM controller sends the data from the hierarchical buffer component to the NVM device (e.g., flash device). The two - level memory buffer supports streams and allows the memory subsystem to scale the number of streams, such as 32 streams, 64 streams, 128 streams, 512 streams, 1024 streams, etc.

[0020] For example, a host system may communicate with the memory subsystem via a high - speed serial computer bus standard, such as the Peripheral Component Interconnect Express (PCIe) standard or Non - Volatile Memory Express (NVMe) using an electrical interface of PCIe. NVMe is an open logical device interface specification for accessing non - volatile storage media attached via an electrical bus. The NVMe of a solid - state storage device has low latency and internal parallelism. Multiple streams are part of the NVMe standard, and it is expected that the number of streams supported by the memory subsystem will scale, for example, from 32 streams to 1024 streams. The two - level memory buffer (a hybrid of DRAM and internal SRAM) supports multiple streams (e.g., multiple NVMe streams) and the scalability of the number of streams. As the number of streams begins to scale, the hierarchical buffer component may support an increasing number of NVMe streams while keeping the cost relatively low and meeting performance and form - factor requirements. The hierarchical buffer component stages the data at the backend and all the high - bandwidth required to keep the backend running at full efficiency.

[0021] A stream provides a way for a host system to identify different accesses to a memory subsystem, whether for read or write access. Streams are separated from each other, and each stream can be considered to be available for a specific host task or application. When a host system uses the memory subsystem to store data, the host system combines all its data. If the host system can provide a large amount of data for various applications or tasks, then the storage medium can be more efficient. Although the data may appear randomly, the host system can identify the data into these different streams and the memory subsystem can place the corresponding data of different streams together in different locations in the NVM backend. Write amplification is an undesirable phenomenon associated with flash memory and SSDs, where the actual amount of information physically written to the storage medium is a multiple of the logical amount that is intended to be written. Streams can reduce the write amplification rate. When data is stored as a stream, the memory subsystem can easily remove the data of a specific stream. For example, the host system can decide to delete a stream, and the memory subsystem can erase those data.

[0022] Aspects of the present invention address the above and other drawbacks by buffering RAIN data in a two-level memory buffer. The RAIN parity data of each of these multiple streams can be accumulated in size, and the two-level memory buffer can store the RAIN data of these multiple streams in a host buffer component and temporarily store it in a hierarchical buffer component. Intelligence is added to the controller to manage the hierarchical host buffer component and the hierarchical buffer component of the two-level memory buffer. The controller can use, for example, firmware to control the use of the hierarchical region and manage the data stream, which includes managing die conflicts in the NVM die (flash device).

[0023] Figure 1 An example computing environment 100 including a memory subsystem 110 is described in accordance with some embodiments of the present invention. The memory subsystem 110 can include media, such as memory components 112A through 112N. The memory components 112A through 112N can be volatile memory components, non-volatile memory components, or a combination of such components. In some embodiments, the memory subsystem is a storage system. An example of a storage system is an SSD. In some embodiments, the memory subsystem 110 is a hybrid memory / storage subsystem. Generally, the computing environment 100 can include a host system 120 that uses the memory subsystem 110. For example, the host system 120 can write data to the memory subsystem 110 and read data from the memory subsystem 110.

[0024] The host system 120 can be a computing device, such as a desktop computer, a laptop computer, a network server, a mobile device, or such a computing device that includes a memory and a processing device. The host system 120 can include or be coupled to the memory subsystem 110 such that the host system 120 can read data from or write data to the memory subsystem 110. The host system 120 can be coupled to the memory subsystem 110 via a physical host interface. As used herein, "coupled to" generally refers to a connection between components, which can be an indirect communication connection or a direct communication connection (e.g., without an intervening component), whether wired or wireless, and includes connections such as electrical, optical, magnetic, etc. Examples of the physical host interface include, but are not limited to, Serial Advanced Technology Attachment (SATA) interface, Peripheral Component Interconnect Express (PCIe) interface, Universal Serial Bus (USB) interface, Fibre Channel, Serial Attached SCSI (SAS), etc. The physical host interface can be used to transfer data between the host system 120 and the memory subsystem 110. When the memory subsystem 110 is coupled to the host system 120 via a PCIe interface, the host system 120 can further utilize the Non-Volatile Memory Express (NVMe) interface to access the memory components 112A to 112N. The physical host interface can provide an interface for passing control, address, data, and other signals between the memory subsystem 110 and the host system 120.

[0025] Memory components 112A to 112N may include any combination of different types of non-volatile memory components and / or volatile memory components. Examples of non-volatile memory components include NAND-type flash memory. Each of memory components 112A to 112N may include one or more memory cell arrays, such as single-level cells (SLCs) or multi-level cells (MLCs) (e.g., triple-level cells (TLCs) or quad-level cells (QLCs)). In some embodiments, a particular memory component may include both an SLC portion and an MLC portion of memory cells. Each of the memory cells may store one or more data bits (e.g., data blocks) used by host system 120. Although non-volatile memory components such as NAND-type flash memory are described, memory components 112A to 112N may be based on any other type of memory such as volatile memory. In some embodiments, memory components 112A to 112N may be (but are not limited to) random access memory (RAM), read-only memory (ROM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), phase change memory (PCM), magnetic random access memory (MRAM), NOR flash memory, electrically erasable programmable read-only memory (EEPROM), and cross-point non-volatile memory cell arrays. The cross-point non-volatile memory array may perform bit storage based on a change in bulk resistance along with a stacked cross-gate format data access array. Additionally, compared to many flash-based memories, the cross-point non-volatile memory may perform in-situ write operations, where non-volatile memory cells may be programmed without first erasing the non-volatile memory cells. Further, the memory cells of memory components 112A to 112N may be grouped into memory cell groups, word lines, word line groups, or data blocks, which may refer to the unit of the memory component for storing data.

[0026] A memory system controller 115 (hereinafter referred to as "controller") may communicate with memory components 112A to 112N to perform operations such as reading data, writing data, or erasing data at the memory components 112A to 112N and other such operations. The controller 115 may include hardware such as one or more integrated circuits and / or discrete components, a two-level memory buffer 119, or a combination thereof. The two-level memory buffer 119 may include a local memory 119A and an external memory 119B. In one embodiment, the local memory 119A has one or more SRAM devices or SRAM components, and the external memory 119B has one or more DRAM devices or DRAM components. In other embodiments, the local memory 119A may have other types of memory devices, including DRAM devices. It should be noted that on-chip memory devices are more expensive than external memory devices. The controller 115 may be an SSD controller that manages a hierarchical host buffer component and a hierarchical buffer component of the two-level memory buffer 119. The SSD controller 115 may use, for example, firmware to have the intelligence of how to use the hierarchical regions and how to manage the data flow, including managing die conflicts in the NVM dies (flash devices). The controller 115 may be a microcontroller, a dedicated logic circuit system (such as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or other suitable processor. The controller 115 may include a processor (processing device) 117 configured to execute instructions stored in the local memory 119A. The local memory 119 for storing instructions may be the same as or different from the local memory of the two-state memory buffer 119. In the illustrative example, the local memory 119A of the controller 115 includes an embedded memory configured to store instructions for performing various processes, operations, logic flows, and routines for controlling the operation of the memory subsystem 110, including handling communication between the memory subsystem 110 and the host system 120. In some embodiments, the local memory 119A may include memory registers for storing memory pointers, fetching data, etc. The local memory 119A may also include a read only memory (ROM) for storing microcode. Although the Figure 1 illustrative memory subsystem 110 has been described as including a controller 115, in another embodiment of the present invention, the memory subsystem 110 may not include a controller 115, but may rely on external control (such as provided by an external host or by a processor or controller separate from the memory subsystem).

[0027] Generally, the controller 115 can receive commands or operations from the host system 120 and can convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory components 112A to 112N. The controller 115 can be responsible for other operations such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between the logical block addresses and the physical block addresses associated with the memory components 112A to 112N. The controller 115 can further include host interface circuitry that communicates with the host system 120 via a physical host interface. The host interface circuitry can convert commands received from the host system into command instructions to access the memory components 112A to 112N, and convert responses associated with the memory components 112A to 112N into information for the host system 120.

[0028] The memory subsystem 110 can also include additional circuitry or components not shown. In some embodiments, the memory subsystem 110 can include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoders and column decoders) that can receive addresses from the controller 115 and decode the addresses to access the memory components 112A to 112N.

[0029] The memory subsystem 110 includes a two-level memory buffer 119 (local memory 119A and external memory 119B) and a two-level buffer controller component 113 that can be used to control the transfer of data between the host system 120, the two-level component buffer 119, and the memory components 112A to 112N. In some cases, the external memory 119B can be implemented as one of the memory components 112A to 112N, and one or more of the memory components 112A to 112N can be NVM components. In some embodiments, the controller 115 includes at least a portion of the two-level buffer controller component 113. For example, the controller 115 can include a processor 117 (processing device) configured to execute instructions stored in the local memory 119A to perform the operations described herein. In some embodiments, the two-level buffer controller component 113 is part of the host system 120, an application, or an operating system.

[0030] The two - level buffer controller component 113 may determine that first write data of a first stream stored in an external memory 119B (also referred to herein as a host buffer component) meets a threshold for programming a first programming unit. The size of the first programming unit corresponds to the programming unit size of an NVM die (e.g., the memory component 112A). The two - level buffer controller component 113 transfers the first write data to a local memory 119A (also referred to herein as a hierarchical buffer component). The two - level buffer controller component 113 writes the first write data from the local memory 119A (hierarchical buffer component) as a first programming unit to the memory component 112A (e.g., the first NVM die). The two - level buffer controller component 113 determines that second write data of a second stream stored in the external memory 119B (host buffer component) meets a threshold for programming a second programming unit. The two - level buffer controller component 113 transfers the second write data from the external memory 119B (host buffer component) to the local memory 119A (hierarchical buffer component). The two - level buffer controller component 113 writes the second write data from the local memory 119A (hierarchical buffer component) as a second programming unit to a second memory component (e.g., a second NVM die in the memory component 112A or another NVM die in another memory component 112B to N).

[0031] Figure 2 is a flowchart of an example method 200 for buffering write data of multiple streams from a host system in a two - level memory buffer according to some embodiments of the present invention. Method 200 may be executed by processing logic, which may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, the hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executing on a processing device), or a combination thereof. In some embodiments, method 200 is executed by Figure 1 the memory two - level buffer controller component 113. Although shown in a particular sequence or order, the order of the processes may be modified unless otherwise specified. Accordingly, the illustrated embodiments should be understood only as examples, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. Additionally, one or more processes may be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.

[0032] At block 210, processing logic determines that first write data of a first stream stored in a host buffer component meets a threshold for programming a first programming unit of an NVM die. Note that the host may write to the respective streams in any order such that the programming unit size may be reached in any order. At block 220, processing logic transfers the first write data from the host buffer component to a staging buffer component. At block 230, processing logic writes the first write data from the staging buffer component to a first NVM die as a first programming unit. At block 240, processing logic determines that second write data of a second stream stored in the host buffer component meets a threshold for programming a second programming unit. At block 250, processing logic transfers the second write data from the host buffer component to the staging buffer component. At block 260, processing logic writes the second write data from the staging buffer component to a second NVM die as a second programming unit.

[0033] In another embodiment, processing logic generates first independent NAND redundancy array (RAIN) data corresponding to the first write data. Processing logic temporarily stores the first RAIN data in the staging buffer component. Processing logic transfers the first RAIN data from the staging buffer component to the host buffer component together with the first write data written to the first die as a first programming unit. The first RAIN data may be transferred concurrently or sequentially with the first write data written to the first NVM die. Processing logic generates second RAIN data corresponding to the second write data. Processing logic temporarily stores the second RAIN data in the staging buffer component. Processing logic transfers the second RAIN data from the staging buffer component to the host buffer component together with the second write data written to the second die as a second programming unit. If there is room, the RAIN data may remain in the staging buffer component; otherwise, the RAIN data is put back into the host buffer component if the host buffer component is not used to make room for RAIN data of another stream. Similarly, the second RAIN data may be transferred concurrently or sequentially with the second write data written to the second NVM die.

[0034] In another embodiment, the processing logic stores first garbage collection (GC) data corresponding to any stream in a host buffer component. Not all streams require GC, so if the first stream requires GC, then the first GC data can correspond to the first stream. Alternatively, the first GC data can correspond to a stream different from the first stream. As described herein, the order in which data moves from the hierarchical buffer component and the host buffer component depends on host access through the host. The processing logic determines that the first GC data in the host buffer component meets a threshold for programming a third programming unit. The processing logic transfers the first GC data from the host buffer component to the hierarchical buffer component. The processing logic writes the first GC data from the hierarchical buffer component to a third NVM die as the third programming unit.

[0035] In another embodiment, the processing logic encodes first write data as it is transferred between the host buffer component and the hierarchical buffer component. In another embodiment, the processing logic stores GC read data in the hierarchical buffer component. The GC read data can be received from an NVM controller associated with one or more NVM dies. The processing logic transfers the GC read data from the hierarchical buffer component to the host buffer component and decodes the GC read data as it is transferred from the hierarchical buffer component to the host buffer component.

[0036] In some embodiments, the write data received from the host system is sequential write data. In other embodiments, the write data received from the host system is random write data.

[0037] Figure 3 is a flowchart of an example method 300 for buffering write data and RAIN data in a two-level memory buffer according to some embodiments of the present invention. Method 300 can be executed by processing logic, which can include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executing on a processing device), or a combination thereof. In some embodiments, method 300 is executed by Figure 1 the memory two-level buffer controller component 113. Although shown in a particular sequence or order, the order of the processes can be modified unless otherwise specified. Accordingly, the illustrated embodiments should be understood only as examples, and the illustrated processes can be executed in a different order, and some processes can be executed in parallel. Additionally, one or more processes can be omitted in various embodiments. Accordingly, not all processes are required in every embodiment. Other process flows are possible.

[0038] At block 310, processing logic determines that first write data of a first stream stored in a host buffer component meets a threshold for programming a first programming unit of a NVM die. The size of the first programming unit corresponds to the size of the NVM die. At block 320, the processing logic transfers the first write data from the host buffer component to a hierarchical buffer component. At block 330, the processing logic writes the first write data from the hierarchical buffer component as the first programming unit to a first NVM die. At block 340, the processing logic generates first RAIN data corresponding to the first write data. At block 350, the processing logic temporarily stores the first RAIN data in the hierarchical buffer component. It should be noted that the first RAIN data is transferred to the hierarchical buffer component before or simultaneously with the first write data. At block 360, the processing logic transfers the first RAIN data from the hierarchical buffer component to the host buffer component.

[0039] In another embodiment, processing logic determines that second write data of a second stream stored in a host buffer component meets a threshold for programming a second programming unit. The processing logic transfers the write data to the hierarchical buffer component, generates second RAIN data corresponding to the second write data, and writes the second write data from the hierarchical buffer component as the second programming unit to a second NVM die. The processing logic also temporarily stores the second RAIN data in the hierarchical buffer component and subsequently transfers the second RAIN data from the hierarchical buffer component to the host buffer component.

[0040] Figure 4 is a flowchart of a data stream of write data, discard item collection data, and RAIN data from multiple streams using a two-level memory buffer according to some embodiments of the present invention. In the following description of the data stream, host system 120 communicates with SSD 400 via an electrical interface. SSD 400 includes an SSD controller 402 having a hierarchical buffer SRAM 406, an external DRAM component 404 (also referred to herein as a main buffer component), and a flash device 408. SSD controller 402 may be the controller 115 described above with respect to Figure 1 and may include firmware for controlling the interaction of these components for the data stream described below. SSD controller 402 includes a RAIN exclusive OR (XOR) engine 410, a low density parity check (LDPC) encoder 412, an LDPC decoder 414, and a NAND controller 416. In other embodiments, the various components of SSD controller 402 may be integrated as separate integrated circuits, such as a separate integrated circuit for NAND controller 416. The hierarchical buffer SRAM may include multiple logical partitions, as Figure 4It is described in [reference]. In other embodiments, the SSD 400 can be other types of memory subsystems and can have different types of memory for the main buffer component and the hierarchical buffer component. Moreover, the LDPC encoder and decoder can be other types of data protection encoders and data protection decoders. In addition, the NAND controller 416 and the flash device 408 can be other types of NVM controllers and NVM storage media.

[0041] In the first data stream 401, the host system 120 performs a sequential write operation 403 to store write data at the SSD 400. The write data can correspond to one or more streams. For example, the SSD 400 receives the first write data 405 of the first stream and the second write data 407 of the second stream, and the SSD controller 402 stores the first write data 405 and the second write data 407 separately in the DRAM component 404. The DRAM component 404 can be organized to store the first write data 405 in a programming unit (PU) buffer, where the size of the PU buffer corresponds to the die size of the flash device 408. Data from more than two streams can be received and stored in separate PU buffers in the DRAM component 404. When the SSD controller 402 determines that the first write data 405 of the first stream stored in the first PU buffer of the DRAM component 404 meets the threshold for programming the first programming unit (PU) (i.e., filling the PU buffer with the first write data), the SSD controller 402 transfers the first write data 405 to the hierarchical buffer SRAM 406. It should be noted that the first stream is the first to become full in the PU buffer at this time, which can be any of the PU buffers, depending on the host. Before storing the first write data 405 in the hierarchical buffer SRAM 406, the LDPC 412 encodes the first write data 405. That is, the first write data 405 flows to the LDPC encoder 412 and then to the hierarchical buffer SRAM 406. Since the first write data 405 is already one PU size, the SSD controller 402 can start programming one of the flash devices 408 using the PU corresponding to the first write data 405 stored in the hierarchical buffer SRAM 406. The SSD controller 402 can send the first write data 405 to the NAND controller 416, or the NAND controller 416 can retrieve the first write data 405 from the hierarchical buffer SRAM 406 to write the first write data 405 to the first of the multiple flash devices 408.

[0042] As part of the first data stream 401, the SSD controller 402 may determine that second write data 407 of a second stream stored in a second PU buffer in the DRAM component 404 meets a threshold for programming the second PU and transmit the second write data 407 to the hierarchical buffer SRAM 406 via the LDPC encoder 412. The NAND controller 416 may write the second write data 407 from the hierarchical buffer SRAM 406 to a second flash device among a plurality of flash devices 408 as a second PU. The hierarchical buffer SRAM 406 may serve as a pipeline for staging programming units for programming by the NAND controller 416. For example, once the first write data 405 is written to the flash device 408, the second write data 407 may be transmitted to the hierarchical buffer SRAM 406 in a staggered manner. Data from more than two sequential write data streams may be received and stored in the hierarchical buffer SRAM 406.

[0043] In some cases, parity information may be generated for sequential write data written to the flash device 408. As Figure 4 illustrated, the RAIN XOR engine 410 may receive the first write data 407 from the NAND controller 416 and may XOR the first RAIN data 409 corresponding to the first write data 405 with the existing RAIN data. The RAIN data is typically stored in the DRAM and moves to the hierarchical buffer SRAM 406 together with the associated write data. The first RAIN data 409 may be temporarily stored in the hierarchical buffer SRAM 406 before being transmitted to the DRAM component 404. This may be performed in conjunction with writing the first write data 405 as a first programming unit to the first flash device. The RAIN XOR engine 410 may receive the second write data 407 from the NAND controller 416 and generate a second RAIN data 411 corresponding to the second write data 407. The second RAIN data 411 may be temporarily stored in the hierarchical buffer SRAM 406 before being transmitted to the DRAM component 404. This may be performed in conjunction with writing the second write data 407 as a second programming unit to the second flash device. Parity data from more than two streams may be generated and stored in the hierarchical buffer SRAM 406 and the DRAM component 404.

[0044] As described above with respect to the first data stream 401, a two-level memory buffer may be used for sequential write data from the sequential write data operation 403. In other embodiments, the two-level memory buffer may be used for random write data received from the host system 120 in conjunction with a random write operation, such as illustrated as the second data stream 421.

[0045] In the second data stream 421, the host system 120 performs a random write operation 423 to store write data at the SSD 400. The random write data can correspond to one or more streams. For example, the SSD 400 receives third write data 425 of a third stream and fourth write data 427 of a fourth stream, and the SSD controller 402 stores the third write data 425 and the fourth write data 427 separately in the DRAM component 404. As described above, the DRAM component 404 can be organized to store data in the PU buffer, where the size of the PU buffer corresponds to the die size of the flash device 408. Data from more than two streams can be received and stored in separate PU buffers in the DRAM component 404. When the SSD controller 402 determines that the third write data 425 of the third stream stored in the third PU buffer of the DRAM component 404 meets the threshold for programming the third PU (i.e., filling the PU buffer with the third write data), the SSD controller 402 transfers the third write data 425 to the hierarchical buffer SRAM 406. Before storing the third write data 425 in the hierarchical buffer SRAM 406, the LDPC encoder 412 encodes the third write data 425. That is, the third write data 425 flows to the LDPC encoder 412 and then to the hierarchical buffer SRAM 406. Since the third write data 425 is already the size of one PU, the SSD controller 402 can start programming one of the flash devices 408 with the PU corresponding to the third write data 425 stored in the hierarchical buffer SRAM 406. The SSD controller 402 can send the third write data 425 to the NAND controller 416, or the NAND controller 416 can retrieve the third write data 425 from the hierarchical buffer SRAM 406 to write the third write data 425 to the third device among the multiple flash devices 408.

[0046] As part of the second data stream 421, the SSD controller 402 can determine that the fourth write data 427 of the second stream stored in the fourth PU buffer of the DRAM component 404 meets the threshold for programming the fourth PU and transfer the fourth write data 427 to the hierarchical buffer SRAM 406 via the LDPC encoder 412. The NAND controller 416 can write the fourth write data 427 from the hierarchical buffer SRAM 406 as the fourth PU to the fourth flash device among the multiple flash devices 408. The hierarchical buffer SRAM 406 can act as a pipeline for staging programming units for programming by the NAND controller 416. For example, once the third write data 425 is written to the flash device 408, the fourth write data 427 can be transferred to the hierarchical buffer SRAM 406 in a staggered manner. Data from more than two random write data streams can be received and stored in the hierarchical buffer SRAM 406.

[0047] In some cases, parity information may be generated for randomly written data written to the flash device 408. As Figure 4 described, the RAIN XOR engine 410 may receive the third write data 427 from the NAND controller 416 and generate the third RAIN data 429 corresponding to the third write data 425. The third RAIN data 429 may be temporarily stored in the hierarchical buffer SRAM 406 before being transmitted to the DRAM component 404. This may be performed by writing the third RAIN data 429 combined with the third write data 425 as the third programming unit to the first flash device. The RAIN XOR engine 410 may receive the fourth write data 427 from the NAND controller 416 (or other NVM controller) and generate the fourth RAIN data 431 corresponding to the fourth write data 427. The fourth RAIN data 431 may be temporarily stored in the hierarchical buffer SRAM 406 before being transmitted to the DRAM component 404. This may be performed by writing the fourth RAIN data 431 combined with the fourth write data 427 as the fourth programming unit to the fourth flash device. Parity data from more than two streams may be generated and stored in the hierarchical buffer SRAM 406 and the DRAM component 404.

[0048] As described above with respect to the first data stream 401 and the second data stream 421, a two-level memory buffer may be used for sequential write data from the sequential write data operation 403 and random write data from the random write data operation 423. In other embodiments, the two-level memory buffer may be used for read data received from the plurality of flash devices 408 before being sent to the host system 120, as illustrated as the third data stream 433.

[0049] In the third data stream 433, the NAND controller 416 transmits the read data from one of the flash devices 408 to the hierarchical buffer SRAM 406 to be temporarily stored in response to a read operation. The read data from the hierarchical buffer SRAM 406 is decoded by the LDPC decoder 414 and sent to the host system 120. In other words, the read data is passed from the NAND controller 416 to the hierarchical buffer SRAM 406, from the hierarchical buffer SRAM 406 to the LDPC decoder 414, and from the LDPC decoder 414 to the host system 120.

[0050] As described above with respect to the third data stream 433, a two-level memory buffer may be used for read data from the read data operation. In other embodiments, the two-level memory buffer may be used for garbage collection (GC) read data from the GC read operation, as illustrated as the fourth data stream 441.

[0051] In the fourth data stream 441, the NAND controller 416 transfers the first GC data 443 to the hierarchical buffer SRAM 406 to be temporarily stored in response to a GC read operation regarding the first stream (the first write data 405). The first GC data 443 from the hierarchical buffer SRAM 406 is decoded by the LDPC decoder 414 and sent to the DRAM component 404. The GC data may also correspond to one or more streams. As described above, the DRAM component 404 may be organized to store the GC data in the PU buffer, where the size of the PU buffer corresponds to the die size of the flash device 408. The GC data from more than two streams may be received and stored in separate PU buffers in the DRAM component 404, such as illustrated as the first GC data 443 and the second GC data 445. When the SSD controller 402 determines that the first GC data 425 of the first stream stored in the fifth PU buffer of the DRAM component 404 meets the threshold for programming the fifth PU (i.e., filling the PU buffer with the first GC data), the SSD controller 402 transfers the first GC data 443 to the hierarchical buffer SRAM 406. Before storing the first GC data 443 in the hierarchical buffer SRAM 406, the LDPC encoder 412 encodes the first GC data 443. That is, the first GC data 443 flows to the LDPC encoder 412 and from the LDPC encoder 412 to the hierarchical buffer SRAM 406. Since the first GC data 443 is already one PU size, the SSD controller 402 can start programming one of the flash devices 408 using the PU corresponding to the first GC data 443 stored in the hierarchical buffer SRAM 406. The SSD controller 402 may send the first GC data 443 to the NAND controller 416, or the NAND controller 416 may retrieve the first GC data 443 from the hierarchical buffer SRAM 406 to write the first GC data 443 to the third device among the plurality of flash devices 408.

[0052] Note that for the garbage collection (GC) read for the discard item collection stream, the GC read can run at the same bandwidth as the write operation and only valid transfer units (TUs) are read from the NAND controller 416. A TU is the basic granularity for data management (e.g., 4096B). Once the GC data is aggregated into the new physical unit (PU) buffer, the backend write bandwidth is shared by the write operations (e.g., 403, 423). The write amplification factor can determine the ratio of the GC data to the host write data. In some embodiments, the GC data from the NAND controller 416 is transferred to the hierarchical buffer SRAM 406 via multiple channels. The multiple channels can share the same buffer, depending on the percentage of valid data. The LDPC decoder 414 reads, processes the decoded data, and then writes the decoded data to the DRAM component 404. The GC data is aggregated into the new PU buffer. The GC write data stream is the same as the write data from the host system 120. Specifically, the GC write data can be sent to the hierarchical buffer SRAM 406 under the same rules as the write data. As Figure 4 illustrated in, the hierarchical buffer SRAM 406 stores the GC read data until it is transferred to the DRAM and also stores the GC write data. In one embodiment, the hierarchical buffer SRAM 406 has two logical partitions: a first logical partition where the write data and / or the GC write data before being written to the flash device 408 are temporarily stored; and a second logical partition where the host read data and / or the GC read data before being transferred to the host system 120 or the DRAM component 404 are temporarily stored. Therefore, in addition to the write data capacity, the hierarchical buffer SRAM 406 should also allocate capacity for the GC read.

[0053] To scale from one stream to many streams (e.g., up to 1024 streams), the DRAM component 404 is the only option for expansion, and the hierarchical buffer SRAM 406 operates as a pipeline where all the host write data and all the GC read data are accumulated in the DRAM component 404 until the threshold for programming the die in the programming stream is met. In one embodiment, the die of the flash device 408 can have a programming unit (PU) size of 330KB per NAND die. In one embodiment, the write amplification (WA) is 2.1. The RAIN storage is also allocated between the DRAM component 404 and the hierarchical buffer SRAM 406. A certain number of RAIN contexts can reside in the hierarchical buffer SRAM 406. As the stream count increases, the exchange of the RAIN contexts between the DRAM component 404 and the hierarchical buffer SRAM 406 can occur. In one embodiment, one or more DMA engines can be used to exchange the RAIN contexts between the DRAM component 404 and the hierarchical buffer SRAM 406.

[0054] In one embodiment based on the aforementioned WA, the bandwidth of the sequential write operation 403 is 4600 MB / s, the bandwidth of the random write operation 423 is 1250 MB / s, and the bandwidth for the host to read data is 6600 MB / s. There is also a bandwidth of 4600 MB / s for sequential write data and 1250 MB / s for random write data when transferring sequential write data from the DRAM component 404 to the hierarchical buffer SRAM 406. The GC write data has a bandwidth of 2625 MB / s when transferring from the DRAM component 404 to the hierarchical buffer SRAM 406. This can be determined by multiplying 1250 by a write magnification factor of 2.1. The random write data and the GC write data can operate together using a bandwidth of 4000 MB / s at a 31 / 32 code rate. The LDPC encoder can operate using a bandwidth of 4600 * 0.92 for sequential write data, 1250 * 0.92 for random write data, and 2625 * 0.92 for GC write data, where 0.92 is the LDPC code rate. The data has an average bandwidth of 5150 MB / s (e.g., 4600 / 0.92 * 32 / 31) when transferring from the hierarchical buffer SRAM 406 to the NAND controller 416. This can be responsible for writing RAIN data to the flash device 408 at a 31 / 32 code rate. One or more DMA engines can exchange RAIN data between the DRAM component 404 and the hierarchical buffer SRAM 406 at a bandwidth of 4000 MB / s. The hierarchical buffer SRAM 406 and the RAIN XOR engine 410 can transfer RAIN data using a bandwidth of 9600 MB / s because the RAIN XOR engine 410 can have a peak rate of 8 * 1200 MB / s. For the read operation, the hierarchical buffer SRAM 406 and the NAND controller 416 can have an average bandwidth of 7174 MB / s (6600 / 0.92) for transferring host read data and GC read data. The LDPC decoder 414 and the hierarchical buffer SRAM 406 can have a bandwidth of 2853 Mb / s for GC read data transferred between the hierarchical buffer SRAM 406 and the DRAM component 404. The bandwidth between the NAND controller 416 and the host system 120 via the hierarchical buffer is 6600 MB / s. The bandwidth between the NAND controller 416 and the DRAM component 404 via the hierarchical buffer is 2625 MB / s. The various bandwidths described above are demonstrated based on the programming unit size, DRAM, and SRAM technologies used. In other embodiments, the individual bandwidths can be different. The Pu size can also be 220 KB, 660 KB, 440 KB, 293 KB, or the like. The PU size can be contributed by the page size and the number of planes. For example, a PU size of 330 KB can be derived from a page size of 18344 bytes / page * 3 pages * 6 planes.The bandwidth can depend on the program time, ONFI transfer time, DRAM transfer time, and programming algorithms, such as single-pass programming or double-pass programming (from 8 programming levels to 16 programming levels).

[0055] In one embodiment, the design requirements of a two-level memory buffer architecture include 33 streams, 16 groups, a host sequential write of 4.6 GB / s, a coding rate of 5.15 GB / s (calculated as 4600 MB / s sequential host write / 0.92 * 32 / 31 LDPC code rate (31 / 32 RAIN code rate)), Rand write FOG = 1M KIOP - 4.1 GB / s (for WA = 1) and 2.1 Rand WrSS, a 64b DDR interface of 3200 MT / x with 15 / 16 ECC consumption, and an ONFI 4.1 interface burst rate of 1200 MT / s. A write amplification factor of 2.1 is estimated to reduce 4600 MB / s in the backend to 4000 MB / s to account for additional FW consumption (4000 MB / s / 4096 = 977 KIOP backend). The write amplification effect seen by the host is that the system is 3.2 (2 * WA - 1). 977 KIOP / 3.2 = 305 KIOP (estimated at 312 KIOP), so 305 KIOP * 4096 = 1250 host write bandwidth. The GC bandwidth after the LDPC decoder is 1250 MB / s * 2.1 = 2625 Mb / s (plus background scan consumption). The main basis for this stream is to construct the programming units in DRAM and save the RAIN background in DRAM to expand the write data storage as the expected number of streams increases over time. As mentioned above, the bandwidth values can vary based on different design requirements of the two-level memory buffer architecture.

[0056] To minimize the on-chip SRAM and provide scalability higher than the above requirements, the host data and RAIN data are stored in DRAM. For this case, the host write data is accumulated in DRAM until there is threshold data that satisfies the die for the programming stream. The SRAM can be used as a hierarchical buffer for the host data, GC data, and RAIN background. When the XOR is completed, the RAIN background needs to be written back to DRAM. Alternatively, the XOR operation can be completed using the RAIN background in DRAM. But when all 8 channels are transferring data out, the XOR bandwidth can burst up to 9.6 GB / s, which would require 19.2 GB / s access to DRAM to read and write the RAIN background. If DRAM cannot keep up, then the ONFI bus can be adjusted, which will not affect performance until the ONFI average bandwidth drops below 800 MT / s.

[0057] Figure 5FIG. 500 illustrates eight streams 502 to 516 striping across eight channels to idle dies according to some embodiments of the present invention. Continuing with the example of the 330 KB PU size above, the programmed bandwidth is 115 MB / s (330,192 B / 2862 μs). The minimum average throughput per channel is 644 MB / s (5.15 GB / 8). At least 45 dies must be turned on to meet the 5.15 GB / s or approximately six dies per channel system write bandwidth. As Figure 5 shown, there are eight channels, each channel for accommodating one of the eight streams. The latency 501 between each stream is determined by the time taken to move data from DRAM to SRAM (plus any firmware consumption).

[0058] In one embodiment describing the hierarchical buffer concept, by storing host write data in DRAM, the on-chip SRAM becomes a hierarchical buffer component that can support a very high bandwidth interface to the ONFI backend. The ONFI maximum bandwidth (9.6 Gb / s) is much higher than the host bandwidth (5.15 GB / s - encoded), so the hierarchical buffer component typically empties faster than it fills.

[0059] Figure 6 FIG. 600 illustrates a hierarchical buffer for programming four streams on a single channel according to some embodiments of the present invention. For a general data stream, the firmware can evaluate the current channel / die activity and available PUs. The firmware then selects the next PU for programming. The firmware initiates data encoding and uses NAND operations to program the NAND controller. The encoder starts filling the hierarchical buffer component 602 with stream 0, as Figure 6Explanation at 601. After filling the hierarchical buffer component 602 with stream 0 host data (and optionally associated RAIN data) at 601, the NAND data transfer 603 starts an XOR operation on the first die 604 (die 0) on the first channel (Ch0 NAND). When the data transfer 603 is complete, the stream 0 host buffer can be released. If there are no other stream 0 host writes pending and the RAIN background is in the hierarchical buffer component 602, then the stream 0 RAIN data should be transferred (via the DMA engine) back to the DRAM. After filling the hierarchical buffer component 602 with stream 1 host data (and optionally associated RAIN data) at 603, the NAND data transfer 607 starts an XOR operation on the second die 606 (die 1) on the same channel (Ch0 NAND). When the data transfer 607 is complete, the same stream defined above for die 0 is complete for die 1. After filling the hierarchical buffer component 602 with stream 2 host data (and optionally associated RAIN data) at 609, the NAND data transfer 609 starts an XOR operation on the third die 608 (die 2). When the data transfer 609 is complete, the same stream defined above for die 0 is complete for die 2. After filling the hierarchical buffer component 602 with stream 3 host data (and optionally associated RAIN data) at 611, the NAND data transfer 612 starts an XOR operation on the fourth die 610 (die 3). When the data transfer 612 is complete, the same stream defined above for die 0 is complete for die 3. This can be done in parallel for all remaining dies, as Figure 6 shown. As shown, data can always move through the hierarchical buffer component 602, but the retention period of data in the hierarchical buffer component 602 is short.

[0060] Since data is organized in programming units in the DRAM, the firmware can have PU selection rules and can prevent or reduce channel or die conflicts. When the host system writes to each stream for the same channel / die, performance degrades regardless of the data stream. To minimize conflicts, the firmware can stagger and randomly order stream stripes across channels and dies. If the hierarchical buffer component supports writes to the same channel / die, then performance may be significantly affected, thereby preventing other operations from using the hierarchical buffer component because the stream data will be stored in the hierarchical buffer component for the duration of the ONFI transfer and the operations preceding it. If multiple PUs target the same channel / die, then the hierarchical buffer component can fill up. To avoid filling the hierarchical buffer component when the host overloads writes to one channel or die, the firmware can follow these rules: 1) Limit the per-channel program operation volume to 2; 2) If a PU targets a busy die, then use a cache program operation or save it in the DRAM until the die becomes idle; or the like. Figure 6Show a sequence in which four back-to-back programs for different dies and channel conflicts with the next stream of data can be transferred to the hierarchical buffer component 602 at 619 while the current NAND data transfer is in progress. In one embodiment, about six PUs fill the hierarchical buffer component 602 during the time it takes to transfer 1 PU to the NAND, so up to 3 channels can look alike, as Figure 6 shown. Alternatively, the hierarchical buffer component 602 can be sized to match more or fewer than six PUs.

[0061] Figure 7 is a diagram illustrating a sequence of program operations with simultaneous completion across all eight channels (which is not possible) according to some embodiments of the present invention. The delay 706 labeled E is the time to transfer data to the hierarchical buffer component and configure the NAND controller for the program operation. In this worst-case scenario, all existing channel programs 704 complete simultaneously and all outstanding program operations 704 are not in the hierarchical buffer component.

[0062] In one embodiment, cache programming can be performed to reduce this problem by performing encoding operations and preparations during the previous program operation. It should be noted that cache programming increases the write cache size by effectively adding another data buffer. Such cache program operations should only be used when targeting dies that have already been used by another programming operation. When this case occurs, the channel programs 704 are staggered so that they do not repeat when writing continues (but will eventually become randomly aligned), as Figure 7 shown. The write bandwidth may require six valid dies per channel. Transferring data to six dies at 300 μs per transfer takes 1.8 ms, leaving about 1 ms of idle channel time for every 6 data transfers. Some outstanding program operations 704 may already be in the hierarchical buffer component.

[0063] In one embodiment for sequential write data, a two-level memory buffer can contain 18 programming units with a PU size of 330 KB, and the hierarchical buffer SRAM can be 5804 KB. For example, host data can have eight PUs and will require 2580 KB in SRAM, and RAIN data can have ten PUs and will require 3225 KB in SRAM. For an effective ONFI rate of 1 GB / s, at least six programming units (rounded up to 8) can be buffered at a time. For RAIN data, two additional programming units can be used compared to host data because the XOR operation makes the RAIN data have a longer retention period in the buffer.

[0064] In another embodiment for random write data, the two-level memory buffer can have 24 programming units, which include seven for host data, nine for RAIN data, and eight for GC data. For this embodiment, the SRAM can be 7739 KB, which includes 2257 for host data, 2902 for RAIN data, and 2580 for GC data. For an effective ONFI rate of 1 GB / s, at least five programming units can be buffered at a time (rounded to 7). For RAIN data, two additional programming units can be used compared to host data because the XOR operation makes the retention period of RAIN data in the buffer longer. For 50% effectiveness in GC data, the read data from 8 channels is used with 4 buffers multiplied by 2 depth, decoded, and sent to the DRAM.

[0065] Since random write data uses more PUs, if 8 MB SRAM and 24 PUs are used, then there are six or more PUs for sequential write data. These additional PUs can be used to permanently store 6 RAIN PUs. This can reduce the RAIN background switch BW by about 18%, depending on the host workload. If there is block-level RAIN protection for the stream, then 2 MB can be saved, reducing the SRAM to 6 MB, or the additional 2 MB can be used to save the RAIN background or for other purposes that the firmware deems appropriate.

[0066] Figure 8 An example machine of the computer system 800 can execute a set of instructions within the computer system 800 to cause the machine to perform any one or more of the methodologies discussed herein. In some embodiments, the computer system 800 can correspond to a host system (e.g., Figure 1 host system 120), which includes, is coupled to, or utilizes a memory subsystem (e.g., Figure 1 memory subsystem 110) or can be used to perform the operations of a controller (e.g., execute an operating system to perform the operations corresponding to Figure 1 two-level buffer controller component 113). In alternative embodiments, the machine can be connected (e.g., networked) to other machines in a LAN, intranet, extranet, and / or the Internet. The machine can operate as a server or client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment.

[0067] The machine can be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network device, a server, a network router, a switch, or a bridge, or any machine capable of executing a set of instructions (sequentially or otherwise) that specify actions to be taken by the machine. Further, although a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0068] Example computer system 800 includes a processing device 802, a main memory 804 (such as read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 806 (such as flash memory, static random access memory (SRAM), etc.), and a data storage system 818, which communicate with each other via a bus 830.

[0069] Processing device 802 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or a processor implementing a combination of instruction sets. Processing device 802 can also be one or more special-purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. Processing device 802 is configured to execute instructions 826 for performing the operations and steps discussed herein. Computer system 800 can further include a network interface device 808 to communicate over a network 820.

[0070] Data storage system 818 can include a machine-readable storage medium 824 (also referred to as a non-transitory computer-readable storage medium) on which is stored one or more sets of instructions 826 or software to embody any one or more of the methodologies or functions described herein. The instructions 826 can also reside, completely or at least partially, within main memory 804 and / or within processing device 802 during execution by computer system 800, and main memory 804 and processing device 802 also constitute machine-readable storage media. Machine-readable storage medium 824, data storage system 818, and / or main memory 804 can correspond to Figure 1 memory subsystem 110.

[0071] In one embodiment, the instructions 826 include implementing a corresponding two-level buffer controller component (such as Figure 1instructions for the functionality of the two - stage buffer controller component 113). Although the machine - readable storage medium 824 is shown as a single medium in the example embodiment, the term "machine - readable storage medium" should be regarded as including a single medium or multiple media that store one or more sets of instructions. The term "machine - readable storage medium" should also be regarded as including any medium that is capable of storing or encoding a set of instructions executable by a machine and causing the machine to perform any one or more of the methodologies of the present invention. The term "machine - readable storage medium" should accordingly be regarded as including (but not limited to) solid - state memory, optical media, and magnetic media.

[0072] Some parts of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data - processing art to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived as a self - consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. For the sake of common usage, it is sometimes convenient to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0073] However, it should be borne in mind that all such and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present invention may refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the registers and memories of the computer system into other data similarly represented as physical quantities within the memories or registers or other such information storage systems of the computer system.

[0074] The present invention also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the intended purpose or it may comprise a general - purpose computer selectively activated or reconfigured by a computer program stored in a computer. Such a computer program may be stored in a computer - readable storage medium, such as (but not limited to) any type of disk (including floppy disks, optical disks, CD - ROMs, and magneto - optical disks), read - only memory (ROM), random - access memory (RAM), EPROM, EEPROM, magnetic or optical cards, or any type of media suitable for storing electronic instructions, each coupled to the computer system bus.

[0075] The algorithms and displays presented herein do not inherently relate to any particular computer or other device. Various general-purpose systems may be used in conjunction with the programs according to the teachings herein, or it may prove convenient to construct more specialized devices to perform the methods. The structure of various such systems will appear as set forth in the following description. Additionally, the present invention is not described with reference to any particular programming language. It should be understood that various programming languages may be used to implement the teachings of the present invention described herein.

[0076] The present invention may be provided as a computer program product or software, which may include a machine-readable medium having instructions stored thereon that can be used to program a computer system (or other electronic device) to perform a process according to the present invention. The machine-readable medium includes any mechanism for storing information in a form readable by a machine (such as a computer). In some embodiments, the machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as read-only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory components, and the like.

[0077] In the foregoing description, embodiments of the present invention have been described with reference to specific example embodiments of the present invention. Obviously, various modifications may be made to the present invention without departing from the broader spirit and scope of the embodiments of the present invention as set forth in the appended claims. The specification and drawings should accordingly be regarded as illustrative rather than restrictive.

Claims

1. A system, comprising: One or more external dynamic random access memory (DRAM) devices having a plurality of programming unit buffers including a first programming unit buffer and a second programming unit buffer; An internal static RAM (SRAM) device; One or more non-volatile memory (NVM) devices; And A processing device operably coupled to the one or more external DRAM devices, the internal SRAM device, and the one or more NVM devices to perform operations including: Transferring the first write data from the first programming unit buffer to the internal SRAM device in response to the first write data satisfying a first programming unit (PU) threshold related to a PU size of a first NVM device among the one or more NVM devices; Writing the first write data from the internal SRAM device as a first programming unit to the one or more NVM devices; Transferring the second write data from the second programming unit buffer to the internal SRAM device in response to the second write data satisfying a second PU threshold related to a PU size of a second NVM device among the one or more NVM devices, wherein the second PU threshold is different from the first PU threshold; And Writing the second write data from the internal SRAM device as a second programming unit to the one or more NVM devices.

2. The system according to claim 1, further comprising: A dynamic random access memory (DRAM) interface coupled to the one or more external DRAM devices; And An NVM controller managing the internal SRAM device and the one or more external DRAM devices.

3. The system according to claim 2, further comprising: A low density parity check (LDPC) encoder coupled between the internal SRAM device and the DRAM interface; And An LDPC decoder coupled between the internal SRAM device and the NVM controller.

4. The system according to claim 2, further comprising: An independent NAND redundant array of independent disks (RAIN) exclusive OR (XOR) engine coupled between the internal SRAM device and the NVM controller, the RAIN XOR engine for performing operations including: Generating first RAIN data corresponding to the first write data; Storing the first RAIN data in the internal SRAM device before being transferred to the one or more external DRAM devices via the DRAM interface; Generating second RAIN data corresponding to the second write data; And Storing the second RAIN data in the internal SRAM device before being transferred to the one or more external DRAM devices via the DRAM interface.

5. The system according to claim 2, further comprising: A low density parity check (LDPC) encoder coupled between the internal SRAM device and the DRAM interface, wherein the LDPC encoder is for receiving the first write data via a host interface, encoding the first write data, and storing the first write data in a first logical partition of the internal SRAM device; And An LDPC decoder, which is coupled between the internal SRAM device and the NVM controller, wherein the NVM controller stores garbage collection (GC) data of discarded items in a second logical partition of the internal SRAM device, and wherein the LDPC decoder is configured to receive the GC data from the internal SRAM device, decode the GC data, and send the GC data to the one or more external DRAM devices via the DRAM interface.

6. The system according to claim 1, wherein the first write data is first sequential write data received from a host system, and the second write data is second sequential write data received from the host system.

7. The system according to claim 1, wherein the first write data is sequential write data received from a host system, and the second write data is random write data received from the host system.

8. The system according to claim 1, wherein the one or more external DRAM devices and the internal SRAM device are included in a memory subsystem.

9. A method, comprising: Determine whether first write data in a first programming unit buffer in one or more external dynamic random access memory (DRAM) devices meets a first programming unit (PU) threshold, the first PU threshold being related to a first PU size of a first non-volatile memory (NVM) device in the one or more NVM devices; In response to the first write data meeting the first PU threshold, transfer the first write data from the first programming unit buffer to an internal static random access memory (SRAM) device; Write the first write data from the internal SRAM device as a first programming unit to the first NVM device in the one or more NVM devices; Determine whether second write data in a second programming unit buffer in the one or more external DRAM devices meets a second PU threshold, the second PU threshold being related to a second PU size of a second NVM device in the one or more NVM devices, wherein the second PU threshold is different from the first PU threshold; In response to the second write data meeting the second PU threshold, transfer the second write data from the second programming unit buffer to the internal SRAM device; and Write the second write data from the internal SRAM device as a second programming unit to the second NVM device in the one or more NVM devices.

10. The method according to claim 9, wherein: The one or more external DRAM devices are coupled to a dynamic random access memory (DRAM) interface; and The internal SRAM device and the one or more external DRAM devices are managed by the NVM controller.

11. The method according to claim 10, wherein: A low density parity check (LDPC) encoder is coupled between the internal SRAM device and the DRAM interface; and An LDPC decoder is coupled between the internal SRAM device and the NVM controller.

12. The method according to claim 10, further comprising: Generate first RAIN data corresponding to the first write data by an independent NAND redundancy array (RAIN) exclusive OR (XOR) engine coupled between the internal SRAM device and the NVM controller; Store the first RAIN data in the internal SRAM device before transmitting it to the one or more external DRAM devices via the DRAM interface; Generate second RAIN data corresponding to the second write data by the RAIN XOR engine; and Store the second RAIN data in the internal SRAM device before transmitting it to the one or more external DRAM devices via the DRAM interface.

13. The method according to claim 10, further comprising: The first write data is received via a host interface by a Low-Density Parity-Check (LDPC) encoder coupled between the internal SRAM device and the DRAM interface. The first write data is encoded by the LDPC encoder. And The first write data is stored in a first logical partition of the internal SRAM device.

14. The method according to claim 13, further comprising: Garbage Collection (GC) data is received from the internal SRAM device by an LDPC decoder coupled between the internal SRAM device and the NVM controller. The GC data is decoded by the LDPC decoder. And The GC data is sent by the LDPC decoder to the one or more external DRAM devices via the DRAM interface.

15. The method according to claim 9, wherein the first write data is first sequential write data received from a host system, and the second write data is second sequential write data received from the host system.

16. The method according to claim 9, wherein the first write data is sequential write data received from a host system, and the second write data is random write data received from the host system.

17. A non-transitory computer-readable medium including instructions that, when executed by a processing device, cause the processing device to perform operations including the following: Determine whether first write data in a first programming unit buffer in one or more external dynamic random access memory (DRAM) devices meets a first programming unit (PU) threshold, where the first PU threshold is related to a first PU size of a first non-volatile memory (NVM) device in the one or more NVM devices; In response to the first write data meeting the first PU threshold, transfer the first write data from the first programming unit buffer to an internal static random access memory (SRAM) device; The first write data from the internal SRAM device is written as a first programming unit to the first NVM device among the one or more NVM devices. Determine whether second write data in a second programming unit buffer in the one or more external DRAM devices meets a second PU threshold, where the second PU threshold is related to a second PU size of a second NVM device among the one or more NVM devices, and the second PU threshold is different from the first PU threshold. In response to the second write data meeting the second PU threshold, transfer the second write data from the second programming unit buffer to the internal SRAM device. And The second write data from the internal SRAM device is written as a second programming unit to the second NVM device among the one or more NVM devices.

18. The non-transitory computer-readable medium according to claim 17, wherein: The one or more external DRAM devices are coupled to a Dynamic Random Access Memory (DRAM) interface; and The internal SRAM device and the one or more external DRAM devices are managed by an NVM controller.

19. The non-transitory computer-readable medium according to claim 18, wherein: A Low-Density Parity-Check (LDPC) encoder is coupled between the internal SRAM device and the DRAM interface; and An LDPC decoder is coupled between the internal SRAM device and the NVM controller.

20. The non-transitory computer-readable medium according to claim 18, further comprising: A Redundant Array of Independent NAND (RAIN) Exclusive-OR (XOR) engine coupled between the internal SRAM device and the NVM controller generates first RAIN data corresponding to the first write data. The first RAIN data is stored in the internal SRAM device before being transmitted to the one or more external DRAM devices via the DRAM interface. The RAIN XOR engine generates second RAIN data corresponding to the second write data. And The second RAIN data is stored in the internal SRAM device before being transmitted to the one or more external DRAM devices via the DRAM interface.

Images