Progressive, low-cost disk redundancy arrays (RAID) for memory devices

Abstract

This application relates to storing data by distributing redundant data blocks across multiple memory devices (e.g., solid-state drives (SSDs)) based on multiple RAID schemes. The electronic system mirrors user data to two separate memory devices. Integrity data is generated based on the user data and stored in the integrity memory device. Upon determination that the integrity data of the user data has been stored in the integrity memory device, the electronic system de-mirrors the user data on at least one of the two separate memory devices. In some embodiments, a copy of the user data is also stored in one of the multiple data memory devices. Integrity data is generated based on the user data and stored in the integrity memory device upon determination that the multiple data memory devices have been filled.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application is a continuation of, and claims priority to, U.S. Patent Application No. 18 / 227256, entitled "Progressive Low - Cost Disk Redundant Array (RAID) for Memory Devices," filed on July 27, 2023, the entire content of which is incorporated herein by reference.

[0002] This application generally relates to memory management, including but not limited to methods, systems, and non - transient computer - readable media for storing data by dispersing redundant data blocks across multiple memory devices (e.g., multiple memory dies within a solid - state drive (SSD)).

Background Art

[0003] In computer systems, memory is used to store instructions and data. Data is processed by one or more processors in the computer system according to instructions stored in memory. Different parts of a computer system utilize multiple memory units that perform different functions. Specifically, a computer system includes non-volatile memory, which functions as a secondary memory device to retain stored data when the computer system's power is cut off. Examples of secondary memory include, but are not limited to, hard disk drives (HDDs) and solid-state drives (SSDs). Secondary memory in computer systems often employs inexpensive disk redundant arrays (RAID), a data storage virtualization technique that combines multiple physical storage devices into one or more logical units for data redundancy and optimized performance. RAID can be configured at different levels within an SSD, improving performance levels and data security. The Storage Networking Industry Association (SNIA) standardized RAID levels and their associated data formats. Data center SSDs typically employ block-level striping with dedicated parity bits (RAID 4) or distributed parity bits (RAID 5) to meet uncorrectable bit error rate (UBER) requirements. However, RAID 4 or RAID 5 are not effectively applicable in some replacement modes that require many open isolation units with frequent asynchronous updates within the XOR strip. To store data involving many open isolation units, it is beneficial to apply practical data storage and verification mechanisms. [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] Various embodiments of this application relate to methods, systems, devices, and non-transient computer-readable media for storing data by distributing redundant data blocks across multiple memory devices (e.g., NAND dies) based on multiple RAID schemes (e.g., RAID 1, RAID 4). A first level of RAID, including RAID 1, is applied to buffer data in response to host writes. This data is mirrored across two separate memory devices (e.g., two NAND dies). A copy of this data is written to a first isolation unit (e.g., a memory block in another memory device). If the memory system fully fills one or more isolation units, including the first isolation unit, the memory system generates integrity data for the data stored in the first isolation unit in a batch and stores this integrity data according to a second level of RAID (e.g., RAID 4 or RAID 5). After the integrity data has been stored, at least one copy of the data stored according to the first level of RAID is invalidated or deactivated. RAID 1 is applied to store data with asynchronous writes that have a fine data granularity level, while RAID 4 and RAID 5 are applied to store data protected by integrity data in bulk. In exemplary configuration modes, RAID 1 provides better performance with a finer data granularity level while utilizing more temporary storage space. Conversely, RAID 4 and RAID 5 save storage space that would be wasted in RAID 1 for data replication, but they cannot provide the same data granularity level as RAID 1. In various embodiments of this application, the two levels of RAID schemes can be used in combination to benefit from both the fine data granularity level of RAID 1 and the high storage space utilization of RAID 4 or RAID 5, thereby meeting the UBER requirements of a memory system at a reasonable hardware cost. [Means for solving the problem]

[0005] In one embodiment, a method for storing data in a memory system (e.g., including multiple memory channels) is implemented in an electronic system. This method includes mirroring user data to two separate memory devices, generating integrity data based on the user data, and storing the integrity data of the user data in an integrity memory device. This method further includes releasing the mirrored user data from at least one of the two separate memory devices upon determination that the integrity data of the user data has been stored in the integrity memory device. In some embodiments, the two separate memory devices and the integrity memory device correspond to different NAND dies of an SSD.

[0006] In some embodiments, the memory zone includes a memory block of an integrity memory device and a plurality of memory blocks of a plurality of data memory devices, including a first data memory device. The method further includes storing a copy of user data in the first data memory device, and the integrity data is generated based on a subset of user data in each subset of data memory devices, including the first data memory device. Furthermore, in some embodiments, the method further includes determining whether one or more memory blocks of the plurality of data memory devices in the memory zone are filled. Integrity data of user data is generated based on the user data and stored in the integrity memory device, according to the determination that one or more memory blocks of the plurality of data memory devices are filled. Also in some embodiments, the method further includes updating the integrity data to exclude the copy of user data stored in the first data memory device from the subset of user data applied to generate the integrity data, and erasing the copy of user data from the first data memory device by modifying a logical-to-physical (L2P) table to disassociate the physical address of the first data memory device where the copy of user data was stored from the corresponding logical address associated with the user data.

[0007] Some implementations of this application include an electronic device that includes one or more processors and a memory that, when executed by one or more processors, stores instructions causing the processors to perform any of the above methods in a memory system (e.g., one or more SSDs).

[0008] Some implementations include a non-temporary, computer-readable storage medium for storing one or more programs. When executed by one or more processors, one or more programs include instructions that cause the processors to perform one of the above methods on a memory system (e.g., one or more SSDs).

[0009] These exemplary embodiments and implementations are mentioned not to limit or define this disclosure, but to provide examples for ease of understanding. Additional embodiments are described in embodiments for carrying out the invention, where further details are provided.

[0010] To gain a better understanding of the various implementations described, please refer to the following "Modes for Carrying Out the Invention" in conjunction with the following drawings, where similar reference numbers refer to corresponding parts throughout the drawings. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram of an exemplary system module in a typical electronic device, according to several embodiments. [Figure 2] This is a block diagram of an exemplary electronic device memory system having one or more memory access queues, according to several embodiments. [Figure 3] An exemplary progressive RAID scheme that integrates two RAID schemes (e.g., RAID 1 and RAID 4) according to several embodiments is shown. [Figure 4] Another exemplary progressive RAID scheme that integrates two RAID schemes, according to several embodiments, is shown. [Figure 5]This is a flowchart illustrating an exemplary data storage method in a memory system according to several embodiments. [Modes for carrying out the invention]

[0012] Throughout the drawings, similar reference numbers indicate corresponding parts.

[0013] The following descriptions refer in detail to specific embodiments illustrated in the accompanying drawings. The following embodiments for carrying out the invention include numerous non-limiting specific details to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives can be used without departing from the claims, and that the subject matter can be carried out without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented in a variety of electronic devices having digital video capabilities.

[0014] Figure 1 is a block diagram of an exemplary system module 100 in a typical electronic device according to several embodiments. The system module 100 in this electronic device includes at least a processor module 102, a memory module 104 for storing programs, instructions, and data, an input / output (I / O) controller 106, one or more communication interfaces such as a network interface 108, and one or more communication buses 140 for interconnecting these components. In some embodiments, the I / O controller 106 enables the processor module 102 to communicate with I / O devices (e.g., keyboards, mice, or trackpads) via a universal serial bus interface. In some embodiments, the network interface 108 includes one or more interfaces for Wi-Fi, Ethernet, and Bluetooth® networks, respectively, enabling the electronic device to exchange data with external sources such as servers or other electronic devices. In some embodiments, the communication bus 140 includes circuitry (sometimes called a chipset) that interconnects the various system components included in the system module 100 and controls communication between them.

[0015] In some embodiments, the memory module 104 includes high-speed random access memory such as DRAM, static random access memory (SRAM), double data rate (DDR) dynamic random access memory (RAM), or other random access solid-state memory devices. In some embodiments, the memory module 104 includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments, the memory module 104 or the non-volatile memory devices within the memory module 104 includes a non-transient computer-readable storage medium. In some embodiments, the system module 100 is provided with a memory slot for receiving the memory module 104. When inserted into the memory slot, the memory module 104 is integrated into the system module 100.

[0016] In some embodiments, the system module 100 further includes one or more components selected from a memory controller 110, one or more solid-state drives (SSDs) 112, a hard disk drive (HDD) 114, a power management integrated circuit (PMIC) 118, a graphics module 120, and a sound module 122. The memory controller 110 is configured to control communication between the processor module 102 and the memory components, including the memory module 104, within the electronic device. In many embodiments, the SSD 112 is configured to apply an integrated circuit assembly to store data in the electronic device based on a NAND or NOR memory configuration. The HDD 114 is a conventional data storage device for storing and retrieving digital information based on an electromechanical magnetic disk. A power connector 116 is electrically connected to receive an external power supply. The PMIC 118 is configured to modulate the received external power supply to other desired DC voltage levels, such as 5V, 3.3V, or 1.8V, depending on the requirements of various components or circuits within the electronic device (e.g., the processor module 102). The graphics module 120 is configured to generate a feed of output images to one or more display devices according to a desired image / video format. The sound module 122 is configured to facilitate the input and output of audio signals to and from electronic devices under the control of a computer program.

[0017] Furthermore, the communication bus 140 interconnects various system components, including components 110 to 122, and controls communication between them.

[0018] Furthermore, those skilled in the art will see that as new data storage technologies are developed for storing information in non-temporary computer-readable storage media within memory modules 104 and SSDs 112, other non-temporary computer-readable storage media may be used. These new non-temporary computer-readable storage media include, but are not limited to, those manufactured from biological materials, nanowires, carbon nanotubes, and individual molecules, for which the respective data storage technologies are currently under development and not yet commercialized.

[0019] Some implementations of this application are oriented towards storing data by applying multiple RAID schemes (e.g., RAID 1 and RAID 4) which involve temporarily distributing redundant data blocks across multiple memory devices (e.g., one or more SSDs, each containing multiple NAND dies). The two levels of RAID schemes are used in combination to benefit from both the fine data granularity level of RAID 1 and the high storage space utilization of RAID 4 or RAID 5. Such integration of multiple RAID schemes can meet the UBER requirements of memory systems at a reasonable hardware cost. Specifically, the first level of RAID includes RAID 1 and is applied to buffer data in response to host writes. This data is mirrored across two separate memory devices (e.g., two NAND dies). Based on the second level of RAID (e.g., RAID 4 or RAID 5), copies of the data are written to the first isolation unit (e.g., a memory block of the data memory device). If the memory system fully fills one or more isolation units, including the first isolation unit, the memory system generates integrity data for the data stored in the first isolation unit in a batch and stores the integrity data according to the second level of RAID. After the integrity data is stored, at least one copy of the data stored according to the first level of RAID is invalidated or removed.

[0020] RAID is a virtualization technology for data storage that combines multiple physical storage devices into single or multiple logical units for data redundancy and optimized performance. The Storage Networking Industry Association (SNIA) has standardized the data formats associated with RAID levels (e.g., RAID 0, …, RAID 6). For example, RAID 0 is implemented based on striping and does not have data mirroring or parity. In RAID 1, data is mirrored on two separate memory devices (e.g., NAND dies). RAID 4 is implemented based on block-level striping with a dedicated parity disk. RAID 5 is implemented based on block-level striping with distributed parity.

[0021] FIG. 2 is a block diagram of a memory system 200 of an exemplary electronic device having one or more memory access queues, according to some embodiments. The memory system 200 is coupled to a host device 220 (e.g., the processor module 102 in FIG. 1) and is configured to store instructions and data over a long period of time, for example when the electronic device is in a sleep state, a standby state, or a shutdown state. The host device 220 is configured to access instructions and data stored in the memory system 200, process the instructions and data to execute an operating system, and execute user applications. The memory system 200 further includes a controller 202 and a plurality of memory channels 204. Each memory channel 204 includes a plurality of memory cells. The controller 202 is configured to execute firmware-level software to bridge the plurality of memory channels 204 to the host device 220.

[0022] Each memory channel 204 includes one or more memory packages 206 (e.g., two memory chips, two memory dies). In one example, each memory package 206 corresponds to a memory die. Each memory package 206 includes multiple memory planes 208, each memory plane 208 further includes multiple memory pages 210. Each memory page 210 includes an ordered set of memory cells, each memory cell identified by its own physical address. In some embodiments, the memory system 200 includes a single-level cell (SLC) die, where each memory cell stores a single data bit. In some embodiments, the memory system 200 includes a multi-level cell (MLC) die, where each memory cell stores two data bits. In one example, each memory cell in a triple-level cell (TLC) die stores three data bits. In another example, each memory cell in a quad-level cell (QLC) die stores four data bits. In yet another example, each memory cell in a penta-level cell (PLC) die stores five data bits. In some embodiments, each memory cell can store any appropriate number of data bits. Compared to non-SLC dies (e.g., MLC dies, TLC dies, QLC dies, PLC dies), SLC dies operate at high speed, high reliability, and long lifespan, but have lower device density and higher cost.

[0023] Each memory channel 204 is coupled to a respective channel controller 214 configured to control internal and external requests to access memory cells within each memory channel 204. In some embodiments, the memory package 206 (e.g., each memory die) corresponds to a respective queue 216 of memory access requests. In some embodiments, each memory channel 204 corresponds to a respective queue 216 of memory access requests. Also, in some embodiments, each memory channel 204 corresponds to a separate and distinct queue 216 of memory access requests. In some embodiments, a subset (but not all) of the plurality of memory channels 204 corresponds to a separate queue 216 of memory access requests. In some embodiments, all of the plurality of memory channels 204 of the memory system 200 correspond to a single queue 216 of memory access requests. Each memory access request is optionally received from within the memory system 200 to manage each memory channel 204 or from outside the host device 220 to write to or read data stored in each channel 204. Specifically, each memory access request includes either a system write request received from the memory system 200 to write to each memory channel 204, a system read request received from the memory system 200 to read from each memory channel 204, a host write request transmitted from the host device 220 to write to each memory channel 204, and a host read request received from the host device 220 to read from each memory channel 204. Note that system read requests (also called background read requests or non-host read requests) and system write requests are issued by the memory controller to implement internal memory management functions including, but not limited to, garbage collection, wear leveling, read disturb reduction, memory snapshot capture, memory mirroring, caching, and memory sparing.

[0024] In some embodiments, the controller 202 further includes, in addition to the channel controller 214, a local memory processor 218, a host interface controller 222, an SRAM buffer 224, and a DRAM controller 226. The local memory processor 218 accesses multiple memory channels 204 based on one or more queues 216 of memory access requests. In some embodiments, the local memory processor 218 writes to and reads to and from the multiple memory channels 204 in memory block units. Data in one or more memory blocks is jointly written to or read from multiple channels. Data within the same memory block is not written in more than one operation. Each memory block optionally corresponds to one or more memory pages. In one example, the size of each memory block jointly written to or read from multiple memory channels 204 is 16KB (e.g., 1 memory page). In another example, the size of each memory block jointly written to or read from multiple memory channels 204 is 64KB (e.g., 4 memory pages). In some embodiments, each page contains 16KB of user data and 2KB of metadata. Furthermore, the number of memory blocks accessed jointly and the size of each memory block are configurable for each of the system read, host read, system write, and host write operations.

[0025] In some embodiments, the local memory processor 218 stores the data written to or read from each memory block in the multiple memory channels 204 in the SRAM buffer 224 of the controller 202. Alternatively, in some embodiments, the local memory processor 218 stores the data written to or read from each memory block in the multiple memory channels 204 in the DRAM buffer 228, which is the main memory used by the processor module 102 (Figure 1). The local memory processor 218 of the controller 202 accesses the DRAM buffer 228 via the host interface controller 222.

[0026] In some embodiments, the memory system 200 includes one or more SSDs, each SSD including a logical-to-physical (L2P) address mapping table 212 (also called L2P table 212) that stores physical addresses to a set of logical addresses, such as logical block addresses (LBAs). In one example, the SSD has a memory capacity of 32 terabytes (i.e., 32 TB) organized into multiple memory sectors, each memory sector storing 4096 bytes (i.e., 4 KB) and individually addressable. The SSD includes 8 billion memory sectors identified by 8 billion physical addresses. At least 33 data bits are required to uniquely represent each individual physical address of the SSD having 8 billion physical addresses. Furthermore, in some embodiments, the SSD includes NAND memory cells, with extra memory space reserved by over-provisioning. For example, the over-provisioning is 25%, and the SSD has 10 billion memory sectors identified by 10 billion physical addresses. To uniquely identify each individual physical address in an SSD with 10 billion physical addresses, at least 34 data bits are required.

[0027] In some embodiments, data in the memory system 200 is grouped into coded blocks, each coded block being called a codeword (for example, the combination of user data 302C, 318A, and 318B in Figure 3). For example, each codeword contains n bits, of which k bits correspond to user data and m bits correspond to integrity data of the user data, where k, m, and n are integers, and n is the sum of k and m. In some embodiments, the memory system 200 includes an integrity engine 230 (e.g., an LDPC engine) and a register file 232, which includes a register arrangement and is linked to the integrity engine 230. The integrity engine 230 is linked to the memory channel 204 via a channel controller 214 and an SRAM buffer 224. Specifically, in some embodiments, the integrity engine 230 has a data path connected to the SRAM buffer 224, which is further connected to the channel controller 214 via a data path controlled by a local memory processor 218. The integrity engine 230 verifies data integrity for each encoded block in the memory channel 204 using variable nodes and check nodes, and messages are exchanged between variable nodes and check nodes during the integrity check process. A subset of these messages is selected and temporarily stored in the register file 232 as variable node data or check node data.

[0028] Figure 3 shows an exemplary progressive RAID scheme 300 that integrates two RAID schemes according to several embodiments. The host device 220 is coupled to a memory system 200 which includes a plurality of memory devices. Examples of memory devices include HDDs or SSDs. Each memory device includes a plurality of memory channels 204, each further containing a plurality of memory planes 208, and each of the plurality of memory planes 208 further contains a plurality of memory pages 210. In some embodiments, each memory channel 204 includes one or more separate memory dies. In some embodiments, user data 302 is mirrored across two separate memory devices 304, 306 to create two copies 302A, 302B of the user data. Integrity data 308 is generated based on the user data 302 and stored in an integrity memory device 310. In accordance with the determination that the integrity data 308 of user data 302 has been stored in the integrity memory device 310, the user data 302 mirrored on at least one of the two separate memory devices 304, 306 is released. In some embodiments, the host device 220 sends host write requests to multiple memory devices. In response to the host write requests, the user data 302 is mirrored on the two separate memory devices 304, 306. Furthermore, in some embodiments, in response to a host write request received from the host device 220, another copy 302C of the user data is stored in the first data memory device 312-1.

[0029] In some embodiments, user data 302 is mirrored to two separate memory devices 304, 306 and then copied to a first data memory device 312-1, and integrity data 308 is generated based on the copy of user data 302C in the first data memory device 312-1. Specifically, the copy of user data 302C is stored in a memory block of the first data memory device 312-1, which together with one or more additional data memory devices 312 (e.g., 312-2, 312-3) and a memory block of the integrity memory device 310 forms a memory zone 314. In other words, in some embodiments, the memory zone 314 includes a memory block 310A of the integrity memory device 310 and multiple memory blocks of multiple data memory devices 312, including the first data memory device 312-1. Each memory block includes multiple memory pages 210 (Figure 2) of the respective memory device 310 or 312. A copy of the user data 302C is stored in the first data memory device 312-1. Integrity data 308 is generated based on the subset of user data in each subset of the data memory device 312, including the first data memory device 312-1. In one example, integrity data 308 is generated based on copies 302C of user data 318A and 318B stored in data memory devices 312-2 and 312-3. In some embodiments, integrity data 308 is generated when at least one memory block of the data memory device 312 in memory zone 314 is filled or closed for further writing. In other words, integrity data 308 is often not generated immediately after the copy of the user data 302C is stored in the first data memory device 312-1.

[0030] In some embodiments, after integrity data 308 is stored in integrity memory device 310, both copies 302A and 302B in two separate memory devices 304 and 306 are invalidated and deactivated. Alternatively, in some embodiments, the first data memory device 312-1 includes one of the two separate memory devices 304 and 306. User data 302 is duplicated in the two separate memory devices 304 and 306 without being stored in an additional, separate, distinct first data memory device. After integrity data 308 is stored in integrity memory device 310, one copy 302A or 302B of the user data stored in memory device 304 or 306 is invalidated and deactivated, and the other copy 302B or 302A of the user data stored in memory device 306 or 304 is used as copy 302C of the user data in the first data memory device 312-1.

[0031] In some embodiments, if a copy of user data 302C is erased from the first data memory device 312-1, the corresponding integrity data 308 is updated to exclude the copy of user data 302C stored in the first data memory device 312-1 from the subset of user data applied to generate the integrity data 308. For example, the integrity data 308 includes a parity check result generated using XOR logic based on the copy of user data 302C stored in memory device 312-1 and the user data 318A, 318B stored in memory devices 312-2, 312-3. The memory system 200 updates the integrity data 308 in response to a request to erase, disable, or release the copy of user data 302 in memory device 312-2 by updating the parity check result using XOR logic based on the user data 318A, 318B stored in memory devices 312-2, 312-3. The L2P table 212 is modified by the memory controller 202 to disassociate the physical address of the first data memory device 312-1, where the user data copy 302C is stored, from the corresponding logical address associated with the user data 302. In some embodiments, the user data copy 302C is physically purged from the first data memory device 312-1. Alternatively, the user data copy 302C remains in the first data memory device 312-1 until it is overwritten by the next data 320. The user data copy 302C is inaccessible because the L2P table does not link its physical address to any logical address. Furthermore, the next data 320 may be written to the first data memory device 312-1 instead of the user data copy 302C. The L2P table is modified to associate the physical address of the first data memory device 312-1 with the next logical address associated with the next data 320. The integrity data 308 is updated based on the next data 320.

[0032] In some embodiments, two separate memory devices 304, 306 have the same memory type. For example, each of the two separate memory devices 304, 306 includes a QLC memory die and has a plurality of memory blocks (e.g., 304A, 304B, 306A, 306B), and each memory block includes a plurality of memory pages 210, each containing a plurality of QLC memory cells. Alternatively, in some embodiments, each of the two separate memory devices 304, 306 includes an SLC memory die and has a plurality of memory blocks (e.g., 304A, 304B, 306A, 306B), and each memory block includes a plurality of memory pages 210, each containing a plurality of SLC memory cells. Alternatively, in some embodiments, each of the two separate memory devices 304, 306 includes an MLC memory die and has a plurality of memory blocks (e.g., 304A, 304B, 306A, 306B), each memory block containing a plurality of memory pages 210, each containing a plurality of MLC memory cells. Alternatively, in some embodiments, each of the two separate memory devices 304, 306 includes a storage class memory (SCM) selected from the group consisting of phase-change memory (PCM), resistive random-access memory (ReRAM), magnetoresistive random-access memory (MRAM), and 3D XPoint memory. The SCM is a type of physical computer memory that combines DRAM, NAND flash memory, and a power supply for data persistence. In some embodiments, the SCM treats non-volatile memory as DRAM and includes it in the server's memory space.

[0033] Alternatively, in some embodiments, two separate memory devices 304, 306 have different memory types. For example, memory device 304 includes a QLC-based memory die and is used as a first data memory device 312-1 that stores a copy of user data 302C. Memory device 306 is optionally based on SLC, MLC, or SCM. User data 302B copied to memory device 306 is released or erased according to the determination that the integrity data 308 (e.g., parity data) of user data 302 has been stored in the integrity memory device 310.

[0034] In some embodiments, the integrity memory device 310 includes either an MLC memory die or an SLC memory die and has a plurality of memory blocks (e.g., 310A, 310B), each memory block containing a plurality of memory pages, each containing a plurality of MLC or SLC memory cells. In one example, RAID 5 is applied to generate integrity data 308, and the integrity memory device 310 storing the integrity data 308 includes SLC memory cells, thereby benefiting from the durability of the SLC memory cells. The integrity memory device 310 may not include TLC and QLC memory cells or higher-level memory cells if the durability level of TLC or QLC is relatively low. Alternatively, in some embodiments, the integrity memory device 310 includes an SCM selected from the group consisting of PCM, ReRAM, MRAM, and 3D XPoint memory. In some embodiments, the data memory device 312 and the integrity memory device 310 have the same memory type.

[0035] In some embodiments, each of the two separate memory devices 304, 306 and the integrity memory device 310 includes at least one separate memory die of the memory system 200.

[0036] In some embodiments (Figure 3), integrity data 308 is stored according to a RAID 4 scheme in which the integrity data generated for the memory blocks of the data memory device 312 is integrated and stored, for example, in a dedicated integrity memory device 310. Alternatively, in some embodiments, integrity data 308 is stored according to a RAID 5 scheme in which the integrity data generated for the memory blocks of the data memory device 312 is distributed and stored. The memory blocks of integrity data are evenly distributed across memory devices 310 and 312. For example, integrity data is stored in memory blocks 310A, 312A, 312B, and 312C, respectively, based on user data stored in three corresponding memory blocks of user data.

[0037] Figure 4 shows another exemplary progressive RAID scheme 400 that integrates two RAID schemes according to several embodiments. In one example, user data 302 is stored based on RAID 1 and RAID 4 schemes. In another example, user data 302 is stored based on RAID 1 and RAID 5 schemes. In some embodiments, user data 302 is stored based on a first RAID scheme and a second RAID scheme, the first RAID scheme being applied temporarily and having a finer data granularity level than the second RAID scheme. User data 302 is mirrored onto two separate memory devices 304, 306 to create two copies 302A, 302B of user data 302. Integrity data 308 is generated based on user data 302 and stored in an integrity memory device 310. In accordance with the determination that the integrity data 308 of user data 302 has been stored in the integrity memory device 310, the user data 302 mirrored on at least one of the two separate memory devices 304, 306 is released. In some embodiments, a copy 302C of user data 302 is stored in the first data memory device 312-1, and in accordance with the determination that the integrity data 308 of user data 302 has been stored in the integrity memory device 310, the user data 302 mirrored on both of the two separate memory devices 304, 306 is released. Furthermore, in some embodiments, memory device 304 or 306 is released by disassociating the physical address of memory device 304 or 306 from the corresponding logical address, although memory device 304 or 306 may or may not be physically purged.

[0038] In other words, copies 302A and 302B of user data are temporarily stored in a replication manner until integrity data 308 is generated and stored for the same copy 302C of user data. The generation of integrity data 308 is delayed compared to the replication of copies 302A and 302B of user data because it must wait until the memory block 302 containing the user data corresponding to the integrity data 308 is filled or closed. In some embodiments, the memory zone 314 includes one or more data memory devices 312 (e.g., 312-1, 312-2, 312-3) and memory blocks 312A-1, 312-2A, 312-3A, 310A of the integrity memory device 310, each memory block containing multiple memory pages 210. Integrity data 308 may not be generated immediately after the copy 302C of user data is stored in the first data memory device 312-1. Instead, integrity data 308 is generated when at least one memory block 312-1A, 312-2A, or 312-3A of the data memory device 312 in memory zone 314 is filled or closed for further writing. For example, the memory controller 202 determines that memory block 312-1A, which contains a copy 302C of user data 302, has been filled or closed for further writing, and that memory block 310A of integrity data has been generated and stored based on memory block 312-1A of user data. Furthermore, in some embodiments, memory block 312-2A is neither filled / closed nor used to generate memory block 310A of integrity data. Alternatively, additionally, in some embodiments, memory blocks 312-2A, 312-3A are filled and used together with memory block 312-1A to generate memory block 310A of integrity data.

[0039] In other words, in some embodiments, multiple data blocks 402A-402C of the data file 402 are stored in multiple data memory devices (e.g., 304, 306, 312-1, 312-2, 312-3, 310). The multiple data blocks 402A-402C include a first data block 402A which further contains user data 302. The first data block 402A is mirrored in two separate memory devices 304, 306, and the first data block 402A is stored in a first data memory device 312-1 of multiple data memory devices 312. Furthermore, in some embodiments, the multiple data blocks 402A-402C of the data file 402 are stored according to a predetermined low-cost disk redundancy array (RAID) level. The predetermined RAID level is selected from RAID 4 and RAID 5, and the first data block of the data file is stored in the first data memory device 312-1. User data 302 is mirrored to two separate memory devices 304 and 306 according to RAID 1.

[0040] In some embodiments, user data 302 includes first user data. Data file 402 further includes second user data 404. The second user data 404 is mirrored to two separate corresponding memory devices, distinct from the multiple data memory devices 312, according to RAID 1. In some embodiments, the second user data 404 is mirrored to two separate memory devices 304, 306 before integrity data 308 is generated based on the first user data 302 and stored in integrity memory device 310. In some embodiments, both user data 302, 404 are stored in memory block 312-1A of the first data memory device 312-1, and when memory block 312-1A is filled or closed, memory block 310A of integrity data is generated.

[0041] Alternatively, in some embodiments, the second user data 404 is mirrored across two separate memory devices 304, 306 after the integrity data 308 is generated based on the first user data 302 and stored in the integrity memory device 310. The user data 302, 404B are stored separately in memory blocks 312-1A, 312-2A of two separate data memory devices 312-1, 312-2. Regardless of whether memory blocks 312-2A, 312-3A are filled or closed, when memory block 312-1A is filled or closed, the integrity data memory block 310A is generated. After memory block 312-1A is filled or closed, the integrity data memory block 310A is updated based on memory block 312-1A. Specifically, if memory block 312-1A is filled or closed, memory blocks 312-2A and 312-3A may not be filled or closed, and memory block 310A containing integrity data may be generated based on memory block 312-1A. If second user data 404B is written to memory block 312-2A, and then memory block 312-2A is filled or closed, memory block 312-3A may not be filled or closed, and memory block 310A containing integrity data may be generated or updated based on the data stored in memory blocks 312-1A and 312-2A.

[0042] Alternatively, if memory block 312-1A is filled or closed after a copy 302C of user data 302 has been written, memory blocks 312-2A and 312-3A are already filled or closed, and memory block 310A of integrity data is generated based on the data stored in memory blocks 312-1A, 312-2A, and 312-3A. If memory block 312-2A is disabled or released, memory block 310A of integrity data is generated or updated based on the data stored in memory blocks 312-1A and 312-3A, regardless of whether the corresponding user data has been physically purged from memory block 312-2A. If the second user data 404B is written to memory block 312-2A, and then memory block 312-2A is filled or closed, the integrity data memory block 310A is generated or updated based on the data stored in memory blocks 312-1A, 312-2A, and 312-3A.

[0043] In some embodiments, user data 406 is stored in memory block 312-1A of the first data memory device 312-1 and corresponds to integrity data 408 stored in memory block 310A of the integrity memory device 310. If user data 406 is erased from the first data memory device 312-1, the integrity data 408 is updated to exclude the user data 406 stored in the first data memory device 312-1 from the subset of user data applied to generate the integrity data 408 (operation 410). The L2P table 212 is updated to disassociate the physical address of the first data memory device 312-1 where the user data 406 was stored from the corresponding logical address associated with the user data 406 (operation 412). Furthermore, in some embodiments, instead of user data 406, next data 414 is written to the first data memory device 312-1, and the L2P table 212 is updated to associate the physical address of the first data memory device 312-1 with the next logical address associated with next data 414 (operation 416). Integrity data 408 is also updated based on next data 414 (operation 418). In some embodiments, user data 406 includes a copy 302C of user data 302 and is replaced with next data 320 (Figure 3). This updates the integrity data 308 corresponding to the copy 302C of user data 302.

[0044] In some embodiments, data is stored as redundant data blocks and distributed across multiple memory devices (e.g., NAND dies) based on multiple RAID schemes (e.g., RAID 1 and RAID 4). The first level of RAID includes RAID 1 and is applied to buffer data in response to host writes. This data is mirrored across two separate memory devices (e.g., NAND dies). A copy of this data is written to the first isolation unit 312-1A. When the memory system fully fills one or more isolation units (e.g., 312-1A, 312-2A, and / or 312-3A) that form a memory zone 314 including the first isolation unit, the memory system generates a memory block 310A of integrity data of the data stored in the first isolation unit in a batch, and stores the integrity data according to a second level of RAID (e.g., RAID 4 or RAID 5). RAID 1 is applied to store data with asynchronous writes at a fine data granularity level, while RAID 4 and RAID 5 are applied to store data protected by integrity data when integrity data is generated at the memory block level. After integrity data is stored, at least one copy of the data stored according to the first level of RAID is invalidated or deactivated. Only one copy of the user data is stored together with the corresponding integrity data. Memory space used to store a second copy of the user data is saved, but data integrity is maintained. In this way, RAID 1 is applied at a fine data granularity level, while RAID 4 or RAID 5 is applied at a large data block level, thereby saving space in the memory system 200 without compromising data integrity.

[0045] Figure 5 is a flowchart of an exemplary data storage method 500 in a memory system according to several embodiments. This method 500 is implemented in an electronic system including a memory system 200 (Figure 2), which optionally includes a plurality of memory devices (e.g., NAND dies). The electronic system mirrors user data 302 to two separate memory devices 304, 306 (operation 502) (Figure 3), and generates integrity data 308 based on the user data 302 (operation 504). The integrity data 308 of the user data 302 is stored in an integrity memory device 310 (operation 506). Upon determining that the integrity data 308 of the user data 302 has been stored in the integrity memory device 310, the electronic system releases the mirrored user data 302 from at least one of the two separate memory devices 304, 306 (operation 508).

[0046] In some embodiments, the memory zone 314 (Figure 3) includes a memory block of the integrity memory device 310 (e.g., 310A) and multiple memory blocks of multiple data memory devices 312, including the first data memory device 312-1 (e.g., 312-1A, 312-2A, 312-3A) (operation 510). The electronic system stores a copy 302C of user data 302 in the first data memory device 312-1 (operation 512). Integrity data 308 is generated based on a subset of user data in each subset of data memory devices 312, including the first data memory device 312-1. Furthermore, in some embodiments, the electronic system determines whether one or more memory blocks of the multiple data memory devices 312 in the memory zone 314 are filled (operation 514). Integrity data 308 of user data 302 is generated based on user data 302 according to the determination that one or more memory blocks (e.g., memory blocks 312-1A, 312-B, or 312-C) of a plurality of data memory devices 312 have been filled, and is stored in the integrity memory device 310. Furthermore, in some embodiments, the memory system receives a host write request. User data 302 is mirrored in two separate memory devices 304, 306, and a copy 302C of user data 302 is stored in the first data memory device 312-1 in response to the host write request.

[0047] In some embodiments, the electronic system further removes the copy 302C of user data 302 stored in the first data memory device 312-1 from the subset of user data 302 applied to update the integrity data 308 and generate the integrity data 308 (operation 518), and erases the copy 302C of user data 302 from the first data memory device 312-1 by modifying the logical-to-physical (L2P) table 212 (Figure 2) to disassociate the physical address of the first data memory device 312-1 where the copy 302C of user data 302C is stored with the corresponding logical address associated with user data 302 (operation 520) (operation 516). Furthermore, in some embodiments, the electronic system writes the next data 320 (Figure 3) in place of the user data 302 to the first data memory device 312-1 (operation 522), modifies the L2P table 212 to associate the physical address of the first data memory device 312-1 with the corresponding next logical address associated with the next data 320 (operation 524), and updates the integrity data 308 based on the next data 320 (operation 526). Alternatively, in some embodiments, a copy 302C of the user data 302 is erased from the first data memory device 312-1 by purging the copy 302C of the user data 302 from the first data memory device 312-1.

[0048] In some embodiments, each of the two separate memory devices 304, 306 includes a quad-level cell (QLC) memory die and has a plurality of memory blocks, each memory block containing a plurality of memory pages 210, each containing a plurality of quad-level memory cells. Alternatively, in some embodiments, each of the two separate memory devices 304, 306 includes either a single-level cell (SLC) memory die or a multi-level cell (MLC) memory die. In some embodiments, the integrity memory device 310 includes either an MLC memory die or an SLC memory die and has a plurality of memory blocks (e.g., 310A, 310B in Figure 3), each memory block containing a plurality of memory pages, each containing a plurality of MLCs or SLCs. Alternatively, in some embodiments, at least one of the two separate memory devices 304, 306 and the integrity memory device 310 includes a storage class memory (SCM) selected from the group consisting of phase-change memory (PCM), resistive random-access memory (ReRAM), magnetoresistive random-access memory (MRAM), and 3D XPoint memory.

[0049] In some embodiments, the electronic system stores multiple data blocks of the data file 402 (e.g., 402A-402C in Figure 4) in multiple data memory devices 312. The multiple data blocks include a first data block 402A further containing user data 302, the user data 302 being mirrored on two separate memory devices 304, 306, and the first data block 402A being stored on the first data memory device 312-1 of the multiple data memory devices 312. Furthermore, in some embodiments, the multiple data blocks of the data file 402 are stored according to a predetermined low-cost disk redundancy array (RAID) level. The predetermined RAID level is selected from RAID 4 and RAID 5. The user data 302 is mirrored on two separate memory devices 304, 306 according to RAID 1. In some embodiments, the user data 302 includes first user data, and the data file further includes second user data 404 (Figure 4). The second user data 404 is stored in the second data memory device 312-2, but the second user data 404 is mirrored to two corresponding memory devices separate from the multiple data memory devices 312, according to RAID 1.

[0050] In some embodiments, user data 302 includes first user data. The electronic system generates integrity data 308 based on the first user data 302 and mirrors the second user data 404 to two separate memory devices 304, 306 before storing the integrity data 308 of the first user data 302 in the integrity memory device 310.

[0051] In some embodiments, each of the two separate memory devices 304, 306 and the integrity memory device 310 includes one or more memory dies. In some embodiments, each of the two separate memory devices 304, 306 and the integrity memory device 310 includes a separate memory die of the memory system. In some embodiments, integrity data 308 is generated and stored based on a copy 302C of user data 302 and is not generated for user data 302 mirrored on the two separate memory devices 304, 306.

[0052] Each of the identified elements may be stored in one or more of the aforementioned memory devices and may correspond to a set of instructions that perform the function described above. The identified modules or programs (i.e., sets of instructions) do not necessarily have to be implemented as separate software programs, procedures, modules, or data structures, and various subsets of these modules may be combined or rearranged in various embodiments. In some embodiments, memory optionally stores a subset of the identified modules and data structures. Furthermore, memory optionally stores additional modules and data structures not mentioned above.

[0053] Various embodiments of this application relate to methods, systems, devices, and non-transient computer-readable media for storing data by distributing redundant data blocks across multiple memory devices (e.g., NAND dies) based on multiple RAID schemes (e.g., RAID 1, RAID 4). In other words, data is stored in a progressive RAID scheme, where RAID 1 is applied to store data at a fine data granularity level, until the stored data fills one or more isolation units or XOR strips, thereby generating integrity data 308 for the stored data. Copies of the data are stored in QLC and protected by integrity data, so that the duplicated data stored in RAID 1 is invalidated or deactivated. More specifically, in one example, for any host write, user data is duplicated and written simultaneously to two QLC memory dies containing a target memory zone and a QLC copy block. These two QLC memory dies comprise two separate memory dies. In the case of RAID 4 or RAID 5, integrity data cannot be updated unless the zone is full. In some embodiments, integrity data is updated only when a zone group 316 (Figure 3), which includes multiple zones, becomes full. Integrity data may be generated, and QLC copy blocks may be invalidated and released. Also, if zone group 316 (Figure 3) is reset (e.g., invalidated or released), the association between zone group 316 (Figure 3) and its corresponding logical address is broken, and the integrity data generated based on zone group 316 (Figure 3) is updated without using the data in zone group 316 (Figure 3).

[0054] By integrating multiple RAID schemes, the memory die can meet the memory system's memory endurance requirements (e.g., >1.5K program / erase (P / E) cycles) and UBER requirements (e.g., <10-) for architectures that depend on the NAND placement mode and cannot directly apply RAID 4 / RAID 5 schemes. 14This can be achieved at a reasonable hardware cost. SSD systems can still achieve the advantages of NAND placement mode architectures, such as reduced write amplification and reduced over-provisioning, while achieving the desired reliability and performance.

[0055] In some embodiments, data is replicated to SLC-based cache memories 304, 306 according to a RAID 1 scheme. Integrity data is stored in QLC-based memory 310 that meets memory endurance requirements (e.g., >3K P / E cycles). In some embodiments, RAID 1 is generalized to a smaller m+1 RAID 4 or RAID 5, where m synchronous writes are performed by the host system.

[0056] The terms used in describing the various implementations in this specification are intended to describe only specific implementations, and not to limit them. The singular forms “a,” “an,” and “the” used in the various implementations and claims also include the plural form unless otherwise clearly specified in the context. Furthermore, the terms “and / or” used herein indicate and encompass any possible combination of one or more of the related enumerated items. Additionally, the terms “include,” “compose,” and / or “equip” used herein merely identify the presence of the described features, integers, steps, actions, elements, and / or components, and do not preclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof. Furthermore, while terms such as “first,” “second,” etc., are used to describe various elements in this specification, it should be understood that these elements should not be limited by these terms. These terms are merely used to distinguish one element from another.

[0057] In this specification, the term "if" may be interpreted, depending on the context, as meaning "when," "if," "in response to a determination," "in response to detection," or "according to a determination that." Similarly, the phrase "if determined to" or "[the described condition or event] is detected" may be interpreted, depending on the context, as meaning "when determined to" or "in response to a determination" or "[the described condition or event] is detected" or "[the described condition or event] is detected" or "[according to a determination that "[the described condition or event] has been detected."

[0058] The above description has been made with reference to specific embodiments for the sake of clarity. However, the above exemplary description is not exhaustive and is not intended to limit the description to the exact form disclosed in the claims. Many modifications and variations are possible based on the above teachings. These embodiments have been selected and described in order to best illustrate the principle of operation and practical applications and to be understood by those skilled in the art.

[0059] The various accompanying drawings illustrate multiple logical stages in a specific order, but the order-independent stages may be rearranged, combined with other stages, or separated. While some rearrangements and other groupings have been specifically mentioned, other rearrangements and groupings are obvious to those skilled in the art, and therefore the rearrangements and groupings presented herein are not an exhaustive list of alternatives. Furthermore, it should be noted that the above stages can be implemented by hardware, firmware, software, or any combination thereof.

Claims

1. A data storage method, The steps include mirroring user data to two separate memory devices, A step of generating integrity data based on the user data, The steps include storing the integrity data of the user data in an integrity memory device, The steps include: releasing the user data that has been mirrored on at least one of the two separate memory devices, in accordance with the determination that the integrity data of the user data has been stored in the integrity memory device; Methods that include...

2. The memory zone includes the memory block of the integrity memory device and the memory blocks of a plurality of data memory devices, including the first data memory device. The method further includes the step of storing a copy of the user data in the first data memory device, wherein the integrity data is generated based on a subset of user data in each subset of data memory devices, including the first data memory device. The method according to claim 1.

3. The method according to claim 2, further comprising the step of determining whether one or more memory blocks of the plurality of data memory devices in the memory zone are filled, wherein the integrity data of the user data is generated based on the user data and stored in the integrity memory device in accordance with the determination that one or more memory blocks of the plurality of data memory devices are filled.

4. The integrity data is updated to exclude the copy of the user data stored in the first data memory device from the subset of user data applied to generate the integrity data, By modifying the logical-to-physical (L2P) table to disassociate the physical address of the first data memory device where the copy of the user data is stored from the corresponding logical address associated with the user data, The method according to claim 2 or 3, further comprising the step of erasing the copy of the user data from the first data memory device.

5. The steps include writing the following data to the first data memory device instead of the user data, The steps include modifying the L2P table to associate the physical address of the first data memory device with the next logical address associated with the next data, The steps include updating the integrity data based on the following data: The method according to claim 4, further comprising:

6. The method according to claim 4 or 5, wherein the copy of the user data is erased from the first data memory device by the step of purging the copy of the user data from the first data memory device.

7. The method according to any one of claims 2 to 6, further comprising the step of receiving a host write request, wherein the user data is mirrored to the two separate memory devices, and the copy of the user data is stored in the first data memory device in response to the host write request.

8. The method according to any one of claims 1 to 7, wherein each of the two separate memory devices includes a quad-level cell (QLC) solid-state drive (SSD) die and has a plurality of memory blocks, each of which has a plurality of memory pages, each of which has a plurality of quad-level memory cells.

9. The method according to any one of claims 1 to 8, further comprising the step of storing a plurality of data blocks of a data file in a plurality of data memory devices, wherein the plurality of data blocks further include a first data block containing user data, and if the first data block is stored in the first data memory device of the plurality of data memory devices, the user data is mirrored to the two separate memory devices.

10. The plurality of data blocks of the data file are stored according to a predetermined low-cost disk redundancy array (RAID) level, and the predetermined RAID level is selected from RAID 4 and RAID 5. The method according to claim 9, wherein the user data is mirrored to the two separate memory devices according to RAID 1.

11. The user data includes first user data, the data file further includes second user data, and the method is The method according to claim 9 or 10, further comprising the step of mirroring the second user data to two separate corresponding memory devices, in accordance with RAID 1, if the second user data is stored in the second data memory device.

12. The method according to any one of claims 1 to 11, wherein each of the two separate memory devices and the integrity memory device includes one or more memory dies.

13. The method according to any one of claims 1 to 12, wherein each of the two separate memory devices includes either a single-level cell (SLC) memory die or a multi-level cell (MLC) memory die.

14. The method according to any one of claims 1 to 13, wherein the integrity memory device includes either an MLC memory die or an SLC memory die and has a plurality of memory blocks, each of which memory blocks includes a plurality of memory pages, each of which includes a plurality of MLCs or SLCs.

15. The method according to any one of claims 1 to 14, wherein at least one of the two separate memory devices and the integrity memory device includes a storage class memory (SCM) selected from the group consisting of phase-change memory (PCM), resistive random-access memory (ReRAM), magnetoresistive random-access memory (MRAM), and 3D XPoint memory.

16. The method according to any one of claims 1 to 15, wherein each of the two separate memory devices and the integrity memory device includes a separate memory die of the memory system.

17. The method according to any one of claims 1 to 16, wherein the integrity data is generated and stored based on a copy of the user data, and there is no step of generating data for the user data mirrored on the two separate memory devices.

18. The user data includes first user data, and the method is The method according to any one of claims 1 to 17, further comprising an instruction to generate the integrity data based on the first user data, and to mirror the second user data to two separate memory devices before storing the integrity data of the first user data in the integrity memory device.

19. One or more processors, An electronic device comprising: a memory that, when executed by one or more processors, stores instructions causing the processors to perform the method according to any one of claims 1 to 18.

20. A non-temporary computer-readable storage medium that, when executed by one or more processors, stores instructions causing the processors to perform the method according to any one of claims 1 to 18.

Images