Progressive redundant array of inexpensive disks (RAID) for storage devices

Abstract

The present application relates to storing data by allocating redundant data blocks across multiple storage devices (e.g., solid state disks (SSDs)) and based on multiple RAID schemes. An electronic system mirrors user data on two distinct storage devices. Integrity data is generated based on the user data, and the integrity data is stored on an integrity storage device. In accordance with a determination that the integrity data of the user data is stored on the integrity storage device, the electronic system releases the user data mirrored on at least one of the two distinct storage devices. In some embodiments, a copy of the user data is also stored on one of a plurality of data storage devices. The integrity data is generated and stored on the integrity storage device based on the user data in accordance with a determination that the plurality of data storage devices are filled.

Description

[0001] Related applications

[0002] This application is a continuation-into-priority application of U.S. Patent Application No. 18 / 227,256, filed July 27, 2023, entitled "Progressive Redundant Array of Inexpensive Disks (RAID) for Memory Devices," which is incorporated herein by reference in its entirety. Technical Field

[0003] This application generally relates to memory management, including but not limited to methods, systems, and non-transitory computer-readable media for storing data by allocating redundant data blocks across multiple storage devices, such as multiple storage dies in a solid-state drive (SSD). Background Technology

[0004] Memory is used in computer systems to store instructions and data. Data is processed by one or more processors in the computer system according to instructions stored in memory. Multiple storage units are used in different parts of a computer system to serve different functions. Specifically, computer systems include non-volatile memory, which acts as auxiliary storage to retain the data stored thereon when the computer system is decoupled from power. Examples of auxiliary storage include, but are not limited to, hard disk drives (HDDs) and solid-state drives (SSDs). Computer systems often use Redundant Array of Independent Disks (RAID) for auxiliary storage, a virtualized data storage technology that combines multiple physical storage drives into a single or multiple logical units for data redundancy and performance optimization. RAID can be configured into different tiers within SSDs to achieve improved performance levels and data security. The Storage Networking Industry Association (SNIA) has standardized RAID tiers and their associated data formats. Data center SSDs typically employ block-level striping with dedicated parity bits (RAID 4) or distributed parity bits (RAID 5) to meet uncorrectable bit error rate (UBER) requirements. However, RAID 4 or RAID 5 cannot be effectively applied in some alternative modes that require frequent asynchronous updates within XOR stripes of many open isolation cells. It would be beneficial to apply practical data storage and verification mechanisms to store data involving many open isolation cells. Summary of the Invention

[0005] Various embodiments of this application relate to methods, systems, apparatus, and non-transitory computer-readable media for storing data by distributing redundant data blocks across multiple storage devices (e.g., NAND dies) and based on multiple RAID schemes (e.g., RAID 1 and RAID 4). A first-level RAID includes RAID 1 and is used to buffer data in response to host writes. The data is mirrored on two dissimilar storage devices (e.g., two NAND dies). A copy of the data is written to a first isolation unit (e.g., a storage block of another storage device). When the storage system is fully filled with one or more isolation units containing the first isolation unit, the storage system generates integrity data for storing the data in the first isolation unit in batches and stores the integrity data according to a second-level RAID (e.g., RAID 4 or RAID 5). After storing the integrity data, at least one copy of the data stored according to the first-level RAID is invalidated or released. RAID 1 is used to store data with fine-grained data at the asynchronous write level, while RAID 4 and RAID 5 are used to store data protected by integrity data in batches. In the example placement mode, RAID 1 provides better performance at a finer data granularity while utilizing more temporary storage space. Conversely, RAID 4 and RAID 5 save the storage space wasted on data duplication in RAID 1 and cannot provide the same data granularity as RAID 1. In various embodiments of this application, two tiers of RAID schemes are applied jointly to benefit from both the fine data granularity of RAID 1 and the high storage space utilization of RAID 4 or RAID 5, thereby meeting the UBER requirements of the storage system at a reasonable hardware cost.

[0006] In one aspect, a method is implemented at an electronic system to store data on a storage system (e.g., comprising multiple storage channels). The method includes: mirroring user data on two dissimilar storage devices; generating integrity data based on the user data; and storing the integrity data of the user data on the integrity storage device. The method further includes, based on a determination that the integrity data of the user data is stored on the integrity storage device, releasing the user data mirrored on at least one of the two dissimilar storage devices. In some embodiments, the two dissimilar storage devices and the integrity storage device correspond to different NAND dies of an SSD.

[0007] In some embodiments, the storage area includes storage blocks of the integrity storage device and multiple storage blocks of multiple data storage devices including a first data storage device. The method further includes storing a copy of the user data on the first data storage device, and the integrity data is generated based on a subset of user data from each data storage device in a subset of data storage devices including the first data storage device. Furthermore, in some embodiments, the method further includes determining whether one or more storage blocks of the multiple data storage devices in the storage area are full. The integrity data of the user data is generated based on the user data and stored on the integrity storage device based on the determination that one or more storage blocks of the multiple data storage devices are full. Additionally, in some embodiments, the method includes erasing the copy of the user data from the first data storage device by: updating the integrity data to exclude the copy of the user data stored on the first data storage device from the subset of user data used to generate the integrity data; and modifying the logical-physical (L2P) table to deassociate the physical address of the first data storage device storing the copy of the user data with the corresponding logical address associated with the user data.

[0008] Some embodiments of this application include an electronic device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the processors to perform any of the methods described above on a storage system (e.g., one or more SSDs).

[0009] Some implementations include a non-transitory computer-readable storage medium storing one or more programs. The one or more programs contain instructions that, when executed by one or more processors, cause the processors to perform any of the methods described above on a storage system (e.g., one or more SSDs).

[0010] These illustrative embodiments and implementations are mentioned not to limit or restrict this disclosure, but to provide examples to aid in understanding it. Additional embodiments are discussed in the detailed description and are further described therein. Attached Figure Description

[0011] To better understand the various described embodiments, the following detailed description should be taken in conjunction with the accompanying drawings, in which similar reference numerals refer to corresponding parts throughout the drawings.

[0012] Figure 1 This is a block diagram of an example system module in a typical electronic device according to some embodiments.

[0013] Figure 2 This is a block diagram of a storage system of an example electronic device having one or more memory access queues according to some embodiments.

[0014] Figure 3 An example progressive RAID scheme that integrates two RAID schemes (e.g., RAID 1 and RAID 4) according to some embodiments is shown.

[0015] Figure 4 Another example of a progressive RAID scheme that integrates two RAID schemes according to some embodiments is shown.

[0016] Figure 5 This is a flowchart of an example method for storing data in a storage system according to some embodiments.

[0017] Throughout the various views in the accompanying drawings, similar reference numerals refer to the corresponding parts. Detailed Implementation

[0018] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. Numerous non-limiting specific details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0019] Figure 1 This is a block diagram of an example system module 100 in a typical electronic device according to some embodiments. System module 100 in this electronic device includes at least: a processor module 102; a storage module 104 for storing programs, instructions, and data; an input / output (I / O) controller 106; one or more communication interfaces, such as a network interface 108; and one or more communication buses 140 for interconnecting these components. In some embodiments, the I / O controller 106 allows the processor module 102 to communicate with I / O devices (e.g., a keyboard, mouse, or trackpad) via a universal serial bus interface. In some embodiments, the network interface 108 includes one or more interfaces for Wi-Fi, Ethernet, and Bluetooth networks, each allowing the electronic device to exchange data with external sources such as a server or another electronic device. In some embodiments, the communication bus 140 includes a circuitry (sometimes referred to as a chipset) that interconnects and controls communication between the various system components included in system module 100.

[0020] In some embodiments, storage module 104 includes high-speed random access memory, such as DRAM, static random access memory (SRAM), dual data rate (DDR) dynamic random access memory (RAM), or other random access solid-state storage devices. In some embodiments, storage module 104 includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash storage devices, or other non-volatile solid-state storage devices. In some embodiments, storage module 104, or alternatively, the non-volatile storage device within storage module 104, includes a non-transitory computer-readable storage medium. In some embodiments, a storage slot is reserved on system module 100 for receiving storage module 104. Once inserted into the storage slot, storage module 104 is integrated into system module 100.

[0021] In some embodiments, system module 100 further includes one or more components selected from storage controller 110, one or more solid-state drives (SSDs) 112, hard disk drives (HDDs) 114, power management integrated circuits (PMICs) 118, graphics module 120, and audio module 122. Storage controller 110 is configured to control communication between processor module 102 in the electronic device and the storage components including storage module 104. SSD 112 is configured to use integrated circuit assemblies to store data in the electronic device and, in many embodiments, is configured based on NAND or NOR memory. HDD 114 is a conventional data storage device for storing and retrieving digital information based on electromechanical disks. Power connector 116 is electrically coupled to receive external power. PMIC 118 is configured to modulate the received external power to other desired DC voltage levels, such as 5 V, 3.3 V, or 1.8 V, according to the needs of various components or circuits within the electronic device (e.g., processor module 102). The graphics module 120 is configured to generate an output image for feeding to one or more display devices according to an ideal image / video format. The audio module 122 is configured to facilitate the input of audio signals to and output of audio signals from the electronic device under the control of a computer program.

[0022] It should be noted that the communication bus 140 will also include various system components 110 to 122 interconnected and control communication therebetween.

[0023] Furthermore, those skilled in the art will recognize the use of other non-transitory computer-readable storage media because new data storage technologies have been developed for storing information in non-transitory computer-readable storage media within storage module 104 and SSD 112. These new non-transitory computer-readable storage media include, but are not limited to, media made of biological materials, nanowires, carbon nanotubes, and individual molecules, even if the corresponding data storage technologies are currently under development and not yet commercialized.

[0024] Some embodiments of this application involve storing data by applying multiple RAID schemes (e.g., RAID 1 and RAID 4), including temporarily distributing redundant data blocks across multiple storage devices (e.g., one or more SSDs, each containing multiple NAND dies). The two tiers of RAID schemes are applied jointly to benefit from both the fine-grained data level of RAID 1 and the high storage space utilization of RAID 4 or RAID 5. This integration of multiple RAID schemes can meet the UBER requirements of the storage system at a reasonable hardware cost. Specifically, the first tier of RAID includes RAID 1 and is used to buffer data in response to host writes. The data is mirrored on two dissimilar storage devices (e.g., two NAND dies). Based on the second tier of RAID (e.g., RAID 4 or RAID 5), a copy of the data is written in a first isolation unit (e.g., a storage block of a data storage device). When the storage system is completely filled with one or more isolation units containing the first isolation unit, the storage system generates integrity data for the data stored in the first isolation unit via batching and stores the integrity data according to the second tier of RAID. After storing the integrity data, at least one copy of the data stored according to the first tier RAID is invalidated or released.

[0025] RAID is a virtualized data storage technology that combines multiple physical storage drives into a single or multiple logical units for data redundancy and performance optimization. The Storage Networking Industry Association (SNIA) has standardized RAID levels (e.g., RAID 0, ..., and RAID 6) and their associated data formats. For example, RAID 0 is implemented based on striping and does not have data mirroring or parity. In RAID 1, data is mirrored across two dissimilar storage devices (e.g., NAND dies). RAID 4 is implemented based on block-level striping via dedicated parity disks. RAID 5 is implemented based on block-level striping with distributed parity.

[0026] Figure 2This is a block diagram of a storage system 200 of an example electronic device having one or more memory access queues according to some embodiments. The storage system 200 is coupled to a host device 220 (e.g., ...). Figure 1 The processor module 102 is configured to store instructions and data for extended periods, such as when the electronic device is in sleep, hibernation, or powered off. The host device 220 is configured to access and process the instructions and data stored in the storage system 200 to run the operating system and execute user applications. The storage system 200 further includes a controller 202 and a plurality of storage channels 204. Each storage channel 204 contains a plurality of storage units. The controller 202 is configured to execute firmware-level software to bridge the plurality of storage channels 204 to the host device 220.

[0027] Each storage channel 204 includes one or more storage packages 206 (e.g., two storage chips, two storage dies). In one example, each storage package 206 corresponds to one storage die. Each storage package 206 includes multiple storage planes 208, and each storage plane 208 further includes multiple storage pages 210. Each storage page 210 includes a set of ordered storage cells, and each storage cell is identified by a corresponding physical address. In some embodiments, the storage system 200 includes a single-level cell (SLC) die, and each storage cell stores a single data bit. In some embodiments, the storage system 200 includes a multi-level cell (MLC) die, and each storage cell stores two data bits. In one example, each storage cell of a three-level cell (TLC) die stores three data bits. In another example, each storage cell of a four-level cell (QLC) die stores four data bits. In yet another example, each storage cell of a five-level cell (PLC) die stores five data bits. In some embodiments, each storage cell may store any suitable number of data bits. Compared to non-SLC dies (e.g., MLC dies, TLC dies, QLC dies, PLC dies), SLC dies operate at higher speeds, with higher reliability and longer lifespans; however, they have lower device density and higher prices.

[0028] Each memory channel 204 is coupled to a corresponding channel controller 214, which is configured to control internal and external requests for accessing memory cells in the corresponding memory channel 204. In some embodiments, each memory package 206 (e.g., each memory die) corresponds to a corresponding memory access request queue 216. In some embodiments, each memory channel 204 corresponds to a corresponding memory access request queue 216. Additionally, in some embodiments, each memory channel 204 corresponds to a distinct and different memory access request queue 216. In some embodiments, a subset (less than all) of the plurality of memory channels 204 corresponds to distinct memory access request queues 216. In some embodiments, all of the plurality of memory channels 204 of the memory system 200 correspond to a single memory access request queue 216. Each memory access request is optionally received internally from the memory system 200 to manage the corresponding memory channel 204, or externally from the host device 220 to write or read data stored in the corresponding channel 204. Specifically, each memory access request includes one of the following: a system write request received from storage system 200 to write to the corresponding storage channel 204; a system read request received from storage system 200 to read from the corresponding storage channel 204; a host write request originating from host device 220 to write to the corresponding storage channel 204; and a host read request received from host device 220 to read from the corresponding storage channel 204. It should be noted that system read requests (also referred to as background read requests or non-host read requests) and system write requests are dispatched by the storage controller to implement internal memory management functions, including but not limited to garbage collection, wear leveling, read interference mitigation, memory snapshot capture, memory mirroring, caching, and memory spare.

[0029] In some embodiments, in addition to channel controller 214, controller 202 further includes local storage processor 218, host interface controller 222, SRAM buffer 224, and DRAM controller 226. Local storage processor 218 accesses multiple storage channels 204 based on one or more memory access request queues 216. In some embodiments, local storage processor 218 writes to and reads from the multiple storage channels 204 based on storage blocks. Data from one or more storage blocks is jointly written to or read from the multiple channels. Data in the same storage block is not written via more than one operation. Each storage block optionally corresponds to one or more storage pages. In one example, each storage block to be jointly written to or read from the multiple storage channels 204 has a size of 16 KB (e.g., one storage page). In another example, each storage block to be jointly written to or read from the multiple storage channels 204 has a size of 64 KB (e.g., four storage pages). In some embodiments, each page has 16 KB of user data and 2 KB of metadata. Additionally, the number of storage blocks to be jointly accessed and the size of each storage block can be configured for each of the system read, host read, system write, and host write operations.

[0030] In some embodiments, the local storage processor 218 stores data to be written to or read from each of the plurality of storage channels 204 in the SRAM buffer 224 of the controller 202. Alternatively, in some embodiments, the local storage processor 218 stores data to be written to or read from each of the plurality of storage channels 204 in the DRAM buffer 228, the DRAM buffer being configured by the processor module 102 ( Figure 1 The main memory used. The local storage processor 218 of controller 202 accesses the DRAM buffer 228 via host interface controller 222.

[0031] In some embodiments, storage system 200 includes one or more SSDs, and each SSD has a logical-to-physical (L2P) address mapping table 212 (also referred to as L2P table 212) that stores a set of logical addresses (e.g., logical block addresses (LBAs)) of physical addresses. In one example, the SSD has a storage capacity of 32 terabytes (i.e., 32 TB) organized into multiple storage sectors, and each storage sector stores 4096 bytes (i.e., 4 KB) and is individually addressable. The SSD contains 8 billion storage sectors identified by 8 billion physical addresses. At least 33 data bits are required to uniquely represent each individual physical address of the SSD with 8 billion physical addresses. Furthermore, in some embodiments, the SSD includes NAND storage cells and reserves additional storage space through overprovisioning. For example, the overprovisioning is 25%, and the SSD has 10 billion storage sectors to be identified by 10 billion unique physical addresses. At least 34 data bits are required to uniquely identify each individual physical address of an SSD with 10 billion physical addresses.

[0032] In some embodiments, the data in the storage system 200 is divided into encoded blocks, and each encoded block is called a codeword (e.g., Figure 3 (A combination of user data 302C, 318A, and 318B). For example, each codeword contains... k The units digit corresponds to user data and m A complete set of data corresponding to user data. n Ones place, of which k , m and n It is an integer, and n for k and m The sum of all data. In some embodiments, the storage system 200 includes an integrity engine 230 (e.g., an LDPC engine) and a register file 232 containing an arrangement of registers coupled to the integrity engine 230. The integrity engine 230 is coupled to the storage channel 204 via a channel controller 214 and an SRAM buffer 224. Specifically, in some embodiments, the integrity engine 230 has a data path connection to the SRAM buffer 224, which is further connected to the channel controller 214 via a data path controlled by a local storage processor 218. The integrity engine 230 is configured to use variable nodes and check nodes to verify the data integrity of each coded block of the storage channel 204, and to exchange messages between the variable nodes and check nodes during the integrity check process. A subset of these messages is selected and temporarily stored in the register file 232 as variable node data or check node data.

[0033] Figure 3An example progressive RAID scheme 300 integrating two RAID schemes according to some embodiments is shown. A host device 220 is coupled to a storage system 200 comprising multiple storage devices. Examples of storage devices are HDDs or SSDs. Each storage device includes multiple storage channels 204, each of which further includes multiple storage planes 208, and each storage plane further includes multiple storage pages 210. In some embodiments, each storage channel 204 includes one or more dissimilar storage dies. In some embodiments, user data 302 is mirrored on two dissimilar storage devices 304 and 306 to produce two copies 302A and 302B of the user data. Integrity data 308 is generated based on the user data 302, and the integrity data is stored on an integrity storage device 310. Based on the determination that the integrity data 308 of the user data 302 is stored on the integrity storage device 310, the user data 302 mirrored on at least one of the two dissimilar storage devices 304 and 306 is released. In some embodiments, the host device 220 sends host write requests to the multiple storage devices. User data 302 is mirrored on two dissimilar storage devices 304 and 306 in response to a host write request. Furthermore, in some embodiments, another copy 302C of the user data is stored on a first data storage device 312-1 in response to a host write request received from host device 220.

[0034] In some embodiments, after mirroring in two dissimilar storage devices 304 and 306, user data 302 is copied to the first data storage device 312-1, and integrity data 308 is generated based on a copy 302C of the user data in the first data storage device 312-1. Specifically, the copy 302C of the user data is stored in a storage block of the first data storage device 312-1, which, together with one or more additional data storage devices 312 (e.g., 312-2, 312-3) and a storage block of the integrity storage device 310, forms a storage area 314. In other words, in some embodiments, storage area 314 includes a storage block 310A of the integrity storage device 310 and multiple storage blocks of multiple data storage devices 312 containing the first data storage device 312-1. Each storage block contains multiple storage pages 210 of the corresponding storage device 310 or 312. Figure 2A copy 302C of the user data is stored on the first data storage device 312-1. Integrity data 308 is generated based on a subset of the user data in each data storage device within a subset of the data storage devices 312 that include the first data storage device 312-1. In one example, integrity data 308 is generated based on copies 302C of user data 318A and 318B stored in data storage devices 312-2 and 312-3. In some embodiments, integrity data 308 is generated when at least one of the storage blocks in the data storage devices 312 of storage area 314 is filled or closed so that no further writing is possible. In other words, in many cases, integrity data 308 is not generated once the copy 302C of the user data is stored in the first data storage device 312-1.

[0035] In some embodiments, after integrity data 308 is stored in integrity storage device 310, both copies 302A and 302B in the two dissimilar storage devices 304 and 306 are invalidated and released. Alternatively, in some embodiments, the first data storage device 312-1 comprises one of the two dissimilar storage devices 304 and 306. User data 302 is duplicated in both dissimilar storage devices 304 and 306 and is no longer stored in an additional separate and dissimilar first data storage device. After integrity data 308 is stored in integrity storage device 310, one copy 302A or 302B of the user data stored in storage device 304 or 306 is invalidated and released, while another copy 302B or 302A of the user data stored in storage device 306 or 304 serves as copy 302C of the user data in the first data storage device 312-1.

[0036] In some embodiments, when a copy 302C of user data is erased from the first data storage device 312-1, the corresponding integrity data 308 is updated to exclude the copy 302C of user data stored on the first data storage device 312-1 from the subset of user data used to generate the integrity data 308. For example, the integrity data 308 includes a parity check result generated using XOR logic based on the copy 302C of user data stored in storage device 312-1 and user data 318A and 318B stored in storage devices 312-2 and 312-3. In response to a request to erase, invalidate, or release a copy 302 of user data in storage device 312-2, the storage system 200 updates the integrity data 308 by using XOR logic to update the parity check result based on the user data 318A and 318B stored in storage devices 312-2 and 312-3. The storage controller 202 modifies the L2P table 212 to deassociate the physical address of the first data storage device 312-1 storing the copy 302C of user data with the corresponding logical address associated with the user data 302. In some embodiments, the copy 302C of user data is physically removed from the first data storage device 312-1. Alternatively, the copy 302C of user data remains in the first data storage device 312-1 until it is overwritten by subsequent data 320. Because the L2P table no longer links its physical address to any logical address, the copy 302C of user data is inaccessible. Furthermore, in some cases, subsequent data 320 is written to the first data storage device 312-1 instead of the copy 302C of user data. The L2P table is modified to associate the physical address of the first data storage device 312-1 with the subsequent logical address associated with the subsequent data 320. Integrity data 308 is updated based on the subsequent data 320.

[0037] In some embodiments, the two dissimilar storage devices 304 and 306 have the same memory type. For example, each of the two dissimilar storage devices 304 and 306 includes a QLC memory die and has multiple memory blocks (e.g., 304A, 304B, 306A, 306B), and each memory block includes multiple memory pages 210, wherein each of them includes multiple QLC memory cells. Alternatively, in some embodiments, each of the two dissimilar storage devices 304 and 306 includes an SLC memory die and has multiple memory blocks (e.g., 304A, 304B, 306A, 306B), and each memory block includes multiple memory pages 210, wherein each of them includes multiple SLC memory cells. Alternatively, in some embodiments, each of the two dissimilar storage devices 304 and 306 includes an MLC memory die and has multiple memory blocks (e.g., 304A, 304B, 306A, 306B), and each memory block includes multiple memory pages 210, each of which includes multiple MLC memory cells. Alternatively, in some embodiments, each of the two dissimilar storage devices 304 and 306 includes a storage class memory (SCM) selected from the group consisting of: phase-change memory (PCM), resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), and 3DXPoint memory. An SCM is a type of physical computer memory that combines DRAM, NAND flash memory, and power for data persistence. In some embodiments, the SCM treats non-volatile memory as DRAM and includes it in the server's storage space.

[0038] Alternatively, in some embodiments, the two dissimilar storage devices 304 and 306 have different memory types. For example, storage device 304 includes a QLC-based memory die and serves as a first data storage device 312-1 for storing a copy of user data 302C. Storage device 306 is optionally based on SLC, MLC, or SCM. User data 302B duplicated on storage device 306 is released or erased based on a determination that the integrity data 308 of user data 302 (e.g., parity data) is stored on integrity storage device 310.

[0039] In some embodiments, integrity storage device 310 includes one of MLC and SLC memory dies and has multiple storage blocks (e.g., 310A and 310B), with each block containing multiple storage pages, each of which contains multiple MLC or SLC memory cells. In one example, RAID 5 is applied to generate integrity data 308, and the integrity storage device 310 storing the integrity data 308 includes SLC memory cells, thereby benefiting from the durability of the SLC memory cells. In some cases, given the relatively low durability level of TLC or QLC, integrity storage device 310 does not include TLC and QLC memory cells or any higher-level memory cells. Alternatively, in some embodiments, integrity storage device 310 includes an SCM selected from the group consisting of PCM, ReRAM, MRAM, and 3D XPoint memory. In some embodiments, data storage device 312 has the same memory type as integrity storage device 310.

[0040] In some embodiments, each of the two dissimilar storage devices 304 and 306 and the integrity storage device 310 includes at least one dissimilar storage die of the storage system 200.

[0041] In some embodiments ( Figure 3 In this embodiment, integrity data 308 is stored according to a RAID 4 scheme, wherein integrity data generated for storage blocks of data storage device 312 is stored in a merged manner, for example, in a dedicated integrity storage device 310. Alternatively, in some embodiments, integrity data 308 is stored according to a RAID 5 scheme, wherein integrity data generated for storage blocks of data storage device 312 is stored in a distributed manner. The storage blocks of integrity data are evenly distributed across storage devices 310 and 312. For example, integrity data is stored in each of storage blocks 310A, 312A, 312B, and 312C based on user data stored in three corresponding storage blocks of user data.

[0042] Figure 4Another example of a progressive RAID scheme 400 integrating two RAID schemes according to some embodiments is shown. In one example, user data 302 is stored based on RAID 1 and RAID 4 schemes. In another example, user data 302 is stored based on RAID 1 and RAID 5 schemes. In some embodiments, user data 302 is stored based on a first RAID scheme and a second RAID scheme, wherein the first RAID scheme is temporarily applied and has a finer data granularity than the second RAID scheme. User data 302 is mirrored on two dissimilar storage devices 304 and 306 to produce two copies 302A and 302B of user data 302. Integrity data 308 is generated based on user data 302, and said integrity data is stored on integrity storage device 310. Based on the determination that the integrity data 308 of user data 302 is stored on integrity storage device 310, the user data 302 mirrored on at least one of the two dissimilar storage devices 304 and 306 is released. In some embodiments, a copy 302C of user data 302 is stored on a first data storage device 312-1, and based on the determination that the integrity data 308 of user data 302 is stored on an integrity storage device 310, the mirrored user data 302 on both dissimilar storage devices 304 and 306 is released. Furthermore, in some embodiments, storage devices 304 or 306 are released by deassociating their physical addresses with their corresponding logical addresses, and may or may not be physically cleared.

[0043] In other words, copies 302A and 302B of user data are temporarily stored in a duplicate manner until integrity data 308 is generated and stored for copy 302C of the same user data. The generation of integrity data 308 is delayed from the duplication of copies 302A and 302B of user data because it must wait until storage block 302 containing the user data corresponding to integrity data 308 is filled or closed. In some embodiments, storage area 314 includes each of one or more data storage devices 312 (e.g., 312-1, 312-2, and 312-3) and storage blocks 312A-1, 312-2A, 312-3A, or 310A of integrity storage device 310, and each storage block contains multiple storage pages 210. In some cases, integrity data 308 is not generated once copy 302C of user data is stored in the first data storage device 312-1. Alternatively, integrity data 308 is generated when at least one of storage blocks 312-1A, 312-2A, or 312-3A of the data storage device 312 of storage area 314 is filled or closed to prevent further writing. For example, storage controller 202 determines that storage block 312-1A containing a copy 302C of user data 302 is filled or closed to prevent further writing, and generates and stores integrity data storage block 310A based on storage block 312-1A containing user data. Furthermore, in some embodiments, storage block 312-2A is neither filled / closed nor used to generate integrity data storage block 310A. Alternatively and additionally, in some embodiments, storage blocks 312-2A and 312-3A are also filled and used in conjunction with storage block 312-1A to generate integrity data storage block 310A.

[0044] In other words, in some embodiments, multiple data blocks 402A to 402C of data file 402 are stored on multiple data storage devices (e.g., 304, 306, 312-1, 312-2, 312-3, and 310). The multiple data blocks 402A to 402C include a first data block 402A that further contains user data 302. The first data block 402A is mirrored on two dissimilar storage devices 304 and 306, and is stored on a first data storage device 312-1 of the multiple data storage devices 312. Furthermore, in some embodiments, multiple data blocks 402A to 402C of data file 402 are stored according to a predefined Redundant Array of Inexpensive Disks (RAID) tier. The predefined RAID tier is selected from RAID 4 and RAID 5, and the first data block of the data file is stored on the first data storage device 312-1. According to RAID 1, user data 302 is mirrored on two dissimilar storage devices 304 and 306.

[0045] In some embodiments, user data 302 includes first user data. Data file 402 further includes second user data 404. According to RAID 1, the second user data 404 is mirrored on two corresponding storage devices different from the plurality of data storage devices 312. In some embodiments, the second user data 404 is mirrored on two different storage devices 304 and 306 before integrity data 308 is generated based on the first user data 302 and the integrity data is stored in integrity storage device 310. In some embodiments, both user data 302 and 404A are stored in storage block 312-1A of the first data storage device 312-1, and storage block 310A of integrity data is generated when storage block 312-1A is filled or closed.

[0046] Alternatively, in some embodiments, after integrity data 308 is generated based on first user data 302 and stored in integrity storage device 310, second user data 404 is mirrored on two dissimilar storage devices 304 and 306. User data 302 and 404B are stored separately in storage blocks 312-1A and 312-2A of the two dissimilar data storage devices 312-1 and 312-2. When storage block 312-1A is filled or closed, storage block 310A of integrity data is generated independently of whether storage blocks 312-2A and 312-3A are filled or closed. After storage block 312-1A is filled or closed, storage block 310A of integrity data is updated based on storage block 312-1A. Specifically, in some cases, when storage block 312-1A is filled or closed, storage blocks 312-2A and 312-3A are not filled or closed, and storage block 310A, which generates integrity data based on storage block 312-1A, is generated. When second user data 404B is written to storage block 312-2A and storage block 312-2A is subsequently filled or closed, storage block 312-3A is not filled or closed, and storage block 310A, which generates or updates integrity data based on the data stored in storage blocks 312-1A and 312-2A, is generated.

[0047] Alternatively, in some cases, when storage blocks 312-1A are filled or closed after a copy 302C of user data 302 is written, storage blocks 312-2A and 312-3A have been filled or closed, and storage block 310A containing integrity data is generated based on the data stored in storage blocks 312-1A, 312-2A, and 312-3A. When storage block 312-2A fails or is released, storage block 310A containing integrity data is generated or updated based on the data stored in storage blocks 312-1A and 312-3A, independent of whether the corresponding user data is physically cleared from storage block 312-2A. When second user data 404B is written to storage block 312-2A and storage block 312-2A is subsequently filled or closed, storage block 310A containing integrity data is generated or updated based on the data stored in storage blocks 312-1A, 312-2A, and 312-3A.

[0048] In some embodiments, user data 406 is stored in storage block 312-1A of the first data storage device 312-1 and corresponds to integrity data 408 stored in storage block 310A of the integrity storage device 310. When user data 406 is erased from the first data storage device 312-1, integrity data 408 is updated (operation 410) to exclude user data 406 stored on the first data storage device 312-1 from the subset of user data used to generate integrity data 408. L2P table 212 is updated (operation 412) to deassociate the physical address of the first data storage device 312-1 storing user data 406 with the corresponding logical address associated with user data 406. Furthermore, in some embodiments, subsequent data 414 is written to the first data storage device 312-1 in place of user data 406, and L2P table 212 is updated (operation 416) to associate the physical address of the first data storage device 312-1 with the subsequent logical address associated with subsequent data 414. Integrity data 408 is also updated based on subsequent data 414 (operation 418). In some embodiments, user data 406 includes a copy 302C of user data 302, and is updated with subsequent data 320 ( Figure 3 Replace. Accordingly, update the integrity data 308 corresponding to copy 302C of user data 302.

[0049] In some embodiments, data is stored as redundant data blocks, spanning multiple storage devices (e.g., NAND dies) and distributed based on multiple RAID schemes (e.g., RAID 1 and RAID 4). A first-level RAID includes RAID 1 and is used to buffer data in response to host writes. The data is mirrored on two dissimilar storage devices (e.g., NAND dies). A copy of the data is written to a first isolation unit 312-1A. When the storage system completely fills one or more isolation units (e.g., 312-1A, 312-2A, and / or 312-3A) to form a storage area 314 containing the first isolation unit, the storage system generates storage blocks 310A for integrity data of the data stored in the first isolation unit via batching, and stores the integrity data according to a second-level RAID (e.g., RAID 4 or RAID 5). RAID 1 is used to store data with fine-grained data at the asynchronous write level, and RAID 4 or RAID 5 is used to store data protected by integrity data as integrity data is generated at the storage block level. After storing the integrity data, at least one copy of the data stored according to the first RAID level is invalidated or released. Only one copy of the user data stores the corresponding integrity data. This saves storage space used to store a second copy of the user data, while data integrity remains available. Thus, RAID 1 is applied at the fine data granularity level, and RAID 4 or RAID 5 is applied at the large data block level, thereby saving 200 units of storage system space without compromising data integrity.

[0050] Figure 5 This is a flowchart of an example method 500 for storing data in a storage system according to some embodiments. Method 500 is implemented in a storage system 200 (… Figure 2 The electronic system of the storage system 200 optionally includes multiple storage devices (e.g., NAND dies). The electronic system mirrors user data 302 across two dissimilar storage devices 304 and 306 (operation 502). Figure 3 The system generates (operation 504) integrity data 308 based on user data 302. The integrity data 308 of user data 302 is stored (operation 506) on integrity storage device 310. Based on the determination that the integrity data 308 of user data 302 is stored on integrity storage device 310, the electronic system releases (operation 508) the user data 302 mirrored on at least one of two dissimilar storage devices 304 and 306.

[0051] In some embodiments, storage area 314 ( Figure 3The system includes (operation 510) storage blocks (e.g., 310A) of integrity storage device 310 and multiple storage blocks (e.g., 312-1A, 312-2A, and 312-3A) of multiple data storage devices 312 including first data storage device 312-1. The electronic system stores (operation 512) a copy 302C of user data 302 on the first data storage device 312-1. Integrity data 308 is generated based on a subset of user data from each data storage device within a subset of the data storage devices 312 including first data storage device 312-1. Furthermore, in some embodiments, the electronic system determines (operation 514) whether one or more storage blocks of the multiple data storage devices 312 of storage area 314 are full. Integrity data 308 of user data 302 is generated based on user data 302 and stored on integrity storage device 310 based on the determination that one or more storage blocks (e.g., storage blocks 312-1A, 312-B, or 312-C) of the multiple data storage devices 312 are full. Furthermore, in some embodiments, the storage system receives a host write request. User data 302 is mirrored on two dissimilar storage devices 304 and 306, and a copy 302C of user data 302 is stored on a first data storage device 312-1 in response to the host write request.

[0052] In some embodiments, the electronic system further erases (operation 516) a copy 302C of user data 302 from the first data storage device 312-1 by: updating (operation 518) integrity data 308 to exclude the copy 302C of user data 302 stored on the first data storage device 312-1 from the subset of user data 302 applied to generate integrity data 308; and modifying (operation 520) logical-physical (L2P) table 212 ( Figure 2 This is to deassociate the physical address of the first data storage device 312-1 storing the copy 302C of user data 302 with the corresponding logical address associated with the user data 302. Furthermore, in some embodiments, the electronic system will subsequently store data 320 (…). Figure 3 The process involves replacing user data 302 with (operation 522) writing it into the first data storage device 312-1, modifying (operation 524) the L2P table 212 to associate the physical address of the first data storage device 312-1 with the subsequent logical address associated with subsequent data 320, and updating (operation 526) the integrity data 308 based on the subsequent data 320. Alternatively, in some embodiments, the copy 302C of user data 302 is erased from the first data storage device 312-1 by wiping the copy 302C of user data 302 from the first data storage device 312-1.

[0053] In some embodiments, each of the two dissimilar storage devices 304 and 306 includes a four-level cell (QLC) memory die and has multiple memory blocks, with each memory block containing multiple memory pages 210, wherein each of them contains multiple four-level memory cells. Alternatively, in some embodiments, each of the two dissimilar storage devices 304 and 306 includes one of the following: a single-level cell (SLC) memory die and a multi-level cell (MLC) memory die. In some embodiments, the integrity storage device 310 includes one of an MLC memory die and an SLC memory die and has multiple memory blocks (e.g., Figure 3 (310A and 310B in the example), and each storage block contains multiple storage pages, each of which contains multiple MLCs or SLCs. Alternatively, in some embodiments, at least one of the two dissimilar storage devices 304 and 306 and the integrity storage device 310 includes a storage class memory (SCM) selected from the group consisting of: phase-change memory (PCM), resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), and 3D XPoint memory.

[0054] In some embodiments, the electronic system will store multiple data blocks of data file 402 (e.g., Figure 4 Data blocks 402A to 402C are stored on multiple data storage devices 312. The multiple data blocks include a first data block 402A that further includes user data 302, and the user data 302 is mirrored on two dissimilar storage devices 304 and 306, while the first data block 402A is stored on a first data storage device 312-1 of the multiple data storage devices 312. Furthermore, in some embodiments, multiple data blocks of the data file 402 are stored according to a predefined Redundant Array of Inexpensive Disks (RAID) hierarchy. The predefined RAID hierarchy is selected from RAID 4 and RAID 5. According to RAID 1, the user data 302 is mirrored on two dissimilar storage devices 304 and 306. In some embodiments, the user data 302 includes first user data, and the data file further includes second user data 404 (…). Figure 4 When the second user data 404 is stored on the second data storage device 312-2, the second user data 404 is mirrored on two corresponding storage devices different from the plurality of data storage devices 312 according to RAID 1.

[0055] In some embodiments, user data 302 includes first user data. Before generating integrity data 308 based on the first user data 302 and storing the integrity data 308 of the first user data 302 in the integrity storage device 310, the electronic system mirrors the second user data 404 on two dissimilar storage devices 304 and 306.

[0056] In some embodiments, each of the two dissimilar storage devices 304 and 306 and the integrity storage device 310 includes one or more storage dies. In some embodiments, each of the two dissimilar storage devices 304 and 306 and the integrity storage device 310 includes dissimilar storage dies of the storage system. In some embodiments, integrity data 308 is generated and stored based on a copy 302C of user data 302, and the integrity data is not generated for user data 302 mirrored on the two dissimilar storage devices 304 and 306.

[0057] Each of the elements identified above may be stored in one or more of the previously mentioned storage devices and corresponds to a set of instructions for performing the functions described above. The modules or programs identified above (e.g., instruction sets) need not be implemented as separate software programs, processes, modules, or data structures, and therefore, in various embodiments, various subsets of these modules may be combined or otherwise rearranged. In some embodiments, the memory optionally stores a subset of the modules and data structures identified above. Furthermore, the memory optionally stores additional modules and data structures not described above.

[0058] Various embodiments of this application relate to methods, systems, apparatus, and non-transitory computer-readable media for storing data by distributing redundant data blocks across multiple storage devices (e.g., NAND dies) and based on multiple RAID schemes (e.g., RAID 1 and RAID 4). In other words, data is stored using progressive RAID schemes, where RAID 1 is applied to store data at a fine-grained data level until the stored data fills one or more isolation cells or XOR stripes, thereby generating integrity data 308 for the stored data. Since copies of the data are stored in QLCs and protected by integrity data, duplicate data stored in RAID 1 is invalidated or released. More specifically, in one example, for any host write, user data is duplicated and simultaneously written to two QLC storage dies containing a target storage area and a QLC replica block. These two QLC storage dies contain two distinct storage dies. For RAID 4 or RAID 5, if one zone is not full, updates to the integrity data are not available. In some embodiments, updates are only made to zone groups 316 containing multiple zones ( Figure 3 Integrity data is updated only when the region is full. In some cases, integrity data is generated, and the QLC replica block is invalidated and released. Additionally, when resetting (e.g., invalidating or releasing) region group 316... Figure 3 When, district group 316 ( Figure 3 ) is unassociated with the corresponding logical address, and in the unused zone group 316 ( Figure 3Updating data based on district group 316 in the case of data in ) Figure 3 The generated integrity data.

[0059] For architectures that rely on NAND placement patterns (where RAID4 / RAID5 schemes cannot be directly applied), the integration of multiple RAID schemes allows storage dies to meet the storage persistence requirements (e.g., >1.5K program / erase (P / E) cycles) and UBER requirements (e.g., <10-) of the storage system at a reasonable hardware cost. 14 SSD systems can still realize the benefits of NAND placement mode architecture, such as lower write amplification and reduced overprovisioning, while achieving the desired reliability and performance.

[0060] In some embodiments, data is duplicated in SLC-based caches 304 and 306 according to a RAID 1 scheme. Integrity data is stored in QLC-based memory 310 that meets storage durability requirements (e.g., >3K P / E cycles). In some embodiments, if there is a host system... m If there are multiple synchronous writes, then RAID1 is generalized to a small number. m+1 RAID 4 or RAID 5.

[0061] The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be restrictive. As used in the description of the various described embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or,” as used herein, refers to and covers any and all possible combinations of one or more of the associated listed items. It should be further understood that the terms “includes,” “including,” “comprises,” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Additionally, it should be understood that although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another.

[0062] As used herein, depending on the context, the term “if” may optionally be interpreted as meaning “when”, “after”, “in response to determination”, “in response to detection”, or “according to determination”. Similarly, depending on the context, the phrase “if determination” or “if [the stated condition or event] is detected” may optionally be interpreted as meaning “after determination”, “in response to determination”, “after detection of [the stated condition or event]”, “in response to detection of [the stated condition or event]”, or “according to determination of detection of [the stated condition or event]”.

[0063] For illustrative purposes, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The described embodiments were chosen and described in order to best explain the operating principles and practical applications, thereby enabling those skilled in the art to understand them.

[0064] Although the various figures illustrate several logical stages in a specific order, stages independent of this order can be reordered and other stages can be combined or decomposed. While some reorderings or other groupings are specifically mentioned, other reorderings or groupings will be apparent to those skilled in the art, and therefore the orderings and groupings presented herein are not an exhaustive list of alternatives. Furthermore, it should be understood that the stages can be implemented in hardware, firmware, software, or any combination thereof.

Claims

1. A method for storing data, comprising: Mirror user data on two different storage devices; Generate integrity data based on the user data; The integrity data of the user data is stored on an integrity storage device; as well as Based on the determination that the integrity data of the user data is stored on the integrity storage device, the user data mirrored on at least one of the two dissimilar storage devices is released.

2. The method according to claim 1, wherein the storage area comprises storage blocks of the integrity storage device and multiple storage blocks of multiple data storage devices comprising the first data storage device, the method further comprising: A copy of the user data is stored on the first data storage device, and the integrity data is generated based on a subset of the user data of each data storage device in a subset of data storage devices that include the first data storage device.

3. The method according to claim 2, further comprising: Determine whether one or more storage blocks of the plurality of data storage devices in the storage area are full, wherein the integrity data of the user data is generated based on the user data and stored on the integrity storage device based on the determination that one or more storage blocks of the plurality of data storage devices are full.

4. The method according to claim 2 or 3, further comprising erasing the copy of the user data from the first data storage device by: Update the integrity data to exclude the copy of the user data stored on the first data storage device from the subset of user data used to generate the integrity data; and Modify the logical-physical (L2P) table to deassociate the physical address of the first data storage device storing the copy of the user data with the corresponding logical address associated with the user data.

5. The method of claim 4, further comprising: Subsequent data will replace the user data and be written into the first data storage device; Modify the L2P table to associate the physical address of the first data storage device with the subsequent logical address associated with the subsequent data; as well as The integrity data is updated based on the subsequent data.

6. The method of claim 4 or 5, wherein the copy of the user data is erased from the first data storage device by: The copy of the user data is cleared from the first data storage device.

7. The method according to any one of claims 2 to 6, further comprising: A host write request is received, wherein the user data is mirrored on two dissimilar storage devices, and the copy of the user data is stored on the first data storage device in response to the host write request.

8. The method according to any one of claims 1 to 7, wherein each of the two dissimilar storage devices comprises a four-level cell (QLC) solid-state drive (SSD) die and has a plurality of storage blocks, and each storage block comprises a plurality of storage pages, wherein each comprises a plurality of four-level storage cells.

9. The method according to any one of claims 1 to 8, comprising: Multiple data blocks of a data file are stored on multiple data storage devices, wherein the multiple data blocks include a first data block that further contains user data, and when the first data block is stored on a first data storage device among the multiple data storage devices, the user data is mirrored on the two dissimilar storage devices.

10. The method according to claim 9, wherein: The plurality of data blocks of the data file are stored according to a predefined Redundant Array of Inexpensive Disks (RAID) level, wherein the predefined RAID level is selected from RAID 4 and RAID 5; and The user data is mirrored on the two dissimilar storage devices according to RAID 1.

11. The method according to claim 9 or 10, wherein the user data comprises first user data, and the data file further comprises second user data, the method further comprising: When the second user data is stored on the second data storage device, the second user data is mirrored on two corresponding storage devices that are different from the plurality of data storage devices according to RAID 1.

12. The method according to any one of claims 1 to 11, wherein each of the two dissimilar storage devices and the integrity storage device comprises one or more storage dies.

13. The method according to any one of claims 1 to 12, wherein each of the two dissimilar storage devices comprises one of the following: a single-level cell (SLC) memory die and a multi-level cell (MLC) memory die.

14. The method according to any one of claims 1 to 13, wherein the integrity storage device comprises one of an MLC storage die and an SLC storage die and has a plurality of storage blocks, and each storage block comprises a plurality of storage pages, wherein each of the two comprises a plurality of MLC or SLC.

15. The method according to any one of claims 1 to 14, wherein at least one of the two dissimilar storage devices and the integrity storage device comprises a storage class memory (SCM) selected from the group consisting of: phase-change memory (PCM), resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), and 3D XPoint memory.

16. The method according to any one of claims 1 to 15, wherein each of the two dissimilar storage devices and the integrity storage device comprises a dissimilar storage die of a storage system.

17. The method according to any one of claims 1 to 16, wherein the integrity data is generated and stored based on a copy of the user data, and the integrity data is not generated for user data mirrored on two dissimilar storage devices.

18. The method according to any one of claims 1 to 17, wherein the user data comprises first user data, the method further comprising instructions for: Before generating the integrity data based on the first user data and storing the integrity data of the first user data on the integrity storage device, the second user data is mirrored on the two dissimilar storage devices.

19. An electronic device comprising: One or more processors; as well as A memory having instructions stored thereon, which, when executed by the one or more processors, cause the processors to perform the method according to any one of claims 1 to 18.

20. A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 18.

Images