Distributed raid for parity-based flash memory devices

Abstract

The present disclosure relates to utilizing a storage system that improves memory storage and access by utilizing a distributed erasure code RAID. For example, the storage system stores a mapping table within volatile memory of one or more SSDs on a set of redundant array of independent disks (RAID) SSDs. In particular, a mapping table stored in one SSD in the set of SSDs includes mapping information corresponding to data stored in another SSD in the set of SSDs. In this manner, the storage system can significantly reduce the number of operations required to write new data to the SSD set compared to previous methods that require multiple reads and multiple writes.

Description

Background Art

[0001] In recent years, significant advances have been made in both hardware and software regarding digital storage and memory devices. For example, solid-state drives (SSDs) typically implement solid-state storage in the form of a controller and NAND flash memory or chips, which continue to improve in capacity and performance. For example, many SSDs now include dynamic random access memory (DRAM) to improve data access and storage with respect to NAND flash memory.

[0002] Additionally, various existing computer systems utilize SSD arrays to provide enhanced data protection. For example, redundant array of independent disks (RAID) and similar schemes provide data protection through redundancy. In particular, some RAID schemes (such as RAID4, RAID5, and RAID6) utilize parity-based schemes to recover lost data in the event of a drive failure. Despite these and other advances, problems remain regarding data storage on NAND flash storage devices (e.g., SSDs) that utilize parity-based data protection schemes.

[0003] To illustrate, existing computer systems that utilize parity-based RAID across multiple SSDs (and similar schemes) are inefficient. In particular, while SSDs with NAND flash memory are capable of writing large and small amounts of data to memory, the workload of these SSDs may require a significantly greater number of small writes (e.g., memory block writes that are smaller than a full RAID stripe of the SSD). In practice, these SSDs perform a large number of random small memory block writes to maintain existing data and store new data. However, currently, when writing new data to an SSD disk in a collection of SSDs, the SSDs require additional operations and steps.

[0004] To illustrate, each time data is written to an SSD with a parity-based RAID scheme, many existing computer systems must perform at least two read operations and two write operations. For example, in the case where a small block write does not cover the entire data sector, an existing computer system writes new data to an available sector, reads old data, and reads the corresponding parity data. Then, the computer system uses the old data and the corresponding parity to remove the old data from the parity (e.g., via an XOR operation). In addition, the computer system determines a new parity based on the new data (e.g., via another XOR operation), and returns a new parity for the new data write. As noted above, a write often results in two read operations and two write operations, which results in inefficient resource utilization, causing additional overhead operations (e.g., additional garbage collection, faster wear of drive components due to additional drive use, and longer delays in writing data). In addition, these problems are complex when multiple small random writes need to be performed.

[0005] To solve the above problem, some existing computer systems utilize an additional hardware controller (e.g., a RAID controller or a memory controller) as an intermediary between a host device and an SSD. In this case, the hardware controller requires additional DRAM to maintain a mapping table in which data and parity are stored across a set of SSDs. However, the additional DRAM on the hardware controller requires additional components (and costs), which are typically not included in most computing devices. For example, some components add additional SSDs only for storing data from DRAM. Additionally, if the hardware controller fails or is damaged (e.g., the memory table is lost due to a power outage), access to the SSD may be temporarily or permanently blocked. Therefore, restoring the hardware controller requires a lot of time and / or multiple steps, which can further result in significant performance losses. In addition, these existing computer systems unnecessarily increase the components and costs required for operation.

[0006] These problems and others result in significant inefficiencies with respect to existing computing systems that utilize parity-based RAID SSD schemes and similar schemes for data storage and protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] DETAILED DESCRIPTION Through use of the drawings, additional specificity and detail are provided for one or more implementations briefly described below.

[0008] Figure 1A to Figure 1B A schematic diagram of a computing device illustrating a storage system implemented in accordance with one or more embodiments.

[0009] Figure 2An example workflow for efficiently managing and writing new data utilizing a parity-based RAID SSD scheme in accordance with one or more implementations is illustrated.

[0010] Figure 3 A block diagram illustrating an asymmetric implementation of a distributed parity-based RAID storing a mapping table on an SSD in accordance with one or more implementations.

[0011] Figure 4 A block diagram illustrates a symmetric implementation of a distributed parity-based RAID storing a mapping table across multiple SSDs in accordance with one or more implementations.

[0012] Figure 5 A block diagram illustrating a distributed parity-based RAID with an external storage controller in accordance with one or more implementations.

[0013] FIG. 6A to FIG. 6B A sequence diagram illustrating efficient maintenance, writing, and accessing of data on a distributed parity-based RAID in accordance with one or more implementations.

[0014] Figure 7 An example of a sequence of actions to improve memory storage and access by utilizing distributed parity-based RAID to store mapping tables across multiple SSDs is illustrated in accordance with one or more implementations.

[0015] Figure 8 The diagram illustrates some of the components that may be included within a computer system. DETAILED DESCRIPTION

[0016] Implementations of the present disclosure provide systems, methods, and computing devices that improve memory storage and access by utilizing a distributed parity-based RAID storage system, provide benefits, and / or solve one or more of the aforementioned or other problems in the field of technology. For example, the storage system stores a mapping table in a volatile memory of one or more SSDs on a RAID SSD set. In particular, a mapping table stored in an SSD in the SSD set includes mapping information corresponding to data stored in one or more SSDs in the SSD set. In this way, the storage system can significantly reduce the number of operations required to write new data to the SSD set, from multiple reads and multiple writes to one or two writes.

[0017] To illustrate, in many embodiments, the storage system (i.e., a parity-based RAID storage system) maintains data in non-volatile memory (e.g., NAND flash memory) across a set of SSDs (solid-state drives), wherein a parity-based error-coding SSD set (e.g., RAID4 or RAID5) includes a primary SSD and one or more secondary SSDs. In some embodiments, the storage system stores some or all of a mapping table in a volatile memory (e.g., DRAM) of the primary SSD, the mapping table indicating the location of a sector of data at a first non-volatile memory location (e.g., a memory block) within one of the secondary SSDs. Additionally, the storage system may write a new data segment at a second non-volatile memory location within the secondary SSD, and write a new parity for the new data segment at another non-volatile memory location within the set of SSDs. Further, the storage system may update the mapping table within the volatile memory on the primary SSD to indicate a new non-volatile memory location of the new data segment on the secondary SSD.

[0018] In additional embodiments, the storage system maintains one or more additional portions of the mapping table on one or more secondary SSDs in the set of SSDs. For example, the storage system stores a different portion of the mapping table in DRAM on each SSD in the set of SSDs (excluding parity disks). In many embodiments, the portion of the mapping table stored on one SSD includes mappings for one or more other SSDs in the set of SSDs.

[0019] In some embodiments, the storage system can facilitate a storage controller (e.g., a RAID controller) to operate on one or more of the SSDs. For example, the storage system implements a storage controller on a master SSD. In one or more embodiments, the storage system distributes a storage controller across multiple SSDs in a set of SSDs (e.g., multiple SSDs each perform some storage controller functions). In various implementations, the storage controller (e.g., a RAID controller) is located outside the storage system. However, in these embodiments, the storage system can still maintain a mapping table within a volatile memory (e.g., DRAM) of one or more SSDs that reduces the number of operations required to write new data fragments (e.g., small memory block writes).

[0020] As described herein, the storage system provides several technical advantages in terms of computing flexibility and efficiency over existing computing systems. In fact, the storage system provides several practical applications that deliver benefits and / or solve problems associated with SSD memory devices and systems.

[0021] As an overview, because of the architecture of NAND flash memory chips, it is often not possible to overwrite old data with new data at the same location (e.g., memory block). Instead, the new data is written to the new location, and the old data is marked as invalid. In addition, in order to maintain the data, the NAND flash memory chip may need to refresh the data, which may involve moving valid data out of the sectors that include invalid data. Accordingly, when utilizing NAND flash memory chips, a garbage collection process may be performed to discard old invalid data and recover available memory block space.

[0022] Additionally, in order to keep track of where data is stored across a set of SSDs, a mapping table can be used to map the logical addresses of data segments to physical addresses within the NAND flash memory chips. Conventionally, the mapping table is stored within a DRAM associated with a flash translation layer (FTL) of a controller. In some cases, the controller is a hardware device. In other implementations, the controller is software implemented by a host operating system (OS) or other component. However, as noted above, conventional systems are inefficient and prone to data loss. In contrast, the implementation of the storage system disclosed herein is efficient.

[0023] To illustrate, in various implementations, the storage system improves efficiency by maintaining a mapping table in one or more SSDs in a parity-based RAID storage system. For example, the storage manager maintains a mapping table in DRAM (i.e., volatile memory) on the master SSD, which indicates the storage location of data on one or more secondary drives and in some cases on the master drive. In this way, the storage system first generates a mapping and then performs SSD operations, which reduces the number of operations required to write new data to the SSD in the SSD set. In fact, in many implementations, the number of operations is reduced from four or more operations (e.g., at least two reads and two writes) to two or fewer operations (e.g., a single write or two writes). In addition, by storing the mapping table in DRAM, the storage system improves the retrieval rate of accessing the mapping table and finding the location of the stored data on the NAND flash memory.

[0024] Additionally, the storage system can improve efficiency by storing new data in volatile memory until multiple new data segments are ready to be written. In these embodiments, the storage system can write multiple random small write operations to locations in the SSD set with a single parity for all new data. Therefore, the storage system does not have to perform parity writes every time a new data segment is added, which can further improve efficiency.

[0025] In addition, the storage system improves efficiency and accuracy by maintaining portions of a mapping table on different SSDs within a set of SSDs, which also provides the benefit of implementing a symmetric implementation across all SSDs in the set of SSDs. For example, if the set of SSDs includes N drives plus a parity drive (e.g., N+1 drives), the storage system can divide 1 / N of the mapping table and store it on the N drives, such as in the DRAM of each of the N SSDs. For example, a first SSD includes a first portion of a mapping table that includes data locations for data on the N+1 drives and a second SSD includes a second portion of a mapping table that includes additional data locations for additional data on the N+1 drives. In this implementation, in the event of an SSD failure, only portions of the mapping table are lost. Further, in some embodiments, the storage system may be able to quickly recover and restore lost mapping table information, along with recovering data from the lost SSD.

[0026] In addition, the storage system improves efficiency and accuracy by utilizing a storage controller within one or more SSDs in the SSD set. For example, in various implementations, the storage manager operates without a traditional storage controller. In this way, the storage system avoids the computational, capital, and operating costs of such controllers. However, if a central storage controller is available, the storage system can utilize the storage controller in processes related to maintaining mapping tables within DRAM of the SSDs in the SSD set.

[0027] As described in the foregoing discussion, the present disclosure utilizes various terms to describe the features and advantages of one or more implementations described herein. For example, the term "solid state drive" (SSD) refers to a type of flash memory device that includes a flash memory chip, a dynamic random access memory (DRAM), and a controller. In various implementations, the flash memory chip is a NAND flash memory. In some implementations, the controller is configured to receive and provide data via an input / output (I / O) bus. In various implementations, the SSD is used to host virtual machines and other computing components.

[0028] As used herein, the term "erasure coding" (sometimes referred to as error coding and / or "EC") refers to a data protection method in which data is segmented, encoded with redundant data fragments, and stored across a collection of different locations or storage media to enable the data to be recovered in the event that some portions of the data are erased. Erasure codes may include parity checks in the form of simple parity checks, such as RAID4, RAID5, and RAID6. Additionally, erasure codes may also include more complex erasure codes, such as Reed-Solomon codes.

[0029] As used herein, "RAID" or "Redundant Array of Independent Disks" refers to combining multiple physical drives (e.g., SSDs) into a single storage unit. In various implementations, a RAID configuration provides redundancy to protect against drive failures. In some implementations, one or more storage controllers implement a RAID scheme. Additionally, the RAID schemes described herein refer to erasure code RAID schemes (including parity-based RAID schemes) that utilize parity to protect against partial or complete hardware failures. In practice, parity can be used to recover lost data in the event of a drive failure. For example, the storage system disclosed herein works with RAID4, RAID5, RAID6, and possibly additional future parity-based RAID schemes.

[0030] It is worth noting that while the RAID scheme described herein is particularly a parity-based RAID scheme, the techniques, methods, and approaches of the storage system can also be applied to more advanced erasure coding schemes. For example, the implementations described herein can work with other erasure coding schemes (such as Reed-Solomon codes) that utilize redundant information to mitigate drive failures for a set of drives (e.g., SSDs) that are logically interconnected to form a unified storage unit.

[0031] Additionally, the term "storage mapping table" (or simply mapping table) refers to a data structure that indicates the correlation between the logical block address and the physical location of the data for storage. For example, the mapping table indicates which SSD drive the target data is stored on, along with the physical address (e.g., memory block) on the drive. Often, for NAND chips, the mapping table is constantly updated to accurately reflect the data moved between one or more SSDs. In various implementations, the mapping table is stored in a volatile memory, such as DRAM. In some implementations, the mapping table or a copy of the mapping table is stored in a non-volatile memory (NVM), such as NAND flash memory.

[0032] As used herein, a "cloud computing system" refers to a network of connected computing devices that provides various services to computing devices (e.g., client devices, server devices, provider devices, customer devices, etc.). For example, as mentioned above, a distributed computing system may include a collection of physical server devices (e.g., server nodes) organized in a hierarchy that includes clusters, computing regions, virtual local area networks (VLANs), racks, fault domains, etc. In various implementations, the network is a virtual network or a network with a combination of virtual and real components. Cloud computing systems often utilize computing devices that include parity-based SSD solutions to facilitate cloud storage, virtual machines, and / or other services.

[0033] Additional details are now provided about the components and elements of the storage system. For example, Figure 1A to Figure 1B A computing device is illustrated that implements a storage system according to one or more embodiments. In particular, Figure 1A includes a computing device 102 and Figure 1B Additional details regarding components and elements of storage system 106 are provided.

[0034] like Figure 1A As shown, the computing device 102 includes an operating system 104 and a storage system 106. In various embodiments, the operating system 104 is associated with a processor, applications, and other computing components. Although the storage system 106 is shown as being separate from the operating system 104, the computing device 102 can be implemented at various locations within the computing device 102 (e.g., on separate hardware components), including being implemented in part or in whole by the operating system 104. In various implementations, the storage system 106 is implemented on a different device or multiple devices. Additionally, in many implementations, the computing device 102 implements a parity-based RAID scheme that utilizes parity-based error encoding to mitigate SSD drive failures. Figure 8 Additional details regarding computing devices, such as computing device 102, are provided.

[0035] In various implementations, computing device 102 is part of a cloud computing system. For example, computing device 102 is used for cloud storage, virtual machine operations, or various services provided by a cloud computing system that relies on a parity-based RAID solution. In alternative implementations, computing device 102 operates as a separate computing system and / or belongs to a non-cloud computing system.

[0036] like Figure 1A As shown, computing device 102 includes a storage system 106 having N solid state drives (SSDs), where N represents a non-zero integer. As shown, the N SSDs include solid state drive 0 110a, solid state drive 1 110b, and solid state drive N 110n (collectively referred to as SSD 110). Each SSD in SSD 110 includes a controller 112, a volatile memory 114, and a non-volatile memory 116.

[0037] In various implementations, SSD refers to a NAND flash memory storage device. For example, the non-volatile memory 116 includes physical NAND flash memory chips ("NAND chips"). For example, multiple NAND chips are connected to the controller 112. For illustration, the non-volatile memory 116 includes 8 NAND chips, each of which is connected to the controller 112. Further, each of the 8 NAND chips is connected to the bus of additional 4 NAND chips to form a set of 32 NAND chips.

[0038] According to some implementations, the non-volatile memory 116 can process storage access commands received via an input / output (I / O) bus. In some implementations, the controller 112 can be implemented as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microcontroller, and / or a microprocessor. In addition, the controller 112 can be integrated inside the SSD or can be located outside the SSD. Further, in various implementations, the controller 112 can facilitate read and write operations of the NAND chip of the SSD so that the operating system 104 does not need to perform buffering operations on the SSD.

[0039] In several implementations, the controller 112 may include a flash translation layer (FTL). In one or more implementations, the FTL is a layer of hardware and / or software that performs various operations on the SSD. For example, the FTL enables the storage system 106 to perform logical-physical address translation, garbage collection (GC), bad block management (BBM), page allocation, and error correction code (ECC). In various implementations, the controller 112 is connected to the volatile memory 114 via a connection enabled by the FTL.

[0040] In one or more implementations, the volatile memory 114 includes DRAM (or other types of RAM) that acts as a cache or buffer to store data waiting to be written to the non-volatile memory 116. In additional implementations, the volatile memory 114 stores a mapping table (or portion of a mapping table) for data stored on the non-volatile memory 116. In various implementations, the volatile memory 114 lacks the capacity to store a complete version of the mapping table.

[0041] In many implementations, SSDs 110 are identical drives. In alternative implementations, SSDs 110 are different. For example, one of SSDs 110 has a different storage size, chip manufacturer type, block or sector size, firmware version, and / or production version.

[0042] In some implementations, SSD 110 is implemented in a RAID scheme or system (e.g., RAID5). For example, in some instances, the RAID scheme includes N+1 SSD storage devices that provide the capacity of N devices. In various embodiments, the additional capacity of the "+1" drive can be used to store bit parity (or simply parity), and parity can be used to recreate data from a drive that has stopped working or failed. Additionally, in various implementations, SSD 110 utilizes another type of erasure code that can recover lost data from more than one failed SSD. For example, given N+M SSDs, up to M SSD failures can be recovered.

[0043] In an example implementation, the SSD has a native sector size of 4KB. In some instances, 512B still supports 512B addressing, which is not aligned to a 4KB boundary and may be required to perform read-modify-write operations. In various implementations, the storage system 106 performs interleaving (e.g., in multiples of 4KB) when implementing RAID on the SSD 110. Additionally, a single RAID stripe may have N×4KB sectors of data, where each sector is stored in a different SSD in the set of SSDs plus a 4KB sector, which is a bit parity for these N sectors.

[0044] Additionally, when writing data to SSD 110, if a single write is large enough to cover the entire RAID stripe, the RAID controller (i.e., storage system 106) can calculate parity across the new data for N sectors and write all N+1 sectors at once. However, if storage system 106 is writing data to a single sector of non-volatile memory 116 without all the data required to calculate the new parity, then the following conventional method provided above may encounter problems and inefficiencies. In fact, as noted above, existing computer systems must write new data to a block, read old data from another block, read old parity from another block, perform calculations to remove the old data from the parity, and write the new parity to the new block. Therefore, the process of this single write becomes two reads and two writes and various calculation steps, which causes performance degradation.

[0045] In contrast to existing computer systems, the storage system 106 can reduce the write process to a single write (when the parity write is merged with other new writes). More specifically, the storage system 106 performs an average of 1+1 / N writes for each sector write (i.e., every N writes plus the parity write becomes N+1 writes). In various embodiments, the storage system 106 accomplishes this by maintaining a mapping table in volatile memory 114 on one or more of the SSDs 110. Further, in many embodiments, the mapping table stored on one SSD includes mappings for other SSDs in the SSD 110. In practice, an SSD storing its own mapping table in its own DRAM would provide no protection or redundancy in the event of a drive failure because the drive mapping would be completely lost.

[0046] Additionally, unlike hard disk drives (HDDs), SSDs require constant remapping. For example, the first time a piece of data is stored, it is written to a first specific location in the NAND. Then, the next time the piece of data is written, it is written to some other location. The third time the piece of data is written, it will land in yet another location. Very rarely will a piece of data land in the same location as the previous version. In fact, for NAND, SSDs can utilize a multi-step erase and garbage collection process before a block is rewritten. Often, the garbage collection process involves erasing multiple adjacent blocks together and moving any valid data to a new location without marking blocks for deletion (e.g., TRIM blocks).

[0047] In various embodiments, SSD 110 uses NVMe and / or PCIe as an interface to communicate with operating system 104 and / or other SSDs. In one or more embodiments, storage system 106 uses PCIe to communicate with server or desktop computer processors or peripherals (e.g., SSD, NIC, GPU) via an I / O bus. In some embodiments, storage system 106 uses NVMe over PCIe to communicate between SSDs 110. In various embodiments, storage system 106 uses an NVMe-like interface to communicate between SSDs 110. As used herein, NVMe (Non-Volatile Memory Host Controller Interface Specification) refers to a non-volatile memory host controller interface that is an open logical device interface for accessing non-volatile storage media of a computing device (e.g., typically connected via a PCI Express (PCIe) bus).

[0048] Figure 1B Additional details are provided regarding the components and elements of the storage system 106 as noted above. In particular, Figure 1B Additional details are provided regarding the capabilities and components of storage system 106. As shown, storage system 106 includes various components and elements. For example, storage system 106 includes a read / write manager 120, a mapping manager 122, a parity manager 124, an error correction manager 126, and a garbage collection manager 128. Additionally, storage system 106 includes SSD 110 introduced above.

[0049] In various implementations, controller 112 facilitates one or more of the actions performed by storage system 106. For example, controller 112 enables operation of read / write manager 120, mapping manager 122, parity manager 124, error correction manager 126, and garbage collection manager 128.

[0050] As shown, the storage system 106 includes a read / write manager 120. In one or more implementations, the read / write manager 120 receives, accesses, provides, edits, creates, or otherwise manages data and data segments that are written to and read from the SSDs. For example, the read / write manager 120 writes data to a block of the non-volatile memory 116 and retrieves data from the same block.

[0051] As shown, the storage system 106 includes a mapping manager 122. In various embodiments, the mapping manager 122 facilitates the identification, determination, access, or otherwise management of mapping data between logical storage locations and physical storage locations of the SSD (non-volatile memory 116). In various embodiments, the mapping manager 122 generates a mapping table to store the mapping. As noted above, the mapping manager 122 can store the mapping table in the volatile memory 114 of one or more SSDs in the SSD 110. In many embodiments, the mapping table (or a portion of the complete mapping table) on one SSD includes mapping information for data stored on other SSDs.

[0052] As also shown, the storage system 106 includes a parity manager 124. In various implementations, the parity manager 124 generates, identifies, calculates, edits, modifies, accesses, provides, or otherwise manages parity (e.g., bit parity). For example, the parity manager 124 determines parity for one or more blocks being written. In some implementations, the parity manager 124 employs a reverse process to recover data lost on a failed drive.

[0053] As shown, the storage system 106 includes an error correction manager 126. In various embodiments, the error correction manager 126 generates, identifies, calculates, compiles, modifies, or otherwise manages error corrections. For example, the error correction manager 126 utilizes various methods and algorithms to ensure the accuracy and integrity of the data stored on the SSD 110.

[0054] As also shown, storage system 106 includes a garbage collection manager 128. In various embodiments, garbage collection manager 128 detects, removes, or otherwise manages garbage collection on SSD 110. For example, garbage collection manager 128 performs garbage collection to free up free space and blocks within SSD 110.

[0055] Additional details regarding an example implementation of the storage system 106 are discussed in relation to the following figure. For illustration, Figure 2An example workflow for efficiently managing and writing new data using an SSD with a parity-based RAID scheme according to one or more embodiments is provided. As shown, Figure 2 An overview 200 of the storage system 106 is illustrated.

[0056] like Figure 2 As shown, the overview 200 includes a first element 210 showing data maintained in non-volatile memory across a set of SSDs using parity-based error coding. For example, in various embodiments, the storage system 106 generates a mapping table that links logical addresses of data to their corresponding physical locations (e.g., memory blocks) stored across a RAID or RAID-like storage scheme (e.g., a set of SSDs) that utilizes parity to protect against drive loss. In addition, the storage system 106 stores the mapping table on one or more of the SSDs in the set of SSDs.

[0057] As shown, the overview 200 includes a second element 220 that maintains multiple portions of a mapping table across multiple SSDs, the multiple portions of the mapping table mapping to additional data fragments stored in other SSDs in the SSD set. For example, according to some embodiments, the storage system 106 divides the mapping table into multiple portions. In addition, the storage system 106 stores the first portion in a volatile memory of a first SSD in the SSD set. Additionally, the first portion may include mappings corresponding to physical addresses (e.g., non-volatile memory) of other SSDs in the SSD set. Further, in some embodiments, the first portion may include some mappings of data stored on the first SSD. Additionally, the storage system 106 stores the second portion of the mapping table in a volatile memory of a second SSD in the SSD set. For example, the second portion may include mappings corresponding to physical addresses on the first SSD. Additionally, the second portion may include mappings to physical addresses on other SSDs in the SSD set. In this way, if one of the SSDs fails, the parity and mappings on the other SSDs in the SSD set can be used to recover the second SSD.

[0058] As shown, overview 200 includes updating a third element 230 of a set by writing new data to the set without reading previously written data segments or their corresponding parity segments. For example, in various embodiments, storage system 106 receives new data segments to store in a set of SSDs. In response, storage system 106 writes the new data to an available block of a non-volatile memory. In many instances, storage system 106 writes the new data segment with other new data segments. Further, the storage system generates and writes parity for the new data segment.

[0059] Additionally, in various examples, the storage system 106 marks old data for deletion, which will be cleared during future garbage collection. Further, when a new data segment is replacing an existing data segment, the storage system 106 updates the corresponding mapping table portion to map to the location (e.g., memory block) of the new data segment. Additional details on storing mapping tables across SSDs in the SSD set and efficiently writing new data segments will be described below in conjunction with Figures 3 to 6B is provided.

[0060] Figure 3 A block diagram of an asymmetric implementation of a distributed RAID is illustrated, which stores a mapping table on an SSD according to one or more implementations. As shown, Figure 3 The host 302 and a set of SSDs (eg, SSD 0 304a, SSD 1 304b, SSD N 304n) are included. In various implementations, Figure 3 The components shown in are part of a computing device, such as computing device 102 .

[0061] As shown, the SSD in the SSD set includes a controller 306 and a NAND 312, which respectively represent the above-mentioned Figure 1A 1 and 10. In addition, SSD 0 304a includes DRAM 308, which represents an example of volatile memory 114 described above. Although not shown, other SSDs in the SSD set may or may not include DRAM. Further, host 302 may represent operating system 104 or hardware controller as described above.

[0062] As noted, Figure 3 An asymmetric distributed RAID 300 (i.e., an asymmetric distributed parity-based RAID) across N+1 SSDs (i.e., a set of SSDs) is shown, such as a RAID 4 or RAID 5 scheme, where the SSDs are numbered from 0 to N. Figure 3 In the illustrated design, it is assumed that SSD 0 304a has sufficient DRAM 308 capacity to hold a mapping table 310 that maps addresses for all of the N SSDs. For simplicity, SSD N 304n holds parity, but it can be any of the SSDs (e.g., SSD 0 304a can be parity). In one or more embodiments, parity is distributed across the set of SSDs.

[0063] In various embodiments, the controller 306 in each SSD may represent an SSD controller application specific integrated circuit (ASIC). As noted above, the controller may enable the storage system 106 to perform various actions related to storing and accessing data on the SSD. In particular, Figure 3 In FIG. 3 , the controller 306 on SSD 0 304a includes a mapping table 310. In some embodiments, SSD 0 304a is the only SSD in the set of SSDs that exposes the namespace to the host 302. Accordingly, in Figure 3 In the example, SSD 0 304a represents a primary SSD, and SSD 1 304b to SSD N 304n represent secondary SSDs. Additionally, as used herein, in the context of NVMe technology, a namespace refers to a set of logical block addresses (LBAs) accessible to host software. For example, the controller 306 provides the host 302 with access to a target namespace using a namespace ID (NSID).

[0064] In one or more implementations, a mapping table 310 (e.g., a flash translation layer (FTL) mapping table) is stored in DRAM 308 of the primary SSD (i.e., SSD 0 304a). In some examples, the mapping table 310 maps at a granularity of 4KB and a RAID stripe, with N×4KB data blocks and one 4KB parity block.

[0065] Further, in various embodiments, the primary SSD at least initially handles read and write commands from the host 302. In the case of block writes, the primary SSD may combine N writes of 4KB each into a RAID stripe and write individual blocks to the NAND of the corresponding secondary SSD (e.g., at an arbitrary LBA). In addition, the primary SSD may calculate parity and store it to a parity SSD (e.g., SSD N 304n).

[0066] With respect to I / O bus transfers, the storage system 106 can utilize various options, including that the primary SSD initially receives the command. As an example, if the command is a 4KB block write command, the storage system 106 causes the primary SSD to determine the RAID stripe placement within the secondary SSD and forwards the command to the appropriate secondary SSD. The SSD can communicate with the host 302 to obtain data (e.g., via DMA (direct memory access)) and write data. In this example, the primary SSD can also DMA data for calculating parity.

[0067] The above examples assume that the DMA is ongoing and the DMA data buffer remains unchanged. However, if the data buffer may change, the storage system 106 can cause the primary SSD to DMA data from the host, calculate parity, and pass the data to the appropriate secondary SSD to write to its NAND storage. In the latter case, the storage system 106 can utilize point-to-point writes for PCIe interfaces and / or NVMe I / O writes. In an alternative embodiment, the storage system 106 can rely on the use of a controller memory buffer (CMB) in the secondary SSD.

[0068] In some instances, a single read or write operation is about multiple 4KB blocks. In the case of writing multiple 4KB blocks, each 4KB memory block write can be treated as a separate 4KB write and processed as described above. In the case of reading multiple 4KB blocks, the storage system 106 can send a separate 4KB read to the secondary SSD. For example, the secondary SSD can DMA data to the host 302. Additionally, the secondary SSD can send a response to the primary SSD, which can then send a response to the host 302. In the case of a read greater than 4KB, the storage system 106 can cause the primary SSD to split it into multiple 4KB reads. Further, in the case where the write is not aligned to a 4KB boundary, the storage system 106 can cause the primary SSD to perform a read-modify-write operation.

[0069] In various embodiments, the storage system 106 enables the primary SSD to manage space in the secondary SSD. As a first example, if the NAND in the secondary SSD is the same as the NAND in the primary SSD, the storage system 106 can manage the NAND at the block level. For example, the storage system 106 enables the primary SSD to allocate stripes of erased blocks (e.g., one per SSD, including the primary SSD itself) and program data therein. Here, the entire stripe can be closed at once. Then, at some time in the future, the storage system 106 performs garbage collection (GC) on the entire stripe to make it available for reuse. Additionally, the storage system 106 can perform bad block management on each individual SSD while letting the primary SSD handle other operations.

[0070] In a second example, if the secondary SSD has (or can have) a different NAND type than the primary SSD, the storage system 106 can employ a higher level of abstraction. To illustrate, rather than managing erase blocks, the storage system 106 causes the secondary SSD to present a linear LBA range to the primary SSD. In various embodiments, this range is the same across all SSDs (including the primary SSD itself). In the case of varying capacities between SSDs, in some instances, the storage system 106 utilizes the capacity of the smallest SSD. In other instances, the storage system 106 utilizes the capacity of the other SSD, either the primary SSD or the secondary SSD.

[0071] In additional embodiments, the storage system 106 can cause the primary SSD to use each range in the LBA range in turn, including its own range. For example, after the first pass, if the sequential write encounters valid memory blocks, the storage system 106 can cause the primary SSD to skip them and trigger garbage collection (GC) in the SSD.

[0072] In various embodiments, the storage system 106 can utilize a circular buffer to manage writing data to the SSDs of the set of SSDs. For example, the storage system 106 can use a circular buffer to handle and resolve the problem of unequal SSD sizes in the set of SSDs. For example, by using a circular buffer, the storage system 106 can cause each SSD to use its own boundaries for garbage collection, thereby resolving the problem of SSDs having unequal erase block sizes. In this case, the storage system 106 can cause each SSD to perform its own garbage collection to completely resolve the problem of unequal erase block sizes in the set of SSDs.

[0073] In various embodiments, the storage system 106 utilizes equal circular buffers across the SSDs in the set of SSDs. In other words, in these embodiments, the size of the entire circular buffer is the same, even if the erase block sizes are different. Here, although the circular buffer is the same size globally, the storage system 106 can cause each of the SSDs to utilize its own boundaries, including garbage collection boundaries. For example, given an entire circular buffer of 1GB in size, if a first SSD has an erase boundary of 16MB and a second SSD has an erase boundary of 32MB, the smaller SSD will need to be executed 64 times, while the larger SSD will only be executed 32 times. In this way, the storage system 106 does not need to worry about the garbage collection boundaries of individual SSDs, and can cause each SSD to perform garbage collection in turn according to its own size.

[0074] To illustrate, if all of the SSDs in a collection of SSDs are exactly the same model, then it is relatively easy to perform write and garbage collection operations for the storage system 106. In particular, the storage system 106 can split the controller across each of the SSDs and perform operations regardless of whether the controller is split. However, if the SSDs differ in any way (e.g., different models, different capacities, different block sizes, different garbage collection boundaries, different manufacturers, or different device generations), problems can arise, and particularly with respect to garbage collection. For example, if garbage collection on each SSD occurs at different boundaries, the storage system 106 may over-delete portions of the SSDs.

[0075] In various embodiments, garbage collection may incur overhead in the embodiments described above. However, existing computer systems also experience similar garbage collection overhead. Further, in many embodiments, storage system 106 performs less garbage collection than existing computer systems because existing storage systems 106 perform double writes that require twice as much garbage collection.

[0076] As additional details regarding the circular buffer, in some instances, the storage system 106 can cause the primary SSD to promote LBA ranges from each SSD in a manner that makes it simple for the secondary SSDs to manage their NAND space with minimal memory requirements. In various embodiments, the storage system 106 can cause the primary SSD to manage garbage collection so that the secondary SSDs do not have to track which blocks are valid. To ensure this implementation, the storage system 106 can utilize reserved space in the LBA range for garbage collection. However, in some cases, individual SSDs will reserve a small portion of blocks, primarily for bad block management. In some embodiments, if multiple namespaces are supported, each namespace can have its own LBA range, and the size of the LBA range will determine the number of reserved spaces (OP) for that namespace. Additionally, different namespaces can have different numbers of reserved spaces.

[0077] In some embodiments, when the namespace is first allocated, the associated LBA range of one or more SSDs in the set of SSDs is marked as ready for deletion (e.g., the blocks are in a TRIM state). In these embodiments, the storage system 106 can cause the primary SSD to use the LBA range as a circular buffer that writes blocks from beginning to end. For example, when the primary SSD receives new memory block writes, the storage system 106 causes the primary SSD to eventually overwrite some previously written memory blocks. In various embodiments, the storage system 106 does not need to send TRIM commands to the secondary SSDs storing these overwritten blocks because the primary SSD will remember that these blocks have invalid addresses.

[0078] In additional embodiments, during subsequent writes to the circular buffer, the primary SSD may encounter valid and invalid blocks. Here, the storage system 106 can cause the primary SSD to direct writes to invalid blocks in order while skipping valid blocks. In some embodiments, the storage system 106 can cause the secondary SSD to maintain at least one free super block to handle these writes.

[0079] In the case of skipping a block, in various embodiments, the storage system 106 can cause the existing data to be copied to the new Super Block. Then, when the new Super Block is filled, the storage system 106 can cause the old Super Block to be erased because the old Super Block will not contain any valid data. In this way, the storage system 106 facilitates the secondary SSD to maintain the mapping at the Super Block granularity instead of the 4KB block granularity, thereby greatly reducing the memory requirements.

[0080] Figure 3 shows an asymmetric implementation of a distributed parity-based RAID, and Figure 4 The concepts are advanced to a symmetric implementation of a distributed parity-based RAID. In particular, Figure 4 A block diagram of a symmetric implementation of a distributed RAID 400 is illustrated that stores mapping tables across multiple SSDs according to one or more implementations. As shown, Figure 4 In various embodiments, the host 302 and a set of SSDs (eg, SSD 0 414a, SSD 1 414b, SSD N 414n) are included. Figure 4 The components shown in are parts of a computing device, such as computing device 102 .

[0081] As shown, each SSD in the SSD set includes the above combination Figure 3 Controller 306 and NAND 312 are described. Additionally, unlike asymmetric distributed RAID 300, the SSDs in the set of SSDs in symmetric distributed RAID 400 each include DRAM (e.g., volatile memory). In fact, in many embodiments, the DRAM on the SSD is often too small to accommodate a complete mapping table for multiple SSDs.

[0082] Further, as shown, each DRAM includes a mapping table (e.g., a portion of a mapping table). In particular, SSD0 414a includes a first DRAM 408a having a first mapping table 410a, and SSD1 414b includes a second DRAM 408b having a second mapping table 410b. This may continue until SSD N-1 includes an nth DRAM having an nth mapping table (i.e., each of these SSDs has 1 / n of the complete mapping table).

[0083] In some embodiments, SSD N 414n is a parity drive and does not include a mapping table. According to some embodiments, SSD N 414n is a parity drive that also includes a 1 / nth quota of mapping table 410n in DRAM. In various embodiments, parity is rotated between the span drives and SSD N 414n includes a mapping table within DRAM 408n.

[0084] The storage system 106 may utilize a mapping table stored locally on an SSD outside of an external controller, as illustrated by symmetric distributed RAID 400. For example, in various implementations, the storage system 106 generates a complete mapping table that maps all locations in the NAND locations of data stored across a set of SSDs, and then divides it into N parts that are stored on N SSD drives (e.g., all SSDs except the parity SSD).

[0085] In some embodiments, the storage manager provides a mapping table to the SSD driver that does not include a mapping to any NAND location on the SSD (i.e., all mappings to other SSDs). For example, the first mapping table 410a does not include any mappings to data stored in the NAND on SSD 0 414a. In one or more embodiments, the storage manager may provide a mapping table to the SSD driver that includes some mappings to its own NAND. However, in these embodiments, the mapping table also includes mappings for data stored in the NAND of other SSDs in the set of SSDs. In practice, the SSD includes mapping tables for other SSDs in a group for a subset of the range (e.g., the set depends on the RAID scheme minus one SSD's set of SSDs), rather than the SSD only including mappings for its own NAND (e.g., tied to itself).

[0086] In various examples, when the full mapping table is divided into multiple parts, the mapping table parts are mutually exclusive, meaning that logical addresses and corresponding physical NAND addresses are not repeated in different mapping table parts. In other examples, the storage system 106 can allow and / or provide some overlap between parts. For example, depending on the volatile memory available on the SSDs, the storage system 106 can provide the same logical mapping of physical locations on SSD 1 for SSD 0 and SSD N-1.

[0087] In one or more embodiments, a mapping table on one SSD partially maps to another SSD in the set of SSDs. For example, a second mapping table 410b on SSD 1 414b includes a dedicated mapping for SSD 0 414a. In other embodiments, a second mapping table 410b on SSD 1 414b includes mappings to SSD 0, SSD 1, SSD N-1, and SSD N. As noted, in these embodiments, the storage system 106 can exclude the mapping of the mapping table on the SSD to its own NAND, or the storage system 106 can allow the mapping table to map to its own NAND.

[0088] In many embodiments, the SSD set in the symmetrical distributed RAID 400 includes identical SSDs, each including sufficient DRAM capacity for match mapping to NAND plus sufficient capacity for temporary storage of data fragments so that the storage system 106 can perform other functions, as described above. In some embodiments, each SSD in the SSD set is capable of serving as a master SSD for some or all of the SSD sets in the SSD set.

[0089] To illustrate, in various embodiments, each SSD will act as a master SSD for 1 / N portion of each SSD's capacity (e.g., distributed across a set of SSDs). In some embodiments, each SSD may expose (N-1) / N portions of its NAND capacity as a secondary SSD for use by other SSDs in the set of SSDs. In some embodiments, an SSD acting as a master SSD may control more than 1 / N portion of the SSDs in the set of SSDs (but less than N SSDs in the set of SSDs).

[0090] In various implementations, an SSD may be presented to the host 302 as a separate SSD, and the namespace created within the SSD is adapted to the mapping table and DRAM of the SSD. In other words, each SSD (e.g., the controller 306 on each SSD) does not generate and fill a mapping table that is larger than its DRAM capacity. Instead, each SSD includes a smaller portion of the complete mapping table that is spread across all SSDs. This also provides the advantage of avoiding performance hot spots on any one SSD, and enables the storage system 106 to perform overcommitment of capacity and performance more efficiently.

[0091] As provided above, the architecture of symmetrical distributed RAID 400 can tolerate the failure of the NAND of at least one SSD (e.g., back-end failure). However, if the controller on the SSD that is the master SSD fails (e.g., front-end failure), the mapping table derived by the SSD may become unavailable. However, the remaining SSDs in the SSD set and their mapping tables will continue to operate.

[0092] The above scenario assumes that when an SSD fails, the entire SSD fails. However, if there is isolation between the NAND (e.g., back end) of the primary SSD and the controller and / or DRAM (e.g., front end), then a failure in the back end may not affect the front end (or vice versa). If so, the mapping table on the SSD will continue to work and the NAND data can be recovered via parity.

[0093] In some embodiments, the storage system 106 causes the primary SSD front end of each SSD to periodically save its FTL state (e.g., its mapping table) to a different secondary SSD (e.g., NAND on another SSD). In this way, if an SSD's front end fails, its mapping table can be retrieved almost losslessly. In some embodiments, the storage system 106 may have to rebuild the mapping table to apply any changes made since the last save checkpoint. In various embodiments, the storage system 106 may also have to repeat any outstanding commands that were lost in the failed front end (e.g., via the NVMe driver).

[0094] Additionally, in many embodiments, only N of the N+1 SSDs are primary SSDs because the capacity of one SSD is utilized for parity. In these embodiments, if the front end of the SSD fails, the N+1 SSD can take over the failed functionality in a seamless or nearly seamless manner (e.g., without physically replacing the drive).

[0095] In various embodiments, in the event of a power failure resulting in loss of volatile memory (e.g., DRAM), the storage system 106 can use the remaining power to copy the SSD's mapping table to NAND on the same SSD or a different SSD. For example, the SSD includes one or more backup capacitors that provide sufficient power to copy its mapping table (or portion of the mapping table) to non-volatile memory.

[0096] In some embodiments, if the mapping table needs to be rebuilt, the storage system 106 can perform a reverse mapping of the physical data to determine the corresponding logical address. In these embodiments, when writing data to the physical location, the storage system 106 also causes the logical address of the sector to be written. Additionally, in these embodiments, the storage system 106 can scan the SSDs and rebuild the mapping table, and / or split the mapping table into mapping table portions for storage across a set of SSDs, as described above.

[0097] Figure 5 A block diagram of a symmetrical distributed RAID with an external storage controller 520 is illustrated according to one or more embodiments. As shown, Figure 5 The host 302 and a set of SSDs (eg, SSD 0 514a, SSD 1 514b, SSDN 514n) are included. In addition, the SSD may include an SSD controller 522, a NAND 312, and a DRAM 508 having a mapping table 510. The NAND 312, the DRAM 508, and the mapping table 510 may be combined with the above. Figure 4 The described NAND 312, DRAM 408a, and first mapping table 410a are similar.

[0098] As shown, the host 302 includes a storage controller 520. In various embodiments, the storage system 106 causes the storage controller 520 to perform one or more functions of the master SSD described above. However, although the storage controller 520 is shown to be external to the set of SSDs, portions of the mapping table are stored across the DRAM of the SSDs as described above. In fact, the storage controller 520 does not need an external DRAM that includes a complete copy of the mapping table.

[0099] As mentioned, the SSD may include an SSD controller 522. In some embodiments, the SSD controller 522 is physically similar to the controller 306 described above, but performs fewer functions. For example, the storage system 106 performs SSD control functions via the SSD controller 522, while leaving other coordinated memory functions to the storage controller 520.

[0100] To elaborate, in some instances, the RAID functionality is performed by a centralized RAID controller that controls multiple storage devices (e.g., SSDs or HDDs). The asymmetric distributed RAID 300 and symmetric distributed RAID 400 described in this embodiment assume that there is no such controller. In this way, the storage system 106 avoids the cost of such a controller, which, in addition, is generally not available for NVMe SSDs.

[0101] However, if such a controller happens to be available, the storage system 106 may utilize it, as in Figure 5 520 in the external storage controller 520. For example, as noted above, one problem with centralized RAID controllers is that small memory block writes incur the overhead of two reads and two writes. The above embodiments show how the storage system 106 can avoid this problem by using the mapping table stored in the volatile memory of one or more SSDs to perform RAID functions. In fact, as in Figure 5 As shown in , the storage system 106 implements parity generation and data reconstruction (when needed) via the storage controller 520 (centralized RAID controller), but the storage system 106 retains the mapping table on one or more SSDs in the set of SSDs.

[0102] Accordingly, in various embodiments, the storage system 106 maintains a mapping table in the DRAM in the SSD as described above. In this embodiment, the storage system 106 causes the SSD to still redirect read commands as described above. However, the storage system 106 can cause a centralized RAID controller (e.g., storage controller 520) to place memory block writes. In addition, when a RAID rebuild is needed, the storage system 106 can cause the storage controller 520 to read portions of the mapping table from the SSD.

[0103] Now let’s turn to the next figure. FIG. 6A to FIG. 6B A sequence diagram for efficiently maintaining, writing, and accessing data on a distributed parity-based RAID is illustrated according to one or more embodiments. As shown, FIG. 6A to FIG. 6B The system includes a host 602 (e.g., operating system 104 and / or host 302) and a storage system 106, which includes a primary SSD 604 and a secondary SSD 606. The primary SSD 604 and the secondary SSD 606 may represent similar components as described above. As further shown, FIG. 6A to FIG. 6B A series of actions 600 performed by storage system 106 are included.

[0104] As shown, a series of acts 600 includes an act of maintaining data across a set of SSDs using parity-based error coding 610. For example, as described above, the storage system 106 may store data in a RAID 5 scheme that includes N+1 SSDs for storing data across N SSDs (and the capacity of the SSDs for parity).

[0105] As shown, the series of acts 600 includes an act 612 of creating a mapping table stored in DRAM across a set of SSDs. In various embodiments, the storage system 106 generates and distributes the mapping table across multiple SSDs in the set of SSDs. In particular, the storage system 106 stores portions of the mapping table in volatile memory (e.g., DRAM). In practice, as described above, the storage system 106 stores the mapping table within the set of SSDs, rather than storing the mapping table externally associated with an external storage controller. Furthermore, as described above, in some embodiments, the storage system 106 utilizes one or more storage controllers within the set of SSDs.

[0106] As also shown, the series of acts 600 includes an act 614 of receiving new data from the host 602. For example, in various embodiments, the storage system 106 receives new data to store in the set of SSDs, or receives new data to replace old segments of data stored in the set of SSDs. In various embodiments, the storage system 106 accumulates multiple segments of the new data before writing the new data to available memory blocks (e.g., NAND space) in the set of SSDs. For example, the storage system 106 stores the received data segments in volatile memory (e.g., DRAM) on the primary SSD 604 until it has a sufficient number of data segments.

[0107] After receiving the new data, the storage system 106 can determine where to write the data, as shown in act 616. For example, the storage system 106 can determine at the primary SSD 604 to write the new data to a memory block within the NAND on the secondary SSD 606.

[0108] As shown, the series of acts 600 includes an optional act 618 that utilizes a circular buffer to write new data across the set of SSDs in available memory blocks. As noted above, in various embodiments, the storage system 106 utilizes a circular buffer to address issues with unequal SSD sizes in the set of SSDs, and, in particular, issues that arise when garbage collection boundaries are uneven.

[0109] As shown, the series of behaviors 600 includes an behavior 620 of determining parity for the new data. For example, the storage system 106 causes the primary SSD 604 to calculate parity for the new data. In some instances, the storage system 106 can also determine the location of the memory where the parity is written. In the case where multiple fragments of the received data are written together, the storage system 106 can reduce the processing steps by calculating the parity once for different small memory block writes. In this way, the storage system can perform a write for the new data fragment, and the one parity write is shared by multiple fragments of the received data (for example, 5 data fragments + 1 parity = 6 writes, instead of 10 writes for 5 data fragments and 5 parity changes). Further, as shown in behavior 622 of the series of behaviors 600, the storage system 106 can write parity.

[0110] As shown, the series of acts 600 includes an act of updating one or more mapping table portions across the set of SSDs 624. For example, the storage system 106 causes the primary SSD 604 to update the mapping for the new data segment in the corresponding mapping table portion. As provided above, the storage system 106 can continuously update the mapping table to maintain an accurate mapping between the logical memory addresses and the corresponding physical NAND memory locations within the set of SSDs.

[0111] like Figure 6B As shown, the series of behaviors 600 includes an behavior 626 of performing garbage collection to clear obsolete data. For example, in various embodiments, the storage system 106 causes one or more SSDs to perform garbage collection to free up available memory sectors. As provided above, the primary SSD 604 can perform garbage collection on itself, and in some cases, can also perform garbage collection on one or more of the secondary SSDs 606.

[0112] As shown, a series of behaviors 600 includes an optional behavior 630, which copies a portion of the mapping table to another SSD for backup. As provided above, in various embodiments, the storage system 106 can cause the primary SSD 604 to record periodic backups of its memory table in a non-volatile memory (e.g., NAND memory) of one of the secondary SSDs 606. In this way, if the primary SSD 604 fails, the storage system 106 can recover the lost mapping table more quickly and efficiently with little data loss. Further, a power outage will not cause the saved copy of the mapping table to be lost.

[0113] As shown, the series of acts 600 includes an act of providing requested stored data 632. For example, the host 602 requests access to data stored in the set of SSDs. Accordingly, the storage system 106 causes the primary SSD 604 to look up the data from the mapping table and (e.g., together with the primary SSD 604 or the secondary SSD 606) provides the requested data to the host 602.

[0114] As shown, the series of acts 600 also includes an act 634 of detecting a failure of an SSD. For example, the storage system 106 determines that some or all of the SSDs in the set of SSDs have failed. In some cases, the failure is temporary, such as a power outage. In other cases, the failure is more permanent, such as a drive failure. Additionally, as shown, the series of acts 600 also includes an act 636 of using parity to retrieve lost data. For example, the storage system 106 uses parity to detect any lost data in the NAND, and / or recover any saved data from the DRAM, such as a backed-up mapping table. Otherwise, the storage system 106 can quickly rebuild the mapping table.

[0115] Now go to Figure 7 , which illustrates an example flow chart including a series of acts 700 for improving memory storage and access by utilizing a distributed parity-based RAID that stores mapping tables across multiple SSDs in accordance with one or more embodiments. Figure 7 The diagrams illustrate behaviors according to one or more embodiments, and alternative embodiments may omit, add, reorder, and / or modify any of the behaviors shown. Figure 7 The actions in may be performed as part of a method. Alternatively, the non-transitory computer readable medium may include instructions that, when executed by one or more processors, cause the computing device to perform Figure 7 In a further embodiment, the system may perform Figure 7 behavior in.

[0116] As shown, a series of acts 700 includes an act 710 of maintaining data in non-volatile memory across a set of SSDs. For example, act 710 may include maintaining data in non-volatile memory across a set of SSDs including a primary SSD and one or more secondary SSDs, wherein the set of SSDs utilizes parity-based error encoding to mitigate SSD drive failures. In various embodiments, act 710 includes maintaining data in NAND memory across a set of SSDs, the set of SSDs including a first SSD and a second SSD.

[0117] In various embodiments, act 710 includes utilizing a storage controller to control storing, reading, and writing data blocks across a set of SSDs. In one or more embodiments, the storage controller is external to the set of SSDs. In various embodiments, the storage controller is distributed across the set of SSDs.

[0118] In several embodiments, act 710 includes receiving a new data segment. In some instances, the new data segment updates the stored data segment. In an example embodiment, the parity-based error encoding is utilized to mitigate SSD drive failures, the error encoding distributes data across SSDs within the SSD set, and / or includes rotating parity across SSDs within the SSD set.

[0119] As further shown, the series of acts 700 includes an act 720 of storing a portion of a mapping table within a volatile memory of the primary SSD, the portion indicating locations of data segments on the secondary SSD. For example, act 720 may involve storing at least a portion of a mapping table within a volatile memory of the primary SSD, the at least a portion of the mapping table indicating first non-volatile memory locations where segments of data within one or more secondary SSDs are located.

[0120] In various embodiments, act 720 includes storing a first portion of a mapping table in a first DRAM on a first SSD, the portion indicating a location of a first segment of data in NAND storage of a second SSD; and / or storing a second portion of a mapping table in a second DRAM of a second SSD, the second portion of the mapping table indicating a location of a second segment of data in NAND storage of the first SSD. In one or more embodiments, act 720 includes storing a first portion of a mapping table in a volatile memory (e.g., DRAM) of a primary SSD, and / or storing at least a second portion of a mapping table in a volatile memory of one or more secondary SSDs.

[0121] As further shown, the series of acts 700 includes an act 730 of writing a new data segment to a secondary SSD. For example, the act 730 can include writing the new data segment at a second non-volatile memory location within one or more secondary SSDs. In one or more implementations, the act 730 includes utilizing a circular buffer to write the new data segment. In some implementations, the act 730 includes writing the new data segment to a size that is less than a complete memory sector.

[0122] As further shown, the series of acts 700 includes an act 740 of writing new parity for the new data segment within the set of SSDs. For example, act 740 can involve writing new parity for the new data segment at a third non-volatile memory location within the set of SSDs. In some implementations, act 740 includes writing the new data segment and the new parity to the set of SSDs without reading the data segment (e.g., or reading surrounding old data) or the parity corresponding to the data segment.

[0123] In some embodiments, act 740 includes: a second DRAM on the second SSD includes a second portion of a mapping table (the portion indicating that the second data segment is located in a second non-volatile memory location on the first SSD); a first portion of a mapping table in the first DRAM of the first SSD (e.g., a storage mapping table) also indicates that the second segment of data is located in a second non-volatile memory location on the first SSD; and / or a first portion of a mapping table in the first DRAM of the first SSD also indicates that the third segment of data is located in a third non-volatile memory location in the set of SSDs (not on the first SSD or the second SSD). In some embodiments, parity for the first data segment is located in a second non-volatile memory location (e.g., a memory block) in the set of SSDs (not on the second SSD).

[0124] As further shown, the series of acts 700 includes an act 750 of updating a mapping table at the primary SSD indicating a new location of the new data segment on the secondary SSD. For example, act 750 may include updating a mapping table within a volatile memory on the primary SSD to indicate a new non-volatile memory location of the new data segment.

[0125] In various embodiments, the series of acts 700 includes: storing a copy of at least a portion of the mapping table within a fourth non-volatile memory location within the one or more secondary SSDs. In some embodiments, the series of acts 700 includes: after writing the new data segment at the second non-volatile memory location within the one or more secondary SSDs, removing the data segment from the first non-volatile memory location within the one or more secondary SSDs using a garbage collection process. In certain embodiments, the series of acts 700 includes: writing the additional new data segment at the non-volatile memory location of multiple SSDs across the one or more secondary SSDs, wherein the multiple SSDs include different specifications (e.g., different sizes, capacities, block sizes, garbage collection blocks, manufacturers, software versions, etc.), and wherein writing the additional new data segment includes writing the additional new data segment to the multiple SSDs having different specifications using a circular buffer. In some embodiments, the non-volatile memory of each SSD in the set of SSDs is managed as one or more circular buffers.

[0126] A "network" is defined as one or more data links that enable electronic data to be transmitted between computer systems and / or modules and / or other electronic devices. When information is transferred or provided to a computer over a network or other communications connection (whether hardwired, wireless, or a combination of hardwired and wireless), the computer properly views the connection as a transmission medium. Transmission media may include networks and / or data links, which may be used to carry the required program code means in the form of computer-executable instructions or data structures and may be accessed by general-purpose or special-purpose computers. Combinations of the above should also be included within the scope of computer-readable media.

[0127] In addition, the network described herein may represent a network or a collection of networks (such as the Internet, an intranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a cellular network, a wide area network (WAN), a metropolitan area network (MAN), or a combination of two or more such networks) through which one or more computing devices can access the storage system 106. In practice, the network described herein may include one or more networks that use one or more communication platforms or technologies for transmitting data. For example, a network may include the Internet or other data links that enable electronic data to be transmitted between corresponding client devices and components of a cloud computing system (e.g., a server device and / or a virtual machine thereon).

[0128] Further, program code means in the form of computer executable instructions or data structures may be automatically transferred from a transmission medium to a non-volatile computer readable storage medium (device) (or vice versa) upon reaching various computer system components. For example, computer executable instructions or data structures received over a network or data link may be buffered in RAM within a network interface module (NIC) and then ultimately transferred to RAM at the computer system and / or to a less volatile computer storage medium (device) on the computer system. Thus, it should be understood that a non-volatile computer readable storage medium (device) may be included in a computer system component that also (or even primarily) utilizes a transmission medium.

[0129] Computer executable instructions include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform certain functions or a set of functions when executed by a processor. In some embodiments, computer executable instructions are executed by a general-purpose computer, which transforms the general-purpose computer into a special-purpose computer that implements the elements of the present disclosure. Computer executable instructions can be, for example, binary files, intermediate format instructions (such as assembly language), or even source code. Although structural features and / or method performance specific languages ​​have described the subject matter, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or behaviors described above. On the contrary, the described features and behaviors are disclosed as example forms of implementing the claims.

[0130] Figure 8 Illustrated are certain components that may be included within computer system 800. Computer system 800 may be used to implement the various computing devices, components, and systems described herein.

[0131] In various embodiments, computer system 800 may represent one or more client devices, server devices, and / or other computing devices described above. For example, computer system 800 may refer to various types of network devices capable of accessing data on a network, cloud computing system, or other system. For example, a client device may refer to a mobile device, such as a mobile phone, a smart phone, a personal digital assistant (PDA), a tablet computer, a laptop computer, or a wearable computing device (e.g., a headset or a smart watch). A client device may also refer to a non-mobile device, such as a desktop computer, a server node (e.g., from another cloud computing system), or other non-portable device.

[0132] The computer system 800 includes a processor 801. The processor 801 may be a general-purpose single-chip or multi-chip microprocessor (e.g., Advanced RISC (Reduced Instruction Set Computer), Machine (ARM)), a dedicated microprocessor (e.g., a digital signal processor (DSP)), a microcontroller, a programmable gate array, etc. The processor 801 may be referred to as a central processing unit (CPU). Although Figure 8 Computer system 800 shows only a single processor, in an alternative configuration, a combination of processors could be used (eg, an ARM and DSP).

[0133] The computer system 800 also includes a memory 803 in electronic communication with the processor 801. The memory 803 can be any electronic component capable of storing electronic information. For example, the memory 803 can be embodied as a random access memory (RAM), a read-only memory (ROM), a magnetic disk storage medium, an optical storage medium, a flash memory device in a RAM, an onboard memory included in a processor, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) memory, a register, etc., including combinations thereof.

[0134] Instructions 805 and data 807 may be stored in memory 803. Instructions 805 may be executed by processor 801 to implement some or all of the functionality disclosed herein. Execution of instructions 805 may involve the use of data 807 stored in memory 803. Any of the various module and component examples described herein may be implemented in part or in whole as instructions 805 stored in memory 803 and executed by processor 801. Any of the various data examples described herein may be an example of data 807 stored in memory 803 and used during execution of instructions 805 by processor 801.

[0135] The computer system 800 may also include one or more communication interfaces 809 for communicating with other electronic devices. The one or more communication interfaces 809 may be based on wired communication technology, wireless communication technology, or both. Some examples of the one or more communication interfaces 809 include a universal serial bus (USB), an Ethernet adapter, a wireless adapter operating according to the Institute of Electrical and Electronics Engineers (IEEE) 802.11 wireless communication protocol, Wireless communication adapter and infrared (IR) communication port.

[0136] The computer system 800 may also include one or more input devices 811 and one or more output devices 813. Some examples of the one or more input devices 811 include keyboards, mice, microphones, remote controls, buttons, joysticks, trackballs, touchpads, and light pens. Some examples of the one or more output devices 813 include speakers and printers. A specific type of output device that is typically included in the computer system 800 is a display device 815. The display device 815 used with the implementations disclosed herein may utilize any suitable image projection technology, such as a liquid crystal display (LCD), a light emitting diode (LED), gas plasma, electroluminescence, etc. A display controller 817 may also be provided for converting data 807 stored in the memory 803 into text, graphics, and / or (appropriate) dynamic images, which are shown on the display device 815.

[0137] The various components of the computer system 800 may be coupled together via one or more buses, which may include a power bus, a control signal bus, a status signal bus, a data bus, etc. For clarity, Figure 8 Various buses are illustrated as bus system 819 .

[0138] It will be appreciated by those skilled in the art that the present disclosure can be practiced in a network computing environment with various types of computer system configurations, including personal computers, desktop computers, notebook computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network personal computers, minicomputers, mainframe computers, mobile phones, personal digital assistants (PDAs), tablet computers, pagers, routers, switches, etc. The present disclosure can also be practiced in a distributed system environment, where local and remote computer systems are connected via a network (whether by a hardwired data link, a wireless data link, or a combination of hardwired and wireless data links) to perform tasks together. In a distributed system environment, program modules can be located in local and remote memory storage devices.

[0139] The technology described herein can be implemented in the form of hardware, software, firmware or any combination thereof, unless specifically described as being implemented in a particular manner. Any features described as modules, components, etc. can also be implemented together in an integrated logic device, or implemented separately as discrete but interoperable logic devices. If implemented in software, the technology can be implemented at least in part by a non-volatile processor-readable storage medium including instructions, which, when executed by at least one processor, executes one or more methods described herein. The instructions can be organized into routines, programs, objects, components, data structures, etc., which can perform specific tasks and / or implement specific data types, and can be combined or distributed as needed in various implementations.

[0140] Computer readable media can be any available media accessible to a general or special purpose computer system. A computer readable medium storing computer executable instructions is a non-volatile computer readable storage medium (device). A computer readable medium carrying computer executable instructions is a transmission medium. Therefore, by way of example and not limitation, implementations of the present disclosure may include at least two distinct computer readable media: a non-volatile computer readable storage medium (device) and a transmission medium.

[0141] As used herein, non-volatile computer-readable storage media (devices) may include RAM, ROM, EEPROM, CD-ROM, solid-state drives (“SSD”) (e.g., RAM-based), flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general or special purpose computer.

[0142] The steps and / or actions of the methods described herein may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is required for the proper operation of the described method, the order and / or use of steps and / or specific actions may be modified without departing from the scope of the claims.

[0143] The term "determining" encompasses a wide variety of actions, and thus, "determining" may include computing, calculating, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), ascertaining, etc. Furthermore, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. Furthermore, "determining" may also include resolving, selecting, choosing, establishing, etc.

[0144] The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements in addition to the listed elements. Additionally, it should be understood that references to "one embodiment" or "an embodiment" in this disclosure are not intended to be interpreted to exclude the existence of additional implementations that also include the listed features. For example, any element or feature described herein with respect to an implementation may be combined with any element or feature of any other implementation described herein, so long as they are compatible.

[0145] The present disclosure may be implemented in other specific forms without departing from its spirit or characteristics. The described implementation should be considered illustrative rather than restrictive. Therefore, the scope of the present disclosure is indicated by the appended claims rather than the aforementioned embodiments. Equivalent changes within the meaning and scope of the claims should be included within their scope.

Claims

1. A storage system, comprising: A set of solid state drives (SSDs), the set of SSDs utilizing erasure coding to mitigate SSD drive failures, wherein each SSD in the set of SSDs comprises a dynamic random access memory (DRAM) and a non-volatile memory; Storage controller; as well as a mapping table that maps locations of data stored in the non-volatile memory across the set of SSDs, Wherein a first DRAM on a first SSD in the set of SSDs includes a first portion of the mapping table, the first portion of the mapping table indicating that a first segment of the data is located in a first non-volatile memory location on a second SSD in the set of SSDs.

2. The storage system of claim 1, wherein a second DRAM on the second SSD includes a second portion of the mapping table, the second portion of the mapping table indicating that a second segment of the data is located within a second non-volatile memory location on the first SSD.

3. The storage system according to claim 1, wherein: The first portion of the mapping table in the first DRAM of the first SSD further indicates that a second segment of the data is located in a second non-volatile memory location on the first SSD; and The first portion of the mapping table in the first DRAM of the first SSD also indicates that a third segment of the data is located in a third non-volatile memory location in the set of SSDs other than on the first SSD or the second SSD. 4 . The storage system of claim 1 , wherein the non-volatile memory in each SSD in the set of SSDs is managed as one or more circular buffers.

5. The storage system of claim 1, wherein the storage controller is distributed across the set of SSDs.

6. A computer-implemented method comprising: maintaining data in a non-volatile memory across a set of solid-state drives (SSDs), the set of SSDs comprising a primary SSD and one or more secondary SSDs, wherein the set of SSDs utilizes erasure coding to mitigate SSD drive failures; storing at least a portion of a mapping table in a volatile memory of the primary SSD, the at least a portion of the mapping table indicating a first non-volatile memory location where a segment of the data in the one or more secondary SSDs is located; writing a new data segment at a second non-volatile memory location within the one or more secondary SSDs; writing new parity for the new data segment at a third non-volatile memory location within the set of SSDs; as well as The mapping table within the volatile memory on the primary SSD is updated to indicate a new non-volatile memory location of the new data segment.

7. The computer-implemented method of claim 6, wherein writing the new segment of data and the new parity to the set of SSDs does not include reading the segment of data or the parity corresponding to the segment of data.

8. The computer-implemented method of any one of claims 6 to 7, further comprising receiving the new data segment, wherein the new data segment updates the segment of the data.

9. The computer-implemented method of any one of claims 6 to 8, further comprising storing a copy of the at least one portion of the mapping table in a fourth non-volatile memory location within the one or more secondary SSDs.

10. The computer-implemented method of any one of claims 6 to 9, further comprising: After writing the new data segment at the second non-volatile memory location within the one or more secondary SSDs, a garbage collection process is utilized to remove the segment of data from the first non-volatile memory location within the one or more secondary SSDs.

11. The computer-implemented method of claim 6, wherein writing the new data segment at the second non-volatile memory location comprises writing the new data segment as less than a complete memory sector of data.

12. The computer-implemented method of claim 6, wherein storing the at least one portion of the mapping table within a volatile memory of the primary SSD comprises: A first portion of the mapping table is stored in a volatile memory of the primary SSD, and at least a second portion of the mapping table is stored in a volatile memory of the one or more secondary SSDs.

13. The computer-implemented method of claim 6, further comprising: Using a storage controller to control reading and writing of data segments across the set of SSDs; as well as Writing additional new data segments at non-volatile memory locations across multiple SSDs in the one or more secondary SSDs, wherein the multiple SSDs include different specifications, wherein writing the additional new data segments includes utilizing a circular buffer to write the additional new data segments to the multiple SSDs having different specifications.

14. The computer-implemented method of claim 6, wherein the erasure code comprises parity-based error coding used to mitigate SSD drive failures, comprising: Distributing data across the SSDs within the set of SSDs; or Parity is rotated across the SSDs within the set of SSDs.

15. A computer program configured to perform the method of any one of claims 6 to 14 when executed on a processor of a computing system.

Images