SSD and storage array based on erasure code mechanism

Abstract

This invention provides an SSD and storage array based on erasure coding mechanism. The SSD includes a NAND flash memory array and a controller chip. The controller chip includes SRAM, a CPU, and an erasure coding hardware accelerator. The erasure coding hardware accelerator is used to generate parity blocks and repair data, and includes a parity data generation module and a data repair module. The SSD provided by this invention integrates an erasure coding hardware accelerator into the controller chip, thereby applying erasure coding technology to the SSD and improving its storage efficiency and reliability.

Description

Technical Field

[0001] This invention relates to the field of storage, specifically to the field of solid-state drives (SSDs), and more specifically, to an SSD and storage array based on erasure coding. Background Technology

[0002] Solid State Drives (SSDs) store data in NAND flash memory arrays. Their basic operating units are physical pages (typically 16KB or larger) and physical blocks (composed of multiple pages). The reliability of the data stored in an SSD is crucial. In addition to user data, the NAND flash memory array also stores critical firmware data, especially firmware runtime system data and firmware program code data. If the firmware runtime system data is corrupted, all user data will become inaccessible, while firmware program failure will render the SSD unusable.

[0003] On the one hand, in existing SSDs, the protection of critical firmware data mainly adopts a simple redundancy scheme based on physical storage units: one is physical page-level dual backup, which involves writing the same system data (such as a mapping table) to two independent physical pages. This scheme has significant drawbacks: First, the size of effective system data is usually a few KB, far smaller than the fixed capacity of a physical page. During writing, invalid data must be used to fill the remaining space, resulting in a large amount of wasted storage space; second, this scheme only provides another identical copy of the data. If the data itself has an uncorrectable error (UNC) at the bit level due to reasons such as charge leakage in the storage unit, both copies will be invalid and cannot be repaired. The other is physical block-level repeated writing, which involves repeatedly writing the complete firmware program data to all physical pages of a physical block and backing it up in another physical block. Although this scheme improves reliability, the storage efficiency is extremely low (usually less than 1%), and a physical block can only store a single type of data, which cannot meet the need to mix and store multiple types of firmware data to improve space utilization.

[0004] On the other hand, erasure coding technology has been maturely applied to improve data reliability. Its core idea is to divide a piece of data into K data blocks and encode them into M parity blocks. If any K data blocks or parity blocks survive, the original data can be recovered, thus achieving a good balance between fault tolerance and storage overhead. However, this type of technology is typically geared towards data center-level distributed storage objects, designed based on randomly accessible disk media, and handling large datasets in the MB / GB range. Applying erasure coding technology, used for macroscopic objects, to protect the firmware data inside an SSD presents significant challenges: implementing data management at the granular level of data blocks and parity blocks through erasure coding incurs very high computational overhead. In data center-level distributed storage scenarios, this computational overhead is acceptable due to ample CPU resources; however, the computational resources of existing SSDs are extremely limited. Directly using the SSD's CPU for computation would lead to significant management latency, causing the SSD's performance to degrade to an unacceptable level.

[0005] In summary, existing SSDs all rely on replication mechanisms to store important data, which have limited storage efficiency and data reliability. Furthermore, due to the limited CPU computing power of SSDs, erasure coding mechanisms cannot be smoothly applied to SSDs.

[0006] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide an SSD based on erasure coding mechanism.

[0008] The objective of this invention is achieved through the following technical solution: According to a first aspect of the present invention, the SSD based on erasure coding mechanism provided by the present invention includes a NAND flash memory array and a controller chip. Data is stored in the NAND flash memory array by the SSD in the form of multiple data blocks and parity blocks of equal size. The parity blocks are used to repair erroneous data when errors occur in the data stored in the NAND flash memory array. The controller chip includes SRAM, a CPU, and an erasure coding hardware accelerator, wherein: the SRAM is used to cache data to be stored waiting to be written to the NAND flash memory array, data to be repaired read from the NAND flash memory array, and parity blocks and repair data generated by the erasure coding hardware accelerator. The CPU calculates check block generation parameters based on the data to be stored and data repair parameters based on the data to be repaired. The check block generation parameters include the address of the data to be stored in SRAM, the number of data blocks to be stored, the size of each data block, the specified number of check blocks to be generated, and the specified SRAM address for storing the check blocks. The data repair parameters include the address of the data to be repaired in SRAM, the number of data blocks to be repaired, the size of each data block to be repaired, the number of check blocks to be repaired, the location information of the erroneous data blocks, and the specified SRAM address for storing the repaired data. An erasure coding hardware accelerator is used to generate check blocks and repaired data, including a check data generation module and a data repair module. The check data generation module is used to: obtain the check block generation parameters calculated by the CPU; obtain the data to be stored based on its address in SRAM; and, based on the obtained check block generation parameters and the data to be stored, generate a specified number of check blocks using an erasure coding algorithm and then place the generated check blocks into the specified SRAM address. The data repair module is used to: obtain data repair parameters calculated by the CPU; obtain the data to be repaired according to the address of the data to be repaired in SRAM; and, based on the obtained data repair parameters and the data to be repaired, complete the data repair using erasure coding algorithm and place the repaired data at the specified SRAM address.

[0009] The beneficial effects of this solution are: integrating erasure coding hardware accelerators into the main control chip, using dedicated erasure coding hardware accelerators to replace the CPU in generating check blocks and repairing damaged data, avoiding the management latency caused by excessive CPU resource consumption, and realizing the application of erasure coding on SSDs.

[0010] Optionally, the erasure coding hardware accelerator is a fixed-function circuit.

[0011] The beneficial effect of this approach is that, compared to using a programmable core (such as an FPGA), using a fixed-function circuit (ASIC) can achieve higher performance with lower power consumption.

[0012] Optionally, the NAND flash array is configured with an error correction mechanism based on a fixed-size low-density parity check code, where the size of each data block to be stored is equal to the size of the low-density parity check code.

[0013] The beneficial effect of this scheme is that it aligns the length of the "block" of the upper-layer erasure coding with the "processing unit" of the lower-layer LDPC. This means that if a data block is corrupted, both LDPC and erasure coding mechanisms can repair the data, thus making the data more reliable.

[0014] Optionally, the NAND flash array includes multiple physical blocks, each physical block includes multiple physical pages, each physical page has a fixed storage capacity, data blocks and parity blocks are stored in the same physical page, and the total size of the data blocks and parity blocks is equal to the fixed storage capacity of the physical page.

[0015] The beneficial effect of this scheme is that when writing system data of only a few KB as mentioned in the background technology, there is no need to fill in invalid data, but instead fill in useful check blocks, thus avoiding the waste of storage space.

[0016] Optionally, the data to be stored in the SRAM includes multiple different types of data, and the NAND flash array includes multiple physical blocks, each physical block including multiple physical pages. Within the same physical block, there are parity blocks and data blocks of multiple different types of data, while within the same physical page, there are only parity blocks or data blocks of one type of data.

[0017] The beneficial effects of this scheme are: it allows different types of data to share the same check block, so different types of data can be written to the same physical block, and even when writing firmware data, they do not need to be completely repeated, which greatly improves storage efficiency.

[0018] Optionally, multiple different types of data refer to key data including firmware runtime system data and firmware program data.

[0019] Optionally, the ratio between the number of data blocks and the number of parity blocks is determined based on the desired storage efficiency and fault tolerance.

[0020] The beneficial effect of this solution is that it can balance storage efficiency and fault tolerance based on the importance of the data.

[0021] Optionally, the ratio between the number of data blocks and the number of check blocks can be one to one, two to one, or three to two.

[0022] The beneficial effects of this scheme are: it can balance storage efficiency and fault tolerance based on the importance of the data. When the ratio of data blocks to parity blocks is 2:1, the storage efficiency is 50% and the fault tolerance is 50%; when the ratio is 2:1, the storage efficiency is 66.6% and the fault tolerance is 33.3%; when the ratio is 3:2, the storage efficiency is 60% and the fault tolerance is 40%.

[0023] Optionally, the main control chip also includes a NAND flash memory controller for reporting the number of erroneous data blocks to the CPU. The CPU is configured to trigger the data repair module to repair the data to be repaired when the number of received erroneous data blocks is less than or equal to the number of check blocks in the data repair parameters.

[0024] The beneficial effect of this solution is that it allows SSDs to shift from "passive error correction" to "active repair," prioritizing the initiation of a better erasure coding repair path instead of the traditional read retry.

[0025] According to a second aspect of the present invention, a storage array is provided, comprising a controller and storage units, the controller being used to manage the storage units, the storage units being composed of a plurality of SSDs based on erasure coding as described in the first aspect of the present invention.

[0026] Compared with the prior art, the advantages of the present invention are as follows: The SSD provided by the present invention integrates erasure coding hardware accelerator in the main control chip, thereby realizing the application of erasure coding technology in SSD, improving the storage efficiency and storage reliability of SSD. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the architecture of an SSD based on erasure coding mechanism according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating a data storage method in a physical page of an SSD based on erasure coding mechanism according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating a data storage method in a physical block for an SSD based on erasure coding mechanism according to an embodiment of the present invention. Figure 4 This is a schematic diagram of erroneous data blocks in the physical pages of an SSD based on erasure coding mechanism according to an embodiment of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] As mentioned in the background section, existing SSDs are all based on a copy backup mechanism to store important data. Their storage efficiency and data reliability are limited. Furthermore, due to the limited CPU computing power of SSDs, erasure coding mechanisms cannot be smoothly applied to SSDs.

[0030] To address the problems existing in current SSDs, the inventors, through analysis of the erasure coding calculation process, discovered that fixed-function circuits (ASICs) can replace the CPU in calculating erasure codes, significantly improving computational efficiency. Specifically, the calculation of erasure codes (especially the most commonly used Reed-Solomon codes) can be mathematically reduced to linear operations over a specific algebraic field (Galois field, GF), and its computational characteristics include: High degree of parallelism. When computing check blocks, each check block is a linear combination of all the original data blocks, which means that the computation of multiple check blocks can be performed simultaneously and independently. For example, to generate M check blocks, ideally they can be computed simultaneously using M parallel computing units.

[0031] A regular data access pattern. The encoding or decoding process involves reading consecutive blocks of data in a fixed pattern, multiplying them by a fixed coefficient, and then summing them. This pattern is very regular and is well-suited for implementation using pipelining and fixed-address step direct memory access (DMA), without the need for complex cache prediction and out-of-order execution.

[0032] It is computationally intensive, but its logic control is simple. The majority of the computation time is spent on multiplication and addition over the domain, rather than complex branching or data dependency resolution. The algorithm flow is fixed: "read data, matrix multiplication or iterative calculation, output result", without complex control flows such as if-else statements or loop jumps.

[0033] Deterministic computational complexity. For fixed parameters, the number of operations required to complete one encoding or decoding operation is deterministic and predictable, which facilitates precise planning of the number of computing units and clock cycles during hardware design.

[0034] These characteristics demonstrate that erasure coding is well-suited for computation by ASICs. If such a hardware processing unit could be added to an SSD, an SSD based on erasure coding could be designed, fundamentally improving the storage efficiency and fault tolerance of existing SSDs. Based on this design concept, this invention provides an SSD based on erasure coding.

[0035] In summary, the erasure coding-based SSD provided by this invention includes a NAND flash memory array and a controller chip. Data is stored in the NAND flash memory array by the SSD in the form of multiple data blocks and parity blocks of equal size. The parity blocks are used to repair erroneous data when errors occur in the data stored in the NAND flash memory array. The controller chip includes SRAM, a CPU, and an erasure coding hardware accelerator. The SRAM is used to cache data to be stored awaiting writing to the NAND flash memory array, data to be repaired read from the NAND flash memory array, and parity blocks and repair data generated by the erasure coding hardware accelerator. The CPU is used to calculate parity block generation parameters based on the data to be stored and data repair parameters based on the data to be repaired. The parity block generation parameters include the address of the data to be stored in the SRAM, the number of data blocks to be stored, the size of each data block, a specified number of parity blocks to be generated, and a specified SRAM address for storing the parity blocks. The data repair parameters include the address of the data to be repaired in the SRAM, the number of data blocks to be repaired, the size of each data block to be repaired, the number of parity blocks to be repaired, the location information of the erroneous data blocks, and the specified SRAM address for storing the repair data. The erasure coding hardware accelerator is used to generate check blocks and repair data. It includes a check data generation module and a data repair module. The check data generation module is responsible for: obtaining check block generation parameters calculated by the CPU; retrieving the data to be stored from the SRAM based on its address; generating a specified number of check blocks using the erasure coding algorithm based on the obtained check block generation parameters and the data to be stored; and placing the generated check blocks into the specified SRAM address. The data repair module is responsible for: obtaining data repair parameters calculated by the CPU; retrieving the data to be repaired from the SRAM based on its address; and completing the data repair using the erasure coding algorithm based on the obtained data repair parameters and the data to be repaired, and placing the repaired data into the specified SRAM address.

[0036] As can be seen, this invention integrates erasure coding hardware accelerator into the SSD controller chip. By using a dedicated erasure coding hardware accelerator to replace the CPU in generating check blocks and repairing damaged data, the management latency caused by consuming a large amount of CPU resources is avoided, thus realizing the application of erasure coding on SSDs.

[0037] To better understand the present invention, a detailed description is provided below with reference to specific embodiments.

[0038] Reference Figure 1The erasure coding hardware accelerator is integrated into the main control chip and connected to the CPU and SRAM respectively. It can receive calculation instructions from the CPU without obstruction, read the data to be stored or repaired in the SRAM, and register the generated verification block or repaired data in the SRAM. The data is eventually written to the NAND flash array through subsequent processes.

[0039] According to one embodiment of the present invention, the erasure coding hardware accelerator is a fixed-function circuit. Compared with implementation using a programmable core (such as an FPGA), implementation using an ASIC can achieve higher performance with lower power consumption.

[0040] According to one embodiment of the present invention, the NAND flash memory array is configured with an error correction mechanism based on a fixed-size low-density parity check (LDPC) code, wherein the size of each data block of the data to be stored is equal to the size of the LDPC code. For example, refer to... Figure 2 If data needs to be stored in a 16KB page and the LDPC size is 1KB, then the data is also divided into multiple data blocks of 1KB each. This invention aligns the length of the upper-layer erasure coding "block" with the length of the lower-layer LDPC "processing unit". In this way, if a data block is corrupted, both LDPC and erasure coding mechanisms can repair the data, thus improving data reliability.

[0041] According to one embodiment of the present invention, the data block and the parity block are stored in the same physical page, and the total size of the data block and the parity block is equal to the fixed storage capacity of the physical page. Still referring to... Figure 2 If data needs to be stored in a 16KB page, then Kn data blocks and Mn parity blocks are all stored in this page, and Kn + Mn = 16. In the event of a data error, as long as at least Kn data blocks or parity blocks survive, the original data can be 100% recovered. Therefore, when writing system data of only a few KB as mentioned in the background art using the SSD provided by this invention, there is no need to fill the page with invalid data; instead, useful parity blocks are filled, avoiding waste of storage space.

[0042] According to one embodiment of the present invention, the data to be stored in the SRAM includes multiple data of different types. Specifically, a parity block and data blocks of multiple data types are stored within the same physical block, while only a parity block or a data block of one type of data is stored within the same physical page. (Refer to...) Figure 3Different types of data, including data blocks and parity blocks, are stored in a single BLOCK space and written to locations P0, P1, and P2 respectively. For critical data such as firmware programs, these blocks can be repeatedly written to more PAGEs until the entire BLOCK is full. Here, P represents PAGE, K represents data block, D represents data type, B represents the number of writes, and M represents parity block. The SSD provided by this invention allows different types of data to share parity blocks, thus enabling different types of data to be written to the same physical block without complete duplication, even when writing important firmware program data, significantly improving storage efficiency. Furthermore, when any data block contains an uncorrected null terminator (UNC), error correction algorithms can be used to recover the correct data, improving data reliability.

[0043] According to one embodiment of the present invention, even critical data such as firmware runtime system data and firmware program data can be stored in the same physical block of the SSD provided by the present invention, thereby saving storage space and eliminating concerns about reliability.

[0044] According to one embodiment of the present invention, the ratio between the number of data blocks and the number of parity blocks is determined based on the desired storage efficiency and fault tolerance. This allows for a balance between storage efficiency and fault tolerance based on the importance of the data; if high storage efficiency is desired, the number of parity blocks can be reduced, and vice versa, if high fault tolerance is desired, the number of parity blocks can be increased.

[0045] According to one embodiment of the present invention, the ratio of the number of data blocks to the number of parity blocks is 1:1, 2:1, or 3:2. Specifically, when the ratio is 2:1, the storage efficiency is 50% and the fault tolerance is 50%; when the ratio is 2:1, the storage efficiency is 66.6% and the fault tolerance is 33.3%; and when the ratio is 3:2, the storage efficiency is 60% and the fault tolerance is 40%. These three ratios are classic configurations, each with its own emphasis.

[0046] According to one embodiment of the present invention, the main control chip further includes a NAND flash memory controller for reporting the number of erroneous data blocks to the CPU. The CPU is configured to trigger the data repair module to repair the data to be repaired when the number of received erroneous data blocks is less than or equal to the number of check blocks in the data repair parameters. When a data error occurs, this embodiment prioritizes initiating a better erasure coding repair path instead of the traditional read retry, allowing the SSD to move from "passive error correction" to "active repair".

[0047] To better understand the SSD error data repair process provided by this invention, an example will be used to illustrate it below. (Refer to...) Figure 4When a UNC occurs in data block K9 of a page, the flash memory controller (NFC) reports the information to the CPU. After receiving the information, the CPU reads the page's "archive" (metadata) from the NAND, obtains the encoding parameters used when it was written (K=11, M=5), and determines that the current number of erroneous blocks is within the fault tolerance range (there are 5 parity blocks, which is greater than the number of erroneous data blocks by 1). Therefore, the CPU decides to abandon the traditional time-consuming "voltage retry" scheme and instead use the more efficient "erasure coding repair" scheme. Then, the CPU "commands" the NFC to read all the data of the entire page (including good and bad blocks) into the SRAM buffer and generate an error bitmap (e.g., 0x200 indicates that the 9th data block is faulty). All the parameters required for repair are written to the erasure coding hardware accelerator. After obtaining all the parameters of the CPU configuration and the start command, the erasure coding hardware accelerator reads the data blocks and check blocks marked as "intact" from the SRAM. Using these (K+M-1) intact data blocks and check blocks, it accurately calculates the original correct data of the damaged block (K9), and then writes the repaired data back to the position corresponding to K9 in the SRAM buffer.

[0048] Finally, the inventors verified the erasure coding generation performance of the SSD provided by this invention through comparative testing. It is also based on generating 8 1KB parity blocks from 8 1KB data blocks. If the CPU of the SSD main controller chip is used for calculation (simulation), the completion time is in the second range; if the calculation is performed by ordinary PC software algorithm, the completion time is about 500us; while if the calculation is performed by the erasure coding hardware accelerator of this invention, the completion time is only 15us.

[0049] In summary, the SSD provided by this invention integrates an erasure coding hardware accelerator into the main control chip, thereby applying erasure coding technology to the SSD and improving the storage efficiency and reliability of the SSD.

[0050] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0051] The present invention may be a storage array, which includes a controller and storage units, wherein the controller is used to manage the storage units, and the storage units are composed of multiple SSDs as described in the present invention.

[0052] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. An SSD based on erasure coding mechanism, comprising a NAND flash memory array and a controller chip, characterized in that, Data is stored in the NAND flash memory array by the SSD in the form of multiple data blocks and parity blocks of equal size. The parity blocks are used to repair erroneous data when errors occur in the data stored in the NAND flash memory array. The main control chip includes SRAM, CPU, and erasure coding hardware accelerator, wherein: SRAM is used to cache data waiting to be written to the NAND flash array, data to be repaired read from the NAND flash array, and check blocks and repair data generated by the erasure coding hardware accelerator; The CPU is used to calculate the check block generation parameters based on the data to be stored, and to calculate the data repair parameters based on the data to be repaired. The check block generation parameters include the address of the data to be stored in SRAM, the number of data blocks to be stored, the size of each data block to be stored, the specified number of check blocks to be generated, and the specified SRAM address for storing the check blocks. The data repair parameters include the address of the data to be repaired in SRAM, the number of data blocks to be repaired, the size of each data block to be repaired, the number of check blocks to be repaired, the location information of the erroneous data blocks, and the specified SRAM address for storing the repaired data. Erasure coding hardware accelerators are used to generate check blocks and repair data, including: The verification data generation module is used to: obtain the verification block generation parameters calculated by the CPU; obtain the data to be stored according to the address of the data to be stored in the SRAM; and generate a specified number of verification blocks using the erasure coding algorithm based on the obtained verification block generation parameters and the data to be stored, and then put the generated verification blocks into the specified SRAM address. The data repair module is used to: obtain data repair parameters calculated by the CPU; obtain the data to be repaired according to the address of the data to be repaired in SRAM; and, based on the obtained data repair parameters and the data to be repaired, complete the data repair using erasure coding algorithm and place the repaired data at the specified SRAM address.

2. The SSD based on erasure coding mechanism according to claim 1, characterized in that, The erasure coding hardware accelerator is a fixed-function circuit.

3. The SSD based on erasure coding mechanism according to claim 1, characterized in that, The NAND flash array is equipped with an error correction mechanism based on a fixed-size low-density parity check code, where the size of each data block to be stored is equal to the size of the low-density parity check code.

4. The SSD based on erasure coding mechanism according to claim 1, characterized in that, A NAND flash array consists of multiple physical blocks, each physical block consists of multiple physical pages, each physical page has a fixed storage capacity, data blocks and parity blocks are stored in the same physical page, and the total size of the data blocks and parity blocks is equal to the fixed storage capacity of the physical page.

5. The SSD based on erasure coding mechanism according to claim 1, characterized in that, The data to be stored in SRAM includes multiple different types of data. The NAND flash array includes multiple physical blocks, and each physical block includes multiple physical pages. Within the same physical block, there are parity blocks and data blocks of multiple different types of data. Within the same physical page, there are only parity blocks or data blocks of one type of data.

6. The SSD based on erasure coding mechanism according to claim 5, characterized in that, Multiple different types of data refer to key data including firmware runtime system data and firmware program data.

7. The SSD based on erasure coding mechanism according to claim 5, characterized in that, The ratio between the number of data blocks and the number of parity blocks is determined based on the desired storage efficiency and fault tolerance.

8. The SSD based on erasure coding mechanism according to claim 7, characterized in that, The ratio between the number of data blocks and the number of check blocks is one to one, two to one, or three to two.

9. The SSD based on erasure coding mechanism according to claim 1, characterized in that, The main control chip also includes a NAND flash memory controller for reporting the number of erroneous data blocks to the CPU. The CPU is configured to trigger the data repair module to repair the data to be repaired when the number of received erroneous data blocks is less than or equal to the number of check blocks in the data repair parameters.

10. A storage array comprising a controller and storage units, the controller being used to manage the storage units, characterized in that, The storage unit consists of multiple SSDs based on erasure coding as described in any one of claims 1-9.

Images