A RAID card adaptive fault tolerance optimization method and system based on erasure code

Abstract

The present invention relates to the field of computer storage technology, and is a method for optimizing adaptive fault tolerance of RAID cards based on erasure codes. The method includes constructing a coding matrix, performing coding operations on original data to generate check data, and optimizing the encoding and decoding process of the erasure code; performing striping based on super pages, and implementing a left-symmetric strategy in each super block; dynamically adjusting the fault tolerance of the erasure code used based on the number of high-wear blocks; triggering the erasure code encoder to generate check data under preset conditions; creating a recovery node and a flash memory read operation, obtaining all read sub-requests and executing corresponding flash memory read operations, starting the erasure code decoder to recover data from the error page to obtain recovered data, and re-layouting the recovered data. The present invention can dynamically adjust the error correction level of the erasure code based on the number of high-wear blocks in each super block, effectively handle errors occurring in the storage medium, and improve the fault tolerance and performance efficiency of the solid-state drive (SSD).

Description

Technical Field

[0001] The present invention relates to the technical field of computer storage, and in particular to a RAID card adaptive fault-tolerant optimization method and system based on erasure codes. Background Art

[0002] In data center and cloud computing environments, the reliability and efficiency of data storage systems are key research areas. Traditional Redundant Array of Independent Disks (RAID) technology enhances data reliability and access speed by implementing mirroring, striping, and combinations thereof. However, with the increasing demand for large-scale data processing, these traditional methods have encountered significant limitations in capacity expansion and performance improvement.

[0003] Erasure coding, as an advanced fault-tolerant coding mechanism, provides data protection capabilities that surpass those of traditional RAID. Through a complex encoding and decoding process, erasure coding can restore the original data even after multiple data corruptions, thus providing a higher level of data protection. However, the application of erasure coding technology is constrained by its high computational complexity and corresponding performance overhead. Traditional RAID schemes face particularly prominent challenges in solid-state drive (SSD) applications. While traditional RAID can correct bit errors within a page, it cannot effectively handle more complex issues such as page errors, block errors, or chip failures. For example, RAID 5 and RAID 6 schemes can only tolerate one and two faults, respectively. This fixed fault tolerance can lead to unnecessary redundancy early in the lifespan of an SSD and may be unable to handle multiple errors within a stripe later in its lifespan. Furthermore, existing SSD erasure coding schemes typically fail to account for the differences in failure rates between early and late life stages of an SSD, limiting their effectiveness in dynamic data storage environments. Therefore, a new RAID storage fault-tolerance scheme is needed that can adaptively adjust its fault tolerance to match the specific requirements of SSDs and other storage media at different stages of their lifecycle, optimizing performance and efficiency in data centers and cloud computing environments. Summary of the Invention

[0004] In order to solve the efficiency and data protection problems in existing storage technologies, the present invention proposes a RAID card adaptive fault-tolerant optimization method and system based on erasure codes. By monitoring the wear status of flash memory blocks in storage devices in real time, the erasure codes of the error correction level are dynamically adjusted according to the number of high-wear blocks in each super block. The error tolerance capability can be adaptively adjusted, and errors that occur in storage media at different stages of their life can be effectively handled, while reducing computational complexity and improving performance efficiency.

[0005] The purpose of the present invention can be achieved by taking the following technical solutions:

[0006] A Raid card adaptive fault tolerance optimization method based on erasure code includes the following steps:

[0007] S1. Construct a coding matrix based on the Vandermonde matrix, perform coding operations on the original data to generate check data, and use the optimized instruction-based erasure code encoding and decoding process provided by the HLS tool;

[0008] S2, striping is performed based on super pages, and a left-symmetric strategy is implemented in each super block;

[0009] S3. Regularly detect the number of high-wear blocks in the super block and dynamically adjust the fault tolerance of the erasure code used based on the number of high-wear blocks.

[0010] S4. Adopt a write cache management strategy based on double-linked list partitioning, trigger the erasure code encoder to generate verification data under preset conditions, and synchronously organize user data and verification data into a stripe;

[0011] S5. Create a recovery node and read flash memory operation, obtain all read sub-requests and execute corresponding read flash memory operations, start the erasure code decoder to recover the data of the error page to obtain recovered data, and re-layout the recovered data.

[0012] Specifically, step S1 includes:

[0013] Construct a coding matrix A based on the Vandermonde matrix form and select a generator polynomial in the Galois field;

[0014] The data to be encoded is divided into multiple blocks, the size of each block is adapted to the requirements of erasure coding, and each data block is encoded on the Galois field based on the encoding matrix A;

[0015] The optimization instructions provided by the HLS tool are used to optimize the encoding and decoding process of erasure codes.

[0016] Specifically, the optimization instructions provided by the HLS tool are used to optimize the encoding and decoding process of the erasure code, including: using the optimization instructions provided by the HLS tool to optimize the storage method of the global array, the expansion and pipelining of the loop, the parallelism of the input / output interface, and the function call;

[0017] The optimization instruction provided by the HLS tool is used to optimize the storage mode of the global array, including: specifying the global array as a dual-port ROM;

[0018] The optimization of the parallelism of the input / output interface using the optimization instructions provided by the HLS tool includes: dividing the array into multiple independent parts through instructions to achieve parallel access, and the division method includes block, loop or complete method.

[0019] Specifically, step S2 includes:

[0020] Define and organize stripes, organize one or more pages with fixed physical addresses into a physical stripe, and arrange the pages in order of physical addresses to form a basic unit for storing data and check information;

[0021] For stripe data update processing, when a data page in a stripe needs to be updated, the updated data page and the subsequent write data are organized into a new physical stripe;

[0022] The garbage collection process is performed based on the super block granularity. When the data in a physical stripe needs to be erased or reorganized, the entire super block will participate in the garbage collection process;

[0023] A left-symmetric parity layout is used across all stripes within each superblock.

[0024] Specifically, step S3 includes:

[0025] Construct a wear block table to record the number of high-wear blocks in each super block. Define a wear threshold W to distinguish between normal blocks and high-wear blocks. When the wear level of a storage block is lower than the threshold W, the block is considered a normal block; otherwise, it is considered a high-wear block.

[0026] Select a suitable erasure coding scheme and use a reasonable distribution strategy to evenly distribute data blocks and parity blocks to different storage nodes or devices;

[0027] According to the number of high-wear blocks recorded in the wear block table, the fault tolerance of the erasure code used in each super block is dynamically adjusted.

[0028] Specifically, step S4 includes:

[0029] S41, dividing the write request issued by the host into multiple write sub-requests;

[0030] S42: Receive a write sub-request and check whether the corresponding write sub-request is matched in the data cache.

[0031] S43. When the write sub-request hits the data cache, further check whether each sector within the data cache request also hits the cache node. For sectors that hit, update the cache node directly and adjust the cache node to the head of the least recently used (LRU) queue of the data cache area. For sectors that miss, determine whether these sectors can be fully accommodated by the data cache area. If these sectors can be fully accommodated by the data cache area, add these sectors to the cache node and move the node to the head of the LRU queue. If the data cache area cannot fully accommodate these sectors, execute step S45.

[0032] S44. When the write sub-request misses in the data cache, determine whether the write sub-request can be completely cached by the data cache. If it can be completely cached, create a new cache node to store the write sub-request and place the newly created cache node at the top of the least recently used (LRU) queue in the data cache. If the data cache cannot completely cache the write sub-request, execute step S45.

[0033] S45. Obtain the erasure code applied to the super block to be written by querying the wear block table, replace the k nodes at the tail of the LRU queue of the data cache area with the data check cache area, start the encoder of the erasure code, cache the generated check page to the data check cache area, create a write flash memory sub-request corresponding to the data page and the check page, and hang it in the corresponding channel write sub-request queue.

[0034] Specifically, step S5 includes:

[0035] S51. Create a recovery node and set the flag bit of the flash memory page in the stripe required for recovery; create the flash memory read operation required for recovery; create all read sub-requests required for recovery, set the read sub-request type, add the read sub-request to the corresponding channel read sub-request queue, and add the recovery node to the end of the recovery queue;

[0036] S52, obtaining a read sub-request from the channel read sub-request queue and executing a corresponding flash memory read operation;

[0037] S53 sets the flag bit of the flash memory read operation. If the flash memory read operation is a flash memory read operation required for passive recovery, the recovery node is polled to find the corresponding recovery operation, a read operation completion flag is set, and the flag bit corresponding to the read operation in the read operation completion flag is set to 1; otherwise, the process goes to step S52 to execute other read sub-requests in the channel.

[0038] S54, determining the completion status of the flash memory read operation. If all flash memory read operations required for the recovery operation are completed, proceed to step S55; otherwise, go to step S52 to process other read sub-requests.

[0039] S55: Start the erasure code decoder to recover the data of the error page and rearrange the recovered data.

[0040] A RAID card adaptive fault-tolerant optimization system based on erasure codes is implemented based on the above-mentioned RAID card adaptive fault-tolerant optimization method based on erasure codes, comprising:

[0041] The erasure code encoding / decoding acceleration module is used to construct a coding matrix based on the Vandermonde matrix, perform encoding operations on the original data to generate check data, and implement the erasure code encoding and decoding process through the optimized instructions provided by the HLS tool;

[0042] The stripe organization module is used to perform striping based on super pages and implement a left-symmetric strategy in each super block.

[0043] An adaptive fault-tolerance module is used to periodically detect the number of high-wear blocks in the super block and dynamically adjust the fault-tolerance capability of the RS erasure code used based on the number of high-wear blocks.

[0044] The write cache module managed by a double-linked list partition is used to adopt a write cache management strategy based on the double-linked list partition. Under preset conditions, it triggers the erasure code encoder to generate verification data and synchronously organizes user data and verification data into a stripe.

[0045] The data recovery and reconstruction module is used to create a recovery node and read the flash memory operation, obtain all read sub-requests and execute the corresponding read flash memory operation, start the RS code decoder to recover the data of the error page to obtain the recovered data, and re-layout the recovered data.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] The present invention provides a RAID card adaptive fault-tolerant optimization method and system based on erasure code. By performing encoding operations on original data to generate verification data, the encoding and decoding process of the optimized instruction-based erasure code provided by the HLS tool is significantly improved compared with the existing software implementation, and overcomes the limitations of the traditional RAID card processing capabilities.

[0048] By striping data based on superpages and implementing a left-symmetric strategy within each superblock, this optimized physical address organization reduces the frequent parity updates required by data page updates within the stripe, thereby improving overall storage efficiency. The number of high-wear blocks within a superblock is regularly monitored, and the fault tolerance of the RS erasure code used is dynamically adjusted based on the number of high-wear blocks. This strategy enhances the fault tolerance of the erasure code, improving its resilience to potential failures, particularly in the later stages of a storage medium's lifespan. This allows for effective handling of errors that occur at different stages of the storage medium's lifespan, while reducing computational complexity and improving performance efficiency.

[0049] A write cache management strategy based on doubly linked list partitioning is employed. This strategy triggers the erasure code encoder to generate parity data at appropriate times, synchronously organizing user data and parity data into a single stripe, ensuring data integrity and consistency. A passive data recovery mechanism provides multiple reconstruction layout options for recovered data, minimizing data recovery time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0051] Figure 1 This is a flow chart of a RAID card adaptive fault tolerance optimization method based on erasure code in an embodiment of the present invention;

[0052] Figure 2 This is a flowchart of a write cache based on double-linked list partition management in an embodiment of the present invention.

[0053] Figure 3 It is a structural diagram of a RAID card adaptive fault-tolerant optimization system and a flash memory controller based on erasure code in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is obvious that the embodiments described are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited to these. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] Example 1:

[0056] like Figure 1 As shown, a general architecture diagram of a Raid card adaptive fault-tolerant optimization method based on erasure code, a Raid card adaptive fault-tolerant optimization method based on erasure code, comprising the following steps:

[0057] S1. Use hardware acceleration technology to optimize the encoding and decoding process of erasure codes, perform encoding operations on the original data to generate check data, and use the remaining data for efficient decoding and recovery when data corruption occurs.

[0058] S11, construct the encoding matrix, construct the encoding matrix A based on the Vandermonde matrix form, in the Galois field (GF 8 ) and select a generating polynomial from .

[0059]

[0060] The encoding matrix A is an m×n order Vandermonde matrix, which is composed of n elements x0, x1, x n-1 The power of 0 to m-1. In the Galois field (GF 8 ) is crucial for constructing the encoding matrix. The encoding matrix A is used to convert the original data block into encoded data, including a checksum block. This checksum block is used to enhance error detection and correction capabilities, allowing for efficient decoding and recovery using the remaining data in the event of data corruption.

[0061] In this embodiment, to represent this encoding matrix in a computer program, a two-dimensional array code_matrix is ​​used to store the encoding matrices of RS (Reed-Solomon) codes under different parameters. Specifically, for a RS(k,m) code, its encoding matrix can be represented by the first k rows and first m columns of the code_matrix array, where k is the number of original data blocks and m is the number of generated check data blocks.

[0062] S12. Divide the data to be encoded into multiple blocks, where the size of each block is adapted to the requirements of RS encoding, and perform encoding on each data block based on the encoding matrix A in the Galois field (GF) to implement a multi-rate RS code encoder.

[0063] Specifically, the encoding process for each data block is performed on the GF field, and the encoding process for each data block includes: encoding k bytes of data each time, and using a method based on logarithm / antilogarithm tables to accelerate the multiplication operation of the finite field.

[0064] S13. Optimize Vitis high-level synthesis (HLS) instructions. Use the optimization instructions provided by Xilinx's HLS tool to optimize the encoding and decoding process of erasure codes. Optimize the storage method of global arrays, loop expansion and pipelining, parallelism of input / output interfaces, and function calls.

[0065] Specifically, optimizing global array storage in HLS involves specifying the global array as a dual-port ROM. This optimization can significantly impact access speed and hardware resource utilization. By specifying the global array as a dual-port ROM, table lookups can be accelerated, which is particularly useful in applications that frequently read fixed data.

[0066] Optimize the parallelism of the input / output interface and implement parallel access by partitioning the array into multiple independent parts through the array_PARTITION instruction. The partitioning method can be block, cyclic, or complete to meet the needs of different data flow-intensive designs.

[0067] S2: Striping is performed based on super pages, and a left-symmetric strategy is implemented within each super block. This strategy optimizes the organization of physical addresses, reducing the frequent parity data updates caused by data page updates within the stripe, thereby improving overall storage efficiency.

[0068] S21. Calculate the index of the decoding matrix in the ROM. When the erasure code used in the stripe is RS(k,1), there is no need to decode the decoding matrix. The lost data block can be obtained by directly performing an XOR operation on the surviving blocks. When the RS(k,m) code that can tolerate m errors is applied in a stripe of width n (i.e., m>=2), the formula for calculating the ROM index of the RS(k,m) decoding matrix can be derived:

[0069]

[0070] where i and j are the positions of the two errors in the stripe, n is the width of the stripe, and f(i,j,m) is the index of the decoding matrix corresponding to RS(k,m) in the ROM.

[0071] S22. Define and organize stripes, organize one or more pages with fixed physical addresses into a physical stripe, and arrange the pages in order of physical addresses to form a basic unit for storing data and check information.

[0072] Specifically, a page with a fixed physical address is a super page. Each physical stripe contains one or more super pages. The number of super pages in each physical stripe is configured according to actual storage requirements and the architecture of the SSD.

[0073] S23: When a data page in a stripe needs to be updated, the original check data is not directly updated. Instead, the updated data page is organized into a new physical stripe together with the subsequent write data.

[0074] S24, invalid data recovery, the garbage collection process is performed based on the super block as the granularity. When the data in a physical stripe needs to be erased or reorganized, the entire super block will participate in the garbage collection process.

[0075] S25. Checksum layout: A left-symmetric checksum layout is used in all stripes within each superblock. Checksum pages are evenly distributed across all planes, and the layout of checksum pages in the superblock is organized across chips and planes.

[0076] Specifically, after determining the redundant encoding used in the super block, in order to maximize the parallelism of the flash memory, a left-symmetric check layout similar to RAID5 is adopted in all stripes within each super block, that is, the check pages are evenly distributed in all planes, and the layout of the check pages in the super block is organized across chips and planes.

[0077] S3. Regularly check the number of high-wear blocks within the superblock and dynamically adjust the fault tolerance of the RS (Reed-Solomon) erasure code based on the number of high-wear blocks. Specifically, dynamically adjust the ratio of the number of data blocks, k, to the number of parity blocks, m. By enhancing the fault tolerance of the erasure code, the code's adaptability to potential failures can be improved.

[0078] S31. Construct a wear block table to record the number of high-wear blocks within each super block. A wear threshold W is defined to distinguish between blocks in normal use and blocks with high wear. When a block's wear level falls below the threshold W, it is considered a normal block; otherwise, it is considered a high-wear block. Recording the number of high-wear blocks within each super block in the wear block table provides a basis for subsequent wear management and erasure code strategy adjustments, enabling precise management of wear within the RAID.

[0079] S32. Select an erasure code, choose a suitable erasure code encoding scheme, and use a reasonable distribution strategy to evenly distribute data blocks and parity blocks to different storage nodes or devices to achieve consistency in data volume on each node, thereby optimizing data recovery efficiency and storage space utilization.

[0080] S33. Wear detection. The frequency of wear detection should be dynamically adjusted based on the difference in bit error rates between the initial and final stages of SSD flash memory block use. In the early stages of a RAID system's lifespan, due to the lower bit error rate, a lower frequency of wear detection can be employed (for example, once every 1000 write / erase (P / E) cycles). As the system enters its later stages of lifespan and the bit error rate increases, the frequency of wear detection should be increased (for example, once every 10 P / E cycles) to more accurately monitor the wear status of the storage media.

[0081] S34. Dynamically adjust the erasure code. According to the number of high-wear blocks recorded in the wear block table, dynamically adjust the fault tolerance of the RS(k,m) erasure code used in each super block.

[0082] Specifically, the ratio of the erasure code parameters k (number of data blocks) and m (number of parity blocks) is adjusted to accommodate changes in the number of high-wear blocks within a superblock. This adjustment is typically performed after garbage collection to ensure data integrity and reliability, preventing data loss caused by simultaneous errors in multiple high-wear blocks.

[0083] S4. Adopt a write cache management strategy based on double-linked list partitioning, trigger the erasure code encoder to generate verification data under preset conditions, and synchronously organize user data and verification data into a stripe to ensure data integrity and consistency.

[0084] like Figure 2 As shown in the figure, the write cache flow chart based on double-linked list partition management is shown. The write cache management strategy based on double-linked list partition triggers the erasure code encoder to generate verification data under preset conditions, and synchronously organizes user data and verification data into a stripe. Specifically, it includes:

[0085] S41. Divide the write request sent by the host into multiple write sub-requests, each sub-request corresponds to a logical page number (lpn), and each sub-request covers several sectors, each sector corresponds to a logical sector number (lsn).

[0086] S42: Receive a write sub-request and check whether the corresponding write sub-request matches the data cache. Specifically, a write sub-request is first received and searched for in the cache nodes of the data cache. If the search matches, step S43 is performed to check if the write sub-request matches. If not, step S44 is performed to check if the write sub-request does not match.

[0087] S43, write sub-request hit processing, when the write sub-request hits in the data cache area, further check whether each sector in the data cache area request is also hit in the cache node; for the hit sectors, directly update in the cache node, and adjust the cache node to the head of the least recently used (LRU) queue of the data cache area; for the missed sectors, determine whether these sectors can be fully accepted by the data cache area. If these sectors can be fully accepted by the data cache area, these sectors are added to the cache node, and the node is also moved to the head of the LRU queue. If the data cache area cannot fully accommodate these sectors, it indicates that the cache area is full, and step S45 is executed at this time to perform a cache replacement operation.

[0088] S44: Processing a missed write sub-request. When a write sub-request misses in the data cache, a determination is made as to whether the write sub-request can be fully cached by the data cache. If it can, a new cache node is created to store the write sub-request and placed at the top of the least recently used (LRU) queue in the data cache. If the data cache cannot fully cache the write sub-request, this indicates that the cache has reached its capacity limit, and step S45 is executed to perform a cache replacement operation.

[0089] S45, cache replacement operation, obtain the EC mode (RS code applied to the super block) corresponding to the super block to be written by querying the wear block table, and replace the k nodes at the tail of the LRU queue of the data cache area with the data verification cache area (k is the number of data blocks in RS). The replacement process is a simple pointer operation. Generate verification data, start the encoder of the RS erasure code, cache the generated verification page to the data verification cache area, that is, organize it into the same stripe as the data page. Create a write flash request, create a write flash sub-request corresponding to the data page and the verification page, and hang it in the corresponding channel write sub-request queue. If all sub-requests of the write request are processed, the cache management of the request is terminated, otherwise go to step (a) to continue processing.

[0090] S5. Create a recovery node and read flash memory operation, obtain all read sub-requests and execute corresponding read flash memory operations, start the RS code decoder to recover the data of the error page to obtain recovered data, and re-layout the recovered data.

[0091] S51. Create a recovery node and read flash memory operations. Create a recovery node rec_node and set the flag bit rec_node->recovery_flag for the flash memory pages in the stripe required for recovery. Create the read flash memory operations required for recovery. Create all read subrequests required for recovery, set the read subrequest type to RECOVERY_SUB, add the read subrequests to the corresponding channel read subrequest queue, and add the recovery node rec_node to the end of the recovery queue.

[0092] S52, obtaining a read sub-request from the channel read sub-request queue and executing a corresponding flash memory read operation;

[0093] S53 sets the flag of the flash memory read operation. If the flash memory read operation is the flash memory read operation required for passive recovery, poll the recovery node, find the corresponding recovery operation rec, set the read operation completion flag rec->complete_flag, and set the flag position corresponding to the read operation in the read operation completion flag rec->complete_flag to 1; otherwise, go to step S52 to execute other read sub-requests in the channel.

[0094] S54, determining the completion status of the flash memory read operation. If all flash memory read operations required for the recovery operation are completed, proceed to step S55; otherwise, go to step S52 to process other read sub-requests.

[0095] Specifically, the completion status of the flash memory read operation is determined by determining whether the complete_flag field of the recovery read operation completion flag and the recovery_flag field of the mark bit of the flash memory page in the stripe are equal. If the two are equal, all the read operations required for the recovery operation are completed; otherwise, go to step S52 to process other read sub-requests.

[0096] S55: Start the RS code decoder to recover the data of the error page and rearrange the recovered data.

[0097] Specifically, the RS code decoder is started to recover the data of the error page, the recovered data is obtained, the recovered flash page data is cached in the data cache area, and the recovered data and subsequent write data are organized into a new stripe and written into the flash array, and the recovery node is deleted from the recovery operation queue to complete the recovery operation.

[0098] By designing a passive data recovery mechanism, a variety of reconstruction layout options are provided for the recovered data. This flexible recovery strategy aims to minimize data recovery time.

[0099] In summary, the present embodiment provides an adaptive fault-tolerance optimization method for RAID cards based on erasure codes. This method generates check data by performing encoding operations on the original data. The encoding and decoding process of the optimized instruction-based erasure code provided by the HLS tool significantly improves the speed and efficiency of encoding and decoding compared to existing software implementations, overcoming the limitations of traditional RAID card processing capabilities. By striping based on super pages and implementing a left-symmetric strategy in each super block, the organization of physical addresses can reduce the frequent updates to check data due to updates to data pages within the stripe, thereby improving overall storage efficiency. By regularly detecting the number of high-wear blocks within the super block, the fault tolerance of the RS erasure code used is dynamically adjusted based on the number of high-wear blocks. In particular, in the later stages of storage media use, this strategy enhances the fault tolerance of the erasure code and improves adaptability to potential faults. It can effectively handle errors that occur at different stages of the storage medium's lifespan, while reducing computational complexity and improving performance efficiency. A write cache management strategy based on doubly linked list partitioning is employed. This strategy triggers the erasure code encoder to generate parity data at appropriate times, synchronously organizing user data and parity data into a single stripe, ensuring data integrity and consistency. A passive data recovery mechanism provides multiple reconstruction layout options for recovered data, minimizing data recovery time.

[0100] Example 2:

[0101] Based on the above-mentioned adaptive fault-tolerant optimization method of Raid card based on erasure code, this embodiment provides an adaptive fault-tolerant optimization system of Raid card based on erasure code, such as Figure 3 The figure shows a schematic diagram of the structure of a RAID card adaptive fault-tolerant optimization system and a flash memory controller based on erasure code. The RAID card adaptive fault-tolerant optimization system based on erasure code includes an erasure code encoding / decoding acceleration module, a stripe organization module, an adaptive fault-tolerant module, a write cache module managed by a double-linked list partition, a write cache module managed by a double-linked list partition, and a data recovery and reconstruction module, wherein:

[0102] A RAID card adaptive fault-tolerant optimization system based on erasure codes is implemented based on the above-mentioned RAID card adaptive fault-tolerant optimization method based on erasure codes, comprising:

[0103] The erasure code encoding / decoding acceleration module is used to construct a coding matrix based on the Vandermonde matrix, perform encoding operations on the original data to generate check data, and implement the erasure code encoding and decoding process through the optimized instructions provided by the HLS tool;

[0104] The stripe organization module is used to perform striping based on super pages and implement a left-symmetric strategy in each super block.

[0105] An adaptive fault-tolerance module is used to periodically detect the number of high-wear blocks in the super block and dynamically adjust the fault-tolerance capability of the RS erasure code used based on the number of high-wear blocks.

[0106] The write cache module managed by a double-linked list partition is used to adopt a write cache management strategy based on the double-linked list partition. Under preset conditions, it triggers the erasure code encoder to generate verification data and synchronously organizes user data and verification data into a stripe.

[0107] The data recovery and reconstruction module is used to create a recovery node and read the flash memory operation, obtain all read sub-requests and execute the corresponding read flash memory operation, start the RS code decoder to recover the data of the error page to obtain the recovered data, and re-layout the recovered data.

[0108] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A Raid card adaptive fault tolerance optimization method based on erasure code, characterized in that: The following steps are involved: S1. Construct a coding matrix based on the Vandermonde matrix, perform coding operations on the original data to generate check data, and optimize the encoding and decoding process of the erasure code using the optimization instructions provided by the HLS tool; S2, striping is performed based on super pages, and a left-symmetric strategy is implemented in each super block; S3. Regularly detect the number of high-wear blocks in the super block and dynamically adjust the fault tolerance of the erasure code used based on the number of high-wear blocks. S4. Adopt a write cache management strategy based on double-linked list partitioning, trigger the erasure code encoder to generate verification data under preset conditions, and synchronously organize user data and verification data into a stripe; The step S4 comprises: S41, dividing the write request issued by the host into multiple write sub-requests; S42: Receive a write sub-request and check whether the corresponding write sub-request is matched in the data cache. S43. When the write sub-request hits the data cache, further check whether each sector within the data cache request also hits the cache node. For sectors that hit, update the cache node directly and adjust the cache node to the head of the least recently used (LRU) queue of the data cache area. For sectors that miss, determine whether these sectors can be fully accommodated by the data cache area. If these sectors can be fully accommodated by the data cache area, add these sectors to the cache node and move the node to the head of the LRU queue. If the data cache area cannot fully accommodate these sectors, execute step S45. S44. When the write sub-request misses in the data cache, determine whether the write sub-request can be completely cached by the data cache. If it can be completely cached, create a new cache node to store the write sub-request and place the newly created cache node at the top of the least recently used (LRU) queue in the data cache. If the data cache cannot completely cache the write sub-request, execute step S45. S45. Obtain the erasure code applied to the superblock to be written by querying the worn block table, replace the k nodes at the tail of the LRU queue in the data cache area with the data check cache area, cache the generated check page in the data check cache area, create write flash memory sub-requests corresponding to the data page and check page, and attach them to the corresponding channel write sub-request queue; S5. Create a recovery node and read flash memory operation, obtain all read sub-requests and execute corresponding read flash memory operations, recover the data of the error page through the erasure code decoder to obtain recovered data, and re-layout the recovered data.

2. The method for adaptive fault-tolerance optimization of RAID cards based on erasure codes according to claim 1, characterized in that: The step S1 comprises: Construct a coding matrix based on the Vandermonde matrix form and select a generator polynomial in the Galois field; The data to be encoded is divided into multiple blocks, the size of each block is adapted to the requirements of erasure coding, and each data block is encoded on the Galois field based on the coding matrix; The optimization instructions provided by the HLS tool are used to optimize the encoding and decoding process of erasure codes.

3. The adaptive fault-tolerant optimization method for RAID cards based on erasure codes according to claim 2, characterized in that: The optimization instructions provided by the HLS tool are used to optimize the encoding and decoding process of the erasure code, including: using the optimization instructions provided by the HLS tool to optimize the storage method of the global array, the expansion and pipelining of the loop, the parallelism of the input / output interface, and the function call; The storage mode of the global array is optimized by utilizing the optimization instructions provided by the HLS tool, including: specifying the global array as a dual-port ROM; The optimization of the parallelism of the input / output interface using the optimization instructions provided by the HLS tool includes: dividing the array into multiple independent parts through instructions to achieve parallel access, and the division method includes block, loop or complete method.

4. The method for adaptive fault-tolerance optimization of RAID cards based on erasure codes according to claim 3, characterized in that: The step S2 comprises: Define and organize stripes, organize one or more pages with fixed physical addresses into a physical stripe, and arrange the pages in order of physical addresses to form a basic unit for storing data and check information; For stripe data update processing, when a data page in a stripe needs to be updated, the updated data page and the subsequent write data are organized into a new physical stripe; The garbage collection process is performed based on the super block granularity. When the data in a physical stripe needs to be erased or reorganized, the entire super block will participate in the garbage collection process; A left-symmetric parity layout is used across all stripes within each superblock.

5. The method for adaptive fault-tolerance optimization of RAID cards based on erasure codes according to claim 1, characterized in that: The step S3 comprises: Construct a wear block table to record the number of high-wear blocks in each super block. Define a wear threshold W to distinguish between normal blocks and high-wear blocks. When the wear level of a storage block is lower than the threshold W, the storage block is considered a normal block; otherwise, the storage block is considered a high-wear block. Select a suitable erasure coding scheme and use a reasonable distribution strategy to evenly distribute data blocks and parity blocks to different storage nodes or devices; According to the number of high-wear blocks recorded in the wear block table, the fault tolerance of the erasure code used in each super block is dynamically adjusted.

6. The method for adaptive fault-tolerance optimization of RAID cards based on erasure codes according to claim 1, characterized in that: The step S5 comprises: S51. Create a recovery node and set the flag bit of the flash memory page in the stripe required for recovery; create the flash memory read operation required for recovery; create all read sub-requests required for recovery, set the read sub-request type, add the read sub-request to the corresponding channel read sub-request queue, and add the recovery node to the end of the recovery queue; S52, obtaining a read sub-request from the channel read sub-request queue and executing a corresponding flash memory read operation; S53 sets the flag bit of the flash memory read operation. If the flash memory read operation is a flash memory read operation required for passive recovery, the recovery node is polled to find the corresponding recovery operation, a read operation completion flag is set, and the flag bit corresponding to the read operation in the read operation completion flag is set to 1; otherwise, the process goes to step S52 to execute other read sub-requests in the channel. S54, determining the completion status of the flash memory read operation. If all flash memory read operations required for the recovery operation are completed, proceed to step S55; otherwise, go to step S52 to process other read sub-requests. S55: Start the erasure code decoder to recover the data of the error page and rearrange the recovered data.

7. The method for adaptive fault-tolerance optimization of RAID cards based on erasure codes according to claim 6, characterized in that: The flag bit of the flash memory page in the stripe is set to: rec_node->recovery_flag; the read operation completion flag is set to: rec->complete_flag; The step S54 includes: determining whether the complete_flag field of the recovery read operation completion flag and the recovery_flag field of the mark bit of the flash memory page in the stripe are equal. If the two are equal, then all the read operations required for the recovery operation are completed, and step S55 is performed; otherwise, step S52 is turned to process other read sub-requests.

8. The method for adaptive fault-tolerance optimization of RAID cards based on erasure codes according to claim 6, characterized in that: The step S55 includes: starting the erasure code decoder to recover the data of the error page, obtaining the recovered flash memory page data, caching the recovered flash memory page data in the data cache area, organizing the recovered data and subsequent write data into a new stripe and writing it into the flash memory array, and deleting the recovery node from the recovery operation queue.

9. The adaptive fault-tolerant optimization system for RAID cards based on erasure codes according to claim 1 is implemented based on the adaptive fault-tolerant optimization method for RAID cards based on erasure codes according to any one of claims 1 to 8, characterized in that: include: The erasure code encoding / decoding acceleration module is used to construct a coding matrix based on the Vandermonde matrix, perform encoding operations on the original data to generate check data, and optimize the erasure code encoding and decoding process through the optimization instructions provided by the HLS tool; The stripe organization module is used to divide stripes based on super pages and implement a left-symmetric strategy in each super block; Adaptive fault-tolerance module, which is used to regularly detect the number of high-wear blocks in the super block and dynamically adjust the fault-tolerance capability of the erasure code used based on the number of high-wear blocks; The write cache module managed by a double-linked list partition is used to adopt a write cache management strategy based on the double-linked list partition. Under preset conditions, it triggers the erasure code encoder to generate verification data and synchronously organizes user data and verification data into a stripe. The data recovery and reconstruction module is used to create recovery nodes and read flash memory operations, obtain all read sub-requests and execute corresponding read flash memory operations, start the erasure code decoder to recover the data of the error page to obtain the recovered data, and re-layout the recovered data.

Images