Degraded read method and apparatus based on erasure code, device, readable storage medium and program product

Abstract

The application relates to a degradation reading method and device based on an erasure code, equipment, a readable storage medium and a program product, and relates to the technical field of data storage. The method comprises the following steps: in response to a reading request for a target data block in erasure code data, reading the target data block, and in the case that the target data block reading fails, determining the number of check blocks and the number of failed blocks included in the erasure code data; in the case that the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks, constructing an initial decoding combination bitmap of the erasure code data; constructing a failed block bitmap according to the positions of the failed blocks and the target data blocks in the initial decoding combination bitmap; based on the initial decoding combination bitmap and the failed block bitmap, constructing a target decoding combination, and performing erasure code decoding processing on the target decoding combination until the degradation reading succeeds or each target decoding combination is traversed. The method can improve the degradation reading efficiency.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a method, apparatus, device, readable storage medium, and program product based on erasure coding for degraded reading. Background Technology

[0002] To ensure the reliability and availability of storage systems, replication or erasure coding (EC) is typically used to implement redundant data storage. In an erasure coding storage system, each stripe contains multiple data blocks and multiple parity blocks. During the data writing process, EC encoding is used to calculate the parity blocks of the stripe, and these parity blocks are stored together with the data blocks. When reading data, if the original data block fails to be read, other data blocks and parity blocks in the stripe can be read using a degraded read approach, and the original data block can be recovered using EC decoding.

[0003] However, in the downgrade read and repair methods of related technologies, there may be a combination of multiple EC decodings. Usually, the data blocks to be decoded are selected according to a certain priority and downgrade read is performed. The read completion condition of this method is unclear. If the data decoded by continuous downgrade read fails to be verified, the same data and verification blocks may be read repeatedly without success. Therefore, the downgrade read method of related technologies has the problem of low efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product based on erasure coding that can improve the efficiency of downgraded reading, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a downgraded read method based on erasure coding, including:

[0006] In response to a read request for a target data block in erasure coding data, the target data block is read, and if the read of the target data block fails, the number of check blocks and the number of invalid blocks included in the erasure coding data are determined.

[0007] If the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, an initial decoded combination bitmap of the erasure coding data is constructed.

[0008] Based on the positions of the failed block and the target data block in the initial decoded combination bitmap, a failed block bitmap is constructed;

[0009] Based on the initial decoded combination bitmap and the failed block bitmap, a target decoded combination is constructed, and erasure coding decoding is performed on the target decoded combination until the downgrade read is successful or each target decoded combination is traversed.

[0010] In this embodiment, when there is a need to read erasure coding data, and the target data block in the erasure coding data fails to be read, the number of check blocks and the number of invalid blocks included in the erasure coding data are determined to determine the number of existing decoding combinations. If there are multiple decoding combinations, an initial decoding combination bitmap of the erasure coding data is constructed, and an invalid block bitmap is further constructed based on the positions of the invalid blocks and the target data block in the initial decoding combination bitmap. Based on the initial decoding combination bitmap and the invalid block bitmap, a target decoding combination is constructed, and erasure coding decoding processing is performed on the target decoding combination until the downgrade read is successful or all the target decoding combinations are traversed. The invalid block bitmap can filter out invalid decoding combinations and find valid target decoding combinations more quickly, thereby shortening the downgrade read process and improving the efficiency of downgrade read.

[0011] In one embodiment, constructing the target decoding combination based on the initial decoding combination bitmap and the failed block bitmap includes:

[0012] The initial decoded combination bitmap is processed using a preset conversion algorithm to generate a corresponding candidate decoded combination bitmap.

[0013] Based on the position of the target data block in the candidate decoding combination bitmap, the value of the target data block is inserted to obtain the updated candidate decoding combination bitmap;

[0014] The updated candidate decoding combination bitmap is verified based on the failed block bitmap, and the decoding combination corresponding to the updated candidate decoding combination bitmap that passes the verification is determined as the target decoding combination.

[0015] In this embodiment, the initial decoding combination bitmap is processed by a preset conversion algorithm to generate a corresponding candidate decoding combination bitmap. By using the principle of permutation and combination, all kinds of decoding combinations can be obtained. Based on the position of the target data block in the candidate decoding combination bitmap, the target data block is inserted and its value is assigned to obtain an updated candidate decoding bitmap. This marks the target data block to be decoded, improving the accuracy of subsequent decoding. In addition, the updated candidate decoding bitmap is verified based on the failed block bitmap, and the decoding combination corresponding to the updated candidate decoding bitmap that passes the verification is determined as the target decoding combination. This avoids selecting decoding combinations with failed blocks for decoding, reduces decoding resource consumption and shortens the downgrade read process, thereby further improving the efficiency of downgrade read.

[0016] In one embodiment, the step of verifying the updated candidate decoding combination bitmap based on the failed block bitmap includes:

[0017] Perform a bitwise AND operation between the failed block bitmap and the updated candidate decoding combination bitmap to obtain a calculated bitmap;

[0018] If the value assigned to the target data block in the computational bitmap is different from other values, it indicates that the updated candidate decoding combination bitmap has passed the verification.

[0019] In this embodiment, a computed bitmap is formed by performing a bitwise AND operation on the updated bitmap to be decoded and the bitmap of the invalid block. Only when the assignment of the computed bitmap at the position of the target data block is different from the assignment of other blocks is the updated bitmap to be decoded considered to have passed the verification and then enter the erasure coding decoding attempt. Otherwise, the decoding combination is excluded to avoid repeated attempts on invalid combinations, thereby reducing the number of downgrade read attempts and further improving the decoding success rate and overall reading efficiency.

[0020] In one embodiment, the method further includes:

[0021] If the number of downgrade reads for the target data block reaches a preset downgrade read count threshold, and the erasure coding data has not undergone strip consistency verification, erasure coding decoding is performed on the target decoding combination of the current downgrade read to obtain the decoded data.

[0022] Perform strip-by-strip consistency verification on the decoded data and the original data of the target decoded combination to obtain the consistency verification results of each data block to be verified in the target decoded combination;

[0023] If, in the consistency verification result characterizing the target decoding combination, some of the data blocks to be verified fail verification, the data blocks to be verified that fail verification are identified as new invalid blocks.

[0024] The failure block bitmap is updated based on the new failure block.

[0025] In this embodiment, by introducing stripe consistency verification during the degraded read process to identify and update new failed blocks, invalid decoding combinations are gradually eliminated and invalid repeated attempts are avoided. If the decoding fails to recover the target data block in the current round of degraded read, it can also identify whether there is a new failed block, providing information for subsequent iterations. At the same time, by judging the stripe consistency verification result, the data block that failed the verification is marked as a new failed block and the failed block bitmap is updated, ensuring that the failed block is removed when constructing the target decoding combination in the future, shortening the degraded read path, thereby improving the efficiency and success rate of degraded read under the premise of controllable resource overhead.

[0026] In one embodiment, the erasure coding-based degradation read method further includes:

[0027] After updating the fault block bitmap according to the new fault block, the steps of constructing a target decoding combination based on the initial decoding combination bitmap and the fault block bitmap, and performing erasure coding decoding processing on the target decoding combination until the downgrade read is successful or each target decoding combination is traversed.

[0028] In this embodiment, after stripe consistency verification and identification of new failed blocks, the failed block bitmap is updated with the new failed block information. The target decoding combination is then reconstructed based on the updated failed block bitmap and the initial decoded combination bitmap for erasure coding decoding, forming an adaptive, iterative decoding process. This eliminates known invalid blocks in each iteration, reducing the number of invalid decoding attempts and further improving the efficiency of downgraded reads. If the target decoding combination fails to decode, the updated bitmap enters the next round of downgraded read filtering, continuing to traverse potential decoding paths until the target data block is successfully read or all possible combinations are exhausted, thus improving the system's fault tolerance and data availability.

[0029] In one embodiment, when the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks, an initial decoded combination bitmap is constructed for the erasure coding data, including:

[0030] If the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks, the target number of data blocks included in the erasure coding data is determined.

[0031] An initial decoding combination bitmap is constructed based on the target number.

[0032] In this embodiment, when there are multiple available decoding combinations, an initial decoding combination bitmap based on the target data block number is explicitly given. The relationship between the target number and the number of check blocks, as well as the relationship between the selected blocks used for decoding and the unselected blocks, are transformed into a specific initial decoding combination bitmap. This can intuitively reflect the composition of the decoding combination and lay the groundwork for generating different target decoding combinations based on the failed block bitmap.

[0033] Secondly, this application also provides a degradation reading device based on erasure coding, comprising:

[0034] The data degradation read module is used to respond to a read request for a target data block in erasure coding data, read the target data block, and determine the number of check blocks and the number of invalid blocks included in the erasure coding data if the read of the target data block fails.

[0035] A combined bitmap construction module is used to construct an initial decoded combined bitmap of the erasure coding data when the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks;

[0036] A failure bitmap construction module is used to construct a failure block bitmap based on the positions of the failure block and the target data block in the initial decoded combination bitmap;

[0037] The data block decoding module is used to construct a target decoding combination based on the initial decoding combination bitmap and the failed block bitmap, and to perform erasure coding decoding processing on the target decoding combination until the downgrade read is successful or each of the target decoding combinations is traversed.

[0038] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0039] In response to a read request for a target data block in erasure coding data, the target data block is read, and if the read of the target data block fails, the number of check blocks and the number of invalid blocks included in the erasure coding data are determined.

[0040] If the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, an initial decoded combination bitmap of the erasure coding data is constructed.

[0041] Based on the positions of the failed block and the target data block in the initial decoded combination bitmap, a failed block bitmap is constructed;

[0042] Based on the initial decoded combination bitmap and the failed block bitmap, a target decoded combination is constructed, and erasure coding decoding is performed on the target decoded combination until the downgrade read is successful or each target decoded combination is traversed.

[0043] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0044] In response to a read request for a target data block in erasure coding data, the target data block is read, and if the read of the target data block fails, the number of check blocks and the number of invalid blocks included in the erasure coding data are determined.

[0045] If the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, an initial decoded combination bitmap of the erasure coding data is constructed.

[0046] Based on the positions of the failed block and the target data block in the initial decoded combination bitmap, a failed block bitmap is constructed;

[0047] Based on the initial decoded combination bitmap and the failed block bitmap, a target decoded combination is constructed, and erasure coding decoding is performed on the target decoded combination until the downgrade read is successful or each target decoded combination is traversed.

[0048] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0049] In response to a read request for a target data block in erasure coding data, the target data block is read, and if the read of the target data block fails, the number of check blocks and the number of invalid blocks included in the erasure coding data are determined.

[0050] If the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, an initial decoded combination bitmap of the erasure coding data is constructed.

[0051] Based on the positions of the failed block and the target data block in the initial decoded combination bitmap, a failed block bitmap is constructed;

[0052] Based on the initial decoded combination bitmap and the failed block bitmap, a target decoded combination is constructed, and erasure coding decoding is performed on the target decoded combination until the downgrade read is successful or each target decoded combination is traversed.

[0053] The aforementioned erasure coding-based degradation reading method, apparatus, computer device, computer-readable storage medium, and computer program product, in response to a read request for a target data block in erasure coding data, read the target data block, and, in the event of a failure to read the target data block, determine the number of check blocks and the number of invalid blocks included in the erasure coding data to determine the number of decoding combinations of the erasure coding data; if the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, it indicates the existence of multiple decoding combinations, and an initial decoding combination bitmap of the erasure coding data is constructed; furthermore, based on the invalid blocks and... The position of the target data block in the initial decoding combination bitmap is used to construct a failed block bitmap, which serves as the basis for filtering subsequent decoding combinations. Furthermore, based on the initial decoding combination bitmap and the failed block bitmap, target decoding combinations are constructed, and erasure coding decoding is performed on the target decoding combinations until the downgrade read is successful or all target decoding combinations are traversed. In the case of multiple decoding combinations, the failed block bitmap is used to filter the candidate decoding combinations constructed based on the initial decoding combinations, avoiding the selection of decoding combinations with failed blocks for decoding. This reduces decoding resource consumption and shortens the downgrade read process, thereby improving the efficiency of downgrade read. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating a degradation read method based on erasure coding in one embodiment;

[0056] Figure 2 This is a schematic diagram of the erasure coding data reading process in one embodiment;

[0057] Figure 3 This is a schematic diagram illustrating the principle of generating an updated bitmap with decoded combination based on an initial decoded combination bitmap in one embodiment.

[0058] Figure 4 This is a flowchart illustrating the strip consistency verification steps in one embodiment;

[0059] Figure 5 This is a schematic diagram illustrating the principle of updating the bitmap of a failed block in one embodiment;

[0060] Figure 6 This is a schematic diagram illustrating the principle of the initial decoding combined bitmap in one embodiment;

[0061] Figure 7 This is a flowchart illustrating a degradation read method based on erasure coding in another embodiment;

[0062] Figure 8 This is a schematic diagram of the decoding and combination calculation process in one embodiment;

[0063] Figure 9 This is a schematic diagram illustrating the principle of the last combination and special values ​​during the traversal process in one embodiment.

[0064] Figure 10 This is a schematic diagram of the strip consistency verification process in another embodiment;

[0065] Figure 11 This is a schematic diagram illustrating the modified realMask principle in one embodiment;

[0066] Figure 12 This is a structural block diagram of a degradation read device based on erasure coding in one embodiment;

[0067] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] As described in the background section, the erasure coding degradation read method in related technologies suffers from low efficiency. The inventors discovered that this problem arises because, to ensure the reliability and availability of storage systems, replication or erasure coding is generally used to improve data redundancy. In an erasure coding storage system, each stripe contains D data blocks and P parity blocks, for example, 4 data blocks and 2 parity blocks, totaling 6 blocks. During the data write process, EC encoding is used to calculate the parity blocks of the stripe and write them to disk along with the data blocks. When there is a read request, if the original data block fails to be read, other data blocks and parity blocks in the stripe can be read using a degradation read method. That is, D blocks are selected from the corresponding stripe and EC decoding is used to recover the unreadable original data block. In the degradation read and original data repair methods of related technologies, degradation reads with less bandwidth are usually achieved by prioritizing the encoding of data blocks and parity blocks to optimize system performance. For example, one approach selects data blocks and parity blocks for degraded reads and original column data repair based on encoding priority. This involves reading D blocks sequentially according to their priority: data blocks, parity blocks in the memory server, and parity blocks in the SSD, before decoding the original column data. This method not only increases memory usage but also requires retrieving the parity block from the log when decoding or post-decoding data verification fails, increasing read overhead. Furthermore, the conditions for completing the degraded read are unclear. If degraded reads continue and the decoded original column data fails verification, the same data blocks and parity blocks may be repeatedly read, resulting in multiple degraded reads failing to recover the original column data. It may also lead to the degraded read being considered a failure as soon as the selected D blocks fail to decode, without exploring all possible combinations of D blocks, thus missing combinations that could correctly decode the original column data and causing data loss. Therefore, the erasure coding degraded read method in related technologies suffers from low efficiency.

[0070] Based on the above reasons, this application provides a degradation read method based on erasure coding. When there is a need to read erasure-coded data, and the target data block in the erasure-coded data fails to be read, the number of check blocks and the number of invalid blocks included in the erasure-coded data are determined to determine the number of existing decoding combinations. If multiple decoding combinations exist, an initial decoding combination bitmap of the erasure-coded data is constructed, and further, based on the positions of the invalid blocks and the target data block in the initial decoding combination bitmap, an invalid block bitmap is constructed. Based on the initial decoding combination bitmap and the invalid block bitmap, a target decoding combination is constructed, and erasure coding decoding processing is performed on the target decoding combination until the degradation read is successful or all target decoding combinations are traversed. The invalid block bitmap can filter out invalid decoding combinations, finding valid target decoding combinations more quickly, thereby shortening the degradation read process and improving the efficiency of degradation reads.

[0071] In one embodiment, such as Figure 1 As shown, a degradation read method based on erasure coding is provided. This embodiment illustrates the application of this method to a distributed storage system deployed on a server. It is understood that this system can also be deployed on a terminal, and the method can also be applied to a system including both a terminal and a server, implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps S102 to S108. Wherein:

[0072] Step S102: In response to a read request for a target data block in erasure coding data, the target data block is read, and if the read of the target data block fails, the number of check blocks and the number of invalid blocks included in the erasure coding data are determined.

[0073] The erasure coding data can be a complete set of data containing data blocks, check blocks, etc., formed by encoding the original data using redundant coding (such as erasure coding). The data block is the basic unit after data segmentation, and it is the logical unit for data storage and processing in the stripe. The check block can be a redundant data block in the erasure coding data used to verify and recover the original data. When some data blocks are lost or cannot be read directly, the data is recovered by decoding the check block with the remaining data blocks. The failed block can be a data block that is marked as unusable or failed during the degradation read or decoding process, including the failed state of the target data block itself that cannot be read, and other data blocks used for decoding.

[0074] The target data block can be the data block in the erasure coding data that corresponds to the read request, and serves as the target object of the read request.

[0075] Optionally, in response to a read operation on a target data block in erasure coding data, the distributed storage system parses the read / write marker and logical address of the target data block based on the read operation, converts the logical address into a physical address, and then reads the target data block using the physical address and the read / write marker. Understandably, if the target data block is successfully read, the distributed storage system returns the read data corresponding to the read / write marker. If the target data read fails, including failure to execute the read operation on the target data block or failure to verify the data returned by the read operation, the distributed storage system determines the number of verification blocks and the number of invalid blocks included in the erasure coding data to determine the number of decoding combinations for the erasure coding data.

[0076] For example, such as Figure 2 The diagram illustrates the data reading process for erasure coding. The distributed storage system's read initiation module is responsible for issuing read-type I / O (input / output) tags and verifying the returned data corresponding to the I / O tags. The read processing module receives read requests from the read initiation module, performs the conversion between logical and physical addresses, sends the converted physical address to the data storage module, and performs degraded read processing if a data block read fails. The data storage module returns the corresponding data block based on the physical address. The read initiation module issues read I / O and verifies the returned data; if verification fails, a degraded read is initiated. When the read processing module determines that a degraded read is needed based on the tag mask carried by the I / O or the result of the original column read, it first queries all data and the physical address of the check block corresponding to the I / O, and then reads all block data in batches at once.

[0077] Step S104: If the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, construct the initial decoding combination bitmap of erasure code data.

[0078] A bitmap (mask) is a binary image tool used to define which areas of an image data are visible and which are not, or which areas are foreground and which areas are background. In this embodiment, a bitmap can be a mapping structure marked by bits to indicate the status, selection, and assignment of blocks. Its uses include indicating which data blocks are available or invalid, and which blocks are included in the current decoding combination.

[0079] The initial decoding combination bitmap can be a bitmap of a set of data blocks that can be used for decoding, constructed at the beginning of the downgrade read phase based on the number of data blocks included in the current erasure coding data.

[0080] The invalid block bitmap can be a set of blocks that are considered invalid in the current decoding context and need to be excluded during the decoding process, recorded in bitmap form.

[0081] Optionally, if the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks, the distributed storage system indicates that there are multiple decoding combinations for the current erasure coding data, thereby constructing an initial decoding combination bitmap for the erasure coding data for subsequent decoding combination filtering.

[0082] Understandably, during the degradation read process of erasure-coded data, the distributed storage system needs to perform different processing depending on the number of failed blocks and parity blocks. Specifically, when the number of failed blocks is greater than the number of parity blocks, decoding is impossible, and the system returns either "degradation read complete" or "decoding failed." When the number of failed blocks equals the number of parity blocks, there is only one decoding combination, so decoding is performed directly based on this unique combination. The decoded data is then further processed, and the system returns either "failure" or "read success" based on the processing result. When the number of failed blocks is less than the number of parity blocks, multiple decoding combinations exist. In this case, a suitable and feasible combination needs to be calculated based on the tags carried by the I / O and the read results of the data blocks and parity blocks in the erasure-coded data stripes for decoding the data.

[0083] Step S106: Construct a fault block bitmap based on the positions of the fault block and the target data block in the initial decoded combination bitmap.

[0084] Optionally, both the failed block and the target data block are invalid data blocks that cannot be read or used for decoding. Therefore, the distributed storage system constructs a failed block bitmap based on the positions of the failed block and the target data block in the initial decoding combination bitmap. This marks the failed blocks and target data blocks in the erasure coding data stripes that cannot be used as decoding data blocks in subsequent decoding, serving as the basis for decoding combination screening. This avoids selecting failed blocks during subsequent decoding combination screening, thus saving decoding resources and decoding time.

[0085] Step S108: Based on the initial decoded combination bitmap and the invalid block bitmap, construct the target decoded combination, and perform erasure coding decoding processing on the target decoded combination until the downgrade read is successful or each target decoded combination is traversed.

[0086] The target decoding combination can be a set of data blocks to be decoded based on the initial decoding combination bitmap and the invalid block bitmap, selected by an algorithm for the current degraded read task. This set includes the data blocks and check blocks that participate in this decoding.

[0087] The erasure coding decoding process can be a process of performing decoding operations based on the available data blocks and check blocks covered by the current target decoding combination to recover the original data of the target data block.

[0088] Degradation reads can be a data read operation in a distributed storage system. It occurs when some data blocks are unavailable due to storage node failures or other reasons, but the system still needs to provide data access services. In this situation, the system utilizes erasure coding to reconstruct the lost data blocks using the remaining available data blocks and checksum blocks, thereby satisfying the read request.

[0089] Optionally, the distributed storage system iteratively constructs candidate decoding combinations through an initial decoding combination bitmap. In each iteration, it also filters the candidate decoding combinations based on the failed block bitmap, i.e., removing decoding combinations involving invalid blocks, thereby obtaining valid target decoding combinations. Further, the distributed storage system uses erasure coding of the data block and check block corresponding to the target decoding combination for decoding processing to restore the original data of the target data block, until a downgrade read is successful or all target decoding combinations are traversed. It is understandable that although valid target decoding combinations can be used for decoding, there is still a probability of decoding failure. For example, the data block involved in the obtained target decoding combination may also be an invalid block, but the distributed storage system fails to detect this invalid block and does not record it in the failed block bitmap.

[0090] In the aforementioned erasure coding-based degradation read method, in response to a read request for a target data block in the erasure coding data, the target data block is read. If the target data block read fails, the number of check blocks and the number of invalid blocks included in the erasure coding data are determined to determine the number of decoding combinations of the erasure coding data. If the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, it indicates the existence of multiple decoding combinations, and an initial decoding combination bitmap of the erasure coding data is constructed. Furthermore, based on the positions of invalid blocks and target data blocks in the initial decoding combination bitmap, an invalid block bitmap is constructed as the basis for filtering subsequent decoding combinations. Further, based on the initial decoding combination bitmap and the invalid block bitmap, target decoding combinations are constructed, and erasure coding decoding processing is performed on the target decoding combinations until the degradation read is successful or all target decoding combinations are traversed. When multiple decoding combinations exist, the invalid block bitmap is used to filter the candidate decoding combinations constructed based on the initial decoding combinations, avoiding the selection of decoding combinations containing invalid blocks for decoding, reducing decoding resource consumption and shortening the degradation read process, thereby improving the efficiency of degradation read.

[0091] In an exemplary embodiment, step S108 involves constructing a target decoding combination based on the initial decoded combination bitmap and the failed block bitmap, including:

[0092] The initial decoded combination bitmap is processed using a preset conversion algorithm to generate a corresponding candidate decoded combination bitmap. Based on the position of the target data block in the candidate decoded combination bitmap, the target data block is inserted and its value is assigned to obtain the updated candidate decoded combination bitmap. The updated candidate decoded combination bitmap is verified based on the failed block bitmap, and the decoded combination corresponding to the updated candidate decoded combination bitmap that passes the verification is determined as the target decoded combination.

[0093] The preset conversion algorithm can be a permutation and combination algorithm that converts the initial decoded combination bitmap into a candidate decoded combination bitmap, such as the 01 conversion method. This is a combination algorithm that uses 0 and 1 to indicate whether an element is selected. It is mainly used to solve the problem of selecting m elements from n elements. Its core idea is to generate all possible combinations through bit operations and sequence transformations. In this embodiment, the 01 conversion method is used to select the combination consisting of blocks with the same number of blocks as the data blocks from the available data blocks and check blocks as the candidate decoded combination. The candidate decoded combination can be a decoded combination obtained by converting the initial decoded combination bitmap through the preset conversion algorithm. It is used to be further verified and selected as the actual target decoded combination in subsequent steps.

[0094] The assignment can be an operation in which a specific position of a specific data block (such as a target data block) is bound to a specific position in the candidate decoding combination bitmap and assigned a corresponding value to form an updated candidate decoding combination bitmap.

[0095] The verification process can be used to check the validity of the updated bitmap of the candidate decoding combination, ensuring that the candidate decoding combination is theoretically usable for decoding.

[0096] Understandably, the initial decoded combination bitmap consists of two types of data. The number of positions in the bitmap that can be assigned values ​​corresponds to the number and value of data blocks and parity blocks in the erasure coding data stripe. For example, the initial decoded combination bitmap consists of 0s and 1s, where a value of 0 indicates that a data block or parity block is not selected as a data block for decoding, and a value of 1 indicates that a data block or parity block is selected as a data block for decoding. For instance, the data structure of the initial decoded combination bitmap also includes positions for recording the number of downgrade reads. For example, before representing the erasure coding stripe, four positions are set to record the number of downgrade reads, with an initial value of 0000. Each downgrade read iteration updates the value assigned to this position. For example, if the first downgrade read is performed using decoded combination, the value becomes 0001, and so on.

[0097] Optionally, the distributed storage system uses a preset conversion algorithm to process the initial decoded combination bitmap and generate a corresponding candidate decoded combination bitmap. For example, if the initial decoded combination bitmap is assigned values ​​of 0 and 1, the preset conversion algorithm used is the 0-1 conversion method. The specific implementation process is as follows: scan the array element values ​​of the initial decoded combination bitmap from left to right for the "10" combination. After finding the first "10" combination, change it to a "01" combination, and move all the "1"s to its left to the leftmost end of the array. This constructs a new candidate decoded combination bitmap. When all n (the number of all 1s contained in the initial decoded combination bitmap) "1"s have been moved to the rightmost end, the last candidate decoded combination bitmap is obtained.

[0098] Furthermore, the distributed storage system inserts the target data block's assignment based on its position in the candidate decoding combination bitmap. In the initial decoding combination bitmap, the order of the positions to be assigned is such that the data blocks in the stripe are arranged from right to left. For example, if the erasure coding data includes four data blocks D0-D3 and two parity blocks P0 and P2, the corresponding order in the initial decoding combination bitmap is: P1, P0, D3, D2, D1, and D0. After the target data block's assignment is inserted, all assignments between the leftmost end of the candidate decoding combination bitmap and the target data block's position in the initial decoding combination bitmap are shifted one position to the left, and the assignments between the rightmost end and the target data block's position are shifted one position to the left, resulting in the updated candidate decoding combination bitmap. Figure 3 The diagram illustrates the principle of generating an updated decoded combined bitmap based on an initial decoded combined bitmap. Here, mask represents the initial decoded combined bitmap, realmask is the updated combined bitmap to be decoded, dataNum is the number of data blocks included in the erasure coding data, and the original column represents the target data block. The same logic applies if the order in the initial decoded combined bitmap is reversed.

[0099] It should be noted that the insertion of the target data block assignment is to mark the position of the data block to be decoded, to avoid decoding errors, and to prevent the constructed bitmap to be decoded from including the target data block. Figure 3 The code uses a red 1 to assign a value to the location of the target data block. Unlike the 1 assigned to other locations, the red 1 indicates that the target data block is a data block to be decoded without data, while other data blocks or check blocks assigned a value of 1 are blocks with corresponding original data.

[0100] Furthermore, the distributed storage system verifies the updated candidate decoding combination bitmap based on the failed block bitmap. It checks whether the updated data to be decoded has selected a failed data block for subsequent decoding, and determines the decoding combination corresponding to the verified updated candidate decoding combination bitmap as the target decoding combination. It can be understood that the verified updated candidate decoding combination bitmap represents the selected data blocks used for decoding that do not involve decoding combinations of failed blocks recorded in the failed block bitmap.

[0101] In this embodiment, the initial decoding combination bitmap is processed using the 01 conversion method to generate a corresponding candidate decoding combination bitmap. By using the principle of permutation and combination, all kinds of decoding combinations can be obtained. Based on the position of the target data block in the candidate decoding combination bitmap, the target data block is inserted and its value is assigned to obtain an updated candidate decoding bitmap. This marks the target data block to be decoded, improving the accuracy of subsequent decoding. In addition, the updated candidate decoding bitmap is verified based on the failed block bitmap, and the decoding combination corresponding to the updated candidate decoding bitmap that passes the verification is determined as the target decoding combination. This avoids selecting decoding combinations with failed blocks for decoding, reduces decoding resource consumption, and shortens the downgrade read process, thereby further improving the efficiency of downgrade read.

[0102] In an exemplary embodiment, the steps in the above embodiments to verify the updated candidate decoding combination bitmap based on the failed block bitmap include:

[0103] Perform a bitwise AND operation between the failed block bitmap and the updated candidate decoder combination bitmap to obtain a computed bitmap. If the value assigned to the target data block in the computed bitmap is different from other values, it indicates that the updated candidate decoder combination bitmap has passed the verification.

[0104] Bitwise AND operation can be performed bitwise AND operation on two bitmaps to obtain a new bitmap.

[0105] The computation bitmap can be the result bitmap of a bitwise AND operation, used to verify whether the candidate decoding combination bitmap is valid.

[0106] Optionally, the distributed storage system performs a bitwise AND operation between the failed block bitmap and the updated candidate decoding combination bitmap to obtain a computed bitmap. Continuing from the previous embodiment, for example, the erasure coding data includes four data blocks D0-D3 and two parity blocks P0 and P2. The target data block is D1, the known failed block is D3, the updated candidate decoding combination bitmap is 110111 (the second bit from the right indicates the position of the target data block, with a special value of 1 inserted; the selected data blocks for decoding are P1, P0, D2, and D0), and the failed block bitmap is 001010. After performing a bitwise AND operation between the candidate decoding combination bitmap and the failed block bitmap, the resulting computed bitmap is 000010, i.e. If the assignment corresponding to the target data block in the calculated bitmap is different from other assignments, the updated candidate decoding combination bitmap passes the verification. For example, if the updated candidate decoding combination bitmap is 101111 (the second bit from the right is the position of the target data block, with a special assignment of 1 inserted, and the selected data blocks for decoding are P1, D3, D2, and D0), and a bitwise AND operation is performed with the invalid block bitmap, the resulting calculated bitmap is 001010. This does not satisfy the verification condition that the assignment corresponding to the target data block in the calculated bitmap is different from other assignments, therefore the updated candidate decoding combination bitmap fails the verification.

[0107] In this embodiment, a computed bitmap is formed by performing a bitwise AND operation on the updated bitmap to be decoded and the bitmap of the invalid block. Only when the assignment of the computed bitmap at the position of the target data block is different from the assignment of other blocks is the updated bitmap to be decoded considered to have passed the verification and then enter the erasure coding decoding attempt. Otherwise, the decoding combination is excluded to avoid repeated attempts on invalid combinations, thereby reducing the number of downgrade read attempts and further improving the decoding success rate and overall reading efficiency.

[0108] In one exemplary embodiment, such as Figure 4 As shown, the degradation read method based on erasure coding described in the above embodiments further includes the following steps S402 to S408. Wherein:

[0109] Step S402: When the number of downgrade reads of the target data block reaches the preset downgrade read count threshold and the erasure code data has not undergone strip consistency verification, erasure code decoding is performed on the target decoding combination of the current downgrade read to obtain the decoded data.

[0110] Among them, strip consistency verification can be a consistency verification performed on data blocks at the strip level to determine the correctness status of each data block to be verified.

[0111] The decoded data can be the data corresponding to the data block to be verified obtained by erasure coding.

[0112] Optionally, as described in the above embodiments, a position is set in the initial decoded combination bitmap to record the number of downgrade reads. Each time a downgrade read is performed, the data is updated to count the number of iterative downgrade reads. When the number of downgrade reads for the target data block reaches a preset downgrade read threshold, and the erasure coding data has not undergone stripe consistency verification, the distributed storage system performs erasure coding decoding on the target decoded combination for the current downgrade read to obtain the decoded data. It should be noted that decoding in this case may not necessarily decode the target data block, because the target decoded combination in this round of decoding may not ultimately be able to effectively recover the original decoded combination of the target data block, but it can obtain the decoded data of other data blocks.

[0113] Furthermore, stripe consistency verification can identify whether there are other faulty blocks in the stripes of erasure coding data, thereby updating the faulty block bitmap. This allows subsequent iterative degradation reads to exclude candidate decoding combinations containing new faulty blocks, further improving degradation read efficiency. Preferably, stripe consistency verification is performed only when the number of degradation reads reaches a preset threshold. Because stripe consistency verification consumes a lot of resources, performing it only after the number of degradation reads reaches the threshold can reduce resource consumption while improving degradation read efficiency. However, in some embodiments, stripe consistency verification can also be performed even if the number of degradation reads has not reached the threshold; this is not specifically limited.

[0114] Step S404: Perform strip consistency verification on the decoded data and the original data of the target decoded combination to obtain the consistency verification results of each data block to be verified in the target decoded combination.

[0115] Optionally, the distributed storage system performs strip-by-strip consistency checks on the decoded data and the original data of the data blocks involved in the target decoding combination, compares whether the decoded data and the original data are consistent, and obtains the consistency check results of each data block to be checked in the target decoding combination.

[0116] Step S406: If some data blocks to be verified fail to be verified in the target decoding combination represented by the consistency verification result, the data blocks to be verified that failed to be verified are identified as new failed blocks.

[0117] Optionally, if some data blocks to be verified fail to be verified in the target decoding combination represented by the consistency verification result, the decoded data of the data block to be verified is inconsistent with the original data, thereby identifying the data block to be verified that failed to be verified as a new invalid block.

[0118] Understandably, the consistency check result also includes two other cases. If the consistency check result indicates that the stripes of each data block to be checked are consistent, the step of constructing the target decoding combination based on the initial decoding combination bitmap and the invalid block bitmap is returned. If the consistency check result indicates that the stripes of each data block to be checked are inconsistent, it is impossible to determine whether the data block to be checked is valid, and no special processing is performed.

[0119] Step S408: Update the failure block bitmap according to the new failure block.

[0120] Optionally, such as Figure 5 The diagram illustrates the principle of updating the failed block bitmap, where failBmp is the failed block bitmap. The distributed storage system determines the position of the new failed block in the failed block bitmap and updates its value to match the values ​​of other failed blocks, thus obtaining a new failed block bitmap. For example, erasure coding data includes four data blocks D0-D3 and two parity blocks P0 and P2. The target data block is D1, the known failed block is D3, the failed block bitmap is 001010, and the new failed block is D0. Therefore, the updated failed block bitmap is 001011.

[0121] In this embodiment, by introducing stripe consistency verification during the degraded read process to identify and update new failed blocks, invalid decoding combinations are gradually eliminated and invalid repeated attempts are avoided. If the decoding fails to recover the target data block in the current round of degraded read, it can also identify whether there is a new failed block, providing information for subsequent iterations. At the same time, by judging the stripe consistency verification result, the data block that failed the verification is marked as a new failed block and the failed block bitmap is updated, ensuring that the failed block is removed when constructing the target decoding combination in the future, shortening the degraded read path, thereby improving the efficiency and success rate of degraded read under the premise of controllable resource overhead.

[0122] In an exemplary embodiment, the erasure coding-based degradation read method described above further includes:

[0123] After updating the fault block bitmap based on the new fault block, the process proceeds to construct the target decoding combination based on the initial decoding combination bitmap and the fault block bitmap, and performs erasure coding decoding on the target decoding combination until the downgrade read is successful or each target decoding combination is traversed.

[0124] Optionally, after updating the failed block bitmap according to the new failed block, the distributed storage system performs the following steps: constructing a target decoding combination based on the initial decoded combination bitmap and the failed block bitmap, and performing erasure coding decoding on the target decoded combination until the downgrade read is successful or each target decoded combination is traversed. Then, the updated failed block bitmap is used to enter the next round of downgrade read iteration.

[0125] In this embodiment, after stripe consistency verification and identification of new failed blocks, the failed block bitmap is updated with the new failed block information. The target decoding combination is then reconstructed based on the updated failed block bitmap and the initial decoded combination bitmap for erasure coding decoding, forming an adaptive, iterative decoding process. This eliminates known invalid blocks in each iteration, reducing the number of invalid decoding attempts and further improving the efficiency of downgraded reads. If the target decoding combination fails to decode, the updated bitmap enters the next round of downgraded read filtering, continuing to traverse potential decoding paths until the target data block is successfully read or all possible combinations are exhausted, thus improving the system's fault tolerance and data availability.

[0126] In an exemplary embodiment, step S104, when the sum of the number of failed blocks and the number of target data blocks is less than the number of parity blocks, constructs an initial decoded combination bitmap for the erasure coding data, including:

[0127] If the sum of the number of invalid blocks and the number of target data blocks is less than the number of parity blocks, determine the target number of data blocks included in the erasure coding data; construct an initial decoding combination bitmap based on the target number.

[0128] It is understandable that the principle of erasure coding decoding is that the stripe of erasure coding data includes k data blocks and m parity blocks, and the sum of k and m is n (m, n and k are all positive integers). These data blocks are stored on n hard drives, which can tolerate the failure of m hard drives (assuming there are k data blocks initially). When no more than m hard drives fail, all the original data can be calculated by arbitrarily selecting k normal data blocks from the k data blocks and m parity blocks.

[0129] Optionally, such as Figure 6 As shown, a schematic diagram of the initial decoding combination bitmap is provided. When the sum of the number of failed blocks and the number of target data blocks is less than the number of parity blocks, it indicates that there are multiple decoding combinations that can be attempted to decode the original data of the target data block. The distributed storage system determines the target number of data blocks included in the stripe of erasure coding data and constructs an initial decoding combination bitmap based on the target number. For example, if the target number is 6 and the number of parity blocks is 3, the constructed initial decoding combination bitmap is 000111111, indicating that for decoding the target data block, 6 blocks need to be selected from 9 blocks for decoding and recovery.

[0130] In this embodiment, when there are multiple available decoding combinations, an initial decoding combination bitmap based on the target data block number is explicitly given. The relationship between the target number and the number of check blocks, as well as the relationship between the selected blocks used for decoding and the unselected blocks, are transformed into a specific initial decoding combination bitmap. This can intuitively reflect the composition of the decoding combination and lay the groundwork for generating different target decoding combinations based on the failed block bitmap.

[0131] In one exemplary embodiment, such as Figure 7 As shown, another method for degraded reading based on erasure coding is provided, including:

[0132] First, the read initiation module issues a read I / O and verifies the returned data. If the verification fails, a downgraded read is initiated. The read processing module determines whether a downgraded read is needed based on the marker mask carried by the I / O or the result of reading the original column (target data block). It first queries all data and the physical address of the check block corresponding to the I / O, and then reads all block data in batches at once to obtain the read results, including the original column and the failed block that failed to be read.

[0133] Then, given multiple decoding combinations, an initial mask (initial decoding combination bitmap) and a failMask (failed block bitmap) are constructed. Initially, both are 0. The mask is updated during degraded reads, with the high-order 4 bytes storing the number of degraded reads. The failMask is updated during striped consistency verification. The initial failBmp (failed block bitmap) is set based on the read results; the data storage module returns that the corresponding bits of blocks that failed to read (failed blocks) are set to 1. The striped consistency verification is determined based on the input IO flag mask. If no verification is performed and the number of degraded reads reaches the verification threshold, striped consistency verification is performed and marked as verified. If the verification is consistent, post-decoding processing is performed directly; if inconsistent, the next decoding combination is calculated until all combinations are traversed or the read is successful. This process includes two key steps: combination calculation and consistency verification, both of which require the use of the IO flag mask and interact with each other. For ease of description, the calculation of decoding combinations is introduced first, followed by the steps of consistency verification.

[0134] Specifically, such as Figure 8The diagram illustrates the decoding combination calculation process. First, it determines whether the input IO flag indicates a first downgrade read. If so, an initial combination (initial decoding combination bitmap) is generated. Otherwise, a 0 / 1 conversion method is used to generate the next combination (candidate decoding combination bitmap) based on the IO flag. The original column information is added to the initial combination to obtain the actual decoding combination (updated candidate decoding combination bitmap). Then, the invalid block bitmap is used to determine if the decoding combination is valid. If valid, it can be used for the next decoding step; otherwise, the 0 / 1 conversion method is used again to obtain the next combination. However, if the current combination is already the last one, failure is returned directly.

[0135] Specifically, for the IO flag mask (initial value 0, special value 1 for original column reads), data block number dataNum, parity block number parityNum, original column index oriIndex, and fail block bitmap failBmp, the first degraded read occurs when the IO flag is the initial value or the special value for original column reads. The initial combination mask for the first degraded read is generated by setting the first dataNum bit of the mask to 1, such as... Figure 6 As shown. Based on the initial combined mask, the original column information is inserted into the corresponding positions in the mask to obtain the real decoded combined mask, realMask. The specific steps are: keep the 0th bit of the mask to the bit before the oriIndex of the original column unchanged, shift the oriIndex to the most significant bit of the mask left by 1 bit, and finally set the oriIndex position to 1 to obtain realMask, as shown. Figure 3 As shown. If the bitwise AND operation between realMask and the invalid block bitmap results in a value equal to the original column, then realMask is valid (target decoding combination). Otherwise, the next combination needs to be generated based on the mask using a 0 / 1 conversion algorithm, and this process is repeated. Figure 8 Continue with steps 3 and 4 until a valid realMask is found or all combinations are traversed.

[0136] If a valid realMask is found, then a block combination for EC decoding has been found. The original column data is calculated using the EC decoding algorithm and these block data, and then processed accordingly. If processing is successful, data is returned, and the mask is brought back to the initiating read module; otherwise, proceed as follows... Figure 7 In the next step, perform strip consistency verification or calculate the next valid combination realMask.

[0137] When the first minIndex bits of the mask (i.e., minIndex = parityNum - 1) are all 0, it indicates that all combinations have been traversed. The mask is then set to a special value of all 1s, allowing for direct comparison of this value in subsequent reads, reducing the number of loops and reads. Figure 9 The diagram shown illustrates the principle of the last combination and special values ​​during the traversal process.

[0138] Specifically, such as Figure 10 As shown, a schematic diagram of the stripe consistency verification process is provided. Stripe consistency verification is performed when the number of downgrade reads reaches a threshold (downgrade read count threshold). Based on information such as dataNum and failBmp, the last realMask can be directly generated. Then, the realMask is corrected using the failed block bitmap. Special processing is required for already failed blocks to ensure that the decoded data has corresponding data read from the storage data module before data comparison can be performed. The schematic diagram of the realMask correction principle is shown below. Figure 11 As shown.

[0139] Among these, comparing multiple data blocks may present three scenarios, where the strips are consistent according to... Figure Two Processing can effectively reduce the number of downgrade reads; when striping is inconsistent and some decoded data is consistent with data in the data storage module, a failMask is set and the failed block bitmap is updated simultaneously. For example... Figure Nine As shown, inconsistent blocks are set as invalid blocks. Subsequent decoding calculations can more quickly exclude combinations containing these invalid blocks, reducing the number of EC decoding calculations. When stripes are inconsistent and all data comparisons are different, it is impossible to determine whether the data block is valid, and no special processing is performed.

[0140] In this embodiment, the method can detect some invalid data blocks, terminate the combination traversal early, reduce the number of downgrade reads and EC decodings, and improve downgrade read performance. It can obtain valid true combinations faster and continuously advances forward, preventing repeated downgrade reads caused by combination backtracking due to changes in the invalid column. By taking a fixed number of high-order records for downgrade reads, when the number reaches a set threshold, it triggers strip consistency verification. By comparing the decoded data and the data returned by the data storage module, invalid data blocks can be detected, reducing the number of invalid combination decodings and reads. Utilizing the fixed position of the original column, the original combination uses one less bit, and the original combination mask is brought back to the upper layer with the data, ensuring that the true combination obtained during the next downgrade read does not backtrack under various abnormal conditions. The failMask set in the consistency verification calculation can help quickly eliminate invalid combinations and reduce invalid EC decoding calculations. Furthermore, the method in this embodiment is simple and fast to calculate, has low randomness, and low additional memory overhead.

[0141] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0142] Based on the same inventive concept, this application also provides an erasure code-based degradation read device for implementing the erasure code-based degradation read method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more erasure code-based degradation read device embodiments provided below can be found in the limitations of the erasure code-based degradation read method described above, and will not be repeated here.

[0143] In one exemplary embodiment, such as Figure 12 As shown, a degradation reading device 1200 based on erasure coding is provided, including: a data degradation reading module 1201, a combined bitmap construction module 1202, an invalid bitmap construction module 1203, and a data block decoding module 1204, wherein:

[0144] The data degradation read module 1201 is used to respond to a read request for a target data block in erasure coding data, read the target data block, and determine the number of check blocks and the number of invalid blocks included in the erasure coding data if the target data block read fails.

[0145] The combined bitmap construction module 1202 is used to construct an initial decoded combined bitmap of erasure coding data when the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks.

[0146] The failure bitmap construction module 1203 is used to construct a failure block bitmap based on the positions of the failure block and the target data block in the initial decoded combined bitmap;

[0147] The data block decoding module 1204 is used to construct the target decoding combination based on the initial decoding combination bitmap and the failed block bitmap, and to perform erasure coding decoding processing on the target decoding combination until the downgrade read is successful or each target decoding combination is traversed.

[0148] Furthermore, in one embodiment, the data block decoding module 1204 is further configured to process the initial decoding combination bitmap using a preset conversion algorithm to generate a corresponding candidate decoding combination bitmap; insert the target data block's assignment according to the position of the target data block in the candidate decoding combination bitmap to obtain an updated candidate decoding combination bitmap; verify the updated candidate decoding combination bitmap according to the failed block bitmap, and determine the decoding combination corresponding to the updated candidate decoding combination bitmap that passes the verification as the target decoding combination.

[0149] Furthermore, in one embodiment, the data block decoding module 1204 is also used to perform a bitwise AND operation on the failed block bitmap and the updated candidate decoding combination bitmap to obtain a calculated bitmap; if the value corresponding to the target data block in the calculated bitmap is different from other values, it indicates that the updated candidate decoding combination bitmap has passed the verification.

[0150] Furthermore, in one embodiment, the data block decoding module 1204 is further configured to: perform erasure coding decoding on the target decoding combination of the current downgraded read when the number of downgrade reads for the target data block reaches a preset downgrade read count threshold and the erasure coding data has not undergone strip consistency verification; perform strip consistency verification on the decoded data and the original data of the target decoding combination to obtain the consistency verification result of each data block to be verified in the target decoding combination; if the consistency verification result indicates that some data blocks to be verified in the target decoding combination have failed verification, determine the data blocks to be verified that have failed verification as new failed blocks; and update the failed block bitmap according to the new failed blocks.

[0151] Furthermore, in one embodiment, the data block decoding module 1204 is also configured to, after updating the fault block bitmap according to the new fault block, perform the following steps: construct a target decoding combination based on the initial decoding combination bitmap and the fault block bitmap, and perform erasure coding decoding processing on the target decoding combination until the downgrade read is successful or each target decoding combination is traversed.

[0152] Furthermore, in one embodiment, the combined bitmap construction module 1202 is also used to determine the target number of data blocks included in the erasure coding data when the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks; and to construct an initial decoding combined bitmap based on the target number.

[0153] Each module in the erasure coding-based degradation read device 1200 described above can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0154] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores erasure coding data, initial decoded combination bitmaps, and failed block bitmaps, among other data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements an erasure coding-based degradation read method.

[0155] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0156] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0157] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0158] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A degradation reading method based on erasure coding, characterized in that, The method includes: In response to a read request for a target data block in erasure coding data, the target data block is read, and if the read of the target data block fails, the number of check blocks and the number of invalid blocks included in the erasure coding data are determined. If the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, an initial decoded combination bitmap of the erasure coding data is constructed. Based on the positions of the failed block and the target data block in the initial decoded combination bitmap, a failed block bitmap is constructed; Based on the initial decoded combination bitmap and the failed block bitmap, a target decoded combination is constructed, and erasure coding decoding is performed on the target decoded combination until the downgrade read is successful or each target decoded combination is traversed.

2. The method according to claim 1, characterized in that, The construction of the target decoding combination based on the initial decoding combination bitmap and the failed block bitmap includes: The initial decoded combination bitmap is processed using a preset conversion algorithm to generate a corresponding candidate decoded combination bitmap. Based on the position of the target data block in the candidate decoding combination bitmap, the value of the target data block is inserted to obtain the updated candidate decoding combination bitmap; The updated candidate decoding combination bitmap is verified based on the failed block bitmap, and the decoding combination corresponding to the updated candidate decoding combination bitmap that passes the verification is determined as the target decoding combination.

3. The method according to claim 2, characterized in that, The step of verifying the updated candidate decoding combination bitmap based on the failed block bitmap includes: Perform a bitwise AND operation between the failed block bitmap and the updated candidate decoding combination bitmap to obtain a calculated bitmap; If the value assigned to the target data block in the computational bitmap is different from other values, it indicates that the updated candidate decoding combination bitmap has passed the verification.

4. The method according to claim 1, characterized in that, The method further includes: If the number of downgrade reads for the target data block reaches a preset downgrade read count threshold, and the erasure coding data has not undergone strip consistency verification, erasure coding decoding is performed on the target decoding combination of the current downgrade read to obtain the decoded data. Perform strip-by-strip consistency verification on the decoded data and the original data of the target decoded combination to obtain the consistency verification results of each data block to be verified in the target decoded combination; If, in the consistency verification result characterizing the target decoding combination, some of the data blocks to be verified fail verification, the data blocks to be verified that fail verification are identified as new invalid blocks. The failure block bitmap is updated based on the new failure block.

5. The method according to claim 4, characterized in that, The method further includes: After updating the fault block bitmap according to the new fault block, the steps of constructing a target decoding combination based on the initial decoding combination bitmap and the fault block bitmap, and performing erasure coding decoding processing on the target decoding combination until the downgrade read is successful or each target decoding combination is traversed.

6. The method according to any one of claims 1 to 5, characterized in that, If the sum of the number of invalid blocks and the number of target data blocks is less than the number of check blocks, an initial decoded combination bitmap is constructed for the erasure coding data, including: If the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks, the target number of data blocks included in the erasure coding data is determined. An initial decoding combination bitmap is constructed based on the target number.

7. A degradation reading device based on erasure coding, characterized in that, The device includes: The data degradation read module is used to respond to a read request for a target data block in erasure coding data, read the target data block, and determine the number of check blocks and the number of invalid blocks included in the erasure coding data if the read of the target data block fails. A combined bitmap construction module is used to construct an initial decoded combined bitmap of the erasure coding data when the sum of the number of failed blocks and the number of target data blocks is less than the number of check blocks; A failure bitmap construction module is used to construct a failure block bitmap based on the positions of the failure block and the target data block in the initial decoded combination bitmap; The data block decoding module is used to construct a target decoding combination based on the initial decoding combination bitmap and the failed block bitmap, and to perform erasure coding decoding processing on the target decoding combination until the downgrade read is successful or each of the target decoding combinations is traversed.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images