Data reconstruction method, medium, device, program product and distributed storage system

Abstract

A data reconstruction method, medium, device, program product and distributed storage system, applied to the distributed storage system comprising a plurality of data nodes, the method comprising: determining a plurality of candidate data nodes where a plurality of target data blocks for reconstructing original data are located; the number of the candidate data nodes is greater than the minimum number of the data nodes for reconstructing the original data; respectively estimating the current transmission performance of the plurality of candidate data nodes based on the historical transmission performance of the plurality of candidate data nodes; requesting a target data block from at least one target data node in the plurality of candidate data nodes, so that after the target data block is requested from the at least one target data node, the total number of the requested data blocks on the plurality of candidate data nodes is positively correlated with the current transmission performance of the plurality of candidate data nodes respectively; the number of the target data nodes is smaller than that of the candidate data nodes; and performing data reconstruction on the original data based on the target data block returned by the at least one target data node.

Description

Technical Field

[0001] The present disclosure relates to the field of distributed storage technology, and in particular to a data reconstruction method, medium, device, program product, and distributed storage system. Background Art

[0002] To improve data fault tolerance in a distributed storage system, multiple data blocks used to reconstruct the original data can be stored on multiple data nodes in the distributed storage system. If the original data is lost or damaged, data blocks can be read from multiple data nodes and reconstructed based on the read data blocks. The throughput of this process is limited by the slowest data node among the multiple data nodes. If the transmission speed of some data nodes in a distributed storage system remains low for an extended period, the distributed storage system's cluster bandwidth cannot be effectively utilized, resulting in low throughput and poor throughput stability. Summary of the Invention

[0003] In a first aspect, an embodiment of the present disclosure provides a data reconstruction method, which is applied to a distributed storage system including multiple data nodes, the method comprising: determining multiple candidate data nodes where multiple target data blocks for reconstructing original data are located; the number of candidate data nodes is greater than the minimum number of data nodes where the target data blocks for reconstructing the original data are located; based on the historical transmission performance of the multiple candidate data nodes, estimating the current transmission performance of the multiple candidate data nodes respectively; requesting a target data block from at least one target data node among the multiple candidate data nodes, so that after requesting the target data block from at least one target data node, the total number of requested data blocks on the multiple candidate data nodes is positively correlated with the current transmission performance of the multiple candidate data nodes respectively; the number of target data nodes is less than the number of candidate data nodes; and reconstructing the original data based on the target data block returned by at least one target data node.

[0004] In some embodiments, the current transmission performance of any one of the multiple target data nodes is better than the current transmission performance of any one of the multiple candidate data nodes except the multiple target data nodes.

[0005] In some embodiments, the historical transmission performance is expressed based on the historical transmission delay, and the current transmission performance is expressed based on the current transmission delay; the current transmission performance of the multiple candidate data nodes is estimated based on the historical transmission performance of the multiple candidate data nodes, including: for any candidate data node, based on the historical transmission delay of the candidate data node, and the total number of data blocks that the candidate data node has not yet completed transmitting, estimating the current transmission delay of the candidate data node.

[0006] In some embodiments, the data blocks that the candidate data node has not yet completed transmitting include: data blocks requested by the data request that the candidate data node is currently processing; data blocks requested by the data request that the candidate data node has received and is in a queued state; and target data blocks stored on the candidate data node.

[0007] In some embodiments, the multiple data nodes correspond one-to-one to multiple cache queues, and the cache queue corresponding to each data node is used to cache data requests received by the data node, and the data requests are used to request data blocks stored by the data node for reconstructing the original data; the data requests received by the candidate data node and in a queued state are cached in the cache queue corresponding to the candidate data node, and after the data block requested by the data request currently being processed by the candidate data node is sent by the candidate data node, the candidate data node uses the next data request in the corresponding cache queue as the data request currently being processed by the candidate data node.

[0008] In some embodiments, the method further includes: obtaining multiple data requests received by the candidate data node within a historical time window, the data requests being used to request data blocks on the candidate data node for reconstructing the original data; determining the average transmission delay of the candidate data node within the historical time window based on the average value of the transmission delays of the data blocks requested by the multiple data requests, and determining the historical transmission delay of the candidate data node based on the average transmission delay of the candidate data node within the historical time window.

[0009] In some embodiments, determining the historical transmission delay of the candidate data node based on the average transmission delay of the candidate data node in the historical time window includes: weighting the average transmission delay of the candidate data node in multiple historical time windows to obtain the historical transmission delay of the candidate data node.

[0010] In some embodiments, the data blocks stored by the plurality of data nodes include data blocks to be reconstructed or check blocks obtained by performing erasure coding on the original data.

[0011] In a second aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present disclosure.

[0012] In a third aspect, an embodiment of the present disclosure provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present disclosure when executing the program.

[0013] In a fourth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in any embodiment of the present disclosure when executed by a processor.

[0014] In a fifth aspect, an embodiment of the present disclosure provides a distributed storage system, which includes a metadata node and multiple data nodes, wherein the metadata node is used to distribute data reconstruction tasks to a reconstruction node among the multiple data nodes; the reconstruction node is any one of the multiple data nodes; and the data node is used to execute the method described in any embodiment of the present disclosure in response to receiving the data reconstruction task when the node is the reconstruction node.

[0015] In an embodiment of the present disclosure, multiple candidate data nodes for reconstructing multiple target data blocks of the original data are first determined. Since the number of candidate nodes is greater than the minimum number of data nodes where the target data blocks are located required for data reconstruction, there is redundancy in the candidate data nodes, and the target data blocks can be obtained from only some of the candidate data nodes to reconstruct the original data. Then, based on the current transmission performance of each candidate data node, a target data node is selected from each candidate data node, and the target data blocks are requested from the target data node. Since the total number of data blocks requested on each candidate data node is positively correlated with the current transmission performance of the corresponding candidate data node, candidate data nodes with better current transmission performance will receive data requests for a larger number of data blocks, while candidate data nodes with worse current transmission performance will receive data requests for a smaller number of data blocks. This reduces the situation where data nodes with worse current transmission performance transmit a larger number of data blocks while data nodes with better current transmission performance transmit a smaller number of data blocks, resulting in the data nodes with better transmission performance being idle for a long time, thereby improving the cluster bandwidth utilization of the distributed storage system, improving the data reconstruction throughput, and improving throughput stability.

[0016] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the present disclosure. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0018] Figure 1 Schematic diagram of a distributed storage system according to an embodiment of the present disclosure.

[0019] Figure 2 It is a schematic diagram of the data reconstruction process of an embodiment of the present disclosure.

[0020] Figure 3 4 is a flow chart of a data reconstruction method according to an embodiment of the present disclosure.

[0021] Figure 4 Schematic diagram of the relationship between the cluster bandwidth and the task volume of each data node according to an embodiment of the present disclosure.

[0022] Figure 5 Schematic diagram of the task scheduling process of an embodiment of the present disclosure.

[0023] Figure 6 is a schematic diagram of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0025] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a", "the" and "the" used in this disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of.

[0026] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0027] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure are further described in detail below with reference to the accompanying drawings.

[0028] A distributed storage system is a system that stores data on multiple independent data nodes to achieve distributed storage and management of data. The distributed storage system needs to ensure that the state presented to the outside is consistent and that state rollback does not occur. Figure 1 As shown, the distributed storage system 10 includes a metadata node (MetaNode) 102 and multiple data nodes (ChunkServers) 104. Metanode 102 is a centralized metadata storage node, typically used to store data status information, storage location information, data length information, etc. Data node 104 is a node in the distributed storage system 10 used to store data, and is generally responsible for operations such as writing, storing, reading, and deleting data copies.

[0029] In the distributed storage system 10, to improve data fault tolerance, multiple data blocks used to reconstruct original data can be stored on multiple data nodes 104 of the distributed storage system 10. To improve the reliability of data in the distributed storage system 10, when some data is lost or damaged, the distributed storage system 10 can obtain data blocks from multiple data nodes 104 to reconstruct the lost or damaged data, and write the reconstructed data to other data nodes 104, thereby ensuring data security.

[0030] One data reconstruction method is a data reconstruction method based on erasure codes. In the data reconstruction method based on erasure codes, a section of data is divided into multiple data blocks of equal length, and the multiple data blocks include original data blocks and at least one check block. The original data block includes the original data content that needs to be stored or transmitted, and can be obtained by dividing the original data into blocks according to a certain division method. The check block is redundant check information calculated on the data block, which is used to detect and recover errors or losses in the data block. The check block does not contain the original user data, but is redundant information calculated based on the content of the data block. When any one or more data blocks are lost, the lost data blocks can be reconstructed and recovered based on other data blocks. Different data blocks can be stored on different data nodes 104. As Figure 2As shown, the example of data nodes 104 being equal to 4 is used for illustration. For ease of distinction, the different data nodes 104 are respectively denoted as data node CS1, data node CS2, data node CS3, and data node CS4. Data node CS1 is used to store data block A1, data node CS2 is used to store data block A2, data node CS3 is used to store data block A3, and data node CS4 is used to store data block A4. Data blocks A1, A2, and A3 can be original data blocks, and data block A4 can be a check block. If some of the data blocks A1, A2, and A3 are lost or damaged, the data blocks can be reconstructed using other data blocks. For example, if data block A1 is damaged, data block A1 can be reconstructed using data blocks A2, A3, and A4. Specifically, data nodes CS2, CS3, and CS4 can respectively send data block A2, data block A3, and data block A4 to reconstruction node Y. Reconstruction node Y can reconstruct data based on the received data block A2, data block A3, and data block A4 to obtain reconstructed data block A1, and send the reconstructed data block A1 to data node CS1 for storage. In some embodiments, when the minimum number of original data blocks is k and the number of check blocks is m, the total number of data blocks and check blocks required to reconstruct any original data block is at least k. Assuming k is 8 and m is 3, the total number of data blocks and check blocks required to reconstruct any original data block is at least 8. For example, data reconstruction can be performed using 7 original data blocks and 1 check block, or using 6 original data blocks and 2 check blocks.

[0031] Related technologies typically randomly read data blocks from multiple data nodes to perform data reconstruction. The throughput of this process is limited by the slowest data node among the multiple data nodes. In situations such as heterogeneous devices, partial device failures, high device front-end pressure, and large amounts of data to be copied from devices, the transmission speed of some data nodes in the distributed storage system may remain low for extended periods of time. This prevents the distributed storage system's cluster bandwidth from being effectively utilized, resulting in low data reconstruction throughput and poor throughput stability.

[0032] A related technique draws on the front-end Service Level Agreement (SLA) approach, identifying significantly slow nodes and adding them to a read blacklist. Subsequent reads minimize access to these nodes. However, this approach completely blocks slow nodes, rendering them unable to provide any throughput. This approach is unsuitable for back-end data reconstruction and fails to optimize throughput.

[0033] Based on this, the embodiments of the present disclosure provide a data reconstruction method that can not only utilize slower data nodes to provide a certain throughput, but also reduce the impact of slower data nodes on the overall data reconstruction throughput. The method of the embodiments of the present disclosure is applied to a distributed storage system including multiple data nodes 104, each of which stores data blocks for reconstructing original data. Each piece of original data is reconstructed based on the data blocks on one or more data nodes 104. Figure 3 , the method comprising:

[0034] Step S2: for each piece of original data to be reconstructed, determining multiple candidate data nodes where multiple target data blocks for reconstructing the original data are located; the number of candidate data nodes is greater than the minimum number of data nodes 104 where the target data blocks for reconstructing the original data are located;

[0035] Step S4: estimating the current transmission performance of the multiple candidate data nodes based on the historical transmission performance of the multiple candidate data nodes;

[0036] Step S6: Requesting a target data block from at least one target data node among the plurality of candidate data nodes, such that after requesting the target data block from the at least one target data node, a total number of requested data blocks on the plurality of candidate data nodes is positively correlated with current transmission performance of the plurality of candidate data nodes; the number of target data nodes is less than the number of candidate data nodes;

[0037] Step S8: reconstructing the original data based on the target data block returned by at least one target data node.

[0038] In the embodiment of the present disclosure, there is redundancy in the number of candidate nodes, and the target data blocks can be obtained from only some of the target data nodes among the candidate data nodes to reconstruct the original data. Since the current transmission performance of each candidate data node is taken into account in the process of selecting the target data node, after requesting the target data block from the target data node, the total number of requested data blocks on each candidate data node matches the current transmission performance of the corresponding candidate data node. Therefore, the candidate data node with better current transmission performance will receive data requests for a larger number of data blocks, and the candidate data node with poorer current transmission performance will receive data requests for a smaller number of data blocks. In this way, the data transmission capacity of each data node can be fully utilized, reducing the queuing caused by the mismatch between the number of data blocks required to be transmitted by the node and its capacity, improving the cluster bandwidth utilization of the distributed storage system, and improving the data reconstruction throughput and throughput stability.

[0039] For ease of explanation, it is assumed that the minimum number of data nodes where the target data blocks used to reconstruct the original data are located is k, and the total number of candidate data nodes is n.

[0040] The method of the disclosed embodiment can be executed by a reconstruction node. The reconstruction node can be selected from multiple data nodes 104. For example, the source data node (the data node where the data block for reconstructing the original data is located) can be determined as the reconstruction node, or the destination data node (the data node for storing the reconstructed original data) can be determined as the reconstruction node. Any other data node can also be determined as the reconstruction node.

[0041] Each piece of original data can be reconstructed based on data blocks on at least k data nodes 104. Each data node 104 can store one or more data blocks, which may include both original data blocks and check blocks. During data reconstruction, each data node 104 can send at least some of its data blocks to a reconstruction node for data reconstruction. For ease of description, the following example uses the storage of one data block on each data node 104 as an example.

[0042] In step S2, when data is missing, the metadata node 102 may send a data reconstruction task to the reconstruction node. The reconstruction node may receive the data reconstruction task from the metadata node 102. The data reconstruction task may include information about the original data to be reconstructed and information about each data block used to reconstruct the original data. The original data information includes the storage location and length of the original data, and the data block information includes the storage location and length of the data block. The storage location may include the identification information of the data node 104 storing the original data or data block, as well as the starting storage location of the original data or data block on the corresponding data node 104. For each piece of original data requiring data reconstruction, the data reconstruction task may be used to determine the multiple target data blocks required to reconstruct the original data and determine the candidate data nodes where these multiple target data blocks are located. The reconstruction node may send a data request to each target data node among the candidate data nodes, so that the target data node returns the data block in response to the data request. The reconstruction node then reconstructs the original data based on the data blocks returned by each target data node and returns the reconstructed original data to the data node 104 storing the original data.

[0043] The number n of candidate data nodes is greater than k. Therefore, there is redundancy in the candidate data nodes, and the target data blocks can be obtained from only some of the target data nodes among the candidate data nodes to reconstruct the original data. For example, in an erasure code scenario, assuming that the number of original data blocks is k and the number of check blocks is m, each original data block can be a copy of the original data. Assuming that each candidate data node is used to store a data block (original data block or check block), the number n of candidate data nodes storing data blocks is equal to k+m, and the minimum number of data blocks required to reconstruct the original data is n. That is, the number of candidate data nodes is greater than the minimum number of data blocks required to reconstruct the original data.

[0044] In step S4, the current transmission performance of the n candidate data nodes may be estimated based on their historical transmission performance. The historical transmission performance and the current transmission performance may both be represented by transmission bandwidth (referred to as historical transmission bandwidth and current transmission bandwidth, respectively), or by transmission delay (referred to as historical transmission delay and current transmission delay, respectively), or by other parameters that can characterize the data transmission capabilities of the candidate data nodes.

[0045] Taking the example of expressing historical transmission performance by historical transmission delay and current transmission performance by current transmission delay, for any candidate data node, the current transmission delay of the candidate data node can be estimated based on the historical transmission delay of the candidate data node and the total number of data blocks that the candidate data node has not yet completed transmitting. Among them, the historical transmission delay of the candidate data node can be used to represent the average transmission delay of the candidate node for the data blocks on this node during the historical time period. The larger the historical transmission delay, the longer the time required for the candidate data node to transmit the data blocks on this node during the historical time period, thereby indicating that the transmission performance of the candidate data node during the historical time period is poor. Conversely, the smaller the historical transmission delay, the shorter the time required for the candidate data node to transmit the data blocks on this node during the historical time period, thereby indicating that the transmission performance of the candidate data node during the past period is good.

[0046] The current transmission delay of a candidate data node can be calculated by multiplying its historical transmission delay by the total number of data blocks it has yet to transmit. Assuming the historical transmission delay of a candidate data node is D, the time it takes for the candidate data node to transmit a data block during the historical time period is D0. Assuming the total number of data blocks it has yet to transmit is r, the current transmission delay D of the candidate data node can be expressed as D = D0 * r.

[0047] In some embodiments, the data blocks that a candidate data node has not yet completed transmitting include: data blocks requested by the data request currently being processed by the candidate data node, the number of which is denoted as r1; data blocks requested by data requests already received by the candidate data node and in a queued state, the number of which is denoted as r2; and target data blocks stored on the candidate data node, the number of which is denoted as r3. Assuming that the candidate data node sends each data block serially, the total number of data blocks that the candidate data node has not yet completed transmitting is r = r1 + r2 + r3. Therefore, the current transmission delay D of the candidate data node can be expressed as D = D0 * (r1 + r2 + r3). ​​It should be noted that in this step, whether the candidate data node needs to transmit the data blocks stored on the node has not yet been determined. Here, it is assumed that the candidate data node needs to transmit the data blocks stored on the node, and the current transmission delay of the data blocks on the node is estimated based on this assumption. In subsequent steps, the decision on whether the candidate data node needs to transmit the data blocks on the node will be made based on this current transmission delay.

[0048] In some embodiments, a cache queue can be set for each data node 104. Multiple data nodes 104 correspond to multiple cache queues one by one. The cache queue corresponding to each data node 104 is used to cache data requests received by the data node 104. The data requests received by the data node 104 are used to request data blocks stored by the data node for reconstructing the original data. Figure 2 Taking the system shown as an example, a cache queue corresponding to data node CS1, denoted as cache queue Q1, can be set to store data requests for data blocks on data node CS1. A cache queue corresponding to data node CS2, denoted as cache queue Q2, can also be set to store data requests for data blocks on data node CS2. A cache queue corresponding to data node CS3, denoted as cache queue Q3, can also be set to store data requests for data blocks on data node CS3. A cache queue corresponding to data node CS4, denoted as cache queue Q4, can also be set to store data requests for data blocks on data node CS4. After receiving a data request, each data node 104 can add the data request to the cache queue corresponding to the node. Data node 104 can sequentially read each data request cached in the cache queue and, in response to each data request, transmit the data block stored on the node to the reconstruction node. After the data block is successfully transmitted, the corresponding data request is deleted from the cache queue. After the candidate data node has completed transmitting the data block requested by the data request currently being processed by the candidate data node, the candidate data node may continue to use the next data request in the cache queue corresponding to the node as the data request currently being processed by the node.

[0049] The data requests that have been received by the candidate data node and are in a queued state can be cached in the cache queue corresponding to the candidate data node. In this way, based on the number of data requests in the cache queue corresponding to the candidate data node, the number of data blocks requested by the data requests that have been received by the candidate data node and are in a queued state can be determined. Assuming that each data request is used to request a data block, and assuming that the number of data requests in the cache queue is v, the cache queue can include one data request that is currently being processed and v-1 data requests that have been received and are in a queued state. It can be understood that the above is given for the case where the data node 104 transmits one data block each time. If the data node 104 transmits multiple data blocks each time, the number of data requests that are currently being processed included in the cache queue can be greater than 1.

[0050] In some embodiments, a time window of a preset length may be set, and the average transmission delay of a candidate data node for a data block on the node within the time window may be determined as the historical transmission delay of the candidate data node. Specifically, multiple data requests received by the candidate data node within the historical time window may be obtained, each data request being used to request a data block on the candidate data node for reconstructing the original data. Based on the average of the transmission delays of the data blocks requested by the multiple data requests, the average transmission delay of the candidate data node within the historical time window may be determined, and the historical transmission delay of the candidate data node may be determined based on the average transmission delay of the candidate data node within the historical time window.

[0051] Assuming that the number of data requests received within the historical time window is u, and each data request is used to request a data block, the time when the reconstruction node sends the data request can be determined based on the timestamp carried in the data request (denoted as T1), and the time when the candidate data node completes processing the data request (denoted as T2), that is, the time when the candidate data node successfully sends the data block requested by the data request to the reconstruction node, then the transmission delay of the data block requested by the data request can be recorded as T=T2-T1. The data request received by the candidate data node within the historical time window is the data request with a timestamp within the historical time window. The transmission delay T of the data block requested by each data request received by the candidate data node within the historical time window can be calculated in the above manner, and the average value of the transmission delay T of the data block requested by each data request can be calculated. The average value is determined as the average transmission delay of the candidate data node within the historical time window, that is, the historical transmission delay T_x of the candidate data node, denoted as:

[0052]

[0053] Among them, T i is the transmission delay of the data block requested by the i-th data request within the historical time window.

[0054] Furthermore, the average transmission delay of the candidate data node to the data block on the node in multiple time windows can be obtained respectively, and the historical transmission delay of the candidate data node can be determined based on the average transmission delay in the above multiple time windows. For example, the weight corresponding to each time window can be used to perform weighted processing on the average transmission delay of the candidate data node in multiple historical time windows to obtain the historical transmission delay of the candidate data node. Optionally, the weight corresponding to each historical time window can be determined based on the time interval between the historical time window and the current time. The weight corresponding to the historical time window is inversely correlated with the time interval between the historical time window and the current time, that is, the larger the time interval between the historical time window and the current time, the smaller the weight corresponding to the historical time window. In this way, the influence of the historical time window with a closer time interval to the current time on the calculation result can be increased. In other examples, the weights corresponding to each time window can also be equal, or can be calculated using other methods, which are not limited here.

[0055] The above embodiment illustrates the use of transmission delay to characterize transmission performance. In other examples, transmission bandwidth can also be used to represent transmission performance. Transmission bandwidth and transmission delay can be inversely correlated. In some embodiments, the transmission delay can be calculated first, and then converted to transmission bandwidth based on the conversion relationship between transmission delay and transmission bandwidth. The specific conversion method is not limited in this disclosure.

[0056] In step S6, a target data block can be requested from at least one target data node among the n candidate data nodes, the number of target data blocks is less than the number of candidate data nodes, and the number of target data nodes is greater than or equal to the minimum number k of data nodes where the target data blocks for reconstructing the original data are located. For ease of description, the following explanation is given with the number of target data nodes being k. Among them, the current transmission performance of any one of the multiple target data nodes is better than the current transmission performance of other data nodes among the multiple candidate data nodes except the multiple target data nodes. That is to say, this step can select k nodes with better current transmission performance from the multiple candidate data nodes as target nodes to transmit data blocks. When transmission delay is used to characterize transmission performance, the k nodes with better current transmission performance are the k nodes with smaller current transmission delay. When transmission bandwidth is used to represent transmission performance, the k nodes with better current transmission performance are the k nodes with larger current transmission bandwidth. Since the candidate data nodes with better current transmission performance require a shorter time to transmit data blocks, and the candidate data nodes with worse current transmission performance require a longer time to transmit data blocks, allocating the data block transmission task to the candidate data nodes with better current transmission performance can reduce the situation where some data nodes are idle for a long time after completing their own data block transmission tasks and wait for other nodes to complete data block transmission, thereby improving the throughput of data reconstruction.

[0057] Figure 4 A schematic diagram showing the relationship between the cluster bandwidth of an embodiment of the present disclosure and the task volume of each data node 104 is shown. The metadata node 102 can send data reconstruction tasks to the reconstruction node, and the reconstruction node can establish a replica-level queue (i.e., a cache queue corresponding to each data node 104) and perform replica-level task scheduling. Assume that the system includes six data nodes 104, which are respectively denoted as CS1, CS2, CS3, CS4, CS5, and CS6. The real-time bandwidth (i.e., the current transmission bandwidth) of each data node 104 is shown in the figure. Without the reconstruction node performing task scheduling, data blocks are randomly requested from each data node 104. The task volume of each data node 104 is as follows: Figure 4The distribution of tasks before scheduling is shown in the upper right corner of the figure. The height of the rounded rectangle labeled "Task Amount" represents the number of data block transmission tasks required by the corresponding data node 104. It can be seen that data node CS1 has a larger real-time bandwidth, but the amount of tasks assigned to it is relatively small. Data node CS6 has a smaller real-time bandwidth, but the amount of tasks assigned to it is relatively large. This causes data node CS1 to complete its data block transmission task quickly and wait for data node CS6 to transmit its data block. After task scheduling is performed on the reconfigured node, tasks are allocated based on the current transmission bandwidth of each data node 104. It can be seen that the amount of tasks assigned to each data node 104 matches its bandwidth. Data nodes with larger bandwidths (such as data nodes CS1 and CS4) are assigned larger tasks, while data nodes with smaller bandwidths (such as data nodes CS2, CS5, and CS6) are assigned smaller tasks. This fully utilizes the data transmission capacity of each data node, reduces idle waiting times, and improves system throughput.

[0058] In addition, the original data of the embodiment of the present disclosure can be a data block to be reconstructed in the data to be reconstructed, and the data block to be reconstructed can be a data block of a preset size (for example, 1MB). In a scenario where the foreground load changes and the transmission bandwidth changes dynamically between the various data nodes 104, due to the small request with a scheduling level of 1MB, the reconstruction node can quickly perceive the change in node capacity and make timely adjustments during the scheduling process of subsequent requests. For example, assuming that the size of the data to be reconstructed is 10MB, which includes 10 1MB-sized data blocks to be reconstructed, when reconstructing the first data block to be reconstructed, the current transmission bandwidth of data node CS1 is larger, then the data request for the first data block to be reconstructed is sent to data node CS1; when reconstructing the second data block to be reconstructed, the current transmission bandwidth of data node CS1 is smaller, while the current transmission bandwidth of data node CS2 is larger, then the data request for the first data block to be reconstructed is sent to data node CS2, and so on. In this way, the scheduling of data blocks can be adjusted in a rapid response to changes in the transmission bandwidth of each data node 104.

[0059] In step S8, the reconstruction node can reconstruct the original data based on the target data blocks returned by each target data node. Specifically, the reconstruction node can reconstruct the original data after each target data node successfully returns the target data block. Alternatively, the reconstruction node can also reconstruct the original data based on the currently acquired target data block after any target node returns the target data block, obtain an intermediate reconstruction result, and store the intermediate reconstruction result in the memory. After subsequently receiving the target data block, the original data is reconstructed again based on the subsequently received target data block and the intermediate reconstruction result until a complete reconstruction result of the original data is obtained.

[0060] Figure 5 A schematic diagram of the task scheduling process of an embodiment of the present disclosure is shown. The reconstruction node may include a node selection module for selecting a target data node from a plurality of candidate data nodes. The node selection module first performs node performance estimation, mainly estimating the current transmission performance of each candidate data node based on the length of the cache queue corresponding to each candidate data node and the historical transmission delay of each candidate data node. The present disclosure does not limit the specific method of statistically calculating the current transmission performance. In some embodiments, the historical transmission delay of the candidate data node and the transmission delay of the candidate data node in the latest time window can be obtained, and the historical transmission delay of the candidate data node and the transmission delay of the candidate data node in the latest time window are weighted to obtain the current transmission delay of the candidate data node. The historical transmission delay can also be updated to the current transmission delay for the next calculation of the current transmission delay. Then, the target data node is selected according to a pre-set node selection strategy. Several target data nodes with the shortest current transmission delay that can reconstruct the original data can be selected. As Figure 5As shown in the figure, before scheduling, data nodes CS1, CS2, and CS3 were each assigned two data requests. However, due to the poor transmission performance of data node CS1, it took data node CS1 a long time to transmit its data block. As shown in the figure, data block transmission on data node CS1 began at time t1 and was completed at time t4. Data blocks on data nodes CS2 and CS3, on the other hand, began at time t1 and were completed at time t2. Consequently, data nodes CS2 and CS3 were in an idle waiting state between t2 and t4. After scheduling, data nodes CS1 and CS4 were each assigned one data request, while data nodes CS2 and CS3 were each assigned two data requests. This ensures that the workload of each data node matches its transmission capacity as closely as possible. Data blocks on data nodes CS1, CS2, and CS3 were all transmitted at time t2, while data block transmission on data node CS4 was completed between t2 and t1. This reduces the number of data nodes waiting idlely and achieves load balancing during the data reconstruction process. Furthermore, the embodiment of the present disclosure does not need to shield data nodes with poor transmission capabilities, but instead relies on the negative feedback principle to fully utilize the transmission bandwidth of data nodes with poor transmission capabilities, thereby improving the utilization rate of cluster bandwidth.

[0061] An embodiment of the present disclosure further provides a computer device, which includes at least a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the aforementioned embodiments when executing the program.

[0062] Figure 6 1 shows a more specific hardware structure diagram of a computing device provided by an embodiment of the present disclosure. The device may include: a processor 20, a memory 22, an input / output interface 24, a communication interface 26, and a bus 28. The processor 20, the memory 22, the input / output interface 24, and the communication interface 26 are connected to each other within the device via the bus 28.

[0063] The processor 20 can be implemented using a general-purpose central processing unit, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present disclosure. The processor 20 may also include a graphics card, such as an Nvidia Titan X graphics card or an 1080Ti graphics card.

[0064] The memory 22 can be implemented in the form of a read-only memory (ROM), a random access memory (RAM), a static storage device, a dynamic storage device, etc. The memory 22 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present disclosure are implemented through software or firmware, the relevant program codes are stored in the memory 22 and are called and executed by the processor 20.

[0065] The input / output interface 24 is used to connect input / output modules to enable information input and output. The input / output modules can be configured as components within the device (not shown) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0066] The communication interface 26 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0067] The bus 28 comprises a pathway for transmitting information between the various components of the device, such as the processor 20 , the memory 22 , the input / output interface 24 , and the communication interface 26 .

[0068] It should be noted that although the above device only shows the processor 20, memory 22, input / output interface 24, communication interface 26, and bus 28, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the above device may only include the components necessary to implement the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0069] An embodiment of the present disclosure also provides a distributed storage system 10, which includes a metadata node 102 and multiple data nodes 104, wherein the metadata node 102 is used to distribute data reconstruction tasks to a reconstruction node among the multiple data nodes 104, and the reconstruction node is any one of the multiple data nodes 104; the data node 104 is used to execute the method described in any embodiment of the present disclosure in response to receiving the data reconstruction task when the node is a reconstruction node.

[0070] An embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in any embodiment of the present disclosure when executed by a processor.

[0071] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in any of the aforementioned embodiments when the program is executed by a processor.

[0072] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0073] Through the description of the above implementation methods, it can be seen that those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the embodiments of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments of the present disclosure.

[0074] The systems, devices, modules, or units described in the above embodiments may be implemented by a computer device or entity, or by a product having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0075] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and when implementing the embodiment of the present disclosure, the functions of each module can be implemented in the same one or more software and / or hardware. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0076] The above is only a specific implementation of the embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the embodiment of the present disclosure.

Claims

1. A data reconstruction method, applied to a distributed storage system including multiple data nodes, comprising: For each piece of original data to be reconstructed, determining a plurality of candidate data nodes where a plurality of target data blocks for reconstructing the original data are located; The number of candidate data nodes is greater than the minimum number of data nodes where target data blocks for reconstructing the original data are located; estimating the current transmission performance of each of the plurality of candidate data nodes based on historical transmission performance of the plurality of candidate data nodes; Requesting a target data block from at least one target data node among the plurality of candidate data nodes, so that after requesting the target data block from the at least one target data node, a total number of requested data blocks on the plurality of candidate data nodes is positively correlated with current transmission performance of the plurality of candidate data nodes; The number of the target data nodes is less than the number of the candidate data nodes; The original data is reconstructed based on the target data block returned by the at least one target data node.

2. According to the method of claim 1, the current transmission performance of any one of the multiple target data nodes is better than the current transmission performance of any one of the multiple candidate data nodes except the multiple target data nodes.

3. The method according to claim 1, wherein the historical transmission performance is represented based on historical transmission delay, and the current transmission performance is represented based on current transmission delay; and estimating the current transmission performance of the multiple candidate data nodes based on the historical transmission performance of the multiple candidate data nodes comprises: For any candidate data node, the current transmission delay of the candidate data node is estimated based on the historical transmission delay of the candidate data node and the total number of data blocks that have not yet been transmitted by the candidate data node.

4. The method according to claim 3, wherein the data blocks that have not yet been fully transmitted by the candidate data node include: The data block requested by the data request currently being processed by the candidate data node; The candidate data node has received the data block requested by the data request in the queued state; as well as The target data block stored on the candidate data node.

5. The method according to claim 4, wherein the plurality of data nodes correspond one-to-one to a plurality of cache queues, and the cache queue corresponding to each data node is used to cache data requests received by the data node, wherein the data requests are used to request data blocks stored by the data node for reconstructing original data; The data request that has been received by the candidate data node and is in a queued state is cached in the cache queue corresponding to the candidate data node. After the data block requested by the data request currently being processed by the candidate data node is transmitted by the candidate data node, the candidate data node uses the next data request in the corresponding cache queue as the data request currently being processed by the candidate data node.

6. The method according to claim 3, further comprising: Acquire multiple data requests received by the candidate data node within a historical time window, where the data requests are used to request data blocks on the candidate data node for reconstructing original data; Based on the average value of the transmission delays of the data blocks requested by the multiple data requests, the average transmission delay of the candidate data node within the historical time window is determined, and the historical transmission delay of the candidate data node is determined based on the average transmission delay of the candidate data node within the historical time window.

7. The method according to claim 6, wherein determining the historical transmission delay of the candidate data node based on the average transmission delay of the candidate data node within the historical time window comprises: Weighted processing is performed on the average transmission delays of the candidate data node in multiple historical time windows to obtain the historical transmission delays of the candidate data node.

8. The method according to claim 1, wherein the data blocks stored by the plurality of data nodes include data blocks to be reconstructed or check blocks obtained by performing erasure coding on the original data.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A distributed storage system, comprising a metadata node and a plurality of data nodes; wherein: The metadata node is used to distribute data reconstruction tasks to a reconstruction node among the multiple data nodes; the reconstruction node is any one of the multiple data nodes; The data node is configured to, when the node is the reconstruction node, execute the method according to any one of claims 1 to 8 in response to receiving the data reconstruction task.

Images