A data recovery method, system, device and storage medium

Abstract

The present application provides a data recovery method, system, device and storage medium, which relates to the field of distributed storage technology. The method includes: being applied to any data node in a distributed storage system, where the distributed storage system includes a monitoring node and multiple data nodes, and the monitoring node is communicatively connected to each data node. The method includes: receiving a data recovery request sent by the monitoring node; according to the identifiers of the respective target data nodes included in the data recovery request, reading recovery reference data from each target data node, and obtaining the recovery data of the current cycle based on the recovery reference data; determining the recovery data writing rate of the next cycle of the current cycle according to the amount of service data written in the current cycle, the amount of service data written in the previous cycle of the current cycle, and the amount of recovery data written in the current cycle. In this way, data recovery can be performed without affecting service performance, thereby improving the reliability of the distributed storage system.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular, to a data recovery method, system, device, and storage medium. Background Art

[0002] With the increase in Internet traffic, the rapid growth of access volume and data traffic, distributed storage systems have received increasing attention. When a hard disk failure or node failure occurs in a distributed storage system, data recovery needs to be completed as soon as possible to avoid secondary failures that reduce storage reliability and cause data loss.

[0003] However, data recovery has a greater impact on the business performance of distributed storage systems, resulting in abnormal business access and affecting business read and write operations.

[0004] Therefore, how to perform data recovery without affecting business performance and thereby improve the reliability of distributed storage systems is an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a data recovery method, system, device, and storage medium for the deficiencies in the above-mentioned prior art, which can perform data recovery without affecting business performance and thereby improve the reliability of distributed storage systems.

[0006] To achieve the above purpose, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, an embodiment of this application provides a data recovery method, which is applied to any data node in a distributed storage system. The distributed storage system includes a monitoring node and multiple data nodes. The monitoring node is communicatively connected to each data node. The method includes:

[0008] Receiving a data recovery request sent by the monitoring node. The data recovery request includes the identifiers of at least one target data node, and the target data node is a data node associated with a faulty data node;

[0009] Reading recovery reference data from each target data node according to the identifiers of the target data nodes included in the data recovery request, and obtaining recovery data for the current cycle based on the recovery reference data;

[0010] Determining the write rate of recovery data for the next cycle of the current cycle according to the write volume of business data in the current cycle, the write volume of business data in the previous cycle of the current cycle, and the write volume of recovery data in the current cycle, so as to read recovery reference data from each target data node according to the write rate of recovery data for the next cycle in the next cycle.

[0011] Optionally, determining the write rate of recovery data for the next cycle of the current cycle based on the write volume of service data in the current cycle, the write volume of service data in the previous cycle of the current cycle, and the write volume of recovery data in the current cycle includes:

[0012] Determining the change rate of service data corresponding to the current cycle based on the write volume of service data in the current cycle and the write volume of service data in the previous cycle of the current cycle;

[0013] Determining the write rate of recovery data corresponding to the current cycle based on the write volume of recovery data in the current cycle and the cycle time;

[0014] Calculating and obtaining the write rate of recovery data for the next cycle based on the change rate of service data corresponding to the current cycle, the write rate of recovery data, and the corresponding relationship between the preset change rate of service data and weights.

[0015] Optionally, calculating and obtaining the write rate of recovery data for the next cycle based on the change rate of service data corresponding to the current cycle, the write rate of recovery data, and the corresponding relationship between the preset change rate of service data and weights includes:

[0016] Determining the target weight based on the change rate of service data corresponding to the current cycle and the corresponding relationship between the preset change rate of service data and weights;

[0017] Calculating and obtaining the write rate of recovery data for the next cycle based on the change rate of service data corresponding to the current cycle, the write rate of recovery data, and the target weight.

[0018] Optionally, determining the write rate of recovery data for the next cycle based on the change rate of service data corresponding to the current cycle, the write rate of recovery data, and the target weight includes:

[0019] Multiplying the change rate of service data corresponding to the current cycle by the target weight to obtain an adjustment rate multiple;

[0020] Calculating and obtaining the write rate of recovery data for the next cycle based on the adjustment rate multiple and the write rate of recovery data corresponding to the current cycle.

[0021] Optionally, the data node includes a primary storage module and a secondary storage module. Determining the write rate of recovery data for the next cycle of the current cycle based on the write volume of service data in the current cycle, the write volume of service data in the previous cycle of the current cycle, and the write volume of recovery data in the current cycle includes:

[0022] Calculate the initial recovery data writing rate for the next period based on the service data writing volume in the current period, the service data writing volume in the previous period of the current period, and the recovery data writing volume in the current period;

[0023] Determine the forwarding data writing volume written from the first-level storage module to the second-level storage module within the current period;

[0024] Determine the total data storage amount of the first-level storage module corresponding to the current period based on the service data writing volume, the recovery data writing volume, and the forwarding data writing volume in the current period;

[0025] Determine whether to adjust the initial recovery data writing rate for the next period based on the total data storage amount of the first-level storage module corresponding to the current period and the preset storage amount;

[0026] If so, adjust the initial recovery data writing rate for the next period to obtain the recovery data writing rate for the next period.

[0027] Optionally, after determining the recovery data writing rate for the next period of the current period according to the service data writing volume in the current period, the service data writing volume in the previous period of the current period, and the recovery data writing volume in the current period, the method further includes:

[0028] Determine whether the recovery data writing rate for the next period is less than the preset recovery data writing rate;

[0029] If so, read the recovery data from each of the target data nodes according to the recovery data writing rate for the next period in the next period.

[0030] Optionally, after obtaining the recovery data for the current period according to the recovery reference data, the method further includes:

[0031] Write the recovery data for the current period into the first-level storage module;

[0032] After reaching the preset time, read the recovery data from the first-level storage module based on the read / write speed threshold corresponding to the hard disk in the second-level storage module.

[0033] In a second aspect, an embodiment of the present application further provides a data recovery device, which is applied to any data node in a distributed storage system. The distributed storage system includes a monitoring node and multiple data nodes. The monitoring node is communicatively connected to each data node. The device includes:

[0034] A receiving module, configured to receive a data recovery request sent by the monitoring node, where the data recovery request includes identifiers of at least one target data node, and the target data node is a data node associated with a faulty data node;

[0035] A reading module, configured to read recovery reference data from each of the target data nodes according to the identifiers of the target data nodes included in the data recovery request, and obtain recovery data for the current cycle based on the recovery reference data;

[0036] A determining module, configured to determine a write rate of recovery data for the next cycle according to the amount of service data written in the current cycle, the amount of service data written in the previous cycle of the current cycle, and the amount of recovery data written in the current cycle, so as to read recovery reference data from each of the target data nodes according to the write rate of recovery data for the next cycle in the next cycle.

[0037] Optionally, the determining module is specifically configured to determine a change rate of service data corresponding to the current cycle according to the amount of service data written in the current cycle and the amount of service data written in the previous cycle of the current cycle; determine a write rate of recovery data corresponding to the current cycle according to the amount of recovery data written in the current cycle and the cycle time; calculate and obtain the write rate of recovery data for the next cycle according to the change rate of service data corresponding to the current cycle, the write rate of recovery data, and a corresponding relationship between the change rate of service data and weights preset.

[0038] Optionally, the determining module is further specifically configured to determine a target weight according to the change rate of service data corresponding to the current cycle and the corresponding relationship between the change rate of service data and weights preset; calculate and obtain the write rate of recovery data for the next cycle according to the change rate of service data corresponding to the current cycle, the write rate of recovery data, and the target weight.

[0039] Optionally, the determining module is further specifically configured to multiply the change rate of service data corresponding to the current cycle by the target weight to obtain an adjustment rate multiple; calculate and obtain the write rate of recovery data for the next cycle according to the adjustment rate multiple and the write rate of recovery data corresponding to the current cycle.

[0040] Optionally, the data node includes a primary storage module and a secondary storage module;

[0041] Accordingly, the determining module is further specifically configured to calculate the initial recovery data writing rate for the next cycle according to the service data writing volume in the current cycle, the service data writing volume in the previous cycle of the current cycle, and the recovery data writing volume in the current cycle; determine the forwarding data writing volume written from the primary storage module to the secondary storage module within the current cycle; determine the total data storage volume of the primary storage module corresponding to the current cycle according to the service data writing volume, the recovery data writing volume, and the forwarding data writing volume in the current cycle; determine whether to adjust the initial recovery data writing rate for the next cycle according to the total data storage volume of the primary storage module corresponding to the current cycle and a preset storage volume; if so, adjust the initial recovery data writing rate for the next cycle to obtain the recovery data writing rate for the next cycle.

[0042] Optionally, the determining module is further specifically configured to determine whether the recovery data writing rate for the next cycle is less than a preset recovery data writing rate; if so, read recovery data from each of the target data nodes according to the recovery data writing rate for the next cycle in the next cycle.

[0043] Optionally, the reading module is further configured to write the recovery data in the current cycle into the primary storage module; after a preset time, read the recovery data from the primary storage module based on the read / write speed threshold corresponding to the hard disk in the secondary storage module.

[0044] In a third aspect, an embodiment of the present application provides a distributed storage system, which includes a monitoring node and a plurality of data nodes. The monitoring node is communicatively connected to each of the data nodes, and the data nodes are configured to execute the steps of the data recovery method in the first aspect above.

[0045] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the steps of the data recovery method in the first aspect above.

[0046] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the data recovery method in the first aspect above.

[0047] The beneficial effects of the present application are:

[0048] An embodiment of the present application provides a data recovery method, system, device, and storage medium, which are applied to any data node in a distributed storage system. The distributed storage system includes a monitoring node and multiple data nodes. The monitoring node is communicatively connected to each data node. The method includes: receiving a data recovery request sent by the monitoring node, where the data recovery request includes the identifiers of at least one target data node, and the target data node is a data node associated with a faulty data node; according to the identifiers of the target data nodes included in the data recovery request, reading recovery reference data from each target data node, and obtaining the recovery data of the current period based on the recovery reference data; determining the recovery data write rate of the next period of the current period according to the write volume of business data in the current period, the write volume of business data in the previous period of the current period, and the write volume of recovery data in the current period, so as to read the recovery reference data from each target data node according to the recovery data write rate of the next period in the next period.

[0049] By using the data recovery method provided in the embodiment of the present application, the write volume of recovery data in the current period can be statistically obtained based on the read recovery reference data, and the write volume of business data in the current period can also be statistically obtained based on the read business data. Based on the write volume of business data in the current period and the write volume of business data in the previous period of the current period, the change situation of the write speed of the business data in the current period can be determined. Furthermore, based on the change situation of the write speed of the business data in the current period, the recovery data write rate corresponding to the write volume of recovery data in the current period can be adjusted to obtain the recovery data write rate of the next period of the current period, and the recovery reference data is read from each target data node based on the recovery data write rate of the next period. It can be seen that the recovery data write rate of the next period is determined based on the relevant information of the business data, and the influencing factors of the change situation of the write speed of the business data in the current period are integrated. In this way, data recovery can be performed without affecting the business performance, thereby improving the reliability of the distributed storage system. Description of the Drawings

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0051] Figure 1 A schematic diagram of the scenario of a distributed storage system provided by an embodiment of the present application;

[0052] Figure 2 A schematic diagram of the structure of a data node N provided by an embodiment of the present application;

[0053] Figure 3 A flowchart of a data recovery method provided by an embodiment of the present application;

[0054] Figure 4 A flowchart of another data recovery method provided by an embodiment of the present application;

[0055] Figure 5 A flowchart of yet another data recovery method provided by an embodiment of the present application;

[0056] Figure 6 A flowchart of still another data recovery method provided by an embodiment of the present application;

[0057] Figure 7 A structural diagram of a data recovery method provided by an embodiment of the present application;

[0058] Figure 8 A structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Usually, the components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0061] It should be noted that: like reference numerals and letters denote like items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0062] Before explaining the embodiments of the present application in detail, the application scenarios of the present application will be introduced first. The application scenario may specifically be a scenario of recovering faulty data based on a distributed storage system. Figure 1 A scenario diagram of a distributed storage system provided by an embodiment of the present application, as Figure 1 shown, the system may include a monitoring node and a plurality of data nodes, and the plurality of data nodes such as Figure 1The data nodes 1, 2, 3, and N shown, the monitoring node is communicatively connected to the data nodes 1, 2, 3, and N respectively. It should be noted that the number of data nodes is not limited in this application. The data recovery method in the following examples of this application can be applied to any data node in the distributed storage system, such as data node 1, data node 2, data node 3, or data node N. It should be noted that this application does not limit it.

[0063] Based on Figure 1 the distributed storage system described above, the writing process of business data is introduced first. Exemplarily, the client can send the generated business data to the monitoring node. The monitoring node divides the business data into N business data blocks through the client erasure module thereon and generates M check data blocks, and then sends the N business data blocks and the M check data blocks to multiple data nodes respectively, such as Figure 1 the data nodes 1, 2, and 3 shown above. Here, data node 1 is taken as an example for illustration, and other data nodes are similar. Data node 1 first stores the received data blocks (business data blocks and / or check data blocks) in the primary storage module. After reaching the preset time, the secondary storage module reads the data blocks from the primary storage module and writes the data blocks into the corresponding hard disks.

[0064] Based on Figure 1 the recovery process of faulty data is introduced. The monitoring node can monitor the status information of each data node in real time. If the status information indicates that a data node fails, a data recovery request is generated and sent to the data node for data recovery (such as data node N). Continuing with the above example, assuming that data node 1 fails, then data node N can read the data associated with the data blocks in the faulty data node 1 (i.e., the recovery reference data) from data nodes 2 and 3 respectively based on the identifiers of the target data nodes carried in the data recovery request (such as the identifier of data node 2 and the identifier of data node 3). It should be noted that there may be multiple faulty data nodes, and the data recovery request generated by the monitoring node includes the identifiers of the data nodes associated with the multiple faulty data nodes.

[0065] The following examples of this application are all described with data node N as the execution subject. It can be understood that while data node N reads the recovery reference data from the target data node, it can also receive the business data sent by the client. The following briefly describes the process of data recovery for data node N in combination with Figure 2 Other data nodes for data recovery are similar and will not be described here.

[0066] Figure 2The figure is a schematic structural diagram of a data node N provided by an embodiment of the present application. As Figure 2 shown, the data node N includes a local erasure module, a data volume statistics module, a primary storage module, and a secondary storage module. Among them, the local erasure module, the data volume statistics module, the primary storage module, and the secondary storage module are sequentially communicatively connected. The interface module of the data node N is used to send the recovery reference data read from the target data node to the local erasure module. After the local erasure module performs data recovery processing, recovery data is obtained; the data volume statistics module is used to perform data volume statistics on the service data and the recovery data; the primary storage module is a persistent memory (such as Intel Optane memory), and the secondary storage includes the hard disk in the data node. The data node N first stores the received service data and recovery data in the primary storage module. After a preset time, the secondary storage module reads the data block from the primary storage module and writes the data block into the corresponding hard disk.

[0067] It can be understood that the processing module in the data node N can use the data volume statistics module to determine the write rate of the recovery data in the next cycle based on the write volume of the service data in the current cycle, the write volume of the service data in the previous cycle of the current cycle, and the write volume of the recovery data in the current cycle, generate a write control instruction, and send the write control instruction to the interface module. In the next cycle, the interface module reads the recovery reference data from each target data node based on the recovery data write rate included in the write control instruction. It can be understood that the data volume of the recovery reference data is positively correlated with the data volume of the recovery data. When the data volume of the recovery reference data increases, the data volume of the recovery data also increases. Therefore, the present application can perform data recovery without affecting the service performance, determine the data recovery speed, and thus improve the reliability of the distributed storage system.

[0068] The following uses the accompanying drawings to illustrate the data recovery method mentioned in the present application by way of example. Figure 3 The figure is a schematic flow diagram of a data recovery method provided by an embodiment of the present application. This method is applied to any data node in the above-mentioned distributed storage system, such as the data node N. As Figure 3 shown, this method may include:

[0069] S301. Receive a data recovery request sent by a monitoring node.

[0070] Among them, the data recovery request includes the identifiers of at least one target data node, and the target data node is a data node associated with the faulty data node.

[0071] According to the above description, the monitoring node can monitor the status information of each data node in real time. If the status information indicates that a data node fails, a data recovery request is generated and sent to the data node (such as data node N) for data recovery, that is, data node N receives the data recovery request sent by the monitoring node. Combining Figure 1 For illustration, assume that data node 1 fails. Then data node N can obtain a data recovery request carrying, for example, the identifiers of data node 2 and data node 3.

[0072] It should be noted that there may be multiple failed data nodes. Then the data recovery request may include the identifiers of the data nodes associated with multiple failed data nodes.

[0073] S302. According to the identifiers of each target data node included in the data recovery request, read the recovery reference data from each target data node, and obtain the recovery data for the current period based on the recovery reference data.

[0074] Exemplarily, data node N reads the recovery reference data from each target data node based on the identifiers of each target data node. The locally read recovery reference data can be processed for data recovery by the local erasure module to obtain the recovery data. The data volume statistic module can count the data volume of the recovery data obtained within the preset period (T), that is, the recovery data write volume mentioned below, so as to obtain the recovery data write volume for the current period. Similarly, the business data write volume for the current period can also be counted.

[0075] S203. Determine the recovery data write rate for the next period of the current period according to the business data write volume for the current period, the business data write volume for the previous period of the current period, and the recovery data write volume for the current period, so as to read the recovery reference data from each target data node according to the recovery data write rate for the next period in the next period.

[0076] It can be understood that the data volume statistic module in data node N stores the counted data volume in the associated storage module. After data node N obtains the business data write volume and the recovery data write volume for the current period, it can also read the business data write volume for the previous period of the current period from the storage module associated with the data volume statistic module.

[0077] Exemplarily, the processing module in the data node N can obtain the change in the write speed of the service data in the current cycle based on the write volume of the service data in the current cycle and the write volume of the service data in the previous cycle of the current cycle, and can also determine the write rate of the recovery data in the current cycle according to the write volume of the recovery data in the current cycle. The processing module can obtain the write speed (rate) of the recovery data in the next cycle of the current cycle based on the obtained change in the write speed of the service data in the current cycle.

[0078] After obtaining the write speed of the recovery data in the next cycle of the current cycle, a write control instruction can be generated and sent to the interface module of the data node N. The interface module reads the recovery reference data from each target data node in the next cycle based on the write rate of the recovery data included in the write control instruction.

[0079] In summary, in the data recovery method provided by this application, the write volume of the recovery data in the current cycle can be statistically obtained according to the read recovery reference data, and the write volume of the service data in the current cycle can also be statistically obtained according to the read service data. Based on the write volume of the service data in the current cycle and the write volume of the service data in the previous cycle of the current cycle, the change in the write speed of the service data in the current cycle can be determined. Furthermore, based on the change in the write speed of the service data in the current cycle, the write rate of the recovery data in the current cycle corresponding to the write volume of the recovery data in the current cycle can be adjusted to obtain the write rate of the recovery data in the next cycle of the current cycle, and the recovery reference data is read from each target data node based on the write rate of the recovery data in the next cycle. It can be seen that the write rate of the recovery data in the next cycle is determined based on the relevant information of the service data, incorporating the influencing factors of the change in the write speed of the service data in the current cycle. In this way, data recovery can be performed without affecting the service performance, thereby improving the reliability of the distributed storage system.

[0080] Figure 4 It is a schematic flowchart of another data recovery method provided by an embodiment of this application. As Figure 3 shown, optionally, determining the write rate of the recovery data in the next cycle of the current cycle according to the write volume of the service data in the current cycle, the write volume of the service data in the previous cycle of the current cycle, and the write volume of the recovery data in the current cycle includes:

[0081] S401. Determine the service data change rate corresponding to the current cycle according to the write volume of the service data in the current cycle and the write volume of the service data in the previous cycle of the current cycle.

[0082] S402. Determine the write rate of the recovery data corresponding to the current cycle according to the write volume of the recovery data in the current cycle and the cycle time.

[0083] Exemplarily, assume that the current period is T2 and the previous period of the current period is T1. Then, after obtaining the amount of business data written B in the current period T2 write-T2 and the amount of business data written B in the previous period T1 of the current period write-T1 , the change rate ΔB of the business data corresponding to the current period T2 can be determined according to the following formula write-T :

[0084]

[0085] It can be understood that the change rate of business data can be used to represent the change of the business data writing speed. Specifically, the change rate of business data corresponding to the current period can illustrate whether the business data writing speed is rapidly decreasing, slowly decreasing, slowly increasing or rapidly increasing.

[0086] After obtaining the amount of recovery data written B in the current period T2 recover , the recovery data writing rate S corresponding to the current period T2 can be obtained according to recover-T :

[0087]

[0088] S403. Calculate and obtain the recovery data writing rate of the next period according to the change rate of the business data corresponding to the current period, the recovery data writing rate, and the corresponding relationship between the preset change rate of the business data and the weight.

[0089] Exemplarily, the corresponding relationship between the preset change rate ΔB of the business data write-T and the weight w n can be described as follows

[0090]

[0091] It can be seen from the above that when the absolute value of the business change rate is less than 1, w n is equal to w1 (such as 0.1), and when the absolute value of the business change rate is greater than or equal to 1, w n is equal to w2 (such as 0.2), that is, w1 < w2.

[0092] Optionally, the target weight can be determined according to the change rate of the business data corresponding to the current period and the corresponding relationship between the preset change rate of the business data and the weight; the recovery data writing rate of the next period can be calculated and obtained according to the change rate of the business data corresponding to the current period, the recovery data writing rate, and the target weight.

[0093] Exemplarily, in the change rate ΔB of the business data corresponding to the current period T2 write-T and the recovery data writing rate S recover-TAfter that, based on the correspondence between the business data change rate and the weight, and the magnitude relationship between the absolute value of the business data change rate ΔB write-T and 1, the target weight can be determined. Then, based on the target weight and the business data change rate ΔB weite-T corresponding to the current period, the write rate S recover-T of the recovery data corresponding to the current period is adjusted to obtain the write rate of the recovery data for the next period.

[0094] In this way, by using the target weight calculated based on the change situation of business data writing (such as rapid decrease, slow decrease, slow increase, rapid increase) to calculate the write rate of the recovery data for the next period, it is possible to prevent the write rate of the recovery data from experiencing a cliff-like increase or decrease. For example, when the change situation of business data writing is slow decrease, correspondingly, the write rate of the recovery data for the next period is slow increase.

[0095] Figure 5 FIG. is a schematic flowchart of another data recovery method provided by an embodiment of the present application. As Figure 5 shown, optionally, determining the write rate of the recovery data for the next period according to the business data change rate, the write rate of the recovery data, and the target weight corresponding to the current period includes:

[0096] S501. Multiply the business data change rate corresponding to the current period by the target weight to obtain an adjustment rate multiple.

[0097] S502. Calculate and obtain the write rate of the recovery data for the next period according to the adjustment rate multiple and the write rate of the recovery data corresponding to the current period.

[0098] Among them, after obtaining the business data change rate ΔB write-T corresponding to the current period, the write rate S recover-T of the recovery data corresponding to the current period, and the target weight w n , the write rate S recover-TN of the recovery data for the next period can be calculated according to the following formula:

[0099] S recover-TN = (1 - ΔB write-T * w n ) S recover-T

[0100] For example, assume that the business data write volume B write-T1 in the previous period T1 is 6000 MB, the business data write volume B write-T2 in the current period T2 is 1000 MB, and the write rate S recover-T of the recovery data in the current period T2 is 20 MB / s. Then the business data change rate ΔB write-Tis equal to -5, i.e., the target weight w n takes the value of w2 (such as 0.2).

[0101] Based on this, the change rate of service data ΔB corresponding to the current period write-T can be multiplied by the target weight W n to calculate the adjustment rate multiple (ΔB write-T *w n ) which is equal to -1, and then the write rate S of the recovery data for the next period T3 obtained is recover-TN 40 MB / s.

[0102] It can be seen that the change rate of service data ΔB corresponding to the current period T2 write-T is equal to -5, which proves that the amount of service data written is decreasing. Then, the write rate of the recovery data can be increased, that is, the write rate S of the recovery data for the current period T2 recover-T which is 20 MB / s is adjusted to 40 MB / s, and 40 MB / s is used as the write rate of the recovery data for the next period T3 of the current period. In the next period T3, the recovery reference data can be read faster based on the write rate of the recovery data of 40 MB / s. Furthermore, it can enable the data node (such as data node N) to accelerate the recovery of the faulty data on the premise of less service data, that is, the data recovery speed can be increased without affecting the service performance, and then the reliability of the distributed storage system can be improved.

[0103] The following mainly describes the four cases (rapid decrease, slow decrease, slow increase, and rapid increase) of the above-mentioned change rate of service data ΔB write-T .

[0104] Case of rapid decrease: When the amount of service data written B in the current period T2 write-T2 is less than more than one time the amount of service data written B in the previous period T1 of the current period write-T1 , it proves that the service data write speed is rapidly decreasing, that is, ΔB write-T < -1. It can be understood that the write rate of the recovery data needs to be rapidly increased. According to the corresponding relationship between the change rate of service data ΔB write-T and the weight w n , it can be seen that w n is equal to w2. It can be understood that w2 is a relatively large weight. Then, w2 is used as the target weight to calculate the write rate of the recovery data for the next period, and the write rate of the recovery data can be rapidly increased.

[0105] Case of slow decrease: When the amount of service data written B in the current period T2 write-T2 is less than the amount of service data written B in the previous period T1 of the current period write-T1Within one time of, it is proved that the writing speed of service data is slowly decreasing, that is, -1 ≤ ΔB write-T <0, it can be understood that the writing rate of the corresponding data needs to be slowly increased. According to the service data change rate ΔB mentioned above write-T and the weight w n From the corresponding relationship between them, w n is equal to w1. It can be understood that w1 is a relatively small weight. Then, using w1 as the target weight to calculate the writing rate of the restored data in the next cycle can slowly increase the writing rate of the restored data.

[0106] Slowly increasing situation: The amount of service data written B in the current cycle T2 write-T2 is greater than the amount of service data written B in the previous cycle T1 of the current cycle write-T1 Within one time of, it is proved that the writing speed of service data is slowly increasing, that is, 0 ≤ △B write-T <1. It can be understood that the corresponding writing rate of the restored data needs to be slowly decreased. According to the service data change rate △B mentioned above write-T and the weight w n From the corresponding relationship between them, w n is equal to w1. It can be understood that w1 is a relatively small weight. Then, using w1 as the target weight to calculate the writing rate of the restored data in the next cycle can slowly decrease the writing rate of the restored data.

[0107] Quickly increasing situation: The amount of service data written B in the current cycle T2 write-T2 is greater than the amount of service data written B in the previous cycle T1 of the current cycle write-T1 More than one time of, it is proved that the writing speed of service data is quickly increasing, that is, 1 ≤ △B write-T , it can be understood that the corresponding writing rate of the restored data needs to be quickly decreased. According to the service data change rate △B mentioned above write-T and the weight w n From the corresponding relationship between them, w n is equal to w2. It can be understood that w2 is a relatively large weight. Then, using w2 as the target weight to calculate the writing rate of the restored data in the next cycle can quickly decrease the writing rate of the restored data.

[0108] It can be seen that this can prevent the writing rate of the restored data from experiencing a cliff-like increase or decrease. On the basis of not affecting the overall performance of the service, it can accelerate the data recovery speed, improve the reliability of the system, and balance the data recovery speed and the read-write performance of the service.

[0109] Figure 6 The flowchart of another data recovery method provided by the embodiment of the present application. As Figure 6As shown, optionally, determining the write rate of the recovery data for the next cycle of the current cycle based on the write volume of the service data in the current cycle, the write volume of the service data in the previous cycle of the current cycle, and the write volume of the recovery data in the current cycle includes:

[0110] S601. Calculate the initial write rate of the recovery data for the next cycle based on the write volume of the service data in the current cycle, the write volume of the service data in the previous cycle of the current cycle, and the write volume of the recovery data in the current cycle.

[0111] S602. Determine the write volume of the forwarded data written from the primary storage module to the secondary storage module within the current cycle.

[0112] Among them, the change rate of the service data corresponding to the current cycle can be determined based on the write volume of the service data in the current cycle and the write volume of the service data in the previous cycle of the current cycle; the write rate of the recovery data corresponding to the current cycle can be determined based on the write volume of the recovery data in the current cycle and the cycle time; furthermore, the initial write rate of the recovery data for the next cycle can be calculated based on the change rate of the service data corresponding to the current cycle, the write rate of the recovery data, and the corresponding relationship between the change rate of the service data and the weight preset.

[0113] Combined with Figure 2 for illustration, it can be seen from Figure 2 that the data node includes a primary storage module and a secondary storage module. Whether it is service data or recovery data, it is first written into the primary storage module and then into the secondary storage module.

[0114] Exemplarily, the processing module in the data node N can monitor the forwarded data written from the primary storage module to the secondary storage module and count the write volume of the forwarded data written from the primary storage module to the secondary storage module within the current cycle.

[0115] S603. Determine the total data storage volume of the primary storage module corresponding to the current cycle based on the write volume of the service data, the write volume of the recovery data, and the write volume of the forwarded data in the current cycle.

[0116] Among them, after obtaining the write volume B write-T2 of the service data, the write volume B recover of the recovery data, and the write volume B writetostor of the forwarded data in the current cycle T2, the total data storage volume B stor1-T of the primary storage module corresponding to the current cycle T2 can be calculated according to the following formula:

[0117] B stor1-T = B write-T2 + B recover - B writetostor

[0118] S604. Determine whether to adjust the write rate of the initial recovery data for the next cycle according to the total data storage amount and the preset storage amount of the first-level storage module corresponding to the current cycle.

[0119] S605. If so, adjust the write rate of the initial recovery data for the next cycle to obtain the write rate of the recovery data for the next cycle.

[0120] Exemplarily, the total data storage amount B of the first-level storage module corresponding to the current cycle T2 stor1-T is compared with the preset storage amount B stor1-P If B stor1-T is less than or equal to the preset storage amount B stor1-P , then there is no need to adjust the write rate of the initial recovery data for the next cycle, and directly use the write rate of the initial recovery data for the next cycle as the write rate of the recovery data for the next cycle; if B stor1-T is greater than the preset storage amount B stor1-P , it is determined that the write rate of the initial recovery data for the next cycle needs to be adjusted.

[0121] In an implementable embodiment, if it is determined that the write rate of the initial recovery data for the next cycle needs to be adjusted, the write rate of the initial recovery data for the next cycle can be lowered by a preset value, and then the lowered write rate of the initial recovery data for the next cycle is used as the write rate of the recovery data for the next cycle.

[0122] This can avoid the rapid depletion of the storage space of the first-level storage module and affect the processing performance of business data. That is to say, this can more comprehensively ensure that the fault data is recovered without affecting the business performance.

[0123] Optionally, after determining the write rate of the recovery data for the next cycle of the current cycle according to the write amount of the business data in the current cycle, the write amount of the business data in the previous cycle of the current cycle, and the write amount of the recovery data in the current cycle, the method may include: determining whether the write rate of the recovery data for the next cycle is less than the preset write rate of the recovery data; if so, reading the recovery data from each target data node according to the write rate of the recovery data for the next cycle in the next cycle.

[0124] Exemplarily, after obtaining the write rate S of the recovery data for the next cycle recover-TN , S recover-TN can be compared with the preset write rate S of the recovery data recover-P . If S recover-TN is less than S recover-P , then the data node N can read the recovery data according to the write rate S of the recovery data for the next cycle recover-TNGenerate a write control instruction and send the write control instruction to the interface module. In the next cycle, the interface module can base on the recovery data write rate S included in the write control instruction. recover-TN Read the recovery reference data from each target data node.

[0125] This can avoid the overall performance of the distributed storage system being affected by an overly fast recovery data write rate.

[0126] Optionally, after obtaining the recovery data for the current cycle according to the recovery reference data, the method further includes: writing the recovery data for the current cycle to the primary storage module; after reaching a preset time, reading the recovery data from the primary storage module based on the read / write speed threshold corresponding to the hard disk in the secondary storage module.

[0127] According to the above description, after the local erasure module in the data node performs data recovery, the recovered data can be first written to the primary storage module. It can be understood that the secondary storage module includes multiple hard disks, and each hard disk is pre-set with a read / write speed threshold. After reaching the preset time, the data node can determine the target read speed for the secondary storage module to read the recovered data from the primary storage module according to the read / write speed threshold corresponding to the hard disk, and the target read speed is less than the read / write speed threshold.

[0128] This can avoid a single hard disk having too high a data read / write volume, which affects the overall performance of the distributed storage system.

[0129] Figure 7 It is a schematic structural diagram of a data recovery method provided by an embodiment of the present application, which is applied to any data node in a distributed storage system. The distributed storage system includes a monitoring node and multiple said data nodes, and the monitoring node is communicatively connected to each data node. As Figure 7 shown, the device includes:

[0130] A receiving module 701, configured to receive a data recovery request sent by the monitoring node, where the data recovery request includes the identifiers of at least one target data node, and the target data node is a data node associated with a faulty data node;

[0131] A reading module 702, configured to read recovery reference data from each target data node according to the identifiers of the target data nodes included in the data recovery request, and obtain recovery data for the current cycle according to the recovery reference data;

[0132] A determination module 703, configured to determine a write rate of recovery data for a next cycle according to a write volume of service data in the current cycle, a write volume of service data in the previous cycle of the current cycle, and a write volume of recovery data in the current cycle, so as to read recovery reference data from each target data node according to the write rate of recovery data in the next cycle in the next cycle.

[0133] Optionally, the determination module 703 is specifically configured to determine a change rate of service data corresponding to the current cycle according to the write volume of service data in the current cycle and the write volume of service data in the previous cycle of the current cycle; determine a write rate of recovery data corresponding to the current cycle according to the write volume of recovery data in the current cycle and the cycle time; calculate and obtain a write rate of recovery data for the next cycle according to the change rate of service data corresponding to the current cycle, the write rate of recovery data, and a corresponding relationship between the change rate of service data and weights preset.

[0134] Optionally, the determination module 703 is further specifically configured to determine a target weight according to the change rate of service data corresponding to the current cycle and the corresponding relationship between the change rate of service data and weights preset; calculate and obtain a write rate of recovery data for the next cycle according to the change rate of service data corresponding to the current cycle, the write rate of recovery data, and the target weight.

[0135] Optionally, the determination module 703 is further specifically configured to multiply the change rate of service data corresponding to the current cycle by the target weight to obtain an adjustment rate multiple; calculate and obtain a write rate of recovery data for the next cycle according to the adjustment rate multiple and the write rate of recovery data corresponding to the current cycle.

[0136] Optionally, the data node includes a primary storage module and a secondary storage module;

[0137] Correspondingly, the determination module 703 is further specifically configured to calculate and obtain an initial write rate of recovery data for the next cycle according to the write volume of service data in the current cycle, the write volume of service data in the previous cycle of the current cycle, and the write volume of recovery data in the current cycle; determine a write volume of forwarding data written from the primary storage module to the secondary storage module within the current cycle; determine a total data storage volume of the primary storage module corresponding to the current cycle according to the write volume of service data, the write volume of recovery data, and the write volume of forwarding data in the current cycle; determine whether to adjust the initial write rate of recovery data for the next cycle according to the total data storage volume of the primary storage module corresponding to the current cycle and a preset storage volume; if so, adjust the initial write rate of recovery data for the next cycle to obtain a write rate of recovery data for the next cycle.

[0138] Optionally, the determining module 703 is further specifically configured to determine whether the recovery data writing rate in the next cycle is less than a preset recovery data writing rate; if so, read the recovery data from each target data node according to the recovery data writing rate in the next cycle in the next cycle.

[0139] Optionally, the reading module 702 is further configured to write the recovery data in the current cycle into the primary storage module; after reaching a preset time, read the recovery data from the primary storage module based on the read / write speed threshold corresponding to the hard disk in the secondary storage module.

[0140] The above device is used to execute the method provided in the foregoing embodiment, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0141] The above modules may be one or more integrated circuits configured to implement the above method, for example: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), etc. Again, when the above certain module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. Again, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0142] Figure 8 The following is a schematic structural diagram of an electronic device provided in an embodiment of the present application, as Figure 8 shown, the electronic device may include: a processor 801, a storage medium 802, and a bus 803. The storage medium 802 stores machine-readable instructions executable by the processor 801. When the electronic device runs, the processor 801 communicates with the storage medium 802 through the bus 803, and the processor 801 executes the machine-readable instructions to execute the steps of the above method embodiment. The specific implementation manner and technical effects are similar, and will not be elaborated here.

[0143] Optionally, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the above method embodiment.

[0144] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0147] The above integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The above software functional units stored in a storage medium include several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute some steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (English: Read-Only Memory, abbreviated as: ROM), random access memories (English: Random Access Memory, abbreviated as: RAM), magnetic disks or optical discs that can store program codes.

[0148] It should be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0149] The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application. It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. The above are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A data recovery method, characterized in that, Applied to any data node in a distributed storage system, the distributed storage system includes a monitoring node and multiple such data nodes, and the monitoring node is communicatively connected to each data node. The method includes: Receiving a data recovery request sent by the monitoring node, where the data recovery request includes the identifiers of at least one target data node, and the target data node is a data node associated with a faulty data node; According to the identifiers of the target data nodes included in the data recovery request, reading recovery reference data from each of the target data nodes, and obtaining the recovery data for the current cycle based on the recovery reference data; Determining the recovery data write rate for the next cycle of the current cycle according to the amount of business data written in the current cycle, the amount of business data written in the previous cycle of the current cycle, and the amount of recovery data written in the current cycle, so as to read the recovery reference data from each of the target data nodes according to the recovery data write rate for the next cycle in the next cycle; The determining the recovery data write rate for the next cycle of the current cycle according to the amount of business data written in the current cycle, the amount of business data written in the previous cycle of the current cycle, and the amount of recovery data written in the current cycle includes: Determining the business data change rate corresponding to the current cycle according to the amount of business data written in the current cycle and the amount of business data written in the previous cycle of the current cycle; Determining the recovery data write rate corresponding to the current cycle according to the amount of recovery data written in the current cycle and the cycle time; Calculating and obtaining the recovery data write rate for the next cycle according to the business data change rate corresponding to the current cycle, the recovery data write rate, and the corresponding relationship between the preset business data change rate and the weight; 2. The method according to claim 1, wherein The calculating and obtaining the recovery data write rate for the next cycle according to the business data change rate corresponding to the current cycle, the recovery data write rate, and the corresponding relationship between the preset business data change rate and the weight includes: Determining the target weight according to the business data change rate corresponding to the current cycle and the corresponding relationship between the preset business data change rate and the weight; Calculating and obtaining the recovery data write rate for the next cycle according to the business data change rate corresponding to the current cycle, the recovery data write rate, and the target weight; 3. The method according to claim 2, wherein The determining the recovery data write rate for the next cycle according to the business data change rate corresponding to the current cycle, the recovery data write rate, and the target weight includes: Multiplying the business data change rate corresponding to the current cycle by the target weight to obtain an adjustment rate multiple; Calculating and obtaining the recovery data write rate for the next cycle according to the adjustment rate multiple and the recovery data write rate corresponding to the current cycle.

4. The method according to any one of claims 1-3, characterized in that, The data node includes a primary storage module and a secondary storage module. Determining the recovery data writing rate for the next cycle of the current cycle based on the business data writing volume of the current cycle, the business data writing volume of the previous cycle of the current cycle, and the recovery data writing volume of the current cycle includes: Calculating an initial recovery data writing rate for the next cycle based on the business data writing volume of the current cycle, the business data writing volume of the previous cycle of the current cycle, and the recovery data writing volume of the current cycle; Determining the forwarding data writing volume written from the primary storage module to the secondary storage module within the current cycle; Determining the total data storage amount of the primary storage module corresponding to the current cycle based on the business data writing volume of the current cycle, the recovery data writing volume, and the forwarding data writing volume; Determining whether to adjust the initial recovery data writing rate for the next cycle based on the total data storage amount of the primary storage module corresponding to the current cycle and a preset storage amount; If so, adjusting the initial recovery data writing rate for the next cycle to obtain the recovery data writing rate for the next cycle.

5. The method according to any one of claims 1 to 3, characterized in that After determining the recovery data writing rate for the next cycle of the current cycle based on the business data writing volume of the current cycle, the business data writing volume of the previous cycle of the current cycle, and the recovery data writing volume of the current cycle, the method further includes: Determining whether the recovery data writing rate for the next cycle is less than a preset recovery data writing rate; If so, reading recovery data from each of the target data nodes according to the recovery data writing rate for the next cycle in the next cycle.

6. The method according to claim 4, wherein After obtaining the recovery data for the current cycle according to the recovery reference data, the method further includes: Writing the recovery data for the current cycle into the primary storage module; After a preset time, reading the recovery data from the primary storage module based on the read-write speed threshold corresponding to the hard disk in the secondary storage module.

7. A distributed storage system, characterized in that, The distributed storage system includes a monitoring node and a plurality of data nodes. The monitoring node is communicatively connected to each of the data nodes, and the data nodes are used to execute the steps of the data recovery method according to any one of claims 1-6.

8. An electronic device, characterized in that, Including: A processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the steps of the data recovery method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it executes the steps of the data recovery method according to any one of claims 1-6.

Images