Data reconstruction method and computer readable medium

Abstract

The invention provides a data reconstruction method and a computer readable medium. The data reconstruction method comprises the following steps: storing data blocks in virtual disks to which nodes belong into physical disks to which different nodes belong in a redundant storage mode; when the physical disk breaks down, executing data reconstruction operation on the broken-down physical disk by the broken-down node; when the data reconstruction operation is executed, obtaining an I / O load corresponding to the faulted node, and when the I / O load exceeds a preset threshold value, migrating the reconstruction management module and the I / O management module to which the faulted node belongs to any one of the non-faulted nodes; and executing data reconstruction operation on the failed physical disk by the migrated node. According to the invention, the reconstruction request and the I / O request are ensured not to be interrupted, and the purpose of optimizing the whole performance of the system is achieved.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a data reconstruction method and a computer-readable medium. Background Art

[0002] In distributed storage systems, data is typically stored using replication and erasure coding. Specifically, a segment of data is split into multiple data shards. In the replication method, the data in each data shard is identical, and the multiple data shards are stored on different nodes. In the erasure coding method, multiple parity shards are calculated based on the multiple data shards, and the multiple data shards and parity shards are stored on different nodes. A failure at any node will only result in the loss of one data shard. If any data shard is lost, data reconstruction can be performed using the data shards and parity shards stored on other nodes, thereby recovering the lost data shard on the failed node.

[0003] Data reconstruction usually requires first selecting a target node to run the data reconstruction module that reconstructs the data shards and stores the reconstructed data shards. To obtain other data shards and verification shards corresponding to the lost data shards, the replica method directly stores the data shards in the target node, and the erasure code method uses the data shards and verification shards to calculate the lost data shards and then store them in the target node. In addition, when the application system reads and writes data to the distributed storage system, when it involves data in the same address space, it is also necessary to first select a target node to run the I / O management module. The replica method is for it to determine whether multiple copies are written to different nodes, and the erasure code method is for it to calculate verification shards based on multiple data shards and write them to different nodes.

[0004] In the technical scenario of a computer system or cluster based on a hyper-converged infrastructure (HCI), a distributed storage system is typically composed of storage devices on each node, with the application system running in a virtual machine within each node. To optimize the read and write functions of the application system, a node with a high number of write I / Os is typically selected to run the I / O management module. When data reconstruction occurs, if the reconstruction management module and the I / O management module select the same node, the storage I / O load pressure on that node will increase exponentially. If the reconstruction management module and the I / O management module select different nodes, when accessing the same address space, in order to ensure data read and write consistency, the reconstruction management module and the I / O management module will require some cumbersome data interaction overhead such as locking operations.

[0005] At present, most distributed storage systems use a reconstruction management module and an I / O management module running on the same node. However, there is a drawback in that data reconstruction in distributed storage systems and hyper-converged all-in-one machines based on a hyper-converged architecture takes a long time. The I / O load of data reconstruction is mainly initiated by the reconstruction management module, which controls the number of I / O requests. During data reconstruction, the I / O load of the application system is related to the way the application system is used, and can change at any time, and may even fluctuate greatly. In order to control the IO load of each node in the entire hyper-converged system, the number of reconstruction I / O requests is usually controlled by the reconstruction management module to avoid the total I / O load on a certain node being too large. However, for a distributed system with multiple nodes, the IO load of some nodes may be large, and the reconstruction IO requests need to be reduced, while the IO load of other nodes is small, but because the reconstruction management module is not deployed on this node, more data reconstruction IO requests cannot be initiated. Therefore, the distributed storage system in the prior art and the hyper-converged system based on the distributed storage system have the drawback of uneven I / O load distribution.

[0006] In view of this, it is necessary to improve the data reconstruction method of the hyper-converged system based on the distributed storage system in the prior art to solve the above problems. Summary of the Invention

[0007] The purpose of the present invention is to disclose a data reconstruction method and system for realizing dynamic scheduling on multiple nodes during the reconstruction period, so that the IO load distribution is more balanced, thereby optimizing the response performance of the distributed storage system and the hyper-converged system based on the distributed storage system.

[0008] To achieve one of the above objectives, the present invention provides a data reconstruction method, which runs in a system composed of multiple nodes, and includes:

[0009] The data blocks in the virtual disk of the node are stored in the physical disk of different nodes in a redundant storage manner;

[0010] When a physical disk belonging to a node fails, the reconstruction management module and the I / O management module belonging to the failed node perform a data reconstruction operation on the failed physical disk based on the data blocks in the virtual disk belonging to the failed node stored on physical disks belonging to different nodes;

[0011] When performing a data reconstruction operation, the I / O load corresponding to the failed node is obtained, and when the I / O load exceeds a preset threshold, the reconstruction management module and the I / O management module belonging to the failed node are migrated to any surviving node;

[0012] The reconstruction management module and the I / O management module of the migrated node perform data reconstruction on the failed physical disk based on data blocks in the virtual disk of the failed node stored in physical disks of different nodes.

[0013] As a further improvement of the present invention, a virtual machine and a distributed storage system are run on a single node, a virtual disk is deployed in the virtual machine, and a reconstruction management module, an I / O management module and a physical disk corresponding to the virtual disk are deployed in the distributed storage system.

[0014] As a further improvement of the present invention, the performing of data reconstruction operation on the failed physical disk includes:

[0015] A target physical disk is determined from the failed node, and the reconstruction management module sends a reconstruction request to the I / O management module. The I / O management module sends an I / O request to the physical disks belonging to different nodes based on the reconstruction request. The physical disks belonging to different nodes respond to the I / O request and write the data blocks to the target physical disk.

[0016] As a further improvement of the present invention, the step of migrating the reconstruction management module and the I / O management module of the failed node to any surviving node includes:

[0017] Select a node that has not failed as the target node;

[0018] The reconstruction management module to which the faulty node belongs suspends sending new reconstruction requests, and the I / O management module to which the faulty node belongs suspends sending new I / O requests;

[0019] When the reconstruction request sent before the pause and the I / O request sent before the pause are both executed, the pause position of the current data reconstruction operation is recorded;

[0020] The reconstruction management module and the I / O management module of the failed node are shut down, the reconstruction management module and the I / O management module of the target node are started, and a data reconstruction operation is performed according to the pause position.

[0021] As a further improvement of the present invention, selecting a node that has not experienced a fault as the target node includes:

[0022] The I / O load corresponding to the node that has not failed is obtained, and the target node is determined according to the I / O load.

[0023] As a further improvement of the present invention, the step of migrating the reconstruction management module and the I / O management module of the failed node to any surviving node further includes:

[0024] If the reconstruction request and / or I / O request sent before the pause are not completed within the specified time, a migration failure message is returned, and the reconstruction management module and I / O management module of the failed node are restarted.

[0025] As a further improvement of the present invention, a node service module is deployed in the distributed storage system to record the I / O requests corresponding to the nodes in real time.

[0026] As a further improvement of the present invention, the redundant storage method is a mirror storage method, which copies the data blocks in the virtual disk belonging to the node to form multiple mirror blocks, and stores the mirror blocks in the physical disks belonging to different nodes respectively.

[0027] As a further improvement of the present invention, the redundant storage method is an erasure code storage method, which divides the data blocks in the virtual disk belonging to the node into multiple data slices, and calculates the verification slices based on the data slices, and stores the data slices and the verification slices respectively on the physical disks belonging to different nodes.

[0028] Based on the same inventive concept, the present invention also discloses a computer-readable medium, comprising:

[0029] The computer-readable medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps of the data reconstruction method described in any of the above inventions are executed.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows: in the present invention, data blocks in a virtual disk belonging to a node are stored in a redundant storage manner to physical disks belonging to different nodes; when a physical disk belonging to a node fails, the reconstruction management module and the I / O management module belonging to the failed node perform a data reconstruction operation on the failed physical disk based on the data blocks in the virtual disk belonging to the failed node stored on physical disks belonging to different nodes; when performing the data reconstruction operation, the I / O load corresponding to the failed node is obtained, and when the I / O load exceeds a preset threshold, the reconstruction management module and the I / O management module belonging to the failed node are migrated to any node that has not failed; the reconstruction management module and the I / O management module belonging to the migrated node perform a data reconstruction operation on the failed physical disk based on the data blocks in the virtual disk belonging to the failed node stored on physical disks belonging to different nodes. Based on this, the reconstruction management module and I / O management module of the migrated node replace the reconstruction management module and I / O management module of the failed node. Based on the I / O load of each node, the reconstruction management module and I / O management module are dynamically adjusted to the appropriate node, which not only ensures the uninterrupted operation of reconstruction requests and I / O requests, but also achieves the goal of optimizing the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of the steps of a data reconstruction method of the present invention;

[0032] Figure 2 A schematic diagram of the specific steps for migrating the reconstruction management module and I / O management module of a failed node to any surviving node;

[0033] Figure 3 It is a topological diagram of a computer system consisting of three nodes;

[0034] Figure 4 Schematic diagram of I / O requests from the perspective of virtual disk 111 of node 10 in an example of a computer system composed of three nodes;

[0035] Figure 5 Schematic diagram of I / O requests from the perspective of virtual disk 211 of node 20 in an example of a computer system composed of three nodes;

[0036] Figure 6 Schematic diagram of I / O requests from the perspective of a virtual disk 311 of a node 30 in an example of a computer system composed of three nodes;

[0037] Figure 7 A schematic diagram of I / O requests in an example of a computer system after migration;

[0038] Figure 8 A topological diagram of a computer-readable medium according to the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below with reference to the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in this field based on these embodiments are all within the scope of protection of the present invention.

[0040] Please refer to Figures 1 to 7 As shown, the present invention shows a specific implementation of a data reconstruction method, which runs in a system composed of multiple nodes and is used to perform data reconstruction operations on the failed physical disk based on the data blocks in the virtual disk of the failed node stored in the physical disks of different nodes when the physical disk belonging to the node fails. The system can be, for example, a computer system or cluster based on a hyper-converged architecture, and can also run in a cloud computing platform that deploys multiple computer systems. The cloud platform can be understood as a service or system formed by a hyper-converged all-in-one machine, a computer, a server, a data center or a portable terminal through virtualization technology. The applicant uses the computer system 100 as an example to provide an exemplary explanation below, but this does not limit the scope of protection of the present invention.

[0041] Ginseng Figure 1 As shown, the data reconstruction method includes the following steps S1 to S4.

[0042] Step S1: Data blocks in a virtual disk belonging to a node are stored in a redundant storage manner on physical disks belonging to different nodes.

[0043] Specifically, a system (e.g., computer system 100) is composed of multiple nodes, with multiple virtual machines and distributed storage systems running on a single node. Applications are running within the virtual machines, and virtual disks are deployed within the virtual machines. The virtual disks deployed within the virtual machines are provided by the distributed storage system, and a reconstruction management module, an I / O management module, and physical disks corresponding to the virtual disks are deployed within the distributed storage system. Data blocks within the virtual disk belonging to a single node are stored redundantly on physical disks belonging to different nodes in the system (including the physical disk belonging to the node itself and the physical disks belonging to the remaining nodes in the system excluding the node).

[0044] For example, Figure 3As shown, computer system 100 includes three nodes: node 10, node 20, and node 30. Node 10 deploys a virtual machine 11 and a distributed storage system 12. Virtual machine 11 deploys a virtual disk 111. Distributed storage system 12 deploys a reconstruction management module 122, an I / O management module 121, and physical disks 123 and 124 corresponding to virtual disk 111. Node 20 deploys a virtual machine 21 and a distributed storage system 22. Virtual machine 21 deploys a virtual disk 211. Distributed storage system 22 deploys a reconstruction management module 222, an I / O management module 221, and physical disks 223 and 224 corresponding to virtual disk 211. Node 30 deploys a virtual machine 31 and a distributed storage system 32. Virtual machine 31 deploys a virtual disk 311. Distributed storage system 32 deploys a reconstruction management module 322, an I / O management module 321, and physical disks 323 and 324 corresponding to virtual disk 311. In particular, the reconstruction management module 122 and the I / O management module 121 of the virtual disk 111 are deployed on the same node (i.e., node 10), the reconstruction management module 222 and the I / O management module 221 of the virtual disk 211 are deployed on the same node (i.e., node 20), and the reconstruction management module 322 and the I / O management module 321 of the virtual disk 311 are deployed on the same node (i.e., node 30).

[0045] The redundant storage method for data blocks in a virtual disk belonging to a single node may be, for example, a mirror storage method or an erasure code storage method, which is not specifically limited in this embodiment. In one embodiment, the redundant storage method is a mirror storage method, in which the data blocks in the virtual disk belonging to the node are copied to form multiple mirror blocks, and the multiple mirror blocks are respectively stored in physical disks belonging to different nodes. In another embodiment, the redundant storage method is an erasure code storage method, in which the data blocks in the virtual disk belonging to the node are divided into multiple data shards, and check blocks are calculated based on the data shards, and the multiple data shards and check blocks are respectively stored in physical disks belonging to different nodes.

[0046] More specifically, in the following description, this embodiment is exemplified by taking the redundant storage method as the mirror storage method as an example, and for the specific example of the redundant storage method as the erasure code storage method, reference can be made to the mirror storage method, and this embodiment will not be repeated here.

[0047] Ginseng Figure 3As shown, in this embodiment, the mirror distribution method of virtual disk 111 is the physical disk belonging to node 10 (i.e., physical disk 123 and physical disk 124) and the physical disk belonging to node 20 (i.e., physical disk 223 and physical disk 224), the mirror distribution method of virtual disk 211 is the physical disk belonging to node 20 (i.e., physical disk 223 and physical disk 224) and the physical disk belonging to node 30 (i.e., physical disk 323 and physical disk 324), and the mirror distribution method of virtual disk 311 is the physical disk belonging to node 10 (i.e., physical disk 123 and physical disk 124) and the physical disk belonging to node 30 (i.e., physical disk 323 and physical disk 324).

[0048] Step S2: When a physical disk belonging to a node fails, the reconstruction management module and the I / O management module of the failed node perform data reconstruction operations on the failed physical disk based on the data blocks in the virtual disk belonging to the failed node stored in the physical disks of different nodes.

[0049] Specifically, when a physical disk belonging to a node fails, it is determined that the data blocks in the virtual disk belonging to the failed node are stored on physical disks belonging to different nodes using redundant storage methods, and the reconstruction management module and I / O management module belonging to the failed node perform data reconstruction operations on the failed physical disk based on the determined physical disks belonging to different nodes.

[0050] For example, if the physical disk 123 belonging to node 10 fails, it is determined that the data blocks within the virtual disk 111 belonging to node 10 are stored in a redundant storage manner on the physical disk belonging to node 20, and the reconstruction management module 122 and the I / O management module 121 of node 10 perform data reconstruction operations on the physical disk 123 belonging to node 10 based on the physical disk belonging to node 20.

[0051] In one embodiment, a reconstruction management module and an I / O management module belonging to a failed node perform a data reconstruction operation on a failed physical disk, including: determining a target physical disk from the failed node; the reconstruction management module belonging to the failed node sends a reconstruction request to the I / O management module; the I / O management module sends an I / O request to a physical disk belonging to a different node based on the reconstruction request; the physical disk belonging to the different node responds to the I / O request and writes the data block to the target physical disk. The I / O request includes a read request and a write request.

[0052] For example, Figure 3As shown, if the physical disk 123 belonging to the node 10 fails, another physical disk 123' in the node 10 is determined as the target physical disk, the reconstruction management module 122 belonging to the node 10 sends a reconstruction request to the I / O management module 121, and the I / O management module 121 sends a read request to the physical disk belonging to the node 20 based on the reconstruction request. The physical disk belonging to the node 20 (i.e., physical disk 223 and / or physical disk 224) responds to the read request and creates a write request to write the data block in the physical disk belonging to the node 20 to the target physical disk (i.e., physical disk 123'), thereby realizing the data reconstruction operation on the failed physical disk 123 by the reconstruction management module 122 and the I / O management module 121 belonging to the failed node 10.

[0053] When the physical disk 123 belonging to node 10 fails, and the reconstruction management module 122 and the I / O management module 121 belonging to node 10 perform a data reconstruction operation on the physical disk 123 belonging to node 10 based on the physical disk belonging to node 20, the number of I / O requests corresponding to nodes 10, 20 and 30 respectively.

[0054] Because one data block of virtual disk 111 corresponds to two mirror blocks, when a write request from virtual disk 111 is sent to I / O management module 121, I / O management module 121 clones two new write requests based on the mirror storage method and writes them to the physical disk belonging to node 10 and the physical disk belonging to node 20, respectively. When a read request from virtual disk 111 is sent to I / O management module 121, I / O management module 121 only needs to select one of the two mirror blocks to read the data. Therefore, for I / O management module 121, each write request from virtual disk 111 generates two write requests, and each read request from virtual disk 111 generates one read request. Virtual disk 211 and virtual disk 311 are understood equivalently, that is, for one write request from virtual disk 211, the I / O management module 221 generates two write requests, and for one read request from virtual disk 211, the I / O management module 221 generates one read request; for one write request from virtual disk 311, the I / O management module 321 generates two write requests, and for one read request from virtual disk 311, the I / O management module 321 generates one read request. This embodiment will not be repeated here.

[0055] Since physical disk 123 belonging to node 10 has failed, physical disk 123' is the designated target physical disk, and physical disk 123' does not store any data at this time, physical disk 123' can be considered invalid. However, physical disk 124 has not failed and physical disk 124 stores data at this time, so physical disk 124 can be considered valid. Similarly, physical disk 223 and physical disk 224 belonging to node 20 are both valid. When the reconstruction request sent by the reconstruction management module 122 belonging to node 10 performs a data reconstruction operation on the failed physical disk 123, it is necessary to create a read request to read the mirror block from the physical disk of node 20, and then create a write request to write the read data block to physical disk 123' of node 10. Therefore, for the reconstruction management module 122, the number of read requests and the number of write requests are the same.

[0056] For example, the physical disk 123 of the node 10 fails. Figure 4 As shown, if the I / O management module 121 of virtual disk 111 receives 1000 write requests and 1000 read requests, node 10 receives 1000 write requests and 1000 read requests, and node 20 receives 1000 write requests. If the reconstruction management module 122 of virtual disk 111 sends 2000 read requests to node 20, node 10 accordingly receives 2000 write requests. Therefore, from the perspective of virtual disk 111, node 10 receives 3000 write requests and 1000 read requests, node 20 receives 1000 write requests and 2000 read requests, and node 30 receives 0 write requests and 0 read requests.

[0057] Ginseng Figure 5 As shown, if the I / O management module 221 of virtual disk 211 receives 1000 write requests and 1000 read requests, node 20 receives 1000 write requests and 1000 read requests, and node 30 receives 1000 write requests. At this point, the reconstruction management module 222 of virtual disk 211 does not need to perform any operation. Therefore, from the perspective of virtual disk 211, node 10 receives 0 write requests and 0 read requests, node 20 receives 1000 write requests and 1000 read requests, and node 30 receives 1000 write requests and 0 read requests.

[0058] Ginseng Figure 6As shown, if the I / O management module 321 of virtual disk 311 receives 1000 write requests and 1000 read requests, node 30 receives 1000 write requests and 1000 read requests, and node 10 receives 1000 write requests. If the reconstruction management module 322 of virtual disk 311 initiates 2000 read requests to node 30, node 10 accordingly receives 2000 write requests. Therefore, from the perspective of virtual disk 311, node 10 receives 3000 write requests and 0 read requests, node 20 receives 0 write requests and 0 read requests, and node 30 receives 1000 write requests and 3000 read requests.

[0059] In summary, from the perspective of the entire computer system 100, that is, Figure 4 Medium virtual disk 111, Figure 5 Virtual disk 211 and Figure 6 From the perspective of virtual disk 311, the sum of the I / O request numbers of the three nodes is calculated. Node 10 receives 6,000 write requests and 1,000 read requests, node 20 receives 2,000 write requests and 3,000 read requests, and node 30 receives 2,000 write requests and 3,000 read requests.

[0060] It should be noted that because the reconstruction management module 122 of node 10 needs to read data from other nodes and write it to the target physical disk, although read requests are distributed to other nodes in the system, write requests are all concentrated on node 10, resulting in a sharp increase in the number of I / O requests for node 10. Excessive I / O requests will occupy more CPU and memory resources, causing virtual machine 11 running on node 10 to face resource shortages. At the same time, excessive I / O requests will also overload the underlying storage devices, rapidly increasing request response time and causing read and write request timeouts for virtual disk 111.

[0061] Step S3: When performing data reconstruction operations, obtain the I / O load corresponding to the failed node, and when the I / O load exceeds a preset threshold, migrate the reconstruction management module and I / O management module belonging to the failed node to any surviving node.

[0062] For example, Figure 7 As shown, if the physical disk 123 belonging to the node 10 fails, the node 10 is the failed node. Therefore, when the reconstruction management module 122 and the I / O management module 121 belonging to the node 10 perform the data reconstruction operation, the I / O load corresponding to the node 10 is obtained, and when the I / O load exceeds the preset threshold, the reconstruction management module 122 and the I / O management module 121 belonging to the node 10 are migrated to the node 20, so that the data reconstruction operation is performed by the reconstruction management module 122' and the I / O management module 121' deployed in the node 20.

[0063] In one embodiment, a node service module is deployed within the distributed storage system to record I / O requests corresponding to the node in real time. The recorded I / O requests are recorded in the data node memory and then periodically written to the statistical data file of the system (e.g., computer system 100). Based on this, the I / O load corresponding to the failed node can be directly obtained.

[0064] In the specific implementation process, Figure 2 As shown, migrating the reconstruction management module and the I / O management module of the failed node to any node that has not failed includes the following steps S31 to S34.

[0065] Step S31: Select a node that has not experienced any failure as the target node.

[0066] Step S32: The reconstruction management module to which the faulty node belongs suspends sending new reconstruction requests, and the I / O management module to which the faulty node belongs suspends sending new I / O requests.

[0067] Step S33: When the reconstruction request sent before the pause and the I / O request sent before the pause are both executed, the pause position of the current data reconstruction operation is recorded.

[0068] Step S34: shut down the reconstruction management module and I / O management module of the failed node, start the reconstruction management module and I / O management module of the target node, and perform data reconstruction operations according to the pause position.

[0069] Take the migration of the reconstruction management module 122 and the I / O management module 121 belonging to the node 10 to the node 20 as an example for exemplary explanation. At this time, the node 20 is the target node. The reconstruction management module 122 belonging to the node 10 suspends sending new reconstruction requests, and the I / O management module 121 belonging to the node 10 suspends sending new I / O requests. When the reconstruction requests sent before the reconstruction management module 122 is suspended and the I / O requests sent before the I / O management module 121 is suspended are all executed, the pause position of the current data reconstruction operation is recorded. Shut down the reconstruction management module 122 and the I / O management module 121 belonging to the node 10. Start the reconstruction management module 122' and the I / O management module 121' belonging to the node 20, and continue to execute the data reconstruction operation according to the previously recorded pause position.

[0070] In one embodiment, selecting a healthy node as the target node includes obtaining the I / O load corresponding to the healthy node and determining the target node based on the I / O load. For example, the I / O load corresponding to node 20 and node 30 may be obtained, and the target node may be determined based on the I / O load corresponding to node 20 and node 30, respectively. A specific strategy for determining the target node may be comparing the I / O loads corresponding to node 20 and node 30, and selecting the node with the lower I / O load as the target node, although this embodiment does not specifically limit this.

[0071] In one embodiment, the reconstruction management module and the I / O management module of the failed node are migrated to any node where the failure has occurred, and the process also includes: if the reconstruction request sent before the pause and / or the I / O request sent before the pause are not completed within the specified time, a migration failure message is returned, and the reconstruction management module and the I / O management module of the failed node are restarted.

[0072] Step S4: The reconstruction management module and the I / O management module of the migrated node perform data reconstruction on the failed physical disk based on the data blocks in the virtual disk of the failed node stored in the physical disks of different nodes.

[0073] Exemplarily, the reconstruction management module and I / O management module of the migrated node perform a data reconstruction operation on the failed physical disk, including: determining a target physical disk from the failed node, the reconstruction management module of the migrated node sending a reconstruction request to the I / O management module, the I / O management module sending an I / O request to a physical disk belonging to a different node based on the reconstruction request, and the physical disk belonging to the different node responding to the I / O request and writing data blocks to the target physical disk. Because the target physical disk has already been determined from the failed node when the reconstruction management module and I / O management module of the failed node perform the data reconstruction operation on the failed physical disk in step S2, this step can be omitted in step S4. Alternatively, the target physical disk can be re-determined from the failed node, which is not specifically limited in this embodiment. In short, the reconstruction management module and I / O management module of the migrated node replace the reconstruction management module and I / O management module of the failed node. Based on the I / O load of each node, the reconstruction management module and I / O management module are dynamically adjusted to the appropriate node, which not only ensures the uninterrupted operation of reconstruction requests and I / O requests, but also achieves the goal of optimizing the overall performance of the system.

[0074] For example, the reconstruction management module 122 and I / O management module 121 corresponding to virtual disk 111 on node 10 are migrated to node 20, so that the reconstruction management module 122' and I / O management module 121' belonging to node 20 correspond to virtual disk 111. If I / O management module 121' of virtual disk 111 receives 1000 write requests and 1000 read requests, node 10 receives 1000 write requests and 0 read requests, while node 20 receives 1000 write requests and 1000 read requests. If reconstruction management module 122' of virtual disk 111 initiates 2000 read requests to node 20 and then writes the read data blocks to node 10, node 10 receives 2000 write requests. Therefore, from the perspective of virtual disk 111, node 10 receives 3000 write requests and 0 read requests, node 20 receives 1000 write requests and 3000 read requests, and node 30 receives 0 write requests and 0 read requests. The read and write request counts of the virtual disk 211 and the aforementioned Figure 5 Similarly, node 10 receives 0 write requests and 0 read requests, node 20 receives 1000 write requests and 1000 read requests, and node 30 receives 1000 write requests and 0 read requests. Figure 6 Similarly, node 10 receives 3000 write requests and 0 read requests, node 20 receives 0 write requests and 0 read requests, and node 30 receives 1000 write requests and 3000 read requests.

[0075] Based on this, from the perspective of the entire computer system 100, node 10 receives 6,000 write requests and 0 read requests, node 20 receives 2,000 write requests and 4,000 read requests, and node 30 receives 2,000 write requests and 3,000 read requests. Therefore, compared with the previous example, the I / O load between the three nodes is more balanced.

[0076] Finally, based on the above technical solutions, Figure 8 As shown, the present application also discloses a specific embodiment of a computer-readable medium 900. The computer-readable medium 900 can be configured in whole or in part in a physical computer, server, cluster server, or data center. In this embodiment, a computer-readable medium 900 stores computer program instructions 901. When the computer program instructions 901 are read and executed by a processor 902, the steps in the data reconstruction method disclosed in the above embodiment are executed. Optionally, the computer-readable medium 900 can be configured as a server, and the server runs on a physical device for building a private cloud, hybrid cloud, or public cloud.

[0077] It should be noted that if the integrated unit in this embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (such as a personal computer, a server or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

[0078] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

[0079] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0080] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A data reconstruction method, running in a system consisting of multiple nodes, characterized in that: The data reconstruction method includes: The data blocks in the virtual disk of the node are stored in the physical disk of different nodes in a redundant storage manner; When a physical disk belonging to a node fails, the reconstruction management module and the I / O management module belonging to the failed node perform a data reconstruction operation on the failed physical disk based on the data blocks in the virtual disk belonging to the failed node stored on physical disks belonging to different nodes; When performing a data reconstruction operation, the I / O load corresponding to the failed node is obtained, and when the I / O load exceeds a preset threshold, the reconstruction management module and the I / O management module belonging to the failed node are migrated to any surviving node; The reconstruction management module and the I / O management module of the migrated node perform data reconstruction on the failed physical disk based on data blocks in the virtual disk of the failed node stored in physical disks of different nodes.

2. The data reconstruction method according to claim 1, characterized in that: A virtual machine and a distributed storage system are run on a single node. A virtual disk is deployed in the virtual machine. A reconstruction management module, an I / O management module and a physical disk corresponding to the virtual disk are deployed in the distributed storage system.

3. The data reconstruction method according to claim 2, characterized in that: The performing of a data reconstruction operation on a failed physical disk includes: A target physical disk is determined from the failed node, and the reconstruction management module sends a reconstruction request to the I / O management module. The I / O management module sends an I / O request to the physical disks belonging to different nodes based on the reconstruction request. The physical disks belonging to different nodes respond to the I / O request and write the data blocks to the target physical disk.

4. The data reconstruction method according to claim 3, characterized in that: The step of migrating the reconstruction management module and the I / O management module of the failed node to any surviving node includes: Select a node that has not failed as the target node; The reconstruction management module to which the faulty node belongs suspends sending new reconstruction requests, and the I / O management module to which the faulty node belongs suspends sending new I / O requests; When the reconstruction request sent before the pause and the I / O request sent before the pause are both executed, the pause position of the current data reconstruction operation is recorded; The reconstruction management module and the I / O management module of the failed node are shut down, the reconstruction management module and the I / O management module of the target node are started, and a data reconstruction operation is performed according to the pause position.

5. The data reconstruction method according to claim 4, characterized in that: The selecting a node that has not experienced a fault as the target node includes: The I / O load corresponding to the node that has not failed is obtained, and the target node is determined according to the I / O load.

6. The data reconstruction method according to claim 4, characterized in that: The step of migrating the reconstruction management module and the I / O management module of the failed node to any surviving node further includes: If the reconstruction request and / or I / O request sent before the pause are not completed within the specified time, a migration failure message is returned, and the reconstruction management module and I / O management module of the failed node are restarted.

7. The data reconstruction method according to claim 2, characterized in that: A node service module is deployed in the distributed storage system to record the I / O requests corresponding to the nodes in real time.

8. The data reconstruction method according to claim 1, characterized in that: The redundant storage method is a mirror storage method, which copies the data blocks in the virtual disk belonging to the node to form multiple mirror blocks, and stores the mirror blocks in the physical disks belonging to different nodes respectively.

9. The data reconstruction method according to claim 1, characterized in that: The redundant storage method is an erasure code storage method, which divides the data blocks in the virtual disk belonging to the node into multiple data slices, and calculates the verification slices based on the data slices, and stores the data slices and the verification slices respectively in the physical disks belonging to different nodes.

10. A computer-readable medium, characterized in that include: The computer-readable medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps of the data reconstruction method according to any one of claims 1 to 9 are executed.

Images