Disk migration method, system, device and medium

Abstract

The present invention proposes a disk migration method, system, device and medium. The method obtains the disk capacity, new node information and original node information in the cluster, and determines the number of migrated disks of each original node, thereby obtaining multiple disk migration plans, and migrates the migrated disks in each disk migration plan to the new node respectively to obtain multiple disk migration plans, determines the number of block data to be migrated in each disk migration plan, and uses the disk migration plan with the least number of block data to be migrated as the preferred migration plan for disk migration. The method can achieve cluster expansion with fewer new nodes, less data migration and lower expansion cost.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a disk migration method, system, device and medium. Background Art

[0002] In a distributed erasure storage system, data is distributed on different data nodes in the cluster. Data nodes usually store data on dozens of disks. When the cluster capacity is full, in order to allow the storage cluster to continue to store new data, the available capacity of the cluster is usually increased by expanding nodes. When expanding nodes under the N+M redundancy strategy of erasure, if the number of nodes is expanded according to N+M, the cost of one expansion is huge for users, and a low-cost disk migration solution is urgently needed. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, the present invention provides a disk migration method, system, device and medium to solve the above-mentioned technical problems.

[0004] The present invention provides a disk migration method, the method comprising:

[0005] Obtain the disk capacity, new node information and original node information in the cluster, and determine the number of migrated disks of each of the original nodes. The new node information includes the number of new nodes and the new node available capacity of each new node. The original node information includes the number of original nodes and the original node available capacity of each original node.

[0006] Determine multiple disk groups to be migrated from the original disks of each original node according to the number of migrated disks of each original node, and obtain multiple disk groups to be migrated corresponding to each original node;

[0007] Combine the disk groups to be migrated corresponding to the original nodes respectively to obtain multiple disk migration plans;

[0008] Migrate the disks to be migrated in each disk migration plan to the newly added node to obtain multiple disk migration plans;

[0009] The amount of block data to be migrated of each disk migration scheme is determined, and the disk migration scheme with the least amount of block data to be migrated is used as a preferred migration scheme for disk migration.

[0010] Optionally, the number of disks to be migrated from the original node may be determined by:

[0011] Determine the total available capacity of the current cluster based on the available capacity of the newly added nodes and the available capacity of the original nodes;

[0012] Determine the total number of current cluster nodes based on the number of newly added nodes and the number of original nodes;

[0013] Determine the average total available capacity of each node in the current cluster based on the total available capacity of the current cluster and the total number of nodes in the current cluster;

[0014] The available capacity of the original disks in each original node is obtained, and the number of disks to be migrated from the original node is determined based on the average total available capacity of each node in the current cluster and the available capacity of the original disks.

[0015] Optionally, the method for determining the amount of block data to be migrated in the disk migration solution includes:

[0016] Get the maximum number of block data in the same group that can be written to a node and the number of block data in the same group of each cluster node in the disk migration solution. The cluster nodes include newly added nodes and original nodes.

[0017] The number of block data to be migrated in the disk migration scheme is determined according to the maximum number of block data of the same group that can be written to the node and the number of block data of the same group of each cluster node in the disk migration scheme.

[0018] Optionally, the disk migration scheme includes the migration disk slot information of the newly added node, the migration disk slot information of the original node, and the migration disk slot information of the original node, and the method further includes:

[0019] A disk plug-in prompt message is generated according to the migration disk slot information of the newly added node, the migration disk slot information of the original node, and the migration disk slot information of the original node of the preferred migration plan.

[0020] Optionally, after plugging and unplugging at least a portion of the original disks and at least a portion of the newly added disks according to the preferred migration solution, the method further includes:

[0021] Obtaining disk configuration information of each current disk in the cluster node, the cluster node includes the original node and the newly added node, and the current disk includes the original disk and the newly added disk;

[0022] If the disk configuration information includes disk identification information and node identification information, and the node identification information is the same as the node preset information of the current node, the current disk is controlled to go online;

[0023] If the disk configuration information includes disk identification information and node identification information, and the node identification information is different from the node preset information of the current node, new disk identification information and new node identification information are configured for the current disk, and the new disk identification information and new node identification information are determined as the disk configuration information, and the current disk is controlled to go online;

[0024] If the disk configuration information does not include disk identification information and / or node identification information, the disk identification information and / or node identification information is configured for the current disk, and the disk identification information and node identification information are determined as the disk configuration information to control the current disk to go online.

[0025] Optionally, before controlling the current disk to go online, the method further includes:

[0026] Get and record the disk configuration information of the current disk;

[0027] If the disk configuration information includes disk identification information, node identification information, new disk identification information and new node identification information, the disk configuration information is updated to the new disk identification information and the new node identification information, and the current disk is notified to go online.

[0028] Optionally, the method further includes:

[0029] Get the number of block data to be migrated for the preferred migration solution;

[0030] If the number of block data to be migrated in the preferred migration solution is greater than zero, determine the node block data to be migrated of each cluster node according to the maximum number of block data in the same group that can be written to the node and the number of node block data in the same group of each cluster node in the preferred migration solution;

[0031] At least one target node is selected from each cluster node according to the load balancing principle, and the node block data to be migrated of each cluster node is migrated to the target node.

[0032] Optionally, before selecting at least one target node from each cluster node according to the load balancing principle, the method further includes:

[0033] Clean up the current disks on each cluster node.

[0034] The present invention also provides a disk migration system, the system comprising:

[0035] An acquisition module is used to acquire the disk capacity, new node information and original node information in the cluster, and determine the number of migrated disks of each of the original nodes. The new node information includes the number of new nodes and the new node available capacity of each new node, and the original node information includes the number of original nodes and the original node available capacity of each original node;

[0036] The to-be-migrated disk group determination module is used to determine multiple to-be-migrated disk groups from the original disks of each original node according to the number of migrated disks of each original node, and obtain multiple to-be-migrated disk groups corresponding to each original node;

[0037] A disk migration plan determination module is used to combine the disk groups to be migrated corresponding to each original node to obtain multiple disk migration plans;

[0038] A disk migration plan determination module is used to migrate the migrated disks in each disk migration plan to the newly added node to obtain multiple disk migration plans;

[0039] The preferred migration scheme determining module is used to determine the number of block data to be migrated in each disk migration scheme, and use the disk migration scheme with the least number of block data to be migrated as the preferred migration scheme for disk migration.

[0040] The present invention also provides an electronic device, comprising a processor, a memory and a communication bus;

[0041] The communication bus is used to connect the processor and the memory;

[0042] The processor is used to execute the computer program stored in the memory to implement the method described in any one of the above embodiments.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon.

[0044] The computer program is used to enable a computer to execute the method described in any one of the above embodiments.

[0045] Beneficial effects of the present invention: The present invention proposes a disk migration method, system, device and medium. The method obtains the disk capacity, new node information and original node information in the cluster, and determines the number of migrated disks of each original node, thereby obtaining multiple disk migration plans, and migrates the migrated disks in each disk migration plan to the new node respectively to obtain multiple disk migration plans, determines the number of block data to be migrated in each disk migration plan, and uses the disk migration plan with the least number of block data to be migrated as the preferred migration plan for disk migration. The method can achieve cluster expansion with fewer new nodes, less data migration and lower expansion cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of a disk migration method provided in one embodiment of the present invention;

[0047] Figure 2 It is a specific flow chart of a disk migration method provided in one embodiment of the present invention;

[0048] Figure 3 It is a specific flow chart of determining a preferred migration solution provided in an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of the structure of a distributed storage system provided in one embodiment of the present invention;

[0050] Figure 5 is a structural diagram of a specific example of disk data migration provided in one embodiment of the present invention;

[0051] Figure 6 It is a specific flowchart of disk management provided in one embodiment of the present invention;

[0052] Figure 7 It is a specific flowchart of metadata update provided in one embodiment of the present invention;

[0053] Figure 8 It is a specific flow chart of disk data migration provided in one embodiment of the present invention;

[0054] Fig. 9 It is a structural schematic diagram of a disk migration system provided by an embodiment of the present invention;

[0055] Fig.10 It is a specific structural diagram of a disk migration system provided by an embodiment of the present invention;

[0056] Fig.11 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0058] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0059] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0060] like Figure 1 As shown, this embodiment provides a disk migration method, the method comprising:

[0061] Step S101: Obtain the disk capacity, new node information and original node information in the cluster, and determine the number of migrated disks of each original node.

[0062] The disk capacity is the total disk capacity of the original disks of the original nodes in the cluster and the total disk capacity of the newly added disks in the newly added nodes. If the original disks and the newly added disks are of the same specifications (at least have the same total disk capacity), the disk capacity is the same value as the total disk capacity of the newly added disks or the original disks.

[0063] If there are multiple disks with the same total capacity in the original disks, and / or the total capacity of the newly added disk is different from that of at least one original disk, the disk capacity can be the sum of the total capacity of the newly added disk and the original disk divided by the total number of the newly added disk and the original disk, that is, the disk capacity is the average of the total capacity of the disks. For example, there are m original disks with a total capacity of x, n original disks with a total capacity of y, and p newly added disks with a total capacity of z. In this case, the disk capacity A = (mx+ny+pz) / (m+n+z). For disks with multiple total capacities in the original disks, and / or the total capacity of the newly added disk is different from that of at least one original disk, those skilled in the art can also use other known methods to determine the disk capacity.

[0064] The newly added node information includes the number of newly added nodes and the newly added node available capacity of each newly added node. The newly added node available capacity can be determined based on the number of newly added nodes, the number of newly added disks in each newly added node, and the total disk capacity of each newly added disk. The newly added node available capacity is the sum of the newly added total capacity of the newly added disks in each newly added node.

[0065] The original node information includes the number of original nodes and the original node available capacity of each original node. The original node available capacity is the sum of the available capacity of each original disk in the original node (the remaining capacity excluding the occupied capacity).

[0066] In one embodiment, the method for determining the number of migrated disks of the original node includes:

[0067] Determine the total available capacity of the current cluster based on the available capacity of the newly added nodes and the available capacity of the original nodes;

[0068] Determine the total number of current cluster nodes based on the number of newly added nodes and the number of original nodes;

[0069] Determine the average total available capacity of each node in the current cluster based on the total available capacity of the current cluster and the total number of nodes in the current cluster;

[0070] The available capacity of the original disks in each original node is obtained, and the number of disks to be migrated from the original node is determined based on the average total available capacity of each node in the current cluster and the available capacity of the original disks.

[0071] Optionally, the current total available capacity of the cluster after the node expansion is determined by:

[0072] C=C1+C2 Formula (1);

[0073] Among them, C is the total available capacity of the current cluster, C1 is the available capacity of the original nodes, and C2 is the available capacity of the newly added nodes.

[0074] Optionally, the total number of nodes in the current cluster can be determined by:

[0075] N=N1+N2 formula (2);

[0076] Where N is the total number of nodes in the current cluster, N1 is the number of newly added nodes, and N2 is the number of original nodes.

[0077] Optionally, the average total available amount of each node in the current cluster of each node is determined by:

[0078] A=C / (N1+N2) Formula (3);

[0079] Among them, A is the average total available capacity of each node in the current cluster, C is the total available capacity of the current cluster, N1 is the number of newly added nodes, and N2 is the number of original nodes.

[0080] Optionally, the number of disks to be migrated from the original node may be determined by:

[0081] Mout=|A–Cn| / Cdisk Formula (4);

[0082] Among them, Mout is the number of migrated disks of the original node, A is the average total available amount of each node in the current cluster, Cn is the available capacity of the original disk, and Cdisk is the disk capacity.

[0083] It should be noted that the number of migrated disks determined according to formula (4) may be a decimal number. In this case, a rounding method can be determined by a person skilled in the art as needed, including but not limited to rounding, retaining integer digits, etc. For example, the number of migrated disks of the original node is Mout = ROUND (|A-Cn| / Cdisk).

[0084] When the number of disks to be migrated is rounded off during the determination process, after determining the number of disks to be migrated from each original node and the newly added node, it is also necessary to determine whether the sum of the number of disks to be migrated from each newly added node is less than the sum of the number of newly added disks of the newly added nodes. If the sum of the number of disks to be migrated from the newly added nodes is less than the sum of the number of newly added disks of the newly added nodes, the next step is not required. Otherwise, if the sum of the number of disks to be migrated is greater than the sum of the number of newly added disks of the newly added nodes, several original nodes can be selected to reduce the number of disks to be migrated from the original nodes based on the difference between the sum of the number of disks to be migrated and the sum of the number of newly added disks of the newly added nodes. For example, the difference between the sum of the number of disks to be migrated and the sum of the number of newly added disks of the newly added nodes is N, and the first N nodes with the smallest number of migrated disks are the original nodes with the number of disks to be migrated less by 1.

[0085] Optionally, to ensure cluster load balancing, migrate the migrated disks of the original node to the newly added node, and migrate the newly added disks of the newly added node to the original node, where the number of migrated disks in each node is equal to the number of migrated disks. The number of migrated disks of the newly added node can also be determined using formula (4). That is, the method for determining the number of migrated disks of the newly added node or the original node includes:

[0086] Mout=Min Formula (5);

[0087] Among them, Mout is the number of migrated-out disks of each node, and Min is the number of migrated-in disks of each node.

[0088] Step S102: determining a plurality of disk groups to be migrated out from a plurality of original disks of each original node according to the number of migrated disks of each original node, and obtaining a plurality of disk groups to be migrated out corresponding to each original node.

[0089] Theoretically, the number of disks to be migrated is less than the total number of original disks on the original node. At this time, there are at least two options for selecting the disks to be migrated from an original node, and each combination of the migrated disks forms a disk group to be migrated. For example, an original node includes m original disks, and the number of migrated disks of the original node is n. At this time, determining multiple disk groups to be migrated from the original node is equivalent to selecting n disks from the m original disks. Each selection method is a disk group to be migrated, and the maximum number of disk groups to be migrated is By performing the same operation on each original node in the above manner, multiple disk groups to be migrated corresponding to each original node can be obtained.

[0090] Step S103: respectively combine the disk groups to be migrated corresponding to the original nodes to obtain multiple disk migration solutions.

[0091] That is, each original node takes a disk group to be migrated to form a disk migration plan. Since each original node includes at least two disk groups to be migrated, there are multiple disk migration plans. For example, there are 10 original nodes, and each original node has two disk groups to be migrated. In this case, the number of disk migration plans is 2 10 The specific method of combining the disk groups to be migrated corresponding to each original node can be implemented in a manner known to those skilled in the art.

[0092] Step S104: Migrate the disks to be migrated in each disk migration plan to the newly added node to obtain multiple disk migration plans.

[0093] The newly added node may be one or more nodes. When there are multiple newly added nodes, the number of newly added disks in each newly added node may be the same or different.

[0094] For example, if there are q new nodes, t disk migration plans, and s disks to be migrated in each disk migration plan, and the number of migrated disks for each new node is the same (the number of new disks for each new node is equal), then the number of disk migration plans is indivual.

[0095] In addition, those skilled in the art may also determine the disk migration and disk migration solutions according to other known methods.

[0096] It can be understood that step S103 and step S104 are to migrate disks from the original node to one or more newly added nodes according to the number of migrated disks of the original node, and then select new disks with the same number of migrated disks from the one or more newly added nodes to migrate to the original node until all the original nodes have completed the disk migration and a disk migration plan is obtained. The disk migration plan can be traversed by polling to obtain all feasible disk migration plans.

[0097] Step S105: determining the number of block data to be migrated in each disk migration scheme, and taking the disk migration scheme with the least number of block data to be migrated as the preferred migration scheme for disk migration.

[0098] In one embodiment, the method for determining the amount of block data to be migrated in the disk migration solution includes:

[0099] Get the maximum number of block data in the same group that can be written to a node and the number of block data in the same group of each cluster node in the disk migration solution. The cluster nodes include newly added nodes and original nodes.

[0100] The number of block data to be migrated in the disk migration scheme is determined according to the maximum number of block data of the same group that can be written to the node and the number of block data of the same group of each cluster node in the disk migration scheme.

[0101] Optionally, the number of block data to be migrated is equal to the sum of the number of node migration block data of each cluster node, wherein the number of node migration block data=the number of block data of the same group of nodes-the maximum number of block data of the same group that can be written by the node.

[0102] The following is a specific example to illustrate the determination process of the above-mentioned preferred migration scheme. Taking the erasure mode of 2+1:1 as an example, the maximum number of blocks that can be written to each cluster node is 1 (the maximum number of block data that can be written to the node is 1), the original number of nodes is 3 (N1, N2, N3 respectively), the original number of disks of each original node is 40, the total disk capacity of each original disk is 100GB, and the remaining capacity of each disk (the available capacity of the original disk) is 10GB; a new node N4 is added, the number of newly added disks is 40, the total disk capacity of each newly added disk is 100GB, and the remaining capacity (available capacity) of each disk is 100GB.

[0103] According to formula (1), the total available capacity of the cluster after node expansion is C = (100–90)*40*3+40*100 = 5200 GB;

[0104] According to formula (3), the average total available capacity of each node in the current cluster is A = 5200 / 4 = 1300 GB;

[0105] According to formula (4), the following information can be obtained:

[0106] The number of disks migrated to the N1 node Min: (1300–10*40) / 100=9;

[0107] Number of disks migrated from N1 node Mout: 9;

[0108] The number of disks migrated to the N2 node Min: (1300–10*40) / 100=9;

[0109] Number of disks migrated from N2 node Mout: 9;

[0110] The number of disks migrated to the N3 node Min: (1300–10*40) / 100=9;

[0111] Number of disks migrated from N3 node Mout: 9;

[0112] The number of disks migrated to the N4 node Min: |1300-100*40| / 100=27;

[0113] Number of disks migrated from node N4 (Mout): 27.

[0114] Based on the above information, the following disk migration solution can be obtained:

[0115] Solution 1: Select disks numbered 1-9 from node N1, disks numbered 1-9 from node N2, and disks numbered 1-9 from node N3. Calculate the amount of block data migration if the current 27 disks are migrated to node N4 (node ​​N4 is a newly added node and does not need to consider data migration after its disks are migrated). Query all metadata information on the disks through the disk node ID (node ​​identification information) and disk ID (disk identification information). If there is data 1-1-1 on disks 1-9 of node N1, data 1-1-2 on disks 1-9 of node N2, and data 1-1-3 on disks 1-9 of node N3, then after migration, node N4 has 3 copies of the same group of block information, and the blocks (number of block data to be migrated) that need to be migrated are 3-1=2.

[0116] Solution 2: Select disks numbered 1-9 from node N1, disks numbered 1-9 from node N2, and disks numbered 2-10 from node N3. Calculate the amount of block data migration if the current 27 disks are migrated to node N4 (node ​​N4 is a newly added node and does not need to consider data migration after its disks are migrated). Query all metadata information on the disks through the disk node ID and disk ID. If there is data 1-1-1 on disks 1-9 of node N1, data 1-1-2 on disks 1-9 of node N2, and data 2-1-3 on disks 2-10 of node N3, then after migration, node N4 has two copies of the same group of block information, and the block to be migrated (the number of block data to be migrated) is 2-1=1.

[0117] In the above example, Solution 2 is better than Solution 1.

[0118] According to the above schemes, it is also possible to determine that there are schemes 3 to 729, and the number of blocks that need to be migrated (the number of block data to be migrated) for each scheme can be determined respectively.

[0119] Through the above method until all situations are processed, the disk migration solution with the least number of migrations (the least number of block data to be migrated) is obtained as the preferred solution.

[0120] Optionally, if there are at least two disk migration schemes with an equal number of partitioned data to be migrated and less than the number of partitioned data to be migrated of other disk migration schemes, that is, there are at least two disk migration schemes with the least number of partitioned data to be migrated, then one of the disk migration schemes can be selected as the preferred migration scheme.

[0121] If there are at least two disk migration schemes with the same number of block data to be migrated and less than the number of block data to be migrated of other disk migration schemes, that is, there are at least two disk migration schemes with the least number of block data to be migrated, that is, there are at least two disk migration schemes that can be used as preferred migration schemes, then the method further includes:

[0122] Get the preset disk migration difficulty of each original disk;

[0123] Determine the weight of each preferred migration solution according to the preset disk migration difficulty of each original disk;

[0124] The preferred migration solution with the smallest solution weight is selected as the final migration solution.

[0125] The scheme weight is equal to the sum of the preset disk migration difficulties of the migration disks in each preferred migration scheme. The greater the disk migration difficulty, the greater the preset disk migration difficulty. The preset disk migration difficulty can be preset by a person skilled in the art.

[0126] In one embodiment, the disk migration scheme includes the migration disk slot information of the newly added node, the migration disk slot information of the original node, and the migration disk slot information of the original node. The method further includes:

[0127] A disk plugging and unplugging prompt message is generated based on the migration disk slot information of the newly added node, the migration disk slot information of the original node, and the migration disk slot information of the original node of the preferred migration plan. The disk plugging and unplugging prompt message is used to prompt the disk position to facilitate subsequent disk plugging and unplugging.

[0128] The above method allows users to intuitively know which disks on a node should be removed and install new disks in the vacant slots. When there are multiple new nodes, users can also clearly know which original disks are placed under which new nodes. Such clear prompts will improve the efficiency of disk insertion and removal and make it more convenient for users.

[0129] Optionally, the disk migration plan may further include the migration disk slot information of the newly added node. Accordingly, the preferred migration plan also includes the migration disk slot information, so that the user can know which slots of the newly added disks to be removed for the newly added node.

[0130] Optionally, the disk plugging and unplugging may also be implemented by other methods known to those skilled in the art, not limited to manual plugging and unplugging.

[0131] It is understandable that the newly added node's migration disk slot information, the original node's migration disk slot information, and the original node's migration disk slot information can be determined by the pre-set disk configuration information-slot information mapping relationship of the newly added disk or the original disk. For the original node, when it is known which original disks need to be removed, the disk configuration information of these original disks that need to be removed is also known, and the disk slot information of each original disk that needs to be removed is determined based on the disk configuration information and the pre-set disk configuration information-slot information mapping relationship. For the newly added node, since the preferred migration plan also includes which original nodes are moved into the newly added node, similar to the above method, it can be known to which new node the removed original node is moved into.

[0132] In one embodiment, after plugging and unplugging at least a portion of the original disks and at least a portion of the newly added disks according to the preferred migration scheme (after plugging and unplugging the disks under each newly added node and the original node according to the preferred migration scheme), the method further includes:

[0133] Get the disk configuration information of each current disk in the cluster node. The cluster node includes the original node and the newly added node. The current disk includes the original disk and the newly added disk. The current disk set includes all the original disks and the newly added disks. That is, the current disk is the disk under each cluster node in the cluster after the disk is plugged in and out.

[0134] If the disk configuration information includes disk identification information and node identification information, and the node identification information is the same as the node preset information of the current node, the current disk is controlled to go online. It can be understood that the current disk corresponding to the disk configuration information exists under the same cluster node before and after the disk is plugged in and out. At this time, the upper limit of the current disk can be completed;

[0135] If the disk configuration information includes disk identification information and node identification information, and the node identification information is different from the node preset information of the current node, new disk identification information and new node identification information are configured for the current disk, and the new disk identification information and new node identification information are determined as the disk configuration information to control the current disk to go online. It can be understood that the current disk corresponding to the disk configuration information is located at cluster node A before the disk is plugged in and out, and is located at cluster node B after the disk is plugged in and out. The cluster node where it is located has changed. At this time, it is necessary to reconfigure new disk configuration information for the current disk before the current disk is put online. It should be noted that each cluster node has globally unique node preset information. If a current disk is located at the same cluster node before and after the disk is plugged in and out, then the node identification information of the current disk is consistent with the node preset information. Otherwise, the node of the current disk has changed, and the node preset information of the node after the change is different from the node identification information of the current node.

[0136] If the disk configuration information does not include disk identification information and / or node identification information, configure the disk identification information and / or node identification information for the current disk, and determine the disk identification information and node identification information as the disk configuration information to control the current disk to go online. It can be understood that for the newly added disk, the disk identification information and / or node identification information may not have been configured yet. In this case, the newly added disk does not include disk identification information and / or node identification information and needs to be configured. During the configuration process, the node identification information of the newly added disk is the same as the node preset information of the cluster node where the newly added disk is located. The disk identification information is the globally unique identification information of the newly added disk.

[0137] This process can be regarded as disk management for disks that have been plugged in and out. By checking the disk configuration information of each disk, it is convenient to confirm the identity of each disk, which is convenient for further management of the disk. After the disk is online, the disk becomes available.

[0138] In one embodiment, before controlling the current disk to go online, the method further includes:

[0139] Get and record the disk configuration information of the current disk;

[0140] If the disk configuration information includes disk identification information, node identification information, new disk identification information and new node identification information, the disk configuration information is updated to the new disk identification information and the new node identification information, and the current disk is notified to go online.

[0141] That is, the metadata management module replaces the original disk identification information with the reconfigured new disk identification information and replaces the original node identification information with the new node identification information, and then notifies the current disk to go online. After receiving the online notification, the disk management module controls the current disk to go online.

[0142] After a disk is transferred from one node to another, the metadata information (disk configuration information) on the disk may change, so the metadata needs to be updated. According to the disk node identification information NodeId and disk identification information DiskID that need to be migrated, the data recovery of the disk is suspended and the data recovery is started after the disk migration is completed. During the disk migration process, power outages, network failures and other abnormalities may occur. Before the migration, the disk to be drifted is first recorded in the database, and the new node identification information SrcNodeId, new disk identification information SrcDiskId, node identification information DesNodeID, disk identification information DesDiskId, and migration status of the disk are recorded. After the abnormality is recovered, the migration process can be restarted. The metadata of the data on the original disk is updated to the new node identification information NodeID and the new disk identification information DiskID through the node identification information SrcNodeId and the node identification information SrcDiskId. After the update is completed, the disk is notified to go online.

[0143] The available capacity of the cluster is increased by expanding nodes. When expanding nodes under the erasure redundancy strategy of N+M, if the number of newly expanded nodes is less than N+M, the newly expanded nodes cannot write N+M copies of data. At the same time, the available capacity of the newly expanded nodes is much larger than the available capacity of the existing data nodes in the cluster. This will cause the capacity difference between the various data nodes to be too large, resulting in load imbalance, causing the overall performance of the storage cluster to decline, and the expansion effect cannot be achieved. Therefore, when the number of newly added nodes is less than the erasure redundancy strategy N+M of the cluster, the method also includes:

[0144] Get the number of block data to be migrated for the preferred migration solution;

[0145] If the number of block data to be migrated in the preferred migration solution is greater than zero, determine the node block data to be migrated of each cluster node according to the maximum number of block data in the same group that can be written to the node and the number of node block data in the same group of each cluster node in the preferred migration solution;

[0146] According to the load balancing principle, at least one target node is selected from each cluster node, and the node block data to be migrated of each cluster node is migrated to the target node. In this way, the internal data distribution of the cluster after the disk is plugged in and out can comply with the correction and deletion strategy, and the parent load is more balanced.

[0147] Wherein, determining the node block data to be migrated of each cluster node according to the maximum number of block data of the same group that can be written to the node and the number of block data of the same group of each cluster node in the preferred migration scheme includes:

[0148] If the maximum number of block data in the same group that can be written to a node is greater than or equal to the number of block data in the same group of the node in the cluster, and there is no node block data to be migrated in the cluster node, the cluster node will be selected as the target node to be selected;

[0149] If the maximum number of block data in the same group that can be written to a node is less than the number of block data in the same group of the cluster node, the number of block data to be migrated in the cluster node is equal to the difference between the number of block data in the same group of the cluster node and the maximum number of block data in the same group that can be written to the node, and the block data to be migrated is the number of block data to be migrated selected from the block data in the same group of the cluster node. The selection rules can be confirmed by technical personnel in this field as needed, such as random selection.

[0150] For a cluster node, the number of block data in the same group of nodes may be only one or more. For example, the cluster node includes three different groups of data A, B, and C, where data A includes two block data, data B includes three block data, and data C includes four block data. The current erasure code is 4+2:1, and the maximum number of block data that can be written to each cluster node is 2 (2 / 1=2). At this time, it can be known that there are two block data in the cluster node whose amounts are greater than the maximum number of block data that can be written to the same group of nodes. The number of block data to be migrated in the cluster node is 3 (4-2+3-2=3). You can select one block data from data B and two block data from data C as the block data to be migrated by the node.

[0151] The target node can be a target node with the largest remaining node capacity (node ​​available capacity) among all target nodes to be selected, or the remaining node capacity is arranged from large to small, and the top N target nodes to be selected are used as the target node. The specific selection of the target node can also be implemented according to other methods known to those skilled in the art.

[0152] In one embodiment, before selecting at least one target node from each cluster node according to the load balancing principle, the method further includes:

[0153] Clean up the current disks on each cluster node.

[0154] That is, after the disk is plugged in and out, the disks on each node are cleaned up, so that the available disk capacity of at least a part of the disks may be larger than the available disk capacity recorded when the disk migration plan was initially implemented. On this basis, selecting at least one target node from each cluster node according to the load balancing principle will make the load in the cluster more balanced. The load balancing principle can be understood as migrating the node block data to be migrated to other nodes with relatively larger available capacity. In the case of multiple block data to be migrated, the multiple block data to be migrated can be dispersed to multiple target nodes.

[0155] The disk migration method provided in the above embodiment obtains disk capacity, new node information and original node information of each original node in the cluster, and determines the number of migrated disks of each original node, thereby obtaining multiple disk migration plans, and migrates the migrated disks in each disk migration plan to the new node respectively, to obtain multiple disk migration plans, and determines the number of block data to be migrated in each disk migration plan, and uses the disk migration plan with the least number of block data to be migrated as the preferred migration plan for disk migration, so as to achieve cluster expansion with fewer new nodes, less data migration and lower expansion cost.

[0156] Optionally, by pre-calculating the data distribution and manually intervening (determining the optimal migration plan and then performing disk migration), the data migration required to achieve cluster data load balancing after node expansion can be greatly reduced. Load balancing can be achieved with a small amount of data migration, and data balancing after node expansion can be completed quickly.

[0157] Optionally, after at least a portion of the original disks and at least a portion of the newly added disks are plugged in and out according to the preferred migration plan, the disk configuration information is confirmed and updated, and then the disks are controlled to go online, which also ensures high data reliability.

[0158] In a distributed erasure storage system, data is distributed on different data nodes in the cluster. Data nodes usually store data on dozens of disks. When the cluster capacity is full, in order to allow the storage cluster to continue to store new data, the available capacity of the cluster is usually increased by expanding nodes. When expanding nodes under the N+M redundancy strategy of erasure, if the number of newly expanded nodes is less than N+M, the newly expanded nodes cannot write N+M copies of data. At the same time, the available capacity of the newly expanded nodes is much larger than the available capacity of the existing data nodes in the cluster. This will cause the capacity difference between the data nodes to be too large, resulting in load imbalance, causing the overall performance of the storage cluster to decline, and the expansion effect cannot be achieved. If the expansion is based on the number of N+M nodes, the cost of one expansion is huge for users.

[0159] In order to solve the load imbalance and high cost problems caused by different numbers of newly expanded nodes in the conventional node expansion method, so that the expansion node is no longer limited by the number of newly expanded nodes, the disk migration method provided in the above embodiment is schematically described below through a specific embodiment. Figure 2 , the specific method includes:

[0160] Step S201: Determine a preferred migration solution.

[0161] See also Figure 3 , the preferred migration solution may be determined by:

[0162] Step S2011: Determine the number of disks to be migrated from each original node according to the disk capacity, the number of newly added nodes, the available capacity of the newly added nodes, the number of original nodes, the available capacity of the original nodes, and the available capacity of the original disks under each original node.

[0163] Step S2012: Determine multiple disk migration schemes according to the number of disks to be migrated from each original node.

[0164] Step S2013: Calculate the number of block data to be migrated for each disk migration solution respectively.

[0165] Step S2014: taking a disk migration solution with the least amount of block data to be migrated as a preferred migration solution.

[0166] The number of disks migrated out of each original node is equal to the number of disks migrated in of the original node. The number of disks migrated out can be determined by referring to the above formula (1) to formula (5).

[0167] An exemplary method for determining a preferred migration plan is as follows: according to the number of disks that need to be migrated from the original node i calculated in step S2011, Mini is the number of disks. The nodes 1-N are trained in rotation, and each node i selects Mini disks respectively. The number of blocks that need to be migrated under each combination (the number of block data to be migrated) is calculated, and the combination with the smallest number of block data to be migrated is taken as the final plan.

[0168] For example, the allocatable capacity of each node is calculated based on the existing available capacity of the cluster and the available capacity of the newly added nodes (the average total available capacity of each node in the current cluster), the number of disks that need to be migrated from each node (the number of disks to be migrated) is calculated, and the number of blocks that need to be migrated (the number of blocks to be migrated) under each combination (disk migration plan) is calculated based on the number of disks that need to be migrated.

[0169] The block migration is determined by the erasure data distribution requirement: in the N+M:K erasure code, the maximum number of blocks that can be written to a data node is equal to M / K. The block distribution number on each data node is determined. If there are more than M / K, migration is required. The combination with the least number of block migrations is selected as the preferred migration solution. Optionally, the user is informed of the preferred migration solution in the form of the disk slot number.

[0170] Step S202: According to the preferred migration solution provided in step S201, disk migration is performed manually.

[0171] Step S203: configure disk configuration information and perform disk management.

[0172] The disk management module senses that a disk other than the node is online (a new disk has been migrated to the cluster node), and needs to trigger disk migration. It will carry the disk information to be migrated through the metadata update module to start disk migration.

[0173] For example, when a new disk M is online, a disk ID (DiskID, disk identification information) is assigned to disk M, and the data node ID (NodeID, node identification information) and DiskID are written to the disk together. When the disk is online, it is determined whether the disk is the disk of this node based on the NodeID recorded in the disk. If not, a DiskID is reallocated to the disk (new disk identification information is configured), and the new NodeID (the same as the node preset information of the node where the current disk M is located), DiskID and old NodeID and DiskID of the disk are carried. The metadata update module is notified to trigger the disk migration and wait for the disk metadata update to be completed. After the disk metadata update is completed, the NodeID and DiskID recorded in the disk are updated to the new ones to complete the online process, and the disk becomes available.

[0174] Step S204: metadata update.

[0175] The metadata update module records the disk information to be updated in the database in the format of SrcNodeID (new node identification information)-SrcDiskID (new disk identification information)-DesNodeID (original node identification information)-DesDiskID (original node identification information)-DriftStatus, initializes the migration status to the metadata update status (0, MetaUpdate), and changes the database migration status to the metadata update completion status (1, MetaUpdateOver) after all metadata information of the data is updated. Notify the completion of metadata update.

[0176] After the disk is migrated to another cluster node, the metadata information of the data on the disk will change, so the metadata needs to be updated. According to the original NodeId and DiskID of the disk to be migrated, the data recovery of the disk is suspended, and the data recovery is started after the disk migration is completed. During the disk migration process, power outages, network failures and other abnormalities may occur. Before the migration, the disk to be drifted should be recorded in the database, and the SrcNodeId, SrcDiskId, DesNodeID, DesDiskId, and migration status of the disk should be recorded. After the abnormal recovery occurs, the migration process can be restarted, and the metadata of the data on the original disk is updated to the new NodeID and DiskID through SrcNodeId and SrcDiskId. After the update is completed, notify the disk to go online.

[0177] Step S205: The disk is online.

[0178] After receiving the message that the metadata update is complete, the disk management updates the disk NodeID and DiskID information, and the disk is online.

[0179] Step S206: Disk data migration.

[0180] The data migration module periodically obtains disks with a migration status of MetaUpdateOver from the database for data migration. After the migration is completed, the migration information is removed from the database to complete the disk migration process.

[0181] Taking erasure coding and multiple copies, which are commonly used redundancy strategies in distributed storage, as an example, when a disk is migrated, the distribution of data in the cluster will change, and the data distribution may no longer meet the high reliability of the redundancy strategy. At this time, data migration is used to ensure the high reliability of the data after disk migration. According to the data information distributed on the migration disk, the data distribution of the same group is queried, and the block data block is determined according to the erasure data distribution requirements. If migration is required, the target node is selected according to load balancing, and the migrated block information is recorded in the database in the format of: SrcNodeId-SrcDiskId-DesNodeID-DesDiskId-MigrateStatu. The initial migration status is not started (0, MigrateDefault). After sending the above information to the target data node for data migration, the migration status is modified to (1, MigrateStarting). After waiting for the data node migration to complete, the block data block on SrcNodeId-SrcDiskId is deleted to complete a data migration.

[0182] Compared with the existing method for achieving cluster load balancing after node expansion, the method provided by the above embodiment usually requires global data migration, and the data on the existing data nodes of the cluster are migrated to the newly expanded data nodes, which leads to a large amount of data migration. The large amount of data migration will consume the cluster performance and take a long time, and the cluster cannot quickly reach a load-balanced environment. The disk migration method provided by this embodiment is applied to a distributed storage system. By pre-calculating data distribution and adding manual intervention, the cluster can achieve load balancing under a small amount of data migration after the new node is expanded. Pre-calculation will give a disk migration plan (preferred migration plan), calculate the optimal plan for which disks to migrate to the new data node, and give a disk migration plan in advance before the node is expanded according to the number of disks and disk capacity of the newly expanded node. During pre-calculation, it is ensured that the data is balanced after the disk migration, and the disks between the data nodes in the cluster are allowed to be plugged in and out. Then, the disks are migrated by the user, so as to achieve load balancing after the node expansion while ensuring the high reliability of the data. The redundancy strategy of the storage cluster is met by migrating the plugged-in disks, and the high reliability of the data after the disk migration occurs is ensured.

[0183] The above-mentioned disk migration method is further described below through a specific embodiment.

[0184] See also Figure 4 , Figure 4 The schematic diagram is a structural diagram of a distributed storage system, which includes three metadata management nodes 10 (master node), 101 (backup node), 102 (backup node) and M data nodes (original nodes) 11, 12, 13, ..., each data node contains n data disks (original disks), and the metadata management node coordinates and controls the disk migration process.

[0185] Continue to see Figure 3 , which is a flow chart of determining the optimal migration plan through pre-calculation, and the optimal migration plan through capacity calculation.

[0186] See also Figure 5 , Figure 5 The following is a flowchart of disk data migration. Taking 4+2:1 erasure code as an example, the maximum number of blocks written to each data node is (2 / 1=2). Figure 5The number of blocks (block data) distributed on the data nodes is (2, 2, 1, 1) in sequence. When Disk 1 (Disk1) on DataNode1 is unplugged and plugged into DataNode2, DataNode2 finds that its NodeID (node ​​identification information) is 1, which is not equal to the node (node ​​preset information 2). It then considers that Disk 1 is the disk that needs to be migrated, and reports it to MetaServer (metadata database), along with the original disk configuration information NodeID-DiskID (1-1) and the new NodeID-DiskID (2-3). Upon receiving the message, MetaServer modifies the metadata information of the block originally on 1-1 to 2-3. After the metadata is updated, the distribution of a group of blocks becomes (1, 3, 1, 1). DataNode2 has 3 blocks, which exceeds the maximum allowed data of 2 and does not meet the 4+2:1 strategy. Based on load balancing, DataNode4 is selected as the target node for migrating data, and a copy of the data on DataNode2 is migrated to DataNode4. After completion, the block distribution will become (1, 2, 1, 2), meeting the 4+2:1 high reliability. After all data on the disk is processed, the disk (1-1) is updated to (2-3) to complete the disk migration.

[0187] See also Figure 6 For example, Figure 6 This is a disk management process diagram. The process is executed by the disk management module. The disk management module senses and triggers the disk migration process. The NodeID (node ​​identification information) and DiskID (disk identification information) will be recorded in the diskinfo.conf file of the disk. It is determined whether the diskinfo.conf file exists to determine whether it is an old disk (the original disk of the node is not a new disk). The NodeID and DiskID recorded in diskinfo.conf are read to determine whether it is the disk of this node. If not, it will be reported. After the disk migration is completed, the new NodeID and DiskID will be updated to diskinfo.conf. Specifically, Figure 6As shown, after the disk is inserted, the disk configuration file diskinfo.conf is read to obtain the NodeID (nodeid in the figure) and DiskID (diskid in the figure). If there is no diskinfo.conf file in the disk, it can be determined that the disk is a new disk online, and diskid is assigned to the disk, and nodeid and diskid are written to the diskinfo.conf file, where nodeid is the node preset information of the current node, and the disk is online. If there is a diskinfo.conf file in the disk, it means that the old disk is online, that is, the disk is the original disk, which may be the original disk on the current node or the original disk on another node. It can be determined whether the nodeid on the disk is equal to the nodeid (node ​​preset information) of the current node. If they are equal, the disk is online. If they are not equal, the disk information is reported, and after waiting for the disk metadata to be updated, diskid is assigned to the disk, and nodeid and diskid are written to the diskinfo.conf file. The disk is online and the disk management process is ended.

[0188] See also Figure 7 , Figure 7 A flowchart of metadata update is shown in Figure 1. Figure 7 As shown, first, the disk migration information is recorded in the TBL_DRIFT table of redis in the format of 1-1-2-3-0 (SrcNodeID-SrcDiskID-DesNodeID-DesDiskID-DriftStatu), the data recovery check for disk 1-1 is turned off, and the metadata is changed according to the disk information to be changed, and then TBL_DRIFT is updated to 1-1-2-3-1.

[0189] See also Figure 8 , Figure 8 A flowchart of data migration is shown in FIG. Figure 8As shown, obtain the disk with migration status 1 from the TBL_DRIFT table of Redis, obtain the partition data block information on the disk in turn, obtain the block information of the same group of blocks, calculate the maximum allowed write block number of a node (the maximum number of block data in the same group that can be written by the node) according to the erasure strategy, and determine whether there is a node in the cluster with a block number greater than the maximum allowed write block. If so, it means that data migration is required, and record the block migration information in the TBL_MIGRATE table of Redis, such as 1-1-2-3-0. Select the target node for migration according to load balancing, migrate the node with more block data on the disk to the node with less block data, and update the Redis table TBL_MIGRATE after migration, such as 1-1-2-3-1. After waiting for the target node to report that the migration is successful, delete the original data, and then re-execute to obtain the block information on the disk in turn until the block number of each node is less than or equal to the maximum allowed write number. If the number of blocks of each node is less than or equal to the maximum allowed write number, it means that data migration is not required. The disk migration information is deleted from the TBL_DRIFT table of Redis. The disk migration is completed, the disk data recovery function is enabled, and the process ends.

[0190] Continue with Figure 5 Take the example of as an example, the data migration process is: obtain the migration information 1-1-2-3-1 from the TBL_DRIFT table, query the block data on the disk (2-3) and the block data in the same group, calculate the maximum number of writable blocks for a node according to the erasure strategy, migrate the blocks on the node that are greater than the maximum writable number, and record the migrated block information in the TBL_MIGRATE table in the format of blockid+2-3-4-1-0, select the target node according to the load balancing, send the block information to the target node, and then update the TBL_MIGRATE table to 2-3-4-1-1, wait for the target node to report that the migration is completed, and then send the original data 2-3 block to delete the completed block from 2-3 to 4-1.

[0191] See also Fig. 9 This embodiment provides a disk migration system 900, which includes:

[0192] The acquisition module 901 is used to acquire the disk capacity, the newly added node information and the original node information of each original node in the cluster, and determine the number of migrated disks of each original node;

[0193] The to-be-migrated disk group determination module 902 is used to determine multiple to-be-migrated disk groups from the original node according to the number of migrated disks of the original node, and obtain multiple to-be-migrated disk groups corresponding to each original node;

[0194] The disk migration plan determination module 903 is used to respectively combine the disk groups to be migrated corresponding to the original nodes to obtain multiple disk migration plans;

[0195] The disk migration scheme determining module 904 is used to migrate the migrated disks in each disk migration scheme to the newly added node to obtain multiple disk migration schemes;

[0196] The preferred migration scheme determining module 905 is used to determine the number of block data to be migrated in each disk migration scheme, and use the disk migration scheme with the least number of block data to be migrated as the preferred migration scheme for disk migration.

[0197] In one embodiment, see Fig.10 , Fig.10 A schematic diagram of the structure of another disk migration system is shown in FIG. Fig.10 As shown, the disk migration system 1000 includes a pre-calculation module 1001, a disk management module 1002, a metadata update module 1003, and a data migration module 1004. The pre-calculation module includes an acquisition module, a disk group determination module to be migrated, a disk migration plan determination module, a disk migration plan determination module, and a preferred migration plan determination module. The pre-calculation module calculates which disks on the existing nodes in the cluster need to be migrated to the new migration node through the number and capacity of the disks of the newly added node, and gives the best solution (preferred migration plan). The disk management module is used to manage the online and offline of the disk, determine whether it is a disk on other data nodes and trigger disk migration, and execute the disk management process. The metadata update module is used to receive and process the disk migration request reported by the disk management module, update the metadata of the data written on the migration disk to the new node, and after completion, notify the data migration module to perform data migration and execute the metadata update process. The data migration module is used to query the data distribution of the same group for the data on the disk, migrate the data according to the current cluster redundancy strategy, and execute the disk data migration process.

[0198] In this embodiment, the system is essentially provided with multiple modules for executing the methods in the above embodiments. The specific functions and technical effects can be referred to the above method embodiments, which will not be described in detail here.

[0199] See also Fig.11 , an embodiment of the present invention further provides an electronic device 1100, including a processor 1101, a memory 1102 and a communication bus 1103;

[0200] The communication bus 1103 is used to connect the processor 1101 and the memory 1102;

[0201] The processor 1101 is used to execute the computer program stored in the memory 1102 to implement one or more methods described in the above-mentioned embodiment 1.

[0202] The embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon.

[0203] The computer program is used to enable a computer to execute any of the methods described in the first embodiment.

[0204] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of the steps included in the embodiment 1 of the embodiment of the present application.

[0205] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0206] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0207] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0208] The flow chart and block diagram in the accompanying drawings illustrate the possible implementation architecture, function and operation of the method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0209] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A disk migration method, characterized in that: The method comprises: Obtain the disk capacity, new node information and original node information in the cluster, and determine the number of migrated disks of each of the original nodes. The new node information includes the number of new nodes and the new node available capacity of each new node. The original node information includes the number of original nodes and the original node available capacity of each original node. Determine multiple disk groups to be migrated from the original disks of each original node according to the number of migrated disks of each original node, and obtain multiple disk groups to be migrated corresponding to each original node; Combine the disk groups to be migrated corresponding to the original nodes respectively to obtain multiple disk migration plans; Migrate the disks to be migrated in each disk migration plan to the newly added node to obtain multiple disk migration plans; Determine the amount of block data to be migrated of each disk migration scheme, and use the disk migration scheme with the least amount of block data to be migrated as the preferred migration scheme for disk migration; The method for determining the number of block data to be migrated in the disk migration solution includes: Get the maximum number of block data in the same group that can be written to the node and the number of block data in the same group of each cluster node in the disk migration solution. The cluster nodes include the newly added nodes and the original nodes. The number of block data to be migrated in the disk migration scheme is determined according to the maximum number of block data of the same group that can be written to the node and the number of block data of the same group of each cluster node in the disk migration scheme.

2. The method according to claim 1, characterized in that The number of disks to be migrated from the original node is determined by: Determine the total available capacity of the current cluster based on the available capacity of the newly added nodes and the available capacity of the original nodes; Determine the total number of current cluster nodes based on the number of newly added nodes and the number of original nodes; Determine the average total available capacity of each node in the current cluster based on the total available capacity of the current cluster and the total number of nodes in the current cluster; The available capacity of the original disks in each original node is obtained, and the number of disks to be migrated from the original node is determined based on the average total available capacity of each node in the current cluster and the available capacity of the original disks.

3. The method according to claim 1, characterized in that The disk migration scheme includes the migration disk slot information of the newly added node, the migration disk slot information of the original node, and the migration disk slot information of the original node. The method also includes: A disk plug-in prompt message is generated according to the migration disk slot information of the newly added node, the migration disk slot information of the original node, and the migration disk slot information of the original node of the preferred migration plan.

4. The method according to claim 3, characterized in that After plugging and unplugging at least a portion of the original disks and at least a portion of the newly added disks according to the preferred migration solution, the method further includes: Obtaining disk configuration information of each current disk in the cluster node, the cluster node includes the original node and the newly added node, and the current disk includes the original disk and the newly added disk; If the disk configuration information includes disk identification information and node identification information, and the node identification information is the same as the node preset information of the current node, the current disk is controlled to go online; If the disk configuration information includes disk identification information and node identification information, and the node identification information is different from the node preset information of the current node, new disk identification information and new node identification information are configured for the current disk, and the new disk identification information and new node identification information are determined as the disk configuration information, and the current disk is controlled to go online; If the disk configuration information does not include disk identification information and / or node identification information, the disk identification information and / or node identification information is configured for the current disk, and the disk identification information and node identification information are determined as the disk configuration information to control the current disk to go online.

5. The method according to claim 4, characterized in that Before controlling the current disk to go online, the method further includes: Get and record the disk configuration information of the current disk; If the disk configuration information includes disk identification information, node identification information, new disk identification information and new node identification information, the disk configuration information is updated to the new disk identification information and the new node identification information, and the current disk is notified to go online.

6. The method according to claim 1, characterized in that The method further comprises: Get the number of block data to be migrated for the preferred migration solution; If the number of block data to be migrated in the preferred migration solution is greater than zero, determine the node block data to be migrated of each cluster node according to the maximum number of block data in the same group that can be written to the node and the number of node block data in the same group of each cluster node in the preferred migration solution; At least one target node is selected from each cluster node according to the load balancing principle, and the node block data to be migrated of each cluster node is migrated to the target node.

7. The method according to claim 6, characterized in that Before selecting at least one target node from each cluster node according to the load balancing principle, the method further includes: Clean up the current disks on each cluster node.

8. A disk migration system, characterized in that: The system comprises: An acquisition module is used to acquire the disk capacity, new node information and original node information in the cluster, and determine the number of migrated disks of each of the original nodes. The new node information includes the number of new nodes and the new node available capacity of each new node, and the original node information includes the number of original nodes and the original node available capacity of each original node; The to-be-migrated disk group determination module is used to determine multiple to-be-migrated disk groups from the original disks of each original node according to the number of migrated disks of each original node, and obtain multiple to-be-migrated disk groups corresponding to each original node; A disk migration plan determination module is used to combine the disk groups to be migrated corresponding to each original node to obtain multiple disk migration plans; A disk migration plan determination module is used to migrate the migrated disks in each disk migration plan to the newly added node to obtain multiple disk migration plans; The preferred migration scheme determination module is used to determine the number of block data to be migrated for each disk migration scheme, and use the disk migration scheme with the least number of block data to be migrated as the preferred migration scheme for disk migration, wherein the method for determining the number of block data to be migrated for the disk migration scheme includes: obtaining the maximum number of block data that can be written by the node and the number of block data of the same group of nodes of each cluster node in the disk migration scheme, the cluster nodes including the newly added nodes and the original nodes; determining the number of block data to be migrated for the disk migration scheme according to the maximum number of block data that can be written by the node and the number of block data of the same group of nodes of each cluster node in the disk migration scheme.

9. An electronic device, characterized in that: including a processor, a memory and a communication bus; The communication bus is used to connect the processor and the memory; The processor is configured to execute the computer program stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is used to make a computer execute the method according to any one of claims 1 to 7.

Images