A method, device, equipment and computer storage medium for generating a replication group

Abstract

An embodiment of the present application discloses a method, apparatus, device and computer storage medium for generating a replication group, wherein the method includes: determining multiple replication groups and multiple storage nodes for carrying the multiple replication groups, wherein the data in each replication group is distributed on the storage node carrying the replication group, and the multiple storage nodes are located on multiple hosts, and there are at least two hosts among the multiple hosts, and the number of storage nodes on the two hosts is different; based on the number of the multiple replication groups and the number of storage nodes required for each replication group, the data in each replication group in the multiple replication groups is distributed to at least one storage node in the multiple storage nodes, so that the difference between the number of replication groups carried on each storage node in the multiple storage nodes is less than a first threshold, and the number of storage nodes located on the same host in the at least one storage node carrying each replication group is less than a second threshold.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of Internet service technology, and are related to, but not limited to, a method, apparatus, device, and computer storage medium for generating a replication group. Background Art

[0002] In related technologies, distributed storage systems consist of homogeneous storage nodes. Homogeneous storage nodes refer to multiple storage nodes located on multiple hosts, with each host having the same number of storage nodes. Homogeneous storage systems facilitate load balancing and fault tolerance in distributed storage systems. Simply distributing data evenly across multiple hosts satisfies both load balancing and fault tolerance requirements.

[0003] However, as enterprise equipment upgrades, heterogeneous storage nodes are gradually being introduced into distributed storage systems. How to simultaneously meet the load balancing and fault tolerance requirements of distributed storage systems based on heterogeneous storage nodes has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a method, apparatus, device, and computer storage medium for generating a replication group.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for generating a replication group, the method comprising: determining a plurality of replication groups and a plurality of storage nodes for carrying the plurality of replication groups, wherein the data in each replication group is distributed on the storage node carrying the replication group, the plurality of storage nodes are located on a plurality of hosts, and there are at least two hosts among the plurality of hosts, and the two hosts have different numbers of storage nodes; based on the number of the plurality of replication groups and the number of storage nodes required for each replication group, the data in each replication group in the plurality of replication groups is allocated to at least one storage node in the plurality of storage nodes, so that the difference between the number of replication groups carried on each storage node in the plurality of storage nodes is less than a first threshold, and the number of storage nodes located on the same host in the at least one storage node carrying each replication group is less than a second threshold.

[0007] In a second aspect, an embodiment of the present application provides a device for generating a replication group, the device comprising: a determination module for determining multiple replication groups and multiple storage nodes for carrying the multiple replication groups, wherein the data in each replication group is distributed on the storage nodes carrying the replication group, the multiple storage nodes are located on multiple hosts, and there are at least two hosts among the multiple hosts, and the two hosts have different numbers of storage nodes; an allocation module for allocating the data in each replication group in the multiple replication groups to at least one storage node in the multiple storage nodes based on the number of the multiple replication groups and the number of storage nodes required for each replication group, so that the difference between the number of replication groups carried on each storage node in the multiple storage nodes is less than a first threshold, and the number of storage nodes located on the same host in the at least one storage node carrying each replication group is less than a second threshold.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the above method when executing the program.

[0009] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing executable instructions for causing a processor to execute and implement the above method.

[0010] In an embodiment of the present application, a plurality of replication groups and a plurality of storage nodes for carrying the plurality of replication groups are first determined, and then, based on the number of the plurality of replication groups and the number of storage nodes required for each replication group, the data in each replication group in the plurality of replication groups is distributed to at least one storage node in the plurality of storage nodes, so that the difference between the number of replication groups carried on each storage node in the plurality of storage nodes is less than a first threshold, and the number of storage nodes located on the same host in at least one storage node carrying each replication group is less than a second threshold. In this way, the difference between the number of replication groups carried by each storage node in the plurality of storage nodes can effectively meet the load balancing of the distributed storage system; the number of storage nodes located on the same host in at least one storage node carrying each replication group is less than the second threshold, and when any storage node in any replication group cannot be used due to host network disconnection, power outage, disk failure, misoperation, etc., the remaining storage nodes that are not on the same host can continue to work normally without causing data loss in the replication group. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1A A schematic diagram of an implementation flow of a method for generating a replication group provided in an embodiment of the present application;

[0012] Figure 1BA schematic diagram of four hosts and eight storage nodes provided in an embodiment of the present application;

[0013] Figure 1C A schematic diagram of six hosts and seven storage nodes provided in an embodiment of the present application;

[0014] Figure 2A A schematic diagram of an implementation flow of a method for generating a replication group provided in an embodiment of the present application;

[0015] Figure 2B A schematic diagram of six hosts and seven storage nodes generating seven replication groups according to an embodiment of the present application;

[0016] Figure 3 A schematic diagram of an implementation flow of a method for generating a replication group provided in an embodiment of the present application;

[0017] Figure 4A A schematic diagram of a process for generating all replication groups provided in an embodiment of the present application;

[0018] Figure 4B A schematic diagram of a process for generating any replication group provided in an embodiment of the present application;

[0019] Figure 5 A schematic diagram of the structure of a device for generating a replication group according to an embodiment of the present application;

[0020] Figure 6 A schematic diagram of a hardware entity of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the invention will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application but are not intended to limit the scope of the present application.

[0022] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0023] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0025] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0026] Distributed storage system: A storage system that runs on multiple storage nodes (data centers, storage hosts, or hard disks) connected through a network.

[0027] Replication group: A fault-tolerant unit in a storage system. A replication group consists of multiple nodes. When several nodes in a replication group fail or go offline, data in the replication group will not be lost.

[0028] Fault domain: The set of storage nodes affected by a failure. For example, a host's network disconnection will render all disks on that host unavailable. A distributed storage system that can continue to provide services despite a host failure is said to provide host-level fault domain tolerance.

[0029] Distributed Hash Table (DHT): A distributed storage system. Without the need for a server, each client is responsible for routing a small range and storing a small amount of data, thus achieving addressing and storage for the entire DHT network.

[0030] Ceph Distributed File System: is a unified, distributed file system designed for excellent performance, reliability, and scalability.

[0031] CRUSH: Ceph's algorithm for allocating storage nodes within a replication group.

[0032] A greedy algorithm (also known as a greedy algorithm) always chooses the best solution at the moment when solving a problem. In other words, it doesn't consider the overall optimal solution and only produces a locally optimal solution. While greedy algorithms don't always yield the optimal solution for all problems, they can produce an optimal solution or an approximation of the optimal solution for a wide range of problems.

[0033] Erasure coding (EC) is a data protection method that can split data into fragments, expand and encode redundant data blocks, and store them in different locations, such as disks, storage nodes, or other geographical locations.

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0035] It should be understood that some embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the technical scope of the present application.

[0036] The embodiment of the present application provides a method for generating a replication group, such as Figure 1A As shown, the method includes:

[0037] Step S101: Determine a plurality of replication groups and a plurality of storage nodes for hosting the plurality of replication groups, wherein data in each replication group is distributed on the storage nodes hosting the replication group, the plurality of storage nodes are located on a plurality of hosts, and at least two of the plurality of hosts have different numbers of storage nodes.

[0038] A replication group is a collection of multiple copies of a piece of data. Each copy is called a data replica, and each data replica in a replication group must be stored on a different host. A replication group is a fault-tolerant unit in a storage system. A replication group consists of multiple storage nodes, and different replicas of the same data can be stored on different storage nodes. Failure or offline of several storage nodes in a replication group does not result in data loss within the replication group. For example, erasure coding methods include the generation of replication groups.

[0039] Before generating multiple replication groups, you must first determine the number of storage nodes that will host the data in each replication group. Each replication group requires the same number of storage nodes. During implementation, the number of replication groups to generate and the number of storage nodes required for each replication group can be determined based on actual needs. For example, you might determine that seven replication groups are required, with three storage nodes per replication group.

[0040] The storage nodes to host the replication group can be storage nodes from different hosts or from the same host. Figure 1B In the four hosts h1, h2, h3, and h4 shown, each host includes two storage nodes. Storage nodes 11 and 12 are from the same host, as are storage nodes 21 and 22, storage nodes 31 and 32, and storage nodes 41 and 42. Storage nodes 21, 32, and 41 hosting replication groups are from different hosts.

[0041] A distributed storage system in which multiple storage nodes are located on multiple hosts and at least two of the multiple hosts have different numbers of storage nodes can be called a heterogeneous storage node storage system. Figure 1CThe multiple storage nodes to be used to host the replication group include storage node A on host h1, storage node B on host h2, storage nodes C and D on host h3, storage node E on host h4, storage node F on host h5, and storage node G on host h6. Here, the number of storage nodes on host h3 is different from the number of storage nodes on the other hosts, that is, Figure 1C The distributed storage system shown may be referred to as a heterogeneous storage node storage system.

[0042] Step S102: Based on the number of the multiple replication groups and the number of storage nodes required for each replication group, distribute the data in each replication group in the multiple replication groups to at least one storage node in the multiple storage nodes, so that the difference between the number of replication groups carried on each storage node in the multiple storage nodes is less than a first threshold, and the number of storage nodes located on the same host in at least one storage node carrying each replication group is less than a second threshold.

[0043] For example, the first threshold can be set to 1 based on actual needs, that is, the difference between the number of replication groups carried by each storage node in the multiple storage nodes is less than 1. In this way, the number of replication groups carried by each storage node in the multiple storage nodes is the same, that is, each storage node carries the same number of replication groups, thereby ensuring load balancing.

[0044] In some embodiments, a second threshold value of storage nodes located on the same host among at least one storage node carrying each replication group can be determined based on the number of storage nodes required for each replication group; because the larger the number of storage nodes required for the replication group, the better the fault tolerance performance. For example, in the case where the replication group requires 5 storage nodes, 2 storage nodes can be allowed to fail without affecting the data storage function of the replication group, that is, based on the number of storage nodes required for the replication group being 5, the second threshold value of storage nodes located on the same host among at least one storage node carrying each replication group can be determined to be 2.

[0045] In some embodiments, as Figure 1C As shown, multiple storage nodes include storage node A on host h1, storage node B on host h2, storage nodes C and D on host h3, storage node E on host h4, storage node F on host h5, and storage node G on host h6, where storage nodes C and D are located on the same host h3.

[0046] In some embodiments, the number of multiple replication groups can be determined to be 7 based on actual conditions, the number of storage nodes required for each replication group is 3, and the second threshold of the storage nodes located on the same host in at least one storage node carrying each replication group is 1.

[0047] Using the greedy algorithm, the allocation results are shown in Table 1 below:

[0048]

[0049]

[0050] Table 1

[0051] It can be seen from Table 1 above that storage nodes A, B, C, D, E, F and G each carry 3 of the 7 replication groups, that is, storage nodes A, B, C, D, E, F and G are evenly distributed to 7 different replication groups, and storage nodes C and D in the storage node group on the same host h3 do not carry data of the same replication group at the same time.

[0052] In an embodiment of the present application, a plurality of replication groups and a plurality of storage nodes for carrying the plurality of replication groups are first determined, and then, based on the number of the plurality of replication groups and the number of storage nodes required for each replication group, the data in each replication group in the plurality of replication groups is distributed to at least one storage node in the plurality of storage nodes, so that the difference between the number of replication groups carried on each storage node in the plurality of storage nodes is less than a first threshold, and the number of storage nodes located on the same host in at least one storage node carrying each replication group is less than a second threshold. In this way, the difference between the number of replication groups carried by each storage node in the plurality of storage nodes can effectively meet the load balancing of the distributed storage system; the number of storage nodes located on the same host in at least one storage node carrying each replication group is less than the second threshold, and when any storage node in any replication group cannot be used due to host network disconnection, power outage, disk failure, misoperation, etc., the remaining storage nodes that are not on the same host can continue to work normally without causing data loss in the replication group.

[0053] The embodiment of the present application provides a method for generating a replication group, such as Figure 2A As shown, the method includes:

[0054] Step S201: Determine a plurality of replication groups and a plurality of storage nodes for hosting the plurality of replication groups, wherein data in each replication group is distributed on the storage nodes hosting the replication group, the plurality of storage nodes are located on a plurality of hosts, and at least two of the plurality of hosts have different numbers of storage nodes.

[0055] Step S202: Determine the second threshold value according to the number of storage nodes required by each replication group;

[0056] During implementation, a second threshold for allocating storage nodes belonging to the same host to the same replication group can be effectively determined based on the number of storage nodes required for each replication group, so that the storage nodes allocated to the replication group meet failure domain requirements. For example, if the number of storage nodes required for each replication group is 3, the threshold is determined to be 1. That is, the number of storage nodes located on the same host among at least one storage node supporting each replication group is less than 1.

[0057] Step S203: sort the storage nodes in the plurality of storage nodes to obtain a candidate node list;

[0058] In some embodiments, the Figure 1C As shown, storage node A on host h1, storage node B on host h2, storage nodes C and D on host h3, storage node E on host h4, storage node F on host h5, and storage node G on host h6 are sorted to obtain a candidate node list ABCDEFG.

[0059] Step S204: Determine the storage node that ranks first in the candidate node list as the first storage node in order;

[0060] In some embodiments, as Figure 2B As shown, seven replication groups need to be generated based on the candidate node list ABCDEFG, with each replication group hosted by three storage nodes. Taking the generation of replication group 1 as an example, when generating the first storage node in the node list of replication group 1 and the candidate node list is ABCDEFG, A is determined as the first storage node; when generating the second storage node in the node list of replication group 1 and the candidate node list is BCDEFGA, B is determined as the first storage node; when generating the third storage node in the node list of replication group 1 and the candidate node list is CDEFGAB, C is determined as the first storage node.

[0061] Step S205: If it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is less than the second threshold, allocate the data in the current replication group to the first storage node;

[0062] In some embodiments, as Figure 2B As shown, taking the generation of replication group 1 as an example, when generating the second storage node in the node list of replication group 1 and the candidate list is BCDEFGA, it is necessary to determine whether the first storage node B and the first storage node A that has been generated belong to the same storage node group, that is, to determine whether storage node B and storage node A are deployed on the same host.

[0063] In some embodiments, when it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is less than 1, that is, when there is no storage node in the current replication group that is located on the same host as the first storage node, as shown in FIG. Figure 2B As shown, storage node B and storage node A are deployed on different hosts respectively, and the data in replication group 1 can be distributed to storage node B.

[0064] Step S206: Based on the number of storage nodes required for each replication group, determine that the first storage node is the last storage node in the current replication group, and based on the number of the multiple replication groups, determine that the current replication group is the last replication group among the multiple replication groups, and determine that the storage node allocation is completed for each replication group.

[0065] In some embodiments, as Figure 2B As shown, in the case of generating replication group 7, based on the fact that the number of storage nodes required for each replication group is 3, it can be determined that storage node G is the last storage node of replication group 7; based on the fact that the number of multiple replication groups is 7, and replication group 7 is determined to be the last replication group among the multiple replication groups, it is determined that the storage node allocation for the 7 replication groups is completed.

[0066] In an embodiment of the present application, the second threshold for allocating storage nodes belonging to the same host to the same replication group can be effectively determined based on the number of storage nodes required for each replication group, so that the storage nodes in the allocated replication group meet the requirements of the fault domain. In the event that any storage node in any replication group cannot be used due to host network disconnection, power outage, disk failure, misoperation, etc., the remaining storage nodes that are not on the same host can continue to operate normally and will not cause data loss in the replication group.

[0067] In an embodiment of the present application, multiple storage nodes are sorted to obtain a candidate node list; the storage node ranked first in the candidate node list is determined as the first storage node in order; and if it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is less than the second threshold, the data in the current replication group is allocated to the first storage node. In this way, based on the sorted candidate node list, the multiple storage nodes can be evenly allocated to each replication group, and the storage nodes allocated to the replication group meet the requirements of the failure domain.

[0068] In an embodiment of the present application, when the first storage node is determined to be the last storage node in the current replication group based on the number of storage nodes required by each replication group, and when the current replication group is determined to be the last replication group among the multiple replication groups based on the number of replication groups, the completion of storage node allocation for each replication group is determined. In this way, whether the allocation of storage nodes to each replication group is completed can be determined based on the number of storage nodes required by each replication group, and the completion of storage node allocation for each replication group can be determined based on the number of replication groups.

[0069] The embodiment of the present application provides a method for generating a replication group, such as Figure 3 As shown, the method includes:

[0070] Step S301: Determine a plurality of replication groups and a plurality of storage nodes for hosting the plurality of replication groups, wherein data in each replication group is distributed on the storage nodes hosting the replication group, the plurality of storage nodes are located on a plurality of hosts, and at least two of the plurality of hosts have different numbers of storage nodes.

[0071] Step S302: Determine the second threshold value according to the number of storage nodes required by each replication group;

[0072] Step S303: writing each storage node in the plurality of storage nodes into a candidate node list in a random order;

[0073] In some embodiments, multiple storage nodes may be written into the candidate node list in a random order. For example, Figure 1C The storage node A stored on host h1, the storage node B stored on host h2, the storage nodes C and D stored on host h3, the storage node E stored on host h4, the storage node F stored on host h5, and the storage node G stored on host h6 are randomly sorted to obtain a list of candidate nodes obtained by random sorting, such as ABCDEFG, EFABCDG, etc.

[0074] Step S304: Based on the number of replication groups carried by each storage node that has carried replication groups, rank the storage node with the smallest number of replication groups carried at the top of the candidate node list;

[0075] The candidate node list can record the number of replication groups that each storage node has hosted. When selecting storage nodes, the candidate nodes can be ranked first in ascending order of the number of replication groups each storage node appears in.

[0076] Step S305: Determine the storage node that ranks first in the candidate node list as the first storage node;

[0077] Step S306: If it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is less than the second threshold, allocate the data in the current replication group to the first storage node;

[0078] Step S307: After allocating the first storage node to the current replication group, move the first storage node to the end of the candidate node list to obtain an updated candidate node list;

[0079] In some embodiments, as Figure 2B As shown, based on the candidate node list ABCDEFG, seven replication groups need to be generated, each hosted by three storage nodes. Taking the generation of replication group 1 as an example, after assigning the first storage node A to replication group 1, storage node A is moved to the end of the candidate node list, resulting in an updated candidate node list of BCDEFGA.

[0080] Step S308: Based on the number of storage nodes required for each replication group, determine that the first storage node is the last storage node in the current replication group, and based on the number of the multiple replication groups, determine that the current replication group is the last replication group among the multiple replication groups, and determine that the storage node allocation is completed for each of the multiple replication groups.

[0081] In an embodiment of the present application, the candidate node list may record the number of replication groups that each storage node has hosted. When selecting a storage node, the candidate nodes may be ranked first in ascending order of the number of replication groups each storage node appears in, to ensure load balancing of the distributed storage system.

[0082] In an embodiment of the present application, each storage node in a plurality of storage nodes is written into a candidate node list in a random order, and the candidate node list can also be used to effectively generate multiple replication groups that meet load balancing and fault domain requirements.

[0083] An embodiment of the present application provides a method for generating a replication group, the method comprising:

[0084] Step 321: Determine a plurality of replication groups and a plurality of storage nodes for hosting the plurality of replication groups, wherein data in each replication group is distributed on the storage nodes hosting the replication group, the plurality of storage nodes are located on a plurality of hosts, and at least two of the plurality of hosts have different numbers of storage nodes.

[0085] Step 322: Sort each storage node in the plurality of storage nodes to obtain a candidate node list;

[0086] Step 323: Determine the storage node that ranks first in the candidate node list as the first storage node in order;

[0087] Step 324: If it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is equal to the second threshold, determine the storage node ranked second in the candidate node list as the first storage node.

[0088] In some embodiments, as Figure 2B As shown, in the case of generating the second storage node of replication group 4, it is determined that the first storage node D in the candidate node list DEFGAB and the first storage node C in replication group 4 belong to the same storage node group, that is, it is determined that the number of storage nodes in replication group 4 that are located on the same host as storage node D is equal to 1, and the storage node E ranked second in the candidate node list DEFGAB can be determined as the first storage node in order.

[0089] Step 325: If it is determined that the number of storage nodes located on the same host as the first storage node and the current replication group is less than the second threshold, assign the first storage node to the current replication group;

[0090] In some embodiments, as Figure 2B As shown, when generating the second storage node of replication group 4, if it is determined that the first storage node E and the first storage node C in replication group 4 do not belong to the same storage node group, E is allocated to replication group 4.

[0091] Step 326: Based on the number of storage nodes required for each replication group, if it is determined that the first storage node is not the last storage node in the current replication group, the first storage node is moved to the end of the candidate node list, and the position of the storage node ranked first in the candidate node list is fixed at the first position in the candidate node list, thereby obtaining an updated candidate node list.

[0092] In some embodiments, as Figure 2B As shown, when generating the second storage node E of replication group 4, storage node D is fixed at the first position in the candidate node list, resulting in an updated candidate node list DFGABCE. Similarly, when generating the third storage node F of replication group 4, storage node D is fixed at the first position in the candidate node list, resulting in an updated candidate node list DGABCEF.

[0093] Step 327: sequentially determine the storage node that ranks first in the updated candidate node list as the first storage node until the storage node allocation of the current replication group is completed;

[0094] When it is determined that the number of storage nodes belonging to the same storage node group as the first storage node and the current replication group is less than the second threshold, the first storage node is allocated to the current replication group; based on the number of storage nodes in each replication group, it is determined that the first storage node is the last storage node in the current replication group, and it is determined that the storage node allocation for the current replication group is completed.

[0095] In some embodiments, as Figure 2B As shown, when generating the first storage node of replication group 5, the storage node D ranked first in the updated candidate node list DGABCEF is determined as the first storage node. When generating the second storage node of replication group 5, if it is determined that the first storage node G and the storage node D in replication group 5 do not belong to the same storage node group, the first storage node G is allocated to replication group 5. When generating the third storage node of replication group 5, if it is determined that the first storage node A and the storage nodes D and G in replication group 5 do not belong to the same storage node group, the first storage node A is allocated to replication group 5, thereby completing the storage node allocation of replication group 5.

[0096] Step 328: If it is determined based on the number of the multiple replication groups that the current replication group is not the last replication group among the multiple replication groups, move the first storage node to the end of the candidate node list to obtain an updated candidate node list.

[0097] In some embodiments, as Figure 2B As shown, based on the number of multiple replication groups being 7, when it is determined that replication group 1 is not the last replication group among the 7 replication groups, the first storage node C is moved to the end of the candidate node list to obtain an updated candidate node list DEFGABC.

[0098] Step 329: Determine the storage node that ranks first in the updated candidate node list as the first storage node in order.

[0099] until the storage node allocation of the last replication group among the multiple replication groups is completed;

[0100] When it is determined that the number of storage nodes belonging to the same storage node group as the first storage node and the current replication group is less than the second threshold, the first storage node is allocated to the current replication group; when it is determined that the current replication group is the last replication group among the multiple replication groups based on the number of the multiple replication groups, it is determined that the storage node allocation is completed for each of the replication groups.

[0101] In some embodiments, as Figure 2B As shown, based on the number of replication groups being 7, and in the case where replication group 7 is determined to be the last replication group among the plurality of replication groups, it is determined that storage node allocation is completed for the 7 replication groups.

[0102] In an embodiment of the present application, when it is determined that the first storage node and any other storage node allocated to the current replication group to be allocated belong to the same storage node group, and the number of storage nodes in the current replication group that belong to the same storage node group as the first storage node is equal to the second threshold, the storage node ranked second in the candidate node list is determined as the first storage node in order. In this way, it can effectively avoid the number of storage nodes belonging to the same host in the same replication group being greater than the number threshold, so as to meet the replication group's requirements for the fault domain.

[0103] In an embodiment of the present application, based on the number of storage nodes required for each replication group, if it is determined that the first storage node is not the last storage node in the current replication group, the candidate node list is updated and storage nodes are continuously allocated to the replication group to be allocated until the required number of storage nodes for the replication group is met.

[0104] In an embodiment of the present application, when it is determined based on the number of the multiple replication groups that the current replication group is not the last replication group among the multiple replication groups, the candidate node list is updated and storage nodes are continuously allocated to the remaining replication groups among the multiple replication groups until the requirement on the number of replication groups among the multiple replication groups is met.

[0105] Distributed storage systems store data across different storage nodes. Due to network outages, power outages, disk failures, and operational errors, some storage nodes may experience data loss or unavailability. To ensure data security and availability, fault tolerance is a key feature of distributed systems.

[0106] Fault tolerance is typically achieved by storing redundant data in the system, for example, using a replication scheme to store different copies of the same data on all storage nodes in a replication group.

[0107] Since there are multiple storage nodes in the storage system, determining the data distribution of a certain data (specifically on which storage nodes the copies are stored) is a key issue in the distributed storage system.

[0108] like Figure 1B As shown, each piece of data will be stored in a certain replication group (including storage nodes 21, 32 and 42). In this way, the data is stored on each storage node in the replication group.

[0109] Distributed storage systems typically have several replication groups. There are two common ways to determine the list of nodes in a replication group:

[0110] Method 1: Computation-based: Representative methods include Ceph's CRUSH algorithm and DHT. These systems use the cluster topology as input and calculate the list of storage nodes in each replication group. Computation-based systems often result in different nodes belonging to different numbers of replication groups, leading to uneven load. Furthermore, DHT lacks robustness to fault domains (for example, requiring that nodes in a replication group not belong to the same host).

[0111] Method 2: Table-based lookup: The list of storage nodes corresponding to the replication group is stored in the metadata server. Before accessing data, the node list of the replication group must be queried. When using the table-based lookup method, the list of storage nodes in the replication group must be determined. Common algorithms for determining the node list include random algorithms and round-robin algorithms. Random algorithms, like computation-based methods, can cause uneven load. The round-robin algorithm performs well in homogeneous clusters, but it is difficult to simultaneously meet the requirements of load balancing and fault domains when using heterogeneous algorithms and special parameters.

[0112] The node list of a replication group based on table lookup must meet two conditions:

[0113] Condition 1: Meet the fault domain requirements: For example, the redundancy strategy tolerates single-node failure in the replication group, and the distributed storage system tolerates single-host failure in the replication group. This requires that each storage node in the replication group is distributed on a different host.

[0114] Condition 2: If condition 1 is met, all storage nodes in the same fault domain must be load balanced (the number of fault domains to which each storage node belongs must differ by at most 1). If load balancing of each storage node is not possible, load balancing must be achieved between the storage nodes within each host.

[0115] The embodiment of the present application uses a greedy algorithm to solve the problem of selecting storage nodes in the replication group, and needs to maintain the following data structures: 1. Candidate node list: the initial state is a list containing each storage node in any order; 2. Storage node list corresponding to each replication group: initially empty, and the result is returned after the algorithm is run.

[0116] Figure 4A A schematic diagram of a process for generating all replication groups is provided in an embodiment of the present application, such as Figure 4A As shown, the process of generating all replication groups includes the following steps:

[0117] Step S401: Initialize the candidate node list;

[0118] In some embodiments, all storage nodes in the cluster may be inserted into the candidate node list in any order.

[0119] Step S402: Determine whether all replication groups have been generated;

[0120] If it is determined that all replication groups have been generated, the process ends; if it is determined that all replication groups have been generated, go to step S403.

[0121] Step S403: Generate a list of storage nodes corresponding to each replication group in sequence.

[0122] In some embodiments, the number of replication groups in the replication group storage node list can be determined based on actual needs, and then multiple storage nodes corresponding to each replication group in the replication group storage node list can be generated in sequence. When it is determined that all replication groups in the replication group storage node list have been generated, the determination process ends.

[0123] In this embodiment of the present application, all storage nodes in the cluster are first inserted into the candidate node list in an arbitrary order. Then, when it is determined in the replication group storage node list that all replication groups have been generated, the process is terminated. In this way, a replication group storage node list that meets the required number of replication groups can be obtained.

[0124] Figure 4B A schematic diagram of a process for generating any replication group provided in an embodiment of the present application is shown as follows: Figure 4B As shown, the process of generating any replication group includes the following steps:

[0125] Step S411, initialize the storage node list to be empty;

[0126] Step S412: Determine whether the length of the storage node list is less than the number of storage nodes in each replication group;

[0127] In some embodiments, the number of storage nodes in each replication group can be set based on actual needs. Here, the number of storage nodes in each replication group determines the length of the storage node list. If the length of the storage node list is determined to be less than the number of nodes, the process proceeds to step S413. If the length of the storage node list is determined to be equal to the number of nodes, the process proceeds to step S416, completing the allocation of storage nodes to the replication group.

[0128] Step S413: Determine whether the first storage node in the candidate node list violates the failure domain requirement;

[0129] In some embodiments, the candidate node list is a list of all storage nodes used to configure the replication group, arranged in any order. A failure domain requirement violation can be determined by, for example, allocating three storage nodes to the same replication group, but not having more than one node belonging to the same host. That is, if the number of storage nodes belonging to the same host in the node list of the replication group exceeds one, then the failure domain requirement is violated.

[0130] When a higher number of replicas is used, the fault domain requirement becomes "no more than m (m ≥ 1) nodes in the same replication group can belong to the same host." That is, the fault domain determination at this time needs to be changed accordingly based on the actual situation.

[0131] If it is determined that the first storage node in the candidate node list does not violate the fault domain requirement, go to step S414; if it is determined that the first storage node in the candidate node list violates the fault domain requirement, go to step S415.

[0132] Step S414: add the first node in the candidate node list to the storage node list and move it to the end of the candidate node list;

[0133] When selecting nodes, you can try to add each node to the storage node list in ascending order of the number of nodes that appear in each replication group.

[0134] Step S415: The second storage node in the candidate node list is used as the storage node to be added to the storage node list;

[0135] In some embodiments, the first storage node in the candidate node list can be kept at the first position, and the second storage node in the candidate node list can be added as the storage node in the list of storage nodes to be added. After the second storage node is added to the replication group, it is moved to the end of the candidate node list. This ensures that the skipped node (the first storage node) will be selected first when selecting nodes for the next replication group, thus avoiding uneven load.

[0136] Step S416: End, return to the node list.

[0137] This embodiment of the present application describes how, when determining the number of replication groups in each of multiple replication groups, storage nodes used to configure the replication groups can be appropriately allocated to the replication groups based on a list of candidate nodes. This allows the aforementioned requirements for failure domains and load balancing to be met in any heterogeneous cluster configuration. The methods provided in this embodiment of the present application can also be used to determine storage nodes for replication groups in erasure coding.

[0138] Based on the foregoing embodiments, an embodiment of the present application provides a device for generating a replication group, which includes the modules included, each module includes sub-modules, each sub-module includes a unit, and can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0139] Figure 5 A schematic diagram of the composition structure of the device for generating a replication group provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the apparatus 500 includes:

[0140] A first determining module 501 is configured to determine a plurality of replication groups and a plurality of storage nodes for hosting the plurality of replication groups, wherein data in each replication group is distributed on the storage nodes hosting the replication group, the plurality of storage nodes are located on a plurality of hosts, and at least two of the plurality of hosts have different numbers of storage nodes.

[0141] The allocation module 502 is used to allocate the data in each of the multiple replication groups to at least one storage node among the multiple storage nodes based on the number of the multiple replication groups and the number of storage nodes required for each replication group, so that the difference between the number of replication groups carried on each storage node among the multiple storage nodes is less than a first threshold, and the number of storage nodes located on the same host in the at least one storage node carrying each replication group is less than a second threshold.

[0142] In some embodiments, the allocation module 502 includes a sorting submodule, a first determination submodule, a first allocation submodule and a second determination submodule, wherein the sorting submodule is used to sort each storage node in the multiple storage nodes to obtain a candidate node list; the first determination submodule is used to determine the storage node ranked first in the candidate node list as the first storage node in order; the first allocation submodule is used to allocate the data in the current replication group to the first storage node when it is determined that the number of storage nodes in the current replication group that belong to the same host as the first storage node is less than the second threshold; the second determination submodule is used to determine that the first storage node is the last storage node in the current replication group based on the number of storage nodes required for each replication group, and when it is determined that the current replication group is the last replication group in the multiple replication groups based on the number of the multiple replication groups, determine that the storage node allocation is completed for each of the replication groups.

[0143] In some embodiments, the first determination submodule includes an arranging unit and a determination unit, wherein the arranging unit is used to rank the storage node with the smallest number of carried replication groups at the top of the candidate node list based on the number of replication groups carried on each storage node that has carried replication groups; and the determination unit is used to determine that the storage node ranked first in the candidate node list is the first storage node.

[0144] In some embodiments, the sorting submodule is further configured to write each of the plurality of storage nodes into the candidate node list in a random order. The assigning module 502 further includes a moving submodule configured to, after assigning the first storage node to the current replication group, move the first storage node to the end of the candidate node list to obtain an updated candidate node list.

[0145] In some embodiments, the allocation module 502 also includes: a third determination submodule, a second allocation submodule and a first update submodule, wherein the third determination submodule is used to determine the storage node ranked second in the candidate node list as the first storage node in order when it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is equal to the second threshold; the second allocation submodule is used to allocate the first storage node to the current replication group when it is determined that the number of storage nodes in the current replication group that are located on the same host is less than the second threshold; the first update submodule is used to move the first storage node to the end of the candidate node list and fix the position of the storage node ranked first in the candidate node list at the first position in the candidate node list to obtain an updated candidate node list.

[0146] In some embodiments, the allocation module 502 also includes: a second update submodule and a fourth determination submodule, wherein the second update submodule is used to determine, based on the number of storage nodes required for each of the replication groups, that the first storage node is not the last storage node in the current replication group, and move the first storage node to the end of the candidate node list to obtain an updated candidate node list; the fourth determination submodule is used to determine the storage node that ranks first in the updated candidate node list as the first storage node in order until the storage node allocation of the current replication group is completed.

[0147] In some embodiments, the allocation module 502 also includes: a third update submodule and a fifth determination submodule, wherein the third update submodule is used to determine, based on the number of the multiple replication groups, that the current replication group is not the last replication group among the multiple replication groups, and move the first storage node to the end of the candidate node list to obtain an updated candidate node list; the fifth determination submodule is used to determine the storage node ranked first in the updated candidate node list as the first storage node in order until the storage node allocation of the last replication group among the multiple replication groups is completed.

[0148] In some embodiments, the apparatus further includes a second determination module configured to determine the second threshold value based on the number of storage nodes required by each replication group.

[0149] The description of the above device embodiment is similar to the description of the above method embodiment and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0150] It should be noted that, in the embodiment of the present application, if the above-mentioned method for generating a replication group is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a mobile phone, tablet computer, laptop computer, desktop computer, etc.) to execute all or part of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0151] Correspondingly, an embodiment of the present application provides an electronic device, Figure 6 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the hardware entity of the device 600 includes: a memory 601 and a processor 602, the memory 601 stores a computer program that can be run on the processor 602, and the processor 602 implements the steps in the method for generating a replication group provided in the above embodiment when executing the program.

[0152] The memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or processed by the processor 602 and various modules in the electronic device 600 (for example, image data, audio data, voice communication data and video communication data), which can be implemented through flash memory (FLASH) or random access memory (RAM).

[0153] Correspondingly, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the method for generating a replication group provided in the above embodiment are implemented.

[0154] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0155] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0156] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0158] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0159] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0160] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0161] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words be embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a mobile phone, tablet computer, laptop computer, desktop computer, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0162] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0163] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0164] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0165] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for generating a replication group, characterized in that: The method comprises: Determining a plurality of replication groups and a plurality of storage nodes for hosting the plurality of replication groups, wherein data in each replication group is distributed on the storage nodes hosting the replication group, the plurality of storage nodes are located on a plurality of hosts, and at least two of the plurality of hosts have different numbers of storage nodes on them; Allocate data in each of the multiple replication groups to at least one storage node among the multiple storage nodes based on the number of the multiple replication groups and the number of storage nodes required for each replication group, so that a difference between the number of replication groups carried by each storage node among the multiple storage nodes is less than a first threshold, and a number of storage nodes located on the same host in the at least one storage node carrying each replication group is less than a second threshold; the second threshold is determined based on the number of storage nodes required for each replication group; The allocating data in each of the plurality of replication groups to at least one of the plurality of storage nodes according to the number of the plurality of replication groups and the number of storage nodes required by each replication group includes: Sorting each storage node in the plurality of storage nodes to obtain a candidate node list; Determine the storage node that ranks first in the candidate node list as the first storage node; When it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is less than the second threshold, allocating the data in the current replication group to the first storage node; Based on the number of storage nodes required for each replication group, it is determined that the first storage node is the last storage node in the current replication group, and based on the number of the multiple replication groups, it is determined that the current replication group is the last replication group among the multiple replication groups, and it is determined that the storage node allocation is completed for each replication group.

2. The method according to claim 1, wherein The step of sequentially determining the storage node that ranks first in the candidate node list as the first storage node includes: Based on the number of replication groups carried by each storage node that has carried replication groups, arranging the storage node with the smallest number of carried replication groups at the top of the candidate node list; The storage node that ranks first in the candidate node list is determined as the first storage node.

3. The method according to claim 1, wherein The sorting of each storage node in the plurality of storage nodes to obtain a candidate node list includes: writing each storage node in the plurality of storage nodes into the candidate node list in a random order; The method further comprises: After allocating the first storage node to the current replication group, the first storage node is moved to the end of the candidate node list to obtain an updated candidate node list.

4. The method according to claim 1, wherein The method further includes allocating data in each of the plurality of replication groups to at least one storage node among the plurality of storage nodes based on the number of the plurality of replication groups and the number of storage nodes required for each replication group, such that a difference between the number of replication groups carried by each storage node among the plurality of storage nodes is less than a first threshold, and a number of storage nodes located on the same host among the at least one storage node carrying each replication group is less than a second threshold. If it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is equal to the second threshold, determine the storage node ranked second in the candidate node list as the first storage node; When it is determined that the number of storage nodes located on the same host as the first storage node and the current replication group is less than the second threshold, allocating the first storage node to the current replication group; The first storage node is moved to the end of the candidate node list, and the position of the storage node ranked first in the candidate node list is fixed at the first position in the candidate node list, thereby obtaining an updated candidate node list.

5. The method according to claim 1, wherein The method further includes allocating data in each of the plurality of replication groups to at least one storage node among the plurality of storage nodes based on the number of the plurality of replication groups and the number of storage nodes required for each replication group, such that a difference between the number of replication groups carried by each storage node among the plurality of storage nodes is less than a first threshold, and a number of storage nodes located on the same host among the at least one storage node carrying each replication group is less than a second threshold. Based on the number of storage nodes required by each replication group, if it is determined that the first storage node is not the last storage node in the current replication group, the first storage node is moved to the end of the candidate node list to obtain an updated candidate node list; The storage node that ranks first in the updated candidate node list is determined as the first storage node in order until the storage node allocation of the current replication group is completed.

6. The method according to claim 1, wherein The method further includes allocating data in each of the plurality of replication groups to at least one storage node among the plurality of storage nodes based on the number of the plurality of replication groups and the number of storage nodes required for each replication group, such that a difference between the number of replication groups carried by each storage node among the plurality of storage nodes is less than a first threshold, and a number of storage nodes located on the same host among the at least one storage node carrying each replication group is less than a second threshold. If it is determined based on the number of the multiple replication groups that the current replication group is not the last replication group among the multiple replication groups, move the first storage node to the end of the candidate node list to obtain an updated candidate node list; The storage node that ranks first in the updated candidate node list is determined as the first storage node in order until the storage node allocation of the last replication group in the multiple replication groups is completed.

7. A device for generating a replication group, characterized in that: The device comprises: A first determining module is configured to determine a plurality of replication groups and a plurality of storage nodes for hosting the plurality of replication groups, wherein data in each replication group is distributed on the storage nodes hosting the replication group, the plurality of storage nodes are located on a plurality of hosts, and at least two of the plurality of hosts have different numbers of storage nodes. an allocation module, configured to allocate data in each of the plurality of replication groups to at least one storage node among the plurality of storage nodes based on the number of the plurality of replication groups and the number of storage nodes required for each replication group, so that a difference between the number of replication groups carried by each storage node among the plurality of storage nodes is less than a first threshold, and a number of storage nodes located on the same host among the at least one storage node carrying each replication group is less than a second threshold; the second threshold is determined based on the number of storage nodes required for each replication group; The allocation module is further used to sort each storage node in the multiple storage nodes to obtain a candidate node list; determine the storage node ranked first in the candidate node list as the first storage node in order; when it is determined that the number of storage nodes in the current replication group that are located on the same host as the first storage node is less than the second threshold, allocate the data in the current replication group to the first storage node; based on the number of storage nodes required for each replication group, determine that the first storage node is the last storage node in the current replication group, and when it is determined that the current replication group is the last replication group in the multiple replication groups based on the number of the multiple replication groups, determine that the storage node allocation is completed for each of the replication groups.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the program, the steps in the method according to any one of claims 1 to 6 are implemented.

9. A computer storage medium, characterized in that Executable instructions are stored, which are used to cause a processor to execute the steps of the method according to any one of claims 1 to 6.

Images