A dual-storage pool expansion method and a distributed storage system

Abstract

The application discloses a dual-storage pool expansion method and a distributed storage system, relates to the field of storage pool expansion, and sets a preset expansion strategy according to preset fault domain conditions and a preset primary OSD same node rule. When it is determined that the OSD in a new expansion node is added to the dual-storage pool for expansion, the expansion sequence is determined, the dual-storage pool is expanded based on the expansion sequence and the preset expansion strategy, so that the OSD in the new expansion node is added to the OSD of each first PG of the benchmark pool and the OSD of each second PG of the binding pool, and there are a preset number of groups of same node OSDs in the OSD of the first PG after expansion and the OSD of the second PG after expansion and corresponding binding. The same node OSDs coincide on the same physical node, which is beneficial to subsequent master selection, reduces network pressure, and each group of same node OSDs belongs to different nodes, meets the preset fault domain condition, and improves the performance of the distributed storage system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage pool expansion, in particular to a dual-storage pool expansion method and a distributed storage system. BACKGROUND

[0002] In a distributed storage system, a PG (Placement Group) is a carrier of a placement object, and one PG corresponds to the storage of multiple objects; each PG corresponds to an OSD (Object-based Storage Device), and one PG can correspond to multiple OSDs, and the multiple OSDs corresponding to the PG are referred to as member OSDs of the PG; meanwhile, multiple PGs are present on one OSD, so that the content to be stored is placed on a disk, and a storage pool is an aggregation of PGs. At present, different partitions are divided on a high-speed storage medium such as an SSD (Solid State Disk) to deploy multiple OSDs, and a high-speed cache storage pool (referred to as a high-speed pool) is created by using the OSDs; multiple OSDs are deployed on a low-speed storage medium such as an HDD (Hard Disk Drive) to create a low-speed large-capacity storage pool (referred to as a low-speed pool), and the high-speed pool and the low-speed pool are bound, and the two pools have a corresponding relationship of PGs, that is, a forwarding request of a PG of the high-speed pool can only be forwarded to a PG in the low-speed pool corresponding to the PG; each data request sent from a client passes through the high-speed pool, and then the high-speed pool decides whether to issue the data request to a pool for processing or to forward the data request to the low-speed pool for processing. The forwarding process is determined by a master in a member OSD of each PG, and only the master can bear the read-write service request. The master in the member OSD refers to selecting one of the member OSDs of the PG as a master OSD to bear the read-write service, and the other member OSDs are slave OSDs. Since only the master has the forwarding processing function, when the masters of two corresponding PGs are not on the same node, information needs to be transmitted through a network, and when the pressure of the storage system reaches a certain degree and the network reaches a bottleneck, the speed of forwarding is naturally limited, and the performance of the storage system cannot reach the expectation. Therefore, the masters corresponding to the bound PGs are adjusted to the same physical node as much as possible, and the specific method is as follows: on the premise that PG balancing and master PG balancing have been completed, a storage pool including fewer PGs is taken as a reference pool, the PG members of a storage pool including more PGs are changed, that is, the master of the PG of the reference pool is obtained, and then the corresponding PG in the bound pool is found according to the binding relationship, if the masters of the two PGs are not on the same physical node, a target PG on the same node as the master of the PG in the reference pool is found in the bound pool, and the members of the target PG and the corresponding PG members of the existing bound pool are exchanged as a whole, so that the PG balancing and the master PG balancing are not damaged, and the masters of the two corresponding bound PGs are on the same physical node as much as possible.

[0003] In the prior art, in order to realize reliable storage of data, there is a case of expansion of a dual storage pool (i.e. a high-speed pool and a low-speed pool corresponding to the high-speed pool). However, there is no good solution to reconstruct the primary of the corresponding PG on the same node as much as possible. Usually, the primary is adjusted to the same physical node by the above-mentioned method of adjusting the primary corresponding to the binding PG, which leads to the following problems during expansion: in addition to the data flow from the existing nodes in the storage system to the newly expanded node, there is also internal data reconstruction of the existing nodes due to the adjustment of the primary corresponding to the binding PG, that is, the internal data reconstruction caused by the exchange of a large number of members on the PG in the existing nodes. This redundant reconstruction is meaningless and is not conducive to practical application.

[0004] Therefore, how to provide a solution to the above technical problems to realize the expansion of the dual storage pool is a problem that those skilled in the art urgently need to solve. SUMMARY

[0005] The purpose of the present application is to provide a dual storage pool expansion method and a distributed storage system, which is beneficial to subsequent primary selection, reduces the number of network forwarding service data forwarding due to the non-coincidence of the primary of the corresponding PG, reduces network pressure, saves network bandwidth, has better practicality and applicability, and each group of same-node OSDs belongs to different nodes, meets the preset fault domain condition, and improves the performance and product competitiveness of the distributed storage system.

[0006] To solve the above technical problems, the present application provides a dual storage pool expansion method, comprising:

[0007] determining whether an OSD in a newly expanded node is added to the dual storage pool for expansion;

[0008] if yes, determining the expansion sequence of a reference pool and a binding pool corresponding to the reference pool in the dual storage pool;

[0009] expanding the dual storage pool based on the expansion sequence and a preset expansion strategy, so that the OSD in the newly expanded node is added to the OSD of each first PG of the reference pool and the OSD of each second PG of the binding pool, and there are a preset number of groups of same-node OSDs in the OSD of the first PG after expansion and the OSD of the second PG after expansion and corresponding binding, and each group of same-node OSDs belongs to different nodes; wherein the preset expansion strategy is set according to a preset fault domain condition and a preset primary OSD same-node rule.

[0010] Preferably, the expansion sequence comprises expanding the reference pool first and expanding the binding pool later.

[0011] performing capacity expansion on the dual storage pools based on the capacity expansion sequence and a preset capacity expansion strategy, including:

[0012] performing capacity expansion on the reference pool according to a first reference pool capacity expansion strategy;

[0013] performing capacity expansion on the binding pool according to a first binding pool capacity expansion strategy;

[0014] obtaining a first copy number of the reference pool and a second copy number of the binding pool;

[0015] when the first copy number and the second copy number are not equal, performing secondary capacity expansion on the binding pool based on the first copy number, the second copy number and a preset binding pool re-expansion strategy.

[0016] Preferably, performing capacity expansion on the reference pool according to a first reference pool capacity expansion strategy includes:

[0017] determining current PG reference numbers of all OSDs, the OSDs including OSDs of each first PG in the reference pool and OSDs in the new capacity expansion node;

[0018] determining, based on a preset theoretical bearing strategy, a number of PGs that each OSD of all OSDs should bear;

[0019] for each OSD, determining a set to which the OSD belongs according to the current PG reference number of the OSD and the corresponding number of PGs that the OSD should bear, the set including an over set, an under set, an avg set and an avg_more set;

[0020] for an i-th target OSD to be replaced in a first target set to be replaced, 1≤i≤total number of OSDs in the first target set to be replaced and i is an integer, the following steps are performed:

[0021] determining a target PG including the i-th target OSD to be replaced;

[0022] determining a first replacement OSD from a target replacement set, so as to replace the i-th target OSD to be replaced in the target PG with the first replacement OSD;

[0023] updating the set to which the i-th target OSD to be replaced belongs;

[0024] When the over set is non-empty and the under set is non-empty, the first target set to be replaced is the over set and the target replacement set is the under set; when the over set is non-empty and the under set is empty and the avg set is non-empty, the first target set to be replaced is the over set and the target replacement set is the avg set; when the over set is empty and the under set is non-empty and the avg_more set is non-empty, the first target set to be replaced is the avg_more set and the target replacement set is the under set.

[0025] Preferably, based on a preset theoretical load policy, the number of PGs that each OSD of all OSDs should carry is determined, including:

[0026] The single weight value of each OSD of all OSDs is determined.

[0027] The sum of each single weight value is determined as a total weight value.

[0028] Based on the total number of PGs in the reference pool, the total PG reference number is determined.

[0029] For each OSD, based on the single weight value corresponding to the OSD, the total weight value, the total PG reference number and a preset relationship, the number of PGs that the OSD should carry is determined.

[0030] The preset relationship is:

[0031]

[0032] Wherein, η is the number of PGs that the OSD should carry, A1 is the single weight value, B is the total weight value, and C is the total PG reference number. Indicates the result of rounding down.

[0033] Preferably, according to the current PG reference number of the OSD and the number of PGs that the OSD should carry, the set to which the OSD belongs is determined, including:

[0034] When the current PG reference number of the OSD is equal to the number of PGs that the OSD should carry, the set to which the OSD belongs is determined as the avg set.

[0035] When the current PG reference number is equal to the number of PGs that the OSD should carry plus a preset balance value, the set to which the OSD belongs is determined as the avg_more set.

[0036] ​When the current PG reference number is greater than the PG number to be carried, the set to which the OSD belongs is determined as an over set;

[0037] When the current PG reference number is less than the PG number to be carried, the set to which the OSD belongs is determined as an under set.

[0038] Preferably, the first binding pool is expanded once according to the first binding pool expansion strategy, comprising:

[0039] The set to which each OSD among all OSDs belongs is determined, the set comprising an over set, an under set, an avg set and an avg_more set; the all OSDs comprising the OSDs of each second PG in the binding pool and the OSDs in the newly expanded node;

[0040] For the jth first target OSD in the over set, 1≤j≤the total number of OSDs in the over set and j is an integer, the following steps are performed:

[0041] A first target binding PG including the jth first target OSD is determined;

[0042] For the sth first target binding PG, 1≤s≤the total number of first target binding PGs and s is an integer, the following steps are performed:

[0043] A first target reference PG corresponding to the sth first target binding PG and located in the reference pool is determined;

[0044] It is judged whether the first condition and the second condition are both established, the first condition being that the node corresponding to the OSD of the first target reference PG includes at least one of the newly expanded nodes; the second condition being that the node corresponding to the OSD of the first target reference PG is different from the node corresponding to the jth first target OSD;

[0045] If not, the expansion of the sth first target binding PG is skipped.

[0046] Preferably, when it is determined that the first condition and the second condition are both established, comprising:

[0047] A remaining binding PG corresponding to the sth first target binding PG is determined;

[0048] Among the OSDs of the remaining binding PG, the OSDs satisfying one of a third condition and a fourth condition are determined as candidate OSDs; wherein the third condition is the OSD whose node is the same as the node corresponding to the jth first target OSD; the fourth condition is the OSD whose node is different from the node corresponding to the OSD in the sth first target binding PG;

[0049] determining a remaining candidate OSD from all candidate OSDs;

[0050] determining whether a first assumption condition is met, the first assumption condition being that a current PG reference count of the candidate OSD is less than a set determination value when the candidate OSD is replaced, the set determination value being a sum of a number of PGs that should be borne by the candidate OSD and a preset balancing value, the candidate OSD being any one of the jth first target OSD and the remaining candidate OSD;

[0051] if not, determining a second replacement OSD from a new expansion node corresponding to an OSD in the first target reference PG based on the preset failure domain condition, to replace the remaining candidate OSD and the jth first target OSD in the sth first target binding PG with the second replacement OSD, and updating the set to which all OSDs belong;

[0052] if yes, skipping expansion of the sth first target binding PG and the remaining binding PG corresponding thereto.

[0053] Preferably, after updating the set to which all OSDs belong, the method further comprises:

[0054] when it is determined that the over set is non-empty and the under set is non-empty, replacing a third replacement OSD with an OSD in the new expansion node based on a preset over legacy elimination strategy, the third replacement OSD being an OSD in a binding PG to be replaced in the binding pool corresponding to an OSD in the current over set;

[0055] when it is determined that the over set is empty, the under set is non-empty, and the avg_more set is non-empty, replacing a fourth replacement OSD with an OSD in the new expansion node based on an avg_more elimination strategy and an avg_more legacy elimination strategy, the fourth replacement OSD being an OSD in a binding PG to be replaced in the binding pool corresponding to an OSD in the current avg_more set.

[0056] Preferably, based on the first number of replicas, the second number of replicas, and a preset binding pool re-expansion strategy, performing secondary expansion on the binding pool, comprising:

[0057] determining the set to which each of the OSDs belongs from all OSDs;

[0058] for a zth second target OSD in the over set, 1≤z

[0059] determining a second target binding PG including a zth second target OSD;

[0060] for a tth second target binding PG, 1≤t≤total number of the second target binding PGs and t is an integer, performing the following steps:

[0061] determining a second target reference PG corresponding to the tth second target binding PG and located in the reference pool;

[0062] when the first number of copies is greater than the second number of copies, determining whether a fifth condition and a sixth condition are both established, the fifth condition being that all nodes corresponding to the OSD of the tth second target binding PG are in the nodes corresponding to the OSD of the second target reference PG so that the nodes of the second number of copies are completely corresponding; the sixth condition being that in the nodes corresponding to the OSD of the second target reference PG, there is still the new expansion node except for the completely corresponding nodes of the second number of copies; if not, skipping expansion of the tth second target binding PG;

[0063] when the first number of copies is less than the second number of copies, determining whether the twelfth condition and the seventh condition are both established, the twelfth condition being that all nodes corresponding to the OSD of the second target reference PG are in the nodes corresponding to the OSD of the tth second target binding PG so that the nodes of the first number of copies are completely corresponding; the seventh condition being that the node corresponding to the zth second target OSD in the OSD of the tth second target binding PG is not in the completely corresponding nodes of the first number of copies except for the completely corresponding nodes of the first number of copies; if not, skipping expansion of the tth second target binding PG.

[0064] Preferably, the expansion sequence includes that the binding pool is expanded first and the reference pool is expanded last;

[0065] expanding the dual storage pool based on the expansion sequence and a preset expansion strategy, including:

[0066] expanding the binding pool according to the set to which each OSD in all OSDs belongs and a second binding pool expansion strategy, wherein all OSDs include the OSDs of each second PG in the binding pool and the OSDs in the new expansion nodes;

[0067] expanding the reference pool once according to a second reference pool expansion strategy;

[0068] when the first copy number of the benchmark pool is not equal to the second copy number of the binding pool, performing secondary expansion on the benchmark pool based on the first copy number, the second copy number, OSDs of each first PG in the benchmark pool after the first expansion, OSDs of each corresponding second PG in the binding pool after the first expansion, OSDs in the new expansion node, and a preset benchmark pool re-expansion strategy.

[0069] Preferably, the first expansion on the benchmark pool according to the second benchmark pool expansion strategy comprises:

[0070] determining a set to which each OSD among all OSDs on the benchmark pool and the new expansion node belongs; the set comprises an over set, an under set, an avg set, and an avg_more set;

[0071] for an xth third target OSD in the over set, 1≤x≤total number of OSDs in the over set and x is an integer, the following steps are performed:

[0072] determining a third target benchmark PG comprising the xth third target OSD;

[0073] for a yth third target benchmark PG, 1≤y≤total number of third target benchmark PGs and y is an integer, the following steps are performed:

[0074] determining a first target binding PG corresponding to the yth third target benchmark PG and located in the binding pool;

[0075] determining whether an eighth condition and a ninth condition are established, the eighth condition being that a node corresponding to the OSD of the first target binding PG comprises at least one new expansion node, and the ninth condition being that the node corresponding to the OSD of the first target binding PG is different from a node corresponding to the xth third target OSD;

[0076] if not, skipping expansion on the yth third target benchmark PG;

[0077] if yes, determining a fifth replacement OSD from the new expansion node corresponding to the OSD of the first target binding PG based on the preset failure domain condition, replacing the xth third target OSD in the yth third target benchmark PG with the fifth replacement OSD, and updating the set to which each OSD belongs.

[0078] To solve the above technical problems, the present application further provides a distributed storage system, comprising:

[0079] a memory for storing a computer program;

[0080] Processor, for executing the computer program to realize the steps of the dual storage pool expansion method as described above.

[0081] The application provides a dual storage pool expansion method and a distributed storage system. A preset expansion strategy is set according to preset fault domain conditions and a preset main OSD same node rule. When it is determined that the OSD in a new expansion node is added to the dual storage pool for expansion, the expansion sequence of a reference pool and a corresponding binding pool in the dual storage pool is determined. The dual storage pool is expanded based on the expansion sequence and the preset expansion strategy, so that the OSD in the new expansion node is added to the OSD of each first PG of the reference pool and the OSD of each second PG of the binding pool, respectively. There are a preset number of groups of same node OSDs in the OSD of the expanded first PG and the OSD of the corresponding expanded second PG. The same node OSDs coincide on the same physical node, which is beneficial to subsequent master selection, reduces the number of network forwarding times due to the non-coincidence of the masters of the corresponding PGs, reduces network pressure, saves network bandwidth, and has better practicability and applicability. In addition, each group of same node OSDs belongs to different nodes, meets the preset fault domain conditions, improves the performance and product competitiveness of the distributed storage system. BRIEF DESCRIPTION OF DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the prior art and the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0083] Figure 1 A flowchart of a dual storage pool expansion method provided by the present application;

[0084] Figure 2 A display schematic diagram of a PG corresponding relationship provided by the present application;

[0085] Figure 3 A display schematic diagram of a reference pool and a binding pool PG corresponding relationship provided by the present application;

[0086] Figure 4 A structure schematic diagram of a distributed storage system provided by the present application. DETAILED DESCRIPTION

[0087] The core of the present application is to provide a dual storage pool expansion method and a distributed storage system, which is beneficial to subsequent master selection, reduces the number of network forwarding service data forwarding due to the non-coincidence of the master of the corresponding PG, reduces the network pressure, saves the network bandwidth, has better practicability and applicability, and each group of same node OSDs belong to different nodes, meets the preset fault domain condition, and improves the performance and product competitiveness of the distributed storage system.

[0088] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0089] Please refer to Figure 1 , Figure 1 The present application provides a flowchart of a dual storage pool expansion method.

[0090] In the distributed storage system, the data to be stored is cut according to a fixed size, and the fixed-size data obtained is an object. A PG (Placement Group) is a carrier for placing objects, and its essence can be understood as the aggregation of multiple objects, which is a logical concept. One PG can correspond to the storage of multiple objects. Each PG is corresponded to an OSD (Object-based Storage Device) through a data distribution algorithm. One PG can correspond to multiple OSDs, and the corresponding multiple OSDs can be referred to as member OSDs of the PG. Meanwhile, multiple PGs can exist on one OSD. In this way, the content to be stored is placed on the disk, and the storage pool is the aggregation of PGs.

[0091] In the present embodiment, in order to realize reliable storage of data, the expansion of dual storage pools (i.e. a high-speed pool and a low-speed pool bound thereto) is considered in the prior art. However, there is no good solution to reconstruct the master of the corresponding PG on the same node as much as possible, and the master adjustment usually relies on adjusting the master corresponding to the bound PG to the same physical node, which will lead to redundant reconstruction and is not conducive to practical application. To solve the above technical problems, the present application provides a dual storage pool expansion method, which adjusts the OSD of the corresponding PG in the reference pool and the corresponding bound pool to the same node (the same node refers to the same physical node, such as the same server) as much as possible, reduces redundant reconstruction, and optimizes the storage performance of the distributed storage system.

[0092] The dual storage pool expansion method comprises:

[0093] S11: judging whether there is an OSD in a new expansion node added to the dual storage pool for expansion; if yes, going to S12;

[0094] Specifically, the dual storage expansion method can be applied to a distributed storage system, and the storage system is deployed with a dual storage pool, the dual storage pool includes a benchmark pool and a binding pool, and in the dual storage pool, the storage pool including a smaller total number of PGs is referred to as the benchmark pool, and the storage pool including a larger total number of PGs is referred to as the binding pool. For example, a high-speed pool is a 1024-PG three-copy type storage pool, and a low-speed pool is a 2048-PG 4+2 erasure type storage pool, so the high-speed pool is the benchmark pool, and the low-speed pool is the binding pool. A certain PG on the benchmark pool and a certain PG or several PGs in the binding pool form a unique up-and-down pool corresponding relationship.

[0095] The total number of the new expansion nodes can be one or multiple; the new expansion nodes include multiple OSDs, and the OSDs have not been corresponded to PGs; the total number of all OSDs in the dual storage pool and the identity of the OSDs such as the ID can be recorded in advance, and when it is determined that the total number is increased and the ID is increased, it is determined that the OSD in the new expansion node is added to the dual storage pool for expansion.

[0096] It should be noted that the unique up-and-down pool corresponding relationship of a certain PG on the benchmark pool and a certain PG or several PGs in the binding pool, i.e., the PG corresponding relationship, is further described as follows: no matter the benchmark pool or the binding pool, the total number of PGs included thereon is always an integer power of 2, so even if the total number of PGs of the benchmark pool and the binding pool is different, it is also a relationship of an integer multiple of 2. Therefore, the following PG corresponding relationship is always established: taking the benchmark pool pool1 with a smaller number of PGs as a benchmark, the PGs in the binding pool pool2 with a larger number of PGs can be divided according to the total number of PGs of the benchmark pool pool1, the number of PGs in each part of the division is equal to the total number of PGs of pool1, and each part is one-to-one corresponding to the PG of pool1, so the PG corresponding relationship is obtained; please refer to Figure 2 , Figure 2 A display schematic diagram of the PG corresponding relationship provided by the present application is provided, in which the binding pool pool2 includes 4096 PGs, the benchmark pool pool1 includes 1024 PGs, the binding pool pool2 is divided into 4 parts according to 1024, each part is corresponding to the 1024 PGs of the benchmark pool pool1, and the PG corresponding relationship is obtained, for example, PG0 in pool1 is corresponding to PG0, PG1024, PG2048 and PG2071 in pool2.

[0097] It should be further explained that, before the step S11, as a preferred preparation, the PGs corresponding to the reference pool and the binding pool can be as much as possible coincided on the same physical node under the premise of meeting the failure domain, for example, please refer to Figure 3 , Figure 3 The display schematic diagram of the PG corresponding relationship between the reference pool and the binding pool provided by the present application. As shown in Figure 3 , one PG in the reference pool is PG1.1, and the PGs corresponding to PG1.1 in the binding pool are PG2.1 and PG2.1025, wherein, host.A0 and host.A1 both represent the first physical node, host.B0 and host.B1 both represent the second physical node, host.C0 and host.C1 both represent the third physical node, and so on; the OSDs included in PG1.1 are [osd.a0, osd.b0, osd.c0, osd.d0, osd.e0, osd.f0], the OSDs included in PG2.1 are [osd.a1, osd.b1, osd.c1], and the OSDs included in PG2.1025 are [osd.a2, osd.b2, osd.c2]; the nodes corresponding to osd.a0, osd.a1, and osd.a2 are the same physical node, the nodes corresponding to osd.b0, osd.b1, and osd.b2 are the same physical node, and the nodes corresponding to osd.c0, osd.c1, and osd.c2 are the same physical node, it can be seen that when the master OSD of PG1.1 is osd.a0, the master OSD of PG2.1 is osd.a1, and the master OSD of PG2.1025 is osd.a2, the masters of the three will coincide on the same node, the master OSDs of osd.b0, osd.b1, and osd.b2 are selected, and the master OSDs of osd.c0, osd.c1, and osd.c2 are selected, and the OSDs corresponding to the PGs are as much as possible coincided on the same node, which can facilitate the master selection of the group of PGs, and it is easier to select the master of the group of PGs to the same physical node. The preset master OSD same node rule in the present application is set based on this point, so that the OSDs of the corresponding PGs after expansion are as much as possible coincided on the same physical node, to facilitate the master selection.

[0098] S12: determining the expansion sequence of the reference pool and the binding pool corresponding to the reference pool in the dual storage pool;

[0099] Specifically, the expansion sequence can be that the reference pool is expanded first and then the binding pool is expanded, or the binding pool is expanded first and then the reference pool is expanded.

[0100] S13: Perform expansion on the dual storage pools based on the expansion sequence and a preset expansion strategy, so that the OSDs in the newly expanded nodes are added to the OSDs of each first PG of the benchmark pool and the OSDs of each second PG of the binding pool, and in the OSDs of the first PG after expansion and the OSDs of the second PG after expansion and corresponding binding, there are a preset number of groups of same-node OSDs, and each group of same-node OSDs belongs to different nodes; wherein the preset expansion strategy is set according to a preset failure domain condition and a preset main OSD same-node rule.

[0101] Specifically, the preset failure domain condition essentially includes but is not limited to the node failure domain condition, that is, the OSDs under other nodes are available when any node fails, thereby minimally affecting the storage performance of the PG; the preset main OSD same-node rule essentially means that the OSDs in the corresponding PGs coincide on the same physical node as much as possible, the dual storage pools are expanded based on the expansion sequence and the preset expansion strategy, and the OSDs in the newly expanded nodes will be added to the OSDs of the first PG in the benchmark pool and the OSDs of the second PG in the binding pool. For the sake of description, if the nodes corresponding to the OSDs in the original benchmark pool and the nodes corresponding to the OSDs in the binding pool are all old nodes, the OSDs in the newly expanded nodes replace the original OSDs on the old nodes, so that the business data only flows from the old nodes to the new nodes during expansion, and the setting of the preset expansion strategy ensures good data balance and main correspondence, avoiding redundant reconstruction within the old nodes.

[0102] Further, the first PG in the benchmark pool has a corresponding binding second PG in the binding pool, which is the PG corresponding relationship described above; after expansion, in the OSDs of the first PG and the OSDs of the corresponding binding second PG, there are a preset number of groups of same-node OSDs, and each group of same-node OSDs belongs to different nodes. The preset data group can be: the minimum value of the first copy number of the benchmark pool and the second copy number of the binding pool; in the same group of same-node OSDs, the physical nodes corresponding to each OSD are all the same to facilitate master selection; one group of same-node OSDs corresponds to one physical node, and the physical nodes corresponding to each group are all different to meet the preset failure domain condition; for example, the OSDs of the first PG are [osd.a0, osd.b0, osd.c0], the OSDs of the second PG are [osd.a1, osd.b1, osd.c1], the nodes corresponding to osd.a0 and osd.a1 are the first node, the nodes corresponding to osd.b0 and osd.b1 are the second node, and the nodes corresponding to osd.c0 and osd.c1 are the third node, the first node, the second node, and the third node are three different nodes.

[0103] In summary, the application provides a dual-storage pool expansion method. A preset expansion strategy is set according to preset fault domain conditions and a preset main OSD same node rule. When it is determined that the OSD in the new expansion node is added to the dual-storage pool for expansion, the expansion sequence of the reference pool and the corresponding binding pool in the dual-storage pool is determined. The dual-storage pool is expanded based on the expansion sequence and the preset expansion strategy, so that the OSD in the new expansion node is added to the OSD of each first PG of the reference pool and the OSD of each second PG of the binding pool, and there are a preset number of groups of same node OSDs in the OSD of the expanded first PG and the OSD of the expanded and corresponding second PG. The same node OSDs coincide on the same physical node, which is beneficial to subsequent master selection, reduces the number of network forwarding service data forwarding due to the non-coincidence of the main corresponding PG, reduces the network pressure, saves the network bandwidth, and has better practicability and applicability. In addition, each group of same node OSDs belongs to different nodes, meets the preset fault domain condition, improves the performance of the distributed storage system, and improves the product competitiveness.

[0104] Based on the above-mentioned embodiments:

[0105] As a preferred embodiment, the expansion sequence includes reference pool expansion first and binding pool expansion later.

[0106] Based on the expansion sequence and the preset expansion strategy, the dual-storage pool is expanded, including:

[0107] The reference pool is expanded according to the first reference pool expansion strategy.

[0108] The binding pool is expanded once according to the first binding pool expansion strategy.

[0109] The first number of copies of the reference pool and the second number of copies of the binding pool are obtained.

[0110] When the first number of copies and the second number of copies are not equal, the binding pool is expanded twice based on the first number of copies, the second number of copies, and the preset binding pool re-expansion strategy.

[0111] In this embodiment, the expansion steps when the expansion sequence includes reference pool expansion first and binding pool expansion later are given. When the first number of copies of the reference pool and the second number of copies of the binding pool are not equal, the OSD on the new expansion node may still not meet the requirement of the number of PGs that should be carried after the binding pool is expanded once. Therefore, it is necessary to expand twice. The binding pool is expanded twice based on the first number of copies, the second number of copies, and the preset binding pool re-expansion strategy, so as to finally realize good data balance and main correspondence of the dual-storage pool after expansion.

[0112] As a preferred embodiment, based on the preset theoretical carrying strategy, the number of PGs that should be carried by each OSD in all OSDs is determined, including:

[0113] determining a single weight value of each of the OSDs;

[0114] determining a sum of the single weight values as a total weight value;

[0115] determining a total PG reference number based on a total number of the first PGs in the benchmark pool;

[0116] determining, for each of the OSDs, a number of PGs to be borne by the OSD based on the single weight value corresponding to the OSD, the total weight value, the total PG reference number, and a preset relationship formula;

[0117] The preset relationship formula is:

[0118]

[0119] wherein η is the number of PGs to be borne, A1 is the single weight value, B is the total weight value, and C is the total PG reference number. represents a result of rounding down.

[0120] In this embodiment, a determination manner of the number of PGs to be borne by each of the OSDs is given. Specifically, the single weight value of each of the OSDs is determined. The single weight value can be determined according to the capacity of the OSD and is proportional to the capacity. The greater the capacity, the greater the single weight value, and the more PGs and data can be borne. For example, the single weight value of a 1T capacity can be set to 1. Taking the benchmark pool as an example, the total PG reference number = the total number of PGs of the pool * the first copy number of the benchmark pool. The same applies to the binding pool. Of course, if the k+m redundancy mode is used in the binding pool, the total PG reference number = the total number of PGs of the pool * (k+m of the binding pool), wherein k is the number of data blocks, and m is the number of check blocks. Finally, the number of PGs to be borne by each of the OSDs is determined in combination with the preset relationship formula. An example is given here. Assuming that the disk capacities in the benchmark pool are the same, the weights are all 1, there are 3 nodes and 3 copies, each node has 6 OSDs, and the total number of PGs of the storage pool is 1024. Since it is 3 copies, the total PG reference number of the OSDs is 1024*3. Therefore, the number of PGs to be borne by a single OSD, i.e., the number of PGs to be borne = (1 / 3*6)*1024*3 = 170.

[0121] As a preferred embodiment, a set to which an OSD belongs is determined according to the current PG reference number of the OSD and the number of PGs to be borne by the OSD, including:

[0122] When the current PG reference number of the OSD is equal to the number of PGs to be borne by the OSD, it is determined that the set to which the OSD belongs is an avg set.

[0123] When the current PG reference number is equal to the number of PGs to be borne + a preset balancing value, it is determined that the set to which the OSD belongs is an avg_more set.​

[0124] When the current PG reference number is greater than the PG number to be carried, the set to which the OSD belongs is determined as an over set;

[0125] When the current PG reference number is less than the PG number to be carried, the set to which the OSD belongs is determined as an under set.

[0126] In the embodiment, the implementation of determining the set to which each OSD belongs is further given, which is described above and will not be repeated here. It should be noted that the preferred value of the preset balanced value is 1. The avg set represents that the OSD has been balanced. The avg_more set represents that the OSD is very close to the balanced target. The over set represents that the OSD carries too many PGs and is unbalanced. The under set represents that the OSD carries too few PGs and is unbalanced. It can be understood that, since the addition of the OSD on the new expansion node causes the number of PGs to be carried on the OSD of the old node to decrease, the OSD of the old node is in the over set. The OSD on the new expansion node is in the under set because it does not have a PG on it.

[0127] As a preferred embodiment, the reference pool is expanded according to the first reference pool expansion strategy, which comprises:

[0128] Determine the current PG reference number of all OSDs, including the OSDs of the first PGs in the reference pool and the OSDs in the new expansion node;

[0129] Based on a preset theoretical carrying strategy, determine the PG number to be carried by each OSD of all OSDs;

[0130] For each OSD, determine the set to which it belongs according to the current PG reference number of the OSD and the corresponding PG number to be carried, and the set includes an over set, an under set, an avg set and an avg_more set;

[0131] For the i-th target OSD to be replaced in the first target set to be replaced, 1≤i≤the total number of OSDs in the first target set to be replaced and i is an integer, the following steps are performed:

[0132] Determine the target PG including the i-th target OSD to be replaced;

[0133] Determine the first replacement OSD from the target replacement set, so as to replace the i-th target OSD to be replaced in the target PG with the first replacement OSD;

[0134] Update the set to which the i-th target OSD to be replaced belongs;

[0135] When the over set is non-empty and the under set is non-empty, the first target set to be replaced is the over set and the target replacement set is the under set; when the over set is non-empty and the under set is empty and the avg set is non-empty, the first target set to be replaced is the over set and the target replacement set is the avg set; when the over set is empty and the under set is non-empty and the avg_more set is non-empty, the first target set to be replaced is the avg_more set and the target replacement set is the under set.

[0136] In the embodiment, the step of expanding the benchmark pool under the expansion sequence of expanding the benchmark pool first and then expanding the binding pool is given, and details are shown in the above description. It should be noted that the step of determining the current PG reference number of all OSDs is illustrated as follows: assuming that the OSD of PG1 is (0, 2, 4), the OSD of PG2 is (2, 4, 8), the current PG reference number of OSD0 is 1, the current PG reference number of OSD2 is 2, the current PG reference number of OSD4 is 2, and the current PG reference number of OSD8 is 1. In addition, the step of determining the first replacement OSD from the target replacement set can preferentially select an OSD with a smaller current PG reference number in the target replacement set as the first replacement OSD.

[0137] It should be further noted that the specific execution logic of updating the set to which the ith target OSD to be replaced belongs is still to determine the current PG reference number thereof, to determine the set to which it belongs based on the current PG reference number thereof and the corresponding number of PGs to be carried, and the set includes the over set, the under set, the avg set and the avg_more set. The execution logic used in the following embodiments to update the set to which an OSD belongs is the same logic, and thus will not be described again.

[0138] As an example, the first target to-be-replaced set is the over set, the target replacement set is the under set, the OSD in the over set is replaced by the OSD in the under set, and the set to which the OSD belongs is updated constantly until the current PG reference number of an OSD in the over set is equal to the number of PGs it should carry plus a preset balancing value, and then the OSD is deleted from the over set and added to the avg_more set. Similarly, the current PG reference number of an OSD in the under set will gradually increase until it becomes equal to the number of PGs it should carry, and then the OSD is deleted from the under set and added to the avg set. For example, the i-th target to-be-replaced OSD is OSD0, the target PGs including OSD0 are PG1.0 and PG1.3, the OSD of PG1.0 is [0, 10, 20], the OSD of PG1.3 is [0, 11, 21], the under set includes [50, 51, 52, 54], and the first replacement OSD can be OSD50. Then, OSD50 is used to replace OSD0 in PG1.0, and the OSD of PG1.0 after replacement is [50, 10, 20].

[0139] As a preferred embodiment, the first binding pool is expanded once according to the first binding pool expansion strategy, including:

[0140] Determine the set to which each OSD in all OSDs belongs, and the set includes the over set, the under set, the avg set, and the avg_more set. All OSDs include the OSDs of each second PG in the binding pool and the OSDs in the newly expanded nodes.

[0141] For the j-th first target OSD in the over set, 1≤j≤the total number of OSDs in the over set and j is an integer, the following steps are performed:

[0142] Determine the first target binding PG including the j-th first target OSD.

[0143] For the s-th first target binding PG, 1≤s≤the total number of first target binding PGs and s is an integer, the following steps are performed:

[0144] Determine the first target reference PG corresponding to the s-th first target binding PG and located in the reference pool.

[0145] Determine whether the first condition and the second condition are both true, the first condition is that the node corresponding to the OSD of the first target reference PG includes at least one newly expanded node, and the second condition is that the node corresponding to the OSD of the first target reference PG is different from the node corresponding to the j-th first target OSD.

[0146] If not, skip the expansion of the s-th first target binding PG.

[0147] In this embodiment, the execution logic of the initial expansion of the binding pool after the reference pool expansion is completed is given. The sets to which the OSDs belong are determined to classify the OSDs. According to the characteristics of the binding pool described above, the s-th first target binding PG in the binding pool has a corresponding first target reference PG in the reference pool. Then, it is determined whether the first condition and the second condition are both established. Considering that the OSDs in the reference pool are not necessarily all expanded, such as the OSDs in the avg set, which may not be expanded. Therefore, the first condition is to determine whether the first target reference PG is expanded. As an example, assuming that the OSD of the first target reference PG is [50, 10, 20], the OSD of the s-th first target binding PG is [100, 110, 120, 130, 140, 150], and the node corresponding to OSD 50 is a newly expanded node, the first condition is established. The j-th first target OSD is OSD 100, and the node corresponding to OSD 100 is different from the nodes corresponding to OSD 50, OSD 10, and OSD 20. Therefore, the second condition is established. Otherwise, the expansion of the s-th first target binding PG is skipped, that is, although the s-th first target binding PG is in the over set, it is temporarily not expanded.

[0148] As a preferred embodiment, when it is determined that the first condition and the second condition are both established, it includes:

[0149] Determine the remaining binding PGs corresponding to the s-th first target binding PG.

[0150] Determine the OSDs in the remaining binding PGs that satisfy one of the third condition and the fourth condition as candidate OSDs. The third condition is that the OSDs have the same node as the node corresponding to the j-th first target OSD. The fourth condition is that the OSDs have different nodes from the nodes corresponding to the OSDs in the s-th first target binding PG.

[0151] Determine a remaining candidate OSD from all candidate OSDs.

[0152] Determine whether the first assumption condition is established. The first assumption condition is that the current PG reference number of the hypothetical OSD is less than the set determination value when the hypothetical OSD is replaced. The set determination value is the sum of the number of PGs that the hypothetical OSD should bear and the preset balancing value. The hypothetical OSD is any one of the j-th first target OSD and the remaining candidate OSD.

[0153] If no, based on the preset fault domain condition, a second replacement OSD is determined from a new expansion node corresponding to the OSD in the first target reference PG, to replace the remaining standby candidate OSD and the jth first target OSD in the s th first target binding PG with the second replacement OSD, and update the belonging set of all OSDs;

[0154] If yes, expansion of the s th first target binding PG and its corresponding remaining binding PG is skipped.

[0155] In this embodiment, for the step of determining the remaining binding PGs corresponding to the s th first target binding PG, it should be noted that the number of remaining binding PGs can be one or multiple; the remaining binding PGs can be understood as second PGs in the binding pool and corresponding to the first target reference PG as the s th first target binding PG. As described in the above embodiment, PG0 in pool1 corresponds to PG0, PG1024, PG2048 and PG2071 in pool2, pool1 is the reference pool, pool2 is the binding pool, PG0 in pool1 is the first target reference PG, PG0 in pool2 is the s th first target binding PG, and PG1024, PG2048 and PG2071 in pool2 are the remaining binding PGs, and the number of remaining binding PGs is three.

[0156] For the step of determining a remaining standby candidate OSD from all candidate OSDs, it can be specifically: selecting a candidate OSD with the largest current PG reference number from all candidate OSDs as the final remaining standby candidate OSD, and when there are multiple remaining binding PGs, one remaining binding PG corresponds to one remaining standby candidate OSD. Then, it is judged whether the first assumption condition is established, which is essentially to judge: assuming that the j th first target OSD is OSD10, there is only one remaining binding PG and its corresponding remaining standby candidate OSD is OSD10, then when the assumed OSD is set to OSD10, the current PG reference number of OSD10 will be reduced by 2 at one time, and after the reduction, it can be less than the sum of the number of PGs that OSD10 should bear and the preset balancing value, at this time, it is determined that the first assumption condition is established, and expansion of the s th first target binding PG and its corresponding each remaining binding PG is skipped; otherwise, it is determined that the first assumption condition is not established, and the subsequent replacement step can be performed.

[0157] In addition, the preset fault domain condition can be a node fault domain condition or a cabinet type fault domain condition (for example, the new expansion nodes are respectively in cabinet No. 1, cabinet No. 2 and cabinet No. 3, and the original OSD is selected in cabinet No. 1, so that the OSD selected from the new expansion nodes can only be selected on the OSDs in cabinet No. 2 and cabinet No. 3 to meet the cabinet type fault domain condition), the second replacement OSD is determined from the new expansion nodes corresponding to the OSD in the first target reference PG, when there is more than one new expansion node corresponding to the OSD in the first target reference PG, the node PG reference number of each new expansion node can be calculated, the new expansion node meeting the preset fault domain condition and having the minimum node PG reference number is selected as the final target new expansion node, and the second replacement OSD is selected on the target new expansion node. The number of the second replacement OSDs is multiple, and each of the second replacement OSDs is used to replace each of the remaining candidate OSDs and the jth first target OSD in the s th first target binding PG.

[0158] For example, still based on the example in the above embodiment, it is assumed that the first target reference PG is PG1.0, the OSD of PG1.0 is [50, 10, 20], the s th first target binding PG is PG2.0, the OSD of PG2.0 is [100, 110, 120, 130, 140, 150], the remaining binding PG of the s th first target binding PG is PG2.1, the OSD of PG2.1 is [101, 111, 121, 131, 141, 151], the node corresponding to OSD 50 is a new expansion node, the remaining candidate OSD determined by PG2.1 is OSD 101, the j th first target OSD is OSD 100, the node corresponding to OSD 100 is different from the node corresponding to OSD 50, the node corresponding to OSD 10 and the node corresponding to OSD 20, and when OSD 50 is a hypothetical OSD, the current PG reference number of OSD 50 is greater than or equal to the number of PGs to be borne + 1, the first assumption condition is not established, and expansion can be performed, so that the node corresponding to OSD 50 is the node for selecting the second replacement OSD, and the two OSDs on the node are used to replace OSD 100 of PG2.0 and OSD 101 of PG2.1.

[0159] As a preferred embodiment, after updating the belonging set of all OSDs, the method further comprises:

[0160] When it is determined that the over set is non-empty and the under set is non-empty, based on a preset over legacy elimination strategy, the third replacement OSD is replaced by the OSD in the new expansion node, the third replacement OSD is the OSD in the binding PG to be replaced in the binding pool corresponding to the OSD in the current over set;

[0161] In a case where it is determined that the over set is empty, the under set is non-empty, and the avg_more set is non-empty, OSDs in the newly expanded node are used to replace the fourth replacement OSD based on the avg_more elimination strategy and the avg_more residual elimination strategy. The fourth replacement OSD is an OSD in a binding pool corresponding to a to-be-replaced OSD in a binding PG in the current avg_more set.

[0162] In the embodiment, considering that the expansion of the s-th first target binding PG and its remaining binding PGs will be skipped in a case where the first assumption condition is determined to be true, there can still be residual OSDs after the sets to which all OSDs belong are updated, resulting in a non-empty over set. Therefore, in a case where it is determined that the over set is non-empty and the under set is non-empty (or the over set is non-empty and the avg set is non-empty), the preset over residual elimination strategy is used for processing. The processing steps are the same as those in the above embodiment, and the OSDs on the expanded node are still used to replace the target OSDs in the over set. However, in a case where the first assumption condition is determined to be true, the first target binding PG and other remaining binding PGs are all expanded, and the OSDs in the third replacement OSD are replaced by using the OSDs in the newly expanded node. The third replacement OSD is an OSD in a binding pool corresponding to a to-be-replaced OSD in a binding PG in the current over set. As an example, assuming that the target OSD is OSD100, the OSDs in the corresponding first target binding PG include the OSD100, the remaining binding PGs of the first target binding PG include PG1 and PG2, the remaining to-be-candidate OSD determined by PG1 is OSD100, the remaining to-be-candidate OSD determined by PG2 is OSD110, the first assumption condition is still true when OSD100 is the assumption OSD, and the first assumption condition is not true when OSD110 is the assumption OSD. Therefore, the OSD100 in the first target binding PG and the OSD110 in PG2 can be replaced to achieve expansion, and the OSD100 in PG1 cannot be replaced, that is, cannot be expanded.

[0163] Further, when the over set is empty, the under set is non-empty, and the avg_more set is non-empty, the processing steps are consistent with the above-mentioned embodiments, except that the first target OSD in the jth position in the over set is replaced by the first target OSD in the jth position in the avg_more set, and the OSD on the newly expanded node is still replaced. The preset avg_more elimination strategy is also consistent with the above-mentioned embodiments, except that the set determination value in the first assumption condition is changed to the number of PGs that the hypothetical OSD should bear. When the first assumption condition is determined to be true, the expansion of the s th first target binding PG and its remaining binding PGs is directly skipped. The avg_more legacy elimination strategy is consistent with the preset over legacy elimination strategy execution step in this embodiment, except that the set determination value in the first assumption condition is changed to the number of PGs that the hypothetical OSD should bear. When the first assumption condition is determined to be true, only the remaining binding PGs including the hypothetical OSD when the first assumption condition is true are not expanded, and the first target binding PG and other remaining binding PGs are all expanded. The following embodiments will make a general description of the reference pool expansion, the first binding pool expansion, and the second binding pool expansion.

[0164] As a preferred embodiment, based on the first copy number, the second copy number, and the preset binding pool re-expansion strategy, the binding pool is secondarily expanded, including:

[0165] Determining the set to which each OSD in all OSDs belongs;

[0166] For the zth second target OSD in the over set, 1≤z≤total number of OSDs in the over set and z is an integer, the following steps are performed:

[0167] Determining the second target binding PG including the zth second target OSD;

[0168] For the tth second target binding PG, 1≤t≤total number of second target binding PGs and t is an integer, the following steps are performed:

[0169] Determining the second target reference PG corresponding to the tth second target binding PG and located in the reference pool;

[0170] When the first copy number is greater than the second copy number, it is determined whether the fifth condition and the sixth condition are both true. The fifth condition is that all nodes corresponding to the OSD of the tth second target binding PG are in the nodes corresponding to the OSD of the second target reference PG, so that the nodes of the second copy number completely correspond. The sixth condition is that among the nodes corresponding to the OSD of the second target reference PG, in addition to the completely corresponding nodes of the second copy number, there are still newly expanded nodes. If not, the expansion of the tth second target binding PG is skipped;

[0171] When the first copy number is less than the second copy number, it is determined whether the twelfth condition and the seventh condition are both established. The twelfth condition is that all nodes corresponding to the OSDs of the second target reference PG are in the nodes corresponding to the OSDs of the tth second target binding PG, so that the nodes of the second copy number are completely corresponding. The seventh condition is that, except for the completely corresponding nodes of the first copy number, the node corresponding to the zth second target OSD in the OSD of the tth second target binding PG is not in the completely corresponding nodes of the first copy number. If not, the expansion of the tth second target binding PG is skipped.

[0172] In the embodiment, considering that, when the first copy number is different from the second copy number, after the binding pool is expanded once, the elimination of the under set cannot be realized generally, that is, the data is not balanced, so that the secondary expansion of the binding pool is needed. The specific steps are described above. All OSDs include the OSDs of the second PGs in the binding pool and the OSDs in the newly expanded nodes.

[0173] It should be noted that, when the first copy number is greater than the second copy number, if it is determined that the fifth condition and the sixth condition are established, the subsequent steps of determining that the first condition and the second condition are both established in the above embodiment can be performed, which is not described herein. When the first copy number is less than the second copy number, if it is determined that the twelfth condition and the seventh condition are both established, the subsequent steps of determining that the first condition and the second condition are both established in the above embodiment can be performed. In addition, it is assumed that the binding pool adopts a k+m erasure-type redundancy strategy, so that the second copy number of the binding pool is equal to the value of k+m in the value. K is the number of data blocks, and m is the number of check blocks.

[0174] Specifically, all nodes corresponding to the OSDs of the tth second target binding PG are in the nodes corresponding to the OSDs of the second target reference PG, so that the nodes of the second copy number are completely corresponding. It is assumed that the first copy number is 6, the second copy number is 3, the second target reference PG is PG1.0, the OSDs of PG1.0 are [0, 10, 20, 30, 40, 50], the tth second target binding PG is PG2.0, the OSDs of PG2.0 are [100, 110, 120], it is assumed that the node corresponding to OSD0 and OSD100 is the same node, that is, the first node, the node corresponding to OSD10 and OSD110 is the same node, that is, the second node, and the node corresponding to OSD20 and OSD120 is the same node, that is, the third node, which is referred to as the completely corresponding nodes of the second copy number. The node corresponding to OSD50 is a newly expanded node and is a newly expanded node except for the completely corresponding nodes of the second copy number (the first node, the second node, and the third node), so that it is determined that the fifth condition and the sixth condition are established.

[0175] Similarly, all the nodes corresponding to the OSDs of the second target reference PG are in the nodes corresponding to the OSDs of the tth second target binding PG, and the first copy number of nodes completely corresponding means that assuming the first copy number is 3, the second copy number is 6, the second target reference PG is PG1.0, the OSDs of PG1.0 are [0, 10, 20], the tth second target binding PG is PG2.0, the OSDs of PG2.0 are [100, 110, 120, 130, 140, 150], and the node corresponding to OSD0 is the same node as the node corresponding to OSD100, i.e., the first node, the node corresponding to OSD10 is the same node as the node corresponding to OSD110, i.e., the second node, and the node corresponding to OSD20 is the same node as the node corresponding to OSD120, i.e., the third node, it is called that the first copy number of nodes completely correspond; OSD130 is the zth second target OSD, the node corresponding to OSD130 is not in the first copy number of completely corresponding nodes (the first node, the second node, and the third node), i.e., the node corresponding to OSD130 is not any one of the first node, the second node, and the third node, then it is determined that the twelfth condition and the seventh condition are established.

[0176] As a summary of all the reference pools in the above-mentioned embodiments, first expansion, binding pool once expansion, and binding pool twice expansion, an exemplary description is given as a whole:

[0177] Logical (1) execution logic of first expansion of reference pool:

[0178] Step 1: Obtain the current PG reference number of all OSDs;

[0179] Step 2: Calculate the number of PGs that each OSD should carry;

[0180] Step 3: Classify the OSDs according to step 1 and step 2 to determine their belonging sets, including the over set, the under set, the avg set, and the avg_more set.

[0181] Step 4: Take the OSDs in the over set as target OSDs and take the OSDs in the under set as replacement OSDs to perform PG member adjustment. The specific steps are as follows:

[0182] 1) Traverse the OSDs in the source OSD set, and then traverse the PGs on the OSD, and replace the target OSDs in the OSDs of the PG with the replacement OSDs in the destination set in turn;

[0183] 2) Update the current PG reference number of the target OSD and the replacement OSD, and when the updated current PG reference number does not satisfy the current set condition of the target OSD and the replacement OSD, delete the target OSD and the replacement OSD from the current set and insert them into a new set;

[0184] Step 5: If the over set still exists OSD (i.e. not empty), but the under set is empty or the avg set is not empty, take the OSD in the over set as the target OSD, take the OSD in the avg set as the replacement OSD, repeat the rest of the process described in step 4.

[0185] Step 6: If the over set is empty, the under set is not empty and the avg_more set is not empty, take the OSD in the avg_more set as the target OSD, take the OSD in the under set as the replacement OSD, repeat the rest of the process described in step 4.

[0186] Step 7: After the above steps 4, 5, 6, the over set and the under set will be concentrated into the avg_more set and the avg set, and the expansion of the benchmark pool is completed.

[0187] The execution logic of the one-time expansion of the binding pool when the benchmark pool is expanded first:

[0188] Step 1: The one-time expansion process of the binding pool has been completed in the above step (1);

[0189] Step 2: The same as steps 1, 2, and 3 in the execution logic of the benchmark pool expansion first, determine the belonging set of all OSDs.

[0190] Step 3: According to the characteristics of the binding pool, the corresponding remaining binding pool PG and the corresponding benchmark pool PG of each PG in the binding pool can be found.

[0191] Step 4: Adjust the PG members by taking the OSD in the over set as the target OSD and the expansion node as the replacement node.

[0192] 1) Traverse the OSD in the over set, and then traverse the PG thereon, assuming PG1.

[0193] 2) First, determine whether there is a new expansion node in the node corresponding to the OSD of the benchmark pool PG corresponding to the PG1 (i.e. the first condition described in the above embodiment), and if so, continue to determine whether there is no node corresponding to the target OSD in the node corresponding to the OSD of the benchmark pool PG (i.e. the second condition described in the above embodiment), and if so, continue, otherwise skip the expansion of the PG1.

[0194] 3) According to the PG1, find the corresponding remaining binding PG in the binding pool, and traverse these remaining binding PGs.

[0195] 4) Traverse the OSDs on it, find the OSDs which are in the same node as the target OSD currently traversed (i.e. the third condition described in the above embodiment), or find the OSDs which are in neither the node of the OSD of PG1 nor the node of the OSD of the target OSD currently traversed (i.e. the fourth condition described in the above embodiment), as the candidate OSDs.

[0196] 5) Select one of the candidate OSDs with the largest current PG reference number as the remaining candidate OSD to be replaced.

[0197] 6) After the corresponding remaining binding PGs are traversed, the OSDs to be replaced in each of the remaining binding PGs and PG1 are obtained, the number of OSDs with the same ID in the OSDs to be replaced is counted, and the corresponding current PG reference number of the OSD to be replaced is reduced accordingly. In this stage, it is required to ensure that the current PG reference number of the OSD (i.e. the assumed OSD described in the above embodiment) after the reduction is greater than or equal to the number of PGs it should carry + 1 (i.e. the sum of the number of PGs it should carry and the preset balancing value described in the above embodiment). If it is less than, the expansion of the PG1 and the corresponding remaining binding PGs is skipped.

[0198] 7) When the OSD can be replaced, the expansion node is determined, the OSDs of the reference pool PG corresponding to the PG1 are traversed to obtain the corresponding new expansion node, the new expansion node satisfying the fault domain is selected, and when there are multiple new expansion nodes satisfying the fault domain condition, the target new expansion node with the least node PG reference count is selected, and the OSD to be replaced is selected from the target new expansion node to fill the target position in the PG1 and the corresponding remaining binding PGs.

[0199] 8) Then, the OSD set type is updated.

[0200] 9) When the OSDs in the new expansion node are all in the avg set or the avg_more set, the expansion ends.

[0201] Step 5: Similar to step 4 described above, if the over set is not empty and the under set is not empty (as an extension, if the over set is not empty and the avg set is not empty), the OSD in the over set is taken as the target OSD, the expansion node is taken as the replacement node, and step 4 is performed again.

[0202] The determination of the remaining binding PG corresponding to the PG1 is the same as step 4, and the selection of the new expansion node is also the same as step 4. However, if the current PG reference number of the OSD selected in the remaining binding PG of the PG1 is less than the number of PGs it should carry + 1 after the reduction, the remaining binding PG does not perform the expansion, but the PG1 and the other remaining binding PGs can perform the expansion.

[0203] Step 6: Similar to step 4, if the over set is empty and the avg_more set is non-empty, and the under set is non-empty, the OSD in the avg_more set is taken as the target OSD, and the expansion node is taken as the replacement node, and step 4 is performed again, and the following steps are performed. Among them:

[0204] The determination of the remaining binding PG corresponding to PG1 is the same as step 4, and the selection of the new expansion node is also the same as step 4, wherein when the current PG reference number of the OSD selected by PG1 and its corresponding remaining binding PG is reduced and is less than the number of PGs it should carry, the PG1 and its corresponding remaining binding PG are skipped for expansion (corresponding to the avg_more elimination strategy described in the above embodiment).

[0205] Step 7: Similar to step 6, if the over set is empty and the avg_more set is non-empty, and the under set is non-empty, wherein when the current PG reference number of the OSD selected by PG1 and its corresponding remaining binding PG is reduced and is less than the number of PGs it should carry, the remaining binding PG cannot be expanded, but the PG1 and other remaining binding PGs meet the first assumption condition and can be expanded (corresponding to the avg_more legacy elimination strategy described in the above embodiment).

[0206] Step 8: After the above steps 4, 5, 6, and 7, the benchmark pool is expanded first, and the binding pool is expanded once.

[0207] Logic (3) benchmark pool expansion first, binding pool expansion second, i.e. the execution logic of the second expansion of the binding pool:

[0208] a) When the first number of copies of the benchmark pool is greater than the second number of copies of the binding pool:

[0209] 1. The process is the same as the above logic (2), but in steps 4, 5, 6, and 7 of logic (2), the determination of the corresponding benchmark pool PG of PG1 is changed (i.e. the 2 in steps 4, 5, 6, and 7 is modified):

[0210] New 2): First determine whether the OSD corresponding node of the benchmark pool PG corresponding to PG1 is completely corresponding to the OSD corresponding node of PG1 (i.e. the fifth condition described in the above embodiment), if yes, continue to judge and remove the completely corresponding OSD, and there is still a new expansion node in the OSD corresponding node of the benchmark pool PG (i.e. the sixth condition described in the above embodiment), then the condition is met, and the process continues, otherwise, skip the expansion of PG1.

[0211] b) When the first number of copies of the benchmark pool is less than the second number of copies of the binding pool:

[0212] 1. The process is the same as the above logic (2), but in steps 4, 5, 6, and 7 of logic (2), the determination of the corresponding reference pool PG of PG1 is modified (i.e., the modification of 2 in steps 4, 5, 6, and 7).

[0213] First, determine whether the OSD corresponding node of the reference pool PG corresponding to PG1 is completely corresponding to the OSD corresponding node of PG1 (i.e., the fifth condition described in the above embodiment), if yes, and the target OSD corresponding node in the binding pool PG1 is not in the completely corresponding OSD corresponding node (i.e., the seventh condition described above), the condition is met, continue, otherwise skip the expansion of PG1; it can be understood that the modification for the selection of the new expansion node is that the new expansion node can be selected under the condition of meeting the fault domain, and it is not necessary to be limited to the OSD corresponding new expansion node in the reference pool PG corresponding to PG1, when there are multiple new expansion nodes, the target new expansion node with the least node PG reference number can be further selected for expansion.

[0214] As a preferred embodiment, the expansion sequence includes binding pool expansion first and reference pool expansion last.

[0215] Based on the expansion sequence and the preset expansion strategy, the dual storage pool is expanded, including:

[0216] According to the set to which each OSD in all OSDs belongs and the second binding pool expansion strategy, the binding pool is expanded, wherein all OSDs include the OSDs of each second PG in the binding pool and the OSDs in the new expansion nodes.

[0217] According to the second reference pool expansion strategy, the reference pool is expanded once.

[0218] When the first replica number of the reference pool and the second replica number of the binding pool are not equal, based on the first replica number, the second replica number, the OSDs of each first PG in the once-expanded reference pool, the OSDs of each corresponding second PG in the once-expanded binding pool, the OSDs in the new expansion nodes, and the preset reference pool re-expansion strategy, the reference pool is expanded twice.

[0219] In this embodiment, it is further considered that the expansion sequence can include binding pool expansion first and reference pool expansion last, and when the first replica number of the reference pool and the second replica number of the binding pool are not equal, there is a possibility of twice expansion of the reference pool, the specific execution steps are described above. In the step of expanding the binding pool according to the set to which each OSD in all OSDs belongs and the second binding pool expansion strategy, the second binding pool expansion strategy is similar to the first binding pool expansion strategy described in the above embodiment, except that it does not need to be positioned to the target reference PG. The execution logic of expanding the binding pool first is as follows:

[0220] determining the set to which each OSD of all OSDs belongs, the all OSDs including the OSDs of each second PG in the binding pool and the OSDs in the new expansion node;

[0221] for the lth target OSD in the over set, 1≤l≤total number of OSDs in the over set and l is an integer, performing the following steps:

[0222] determining a target binding PG including the lth target OSD;

[0223] for the oth target binding PG, 1≤o≤total number of target binding PGs and o is an integer, performing the following steps:

[0224] determining a remaining binding PG corresponding to the oth target binding PG;

[0225] determining, from the OSDs of the remaining binding PG, an OSD satisfying one of a fifteenth condition and a sixteenth condition as a candidate OSD; wherein the fifteenth condition (referring to the third condition) is an OSD whose node is the same as the node corresponding to the lth target OSD; and the sixteenth condition (referring to the fourth condition) is an OSD whose nodes are all different from the nodes corresponding to the OSDs in the oth target binding PG;

[0226] determining a remaining candidate OSD (optionally, a candidate OSD with the largest current PG reference number) from all candidate OSDs;

[0227] determining whether a second assumption condition is established, the second assumption condition being that the current PG reference number of the replacement OSD is less than the set determination value when the replacement OSD is assumed, the set determination value being the sum of the number of PGs that the replacement OSD should bear and a preset balancing value, and the replacement OSD being any one of the lth target OSD and the remaining candidate OSD;

[0228] if not, determining a replacement OSD from the new expansion node that satisfies a preset failure domain condition, to replace the lth target OSD in the oth target binding PG and the remaining candidate OSD, and updating the set to which each OSD of all OSDs belongs;

[0229] if yes, skipping expansion of the oth target binding PG and the remaining binding PG corresponding thereto.

[0230] When it is determined that the over set is non-empty and the under set is non-empty, the OSD on the expanded node is still replaced for the target OSD in the over set, and the expansion is performed based on the same preset over legacy elimination strategy. However, when the second assumption condition is established, in addition to the remaining binding PG corresponding to the established assumption OSD, the target binding PG and other remaining binding PGs can be expanded, such as the target binding PG PG2.0 and the OSD [100, 110, 120, 130, 140, 150]; the remaining binding PG corresponding to PG2.0 is PG2.1, and the OSD is [101, 111, 121, 131, 141, 151]. It is assumed that OSD100 and OSD101 are in the same node and are selected to be replaced. OSD100 is the current target OSD. If the current PG reference number of OSD101 is less than the number of PGs it should carry +1 after being reduced by 1, and the current PG reference number of OSD100 is greater than or equal to the number of PGs it should carry +1 after being reduced by 1, then PG2.0 can be expanded, but PG2.1 cannot be expanded.

[0231] When it is determined that the over set is empty, the under set is non-empty, and the avg_more set is non-empty, the expansion is performed based on the similar avg_more elimination strategy and the avg_more legacy elimination strategy. Details are described above and will not be repeated here. The execution logic corresponding to the binding pool first expansion, the benchmark pool first expansion, and the benchmark pool second expansion will be described in the following embodiments.

[0232] As a preferred embodiment, the second benchmark pool expansion strategy is used to expand the benchmark pool once, which includes:

[0233] Determine the set to which each OSD belongs among all OSDs on the new expanded node in the benchmark pool; the set includes the over set, the under set, the avg set, and the avg_more set;

[0234] For the xth third target OSD in the over set, 1≤x≤total number of OSDs in the over set and x is an integer, the following steps are performed:

[0235] Determine the third target benchmark PG including the xth third target OSD;

[0236] For the yth third target benchmark PG, 1≤y≤total number of third target benchmark PGs and y is an integer, the following steps are performed:

[0237] Determine the first target binding PG corresponding to the yth third target benchmark PG and located in the binding pool;

[0238] determining whether the eighth condition and the ninth condition are satisfied, the eighth condition being that the node corresponding to the OSD of the first target binding PG includes at least one newly expanded node, and the ninth condition being that the node corresponding to the OSD of the first target binding PG is different from the node corresponding to the xth third target OSD;

[0239] If not, skip expanding the yth third target reference PG;

[0240] If yes, based on a preset fault domain condition, determine a fifth replacement OSD from the newly expanded node corresponding to the OSD of the first target binding PG, to replace the xth third target OSD in the yth third target reference PG with the fifth replacement OSD, and update the belonging set of all OSDs.

[0241] In the embodiment, the execution logic of expanding the reference pool first and expanding the binding pool once is given, and details are shown in the above description. It should be noted that the first target binding PG refers to the first PG in the binding pool corresponding to the third target reference PG. For example, as described in the above embodiment, PG0 in pool1 corresponds to PG0, PG1024, PG2048 and PG2071 in pool2, pool1 is a reference pool, pool2 is a binding pool, PG0 in pool1 is a third target reference PG, PG0 in pool2 is a first target binding PG corresponding thereto, and PG1024, PG2048 and PG2071 in pool2 are remaining binding PGs corresponding thereto.

[0242] Specifically, as an example, it is assumed that the third target reference PG is PG1.0, the OSD of PG1.0 is [0, 10, 20], the first target binding PG corresponding thereto is PG2.0, the OSD of PG2.0 is [160, 110, 120, 130, 140, 150], the node corresponding to OSD160 is a newly expanded node (it can be seen that the eighth condition is satisfied), OSD0 is a target OSD, the node corresponding to OSD0 is different from the nodes corresponding to OSD160, OSD110, OSD120, OSD130, OSD140 and OSD150 (it can be seen that the ninth condition is satisfied), the node corresponding to OSD160 is a newly expanded node and satisfies the preset fault domain condition, a replacement OSD is selected from the node corresponding to OSD160, and the OSD0 in PG1.0 is replaced with the replacement OSD.

[0243] As in the above embodiment, update the belonging set of all OSDs, if the over set is not empty but the under set is empty, replace the OSD on the expanded node for the target OSD in the over set, repeat the above steps; if the over set is empty and the under set is not empty and the avg_more set is not empty, replace the OSD on the expanded node for the target OSD in the avg_more set, repeat the above steps.

[0244] In addition, when the first number of copies of the reference pool and the second number of copies of the binding pool are not equal, when the reference pool is expanded for the second time, the preset reference pool expansion strategy is similar to the above reference pool expansion execution logic, except that whether the eighth condition and the ninth condition are established is judged, and is replaced by:

[0245] When the first number of copies is greater than the second number of copies, it is judged whether the tenth condition and the eleventh condition are established, the tenth condition is that all nodes corresponding to the OSD of the first target binding PG are in the nodes corresponding to the OSD of the third target reference PG so that the second number of copies of nodes are completely corresponding, and the eleventh condition is that in addition to the completely corresponding nodes of the second number of copies, the nodes corresponding to the target OSD in the OSD of the third target reference PG are not in the completely corresponding nodes of the first number of copies; for example, all nodes corresponding to the OSD of the first target binding PG are in the nodes corresponding to the OSD of the third target reference PG so that the second number of copies of nodes are completely corresponding means: assuming that the first number of copies is 6, the second number of copies is 3, the third target reference PG is PG1.0, the OSD of PG1.0 is [0, 10, 20, 30, 40, 50], the first target binding PG is PG2.0, the OSD of PG2.0 is [100, 110, 120], and the node corresponding to OSD0 and OSD100 is the same node, i.e. the first node, the node corresponding to OSD10 and OSD110 is the same node, i.e. the second node, and the node corresponding to OSD20 and OSD120 is the same node, i.e. the third node, it is called that the second number of copies of nodes are completely corresponding, OSD50 is a target OSD, and the node corresponding to it is not in the second number of copies of completely corresponding nodes (the first node, the second node and the third node), which satisfies the eleventh condition.

[0246] When the first number of copies is less than the second number of copies, it is judged whether the thirteenth condition and the fourteenth condition are established, the thirteenth condition is that all nodes corresponding to the OSD of the third target reference PG are in the nodes corresponding to the OSD of the first target binding PG so that the first number of copies of nodes are completely corresponding; the fourteenth condition is that in the nodes corresponding to the OSD of the first target binding PG, in addition to the completely corresponding nodes of the first number of copies, there are still new expanded nodes.

[0247] As a summary of all the binding pool first expansion, benchmark pool first expansion, binding pool second expansion in the above embodiments, an exemplary description is given as a whole:

[0248] The execution logic of the logical (4) binding pool first expansion:

[0249] Step 1: Get the current PG reference number of all OSDs; calculate the number of PGs that each OSD should carry; classify the OSDs to determine their belonging set, which includes the over set, the under set, the avg set and the avg_more set.

[0250] Step 2: According to the characteristics of the binding pool, the corresponding remaining binding PGs of each PG in the binding pool can be found.

[0251] Step 3: Take the OSD in the over set as the target OSD, and take the expansion node as the replacement node to adjust the PG members.

[0252] 1) Traverse the OSD in the over set, and then traverse the PG on it, assuming PG1.

[0253] 2) According to the PG1, find the corresponding remaining binding PGs in the binding pool and the PGs, traverse these remaining binding PGs, traverse the OSDs on them, find the OSDs in the same node as the target OSD currently traversed (i.e. the fifteenth condition described in the above embodiments), or find the OSDs that do not overlap with the OSD of PG1 (i.e. the sixteenth condition described in the above embodiments), as candidate OSDs.

[0254] 3) Select an OSD with the largest current PG reference number from the candidate OSDs as the remaining candidate OSD to be replaced.

[0255] 4) After traversing these corresponding remaining binding PGs, each remaining binding PG and the OSD to be replaced in PG1 can be obtained, and the number of OSD IDs that are the same is counted in the OSD to be replaced. After the OSD is replaced, the corresponding current PG reference number will be reduced accordingly, and it is necessary to ensure that the current PG reference number of the OSD (i.e. the assumed OSD described in the above embodiments) is greater than or equal to the number of PGs it should carry +1 (i.e. the sum of the number of PGs it should carry and the preset balancing value) after the reduction. If it is less than, skip the expansion of PG1 and its corresponding remaining binding PG.

[0256] 5) When replaceable, determine the expansion node that meets the preset fault domain condition, when there are multiple such new expansion nodes, select the target new expansion node with the least current PG reference number of the node PG, and select the target position of the replacement OSD from the target new expansion node to fill in PG1 and the corresponding remaining binding PG.

[0257] 6) When the OSDs in the new expansion node are all in the avg or avg_more set, the expansion is completed.

[0258] Step 4: Similar to step 3 described above, if the over set is not empty and the under set is not empty (as an extension, if the over set is not empty and the avg set is not empty), the OSD in the over set is selected as the target OSD, and the expansion node is selected as the replacement node, and step 3 is performed again. Wherein:

[0259] If the selected OSD (i.e. the assumed OSD) in the corresponding remaining binding PG of PG1 is less than the number of PGs it should carry +1 after the current PG reference number is reduced, the corresponding remaining binding PG does not expand, but PG1 and other remaining binding PGs can expand.

[0260] Step 5: Similar to step 3 described above, if the over set is empty, the avg_more set is non-empty, and the under set is non-empty, the OSD in the avg_more set is selected as the target OSD, and the expansion node is selected as the replacement node, and step 3 is performed again. Wherein:

[0261] When the current PG reference number of the selected OSD of PG1 and its corresponding remaining binding PG is less than the number of PGs it should carry after being reduced, the PG1 and its corresponding PG are skipped.

[0262] Step 6: Similar to step 5, if the over set is empty, the avg_more set is non-empty, and the under set is non-empty, the OSD in the avg_more set is selected as the target OSD, and the expansion node is selected as the replacement node, and step 3 is performed again. Wherein:

[0263] When the current PG reference number of the selected OSD of PG1 and its corresponding remaining binding PG is less than the number of PGs it should carry after being reduced, the remaining binding PG cannot be expanded, but PG1 and the remaining binding PG meet the condition and can be expanded.

[0264] Step 7: After the above steps 3, 4, 5, and 6, the binding pool is expanded first. The execution logic of the logic (5) binding pool first expansion, benchmark pool one expansion:

[0265] Step 1: The expansion process of the logic (5) is based on the premise that the logic (4) is completed, and the benchmark pool is expanded once.

[0266] Step 2: Determine the set to which all OSDs belong.

[0267] Step 3: According to the characteristics of the dual storage pool, the first binding PG corresponding to each benchmark PG can be found.

[0268] Step 4: Take the OSD in the over set as the target OSD, and take the expansion node as the replacement node to adjust the PG members.

[0269] 1) Traverse the OSD in the over set, and then traverse the PG on it, assuming PG1.

[0270] 2) First, determine whether there is a new expansion node IP in the OSD corresponding to the first binding PG of PG1. If there is, it means that the target benchmark PG is expanded (i.e., the eighth condition described in the above embodiment), and continue to determine whether there is no node corresponding to the target OSD in the OSD of the first binding PG (i.e., the ninth condition described in the above embodiment). If yes, the condition is met, and the expansion of PG1 is continued. Otherwise, skip the expansion of PG1.

[0271] 3) Then determine the expansion node, traverse the first binding PG corresponding to PG1, and get the new expansion node that meets the preset fault domain condition. When there are multiple such new expansion nodes, select the one with the least node PG reference count as the target new expansion node, and select the replacement OSD from the node to fill in PG1.

[0272] 4) Then update the OSD set type.

[0273] 5) When all OSDs in the new expansion node are in the avg or avg_more set, the expansion is complete.

[0274] Step 5: If the over set is non-empty but the under set is empty, take the OSD in the over set as the target OSD, and take the expansion node as the replacement node to repeat the remaining process described in step 4 above.

[0275] Step 6: If the over set is empty, the under set is not empty, and the avg_more set is not empty, take the OSD in the avg_more set as the target OSD, and take the expansion node as the replacement node to repeat the remaining process described in step 4 above.

[0276] Step 7: The execution logic of the benchmark pool expansion once through steps 4, 5, and 6 above.

[0277] The logic (6) binds the pool to expand first, and the execution logic of the second expansion of the benchmark pool after the first expansion of the benchmark pool (based on the completion of the first expansion of the benchmark pool):

[0278] a) The number of benchmark pool copies (the number of data blocks plus the number of check blocks) is greater than the number of copies of the binding pool

[0279] 1. The process is the same as logic (5), but in steps 4, 5, and 6 of logic (5), the determination of the corresponding first binding PG of PG1 is modified (i.e., 2 in steps 4, 5, and 6 is modified):

[0280] First, determine whether the OSD of the corresponding first binding PG of PG1 corresponds to the OSD of PG1 (i.e., the tenth condition described in the above embodiment), and except for the corresponding node member, the source OSD in the benchmark pool PG1 is not in the corresponding node (i.e., the eleventh condition described in the above embodiment). If yes, continue, otherwise skip the expansion of PG1; the selection of the new expansion node is modified as follows: the expansion node that meets the preset fault domain condition can be selected, and it is not necessary to be limited to the new expansion node corresponding to the OSD in the first binding PG. When there are multiple such nodes, the node with fewer PG references is selected as the target new expansion node.

[0281] b) The number of benchmark pool copies (the number of data blocks plus the number of check blocks) is less than the number of copies of the binding pool

[0282] 1. The process is the same as logic (5), but in steps 4, 5, and 6 of logic (5), the determination of the corresponding first binding PG of PG1 is modified (i.e., 2 in steps 4, 5, and 6 is modified):

[0283] First, determine whether the OSD of the corresponding first binding PG of PG1 corresponds to the OSD of PG1 (i.e., the tenth condition described in the above embodiment), and except for the corresponding node member, the source OSD in the benchmark pool PG1 is not in the corresponding node (i.e., the eleventh condition described in the above embodiment). If yes, continue, otherwise skip the expansion of PG1; the selection of the new expansion node is modified as follows: the expansion node that meets the preset fault domain condition can be selected, and it is not necessary to be limited to the new expansion node corresponding to the OSD in the first binding PG. When there are multiple such nodes, the node with fewer PG references is selected as the target new expansion node.

[0284] Please refer to Figure 4 , Figure 4 A structure diagram of a distributed storage system provided by the present application.

[0285] The distributed storage system comprises:

[0286] The memory 21 is used to store a computer program;

[0287] The processor 22 is used to execute the computer program to realize the steps of the double storage pool expansion method as described above.

[0288] For the introduction of the distributed storage system provided in the present application, please refer to the above-mentioned embodiment of the dual-storage pool expansion method, which will not be repeated here.

[0289] The various embodiments described in the specification can be presented with respect to a single-pool expansion method, a dual-storage pool expansion method, or a distributed storage system. The various embodiments are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be mutually referred to. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part. The relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between the entities or operations. Moreover, the terms “include”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement “including a…” does not exclude the presence of another identical element in the process, method, article or device including the element.

[0290] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the disclosed embodiments can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for expanding the capacity of a dual storage pool, characterized in that, include: Determine if there are any OSDs from new expansion nodes added to the dual storage pool for expansion; If so, determine the expansion order of the base pool and the binding pool corresponding to the base pool in the dual storage pool; The dual storage pools are expanded based on the expansion order and a preset expansion strategy, so that the OSDs in the newly expanded nodes are added to the OSDs of each first PG in the baseline pool and the OSDs of each second PG in the bound pool. Furthermore, the expanded OSDs of the first PGs and the corresponding expanded OSDs of the bound second PGs contain a preset number of groups of OSDs from the same node, with each group belonging to a different node. The preset expansion strategy is set according to preset fault domain conditions and preset primary OSD same-node rules. The preset fault domain conditions include ensuring that if any node fails, there are still OSDs available from other nodes. The preset master OSD same node rule means that the OSDs in the corresponding PG are as close as possible to the same physical node. Based on the expansion order and preset expansion strategy, the dual storage pool is expanded. The OSDs in the newly expanded node will be added to the OSDs of the first PG in the baseline pool and the OSDs of the second PG in the binding pool. The first PG in the baseline pool and the second PG in the binding pool are in a PG correspondence relationship.

2. The dual-storage pool expansion method as described in claim 1, characterized in that, The expansion sequence includes expanding the base pool first and then expanding the binding pool; Expanding the dual storage pool based on the expansion order and preset expansion strategy includes: The reference pool is expanded according to the first reference pool expansion strategy; The binding pool is expanded once according to the first binding pool expansion strategy; Obtain the first replica count of the baseline pool and the second replica count of the binding pool; When the number of the first replicas is not equal to the number of the second replicas, the binding pool is expanded a second time based on the number of the first replicas, the number of the second replicas, and the preset binding pool expansion strategy.

3. The dual storage pool expansion method as described in claim 2, characterized in that, The reference pool is expanded according to the first reference pool expansion strategy, including: Determine the current PG reference count for all OSDs, including the OSDs of each first PG in the baseline pool and the OSDs in the newly expanded node; Based on the preset theoretical carrying strategy, determine the number of PGs that each OSD should carry in all OSDs; For each OSD, its set is determined based on the current PG reference count of the OSD and the corresponding number of PGs it should support. The set includes the over set, the under set, the avg set, and the avg_more set. For the i-th target OSD to be replaced in the first target replacement set, where 1 ≤ i ≤ the total number of OSDs in the first target replacement set and i is an integer, perform the following steps: Identify the target PG, including the target OSD to be replaced; Determine a first replacement OSD from the target replacement set, and use the first replacement OSD to replace the i-th target OSD to be replaced in the target PG; Update the set to which the i-th target OSD to be replaced belongs; Wherein, when the over set is not empty and the under set is not empty, the first target set to be replaced is the over set and the target replacement set is the under set; when the over set is not empty, the under set is empty and the avg set is not empty, the first target set to be replaced is the over set and the target replacement set is the avg set; when the over set is empty, the under set is not empty and the avg_more set is not empty, the first target set to be replaced is the avg_more set and the target replacement set is the under set; Specifically, the set to which the OSD belongs is determined based on its current PG reference count and the corresponding number of PGs it should support, including: When the current PG reference count of the OSD is equal to the number of PGs that the OSD should support, the set to which the OSD belongs is determined to be the avg set; When the current PG reference count equals the number of PGs to be supported plus the preset balance value, the set to which the OSD belongs is determined to be the avg_more set; When the current PG reference count is greater than the number of PGs that should be supported, the set to which the OSD belongs is determined to be the over set; When the current PG reference count is less than the number of PGs that should be supported, the set to which the OSD belongs is determined to be the under set.

4. The dual storage pool expansion method as described in claim 3, characterized in that, Based on a pre-defined theoretical carrying strategy, the number of PGs that each OSD should carry is determined, including: Determine the individual weight value for each OSD in all OSDs; The sum of all the individual weight values ​​is determined as the total weight value; The total number of PG references is determined based on the total number of PGs in all first PGs in the baseline pool; For each OSD, the number of PGs that the OSD should support is determined based on the individual weight value corresponding to the OSD, the total weight value, the total number of PG references, and a preset formula. The preset relation is: ； in, The number of PGs that should be carried, B is the single weight value, C is the total weight value, and D is the total number of PG references. Indicates to The result is rounded down.

5. The dual-storage pool expansion method as described in claim 2, characterized in that, The binding pool is expanded once according to the first binding pool expansion strategy, including: Determine the set to which each OSD belongs in all OSDs, including the over set, under set, avg set, and avg_more set; all OSDs include the OSDs of each second PG in the binding pool and the OSDs in the newly expanded node; For the j-th first target OSD in the over set, where 1 ≤ j ≤ the total number of OSDs in the over set and j is an integer, perform the following steps: Determine the first target binding PG, including the j-th first target OSD; For the s-th first target bound PG, 1≤s≤the total number of first target bound PGs and s is an integer, perform the following steps: Determine the first target reference PG that corresponds to the s-th first target binding PG and is located in the reference pool; Determine whether both the first and second conditions are met. The first condition is that the nodes corresponding to the OSD of the first target reference PG include at least one of the newly expanded nodes. The second condition is that the nodes corresponding to the OSD of the first target reference PG are different from the nodes corresponding to the j-th first target OSD. If not, skip the expansion of the PG bound to the s-th first target; Specifically, when the current PG reference count of the OSD is equal to the number of PGs that the OSD should support, the set to which the OSD belongs is determined to be the avg set; when the current PG reference count is equal to the number of PGs that should support plus a preset balance value, the set to which the OSD belongs is determined to be the avg_more set; when the current PG reference count is greater than the number of PGs that should support, the set to which the OSD belongs is determined to be the over set; and when the current PG reference count is less than the number of PGs that should support, the set to which the OSD belongs is determined to be the under set.

6. The dual-storage pool expansion method as described in claim 5, characterized in that, When determining that both the first and second conditions are met, the following applies: Identify the remaining bound PGs corresponding to the s-th first target bound PG; Among the remaining bound PG OSDs, the OSD that satisfies either the third condition or the fourth condition is identified as a candidate OSD; wherein, the third condition is that the node is the same as the node corresponding to the j-th first target OSD; and the fourth condition is that the node is different from the node corresponding to the OSD in the s-th first target bound PG. From all the candidate OSDs, determine one remaining candidate OSD; Determine whether the first assumption condition is true. The first assumption condition is that when the assumed OSD is replaced, its current PG reference count is less than the set judgment value. The set judgment value is the sum of the number of PGs that the assumed OSD should carry and the preset balance value. The assumed OSD is any one of the j-th first target OSD and the remaining candidate OSDs. If not, based on the preset fault domain conditions, a second replacement OSD is determined from the new expansion nodes corresponding to the OSD in the first target baseline PG, so as to replace the remaining candidate OSDs and the j-th first target OSD in the s-th first target bound PG with the second replacement OSD, and update the set to which all OSDs belong. If so, skip the expansion of the s-th first target bound PG and its corresponding remaining bound PGs.

7. The dual-storage pool expansion method as described in claim 6, characterized in that, After updating the set to which all OSDs belong, the process also includes: When it is determined that the over set is not empty and the under set is not empty, based on the preset over legacy elimination strategy, the OSD in the new expansion node is used to replace the third replacement OSD. The third replacement OSD is the OSD in the binding pool that is in the binding PG to be replaced, which corresponds to the OSD in the current over set. When it is determined that the over set is empty, the under set is not empty, and the avg_more set is not empty, based on the avg_more elimination strategy and the avg_more legacy elimination strategy, the OSD control in the new expansion node is used to replace the fourth replacement OSD. The fourth replacement OSD is the OSD in the binding pool that is in the binding PG to be replaced, corresponding to the OSD in the current avg_more set.

8. The dual-storage pool expansion method as described in claim 2, characterized in that, Based on the first number of replicas, the second number of replicas, and the preset binding pool expansion strategy, the binding pool is expanded a second time, including: Determine the set to which each OSD belongs in all OSDs; For the z-th second target OSD in the over set, where 1 ≤ z ≤ the total number of OSDs in the over set and z is an integer, perform the following steps: Identify the second target binding PG, including the z-th second target OSD; For the t-th second target bound PG, 1≤t≤the total number of second target bound PGs and t is an integer, perform the following steps: Determine the second target reference PG that corresponds to the t-th second target bound PG and is located in the reference pool; When the first number of replicas is greater than the second number of replicas, determine whether both the fifth and sixth conditions are met. The fifth condition is that all nodes corresponding to the OSD of the t-th second target bound PG are among the nodes corresponding to the OSD of the second target base PG so that the nodes of the second number of replicas are completely corresponding. The sixth condition is that among the nodes corresponding to the OSD of the second target base PG, apart from the nodes that are completely corresponding to the second number of replicas, there are still newly expanded nodes. If not, skip the expansion of the t-th second target bound PG. When the first number of replicas is less than the second number of replicas, determine whether both the twelfth and seventh conditions are met. The twelfth condition is that all nodes corresponding to the OSD of the second target base PG are among the nodes corresponding to the OSD of the t-th second target bound PG, so that the nodes of the first number of replicas are completely corresponding. The seventh condition is that, apart from the nodes that are completely corresponding to the first number of replicas, the node corresponding to the z-th second target OSD in the OSD of the t-th second target bound PG is not among the nodes that are completely corresponding to the first number of replicas. If not, skip the expansion of the t-th second target bound PG. The set includes the over set, the under set, the avg set, and the avg_more set; when the current PG reference count of the OSD is equal to the number of PGs that the OSD should support, the set to which the OSD belongs is determined to be the avg set; when the current PG reference count is equal to the number of PGs that should support plus a preset balance value, the set to which the OSD belongs is determined to be the avg_more set; when the current PG reference count is greater than the number of PGs that should support, the set to which the OSD belongs is determined to be the over set; when the current PG reference count is less than the number of PGs that should support, the set to which the OSD belongs is determined to be the under set.

9. The dual-memory pool expansion method according to any one of claims 1 to 8, characterized in that, The expansion sequence includes expanding the binding pool first and then expanding the baseline pool; Expanding the dual storage pool based on the expansion order and preset expansion strategy includes: The binding pool is expanded according to the set to which each OSD belongs and the second binding pool expansion strategy, wherein all OSDs include the OSDs of each second PG in the binding pool and the OSDs in the newly expanded node; The reference pool is expanded once according to the second reference pool expansion strategy; When the first replica count of the baseline pool is not equal to the second replica count of the binding pool, the baseline pool is expanded a second time based on the first replica count, the second replica count, the OSD of each first PG in the baseline pool after the first expansion, the OSD of each corresponding second PG in the binding pool after the first expansion, the OSD of the newly expanded node, and the preset baseline pool expansion strategy.

10. The dual-storage pool expansion method as described in claim 9, characterized in that, The reference pool is expanded once according to the second reference pool expansion strategy, including: Determine the set to which each OSD belongs among all OSDs in the baseline pool and the newly expanded node; the set includes the over set, the under set, the avg set, and the avg_more set; For the x-th third target OSD in the over set, where 1 ≤ x ≤ the total number of OSDs in the over set and x is an integer, perform the following steps: Determine the third target baseline PG, which includes the xth third target OSD; For the y-th third target benchmark PG, where 1 ≤ y ≤ the total number of third target benchmark PGs and y is an integer, perform the following steps: Determine the first target binding PG that corresponds to the y-th third target baseline PG and is located in the binding pool; Determine whether the eighth and ninth conditions are met. The eighth condition is that the node corresponding to the OSD of the first target bound PG includes at least one of the newly expanded nodes. The ninth condition is that the node corresponding to the OSD of the first target bound PG is different from the node corresponding to the xth third target OSD. If not, skip the expansion of the y-th third target baseline PG; If so, based on the preset fault domain conditions, a fifth replacement OSD is determined from the new expansion nodes corresponding to the OSD in the first target bound PG, so as to replace the xth third target OSD in the yth third target baseline PG with the fifth replacement OSD, and the set to which all OSDs belong is updated; When the current PG reference count of the OSD is equal to the number of PGs that the OSD should support, the set to which the OSD belongs is determined to be the avg set; when the current PG reference count is equal to the number of PGs that should support plus a preset balance value, the set to which the OSD belongs is determined to be the avg_more set; when the current PG reference count is greater than the number of PGs that should support, the set to which the OSD belongs is determined to be the over set; when the current PG reference count is less than the number of PGs that should support, the set to which the OSD belongs is determined to be the under set.

11. A distributed storage system, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the dual memory pool expansion method as described in any one of claims 1 to 10 when executing the computer program.

Images