Data distribution method and device of distributed storage system, equipment and storage medium

Abstract

The invention relates to the technical field of high availability of computer storage services, and discloses a data distribution method, device and equipment of a distributed storage system and a storage medium, the method comprises the following steps: configuring a plurality of object storage devices for a distributed storage system cluster, and calculating the expected total number of PGs of the cluster based on the expected number of PGs of each object storage device; calculating the number of non-repeated PGs which can be accommodated in the polynomial time based on the number of the object storage devices; and allocating the cluster PGs by adopting a mathematical graph theory based on a size relationship between the number of the receivable non-repeated PGs and the total number of the expected cluster PGs until all the expected total number of the cluster PGs are allocated. According to the method, if two disks are damaged, only one PG at most cannot be written, and if three disks are damaged, only one PG at most cannot be read, so that the risk of data loss is reduced. And if more disks are damaged, the data volume which is possibly lost can be greatly reduced, and the PG range which is possibly influenced when a plurality of data disks are damaged is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of high availability of computer storage services, and in particular to a data distribution method, device, equipment and storage medium of a distributed storage system. Background Art

[0002] Ceph is a distributed storage system that uniquely provides object, block, and file storage capabilities in a unified system. Ceph eliminates the reliance on a single central node in the system and implements the design concept of a decentralized structure. It can form a large cluster of multiple servers and integrate the disk resources of these machines to form a large resource pool.

[0003] In a production environment, Ceph cluster availability issues caused by disk damage are one of the most common problems encountered during the daily maintenance of cloud storage. For a Ceph storage pool that is most commonly set to 3 copies, when an OSD (Object Storage Device, referred to as OSD, object storage device) has a problem, the PG (placement group, referred to as PG) assigned to this OSD enters the Degraded state (the Degraded state refers to the state where there is a faulty or offline hard disk in the RAID system, but the system can still read and write data normally) and can still provide services. However, when two OSDs have problems at the same time, the PG that happens to have two copies distributed on the two OSDs with problems can no longer process requests. When three or more OSDs have problems at the same time, data loss may occur.

[0004] When an OSD error occurs due to disk damage, a certain amount of data migration will occur in the distributed storage system cluster, which will put pressure on other disks. Therefore, it is not uncommon for more than one disk to be damaged at the same time. Summary of the invention

[0005] In view of this, the present invention provides a data distribution method, device, equipment and storage medium of a distributed storage system to solve the problem of data loss caused by an increase in the range of PGs that may be affected when multiple data disks are damaged.

[0006] In a first aspect, the present invention provides a data distribution method for a distributed storage system, the method comprising:

[0007] Configure multiple object storage devices for the distributed storage system cluster, and calculate the total number of expected PGs in the cluster based on the expected number of PGs for each object storage device;

[0008] Calculate the number of non-repeated PGs that can be accommodated in polynomial time based on the number of object storage devices;

[0009] Based on the relationship between the number of non-duplicate PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster is allocated.

[0010] The data distribution method of the distributed storage system provided by the present invention calculates the number of non-repetitive PGs that can be accommodated based on the number of object storage devices; based on the size relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster is allocated. If two disks are damaged, at most only one PG cannot be written, and if three disks are damaged, at most only one PG cannot be read, thereby reducing the risk of data loss. If more disks are damaged, the amount of data that may be lost will also be greatly reduced, reducing the range of PGs that may be affected when multiple data disks are damaged, thereby improving the availability of the system and solving the problem of data loss caused by the increase in the range of PGs that may be affected when multiple data disks are damaged.

[0011] In an optional implementation, calculating the expected total number of PGs in the cluster based on the expected number of PGs of each object storage device includes:

[0012] Get the number of object storage devices in the distributed storage system cluster, and set the expected number of PGs and the maximum number of PG replicas for each object storage device;

[0013] The expected total number of PGs in the cluster is calculated based on the number of object storage devices, the expected number of PGs, and the maximum number of PG replicas. The result of calculating the expected total number of PGs in the cluster is rounded to a value equal to 2 raised to the power of N.

[0014] The data distribution method of a distributed storage system provided by the present invention obtains the number of object storage devices in a distributed storage system cluster, and sets the expected number of PGs and the maximum number of PG copies for each object storage device, thereby achieving the purpose of calculating the expected total number of PGs in the cluster based on the number of object storage devices, the expected number of PGs and the maximum number of PG copies. The result of calculating the expected total number of PGs in the cluster is rounded to a value equal to 2 to the Nth power, thereby improving the accuracy of the calculated expected total number of PGs in the cluster and providing conditions for the subsequent allocation of cluster PGs.

[0015] In an optional implementation, the non-duplicate PGs that can be accommodated are PGs of object storage devices where no more than one duplicate exists between any two PGs in the distributed storage system cluster;

[0016] The number of non-repeated PGs that can be accommodated in polynomial time based on the number of object storage devices includes:

[0017] Calculate the remainder of the ratio of the number of object storage devices to a preset constant;

[0018] Determine the number of PGs in the object storage device where more than one duplicate exists between any two PGs based on the value of the remainder;

[0019] The number of non-duplicate PGs that can be accommodated in polynomial time is calculated based on the number of object storage devices and the number of PGs in any two object storage devices where more than one duplicate PG exists.

[0020] In an optional implementation, the formula for calculating the number of non-repeating PGs that can be accommodated in polynomial time is as follows:

[0021]

[0022] Where Np is the number of non-duplicate PGs that can be accommodated in polynomial time, k is the number of PGs in object storage devices where any two PGs have more than one duplicate, n is the number of object storage devices, and C n Indicates the number of combinations, that is, the number of combinations of any two PGs selected from n individual object storage devices.

[0023] The data distribution method of the distributed storage system provided by the present invention determines the PG number of object storage devices where any two PGs have more than one duplicate according to the value of the remainder, thereby improving the accuracy of the PG number of object storage devices where any two PGs have more than one duplicate, and further improving the accuracy and availability of the calculation of the number of non-duplicate PGs that can be accommodated based on the number of object storage devices and the number of PGs of object storage devices where any two PGs have more than one duplicate.

[0024] In an optional implementation, before allocating cluster PGs based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster and using mathematical graph theory, the data distribution method of the distributed storage system further includes:

[0025] Each object storage device is regarded as a point on the plane, and any three points are not collinear. At the same time, three replicas are set for each PG, and the three replicas are respectively allocated to three object storage devices. The three object storage devices with allocated replicas are regarded as three points, and the three points are connected to form a triangle.

[0026] In an optional implementation, cluster PGs are allocated based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster and using mathematical graph theory until all the total number of PGs expected in the cluster is allocated, including:

[0027] When the number of non-duplicate PGs that can be accommodated is greater than or equal to the total number of PGs expected by the cluster, the storage node selection algorithm is used to select three object storage devices for the first three copies of the total number of PGs expected by the cluster, and mathematical graph theory is used to find the first triangle in the graph;

[0028] Continue to allocate the second PG in the total number of PGs expected in the cluster, and use the storage node selection algorithm to select three object storage devices for the three copies of the second PG in the total number of PGs expected in the cluster. Use mathematical graph theory to find the second triangle, compare the second triangle with the first triangle, and determine whether there are two or more duplicate object storage devices, that is, whether there are co-side triangles; if so, discard the PG allocation result, increase the number of calculations by one, and re-allocate the second PG;

[0029] Continue to allocate the third PG in the total number of expected PGs of the cluster and find non-colateral triangles until all PGs in the total number of expected PGs of the cluster are allocated, or no more non-colateral triangles are found, and the first non-colateral triangle set and PG allocation results are obtained.

[0030] The data distribution method of the distributed storage system provided by the present invention achieves the purpose of ensuring that at most only one PG enters an unavailable state when two disks are damaged or the object storage device OSD is down, that is, any two PGs cannot have two or more copies distributed in the same object storage device OSD, and the allocation of PG copies can be optimized, so that when two disks of the Ceph cluster storage pool are damaged or the object storage device OSD is down, at most only one PG enters an unavailable state, and when there are three disks, at most only one PG will experience data loss, thereby reducing the range of PGs that may be affected when multiple data disks are damaged, thereby improving the availability of the distributed storage system and reducing the risk of data loss.

[0031] In an optional implementation, based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster is allocated, and the following is further included:

[0032] When the number of PGs in the PG allocation result is less than the total number of PGs expected by the cluster, traverse the first non-colateral triangle set from small to large numbers, select any specified triangle, and then traverse all object storage devices to determine whether two non-colateral triangles with other triangles in the first non-colateral triangle set except the specified triangle can be found. If they can be found, remove the object storage device corresponding to the specified triangle from the first non-colateral triangle set to obtain the second non-colateral triangle set;

[0033] Continue to traverse all triangles in the second non-colateral triangle set until at least 1 / 2 of the maximum number of non-colateral triangles is found.

[0034] The data distribution method of the distributed storage system provided by the present invention traverses the largest non-colateral triangle in the non-colateral triangle set through a greedy algorithm, and finds no less than 1 / 2 of the total number of non-colateral triangles in polynomial time, thereby reducing the range of PGs that may be affected when multiple data disks are damaged. When two object storage device OSDs are damaged, at most one PG cannot process requests. When three object storage device OSDs are damaged, at most one PG's data is lost, thereby improving the availability of the system and reducing the risk of data loss. The method has good practicality and can create considerable value for society and enterprises.

[0035] In a second aspect, the present invention provides a data distribution device of a distributed storage system, the device comprising:

[0036] A PG total calculation module is used to configure multiple object storage devices for a distributed storage system cluster, and calculate the expected total number of PGs in the cluster based on the expected number of PGs of each object storage device;

[0037] A module for calculating the number of non-repetitive PGs that can be accommodated is used to calculate the number of non-repetitive PGs that can be accommodated based on the number of object storage devices;

[0038] The PG allocation module is used to allocate cluster PGs based on the relationship between the number of non-duplicate PGs that can be accommodated and the total number of PGs expected in the cluster and using mathematical graph theory until all the total number of PGs expected in the cluster is allocated.

[0039] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data distribution method of a distributed storage system of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data distribution method of a distributed storage system of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a flow chart of a data distribution method of a distributed storage system according to an embodiment of the present invention;

[0043] Figure 2 is a flow chart of another data distribution method of a distributed storage system according to an embodiment of the present invention;

[0044] Figure 3 is a flow chart of a data distribution method of another distributed storage system according to an embodiment of the present invention;

[0045] Figure 4 is a flow chart of a data distribution method of a distributed storage system according to another embodiment of the present invention;

[0046] Figure 5 It is a line chart of the total number of expected PGs in the cluster under different configurations of the expected number of PGs per OSD according to an embodiment of the present invention;

[0047] Figure 6 is a structural block diagram of a data distribution device of a distributed storage system according to an embodiment of the present invention;

[0048] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0050] The present invention takes a typical distributed storage system data distribution scenario as an example, which has three replica storage pools and a total of 2048 PGs distributed in 150 OSDs. It is assumed that the actual number of distributed PG copies on each OSD is evenly distributed at 41.

[0051] Regardless of the distributed storage system's rack, host and other regular configurations, when two disks are damaged or the OSD is down, there are two PG replica errors, resulting in the probability of PG being inaccessible is approximately:

[0052]

[0053] Among them, the probability that two or more PGs are inaccessible is approximately:

[0054]

[0055] At the same time, when three disks are damaged, there are three PG copy errors, resulting in the probability of PG data loss is approximately:

[0056]

[0057] Among them, the probability of losing two or more PG data is:

[0058]

[0059] in, The number of PG copies actually distributed on each object storage device OSD is evenly distributed as 41. x represents the xth PG combination, and y represents the 41 PGs minus the xth PG combination. It indicates the number of combinations of selecting 41 PGs from 2048-41 PGs, which is used to calculate the probability.

[0060] Note: The calculations here are approximate values. The changes in the number of combinations caused by multi-copy allocation are ignored when calculating the number of combinations.

[0061] The embodiment of the present invention provides a data distribution method for a distributed storage system. By optimizing the allocation of PG copies, when two disks of the distributed storage system cluster storage pool are damaged or the OSD is down, at most only one PG enters an unavailable state, and when there are three disks, at most only one PG loses data, that is, the probability that two or more PGs are inaccessible and the probability of losing data of two or more PGs are both equal to 0, and the formula is expressed as follows:

[0062] P inlt2 =0(5);

[0063] P dnlt2 =0 (6).

[0064] According to an embodiment of the present invention, an embodiment of a data distribution method for a distributed storage system is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0065] In this embodiment, a data distribution method of a distributed storage system is provided, which can be used for the above-mentioned computer terminals, such as central processing units, servers, etc. Figure 1 is a flow chart of a data distribution method of a distributed storage system according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0066] Step S101, configure multiple object storage devices for a distributed storage system cluster, and calculate the total number of expected PGs in the cluster based on the expected number of PGs of each object storage device.

[0067] Specifically, the object storage device, namely OSD, is responsible for storing actual data in the form of objects on the physical disk of each cluster node.

[0068] PG (placement group) is a layer of logic above the object storage device OSD and can be regarded as a logical concept.

[0069] Ceph is a distributed storage system that is high-performance, reliable and scalable. It is used to map objects to PGs. Since it is impossible to track the location at the object level in a system with millions of objects, a placement strategy is used to group PGs, group many objects together, and then store them in the object storage device OSD to facilitate the location and tracking of objects.

[0070] like Figure 4 As shown, assuming that the Ceph cluster has 512 object storage devices OSD, it is expected that there are 100 PGs on each OSD. The expected total number of PGs Ne in the cluster is calculated to be 16384 by calculating the number of object storage device OSDs.

[0071] Step S102, calculating the number of non-repetitive PGs that can be accommodated in polynomial time based on the number of object storage devices.

[0072] Specifically, polynomial time in computational complexity theory refers to the fact that the computation time of a problem is no greater than a polynomial multiple of the problem size.

[0073] The number of non-repeated PGs that can be accommodated in polynomial time means that there is no more than one duplicate OSD PG between any two PGs in the cluster, that is, any two PGs cannot have two or more copies distributed in the same OSD. This PG allocation method can achieve the purpose of only one PG becoming unavailable when two disks are damaged or the OSD goes down. Based on the number of object storage devices, the number of non-repeated PGs Np that can be accommodated in polynomial time can be calculated.

[0074] Step S103, based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster is allocated.

[0075] Specifically, in order to ensure that no two PGs have two or more copies distributed in the same OSD, based on the relationship between the number of non-duplicate PGs that can be accommodated and the total number of PGs expected in the cluster and using mathematical graph theory to distribute all PGs on OSDs, it can be ensured that no two PGs have two or more duplicate OSDs.

[0076] The data distribution method of the distributed storage system provided in this embodiment calculates the number of non-duplicate PGs that can be accommodated based on the number of object storage devices; based on the relationship between the number of non-duplicate PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster are allocated. If two disks are damaged, at most only one PG cannot be written, and if three disks are damaged, at most only one PG cannot be read, and the risk of data loss is reduced. If more disks are damaged, the amount of data that may be lost will also be greatly reduced, reducing the range of PGs that may be affected when multiple data disks are damaged, thereby improving the availability of the system and solving the problem of data loss caused by the increase in the range of PGs that may be affected when multiple data disks are damaged.

[0077] In this embodiment, a data distribution method of a distributed storage system is provided, which can be used in computer terminals, such as central processing units, servers, etc. Figure 2 is a flow chart of a data distribution method of a distributed storage system according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0078] Step S201: configure multiple object storage devices for a distributed storage system cluster, and calculate the total number of expected PGs in the cluster based on the expected number of PGs of each object storage device.

[0079] Specifically, the above step S201 includes:

[0080] Step S2011, obtain the number of object storage devices in the distributed storage system cluster, and set the expected number of PGs and the maximum number of PG copies for each object storage device.

[0081] Specifically, Figure 4 As shown, assuming that the Ceph cluster has 512 object storage device OSDs, it is expected that there are 100 PGs on each OSD, and 3 copies of each PG are on 3 object storage device OSDs. Then the number of object storage device OSDs is 512, and the expected number of PGs is set to 100 for each object storage device, and the maximum number of PG copies is 3.

[0082] Step S2012, calculate the expected total number of PGs in the cluster based on the number of object storage devices, the expected number of PGs and the maximum number of PG copies, and round the result of calculating the expected total number of PGs in the cluster to a value equal to 2 raised to the Nth power.

[0083] Specifically, the formula for calculating the expected total number of PGs in the cluster is as follows:

[0084] The expected total number of PGs in the cluster = the number of object storage devices * the expected number of PGs / the maximum number of replicas.

[0085] Based on the fact that there are 512 object storage device OSDs, the expected number of PGs for each object storage device is set to 100, the maximum number of PG copies is 3, and the total number of PGs expected in the cluster Ne is calculated to be 16384 (16384 is equal to 2 to the 14th power). The total number of PGs expected in the cluster is the number of PGs Ne required for the normal operation of the distributed storage system.

[0086] Step S202, calculating the number of non-repetitive PGs that can be accommodated in polynomial time based on the number of object storage devices.

[0087] Specifically, the PG that can accommodate non-duplicate PGs is a PG of an object storage device where no more than one duplicate PG exists between any two PGs in the distributed storage system cluster. The above step S202 includes:

[0088] Step S2021, calculating a remainder obtained by calculating a ratio of the number of object storage devices to a preset constant.

[0089] For example, the preset constant in this embodiment is 6. When there are n object storage devices OSD, n is set according to the actual situation and is not specifically limited here. The remainder obtained by calculating the ratio of the number of object storage devices to the preset constant can be expressed as nmod 6.

[0090] Step S2022, determining the number of PGs of the object storage device in which more than one duplicate exists between any two PGs according to the value of the remainder.

[0091] Specifically, the number of PGs in the object storage device where any two PGs have more than one duplicate is denoted by k, and the determination method includes:

[0092] When n mod 6 = 0 or 2, k = n / 2;

[0093] When n mod 6 = 1 or 3, k = 0;

[0094] When n mod 6 = 4, k = n / 2 + 1;

[0095] When n mod 6 = 5, k = 4.

[0096] When the Ceph cluster has 512 object storage devices OSD, k = 512 / 2 = 256.

[0097] Step S2023, based on the number of object storage devices and the number of PGs in object storage devices where any two PGs have more than one duplicate, calculate the number of non-duplicate PGs that can be accommodated in polynomial time.

[0098] Specifically, the formula for calculating the number of non-repeating PGs that can be accommodated in polynomial time is as follows:

[0099]

[0100] Where Np is the number of non-duplicate PGs that can be accommodated in polynomial time, k is the number of PGs in object storage devices where any two PGs have more than one duplicate, n is the number of object storage devices, and C n Indicates the number of combinations, that is, the number of combinations of any two PGs selected from n individual object storage devices.

[0101] For example, if the Ceph cluster has 512 object storage devices OSD, the calculation formula for the number of non-repetitive PGs that can be accommodated is:

[0102]

[0103] like Figure 5 As shown, the total number of expected PGs in the cluster is calculated under the configuration of the expected number of PGs of different numbers of object storage devices OSD, and a line chart is obtained (the number of non-duplicate PGs that can be accommodated in the figure has been divided by 2).

[0104] The part of the broken line above the curve is the expected total number of PGs in the cluster under the normal operation of the distributed storage system, and the part of the broken line below the curve is the expected total number of PGs in the cluster that does not meet the normal operation of the distributed storage system.

[0105] Step S203, based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster is allocated. For details, see Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0106] The data distribution method of the distributed storage system provided in this embodiment obtains the number of object storage devices in the distributed storage system cluster, and sets the expected number of PGs and the maximum number of PG copies for each object storage device, thereby achieving the purpose of calculating the expected total number of PGs in the cluster based on the number of object storage devices, the expected number of PGs and the maximum number of PG copies, and rounds the result of calculating the expected total number of PGs in the cluster to a value equal to 2 raised to the Nth power, thereby improving the accuracy of the calculated expected total number of PGs in the cluster, providing conditions for the subsequent allocation of cluster PGs, and determining the number of PGs of object storage devices in which any two PGs have more than one duplicate according to the value of the remainder, thereby improving the accuracy of the number of PGs of object storage devices in which any two PGs have more than one duplicate, thereby improving the accuracy and availability of calculating the number of non-duplicate PGs that can be accommodated based on the number of object storage devices and the number of PGs of object storage devices in which any two PGs have more than one duplicate.

[0107] In this embodiment, a data distribution method of a distributed storage system is provided, which can be used in computer terminals, such as central processing units, servers, etc. Figure 3is a flow chart of a data distribution method of a distributed storage system according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0108] Step S301, configure multiple object storage devices for the distributed storage system cluster, and calculate the total number of expected PGs in the cluster based on the expected number of PGs of each object storage device. Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0109] Step S302, based on the number of object storage devices, calculate the number of non-repeated PGs that can be accommodated in polynomial time. Figure 2 Step S202 of the illustrated embodiment will not be described in detail here.

[0110] Step S303, regard each object storage device as a point on the plane, and any three points are not collinear. At the same time, set three replicas for each PG, and allocate the three replicas to three object storage devices respectively. The three object storage devices with allocated replicas are regarded as three points, and the three points are connected into a triangle.

[0111] Specifically, any two PGs recorded in step S202 cannot have two or more copies distributed in the same object storage device OSD, that is, the triangles represented by any two PGs cannot share a common edge.

[0112] Step S304, based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster are allocated.

[0113] Specifically, the above step S304 includes:

[0114] Step S3041, when the number of non-duplicate PGs that can be accommodated is greater than the total number of PGs expected in the cluster, a storage node selection algorithm is used to select three object storage devices for the first three copies of the total number of PGs expected in the cluster, and mathematical graph theory is used to find the first triangle in the graph.

[0115] Specifically, this embodiment selects the Ceph CRUSH algorithm (Controlled, Reusable, Universal, Scalable Hash) as the storage node algorithm, which is the core algorithm for data distribution and high availability used in Ceph. The CRUSH algorithm provides a way to distribute data to ensure that data is evenly distributed on all nodes in the cluster, while maintaining high availability and consistency of data when the cluster changes dynamically. Through the hierarchical Cluster Map and Placement Rule, PG is mapped to the object storage device OSD.

[0116] like Figure 4 As shown, taking the Ceph cluster with 512 object storage devices OSD as an example, the calculated number of non-duplicate PGs Np that can be accommodated is 21760, and the expected total number of PGs Ne of the cluster is 16384. At this time, when the number of non-duplicate PGs Np that can be accommodated is greater than the expected total number of PGs Ne of the cluster, the CRUSH algorithm usage conditions are met. Based on the expected total number of PGs Ne of the cluster being 16384, the storage node selection algorithm is used to select three object storage device OSDs for the first PG three copies in the expected total number of PGs in the cluster, and mathematical graph theory is used to find the first triangle in the figure. Figure 4 The algorithm in this example refers to the CRUSH algorithm.

[0117] Step S3042, continue to allocate for the second PG in the total number of PGs expected in the cluster, and use the storage node selection algorithm to select three object storage devices for the three copies of the second PG in the total number of PGs expected in the cluster, use mathematical graph theory to find the second triangle, compare the second triangle with the first triangle, and determine whether there are two or more repeated object storage devices, that is, whether there are co-side triangles; if so, discard the PG allocation result, increase the number of calculations by one, and reallocate the second PG.

[0118] Exemplarily, the three copies of the first PG select three object storage devices OSD to form the first triangle, and the three copies of the second PG select three object storage devices OSD to form the second triangle. The second triangle is compared with the first triangle to determine whether there are two or more duplicate object storage devices, that is, whether there are co-side triangles; if so, the PG allocation result is discarded, the number of calculations is increased by one, and the second PG is re-allocated; if not, the allocation result of the second PG is retained.

[0119] Step S3043, continue to allocate the third PG in the total number of expected PGs of the cluster, find non-colateral triangles, until all PGs in the total number of expected PGs of the cluster are allocated, or no more non-colateral triangles are found, and obtain the first non-colateral triangle set and PG allocation results.

[0120] Specifically, continue to select three object storage devices OSD for the third PG in the total number of expected PGs in the cluster, and then form the third triangle. The third triangle is compared with the first triangle and the second triangle respectively to determine whether there are co-lateral triangles. If so, the allocation result of the third PG is discarded, the number of calculations is increased by one, and the third PG is reallocated again. If not, the allocation result of the third PG is retained, and the above steps are repeated until all PGs in the total number of expected PGs in the cluster are allocated, or no more non-co-lateral triangles are found, and the first non-co-lateral triangle set and PG allocation results are obtained.

[0121] After executing step S3031 to step S3033, at least 1 / 3 of the number of all maximum non-colateral triangles has been found, that is, the first non-colateral triangle set includes 1 / 3 of the number of all maximum non-colateral triangles.

[0122] Step S3044, when the number of PGs in the PG allocation result is less than the total number of PGs expected by the cluster, traverse the first non-colateral triangle set from small to large by number, select any specified triangle, and then traverse all object storage devices to determine whether two non-colateral triangles with other triangles in the first non-colateral triangle set except the specified triangle can be found. If they can be found, remove the object storage device corresponding to the specified triangle from the first non-colateral triangle set to obtain the second non-colateral triangle set.

[0123] Specifically, if the calculated PG allocation result does not meet the total number of PGs expected by the cluster, that is, the number of PGs in the PG allocation result is less than the total number of PGs expected by the cluster, further enhanced calculation is performed, including:

[0124] From the first non-colateral triangle set T calculated previously, traverse from small to large numbers, select any triangle T1, assume that it is removed from the first non-colateral triangle set, and then traverse all non-colateral triangles formed by the object storage device OSD to see if two triangles that are not colateral with all triangles in T (excluding T1) can be found. If they can be found, remove triangle T1 from the first non-colateral triangle set T to obtain the second non-colateral triangle set T'.

[0125] Step S3045, continue to traverse all triangles in the second non-colateral triangle set until at least 1 / 2 of the maximum number of non-colateral triangles is found.

[0126] Specifically, continue to traverse all triangles in the second non-colateral triangle set T' to find all non-colateral triangles, where finding all non-colateral triangles is an NP problem. A greedy algorithm can be used to find no less than 1 / 2 of the total number of non-colateral triangles in polynomial time. A greedy algorithm is an algorithm strategy that takes the best or optimal choice in the current state at each step, hoping to lead to the best or optimal result in the world.

[0127] The data distribution method of the distributed storage system provided in this embodiment achieves the purpose of having at most one PG enter an unavailable state when two disks are damaged or OSDs are down, that is, any two PGs cannot have two or more copies distributed in the same object storage device OSD, which can optimize the allocation of PG copies, so that when two disks are damaged or OSDs are down in the Ceph cluster storage pool, at most only one PG enters an unavailable state, and when there are three, at most only one PG will experience data loss, reducing the range of PGs that may be affected when multiple data disks are damaged, so as to improve the availability of the distributed storage system and reduce the risk of data loss. The largest non-colateral triangle in the non-colateral triangle set is traversed by a greedy algorithm, and no less than 1 / 2 of the total number of non-colateral triangles is found in polynomial time, which has good practicality and can create considerable value for society and enterprises.

[0128] As one or more specific application embodiments of the present invention, Figure 4 The data distribution method of the distributed storage system provided by the present invention is further described in detail, and the specific process is as follows:

[0129] Step 1) Assume that the Ceph cluster has 512 object storage devices OSD, and it is expected that there are 100 PGs on each object storage device OSD, and the maximum number of replicas is 3.

[0130] Step 2) According to formula (7), the total number of non-repetitive PGs that can be accommodated in the cluster is 43520, ensuring that the number of non-repetitive PGs that can be calculated in polynomial time is 21760 (half of the number of non-repetitive PGs that can be accommodated). The expected total number of PGs in the cluster (total number of PGs = (number of OSDs * 100) / maximum number of replicas, the result must be rounded to the value closest to the power of 2) is 16384 (a power of 2), which meets the conditions and can be used to complete the allocation using this algorithm.

[0131] Step 3) Calculate the distribution of all PGs according to the expected total number of PGs in the cluster, which is 16,384, to ensure that no two PGs have two or more duplicate OSDs. The distribution of all PGs is calculated according to the specific process of step S303, which will not be repeated here.

[0132] After following the process from step 1) to step 3), if two disks are damaged, at most only one PG cannot be written, and if three disks are damaged, at most only one PG cannot be read, and there is no risk of data loss. If more disks are damaged, the amount of data that may be lost will also be greatly reduced, that is, the probability of two or more PGs being inaccessible and the probability of losing two or more PG data are both equal to 0, as expressed by the following formula:

[0133] P inlt2 =0(5);

[0134] P dnlt2 =0(6).

[0135] The data distribution method of the distributed storage system provided in this embodiment can optimize the allocation of PG copies, so that when two disks of the Ceph cluster storage pool are damaged or the OSD fails, at most only one PG will enter an unavailable state, and when there are three disks, at most only one PG will experience data loss, thereby ensuring the data security of the distributed storage system, having good practicality, and creating considerable value for society and enterprises.

[0136] In the present embodiment, a data distribution device of a distributed storage system is also provided, and the device is used to implement the above-mentioned embodiment and preferred implementation mode, and the descriptions that have been made are not repeated here. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0137] This embodiment provides a data distribution device of a distributed storage system, such as Figure 6 As shown, including:

[0138] The PG total number calculation module 601 is used to configure multiple object storage devices for a distributed storage system cluster, and calculate the expected total number of PGs in the cluster based on the expected number of PGs of each object storage device.

[0139] The module 602 for calculating the number of non-repetitive PGs that can be accommodated is used to calculate the number of non-repetitive PGs that can be accommodated based on the number of object storage devices.

[0140] The PG allocation module 603 is used to allocate cluster PGs based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster and using mathematical graph theory until all the total number of PGs expected in the cluster is allocated.

[0141] In some optional implementations, the PG total number calculation module 601 includes:

[0142] The data acquisition unit is used to obtain the number of object storage devices in the distributed storage system cluster and set the expected number of PGs and the maximum number of PG copies for each object storage device.

[0143] The cluster expected total number of PG calculation unit is used to calculate the cluster expected total number of PGs based on the number of object storage devices, the expected number of PGs and the maximum number of PG copies, and round the result of calculating the cluster expected total number of PGs to a value equal to 2 raised to the Nth power.

[0144] In some optional implementations, the non-repeated PG that can be accommodated is the PG of the object storage device where no more than one duplicate exists between any two PGs in the distributed storage system cluster; the non-repeated PG quantity calculation module 602 that can be accommodated includes:

[0145] The ratio calculation unit is used to calculate the remainder of the ratio of the number of object storage devices to a preset constant.

[0146] The value determination unit is used to determine the number of PGs of the object storage device in which more than one duplicate exists between any two PGs according to the value of the remainder.

[0147] The unit for the number of non-repeating PGs that can be accommodated is used to calculate the number of non-repeating PGs that can be accommodated in polynomial time based on the number of object storage devices and the number of PGs in object storage devices where any two PGs have more than one duplicate. The formula for calculating the number of non-repeating PGs that can be accommodated in polynomial time is as follows:

[0148]

[0149] Where Np is the number of non-duplicate PGs that can be accommodated in polynomial time, k is the number of PGs in object storage devices where any two PGs have more than one duplicate, n is the number of object storage devices, and C n Indicates the number of combinations, that is, the number of combinations of any two PGs selected from n individual object storage devices.

[0150] In some optional implementations, the PG allocation module 603 includes:

[0151] The first PG allocation unit is used to select three object storage devices for the first three copies of PG in the total number of PGs expected by the cluster by using a storage node selection algorithm when the number of non-duplicate PGs that can be accommodated is greater than or equal to the total number of PGs expected by the cluster, and to find the first triangle in the graph by using mathematical graph theory.

[0152] The second PG allocation unit is used to continue allocating the second PG in the total number of PGs expected in the cluster, and use the storage node selection algorithm to select three object storage devices for the three copies of the second PG in the total number of PGs expected in the cluster, use mathematical graph theory to find the second triangle, compare the second triangle with the first triangle, and determine whether there are two or more repeated object storage devices, that is, whether there are co-side triangles; if so, discard the PG allocation result, increase the number of calculations by one, and re-allocate the second PG.

[0153] The third PG allocation unit is used to continue allocating the third PG in the total number of expected PGs of the cluster and find non-colateral triangles until all PGs in the total number of expected PGs of the cluster are allocated, or no more non-colateral triangles are found, to obtain the first non-colateral triangle set and PG allocation results.

[0154] The fourth PG allocation unit is used to traverse the first non-colateral triangle set from small to large according to the number when the number of PGs in the PG allocation result is less than the total number of PGs expected by the cluster, select any specified triangle, and then traverse all object storage devices to determine whether two non-colateral triangles with other triangles in the first non-colateral triangle set except the specified triangle can be found. If they can be found, the object storage device corresponding to the specified triangle is removed from the first non-colateral triangle set to obtain the second non-colateral triangle set.

[0155] The fifth PG allocation unit is used to continue traversing all triangles in the second non-colateral triangle set until at least 1 / 2 of the maximum number of non-colateral triangles is found.

[0156] In some optional implementations, the data distribution device of the distributed storage system further includes:

[0157] The mathematical graph theory module is used to regard each object storage device as a point on a plane, and any three points are not collinear. At the same time, three replicas are set for each PG, and the three replicas are respectively allocated to three object storage devices. The three object storage devices with allocated replicas are regarded as three points, and the three points are connected to form a triangle.

[0158] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0159] The data distribution device of the distributed storage system in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0160] The embodiment of the present invention also provides a computer device having the above Figure 6 The data distribution device of the distributed storage system shown.

[0161] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0162] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0163] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0164] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0165] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0166] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0167] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0168] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0169] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data distribution method for a distributed storage system, characterized in that: The method comprises: Configure multiple object storage devices for the distributed storage system cluster, and calculate the total number of expected PGs in the cluster based on the expected number of PGs for each object storage device; Calculate the number of non-repeated PGs that can be accommodated in polynomial time based on the number of object storage devices; Based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster is allocated.

2. The method according to claim 1, characterized in that Calculating the expected total number of PGs in the cluster based on the expected number of PGs of each object storage device includes: Get the number of object storage devices in the distributed storage system cluster, and set the expected number of PGs and the maximum number of PG replicas for each object storage device; The expected total number of PGs in the cluster is calculated based on the number of object storage devices, the expected number of PGs, and the maximum number of PG replicas, and the result of calculating the expected total number of PGs in the cluster is rounded to a value equal to 2 raised to the power of N.

3. The method according to claim 1, characterized in that: The non-duplicate PGs that can be accommodated are PGs of object storage devices where no more than one duplicate PG exists between any two PGs in the distributed storage system cluster; The calculation of the number of non-repeated PGs that can be accommodated in polynomial time based on the number of object storage devices includes: Calculate the remainder of the ratio of the number of object storage devices to a preset constant; Determine the number of PGs of the object storage device where any two PGs have more than one duplicate according to the value of the remainder; The number of non-duplicate PGs that can be accommodated in polynomial time is calculated based on the number of object storage devices and the number of PGs of any two PGs in which there is more than one duplicate object storage device.

4. The method according to claim 3, characterized in that The formula for calculating the number of non-repeating PGs that can be accommodated in polynomial time is as follows: Where Np is the number of non-duplicate PGs that can be accommodated, k is the number of PGs in the object storage device where any two PGs have more than one duplicate, n is the number of object storage devices, and C n Indicates the number of combinations, that is, the number of combinations of any two PGs selected from n individual object storage devices.

5. The method according to claim 1, characterized in that Before allocating cluster PGs based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster and using mathematical graph theory, the method further includes: Each object storage device is regarded as a point on the plane, and any three points are not collinear. At the same time, three replicas are set for each PG, and the three replicas are respectively allocated to three object storage devices. The three object storage devices with allocated replicas are regarded as three points, and the three points are connected to form a triangle.

6. The method according to claim 5, characterized in that Based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster is allocated, including: When the number of non-repetitive PGs that can be accommodated is greater than or equal to the total number of PGs expected in the cluster, a storage node selection algorithm is used to select three object storage devices for the first three copies of the PG in the total number of PGs expected in the cluster, and mathematical graph theory is used to find the first triangle in the graph; Continue to allocate the second PG in the total number of PGs expected by the cluster, and use the storage node selection algorithm to select three object storage devices for the three copies of the second PG in the total number of PGs expected by the cluster, use mathematical graph theory to find the second triangle, compare the second triangle with the first triangle, and determine whether there are two or more duplicate object storage devices, that is, whether there are co-side triangles; if so, discard the PG allocation result, increase the number of calculations by one, and re-allocate the second PG; Continue to allocate the third PG in the total number of expected PGs of the cluster and find non-colateral triangles until all PGs in the total number of expected PGs of the cluster are allocated, or no more non-colateral triangles are found, and the first non-colateral triangle set and PG allocation results are obtained.

7. The method according to claim 6, characterized in that Based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster, the cluster PGs are allocated using mathematical graph theory until all the total number of PGs expected in the cluster is allocated, and the following is also included: When the number of PGs in the PG allocation result is less than the total number of PGs expected by the cluster, traverse the first non-colateral triangle set from small to large numbers, select any specified triangle, and then traverse all object storage devices to determine whether two non-colateral triangles with other triangles in the first non-colateral triangle set except the specified triangle can be found. If they can be found, remove the object storage device corresponding to the specified triangle from the first non-colateral triangle set to obtain a second non-colateral triangle set; Continue to traverse all triangles in the second non-colateral triangle set until at least 1 / 2 of the maximum number of non-colateral triangles is found.

8. A data distribution device for a distributed storage system, characterized in that: The device comprises: A PG total calculation module is used to configure multiple object storage devices for a distributed storage system cluster, and calculate the expected total number of PGs in the cluster based on the expected number of PGs of each object storage device; A module for calculating the number of non-repetitive PGs that can be accommodated is used to calculate the number of non-repetitive PGs that can be accommodated based on the number of object storage devices; The PG allocation module is used to allocate cluster PGs based on the relationship between the number of non-repetitive PGs that can be accommodated and the total number of PGs expected in the cluster and using mathematical graph theory until all the total number of PGs expected in the cluster is allocated.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data distribution method of the distributed storage system according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data distribution method of the distributed storage system according to any one of claims 1 to 7.

Images