Data distribution method, device and equipment of distributed storage system and storage medium

Abstract

The present application relates to the technical field of computer storage service high availability, and discloses a data distribution method, device and equipment of a distributed storage system and a storage medium, the method comprising: configuring a plurality of object storage devices for a distributed storage system cluster, and calculating a total number of expected PGs of the cluster based on the expected number of PGs of each object storage device; calculating a number of non-repeating PGs that can be accommodated within a polynomial time based on the number of object storage devices; distributing the cluster PGs based on the size relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs of the cluster and using mathematical graph theory, until all of the total number of expected PGs of the cluster are distributed. If two disks are damaged, at most only one PG cannot be written, and if three disks are damaged, at most only one PG cannot be read, thereby reducing the risk of data loss. If more disks are damaged, the amount of data that can be lost will also be greatly reduced, thereby reducing the range of PGs that can be affected when multiple data disks are damaged.

Description

Technical Field

[0001] This invention relates to the field of high availability technology for computer storage services, and specifically to data distribution methods, apparatus, devices, and storage media for distributed storage systems. Background Technology

[0002] Ceph is a distributed storage system that uniquely provides object, block, and file storage capabilities within a unified system. Ceph eliminates the dependency on a single central node, enabling a decentralized architecture that allows multiple servers to form a large cluster, consolidating their disk resources into a large resource pool.

[0003] In production environments, Ceph cluster availability issues caused by disk failures are among the most common problems encountered during routine cloud storage maintenance. For a commonly used Ceph storage pool configured with 3 replicas, when an OSD (Object Storage Device) fails, the Placement Group (PG) allocated to that OSD enters a Degraded state (a state where a RAID system has a faulty or offline hard drive, but the system can still read and write data normally), and can still provide services. However, when two OSDs fail simultaneously, the PGs with two replicas distributed across the two failed OSDs will no longer be able to process requests. When three or more OSDs fail simultaneously, data loss may occur.

[0004] When an OSD error occurs due to disk failure, a certain scale of data migration will occur in the distributed storage system cluster, putting pressure on other disks. Therefore, it is not uncommon for more than one disk to fail at the same time. Summary of the Invention

[0005] In view of this, the present invention provides a data distribution method, apparatus, device and storage medium for a distributed storage system to solve the problem of data loss caused by the increased range of PGs that may be affected when multiple data disks are damaged.

[0006] In a first aspect, the present invention provides a data distribution method for a distributed storage system, the method comprising:

[0007] Configure multiple object storage devices for a distributed storage system cluster, and calculate the total expected number of PGs for the cluster based on the expected number of PGs for each object storage device;

[0008] Calculate the number of non-repeating PGs that can be accommodated in polynomial time based on the number of object storage devices;

[0009] Based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, mathematical graph theory is used to allocate PGs in the cluster until all the expected total number of PGs in the cluster has been allocated.

[0010] The data distribution method for a distributed storage system provided by this invention calculates the number of non-repeating Group Specs (PGs) that can be accommodated based on the number of object storage devices. It then allocates PGs to the cluster using mathematical graph theory, based on the relationship between the number of non-repeating PGs that can be accommodated and the total expected number of PGs in the cluster, until all expected PGs have been allocated. If two disks fail, at most one PG will be unable to be written to; if three disks fail, at most one PG will be unable to be read from, reducing the risk of data loss. Even if more disks fail, the amount of data that may be lost will be significantly reduced, decreasing the range of PGs that may be affected when multiple data disks fail, thereby improving system availability and solving the problem of data loss caused by an increased range of PGs that may be affected when multiple data disks fail.

[0011] In one alternative implementation, calculating the total expected PGs of the cluster based on the expected PGs for each object storage device includes:

[0012] Obtain the number of object storage devices in the distributed storage system cluster, and set the expected number of PGs and the maximum number of replicas for each object storage device;

[0013] The total number of expected PGs in the cluster is calculated based on the number of object storage devices, the expected number of PGs, and the maximum number of replicas of PGs. The result of calculating the total number of expected PGs in the cluster is rounded to a power of 2.

[0014] The data distribution method for a distributed storage system provided by this invention obtains the number of object storage devices in the distributed storage system cluster and sets the expected number of Group Specs (PGs) and the maximum number of PG replicas for each object storage device. This achieves the purpose of calculating the total expected number of PGs in the cluster based on the number of object storage devices, the expected number of PGs, and the maximum number of PG replicas. The result of calculating the total expected number of PGs in the cluster is rounded to a power of 2, which improves the accuracy of the calculated total expected number of PGs in the cluster and provides conditions for subsequent allocation of cluster PGs.

[0015] In one alternative implementation, a non-duplicate PG can be defined as a PG in a distributed storage system cluster where no two PGs have more than one duplicate object storage device.

[0016] The number of non-repeating PGs that can be accommodated in polynomial time, based on the number of object storage devices, includes:

[0017] The remainder obtained by comparing the number of object storage devices with a preset constant;

[0018] The remainder value determines the number of object storage devices where any two PGs have more than one duplicate object storage device.

[0019] The number of non-duplicate PGs that can be accommodated in polynomial time is calculated based on the number of object storage devices and the number of PGs for any two PGs where there is more than one duplicate object storage device.

[0020] In one alternative implementation, the formula for calculating the number of non-repeating PGs that can be accommodated in polynomial time is as follows:

[0021] ;

[0022] in, To accommodate the number of non-repeating PGs in polynomial time, Let n be the number of object storage devices (PGs) where any two PGs have more than one duplicate object storage device. This represents the number of combinations, specifically the number of combinations where two PGs are randomly selected from n individual object storage devices.

[0023] The data distribution method of the distributed storage system provided by the present invention determines the number of PGs in which any two PGs have more than one duplicate object storage device based on the value of the remainder, thereby improving the accuracy of the number of PGs in which any two PGs have more than one duplicate object storage device, and further improving the accuracy and availability of calculating the number of non-duplicate PGs that can be accommodated based on the number of object storage devices and the number of PGs in which any two PGs have more than one duplicate object storage device.

[0024] In an alternative implementation, before allocating cluster PGs based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, and using mathematical graph theory, the data distribution method for the distributed storage system further includes:

[0025] Each object storage device is considered as a point on a plane, and no three points are collinear. Three replicas are set for each PG, and the three replicas are allocated to three object storage devices respectively. The three object storage devices with replicas are considered as three points, and the three points are connected to form a triangle.

[0026] In one optional implementation, the cluster PGs are allocated based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, using mathematical graph theory, until all expected PGs in the cluster have been allocated, including:

[0027] When the number of non-repeating PGs that can be accommodated is greater than or equal to the total number of expected PGs in the cluster, the storage node selection algorithm is used to select three object storage devices for the first PG three-replica in the total number of expected PGs in the cluster, and the first triangle in the graph is found using mathematical graph theory.

[0028] Continue allocating the second PG out of the expected total number of PGs in the cluster, and use the storage node selection algorithm to select three object storage devices for the three replicas of the second PG out of the expected total number of PGs in the cluster. Use mathematical graph theory to find the second triangle, compare the second triangle with the first triangle, and determine whether there are two or more duplicate object storage devices, i.e., whether there is a triangle with the same side. If there is, discard the PG allocation result, increment the calculation count by one, and re-allocate the second PG.

[0029] Continue allocating the third PG in the expected total number of PGs in the cluster, find non-shared triangles, until all PGs in the expected total number of PGs in the cluster have been allocated, or no more non-shared triangles can be found, and obtain the first set of non-shared triangles and the PG allocation results.

[0030] The data distribution method for distributed storage systems provided by this invention achieves the goal of ensuring that at most one Group Required (PG) becomes unavailable when two disks fail or the Object Storage Device (OSD) fails. Specifically, no two PGs can have two or more replicas distributed within the same OSD. This optimizes PG replica allocation, ensuring that in a Ceph cluster storage pool, at most one PG becomes unavailable when two disks fail or the OSD fails, and at most one PG experiences data loss when three disks fail. This reduces the potential impact on the range of PGs affected by multiple disk failures, thereby improving the availability of the distributed storage system and mitigating the risk of data loss.

[0031] In one optional implementation, the allocation of cluster PGs based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, using mathematical graph theory, until all expected PGs in the cluster have been allocated, further includes:

[0032] When the number of PGs in the PG allocation result is less than the total number of expected PGs in the cluster, the first set of non-shared triangles is traversed from smallest to largest number, any specified triangle is selected, and then all object storage devices are traversed to determine whether two non-shared triangles can be found that are not shared with the specified triangle in the first set of non-shared triangles. If they can be found, the object storage device corresponding to the specified triangle is removed from the first set of non-shared triangles to obtain the second set of non-shared triangles.

[0033] Continue iterating through all triangles in the second set of non-shared-side triangles until at least half of the total number of the largest non-shared-side triangles is found.

[0034] The data distribution method for distributed storage systems provided by this invention uses a greedy algorithm to traverse the largest non-shared triangle in the set of non-shared triangles, finding at least half of the total number of non-shared triangles in polynomial time. This reduces the range of PGs that may be affected when multiple data disks fail. When two object storage devices (OSDs) fail, at most one PG cannot process requests; when three object storage devices (OSDs) fail, at most one PG's data is lost. This improves system availability and reduces the risk of data loss, demonstrating good practicality and creating considerable value for society and enterprises.

[0035] In a second aspect, the present invention provides a data distribution device for a distributed storage system, the device comprising:

[0036] The PG total count calculation module is used to configure multiple object storage devices for a distributed storage system cluster and calculate the expected total number of PGs for the cluster based on the expected number of PGs for each object storage device.

[0037] A module for calculating the number of non-duplicate PGs that can be accommodated is used to calculate the number of non-duplicate PGs that can be accommodated based on the number of object storage devices.

[0038] The PG allocation module is used to allocate PGs in the cluster based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, using mathematical graph theory, until all the expected total number of PGs in the cluster has been allocated.

[0039] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the data distribution method of the distributed storage system of the first aspect or any corresponding embodiment described above.

[0040] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the data distribution method of the distributed storage system of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0041] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating a data distribution method for a distributed storage system according to an embodiment of the present invention;

[0043] Figure 2 This is a flowchart illustrating a data distribution method for another distributed storage system according to an embodiment of the present invention;

[0044] Figure 3 This is a flowchart illustrating a data distribution method for another distributed storage system according to an embodiment of the present invention;

[0045] Figure 4 This is a flowchart illustrating a data distribution method for another distributed storage system according to an embodiment of the present invention;

[0046] Figure 5 This is a line graph showing the total expected number of PGs in the cluster under different single OSD expected PG configurations according to an embodiment of the present invention;

[0047] Figure 6 This is a structural block diagram of a data distribution device for a distributed storage system according to an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] This invention takes a typical distributed storage system data distribution scenario as an example, which consists of three replica storage pools with a total of 2048 PGs distributed across 150 OSDs. It is assumed that the actual number of PG replicas distributed on each OSD is uniformly 41.

[0051] Without considering the rack, host, and other rule configurations of the distributed storage system, the probability that two disks fail or the OSD crashes, resulting in two incorrect replicas of the PG and rendering the PG inaccessible, is approximately:

[0052] (1);

[0053] The probability that two or more PGs are inaccessible is approximately:

[0054] (2);

[0055] Meanwhile, when three disks fail, there are three PG replica errors, resulting in a probability of PG data loss of approximately:

[0056] 1.63% (3);

[0057] The probability of losing two or more PG data points is:

[0058] (4);

[0059] in, The actual number of PG replicas distributed on each object storage device OSD is 41, where x represents the x-th PG combination, and y represents the 41 PG combinations minus the x-th PG combination. This represents the number of combinations where 41 PGs are selected from 2048-41 PGs, and is used to calculate the probability.

[0060] Note: All calculations here are approximate values. The changes in the number of combinations due to multiple replica allocation are ignored when calculating the number of combinations.

[0061] This invention provides a data distribution method for a distributed storage system. By optimizing the allocation of PG replicas, it achieves the effect that when two disks fail or an OSD crashes, at most only one PG becomes unavailable in the distributed storage system cluster storage pool, and when three disks fail, at most only one PG experiences data loss. That is, the probability of two or more PGs becoming inaccessible and the probability of losing data from two or more PGs are both equal to 0. The formula is expressed as follows:

[0062] (5);

[0063] (6).

[0064] According to an embodiment of the present invention, a data distribution method embodiment for a distributed storage system is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0065] This embodiment provides a data distribution method for a distributed storage system, which can be used in the aforementioned computer terminals, such as central processing units and servers. Figure 1 This is a flowchart of a data distribution method for a distributed storage system according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0066] Step S101: Configure multiple object storage devices for the distributed storage system cluster, and calculate the total expected number of PGs for the cluster based on the expected number of PGs for each object storage device.

[0067] Specifically, object storage devices, or OSDs, are responsible for storing actual data as objects on the physical magnets of each cluster node.

[0068] PG (placement group) is a logical layer above the object storage device OSD, and can be viewed as a logical concept.

[0069] Ceph, a distributed storage system, is a high-performance, reliable, and scalable distributed storage system used to map objects to Grouped Objects (PGs). Since it's impossible to track the location of objects at the object level in a system with millions of objects, a placement strategy is used to group PGs together and then store them in an Object Storage Device (OSD) for easier object location and tracking.

[0070] like Figure 4 As shown, assuming the Ceph cluster has 512 object storage devices (OSDs) and each OSD is expected to have 100 PGs, the expected total number of PGs Ne in the cluster is 16384 based on the number of object storage devices (OSDs).

[0071] Step S102: Calculate the number of non-repeating PGs that can be accommodated in polynomial time based on the number of object storage devices.

[0072] Specifically, in computational complexity theory, polynomial time refers to the time required to compute a problem that is no greater than a polynomial multiple of the problem size.

[0073] The number of non-repeating PGs that can be accommodated in polynomial time refers to the number of PGs in a cluster where no two PGs have more than one duplicate OSD. In other words, no two PGs can have two or more replicas distributed across the same OSD. This PG allocation method ensures that if two disks fail or an OSD crashes, at most one PG will become unavailable. The number of non-repeating PGs Np that can be accommodated in polynomial time can be calculated based on the number of object storage devices.

[0074] Step S103: Based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, mathematical graph theory is used to allocate the cluster PGs until all the expected total number of cluster PGs has been allocated.

[0075] Specifically, in order to ensure that no two PGs have two or more replicas distributed in the same OSD, based on the relationship between the number of non-duplicate PGs that can be accommodated and the total number of PGs expected in the cluster, and by using mathematical graph theory to distribute all PGs on OSDs, it can be ensured that no two PGs have two or more duplicate OSDs.

[0076] The data distribution method for the distributed storage system provided in this embodiment calculates the number of non-repeating Group Specs (PGs) that can be accommodated based on the number of object storage devices. It then allocates PGs to the cluster using mathematical graph theory, based on the relationship between the number of non-repeating PGs that can be accommodated and the total expected number of PGs in the cluster, until all expected PGs have been allocated. If two disks fail, at most one PG will be unwriteable; if three disks fail, at most one PG will be unreadable, reducing the risk of data loss. Even if more disks fail, the potential amount of data loss will be significantly reduced, decreasing the range of PGs that may be affected when multiple data disks fail, thus improving system availability and solving the problem of data loss caused by an increased range of PGs affected when multiple data disks fail.

[0077] This embodiment provides a data distribution method for a distributed storage system, which can be used in computer terminals such as central processing units and servers. Figure 2 This is a flowchart of a data distribution method for a distributed storage system according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0078] Step S201: Configure multiple object storage devices for the distributed storage system cluster, and calculate the total expected number of PGs for the cluster based on the expected number of PGs for each object storage device.

[0079] Specifically, step S201 includes:

[0080] Step S2011: Obtain the number of object storage devices in the distributed storage system cluster, and set the expected number of PGs and the maximum number of PG replicas for each object storage device.

[0081] Specifically, such as Figure 4 As shown, assuming a Ceph cluster has 512 object storage devices (OSDs), each OSD is expected to have 100 PGs, and each PG has 3 replicas on 3 object storage device OSDs, then there are 512 object storage device OSDs. The expected number of PGs for each object storage device is set to 100, and the maximum number of replicas for a PG is 3.

[0082] Step S2012: Calculate the total number of expected PGs in the cluster based on the number of object storage devices, the expected number of PGs, and the maximum number of replicas of PGs. Round the result of the calculation of the total number of expected PGs in the cluster to a power of 2.

[0083] Specifically, the formula for calculating the expected total number of PGs in the cluster is as follows:

[0084] The expected total number of PGs in the cluster = number of object storage devices * expected number of PGs / maximum number of replicas.

[0085] Based on 512 object storage devices (OSDs), with an expected number of PGs (Group Purposes) of 100 per object storage device and a maximum PG replica count of 3, the expected total number of PGs (Ne) for the cluster is calculated to be 16384 (16384 equals 2 to the power of 14). The expected total number of PGs for the cluster is the total number of PGs (Ne) required for the distributed storage system to operate normally.

[0086] Step S202: Calculate the number of non-repeating PGs that can be accommodated in polynomial time based on the number of object storage devices.

[0087] Specifically, a non-duplicate PG is a PG in a distributed storage system cluster where no two PGs have more than one duplicate object storage device. Step S202 includes:

[0088] Step S2021: Calculate the remainder obtained by the ratio of the number of object storage devices to a preset constant.

[0089] For example, in this embodiment, the preset constant is 6. When there are n object storage devices (OSDs), n is set according to the actual situation and is not specifically limited here. The remainder obtained by calculating the ratio of the number of object storage devices to the preset constant can be expressed as n mod 6.

[0090] Step S2022: Determine the number of PGs where any two PGs have more than one duplicate object storage device based on the remainder value.

[0091] Specifically, the number of object storage devices in which any two PGs have more than one duplicate is denoted by k, and the determination methods include:

[0092] When n mod 6 = 0 or 2, k = n / 2;

[0093] When n mod 6 = 1 or 3, k = 0;

[0094] When n mod 6 = 4, k = n / 2 + 1;

[0095] When n mod 6 = 5, k = 4.

[0096] If a Ceph cluster has 512 object storage devices (OSDs), then k = 512 / 2 = 256.

[0097] Step S2023: Calculate the number of non-duplicate PGs that can be accommodated in polynomial time based on the number of object storage devices and the number of PGs where any two PGs have more than one duplicate object storage device.

[0098] Specifically, the formula for calculating the number of non-repeating PGs that can be accommodated in polynomial time is as follows:

[0099] (7);

[0100] in, To accommodate the number of non-repeating PGs in polynomial time, Let n be the number of object storage devices (PGs) where any two PGs have more than one duplicate object storage device. This represents the number of combinations, specifically the number of combinations where two PGs are randomly selected from n individual object storage devices.

[0101] For example, if a Ceph cluster has 512 Object Storage Devices (OSDs), the formula for calculating the number of non-duplicate Group Storage Devices (PGs) it can accommodate is:

[0102] =21760.

[0103] like Figure 5 As shown, the total number of expected PGs in the cluster is calculated under different configurations of the number of OSDs (Object Storage Devices), and a line graph is obtained (the number of non-duplicate PGs that can be accommodated in the graph has been divided by 2).

[0104] The portion of the broken line above the curve represents the expected total number of Group PGs (PGs) of the cluster under normal operating conditions of the distributed storage system, while the portion below the curve represents the expected total number of PGs of the cluster under normal operating conditions of the distributed storage system.

[0105] Step S203: Based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, mathematical graph theory is used to allocate PGs in the cluster until all expected PGs in the cluster have been allocated. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0106] The data distribution method for a distributed storage system provided in this embodiment obtains the number of object storage devices in the distributed storage system cluster and sets the expected number of Group Items (PGs) and the maximum number of PG replicas for each object storage device. This achieves the goal of calculating the total expected number of PGs in the cluster based on the number of object storage devices, the expected number of PGs, and the maximum number of PG replicas. The result of calculating the total expected number of PGs in the cluster is rounded to a power of 2, improving the accuracy of the calculated total expected number of PGs in the cluster and providing conditions for subsequent allocation of cluster PGs. The number of PGs in the cluster where any two PGs have more than one duplicate is determined based on the remainder value, improving the accuracy of the number of PGs in the cluster where any two PGs have more than one duplicate, thereby improving the accuracy and availability of calculating the number of non-duplicate PGs that can be accommodated based on the number of object storage devices and the number of PGs in the cluster where any two PGs have more than one duplicate.

[0107] This embodiment provides a data distribution method for a distributed storage system, which can be used in computer terminals such as central processing units and servers. Figure 3 This is a flowchart of a data distribution method for a distributed storage system according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0108] Step S301: Configure multiple object storage devices for the distributed storage system cluster, and calculate the total expected number of PGs for the cluster based on the expected number of PGs for each object storage device. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0109] Step S302: Calculate the number of non-repeating PGs that can be accommodated in polynomial time based on the number of object storage devices. See details below. Figure 2 Step S202 of the illustrated embodiment will not be described again here.

[0110] Step S303: Each object storage device is considered as a point on a plane, and no three points are collinear. At the same time, three replicas are set for each PG, and the three replicas are allocated to three object storage devices respectively. The three object storage devices with allocated replicas are considered as three points, and the three points are connected to form a triangle.

[0111] Specifically, in step S202, any two PGs cannot have two or more copies located in the same object storage device (OSD), meaning that the triangles represented by any two PGs cannot share a side.

[0112] Step S304: Based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, mathematical graph theory is used to allocate the cluster PGs until all the expected total number of cluster PGs has been allocated.

[0113] Specifically, step S304 includes:

[0114] Step S3041: When the number of non-repeating PGs that can be accommodated is greater than the total number of expected PGs in the cluster, the storage node selection algorithm is used to select three object storage devices for the first PG three-replica in the total number of expected PGs in the cluster, and the first triangle in the graph is found using mathematical graph theory.

[0115] Specifically, this embodiment selects the Ceph CRUSH algorithm (Controlled, Reusable, Universal, Scalable Hash, the core algorithm used in Ceph for data distribution and high availability. The CRUSH algorithm provides a way to distribute data, ensuring that data is evenly distributed across all nodes in the cluster, while maintaining high availability and consistency even when the cluster changes dynamically). Through a hierarchical Cluster Map and Placement Rule, PGs are mapped to object storage devices (OSDs).

[0116] like Figure 4 As shown, taking a Ceph cluster with 512 object storage devices (OSDs) as an example, the calculated number of non-duplicate PGs that can be accommodated, Np, is 21760. The expected total number of PGs in the cluster, Ne, is 16384. Since the number of non-duplicate PGs that can be accommodated, Np, is greater than the expected total number of PGs in the cluster, Ne, the conditions for using the CRUSH algorithm are met. Based on the expected total number of PGs in the cluster, Ne, being 16384, the storage node selection algorithm is used to select three object storage devices (OSDs) for the first PG with three replicas from the expected total number of PGs in the cluster. Mathematical graph theory is used to find the first triangle in the graph. Figure 4 The algorithm mentioned refers to the CRUSH algorithm.

[0117] Step S3042: Continue to allocate the second PG in the expected total number of PGs in the cluster, and use the storage node selection algorithm to select three object storage devices for the three replicas of the second PG in the expected total number of PGs in the cluster. Use mathematical graph theory to find the second triangle, compare the second triangle with the first triangle, and determine whether there are two or more duplicate object storage devices, i.e., whether there is a triangle with the same side. If there is, discard the PG allocation result, increment the calculation count by one, and re-allocate the second PG.

[0118] For example, the first PG three-replica selects three object storage devices (OSDs) to form the first triangle, and the second PG three-replica selects three object storage devices (OSDs) to form the second triangle. The second triangle is compared with the first triangle to determine whether there are two or more duplicate object storage devices, i.e., whether there is a triangle with shared sides. If there is, the PG allocation result is discarded, the calculation count is incremented by one, and the second PG is re-allocated. If not, the allocation result of the second PG is retained.

[0119] Step S3043: Continue to allocate the third PG in the total expected number of PGs in the cluster, find non-shared triangles, until all PGs in the total expected number of PGs in the cluster have been allocated, or no more non-shared triangles can be found, and obtain the first set of non-shared triangles and the PG allocation results.

[0120] Specifically, select three object storage devices (OSDs) for the third PG in the expected total number of PGs in the cluster, and form a third triangle. Compare the third triangle with the first and second triangles to determine if there are any triangles sharing the same side. If there are, discard the allocation result of the third PG, increment the calculation count by one, and re-allocate the third PG. If not, retain the allocation result of the third PG and repeat the above steps until all PGs in the expected total number of PGs in the cluster have been allocated, or no more triangles sharing the same side can be found, thus obtaining the first set of triangles sharing the same side and the PG allocation results.

[0121] After completing steps S3031 to S3033, at least 1 / 3 of the total number of triangles with the largest non-shared sides has been found, meaning that the first set of triangles with the largest non-shared sides includes 1 / 3 of the total number of triangles with the largest non-shared sides.

[0122] Step S3044: When the number of PGs in the PG allocation result is less than the expected total number of PGs in the cluster, traverse the first set of non-shared triangles in ascending order of their numbers, select any specified triangle, and then traverse all object storage devices to determine whether two non-shared triangles can be found that are not shared with the specified triangle in the first set of non-shared triangles. If they can be found, remove the object storage device corresponding to the specified triangle from the first set of non-shared triangles to obtain the second set of non-shared triangles.

[0123] Specifically, if the calculated PG allocation result does not meet the expected total number of PGs for the cluster, that is, the number of PGs in the PG allocation result is less than the expected total number of PGs for the cluster, further enhancement calculations are performed, including:

[0124] Iterate through the first set of non-shared-side triangles T, calculated earlier, in ascending order of their numbers. Select any triangle T1 and remove it from the first set of non-shared-side triangles. Then, iterate through all non-shared-side triangles composed of object storage devices (OSDs) to see if you can find two triangles that do not share sides with any triangle in T (excluding T1). If you can find them, remove triangle T1 from the first set of non-shared-side triangles T to obtain the second set of non-shared-side triangles T'.

[0125] Step S3045: Continue traversing all triangles in the second set of non-shared-side triangles until at least half of the total number of the largest non-shared-side triangles is found.

[0126] Specifically, we continue to traverse all triangles in the second set of non-shared-side triangles T', finding all non-shared-side triangles. Finding all non-shared-side triangles is an NP-hard problem. A greedy algorithm can find at least half of the total number of non-shared-side triangles in polynomial time. A greedy algorithm is a strategy that makes the best or optimal choice at each step, hoping to achieve a globally optimal result.

[0127] The data distribution method for the distributed storage system provided in this embodiment achieves the goal of ensuring that at most one Group Required (PG) becomes unavailable when two disks fail or an OSD crashes. Specifically, no two PGs can have two or more replicas distributed within the same object storage device (OSD). This optimizes PG replica allocation, ensuring that in a Ceph cluster storage pool, at most one PG becomes unavailable when two disks fail or an OSD crashes, and at most one PG experiences data loss when three disks fail. This reduces the potential impact on the range of PGs affected by multiple disk failures, thereby improving the availability of the distributed storage system and mitigating the risk of data loss. Furthermore, by using a greedy algorithm to traverse the set of non-shared-side triangles, finding at least half of the total number of non-shared-side triangles in polynomial time, this method demonstrates good practicality and can create considerable value for society and enterprises.

[0128] As one or more specific application embodiments of the present invention, combined with Figure 4 The data distribution method of the distributed storage system provided by the present invention will be further described in detail below, and the specific process is as follows:

[0129] Step 1) Assume that the Ceph cluster has 512 object storage devices (OSDs), each OSD is expected to have 100 PGs, and the maximum number of replicas is 3.

[0130] Step 2) According to formula (7), the total number of non-repeating PGs that the cluster can accommodate is 43,520. It is ensured that the number of non-repeating PGs that can be calculated in polynomial time is 21,760 (half of the number of non-repeating PGs that can be accommodated). The expected total number of PGs in the cluster (total number of PGs = (number of OSDs * 100) / maximum number of replicas, the result must be rounded to the nearest power of 2) is 16,384 (a power of 2), which meets the conditions, and the algorithm can be used to complete the allocation.

[0131] Step 3) Calculate the distribution of all PGs based on the expected total number of PGs in the cluster, which is 16384. This ensures that no two PGs have two or more duplicate OSDs. The distribution of all PGs is calculated according to the specific process in step S303, which will not be repeated here.

[0132] Following steps 1) to 3), if two disks fail, at most one PG will be unwriteable; if three disks fail, at most one PG will be unreadable, and there is no risk of data loss. Even if more disks fail, the potential data loss will decrease significantly; that is, the probability of two or more PGs becoming inaccessible and the probability of losing data from two or more PGs are both equal to 0. The formula is as follows:

[0133] (5);

[0134] (6).

[0135] The data distribution method for the distributed storage system provided in this embodiment can optimize the allocation of PG replicas. It ensures that when two disks fail or an OSD crashes in the Ceph cluster storage pool, at most only one PG becomes unavailable, and when three disks fail, at most only one PG experiences data loss. This ensures the data security of the distributed storage system, has good practicality, and can create considerable value for society and enterprises.

[0136] This embodiment also provides a data distribution device for a distributed storage system, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0137] This embodiment provides a data distribution device for a distributed storage system, such as... Figure 6 As shown, it includes:

[0138] The PG total calculation module 601 is used to configure multiple object storage devices for a distributed storage system cluster and calculate the expected total number of PGs for the cluster based on the expected number of PGs for each object storage device.

[0139] The module 602 is used to calculate the number of non-repeating PGs that can be accommodated based on the number of object storage devices.

[0140] The PG allocation module 603 is used to allocate PGs in the cluster based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, and to use mathematical graph theory until all the expected total number of PGs in the cluster has been allocated.

[0141] In some alternative implementations, the PG total calculation module 601 includes:

[0142] The data acquisition unit is used to obtain the number of object storage devices in the distributed storage system cluster, and to set the expected number of PGs and the maximum number of PG replicas for each object storage device.

[0143] The cluster expected total PG calculation unit is used to calculate the cluster expected total PG based on the number of object storage devices, the expected number of PGs, and the maximum number of PG replicas. The result of calculating the cluster expected total PG is rounded to a power of 2.

[0144] In some optional implementations, the accommodating non-duplicate PGs are PGs in the distributed storage system cluster where no two PGs have more than one duplicate object storage device; the accommodating non-duplicate PG count calculation module 602 includes:

[0145] The ratio calculation unit is used to calculate the remainder obtained by the ratio of the number of object storage devices to a preset constant.

[0146] The numerical determination unit is used to determine the number of object storage devices in which any two PGs have more than one duplicate, based on the value of the remainder.

[0147] The cell for accommodating non-repeating PGs is used to calculate the number of non-repeating PGs that can be accommodated in polynomial time, based on the number of object storage devices and the number of PGs for which any two PGs have more than one duplicate. The formula for calculating the number of non-repeating PGs that can be accommodated in polynomial time is as follows:

[0148] ;

[0149] in, To accommodate the number of non-repeating PGs in polynomial time, Let n be the number of object storage devices (PGs) where any two PGs have more than one duplicate object storage device. This represents the number of combinations, specifically the number of combinations where two PGs are randomly selected from n individual object storage devices.

[0150] In some alternative implementations, the PG allocation module 603 includes:

[0151] The first PG allocation unit is used to select three object storage devices for the first PG three-replica in the expected total number of PGs in the cluster when the number of non-repeating PGs that can be accommodated is greater than or equal to the total number of PGs expected in the cluster. It uses the selection of storage nodes algorithm to select three object storage devices and uses mathematical graph theory to find the first triangle in the graph.

[0152] The second PG allocation unit is used to continue allocating the second PG out of the expected total number of PGs in the cluster. It uses the storage node selection algorithm to select three object storage devices for the three replicas of the second PG out of the expected total number of PGs in the cluster. It uses mathematical graph theory to find the second triangle, compares the second triangle with the first triangle, and determines whether there are two or more duplicate object storage devices, i.e., whether there is a triangle with shared sides. If there is, the PG allocation result is discarded, the calculation count is incremented by one, and the second PG is reallocated.

[0153] The third PG allocation unit is used to continue allocating the third PG in the expected total number of PGs in the cluster, finding non-shared triangles, until all PGs in the expected total number of PGs in the cluster have been allocated, or no more non-shared triangles can be found, and obtain the first set of non-shared triangles and the PG allocation results.

[0154] The fourth PG allocation unit is used to traverse the first set of non-shared triangles in ascending order of their numbers when the number of PGs in the PG allocation result is less than the total number of expected PGs in the cluster. It selects any specified triangle and then traverses all object storage devices to determine whether two non-shared triangles can be found that are not shared with the specified triangle in the first set of non-shared triangles. If they can be found, the object storage device corresponding to the specified triangle is removed from the first set of non-shared triangles to obtain the second set of non-shared triangles.

[0155] The fifth PG allocation unit is used to continue traversing all triangles in the second set of non-common triangles until at least half of the total number of the largest non-common triangles is found.

[0156] In some alternative implementations, the data distribution apparatus of the distributed storage system further includes:

[0157] The mathematical graph theory module is used to treat each object storage device as a point on a plane, and any three points are not collinear. At the same time, three copies are set for each PG, and the three copies are allocated to three object storage devices respectively. The three object storage devices with copies are regarded as three points, and the three points are connected to form a triangle.

[0158] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0159] In this embodiment, the data distribution device of the distributed storage system is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0160] This invention also provides a computer device having the above-described features. Figure 6 The data distribution device of the distributed storage system shown is shown.

[0161] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 7 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 7 Take a processor 10 as an example.

[0162] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0163] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0164] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0165] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0166] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0167] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0168] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0169] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A data distribution method for a distributed storage system, characterized in that, The method includes: Configure multiple object storage devices for a distributed storage system cluster, and calculate the total expected number of PGs for the cluster based on the expected number of PGs for each object storage device; Calculate the number of non-repeating PGs that can be accommodated in polynomial time based on the number of object storage devices; Based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, mathematical graph theory is used to allocate PGs in the cluster until all the expected total number of PGs in the cluster has been allocated. Based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, mathematical graph theory is used to allocate PGs in the cluster until all expected PGs in the cluster have been allocated, including: When the number of non-repeating PGs that can be accommodated is greater than or equal to the total number of expected PGs in the cluster, the storage node selection algorithm is used to select three object storage devices for the first PG three-replica in the total number of expected PGs in the cluster, and the first triangle in the graph is found using mathematical graph theory. Continue allocating the second PG out of the expected total number of PGs in the cluster, and use the storage node selection algorithm to select three object storage devices for the three replicas of the second PG out of the expected total number of PGs in the cluster. Use mathematical graph theory to find the second triangle, compare the second triangle with the first triangle, and determine whether there are two or more duplicate object storage devices, i.e., whether there is a triangle with the same side. If there is, discard the PG allocation result, increment the calculation count by one, and re-allocate the second PG. Continue allocating the third PG in the expected total number of PGs in the cluster, finding non-shared triangles, until all PGs in the expected total number of PGs in the cluster have been allocated, or no more non-shared triangles can be found, to obtain the first set of non-shared triangles and the PG allocation results.

2. The method according to claim 1, characterized in that, The calculation of the total expected PG of the cluster based on the expected PG of each object storage device includes: Obtain the number of object storage devices in the distributed storage system cluster, and set the expected number of PGs and the maximum number of replicas for each object storage device; The total number of expected PGs in the cluster is calculated based on the number of object storage devices, the expected number of PGs, and the maximum number of replicas of PGs. The result of calculating the total number of expected PGs in the cluster is rounded to a power of 2.

3. The method according to claim 1, characterized in that, The term "accommodating non-duplicate PGs" refers to PGs in a distributed storage system cluster where no two PGs have more than one duplicate object storage device. The calculation of the number of non-repeating PGs that can be accommodated in polynomial time based on the number of object storage devices includes: The remainder obtained by comparing the number of object storage devices with a preset constant; The number of PGs in which any two PGs have more than one duplicate object storage device is determined based on the value of the remainder. The number of non-duplicate PGs that can be accommodated in polynomial time is calculated based on the number of object storage devices and the number of PGs for which any two PGs have more than one duplicate object storage device.

4. The method according to claim 3, characterized in that, The formula for calculating the number of non-repeating PGs that can be accommodated in polynomial time is as follows: ； in, To accommodate a certain number of non-repeating PGs, Let n be the number of object storage devices (PGs) where any two PGs have more than one duplicate object storage device. This represents the number of combinations, specifically the number of combinations where two PGs are randomly selected from n individual object storage devices.

5. The method according to claim 1, characterized in that, Before allocating cluster PGs based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, and using mathematical graph theory, the method further includes: Each object storage device is considered as a point on a plane, and no three points are collinear. Three replicas are set for each PG, and the three replicas are allocated to three object storage devices respectively. The three object storage devices with allocated replicas are considered as three points, and the three points are connected to form a triangle.

6. The method according to claim 1, characterized in that, Based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, and using mathematical graph theory to allocate PGs in the cluster, until all expected PGs in the cluster have been allocated, the following steps are also included: When the number of PGs in the PG allocation result is less than the expected total number of PGs in the cluster, the first set of non-shared triangles is traversed from smallest to largest number, any specified triangle is selected, and then all object storage devices are traversed to determine whether two non-shared triangles can be found that are not shared with the other triangles in the first set of non-shared triangles except for the specified triangle. If they can be found, the object storage device corresponding to the specified triangle is removed from the first set of non-shared triangles to obtain the second set of non-shared triangles. Continue iterating through all triangles in the second set of non-shared-side triangles until at least half of the total number of the largest non-shared-side triangles is found.

7. A data distribution device for a distributed storage system, characterized in that, The device includes: The PG total count calculation module is used to configure multiple object storage devices for a distributed storage system cluster and calculate the expected total number of PGs for the cluster based on the expected number of PGs for each object storage device. A module for calculating the number of non-duplicate PGs that can be accommodated is used to calculate the number of non-duplicate PGs that can be accommodated based on the number of object storage devices. The PG allocation module is used to allocate PGs in the cluster based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, and using mathematical graph theory, until all the expected total number of PGs in the cluster has been allocated. Based on the relationship between the number of non-repeating PGs that can be accommodated and the total number of expected PGs in the cluster, mathematical graph theory is used to allocate PGs in the cluster until all expected PGs in the cluster have been allocated, including: When the number of non-repeating PGs that can be accommodated is greater than or equal to the total number of expected PGs in the cluster, the storage node selection algorithm is used to select three object storage devices for the first PG three-replica in the total number of expected PGs in the cluster, and the first triangle in the graph is found using mathematical graph theory. Continue allocating the second PG out of the expected total number of PGs in the cluster, and use the storage node selection algorithm to select three object storage devices for the three replicas of the second PG out of the expected total number of PGs in the cluster. Use mathematical graph theory to find the second triangle, compare the second triangle with the first triangle, and determine whether there are two or more duplicate object storage devices, i.e., whether there is a triangle with the same side. If there is, discard the PG allocation result, increment the calculation count by one, and re-allocate the second PG. Continue allocating the third PG in the expected total number of PGs in the cluster, finding non-shared triangles, until all PGs in the expected total number of PGs in the cluster have been allocated, or no more non-shared triangles can be found, to obtain the first set of non-shared triangles and the PG allocation results.

8. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the data distribution method of the distributed storage system according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the data distribution method of the distributed storage system according to any one of claims 1 to 6.

Images