Storage pool capacity determination method, apparatus, device, and medium

Abstract

The application provides a storage pool capacity determination method, device, equipment and medium. The method comprises the following steps: obtaining a crush rule, traversing the crush rule, determining a take step in the crush rule and a project subject in the take step, determining a weight ratio of each osd according to a choose step corresponding to each osd in the project subject, and finally determining the capacity of a target storage pool according to the weight ratio of each osd in the crush rule, so that the capacity of the target storage pool obtained is more accurate and reasonable.

Description

Technical Field

[0001] This application relates to the field of cluster technology, and in particular to a method, apparatus, device and medium for determining the capacity of a storage pool. Background Technology

[0002] Crush is Ceph's distributed selection algorithm for data storage and is the core of the Ceph storage engine. Each OSD has a weight, and a crush rule is specified when a storage pool is created.

[0003] In existing technologies, the weight of each OSD and the total weight of all OSDs are obtained by traversing the CRUSH rules. Then, the weight ratio of each OSD is calculated based on the weight of each OSD and the total weight of all OSDs, and the capacity of the storage pool is obtained based on the weight ratio. However, there is a problem that the calculated capacity is inaccurate. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for determining the capacity of a storage pool, in order to solve the problem of inaccurate capacity calculations in the prior art.

[0005] In a first aspect, this application provides a method for determining the capacity of a storage pool, including:

[0006] Retrieve the crush rules for the target storage pool, which is a storage pool in the Ceph cluster;

[0007] Traverse the crush rules to determine the take steps within the crush rules and the project themes within the take steps;

[0008] Based on the choose steps corresponding to each OSD in the project topic, determine the weight ratio of each OSD;

[0009] The capacity of the target storage pool is determined based on the weight ratio of each OSD in the crush rules.

[0010] In this application, the weight ratio of each OSD is determined based on the choose step corresponding to each OSD in the project topic, including:

[0011] For each OSD, obtain the corresponding choose step and the parameter values ​​in the choose step. The types of choose steps include chooseleaf indep, chooseleaf firstn, choose indep, and choose firstn. The parameter values ​​are greater than zero, equal to zero, or less than zero.

[0012] Based on the parameter values, determine the number of OSDs to select in the choose step;

[0013] The weight ratio of each OSD is determined based on the number of selections corresponding to each OSD.

[0014] In this application, the number of OSDs selected in the choose step is determined based on parameter values, including:

[0015] If the parameter value is greater than zero, the quantity is selected as the smaller value between the parameter value and the redundancy of the target storage pool;

[0016] If the parameter value is zero, then the redundancy of the target storage pool is selected.

[0017] If the parameter value is less than zero, the selected quantity is the difference between the parameter value and the redundancy of the target storage pool.

[0018] In this application, the weight ratio of each OSD is determined based on the number of selections corresponding to each OSD, including:

[0019] For each OSD, obtain the corresponding choose step for the OSD and the total weight of all OSDs in the choose step;

[0020] The weight ratio of each OSD is obtained based on the weight of each OSD, the total weight and number of selections for each OSD, and the number of partitions in the target storage pool.

[0021] In this application, the weight ratio of each OSD is obtained based on the weight of each OSD, the total weight and selection number corresponding to each OSD, and the number of partitions in the target storage pool, satisfying the following:

[0022] ;

[0023] in, For the first The weight ratio of each OSD For the first The weight of each OSD For the first Number of selections corresponding to each OSD For the first The total weight corresponding to each OSD This represents the number of partitions.

[0024] In this application, the capacity of the target storage pool is determined based on the weight ratio of each OSD in the crush rule, including:

[0025] Store the weight ratio of each OSD in a data container;

[0026] When traversing to the last item step in the crush rule, obtain the weight ratio of all OSDs in the crush rule according to the data container;

[0027] The effective capacity corresponding to each OSD is obtained by the ratio of the capacity of each OSD to the weight ratio of each OSD.

[0028] Based on the capacity size of each available capacity, select the target available capacity with the smallest capacity from all available capacities;

[0029] The capacity of the target storage pool is obtained based on the target effective capacity and the number of partitions in the target storage pool.

[0030] In this application, the weight ratio of each OSD is stored in a data container, including:

[0031] For each OSD, determine whether the data container already contains the OSD's weight ratio;

[0032] If the data container already contains the weight ratio of OSDs, and the weight ratio of OSDs to be stored is less than the weight ratio of those already stored, then the weight ratio of those already stored is updated according to the weight ratio to be stored.

[0033] If the data container does not contain the weight ratio of OSDs, then the weight ratio of OSDs will be stored in the data container.

[0034] Secondly, this application provides a storage pool capacity determination device, comprising:

[0035] The acquisition module is used to obtain the crush rules of the target storage pool, which is a storage pool in the Ceph cluster.

[0036] The traversal module is used to traverse crush rules, determine the take steps in the crush rules, and the project topics in the take steps;

[0037] The weight ratio module is used to determine the weight ratio of each OSD based on the weight of each OSD in the project topic and the corresponding choose step of each OSD.

[0038] The capacity module is used to determine the capacity of the target storage pool based on the weight ratio of each OSD in the crush rules.

[0039] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0040] The memory stores instructions that the computer executes;

[0041] The processor executes computer-executable instructions stored in memory to implement the method in the first aspect.

[0042] Fourthly, this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method in the first aspect.

[0043] The storage pool capacity determination method, apparatus, device, and medium provided in this application obtain the crush rules, traverse the crush rules, determine the take step in the crush rules and the project topic in the take step, then determine the weight ratio of each OSD according to the choose step corresponding to each OSD in the project topic, and finally determine the capacity of the target storage pool according to the weight ratio of each OSD in the crush rules, so that the capacity of the target storage pool is more accurate and reasonable. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] Figure 1 A schematic diagram illustrating a scenario for determining the capacity of a storage pool, provided as an embodiment of this application;

[0046] Figure 2 A flowchart illustrating a method for determining storage pool capacity provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the cluster storage space tree provided in an embodiment of this application;

[0048] Figure 4 This is a schematic diagram of a storage pool capacity determination device provided in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0050] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0052] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0053] Crush is the distributed selection algorithm for Ceph data storage and the core of the Ceph storage engine. A crush rule is specified when a storage pool is created.

[0054] In a Ceph cluster, all devices are considered as buckets, such as OSDs, hosts, and roots, forming a hierarchical bucket tree. Each OSD has a weight, and a crush rule is specified when creating a storage pool. The crush rule specifies which root to select and on which hosts and OSDs to create the storage pool.

[0055] The crush rule is, for example:

[0056] rule ecrule2

[0057] {

[0058] id 2

[0059] type erasurestep set chooseleaf tries 5

[0060] Step set choose tries 100

[0061] Step take default

[0062] Step Take Host 1

[0063] step chooseleaf indep 1 type osd

[0064] Step emit

[0065] Step Take Host2

[0066] step chooseleaf indep 1 type osd

[0067] Step emit

[0068] Step Take Host3

[0069] step chooseleaf indep 1 type osd

[0070] Step emit

[0071] Step emit

[0072] }

[0073] The current Ceph algorithm for calculating storage pool capacity is: a map is set up to record the weight ratio of each OSD, and the map...<int, float> m.

[0074] 1. Traverse each step of this rule: When encountering a take type step, traverse all OSDs under the item of the step (e.g., default), calculate the total weight sum of all OSDs, and then calculate the weight ratio Wi / sum of each OSD (Wi is the weight of the i-th OSD).

[0075] 2. Iterate through each OSD in m, find the capacity of each OSD, calculate the effective capacity (OSD capacity / OSD weight ratio) for each OSD, and then find the smallest effective capacity available.

[0076] 3. Calculate the number of partitions (rate) of the storage pool based on the storage pool type: When the storage pool type is a replica type, the rate is the number of replicas (e.g., 3 replicas, so it is 3). When the storage pool type is an EC type, the rate is (m+k) / k (m parity blocks, k data blocks).

[0077] 4. The capacity of the storage pool is equal to avail / rate.

[0078] The weighting ratio refers to the ratio of the OSD's capacity to the effective capacity of the storage pool, used to determine how data is distributed across the OSDs in the cluster. OSDs with higher weighting ratios are more likely to be selected as data storage locations.

[0079] Current technology sometimes calculates a smaller capacity than the actual storage pool capacity. Furthermore, two identical crush rules can produce different calculated storage pool capacities using existing technology. Therefore, current technology suffers from inaccurate capacity calculations.

[0080] To address the issue of inaccurate capacity calculations in existing technologies, the inventors discovered that a method can be used to iterate through the crush rules, determine the take steps and project topics within those steps, group projects by topic, determine the weight ratio of each OSD based on its corresponding choose step, and finally determine the target storage pool capacity based on the weight ratio of each OSD in the crush rules.

[0081] The following describes the application scenarios of the storage pool capacity determination method provided in the embodiments of this application.

[0082] Figure 1 This is a schematic diagram illustrating a scenario for determining the capacity of a storage pool, as provided in an embodiment of this application. Figure 1 As shown, the scenario includes a target storage pool, a crush rule corresponding to the target storage pool, and a server. The server is used to obtain the crush rule, traverse the crush rule, determine the take step in the crush rule, and the project topic in the take step. Then, based on the choose step corresponding to each OSD in the project topic, the weight ratio of each OSD is determined. Finally, based on the weight ratio of each OSD in the crush rule, the capacity of the target storage pool is determined.

[0083] Figure 2 This is a flowchart illustrating a method for determining storage pool capacity provided in an embodiment of this application, as shown below. Figure 2 As shown, the method includes:

[0084] S201. Obtain the crush rules for the target storage pool, which is a storage pool in the Ceph cluster.

[0085] In this context, a Ceph cluster can refer to an open-source distributed storage system that supports object storage, block devices, and file systems. A Ceph cluster typically consists of multiple storage nodes that can be deployed in different data centers or racks to meet varying capacity and performance requirements.

[0086] Crush rules, derived using the Crush (Controlled Replication Under Scalable Hashing) algorithm, are used to construct target storage pools and are a core algorithm in the Ceph distributed storage system. They are primarily used to calculate the storage location of data objects, ensuring a balanced distribution of data across the cluster and supporting high availability and flexible cluster scalability.

[0087] S202. Traverse the crush rules to determine the take steps in the crush rules and the project topics in the take steps.

[0088] Traversal can refer to reading line by line according to the crush rule until all lines have been read.

[0089] The `take` step can refer to the step in the `crush` rule used to select an item, which is used to determine the starting point of the algorithm. For example, in `step take default`, the item is `default`, and in `step take host1`, the item is `host1`. The `take` and `emit` steps appear in pairs; each `take` step has a corresponding `emit` step.

[0090] In this application, the project topic is determined through the take step, and then all OSDs under the project topic are statistically analyzed to classify OSDs according to different project topics, which facilitates the subsequent weight ratio calculation.

[0091] S203. Determine the weight ratio of each OSD based on the choose step corresponding to each OSD in the project topic.

[0092] OSD can refer to Object Storage Device, which is a basic storage unit responsible for the storage, retrieval, and management of data copies of objects.

[0093] The `choose` step can have several types: `chooseleaf indep`, `chooseleaf firstn`, `choose indep`, and `choose firstn`. `chooseleaf indep` means independently selecting one OSD from the candidate OSDs to store data in the current step. `chooseleaf firstn` means selecting the first N OSDs from the candidate OSDs to store data. `choose indep` means independently selecting one node from the candidate nodes (which can be of any type) in the current step. `choose firstn` means selecting the first N nodes from the candidate nodes.

[0094] For example, in the rule `step chooseleaf indep 1 type osd`, `chooseleaf` means that in the current step, a single OSD will be selected from the list of candidate OSDs as the data storage location. The `chooseleaf` operation is typically used at the end of a rule to ensure that the final selection is a leaf node (i.e., the actual OSD). `indep` indicates that an independent selection strategy will be used when choosing an OSD, rather than relying on the selection results of previous steps. This means that the selection in the current step will not be affected by the selections in previous steps. `1` indicates that an OSD will be selected in the current step. If the number is greater than 1, then the corresponding number of OSDs will be selected to store multiple replicas of the data. `type osd` specifies that the node type to be selected is an OSD. In a Ceph cluster, an OSD is a daemon responsible for storing data.

[0095] In this application, each project topic includes multiple OSDs, and each OSD is selected through a choose step. Therefore, each OSD has a corresponding choose step, and the weight ratio of the OSD can be obtained through the choose step.

[0096] In some implementations, determining the weight ratio of each OSD based on the choose step corresponding to each OSD in the project topic may include:

[0097] For each OSD, obtain the corresponding choose step and the parameter values ​​in the choose step. The types of choose steps include chooseleaf indep, chooseleaf firstn, choose indep, and choose firstn. The parameter values ​​are greater than zero, equal to zero, or less than zero.

[0098] Based on the parameter values, determine the number of OSDs to select in the choose step;

[0099] The weight ratio of each OSD is determined based on the number of selections corresponding to each OSD.

[0100] The parameter value can refer to the numerical value after the `choose` keyword in the `choose` step. For example, the parameter value in `stepchooseleaf indep 1 type osd` is 1.

[0101] The number of OSDs to be selected in this choose step can be referred to as the number of OSDs to be selected.

[0102] In this application, for each OSD selected in the project topic, there is a corresponding choose step. The number of OSDs selected in this choose step can be determined from the parameter values ​​in this choose step, and the weight ratio of the OSDs can be obtained from the number of OSDs selected in this step.

[0103] For example, project topic host1 includes OSD1, OSD2, and OSD3. If the choose step is `chooseleaf indep 1 type osd`, it means randomly selecting one OSD from OSD1, OSD2, and OSD3 for data storage. In this case, the probability of each OSD being selected is 1 / 3. If the choose step is `chooseleaf indep 3 type osd`, it means randomly selecting three OSDs from OSD1, OSD2, and OSD3 for data storage. In this case, the probability of each OSD being selected is 1. It can be seen that different selection numbers correspond to different probabilities of each OSD being selected. Different probabilities of each OSD being selected result in different weight ratios. The higher the probability of a particular OSD being selected, the higher its weight ratio in the target storage pool. Therefore, this application, based on the parameter values ​​in the choose step, fully considers the probability of each OSD being selected, thereby more accurately determining the weight ratio of each OSD.

[0104] Furthermore, the parameter value can be greater than zero, equal to zero, or less than zero. Based on the parameter value, the number of OSDs to be selected in the choose step can be determined, and may include:

[0105] If the parameter value is greater than zero, the quantity is selected as the smaller value between the parameter value and the redundancy of the target storage pool;

[0106] If the parameter value is zero, then the redundancy of the target storage pool is selected.

[0107] If the parameter value is less than zero, the selected quantity is the difference between the parameter value and the redundancy of the target storage pool.

[0108] Redundancy refers to the degree of duplication or backup of data storage within a system. Specifically, to ensure high data reliability and fault tolerance, Ceph automatically stores multiple copies of each data in the background. This ensures that data is not lost and remains online even in the event of hard drive failure, server malfunction, or power outage. This redundancy design allows Ceph to recover complete data from the remaining copies even if some components fail, thus guaranteeing the stability of the entire storage system and the security of the data.

[0109] Specifically, redundancy methods can include EC type and replica type. For EC type, the redundancy is the ratio of the sum of the number of data blocks and the number of parity blocks to the number of data blocks. For replica type, the redundancy is the number of replicas.

[0110] In some implementations, determining the weight ratio of each OSD based on the number of selections for each OSD may include:

[0111] For each OSD, obtain the corresponding choose step for the OSD and the total weight of all OSDs in the choose step;

[0112] The weight ratio of each OSD is obtained based on the weight of each OSD, the total weight and number of selections for each OSD, and the number of partitions in the target storage pool.

[0113] The weight of an OSD can refer to its storage capacity, performance, or other factors, and is set by the Ceph administrator during configuration.

[0114] The total weight (sum) can refer to the sum of the weights of all OSDs under the project topic to obtain the total weight.

[0115] The number of partitions in the target storage pool can refer to the size of the target storage pool. For EC type, the number of partitions in the target storage pool is the sum of the number of data blocks and the number of parity blocks. For replica type, the number of partitions in the target storage pool is the number of replicas.

[0116] Specifically, based on the weight of each OSD, the total weight and selection number of each OSD, and the number of partitions in the target storage pool, the formula for obtaining the weight ratio of each OSD can be:

[0117] ;

[0118] in, For the first The weight ratio of each OSD For the first The weight of each OSD For the first Number of selections corresponding to each OSD For the first The total weight corresponding to each OSD This represents the number of partitions.

[0119] In this application, the weight ratio of OSDs is adjusted by selecting a certain number of OSDs, which fully considers the probability of OSDs being selected in actual use. Compared with the prior art, which directly uses the ratio of the weight of an OSD to the total weight as the weight ratio of the OSD, the weight ratio of OSDs obtained in this application is more accurate and reasonable.

[0120] S204. Determine the capacity of the target storage pool based on the weight ratio of each OSD in the crush rule.

[0121] In this application, after obtaining the weight ratio of each OSD, the capacity of the target storage pool can be obtained through the weight ratio and capacity of the OSD.

[0122] Specifically, since there are multiple OSDs, there are also multiple target storage pool capacities. To further filter them, the capacity of the target storage pool is determined based on the weight ratio of each OSD in the Crush rule, which may include:

[0123] Store the weight ratio of each OSD in a data container;

[0124] When traversing to the last item step in the crush rule, obtain the weight ratio of all OSDs in the crush rule according to the data container;

[0125] The effective capacity corresponding to each OSD is obtained by the ratio of the capacity of each OSD to the weight ratio of each OSD.

[0126] Based on the capacity size of each available capacity, select the target available capacity with the smallest capacity from all available capacities;

[0127] The capacity of the target storage pool is obtained based on the target effective capacity and the number of partitions in the target storage pool.

[0128] Here, a data container can refer to a container used to calculate the weight ratio of OSDs. For example, the weight ratio of each OSD can be recorded using a map tool.<int, float> In om, the data container is om.

[0129] OSD capacity refers to the size of the OSD's storage capacity, which is set by the Ceph administrator during configuration.

[0130] Effective capacity refers to the storage capacity of the target memory.

[0131] In this application, after traversing the crush rules, it means that the weight ratio of each OSD is recorded in the data container. Based on the capacity of the OSD and the weight ratio, the effective capacity of the target storage pool can be obtained. Each OSD corresponds to an effective capacity. In order to make the calculated capacity of the target storage pool meet the actual capacity of the target storage pool, the effective capacity with the smallest capacity is selected from each effective capacity as the target effective capacity. Then, the ratio of the effective capacity to the number of partitions is used as the capacity of the target storage pool, thereby obtaining the capacity size that the target storage pool can provide, excluding the backup capacity.

[0132] In some implementations, storing the weight ratio of each OSD in a data container may include:

[0133] For each OSD, determine whether the data container already contains the OSD's weight ratio;

[0134] If the data container already contains the weight ratio of OSDs, and the weight ratio of OSDs to be stored is less than the weight ratio of those already stored, then the weight ratio of those already stored is updated according to the weight ratio to be stored.

[0135] If the data container does not contain the weight ratio of OSDs, then the weight ratio of OSDs will be stored in the data container.

[0136] In this application, since the project topics in the take step may include the same OSD, for example, please refer to [reference needed]. Figure 3 In the cluster storage space tree diagram provided in this application embodiment, both default and host1 include osd1. Therefore, different values ​​may be obtained when calculating the weight ratio of osd. In order to minimize the target effective capacity, the weight ratio with the larger value is used as the weight ratio of osd and stored in the data container, so that the target storage pool capacity is more accurate and reasonable.

[0137] For example, assuming the target storage pool is an EC storage pool with k=2 and m=1, where k is the number of data blocks and m is the number of parity blocks, the crush rule used is:

[0138] rule ecrule2

[0139] {

[0140] id 2

[0141] type erasurestep set chooseleaf tries 5

[0142] Step set choose tries 100

[0143] Step take default

[0144] Step Take Host 1

[0145] step chooseleaf indep 1 type osd

[0146] Step emit

[0147] Step Take Host2

[0148] step chooseleaf indep 1 type osd

[0149] Step emit

[0150] Step Take Host3

[0151] step chooseleaf indep 1 type osd

[0152] Step emit

[0153] Step emit

[0154] }

[0155] First, a map is used to record the weight ratio of each OSD.<int,float> om.

[0156] Step 1: Iterate through each step of this rule (step types include take, choose, and emit).

[0157] When the step type is take, record the item for this step and get take_item (item topic).

[0158] When the step type is choose (including chooseleaf indep, chooseleaf firstn, chooseindep, choose firstn), choose_val (number of selections) is calculated based on the choose's val (parameter value). When choose's val equals 0, choose_val is the pool size (number of partitions in the target storage pool); when choose's val is less than 0, choose_val is choose's val + pool size; when choose's val is greater than 0, choose_val is the minimum value between choose's val and pool size.

[0159] When the step type is emit, retrieve all OSDs under the take_item of the take corresponding to emit. Based on the total weight sum of all OSDs in the choose where each OSD is located, record the weight Wi of each OSD. Then check if the OSD is in om. If not, the weight ratio of the OSD is m[id] = Wi * choose_val / (sum * pool size) (where id is the id of the OSD). Otherwise, m[id] is the maximum value between m[id] and Wi * choose_val / (sum * pool size).

[0160] Take and emit appear in pairs. This step is used to calculate the weight ratio of all OSDs under each take / emit pair.

[0161] Step 2:

[0162] Iterate through each OSD in the om calculated in step 1, find the capacity of this OSD, calculate the effective capacity (OSD capacity / OSD weight ratio) through this OSD, and find the smallest effective capacity available.

[0163] Step 3:

[0164] The redundancy of a storage pool is calculated based on its type: for replicas, it is the number of replicas (e.g., 3 replicas, it is 3); for eC, it is (m+k) / k (m parity blocks, k data blocks).

[0165] Step 4:

[0166] Calculate storage pool capacity: avail / rate.

[0167] The storage pool capacity determination method provided in this application embodiment obtains the crush rules, traverses the crush rules, determines the take step in the crush rules and the project topic in the take step, then determines the weight ratio of each OSD according to the choose step corresponding to each OSD in the project topic, and finally determines the capacity of the target storage pool according to the weight ratio of each OSD in the crush rules, so that the capacity of the target storage pool is more accurate and reasonable.

[0168] Figure 4 This is a schematic diagram of a storage pool capacity determination device provided in an embodiment of this application, as shown below. Figure 4 As shown, the device 40 includes:

[0169] The acquisition module 401 is used to acquire the crush rules of the target storage pool, which is a storage pool in the Ceph cluster.

[0170] Traversal module 402 is used to traverse crush rules, determine the take steps in crush rules, and the project topics in take steps;

[0171] The weight ratio module 403 is used to determine the weight ratio of each OSD based on the weight of each OSD in the project topic and the choose step corresponding to each OSD.

[0172] Capacity module 404 is used to determine the capacity of the target storage pool based on the weight ratio of each OSD in the crush rule.

[0173] In some implementations, the weighting module 403 is further used for:

[0174] For each OSD, obtain the corresponding choose step and the parameter values ​​in the choose step. The types of choose steps include chooseleaf indep, chooseleaf firstn, choose indep, and choose firstn. The parameter values ​​are greater than zero, equal to zero, or less than zero.

[0175] Based on the parameter values, determine the number of OSDs to select in the choose step;

[0176] The weight ratio of each OSD is determined based on the number of selections corresponding to each OSD.

[0177] In some implementations, the weighting module 403 is further used for:

[0178] If the parameter value is greater than zero, the quantity is selected as the smaller value between the parameter value and the redundancy of the target storage pool;

[0179] If the parameter value is zero, then the redundancy of the target storage pool is selected.

[0180] If the parameter value is less than zero, the selected quantity is the difference between the parameter value and the redundancy of the target storage pool.

[0181] In some implementations, the weighting module 403 is further used for:

[0182] For each OSD, obtain the corresponding choose step for the OSD and the total weight of all OSDs in the choose step;

[0183] The weight ratio of each OSD is obtained based on the weight of each OSD, the total weight and number of selections for each OSD, and the number of partitions in the target storage pool.

[0184] In some implementations, the weighting module 403 is also used to satisfy:

[0185] ;

[0186] in, For the first The weight ratio of each OSD For the first The weight of each OSD For the first Number of selections corresponding to each OSD For the first The total weight corresponding to each OSD This represents the number of partitions.

[0187] In some embodiments, the capacity module 404 is further used for:

[0188] Store the weight ratio of each OSD in a data container;

[0189] When traversing to the last item step in the crush rule, obtain the weight ratio of all OSDs in the crush rule according to the data container;

[0190] The effective capacity corresponding to each OSD is obtained by the ratio of the capacity of each OSD to the weight ratio of each OSD.

[0191] Based on the capacity size of each available capacity, select the target available capacity with the smallest capacity from all available capacities;

[0192] The capacity of the target storage pool is obtained based on the target effective capacity and the number of partitions in the target storage pool.

[0193] In some embodiments, the capacity module 404 is further used for:

[0194] For each OSD, determine whether the data container already contains the OSD's weight ratio;

[0195] If the data container already contains the weight ratio of OSDs, and the weight ratio of OSDs to be stored is less than the weight ratio of those already stored, then the weight ratio of those already stored is updated according to the weight ratio to be stored.

[0196] If the data container does not contain the weight ratio of OSDs, then the weight ratio of OSDs will be stored in the data container.

[0197] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 includes:

[0198] The electronic device 50 may include a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a communication component 503, and other components. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0199] In the specific implementation process, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above method.

[0200] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0201] In the above Figure 5 In the illustrated embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0202] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0203] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0204] In some embodiments, a computer program product is also provided, comprising a computer program or instructions that, when executed by a processor, implement the steps in any of the methods described above.

[0205] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0206] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0207] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the methods provided in embodiments of this application.

[0208] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0209] According to one aspect of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium.

[0210] Since the instructions stored in the storage medium can execute the steps of any of the methods provided in the embodiments of this application, the beneficial effects that any of the methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0211] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0212] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for determining the capacity of a storage pool, characterized in that, The method comprises the following steps: obtaining a crush rule of a target storage pool in a Ceph cluster; traversing the crush rule to determine a take step in the crush rule and an item subject in the take step; determining a weight ratio of each osd according to a choose step corresponding to each osd in the item subject; storing the weight ratio of each osd in a data container; when traversing to the last item step in the crush rule, obtaining the weight ratio of all osds in the crush rule according to the data container; obtaining an effective capacity corresponding to each osd according to a ratio of a capacity of each osd and the weight ratio of each osd; selecting a target effective capacity with the smallest capacity from all effective capacities according to the capacity of each effective capacity; obtaining the capacity of the target storage pool according to the target effective capacity and the number of partitions of the target storage pool.

2. The method of claim 1, wherein, The method of determining the weight ratio of each osd according to the choose step corresponding to each osd in the item subject comprises the following steps: for each osd, obtaining a choose step corresponding to the osd and a parameter value in the choose step, the type of the choose step comprising a chooseleaf indep type, a chooseleaf firstn type, a choose indep type and a choose firstn type, the parameter value being greater than zero, equal to zero or less than zero; determining the selection number of osds in the choose step according to the parameter value; determining the weight ratio of each osd according to the selection number corresponding to each osd.

3. The method of claim 2, wherein, The method of determining the selection number of osds in the choose step according to the parameter value comprises the following steps: if the parameter value is greater than zero, the selection number is the smaller one of the parameter value and the redundancy of the target storage pool; if the parameter value is equal to zero, the selection number is the redundancy of the target storage pool; if the parameter value is less than zero, the selection number is the difference between the parameter value and the redundancy of the target storage pool.

4. The method of claim 2, wherein, The method of determining the weight ratio of each osd according to the selection number corresponding to each osd comprises the following steps: for each osd, obtaining a choose step corresponding to the osd and a total weight of all osds in the choose step; obtaining the weight ratio of each osd according to the weight of each osd, the total weight corresponding to each osd and the selection number, and the number of partitions of the target storage pool.

5. The method of claim 4, wherein, The method of obtaining the weight ratio of each osd according to the weight of each osd, the total weight corresponding to each osd and the selection number, and the number of partitions of the target storage pool satisfies the following condition: ； wherein, is the weight ratio of the first osd, is the weight of the first osd, is the selection number corresponding to the first osd, is the total weight corresponding to the first osd, is the number of partitions.​​​​ 6. The method of claim 1, wherein, The method of storing the weight ratio of each osd in the data container comprises the following steps: for each osd, determining whether the weight ratio of the osd has been included in the data container; If the data container already contains the weight ratio of the osd, and the to-be-stored weight ratio of the osd is smaller than the stored weight ratio, the stored weight ratio is updated according to the to-be-stored weight ratio; If the data container does not contain the weight ratio of the osd, the weight ratio of the osd is stored into the data container.

7. A storage pool capacity determining apparatus characterized by comprising: The method comprises the steps of: obtaining a crush rule of a target storage pool, wherein the target storage pool is a storage pool in a Ceph cluster; traversing the crush rule to determine a take step in the crush rule and a project subject in the take step; determining a weight ratio of each osd according to a weight of each osd in the project subject and a corresponding choose step of each osd; determining a capacity of the target storage pool according to the weight ratio of each osd in the crush rule; the capacity module is further configured to store the weight ratio of each osd into a data container, and when the last item step in the crush rule is traversed, obtain the weight ratio of all osds in the crush rule according to the data container, obtain an effective capacity corresponding to each osd according to a ratio of the capacity of each osd and the weight ratio of each osd, and select a target effective capacity with the smallest capacity from all effective capacities according to the capacity of each effective capacity; obtain the capacity of the target storage pool according to the target effective capacity and a partition number of the target storage pool.

8. An electronic device, comprising: The method comprises the steps of: a processor and a memory connected to the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method in any one of claims 1-6.

Images