Storage capacity balancing method and device, electronic equipment and storage medium

Abstract

The invention discloses a storage capacity balancing method and device, electronic equipment and a computer readable storage medium. The storage capacity balancing method comprises the steps that the current available capacity of each object storage device OSD in a distributed storage system is obtained; determining an expected balance capacity based on the current available capacity of each OSD; the expected balance capacity refers to a capacity value when the available capacity of each OSD is balanced; determining the configuration weight of each OSD based on the expected balance capacity and the current available capacity of each OSD; determining an OSD polling sequence based on the configuration weight of each OSD; the OSD polling sequence is used for representing the sequence of redirected write-in objects of the OSDs; and selecting each object mapped by the ROW service, and distributing the object selected each time to the corresponding OSD according to the OSD polling sequence. High capacity balance can be achieved among OSDs, and the foreground service IO performance is not affected.

Description

Technical Field

[0001] The present application relates to the field of storage, and in particular to a storage capacity balancing method, device, electronic device and computer-readable storage medium. Background Art

[0002] In the related technology, the distributed storage system can split the file into multiple objects of fixed size, and then map the objects to the placement group (PG) through the hash algorithm, and finally place the PG in the object storage device (OSD) through the controlled scalable copy hashing algorithm (Controlled Replication Under Scalable Hashing, Crush). However, the Crush algorithm is essentially a pseudo-random algorithm, and there is a problem of uneven placement. When any disk in the distributed storage system is full, the entire system will be marked as full to prevent the client from continuing to write. The uneven distribution of data greatly reduces the effective storage space utilization of the distributed storage system. Therefore, it is necessary to balance the storage capacity of the distributed storage system to improve the effective storage space utilization.

[0003] The traditional balancing method is to re-migrate the written data after the business data is written into the distributed storage system when the storage capacity is unbalanced. On the one hand, the balance can only be adjusted in units of PG. PG is a logical unit that carries objects. If the distribution deviation of the hash is not considered, assuming that the probability of each object falling into each PG is equal, then the number of PGs on each OSD cannot be absolutely balanced, and the capacity balance of each OSD is poor. On the other hand, the migration of data from the old OSD to the new OSD increases the amount of data to be migrated, occupies additional system resources, generates write amplification, and reduces the performance of the foreground business input / output (IO) during the data migration process. Summary of the invention

[0004] In view of the above technical problems, the present application provides a storage capacity balancing method, device, electronic device and computer-readable storage medium, and the technical solution is as follows:

[0005] According to a first aspect of the present application, a storage capacity balancing method is provided, the method comprising:

[0006] Obtaining the current available capacity of each object storage device OSD in the distributed storage system, wherein the distributed storage system can map each object split from the file to be stored to the ROW service based on redirection of ROW on write;

[0007] Determine the expected balanced capacity based on the current available capacity of each OSD; the expected balanced capacity refers to the capacity value when the available capacity of each OSD is balanced;

[0008] Determining a configuration weight of each OSD based on the expected balanced capacity and the current available capacity of each OSD;

[0009] Determine an OSD polling order based on the configuration weight of each OSD; the OSD polling order is used to represent the order in which each OSD is redirected to write an object;

[0010] Each object mapped by the ROW service is selected, and each selected object is distributed to the corresponding OSD according to the OSD polling order.

[0011] According to a second aspect of the present application, a storage capacity balancing device is provided, the device comprising:

[0012] An acquisition unit, used for acquiring the current available capacity of each object storage device OSD in the distributed storage system, wherein the distributed storage system can map each object split from the file to be stored to the ROW service based on redirection of ROW on write;

[0013] A determination unit, configured to determine an expected balanced capacity based on the current available capacity of each OSD; the expected balanced capacity refers to a capacity value when the available capacity of each OSD is balanced;

[0014] The determining unit is further configured to determine a configuration weight of each OSD based on the expected balancing capacity and the current available capacity of each OSD;

[0015] The determining unit is further used to determine an OSD polling order based on the configuration weight of each OSD; the OSD polling order is used to represent the order in which each OSD is redirected to write an object;

[0016] The distribution unit is used to select each object of the ROW service mapping, and distribute the selected object each time to the corresponding OSD according to the OSD polling order.

[0017] According to a third aspect of the present application, an electronic device is provided, the electronic device comprising:

[0018] processor;

[0019] a memory for storing processor-executable instructions;

[0020] The processor is configured to implement the method as described in the first aspect.

[0021] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the method described in the first aspect are implemented.

[0022] The technical solution provided by the present application is based on the data writing of the Redirect On Write (ROW) service. At the stage when the object on the ROW service is mapped to the OSD, the expected balanced capacity is determined according to the current available capacity of each OSD in the distributed storage system, that is, the capacity value when the available capacity of each OSD is balanced, and the configuration weight of each OSD is determined based on the expected balanced capacity and the current available capacity of each OSD. Then, the OSD polling order is determined based on the configuration weight of each OSD, that is, the order in which each OSD is redirected to write objects, and each object mapped by the ROW service is selected. The object is split from the file to be stored, and each selected object is distributed to the corresponding OSD according to the OSD polling order. Under the premise of uneven storage capacity of the system, relying on this method for subsequent data writing can achieve a higher capacity balance among the OSDs. At the same time, since the balance is achieved through the writing of subsequent data, no additional data migration is required, which reduces system overhead and does not affect the foreground business IO performance.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0025] Figure 1 It is a schematic diagram of a storage scenario in the related art;

[0026] Figure 2 It is a flowchart of a storage capacity balancing method according to an embodiment of the present application;

[0027] Figure 3 This is a schematic diagram of a storage scenario of an embodiment of the present application;

[0028] Figure 4 This is a schematic diagram of an object distribution process according to an embodiment of the present application;

[0029] Figure 5 This is a schematic diagram of OSD capacity change in one embodiment of the present application;

[0030] Figure 6 It is a structural schematic diagram of a storage capacity balancing device according to an embodiment of the present application;

[0031] Figure 7 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of this application.

[0033] See also Figure 1 , the following is an exemplary introduction to a storage scenario of a distributed storage system in the related technology:

[0034] like Figure 1 As shown, in the related art, the distributed storage system can split the file (File) into multiple objects (Object) according to a fixed size, and then map the object to the placement group (PG) through a hash algorithm, and finally place the PG in the object storage device (OSD) through a controlled scalable copy hashing algorithm (Controlled Replication Under Scalable Hashing, Crush) or a consistent hashing algorithm. In this way, the object and its copy can be placed in the storage device in a deterministic manner only by calculation without relying on the central metadata management service. However, the Crush algorithm is essentially a pseudo-random algorithm, and there is a problem of uneven placement. When any disk in the distributed storage system is full, the entire system will be marked as full to prevent the client from continuing to write. The uneven distribution of data greatly reduces the effective storage space utilization of the distributed storage system. In the Ceph distributed storage system, when the system is marked as full, its overall effective storage space utilization may be less than 70%. Therefore, it is necessary to balance the storage capacity of the distributed storage system to improve the effective storage space utilization.

[0035] The traditional balancing method is to re-migrate the written data after the business data is written to the distributed storage system when the storage capacity is unbalanced. Taking the Ceph distributed storage system as an example, the Ceph distributed storage system adjusts the PG distribution result after placing the PG on the OSD through the load balancing module (Ceph Manager) after Crush, and records the new mapping relationship with upmap entries to achieve fine control of the mapping between PG and OSD to achieve the purpose of balanced distribution of PG. For example, assuming that OSDs [OSD1, OSD2, OSD3] have a total of 10 PGs, and after Crush distribution, the distribution of PG on OSD1, OSD2, and OSD3 is [2,3,5], that is, 2 PGs on OSD1, 3 PGs on OSD2, and 5 PGs on OSD3. Ceph Manager records upmap<PG_X,OSD3,OSD1> , which means remapping PG_X on OSD3 to OSD1. After adjustment, the new PG distribution is [3, 3, 4], which tends to be balanced.

[0036] For the above-mentioned balancing method, on the one hand, the balance can only be adjusted in units of PG. PG is a logical unit that carries objects. If the distribution deviation of the hash is not considered, and it is assumed that the probability of each object falling into each PG is equal, then the number of PGs on each OSD cannot be absolutely balanced, and the capacity balance of each OSD is poor; on the other hand, data is migrated from the old OSD to the new OSD, which increases the amount of migrated data, occupies additional system resources, generates write amplification, and reduces the performance of the foreground business input / output (IO) during the data migration process.

[0037] In view of the above problems, the present application provides a storage capacity balancing method for balancing the capacity of a distributed storage system based on write-time redirection ROW, which can achieve a high capacity balance among various OSDs while reducing system overhead without affecting the foreground business IO performance. Figure 2 As shown, the method comprises the following steps:

[0038] S201. Obtain the current available capacity of each OSD in the distributed storage system.

[0039] The distributed storage system can map each object split from the to-be-stored file to the ROW service based on redirecting the ROW on write.

[0040] S202: Determine the expected balancing capacity based on the current available capacity of each OSD.

[0041] The expected balanced capacity refers to the capacity value when the available capacities of the OSDs are balanced.

[0042] S203: Determine a configuration weight of each OSD based on the expected balancing capacity and the current available capacity of each OSD.

[0043] S204: Determine an OSD polling order based on the configuration weight of each OSD.

[0044] The OSD polling order is used to represent the order in which each OSD is redirected to write an object.

[0045] S205: Select each object mapped by the ROW service, and distribute each selected object to a corresponding OSD according to the OSD polling order.

[0046] The technical solution provided in the embodiment of the present application is based on the data writing of the Redirect On Write (ROW) service. At the stage when the object on the ROW service is mapped to the OSD, the expected balanced capacity is determined according to the current available capacity of each OSD in the distributed storage system, that is, the capacity value when the available capacity of each OSD is balanced, and the configuration weight of each OSD is determined based on the expected balanced capacity and the current available capacity of each OSD. Then, the OSD polling order is determined based on the configuration weight of each OSD, that is, the order in which each OSD is redirected to write objects, and each object mapped by the ROW service is selected. The object is split from the file to be stored, and each selected object is distributed to the corresponding OSD according to the OSD polling order. Under the premise of uneven storage capacity of the system, relying on this method to write subsequent data can achieve a higher capacity balance among the OSDs. At the same time, since the balance is achieved by writing subsequent data, no additional data migration is required, which reduces system overhead and does not affect the foreground business IO performance.

[0047] The distributed storage system of the embodiment of the present application can use the Redirect On Write (ROW) service. ROW is a data storage technology, which is usually used in the snapshot function. Based on the ROW service, when the original data volume creates a snapshot, all write operations will be redirected to a new storage area.

[0048] The distributed storage system of the embodiment of the present application is based on ROW service, which can transform the one-layer mapping of objects to OSDs in the traditional distributed storage system into a two-layer mapping of objects to storage nodes and ROW service to OSDs.

[0049] Combine the following Figure 3 , the above-mentioned "two-layer mapping from object to ROW service of storage node, and ROW service to OSD" is introduced as follows:

[0050] Specifically, it is first necessary to map each object split from the file to be stored to the ROW service: As an example, there may be a logical grouping for object management in the write-time redirection process in the ROW service, and the logical grouping can be understood as a special placement group (that is, different from the placement group PG carried on the OSD in the distributed storage system), which can be responsible for handling the redirection of objects and ensuring that the newly written objects are stored in the correct location; as another example, in order to distinguish it from the placement group PG carried on the OSD, the logical grouping existing in the ROW service in the embodiment of the present application is referred to as "ROW-PG (Redirect-on-Write Placement Group)" ( Figure 3 In the figure, "ROW-PGs" represents multiple ROW-PGs), it can be understood that each ROW-PG can correspond to one ROW service, and there can be multiple ROW-PGs under each ROW service; as another example, each object split from the file to be stored is mapped to the ROW service, that is, the object can be first placed in the ROW-PG through the HASH algorithm, and the ROW-PG is then mapped to a unique ROW service through the Crush algorithm (for example, one ROW-PG corresponds to one ROW service).

[0051] Then, the objects mapped by the ROW service are distributed to the OSD, that is, as described in S205, each object mapped by the ROW service is selected, and each selected object is distributed to the corresponding OSD according to the determined OSD polling order. As an example, Figure 3 As shown, each object selected can be distributed to the placement group (PG) carried by the corresponding OSD according to the determined OSD polling order. The placement group can also be called the redundant placement group to which the OSD belongs. In order to distinguish it from the "ROW-PG" mentioned above, the placement group carried by the OSD in the embodiment of the present application is referred to as "OSD-PG".

[0052] It can be understood that in a distributed storage system, the placement group (PG) carried by the OSD, that is, the above-mentioned OSD-PG, is a logical grouping in the storage pool (Pool) and is used to manage the storage and replication of objects. Each OSD-PG can contain a set of objects, and these objects may be physically distributed on different OSDs. The creation of OSD-PG is specified when the storage pool is created, and it is related to the specified number of replicas. For example, for an OSD-PG with a three-copy redundancy strategy, there will be three identical PGs on three different OSDs, that is, three OSDs can be mapped under the same PG (that is, Figure 3The storage nodes corresponding to the three copies shown in the figure are similar to the storage nodes corresponding to the three copies of the object. The existence of OSD-PG on OSD is similar to the existence of directory. As an example, OSD-PG can finally map the storage nodes corresponding to the multiple copies of the object through the Crush algorithm (such as Figure 3 The three storage nodes (Node) included in the OSDs in the example. It can be understood that Figure 3 The "OSD-PGs" shown may refer to multiple OSD-PGs, and "OSDs" may refer to multiple OSDs.

[0053] For example, object OBJ_ID_1 is placed in ROW_1_PG_1 through HASH calculation. The service ROW_1 corresponding to ROW_1_PG_1 determines OSD_PG_1 through OSD polling order. Taking OSD_PG_1 as an example of three-copy redundancy strategy, it is finally placed on [OSD1, OSD2, OSD3] of OSD_PG_1. ROW_1 can record redirection information<OBJ_ID_1,OSD_PG_1> The redirection mapping from object to OSD is now completed.

[0054] As an example, the distributed storage system can split the file to be stored into multiple objects according to a fixed size. After the file to be stored is split into multiple objects according to a fixed size, each split object can be mapped to a corresponding ROW service. As an example, a ROW service can correspond to one or more objects, and an object corresponds to one ROW service; Figure 3 As shown, as another example, the object can be placed in the ROW-PG through the HASH algorithm, and the ROW-PG can be mapped to a unique ROW service (i.e., one ROW-PG corresponds to one ROW service) through the Crush algorithm. As another example, the distributed storage system can also split the file to be stored into multiple objects according to non-fixed sizes, which is not specifically limited.

[0055] The current available capacity of each OSD can be obtained in various ways. As an example, the OSD can first count its own current available capacity and report the counted current available capacity to the monitoring service of the distributed storage system. This monitoring service can publish the current available capacity of each OSD according to a preset publishing period. As another example, the OSD can also count the information of the redundant placement group to which it belongs (i.e., the information of one or more OSD-PGs carried on the OSD) and report the counted redundant placement group to which it belongs to the monitoring service of the distributed storage system. This monitoring service can also publish the information of the redundant placement group to which each OSD belongs according to a preset publishing period. As another example, the monitoring service can publish both the current available capacity of the OSD and the information of the redundant placement group to which it belongs at the same time, or it can publish them separately. There is no specific limitation on this. As another example, the primary replicas of one or more OSD-PGs carried on the OSD are all this OSD. As another example, for a ROW-based distributed storage system, the current available capacity of each OSD published by the monitoring service can be subscribed through the ROW service, or the information of the redundant placement group to which each OSD belongs published by the monitoring service can be subscribed through the ROW service. It should be noted that the above introduction to the method of obtaining the current available capacity is only an exemplary display. In actual applications, there may be other obtaining methods, and there is no specific limitation on this.

[0056] The following is an exemplary description of the primary replica:

[0057] Taking the redundant placement group OSDs corresponding to OSD-PG1 as [OSD1, OSD2, OSD3] as an example, OSD1 ranked first in this list can be the primary replica of this OSD-PG1. Then, among the one or more OSD-PGs carried by OSD1, this PG1 can be included. The display of the current available capacity of the OSD and the information of the OSD-PGs carried by it can be shown in the form of a list: [<OSD1, current available capacity, <PG_1, PG_4, PG7>>, <OSD2, current available capacity, <PG_2, PG_5, PG_8>>,...].

[0058] There are various ways to determine the expected balanced capacity. As an example, the expected balanced capacity can be determined based on the minimum value among the current available capacities of each OSD.

[0059] There are many ways to determine the expected balancing capacity based on the minimum value of the current available capacity of each OSD. As an example, the correction parameters used to correct the above-mentioned expected balancing capacity can be determined based on the set OSD capacity deviation accuracy, and then the expected balancing capacity is determined based on the determined correction parameters and the minimum value of the current available capacity of each OSD. As another example, if the current available capacity is represented by the percentage of the available capacity of the OSD to the total capacity, and the capacity value of the expected balancing capacity is represented by a percentage, then the correction parameter can be determined within the range indicated by the set OSD capacity deviation accuracy. The correction parameter can be represented by a percentage value. The above-mentioned expected balancing capacity can be the difference between the minimum value of the current available capacity of each OSD and the correction parameter. The difference between the minimum value and the correction parameter represents the expected OSD with the smallest current available capacity. Then, the amount of data that accounts for the correction parameter in the total capacity of the OSD is written, and the distributed storage system achieves capacity balance. In the process of determining the expected balanced capacity based on the above minimum value, a correction parameter is introduced. That is, in the process of capacity balancing, the OSD is expected to write the amount of data represented by the correction parameter on the basis of reaching the minimum value, so as to achieve the expected balanced capacity, instead of directly using the minimum value as the expected balanced capacity. This can make the capacity balancing process smoother and more friendly to the distributed storage system, thereby ensuring system performance. At the same time, the correction parameter is determined based on the OSD capacity deviation accuracy (specification capacity deviation accuracy), which can ensure the OSD product specification requirements.

[0060] As an example, assuming that the total capacity of each OSD is C, the current available capacity of the OSD is the percentage of the available capacity U of the OSD and the total capacity C. The percentage value can be rounded and multiplied by 100 to convert it into an integer, thereby simplifying the calculation and reducing the CPU overhead of the system. For example, if the total capacity of the OSD is 1000G and the available capacity is 511G, the current available capacity can be calculated as <51> .

[0061] The following is an example: As an example, assuming that the current available capacity of the set of OSDs [OSD1, OSD2, OSD3] of each OSD is [X, Y, Z], the minimum value MIN among X, Y, Z can be taken. Since the embodiment of the present application can rely on the data subsequently written in the distributed storage system to achieve capacity balance, it is necessary to set an expected balanced capacity N% as mentioned above. The expected balanced capacity N% is that when the capacity of each OSD in the distributed storage system is expected to be absolutely balanced, the percentage value of the available capacity of each OSD is N%. As another example, taking the OSD capacity deviation accuracy of 3% as an example, in order to make each OSD in the distributed storage system reach a capacity balance state as soon as possible, a correction parameter (for example, the correction parameter is 1%) can be determined according to the range indicated by the OSD capacity deviation accuracy, and the expected balanced capacity N% is set to the difference between the minimum value and the correction parameter, that is, MIN-1 (N=MIN-1), that is, it is expected that when the OSD with the smallest available capacity writes 1% of its total capacity of data, the distributed storage system reaches capacity balance.

[0062] There are many ways to determine the configuration weight of each OSD. As an example, the current available capacity of an OSD can be represented by a percentage of the available capacity of the OSD and its total capacity, and the capacity value of the expected balanced capacity can be represented by a percentage. The difference between the current available capacity and the expected balanced capacity of each OSD can be determined separately, and the configuration weight of each OSD can be determined based on the determined difference.

[0063] It is worth noting that the above-mentioned method for determining the configuration weight of each OSD is only an exemplary display. In practical applications, other determination methods are not excluded and are not specifically limited thereto.

[0064] As an example, based on the above expected balanced capacity, the total amount of data expected to be written in each OSD can be calculated as follows: the expected balanced capacity is subtracted from the current available capacity of each OSD to obtain the expected capacity to be written for each OSD, and the sum of the expected capacities to be written for all OSDs is taken as the total amount of data expected to be written in each OSD; and the value of the expected total amount of data to be written is set as E, then the ratio of the expected capacity to be written for each OSD to the expected total amount of data to be written can be set as the configuration weight of the OSD. For example, the configuration weight W1 of the above OSD1 is (C*(XN)%) / E. Since the values ​​of C and E are equivalent to all OSDs, the configuration weight W1 of OSD1 can be simplified to XN, and the configuration weights [W1, W2, W3] of the above OSDs [OSD1, OSD2, OSD3] are [(XN, (YN), (ZN)].

[0065] There are many ways to determine the OSD polling order. As an example, based on the current weights of each OSD, the OSD with the largest current weight value can be selected as the selected OSD and recorded in order in the OSD table, and the sum of the configuration weights of each OSD is determined as the reference weight; the current weight of the selected OSD is subtracted from the reference weight to obtain the current weight of the selected OSD; the current weight of each OSD is adjusted so that the current weight of each OSD is increased by the configuration weight of the OSD; if the current weight of at least one OSD is not the configuration weight of the OSD, then the step of selecting the OSD with the largest current weight value as the selected OSD based on the current weights of each OSD is returned; if the current weights of each OSD are the configuration weights of each OSD, the OSD polling order is generated based on the OSDs recorded in order in the OSD table. Through the above method, multiple rounds of selection can be performed in each OSD. In each round, the OSD with the largest current weight in this round is selected and recorded from each OSD to obtain an OSD polling order containing multiple rounds. In each round of selection, the current weight of the current round is adjusted by the configuration weight of the OSD, so that the current weight of the OSD changes after each round of selection, so that the capacity change of each OSD is smoother during the capacity balancing process, which is more friendly to the system.

[0066] The following table 1 provides an exemplary introduction to the method of determining the OSD polling order:

[0067] Table 1

[0068]

[0069] According to the configuration weights [W1, W2, W3] of the OSDs [OSD1, OSD2, OSD3] described above, a capacity-weighted polling algorithm is executed to determine the polling order of each OSD, and the OSD polling order is constructed by pre-calculation, as shown in Table 1:

[0070] (1) First, the configuration weights [W1, W2, W3] of OSDs [OSD1, OSD2, OSD3] are taken as the current weights [CW1, CW2, CW3] of OSDs in the first round, and the sum of the configuration weights is calculated as W = W1 + W2 + W3.

[0071] (2) In this round, the OSD with the largest current weight is selected, the currently selected OSD is recorded, and then the current weight of the OSD is subtracted from the sum of the configured weights W to obtain the current weight after selection.

[0072] (3) In this round, the current weight of each OSD is added to its corresponding configuration weight to obtain the current weight before the next round of selection.

[0073] (4) Record the OSD and the OSD-PG information it carries in OSD polling order.

[0074] (5) Repeat steps (1) to (4) until the current weights [CW1, CW2, CW3] are equal to the configured weights [W1, W2, W3] again (i.e., the last round - the eighth round shown in Table 1), then exit the pre-calculation process and obtain the final OSD polling order.

[0075] For example: Assume that the total capacity of a single OSD is 1000G, and the available capacity of OSDs [OSD1, OSD2, OSD3] is [470GB, 480GB, 500GB], then the current available capacity reported by OSDs [X, Y, Z] = [47, 48, 50]. According to the method of determining the expected balanced capacity described above, it is calculated that MIN = 47, N = MIN-1 = 46, and the configuration weights [W1, W2, W3] = [1, 2, 4] are obtained. The current weights of OSDs in the first round [CW1, CW2, CW3] = [1, 2, 4], and the sum of the weights W = W1+W2+W3 = 7. Therefore, in the first round, the current weight of OSD3 is 4, which is the largest. After OSD3 with the largest weight is selected in the first round, the weight of OSD3 is subtracted from W to obtain the weight after selection [CW1, CW2, CW3] = [1, 2, -3]. The weight after selection is added to the configuration weight [W1, W2, W3] to obtain the weight before the next selection [CW1, CW2, CW3] = [2, 4, 1]. By analogy, the execution results of the capacity weighted polling algorithm are shown in Table 1. In the eighth round, [CW1, CW2, CW3] is equal to [W1, W2, W3] again, so the constructed OSD polling order L = [OSD3, OSD2, OSD3, OSD1, OSD3, OSD2, OSD3].

[0076] As another example, another method of determining the OSD polling order may include: taking the proportion of the configuration weight of each OSD in the sum of the configuration weights of all OSDs as the selection probability of the OSD, performing multiple rounds of selection among each OSD, and recording the OSDs selected in each round in sequence in the OSD table; if the number of OSDs recorded in the OSD table is greater than or equal to a quantity threshold, generating an OSD polling order based on the OSDs recorded in sequence in the OSD table.

[0077] It is worth noting that the above-mentioned introduction to the method of determining the OSD polling order is only an exemplary display. In practical applications, other determination methods are not excluded and are not specifically limited to this.

[0078] There are many ways to distribute the objects selected each time to the corresponding OSD according to the OSD polling order. As an example, the objects selected each time can be distributed to the redundant placement group to which the corresponding OSD belongs according to the OSD polling order; wherein each OSD is the primary copy of any redundant placement group to which the OSD belongs. The object distribution process can be distributed based on the redundancy strategy. If the redundant placement group includes a single copy, the same object can be distributed to the single copy. If the redundant placement group includes multiple copies, the same object can be distributed to multiple copies under the redundant placement group at the same time to achieve data redundancy. For example, the OSD-PG carried by OSD1 includes PG1. Assuming that PG1 is a three-copy redundancy strategy, PG1 includes OSD1 as the primary copy, and other copies OSD4 and OSD5. When the same object is distributed to PG1, it can be copied three times and distributed to OSD1, OSD4 and OSD5 at the same time.

[0079] It is understandable that if the redundancy strategy of the redundant placement group is multiple copies, then multiple OSDs belonging to the same redundant placement group, that is, multiple copies of the redundant placement group, generally have the same available capacity.

[0080] As an example, after each selected object is distributed to the redundant placement group to which the corresponding OSD belongs according to the OSD polling order, the mapping relationship <object, OSD-PG> of the object to the redundant placement group (PG) to which the object is distributed (redirected write) can be persisted, and subsequent read requests for the object can be addressed to the corresponding PG through the persisted redirection information.

[0081] See also Figure 4 , the following is an exemplary introduction to the specific process of selecting each object of the ROW service mapping and distributing each selected object to the corresponding OSD according to the OSD polling order:

[0082] In a ROW-based distributed storage system, when a business write IO is distributed to a storage node, the ROW service obtains the redirection information (i.e., the information of the OSD to be distributed) in a polling manner according to the pre-calculated results (OSD polling order), redirects the object and writes it, and then persists the redirection information. This process specifically includes:

[0083] Get the OSD polling order L, such as Figure 4As shown, assuming that the cursor currently selected based on the OSD polling order is CURR, after obtaining the OSD corresponding to CURR and writing the objects currently selected from the objects mapped by the ROW service to the OSD, CURR can be moved to the next OSD in L and the above steps can be repeated. For example: Taking the OSD polling order L = [OSD3, OSD2, OSD3, OSD1, OSD3, OSD2, OSD3] as an example, assuming that the file to be stored is split into 7 objects, and these 7 objects are mapped in the ROW service, then an object can be selected from these 7 objects, and the selection method for each selection can be random selection or selection in a specific order. For example, the first object of the 7 objects is selected in the first selection, and according to the OSD polling order L, the first object is first written to OSD3. In the second selection, the second object of the 7 objects is selected, and according to the OSD polling order L, the second object is written to OSD2. By analogy, the third object is written to OSD3, the fourth object is written to OSD1, the fifth object is written to OSD3, the sixth object is written to OSD2, and the seventh object is written to OSD3.

[0084] In one embodiment, when the currently selected object is written to the corresponding OSD, if the OSD carries multiple PGs (for example, OSD1 carries PG1, PG4, and PG7), any PG can be randomly selected to write the object, for example, PG1 in OSD1 can be randomly selected to write the object, or it can be written in a specific order, for example Figure 4 As shown, the specific writing order of multiple PGs of OSD1 can be a cyclic order of PG1-PG4-PG7-PG1-… Among them, when the currently selected object is written to any PG in any OSD (hereinafter referred to as the "current OSD"), the next selected object will not be repeatedly written to the "current OSD", but will be written to the next OSD (hereinafter referred to as the "next OSD") according to the OSD polling order. It can be understood that if the order of "current OSD" is repeated in the order after "next OSD" according to the OSD polling order, the object selected later can be written to the "current OSD" again, and the PG that has been written to the object last time in the "current OSD" will not be selected again when it is written by other objects again after the "current OSD" according to the same OSD polling order. Instead, other PGs will be selected. For example, Figure 4As shown, for the four objects split from the same file to be stored, if the first object is written to PG1 of OSD1 according to a certain OSD polling order, and the third object is repeatedly written to OSD1, the third object will not be written to PG1 again, but a PG other than PG1 will be selected for writing, such as PG4 or PG7. If there is a specific writing order between multiple PGs, such as a cyclic order of PG1-PG4-PG7-PG1-…, the third object can be written to PG4.

[0085] like Figure 4 As shown, assuming that in any OSD, the index of the OSD-PG selected for distribution this time is IDX, after obtaining the PG corresponding to IDX and completing the distribution of the currently selected object, if other selected objects are written to the same OSD according to the OSD polling order, IDX is moved to the next OSD-PG in the OSD-PG list corresponding to the same OSD, and the other selected objects are distributed again. For example, in OSD1, after the object selected this time selects PG1 for distribution, IDX moves to PG4, and the selected objects can continue to be distributed in PG4.

[0086] As an example, the ROW service can initiate a redirected write request to the OSD corresponding to the selected OSD-PG. After the redirected write request is successful, the mapping relationship <object, OSD-PG> from the object to the redirected PG is persisted. Subsequent read requests for the object can be addressed to the corresponding PG through the persisted redirection information.

[0087] See also Figure 5 The following is an exemplary display of the change in the current available capacity of the OSD when the capacity of the ROW-based distributed storage system is balanced using the storage capacity balancing method of the embodiment of the present application:

[0088] Assume that there are three object storage devices, OSD1, OSD2, and OSD3. The initial available capacity of OSD1, OSD2, and OSD3 is [50G, 48G, and 47G]. The total capacity of each OSD is 100G. The redundancy strategy is single copy (that is, each PG is mapped to one OSD). The number of PGs can be any, and the configuration weight is [1, 2, 4]. Figure 5 As shown, the horizontal axis is the amount of data newly written to the distributed storage system, and the vertical axis is the current available capacity of the OSD. Through the capacity-weighted polling write algorithm described above to balance the capacity, the capacity of OSD1, OSD2, and OSD3 can be accurately balanced as expected.

[0089] Corresponding to the above method embodiment, the present application embodiment also provides a storage capacity balancing device for balancing the capacity of a distributed storage system based on write-time redirection ROW, see Figure 6 As shown, the device may include:

[0090] The acquisition unit 601 is used to acquire the current available capacity of each object storage device OSD in the distributed storage system, wherein the distributed storage system can map each object split from the file to be stored to the ROW service based on the redirection of ROW on write;

[0091] A determination unit 602 is used to determine an expected balanced capacity based on the current available capacity of each OSD; the expected balanced capacity refers to the capacity value when the available capacity of each OSD is balanced;

[0092] The determining unit 602 is further configured to determine a configuration weight of each OSD based on the expected balancing capacity and the current available capacity of each OSD;

[0093] The determining unit 602 is further configured to determine an OSD polling order based on the configuration weight of each OSD; the OSD polling order is used to represent the order in which each OSD is redirected to write an object;

[0094] The distribution unit 603 is used to select each object of the ROW service mapping, and distribute the selected object each time to the corresponding OSD according to the OSD polling order.

[0095] As an example, the determining unit 602 is specifically configured to determine the expected balanced capacity based on a minimum value of current available capacities of each OSD.

[0096] As an example, the determining unit 602 is specifically configured to determine a correction parameter for correcting the expected balancing capacity based on a set OSD capacity deviation accuracy; and determine the expected balancing capacity based on the correction parameter and the minimum value.

[0097] As an example, the current available capacity is represented by a percentage of the available capacity of the OSD to the total capacity, and the capacity value of the expected balanced capacity is represented by a percentage; the determination unit 602 is specifically used to determine the difference between the current available capacity of each OSD and the expected balanced capacity, and determine the configuration weight of each OSD based on the determined difference.

[0098] The determination unit 602 is specifically used to select the OSD with the largest current weight value as the selected OSD based on the current weight of each OSD, and record it in order in the OSD table, and determine the sum of the configuration weights of each OSD as the reference weight; subtract the reference weight from the current weight of the selected OSD to obtain the current weight of the selected OSD; adjust the current weight of each OSD so that the current weight of each OSD is increased by the configuration weight of the OSD; if the current weight of at least one OSD is not the configuration weight of the OSD, return to the step of selecting the OSD with the largest current weight value as the selected OSD based on the current weights of each OSD; if the current weights of each OSD are the configuration weights of each OSD, generate the OSD polling order based on the OSDs recorded in order in the OSD table.

[0099] The determination unit 602 is specifically used to use the proportion of the configuration weight of each OSD in the sum of the configuration weights of all OSDs as the selection probability of the OSD, perform multiple rounds of selection in each OSD, and record the OSDs selected in each round in the OSD table in sequence; if the number of OSDs recorded in the OSD table is greater than or equal to the quantity threshold, then the OSD polling order is generated based on the OSDs recorded in sequence in the OSD table.

[0100] As an example, the distribution unit 603 is specifically used to distribute the objects selected each time to the redundant placement group to which the corresponding OSD belongs according to the OSD polling order; wherein each OSD is the primary copy of any redundant placement group to which the OSD belongs.

[0101] The present application also provides an electronic device, such as Figure 7 As shown, the electronic device includes: a processor 701; a memory 702 for storing processor executable instructions; wherein the processor 701 is configured to implement the storage capacity balancing method described in any of the above embodiments.

[0102] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the storage capacity balancing method described in any one of the above embodiments is implemented.

[0103] The above is only a specific implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A storage capacity balancing method, characterized in that: include: Obtaining the current available capacity of each object storage device OSD in the distributed storage system, wherein the distributed storage system can map each object split from the to-be-stored file to the ROW service based on redirection of ROW on write; Determine the expected balanced capacity based on the current available capacity of each OSD; the expected balanced capacity refers to the capacity value when the available capacity of each OSD is balanced; Determining a configuration weight of each OSD based on the expected balanced capacity and the current available capacity of each OSD; Determine the OSD polling order based on the configuration weight of each OSD; The OSD polling order is used to represent the order in which each OSD is redirected to write objects; Each object mapped by the ROW service is selected, and each selected object is distributed to the corresponding OSD according to the OSD polling order.

2. The method according to claim 1, characterized in that The determining of the expected balancing capacity based on the current available capacity of each OSD includes: The expected balancing capacity is determined based on a minimum value of the current available capacities of the OSDs.

3. The method according to claim 2, characterized in that The determining the expected balanced capacity based on the minimum value of the current available capacity of each OSD includes: Determining a correction parameter for correcting the expected equalization capacity based on a set OSD capacity deviation accuracy; The expected equalization capacity is determined based on the correction parameter and the minimum value.

4. The method according to claim 3, characterized in that The current available capacity is represented by a percentage of the available capacity of the OSD to the total capacity, and the capacity value of the expected balanced capacity is represented by a percentage; The determining the configuration weight of each OSD based on the expected balanced capacity and the current available capacity of each OSD includes: The difference between the current available capacity of each OSD and the expected balanced capacity is determined respectively, and the configuration weight of each OSD is determined based on the determined difference.

5. The method according to claim 4, characterized in that The determining of the OSD polling order based on the configuration weight of each OSD includes: Based on the current weight of each OSD, the OSD with the largest current weight value is selected as the selected OSD and recorded in order in the OSD table, and the sum of the configuration weights of each OSD is determined as the reference weight; the current weight of the selected OSD is subtracted from the reference weight to obtain the current weight of the selected OSD; The current weight of each OSD is adjusted so that the current weight of each OSD is increased by the configured weight of the OSD; If the current weight of at least one OSD is not the configured weight of the OSD, then return to the step of selecting the OSD with the largest current weight value as the selected OSD based on the current weights of each OSD; If the current weights of the OSDs are all the configured weights of the OSDs, the OSD polling order is generated based on the OSDs recorded in order in the OSD table.

6. The method according to claim 4, characterized in that The determining of the OSD polling order based on the configuration weight of each OSD includes: The proportion of each OSD's configuration weight in the sum of all OSD's configuration weights is used as the selection probability of the OSD, multiple rounds of selection are performed in each OSD, and the OSDs selected in each round are recorded in order in the OSD table; If the number of OSDs recorded in the OSD table is greater than or equal to the number threshold, the OSD polling sequence is generated based on the OSDs recorded in sequence in the OSD table.

7. The method according to claim 1, characterized in that The distributing each selected object to the corresponding OSD according to the OSD polling order includes: The objects selected each time are distributed to the redundant placement groups to which the corresponding OSDs belong according to the OSD polling order; wherein each OSD is the primary copy of any redundant placement group to which the OSD belongs.

8. A storage capacity balancing device, characterized in that: include: An acquisition unit, used to acquire the current available capacity of each object storage device OSD in the distributed storage system, wherein the distributed storage system can map each object split from the to-be-stored file to the ROW service based on redirection of ROW on write; A determination unit, configured to determine an expected balanced capacity based on the current available capacity of each OSD; the expected balanced capacity refers to a capacity value when the available capacity of each OSD is balanced; The determining unit is further configured to determine a configuration weight of each OSD based on the expected balancing capacity and the current available capacity of each OSD; The determining unit is further used to determine the OSD polling order based on the configuration weight of each OSD; The OSD polling order is used to represent the order in which each OSD is redirected to write objects; The distribution unit is used to select each object of the ROW service mapping, and distribute the selected object each time to the corresponding OSD according to the OSD polling order.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images