Storage capacity balancing method and apparatus, electronic device, and storage medium

Abstract

The application discloses a storage capacity balancing method and device, electronic equipment and a computer readable storage medium. The storage capacity balancing method comprises the following steps: acquiring the current available capacity of each object storage device (OSD) in a distributed storage system; determining an expected balancing capacity based on the current available capacity of each OSD; the expected balancing capacity refers to the capacity value when the available capacity of each OSD is balanced; determining the configuration weight of each OSD based on the expected balancing capacity and the current available capacity of each OSD; determining the OSD polling order based on the configuration weight of each OSD; the OSD polling order is used to represent the order in which each OSD is redirected to write an object; selecting each object mapped by a ROW service, and distributing each selected object to the corresponding OSD according to the OSD polling order. The capacity balancing among each OSD can be higher and the foreground business IO performance is not affected.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of storage, in particular to a storage capacity balancing method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] In related technologies, a distributed storage system can split a file into multiple objects according to a fixed size, map the objects to a placement group (PG) through a hash algorithm, and finally place the PG into an object storage device (OSD) through a controlled replication under scalable hashing (Crush) algorithm. However, the Crush algorithm is essentially a pseudo-random algorithm, and there is an imbalance problem. When any disk in the distributed storage system is full, the entire system will be marked as full to prevent the client from continuing to write. Imbalanced data distribution greatly reduces the effective storage space utilization of the distributed storage system. Therefore, it is necessary to balance the storage capacity of the distributed storage system to improve the effective storage space utilization.

[0003] A traditional balancing method is to perform data migration to balance the storage capacity when the storage capacity is unbalanced after the business data is written into the distributed storage system. On the one hand, the balancing can only be adjusted in units of PGs. PG is a logical unit that carries objects. If the distribution deviation of the hash is not considered, assuming that the probability of each object falling into each PG is equal, the number of PGs on each OSD cannot be absolutely balanced, and the balancing of the capacity of each OSD is poor. On the other hand, migrating data from an old OSD to a new OSD increases the amount of data to be migrated, occupies additional system resources, causes write amplification, and reduces the performance of front-end business input / output (IO) during the data migration process. SUMMARY

[0004] To solve the above technical problems, the present application provides a storage capacity balancing method, device, electronic equipment and computer readable storage medium, and the technical solutions are as follows:

[0005] According to a first aspect of the present application, a storage capacity balancing method is provided, which comprises:

[0006] Obtaining the current available capacity of each object storage device (OSD) in the distributed storage system, the distributed storage system being capable of mapping each object split from a to-be-stored file to a write-once-redirect-often (ROW) service based on ROW;

[0007] determine an expected equalization capacity based on the current available capacities of the OSDs; the expected equalization capacity refers to a capacity value when the available capacities of the OSDs are equalized;

[0008] determine configuration weights of the OSDs based on the expected equalization capacity and the current available capacities of the OSDs;

[0009] determine an OSD polling order based on the configuration weights of the OSDs; the OSD polling order is used to represent an order in which the OSDs are redirected to write objects;

[0010] select each object of the ROW service mapping, and distribute each selected object to a corresponding OSD according to the OSD polling order.

[0011] According to a second aspect of the present application, a storage capacity equalization apparatus is provided, and the apparatus comprises:

[0012] an obtaining unit, configured to obtain current available capacities of object storage devices (OSDs) in a distributed storage system, the distributed storage system being capable of mapping each object split from a file to be stored to a write-time redirection (ROW) service based on ROW;

[0013] a determining unit, configured to determine an expected equalization capacity based on the current available capacities of the OSDs; the expected equalization capacity refers to a capacity value when the available capacities of the OSDs are equalized;

[0014] The determining unit is further configured to determine configuration weights of the OSDs based on the expected equalization capacity and the current available capacities of the OSDs.

[0015] The determining unit is further configured to determine an OSD polling order based on the configuration weights of the OSDs; the OSD polling order is used to represent an order in which the OSDs are redirected to write objects.

[0016] a distributing unit, configured to select each object of the ROW service mapping, and distribute each selected object to a corresponding OSD according to the OSD polling order.

[0017] According to a third aspect of the present application, an electronic device is provided, and the electronic device comprises:

[0018] a processor;

[0019] a memory for storing processor-executable instructions;

[0020] The processor is configured to implement the method according to the first aspect.

[0021] According to a fourth aspect of the present application, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the method according to the first aspect.

[0022] The technical solution provided in the present application is based on data writing of a redirect on write (ROW) service. In the stage of mapping objects on the ROW service to OSDs, the expected balanced capacity of each OSD is determined according to the current available capacity of each OSD in the distributed storage system, that is, the capacity value when the available capacity of each OSD is balanced. The configuration weight of each OSD is determined based on the expected balanced capacity and the current available capacity of each OSD. The OSD polling order, that is, the order of redirecting the objects to each OSD, is determined based on the configuration weight of each OSD. Each object split from the file to be stored and mapped by the ROW service is selected, and each selected object is distributed to the corresponding OSD according to the OSD polling order. In the premise of unbalanced storage capacity of the system, subsequent data writing is performed in this way, which can achieve high capacity balance among the OSDs. Since the balance is achieved through subsequent data writing, no additional data migration is required, which reduces the system overhead and does not affect the foreground business IO performance.

[0023] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed in the embodiment or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0025] Figure 1 is a storage scenario in the related art;

[0026] Figure 2 is a flowchart of a storage capacity balancing method according to an embodiment of the present application;

[0027] Figure 3 is a storage scenario according to an embodiment of the present application;

[0028] Figure 4 is an object distribution process according to an embodiment of the present application;

[0029] Figure 5 is an OSD capacity change according to an embodiment of the present application;

[0030] Figure 6 is a structural schematic diagram of a storage capacity balancing device according to an embodiment of the present application;

[0031] Figure 7 is a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art should belong to the scope of protection of the present application.

[0033] Please refer to Figure 1 , the following will be exemplarily introduced the storage scenario of a distributed storage system in the related art:

[0034] As shown in Figure 1 , in the related art, the distributed storage system can split a file (File) into multiple objects (Object) according to a fixed size, then map the objects to placement groups (Placement Group, PG) through a hash algorithm, and finally place the PGs into object storage devices (Object Storage Device, OSD) through a controlled scalable replication hash algorithm (Controlled Replication Under Scalable Hashing, Crush) or a consistent hash algorithm. In this way, the object and its replicas can be determinedly placed into the storage devices only through a calculation method without relying on a central metadata management service. However, the Crush algorithm is essentially a pseudo-random algorithm, and there is a problem of uneven placement. When any disk in the distributed storage system is full, the entire system will be marked as full to prevent the client from continuing to write. The uneven distribution of data greatly reduces the effective storage space utilization of the distributed storage system. In the Ceph distributed storage system, the overall effective storage space utilization may be less than 70% when the system is marked as full. Therefore, it is necessary to balance the storage capacity of the distributed storage system to improve the effective storage space utilization.

[0035] The traditional balancing mode is to perform data migration of the written data to achieve balancing when the storage capacity is unbalanced. Taking the Ceph distributed storage system as an example, the Ceph distributed storage system adjusts the PG distribution result after placing the PG to the OSD through the load balancing module (Ceph Manager) and displays the new mapping relationship in the upmap entry to finely control the mapping between the PG and the OSD and achieve the purpose of balanced distribution of the PG. For example, it is assumed that the OSDs [OSD1, OSD2, OSD3] have 10 PGs in total, and the distribution of the PGs on the OSD1, OSD2 and OSD3 is [2, 3, 5] after the distribution by the Crush, that is, there are 2 PGs on the OSD1, 3 PGs on the OSD2 and 5 PGs on the OSD3. The Ceph Manager records the upmap <PG_X, OSD3, OSD1> to represent the remapping of the PG_X on the OSD3 to the OSD1. The new PG distribution is [3, 3, 4] after the adjustment, which tends to be balanced.

[0036] For the above balancing mode, on the one hand, the balancing can only be adjusted in the unit of PG. The PG is a logical unit of a bearing object. If the distribution deviation of the hash is not considered, it is assumed that the probability of each object falling into each PG is equal. The number of PGs on each OSD cannot be absolutely balanced, and the balancing of the capacity of each OSD is poor. On the other hand, the data is migrated from the old OSD to the new OSD, the amount of migrated data is increased, the additional system resources are occupied, the write amplification is generated, and the performance of the front-end business input / output (IO) is reduced during the migration of the data.

[0037] In view of the above problems, the present application provides a storage capacity balancing method for balancing the capacity of a distributed storage system based on write-time redirection (ROW). The method can achieve high capacity balancing among OSDs, reduce system overhead and not affect the IO performance of the front-end business. Figure 2 As shown in the figure, the method comprises the following steps:

[0038] S201, acquiring the current available capacity of each OSD in the distributed storage system.

[0039] The distributed storage system can map each object split from a to-be-stored file to the ROW service based on the write-time redirection (ROW).

[0040] S202, determining the expected balanced capacity based on the current available capacity of each OSD.

[0041] The expected balanced capacity refers to the capacity value when the available capacity of each OSD is balanced.

[0042] S203, determining configuration weights of the OSDs based on the expected balanced capacity and the current available capacities of the OSDs.

[0043] S204, determining an OSD polling order based on the configuration weights of the OSDs.

[0044] The OSD polling order is used to represent the order in which the OSDs are redirected to write objects.

[0045] S205, selecting each object of the ROW service mapping, and distributing each selected object to a corresponding OSD according to the OSD polling order.

[0046] The technical scheme provided by the embodiment of the application is based on data writing of a Redirect On Write (ROW) service. In the stage of mapping objects on the ROW service to OSDs, the expected balanced capacity of each OSD is determined according to the current available capacities of the OSDs in the distributed storage system, that is, the capacity value when the available capacities of the OSDs are balanced, and the configuration weights of the OSDs are determined based on the expected balanced capacity and the current available capacities of the OSDs, and then the OSD polling order is determined based on the configuration weights of the OSDs, that is, the order in which the OSDs are redirected to write objects. Each object of the ROW service mapping, which is split from a file to be stored, is selected, and each selected object is distributed to a corresponding OSD according to the OSD polling order. In the premise that the storage capacities of the system are unbalanced, subsequent data writing is performed in this way, which can achieve high capacity balance among the OSDs, and since the balance is achieved through subsequent data writing, no additional data migration is needed, system overhead is reduced, and the performance of foreground business IO is not affected.

[0047] The distributed storage system of the embodiment of the application can use a Redirect On Write (ROW) service. ROW is a data storage technology, which is usually used in snapshot functions. Based on the ROW service, after a snapshot is created for an original data volume, all write operations are redirected to a new storage area.

[0048] The distributed storage system of the embodiment of the application can change the one-layer mapping of objects to OSDs in the traditional distributed storage system into two-layer mapping of objects to the ROW service of a storage node and then to OSDs.

[0049] The following will be described in combination with Figure 3 The above-mentioned "two-layer mapping of objects to the ROW service of a storage node and then to OSDs" will be described by way of example:

[0050] Specifically, firstly, each object split from the file to be stored needs to be mapped to a ROW service: as an example, in the ROW service, there can be a logical group for object management in the write-time redirection process, which can be understood as a special placement group (i.e. different from the placement group (PG) carried on the OSD in the distributed storage system), which can be responsible for handling the redirection of objects and ensuring that newly written objects are stored to the correct location; as another example, in order to distinguish from the placement group (PG) carried on the OSD, the logical group existing in the ROW service in the embodiment of the present application is referred to as "ROW-PG (Redirect-on-Write Placement Group)" (ROW-PGs represent multiple ROW-PGs in the specification) Figure 3 It can be understood that each ROW-PG can correspond to a ROW service, and multiple ROW-PGs can exist under each ROW service; as another example, each object split from the file to be stored is mapped to a ROW service, that is, the object can be placed in a ROW-PG through a HASH algorithm first, and the ROW-PG is mapped to a unique ROW service (for example, one ROW-PG corresponds to one ROW service) through a Crush algorithm.

[0051] Then, the objects mapped by the ROW service are distributed to the OSD, that is, as described in S205, each object mapped by the ROW service is selected, and each selected object is distributed to the corresponding OSD according to the determined OSD polling sequence, as an example, as shown in Figure 3 each selected object can be distributed to the placement group (Placement Group, PG) carried by the corresponding OSD according to the determined OSD polling sequence, which can also be referred to as the redundant placement group to which the OSD belongs, in order to distinguish from the "ROW-PG" described above, the placement group carried by the OSD in the embodiment of the present application is referred to as "OSD-PG".

[0052] It can be understood that in the distributed storage system, the placement group (Placement Group, PG) carried by the OSD, that is, the OSD-PG described above, is a logical group in the storage pool (Pool), which is used to manage the storage and replication of objects. Each OSD-PG can contain a group of objects, and these objects can be physically distributed on different OSDs. The creation of the OSD-PG is specified when the storage pool is created, which is related to the specified number of copies, for example, for an OSD-PG with a three-copy redundancy strategy, there will be three identical PGs on three different OSDs, that is, the same PG can be mapped to three OSDs (for example, Figure 3The three replicas shown correspond to storage nodes (Node)), and the OSD-PG exists on the OSD in a form similar to that of a directory. As an example, the last OSD-PG can map the multiple replicas of an object to the storage nodes (e.g. Figure 3 It can be understood that, Figure 3 The "OSD-PGs" shown can refer to multiple OSD-PGs, and the "OSDs" can refer to multiple OSDs.

[0053] For example, the object OBJ_ID_1 is placed in ROW_1_PG_1 through HASH calculation, and the service ROW_1 corresponding to ROW_1_PG_1 determines OSD_PG_1 through OSD round-robin order. Taking the three-replica redundancy strategy of OSD_PG_1 as an example, the object is finally placed on [OSD1, OSD2, OSD3] of OSD_PG_1, and the ROW_1 can record the redirection information <OBJ_ID_1, OSD_PG_1> to complete the redirection mapping of the object to the OSD.

[0054] As an example, the distributed storage system can split a file to be stored into multiple objects according to a fixed size. After the file to be stored is split into multiple objects according to the fixed size, each object can be mapped to a corresponding ROW service. As an example, a ROW service can correspond to one or more objects, and an object can correspond to one ROW service. Figure 3 As another example, as shown, an object can be placed in a ROW-PG through a HASH algorithm first, and the ROW-PG can be mapped to a unique ROW service (i.e., one ROW-PG corresponds to one ROW service) through a Crush algorithm. As another example, the distributed storage system can also split a file to be stored into multiple objects according to a non-fixed size, and no limitation is made in this regard.

[0055] The current available capacity of each OSD can be obtained in various ways. As an example, the OSD can first count its current available capacity, and report the counted current available capacity to a monitoring service of the distributed storage system, and the monitoring service can publish the current available capacity of each OSD at a preset publishing period. As another example, the OSD can also count information of a redundancy placement group to which the OSD belongs (i.e., information of one or more OSD-PGs carried on the OSD), and report the counted information of the redundancy placement group to which the OSD belongs to the monitoring service of the distributed storage system, and the monitoring service can also publish the information of the redundancy placement group to which each OSD belongs at a preset publishing period. As another example, the monitoring service can publish the current available capacity of the OSD and the information of the redundancy placement group to which the OSD belongs at the same time, or can publish the two separately, which is not limited in particular. As another example, the primary and secondary copies of one or more OSD-PGs carried on the OSD are the OSD. As another example, for a ROW-based distributed storage system, the current available capacity of each OSD published by the monitoring service can be subscribed through the ROW service, and the information of the redundancy placement group to which each OSD belongs published by the monitoring service can also be subscribed through the ROW service. It should be noted that the above description of the manner of obtaining the current available capacity is only exemplary, and other obtaining manners can exist in actual applications, which is not limited in particular.

[0056] The primary copy is described below by way of example:

[0057] Taking the redundancy placement group OSDs corresponding to the OSD-PG1 as [OSD1, OSD2, OSD3] as an example, the OSD1 at the head of the list can be the primary copy of the OSD-PG1, and one or more OSD-PGs carried on the OSD1 can include the PG1. The current available capacity of the OSD and the information of the OSD-PG carried thereon can be displayed in the form of a list: [<OSD1, current available capacity, <PG_1, PG_4, PG7>], <OSD2, current available capacity, <PG_2, PG_5, PG_8>], …].

[0058] There are various ways to determine the expected balanced capacity. As an example, the expected balanced capacity can be determined based on the minimum value in the current available capacity of each OSD.

[0059] There are various ways to determine the expected balanced capacity based on the minimum of the current available capacities of the OSDs, as an example, a correction parameter for correcting the expected balanced capacity can be determined based on the set OSD capacity deviation precision, and then the expected balanced capacity is determined based on the determined correction parameter and the minimum of the current available capacities of the OSDs. As another example, if the current available capacity is represented as a percentage of the available capacity to the total capacity of the OSD, and the capacity value of the expected balanced capacity is represented as a percentage, the correction parameter can be determined within the range indicated by the set OSD capacity deviation precision, and the correction parameter can be represented as a percentage value, and the expected balanced capacity can be the difference between the minimum of the current available capacities of the OSDs and the correction parameter, which represents the OSD with the minimum expected current available capacity, and then the data amount with a percentage of the correction parameter in the total capacity of the OSD is written again, and the distributed storage system reaches capacity balance. In the process of determining the expected balanced capacity based on the minimum, the correction parameter is introduced, that is, in the process of capacity balance, the expected OSD writes the data amount represented by the correction parameter based on the minimum, to reach the expected balanced capacity, instead of directly taking the minimum as the expected balanced capacity, which can make the process of capacity balance more smooth and more friendly to the distributed storage system, thereby ensuring system performance. At the same time, the correction parameter is determined based on the OSD capacity deviation precision (specification capacity deviation precision), which can ensure the OSD product specification requirements.

[0060] As an example, assume that the total capacity of each OSD is C, and the current available capacity of the OSD is represented as a percentage of the available capacity U to the total capacity C. The percentage value can be rounded and multiplied by 100 to convert it to an integer, thereby simplifying the calculation and reducing the CPU overhead of the system, for example, the total capacity of the OSD is 1000G, and the available capacity is 511G, then the current available capacity can be counted as <51>.

[0061] As an example, assuming that the current available capacities of the set of OSDs OSDs [OSD1, OSD2, OSD3] are [X, Y, Z], the minimum value MIN of X, Y, and Z can be taken, and since the embodiments of the present application can rely on subsequent data written in the distributed storage system to achieve capacity balance, an expected balance capacity N% described above needs to be set, which means that when each OSD in the distributed storage system is expected to be in an absolutely balanced state, the available capacity percentage value of each OSD is N%. As another example, taking 3% as the OSD capacity deviation accuracy, in order to make each OSD in the distributed storage system reach a state of capacity balance as soon as possible, a correction parameter (for example, the correction parameter is 1%) can be determined according to the range indicated by the OSD capacity deviation accuracy, and the expected balance capacity N% is set as the difference between the minimum value and the correction parameter, that is, MIN-1 (N=MIN-1), which means that when the OSD with the minimum available capacity writes in a data amount of 1% of its total capacity, the distributed storage system reaches capacity balance.

[0062] There are various ways to determine the configuration weight of each OSD, as an example, the current available capacity of an OSD can be represented by the percentage of the available capacity of the OSD to its total capacity, and the capacity value of the expected balance capacity can be represented by a percentage, the difference between the current available capacity and the expected balance capacity of each OSD can be determined respectively, and the configuration weight of each OSD can be determined based on the determined difference size.

[0063] It is worth noting that the above description of the way to determine the configuration weight of each OSD is only exemplary, and in actual application, other determination methods can exist, which are not limited in particular.

[0064] As an example, based on the above expected balance capacity, the total amount of data expected to be written in each OSD can be calculated as follows: the expected balance capacity is subtracted from the current available capacity of each OSD to obtain the expected amount of data to be written in each OSD, and the sum of the expected amounts of data to be written in all OSDs is taken as the total amount of data expected to be written in each OSD; and assuming that the value of the total amount of data expected to be written is E, the ratio of the expected amount of data to be written in each OSD to the total amount of data expected to be written can be set as the configuration weight of the OSD. As an example, the configuration weight W1 of OSD1 is (C*(X-N)%) / E, and since the values of C and E are equivalent for all OSDs, the configuration weight W1 of OSD1 can be simplified as X-N, and the configuration weights [W1, W2, W3] of the above OSDs [OSD1, OSD2, OSD3] are [(X-N, (Y-N), (Z-N)].

[0065] There are various ways to determine the OSD polling order. As an example, the OSD with the largest current weight can be selected based on the current weights of the OSDs, and recorded in the OSD table in order, and the sum of the configuration weights of the OSDs is determined as the reference weight; the current weight of the selected OSD is reduced by the reference weight to obtain the current weight of the selected OSD; the current weight of each OSD is adjusted so that the current weight of each OSD increases by the configuration weight of the OSD; if the current weight of at least one OSD is not the configuration weight of the OSD, return to the step of selecting the OSD with the largest current weight based on the current weights of the OSDs; if the current weight of each OSD is the configuration weight of the OSD, generate the OSD polling order based on the OSDs recorded in the OSD table in order. Through the above method, multiple rounds of selection can be performed in each OSD, and in each round, the OSD with the largest current weight in the round is selected and recorded, and the OSD polling order containing multiple rounds is obtained. In each round of selection, the current weight of the current round is adjusted by the configuration weight of the OSD, so that the current weight of the OSD changes after each round of selection, so that the capacity of each OSD changes more smoothly in the capacity balancing process, and the system is more friendly.

[0066] The determination method of the OSD polling order will be described below by way of example with reference to Table 1.

[0067] Table 1

[0068]

[0069] According to the configuration weights [W1, W2, W3] of the OSDs [OSD1, OSD2, OSD3] described above, the capacity weighted round robin algorithm is executed to determine the polling order of each OSD, and the OSD polling order is constructed by pre-computation, as L shown in Table 1.

[0070] (1) First, the configuration weights [W1, W2, W3] of the OSDs [OSD1, OSD2, OSD3] are taken as the current weights [CW1, CW2, CW3] of the OSDs in the first round, and the sum of the configuration weights is W = W1 + W2 + W3.

[0071] (2) The OSD with the largest current weight in the round is selected, the currently selected OSD is recorded, and the current weight of the selected OSD is reduced by the sum of the configuration weights W to obtain the current weight after selection.

[0072] (3) In the round, the current weight of each OSD is increased by its own configuration weight to obtain the current weight before selection in the next round.

[0073] (4) record the OSD polling sequence and the information of OSD-PG carried by the OSD.

[0074] (5) repeat the steps (1) to (4) until the current weights [CW1, CW2, CW3] are equal to the configuration weights [W1, W2, W3] again (i.e. the last round - the eighth round shown in Table 1), exit the pre-computation process, and obtain the final OSD polling sequence.

[0075] For example, assuming that the total capacity of a single OSD is 1000G, and the available capacities of the OSDs [OSD1, OSD2, OSD3] are [470GB, 480GB, 500GB], the current available capacities [X, Y, Z] reported by the OSDs are [47, 48, 50], according to the determination of the expected balanced capacity described above, it can be calculated that MIN = 47, N = MIN - 1 = 46, and the configuration weights [W1, W2, W3] = [1, 2, 4] can be obtained, and the current weights [CW1, CW2, CW3] of the OSDs in the first round are [1, 2, 4], and the sum of the weights W = W1 + W2 + W3 = 7. Therefore, in the first round, the current weight of OSD3 is 4, which is the maximum, and after selecting the OSD3 with the maximum weight in the first round, the weight of OSD3 is reduced by W to obtain the selected weights [CW1, CW2, CW3] = [1, 2, -3], and the selected weights are added to the configuration weights [W1, W2, W3] to obtain the weights [CW1, CW2, CW3] before the next selection, which is [2, 4, 1], and so on. The execution result of the capacity weighted round robin algorithm is shown in Table 1. In the eighth round, [CW1, CW2, CW3] is equal to [W1, W2, W3] again, so the constructed OSD polling sequence L = [OSD3, OSD2, OSD3, OSD1, OSD3, OSD2, OSD3].

[0076] As another example, another way of determining the OSD polling sequence can include: taking the proportion of the configuration weight of each OSD in the sum of the configuration weights of the OSDs as the selection probability of the OSD, selecting in multiple rounds in the OSDs, and recording the OSD selected in each round in sequence to an OSD table; if the number of OSDs recorded in the OSD table is greater than or equal to a number threshold, generating an OSD polling sequence based on the OSDs recorded in sequence in the OSD table.

[0077] It is worth noting that the above introduction of the way of determining the OSD polling sequence is only exemplary, and in actual application, other determination ways can exist, which are not limited in particular.

[0078] There are various ways to distribute each selected object to the corresponding OSD according to the OSD polling sequence, as an example, each selected object can be distributed to the redundant placement group to which the corresponding OSD belongs according to the OSD polling sequence; wherein each OSD is the primary copy of any redundant placement group to which the OSD belongs. The object distribution process can be distributed based on the redundancy strategy, if the redundant placement group includes a single copy, the same object can be distributed to the single copy, if the redundant placement group includes multiple copies, the same object can be simultaneously distributed to multiple copies under the redundant placement group to achieve data redundancy. For example, OSD1 carries OSD-PG including PG1, assuming that the PG1 is a three-copy redundancy strategy, the PG1 includes OSD1 as the primary copy and other copies OSD4 and OSD5, then the same object can be copied three times when distributed to PG1, and simultaneously distributed to OSD1, OSD4 and OSD5.

[0079] It can be understood that if the redundancy strategy of the redundant placement group is multiple copies, the multiple OSDs belonging to the same redundant placement group, i.e. the multiple copies of the redundant placement group, generally have the same available capacity.

[0080] As an example, after distributing each selected object to the redundant placement group to which the corresponding OSD belongs according to the OSD polling sequence, the mapping relationship of the object to the redundant placement group (PG) to which the object is distributed (redirected to write) <object, OSD-PG> can be persisted, and then the read request of the object can be addressed to the corresponding PG through the persisted redirection information.

[0081] Please refer to Figure 4 The specific process of selecting the ROW service mapping of each object and distributing each selected object to the corresponding OSD according to the OSD polling sequence will be exemplarily introduced as follows:

[0082] In the ROW-based distributed storage system, when the business write IO is distributed to a certain storage node, the ROW service obtains the redirection information (i.e. the information of the OSD to which the object needs to be distributed) in a polling manner according to the pre-computed result (OSD polling sequence), writes the object to the corresponding OSD, and then persists the redirection information. This process specifically includes:

[0083] Obtain the OSD polling sequence L, such as Figure 4As shown, assuming the cursor currently selected based on the OSD polling order is CURR, after obtaining the OSD corresponding to CURR and writing the object selected from the objects currently mapped in the ROW service into the OSD, CURR can be moved to the next OSD in L and the above steps can be repeated. For example, taking the OSD polling order L = [OSD3, OSD2, OSD3, OSD1, OSD3, OSD2, OSD3] as an example, assuming that a file to be stored is split into 7 objects, and the 7 objects are mapped in the ROW service, objects can be selected from the 7 objects, and the selection method each time can be random selection or selection in a specific order. For example, the first object in the 7 objects is selected in the first selection, and according to the OSD polling order L, the first object is first written into OSD3. In the second selection, the second object in the 7 objects is selected, and according to the OSD polling order L, the second object is written into OSD2. In this way, the third object is written into OSD3, the fourth object is written into OSD1, the fifth object is written into OSD3, the sixth object is written into OSD2, and the seventh object is written into OSD3.

[0084] In one embodiment, when the currently selected object is written into the corresponding OSD, if the OSD carries multiple PGs (for example, OSD1 carries PG1, PG4, and PG7), any PG can be randomly selected to write the object, for example, the object is written into PG1 in OSD1, or the object can be written in a specific order, for example Figure 4 As shown, the specific write order of the multiple PGs of OSD1 can be a cyclic order of PG1-PG4-PG7-PG1-…, wherein when the currently selected object is written into any PG in any OSD (hereinafter referred to as “current OSD”), the next selected object will not be written into the “current OSD” again, but will be written into the next OSD (hereinafter referred to as “next OSD”) according to the OSD polling order. It can be understood that if the order of the “current OSD” appears again in the order after the “next OSD” according to the OSD polling order, the object selected thereafter can be written into the “current OSD” again, and the PG in the “current OSD” that has been written with an object last time will not be selected again when it is written with another object in the “current OSD” according to the same OSD polling order, but other PGs will be selected. For example, Figure 4As shown, for the 4 objects split from the same file to be stored, if the first object is written to PG1 of OSD1 and the third object is repeatedly written to OSD1 according to a certain OSD polling order, the third object will not be written to PG1 again, but will be written to a PG other than PG1, such as PG4 or PG7. If there is a specific write order among multiple PGs, such as a cyclic order of PG1-PG4-PG7-PG1-…, the third object can be written to PG4.

[0085] As shown, assuming that in any OSD, the index of the OSD-PG selected for distribution this time is IDX, after obtaining the PG corresponding to IDX and completing distribution of the current selected object, if other selected objects are written to the same OSD according to the OSD polling order, IDX is moved to the next OSD-PG in the OSD-PG list corresponding to the same OSD, and the other selected objects are distributed again. For example, in OSD1, after the selected object selects PG1 for distribution, IDX is moved to PG4, and the selected object can continue to be distributed in PG4. Figure 4

[0086] As an example, the ROW service can initiate a redirect write request to the OSD corresponding to the selected OSD-PG according to the selected OSD-PG. After the redirect write request is successful, the mapping relationship between the object and the redirected PG <object, OSD-PG> is persisted, and subsequent read requests for the object can be addressed to the corresponding PG through the persisted redirection information.

[0087] Referring to Figure 5 , the following illustrates the change in the current available capacity of the OSD when the storage capacity balancing method of the embodiments of the present application balances the capacity of the ROW-based distributed storage system:

[0088] Assuming that there are three object storage devices OSD1, OSD2, and OSD3, the initial available capacities of OSD1, OSD2, and OSD3 are [50G, 48G, 47G], the total capacity of each OSD is 100G, the redundancy strategy is single replication (i.e., one OSD is mapped under each PG), the PG can be any number, and the configuration weight is [1, 2, 4]. As shown in Figure 5 , the horizontal axis is the amount of data newly written to the distributed storage system, and the vertical axis is the current available capacity of the OSD. Through the capacity weighted polling write algorithm described above, the capacities of OSD1, OSD2, and OSD3 can be accurately balanced to the expected balance.

[0089] ​Corresponding to the above method embodiments, the embodiments of the present application also provide a storage capacity balancing device for balancing the capacity of a distributed storage system based on write-time redirection (ROW), as shown in Figure 6 The device can include:

[0090] An acquisition unit 601 is configured to acquire the current available capacity of each object storage device (OSD) in a distributed storage system, wherein the distributed storage system is capable of mapping each object split from a file to be stored to a ROW service based on write-time redirection (ROW);

[0091] A determination unit 602 is configured to determine an expected balancing capacity based on the current available capacity of each OSD; the expected balancing capacity refers to the capacity value when the available capacity of each OSD is balanced.

[0092] The determination unit 602 is further configured to determine a configuration weight of each OSD based on the expected balancing capacity and the current available capacity of each OSD.

[0093] The determination unit 602 is further configured to determine an OSD polling order based on the configuration weight of each OSD; the OSD polling order is used to represent the order in which each OSD is redirected to write an object.

[0094] A distribution unit 603 is configured to select each object mapped by the ROW service, and distribute each selected object to a corresponding OSD according to the OSD polling order.

[0095] As an example, the determination unit 602 is specifically configured to determine the expected balancing capacity based on the minimum value in the current available capacity of each OSD.

[0096] As an example, the determination unit 602 is specifically configured to determine a correction parameter for correcting the expected balancing capacity based on a set OSD capacity deviation precision; and determine the expected balancing capacity based on the correction parameter and the minimum value.

[0097] As an example, the current available capacity is represented by the percentage of the available capacity to the total capacity of the OSD, and the capacity value of the expected balancing capacity is represented by a percentage; the determination unit 602 is specifically configured to determine the difference between the current available capacity and the expected balancing capacity of each OSD respectively, and determine the configuration weight of each OSD based on the determined difference size.

[0098] The determining unit 602 is specifically configured to select an OSD with the maximum current weight value as a selected OSD based on the current weights of the OSDs, and record the selected OSD in an OSD table in sequence, and determine a sum of the configuration weights of the OSDs as a reference weight; subtract the reference weight from the current weight of the selected OSD to obtain the current weight of the selected OSD; adjust the current weight of each OSD to increase the configuration weight of the OSD; if the current weight of at least one OSD is not the configuration weight of the OSD, return to the step of selecting an OSD with the maximum current weight value as a selected OSD based on the current weights of the OSDs; and if the current weight of each OSD is the configuration weight of the OSD, generate the OSD polling sequence based on the OSDs recorded in the OSD table in sequence.

[0099] The determining unit 602 is specifically configured to take the proportion of the configuration weight of each OSD in the sum of the configuration weights of the OSDs as the selection probability of the OSD, select the OSDs in multiple rounds, and record the OSDs selected in each round in an OSD table in sequence; and if the number of the OSDs recorded in the OSD table is greater than or equal to a threshold number, generate the OSD polling sequence based on the OSDs recorded in the OSD table in sequence.

[0100] As an example, the distributing unit 603 is specifically configured to distribute the selected object each time to a redundancy placement group to which the corresponding OSD belongs according to the OSD polling sequence; and each OSD is the master copy of any redundancy placement group to which the OSD belongs.

[0101] The present application also provides an electronic device, as shown in the figure, which comprises a processor 701 and a memory 702 for storing processor-executable instructions; wherein the processor 701 is configured to implement the storage capacity balancing method described in any of the above embodiments. Figure 7

[0102] The present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the storage capacity balancing method described in any of the above embodiments.

[0103] The above is only a specific embodiment of the present application, and it should be noted that for those skilled in the art, without departing from the principle of the present application, a number of improvements and refinements can be made, which should also be regarded as the protection scope of the present application.​

Claims

1. A storage capacity equalization method characterized by, The method comprises the following steps: acquiring current available capacities of object storage devices (OSDs) in a distributed storage system, the distributed storage system being capable of mapping objects split from a file to be stored to a write-once-redirect-many (ROW) service based on ROW; determining an expected equalization capacity based on the current available capacities of the OSDs, the expected equalization capacity being a capacity value when the available capacities of the OSDs are equalized; determining configuration weights of the OSDs based on the expected equalization capacity and the current available capacities of the OSDs; determining an OSD polling sequence based on the configuration weights of the OSDs; the OSD polling sequence representing an order in which the OSDs are redirected to write objects; selecting the objects mapped to the ROW service, and distributing each selected object to a corresponding OSD according to the OSD polling sequence.

2. The method of claim 1, wherein, The method further comprises the following steps: determining the expected equalization capacity based on a minimum value in the current available capacities of the OSDs.

3. The method of claim 2, wherein the step of determining the expected equalization capacity based on the minimum value in the current available capacities of the OSDs comprises the following steps: determining a correction parameter for correcting the expected equalization capacity based on a set OSD capacity deviation precision; determining the expected equalization capacity based on the correction parameter and the minimum value. The current available capacities of the OSDs are represented by percentages of available capacities of the OSDs relative to total capacities, and the capacity value of the expected equalization capacity is represented by a percentage.

4. The method of claim 3, wherein, The method further comprises the following steps: respectively determining differences between the current available capacities of the OSDs and the expected equalization capacity, and determining the configuration weights of the OSDs based on the determined difference sizes. The method further comprises the following steps:

5. The method of claim 4, wherein, selecting an OSD with a maximum current weight as a selected OSD based on the current weights of the OSDs, and recording the selected OSD in an OSD table in order, and determining a reference weight as a sum of the configuration weights of the OSDs; and subtracting the reference weight from the current weight of the selected OSD to obtain a new current weight of the selected OSD; adjusting the current weights of the OSDs so that the current weight of each OSD is increased by the configuration weight of the OSD. If the current weight of at least one OSD is not the configuration weight of the OSD, returning to the step of selecting an OSD with a maximum current weight as a selected OSD based on the current weights of the OSDs. If the current weights of the OSDs are all the configuration weights of the OSDs, generating the OSD polling sequence based on the OSDs recorded in the OSD table in order. The method further comprises the following steps:

6. The method of claim 4, wherein, selecting an OSD with a maximum current weight as a selected OSD based on the current weights of the OSDs, and recording the selected OSD in an OSD table in order, and determining a reference weight as a sum of the configuration weights of the OSDs; and subtracting the reference weight from the current weight of the selected OSD to obtain a new current weight of the selected OSD; adjusting the current weights of the OSDs so that the current weight of each OSD is increased by the configuration weight of the OSD. If the current weight of at least one OSD is not the configuration weight of the OSD, returning to the step of selecting an OSD with a maximum current weight as a selected OSD based on the current weights of the OSDs. If the current weights of the OSDs are all the configuration weights of the OSDs, generating the OSD polling sequence based on the OSDs recorded in the OSD table in order. The method further comprises the following steps: selecting an OSD with a maximum current weight as a selected OSD based on the current weights of the OSDs, and recording the selected OSD in an OSD table in order, and determining a reference weight as a sum of the configuration weights of the OSDs; and subtracting the reference weight from the current weight of the selected OSD to obtain a new current weight of the selected OSD; adjusting the current weights of the OSDs so that the current weight of each OSD is increased by the configuration weight of the OSD. If the number of OSDs recorded in the OSD table is greater than or equal to the number threshold, the OSD polling order is generated based on the OSDs recorded in the OSD table in sequence.

7. The method of claim 1, wherein, The distributing each selected object to the corresponding OSD according to the OSD polling order comprises: Distributing each selected object to a redundancy placement group to which the corresponding OSD belongs according to the OSD polling order; wherein each OSD is a primary copy of any redundancy placement group to which the OSD belongs.

8. A storage capacity equalization apparatus, characterized by comprising: Comprise: An acquisition unit configured to acquire current available capacities of object storage devices (OSDs) in a distributed storage system, the distributed storage system being capable of mapping each object split from a file to be stored to a write-once-redirect-often (ROW) service based on write-once-redirect-often (ROW); A determination unit configured to determine an expected balancing capacity based on the current available capacities of the OSDs; the expected balancing capacity being a capacity value when the available capacities of the OSDs are balanced; The determination unit is further configured to determine configuration weights of the OSDs based on the expected balancing capacity and the current available capacities of the OSDs; The determination unit is further configured to determine an OSD polling order based on the configuration weights of the OSDs; The OSD polling order is used to represent an order in which the OSDs are redirected to write objects; A distribution unit configured to select each object mapped by the ROW service and distribute each selected object to a corresponding OSD according to the OSD polling order.

9. An electronic device, comprising: Comprise: A processor; A memory for storing processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 7. The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 7.

Images