A node capacity release method, device and electronic equipment

Abstract

This application discloses a node capacity release method, apparatus, and electronic device. This embodiment triggers node capacity release based on insufficient domain capacity, such as the current remaining capacity of online storage nodes in the domain, rather than relying on user-defined periodic rules. This ensures normal data storage even in scenarios such as sudden offline storage nodes causing the available online capacity (i.e., storage space) to be unable to handle data writing pressure, or a sudden increase in write data. Furthermore, in this embodiment, the corresponding storage space is released based on the data occupancy of each bucket corresponding to the target domain (the storage space currently occupied by the data in that bucket on the storage nodes of the target domain), achieving the goal of releasing storage space as quickly as possible and balancing the amount of data released from each bucket.

Description

Technical Field

[0001] This application relates to the field of storage, and in particular to a method, apparatus, and electronic device for releasing node capacity. Background Technology

[0002] In applications, in order to ensure the storage of the latest data, such as video recordings in security monitoring, it is often necessary to delete the oldest stored data, such as video recordings, based on a cyclic overwrite mechanism. This can be simply referred to as node capacity release.

[0003] The current cyclic overwrite mechanism automatically deletes the oldest stored data according to user-defined periodic rules, freeing up storage space to ensure that the latest data can be written and stored normally. The periodic rules can be set according to actual needs, such as a 2-day cycle. However, this method of releasing storage space according to a fixed periodic rule often cannot cope with scenarios such as sudden offline of storage nodes leading to insufficient online available capacity (i.e., storage space) to handle data writing pressure, or sudden increases in write data, thus failing to guarantee the normal storage of the latest data. Summary of the Invention

[0004] This application discloses a node capacity release method, apparatus, and electronic device, which triggers node capacity release based on dynamically changing domain capacity to ensure the normal storage of the latest data.

[0005] This application provides a node capacity release method, which is applied to a management node in a cloud storage cluster. The cloud storage cluster also includes storage nodes, and the storage nodes in the cloud storage cluster are divided into at least one domain, with each domain corresponding to at least one bucket used for storing data. The method includes:

[0006] For each domain, based on the current remaining capacity of the online storage nodes in that domain, determine whether that domain is a target domain for which node capacity needs to be released;

[0007] When a domain is determined to be a target domain for which node capacity needs to be released, the target capacity to be released in the target domain is determined based on the total capacity of online storage nodes in the target domain and the current remaining capacity of online storage nodes in the target domain.

[0008] Based on the target capacity and the data occupancy of each bucket corresponding to the target domain, the target data amount to be released in each bucket is determined; wherein, the data occupancy of each bucket refers to the storage space of the storage node in the target domain currently occupied by the data in that bucket;

[0009] For each bucket, based on the target amount of data to be released in that bucket, the corresponding storage node is notified to release the written data in the order of writing time from front to back; wherein the amount of data released is less than or equal to the target amount of data to be released in that bucket.

[0010] This application provides a node capacity release device, which is applied to a management node in a cloud storage cluster. The cloud storage cluster further includes storage nodes, and the storage nodes in the cloud storage cluster are divided into at least one domain, with each domain corresponding to at least one bucket used for storing data. The device includes:

[0011] The determining unit is used, for each domain, to determine whether the domain is a target domain for which node capacity needs to be released, based on the current remaining capacity of the online storage nodes in that domain; and,

[0012] When a domain is determined to be a target domain for which node capacity needs to be released, the target capacity to be released in the target domain is determined based on the total capacity of online storage nodes in the target domain and the current remaining capacity of online storage nodes in the target domain.

[0013] Based on the target capacity and the data occupancy of each bucket corresponding to the target domain, the target data amount to be released in each bucket is determined; wherein, the data occupancy of each bucket refers to the storage space occupied by the data in that bucket in the storage nodes of the target domain.

[0014] The release unit is used to, for each bucket, notify the corresponding storage node to release the written data in the order of writing time from front to back, based on the target data volume to be released in that bucket; wherein, the amount of data to be released is less than or equal to the target data volume to be released in that bucket.

[0015] This application provides an electronic device, which includes: a processor and a machine-readable storage medium;

[0016] The machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0017] The processor is used to execute machine-executable instructions to implement the steps of the method described above.

[0018] As can be seen from the above technical solutions, this embodiment triggers node capacity release based on insufficient domain capacity, such as the current remaining capacity of online storage nodes in the domain, rather than relying on user-defined periodic rules. This ensures that data can be stored normally even in scenarios such as sudden offline of storage nodes causing the online available capacity (i.e., storage space) to be unable to handle data writing pressure, or sudden increase in written data.

[0019] Furthermore, in this embodiment, the corresponding storage space is released according to the data occupancy of each bucket corresponding to the target domain (the storage space currently occupied by the data in the bucket in the storage node of the target domain), thereby achieving the purpose of releasing storage space as soon as possible and balancing the amount of data released from each bucket.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification.

[0022] Figure 1 A flowchart illustrating the method provided in this application embodiment;

[0023] Figure 2 A flowchart illustrating the implementation of step 102 provided in this application embodiment;

[0024] Figure 3 A flowchart illustrating the implementation of step 103 provided in this application embodiment;

[0025] Figure 4 This is a structural diagram of the device provided in the embodiments of this application;

[0026] Figure 5 Provided for the embodiments of this application Figure 4 The hardware structure diagram of the device is shown. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0028] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0029] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0030] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0031] See Figure 1 , Figure 1 This is a flowchart illustrating a method provided in an embodiment of this application. The method is applied to a management node in a cloud storage cluster. The cloud storage cluster also includes storage nodes.

[0032] The management node is used to manage the storage nodes, such as saving index information like the write time of data written to the storage node and the location of the data on the storage node.

[0033] In this embodiment, the cloud storage cluster can be divided into at least one domain, and storage nodes are added to the corresponding domains. A corresponding bucket is created for each domain. Data written by users to a bucket is stored in the storage nodes within the corresponding domain, achieving data isolation by domain. Optionally, in this embodiment, when one domain corresponds to one bucket, this bucket can be simply referred to as the resource pool of that domain.

[0034] In this embodiment, the management node and storage node can also be used to count the data occupancy of each bucket on the node. For example, when writing data, the storage node and management node respectively execute the increment record of the bucket capacity, and when deleting data, the storage node and management node respectively execute the decrement record of the bucket capacity.

[0035] like Figure 1 As shown, the method includes the following steps:

[0036] Step 101: For each domain, determine whether the domain is a target domain for which node capacity needs to be released, based on the current remaining capacity of the online storage nodes in that domain.

[0037] As can be seen, in this step 101, the release of node capacity for the domain is triggered in real time based on the current remaining capacity of the online storage nodes in the domain (i.e., the domain capacity remaining online), without relying on traditional periodic configuration. In this way, even if there are storage nodes offline in the domain, it will not affect the node capacity release provided by the embodiments of this application.

[0038] As for how to trigger node capacity release for a domain based on the current remaining capacity of online storage nodes in the domain (i.e., the domain's remaining online capacity), there are many ways. For example, determine the total current remaining capacity N1 of online storage nodes in the domain; determine the proportion of N1 to the total capacity N2 of all online storage nodes in the domain. If the proportion is less than a preset proportion threshold, the domain is determined to be the target domain for releasing node capacity, that is, node capacity release needs to be triggered for the domain; otherwise, if the proportion is greater than the preset proportion threshold, it is determined that node capacity release does not need to be triggered for the domain.

[0039] As can be seen, in this embodiment, the preset ratio threshold is used as the basis for determining the target domain. In specific implementation, the above-mentioned preset ratio threshold can be set according to actual needs, such as 10%, etc., and this embodiment does not specifically limit it.

[0040] Step 102: When the domain is determined to be the target domain for which node capacity needs to be released, the target capacity to be released in the target domain is determined based on the total capacity of online storage nodes in the target domain and the current remaining capacity of online storage nodes in the target domain.

[0041] Step 102 determines the total capacity to be released in the entire target domain, which is the target capacity mentioned above.

[0042] As for how to determine the target capacity to be released in the target domain based on the total capacity and the current remaining capacity of the online storage nodes in the target domain, the following will explain... Figure 2 Examples will be provided, but will not be elaborated upon here.

[0043] Step 103: Determine the target data volume to be released for each bucket based on the target capacity and the data occupancy of each bucket corresponding to the target domain; wherein, the data occupancy of each bucket refers to the storage space of the storage node in the target domain currently occupied by the data in that bucket.

[0044] As an example, the aforementioned management node is a node deployed in the cloud storage cluster used to perform node capacity release. Under this premise, the management node can determine the data occupancy of each bucket based on the recorded bucket object index table. The bucket object index table is used to record the storage space currently occupied by the data in the bucket on the storage nodes in the target domain.

[0045] As another embodiment, the management node can also be a storage node selected from all storage nodes in the cloud storage cluster to perform node capacity release, such as selecting the online storage node with the highest number as the management node, etc., and this embodiment is not specifically limited. Under this premise, the management node can interact with each storage node to know the storage space occupied by each bucket on each storage node, and deduce the data usage of each bucket based on the storage space occupied by each bucket on each storage node. In this embodiment, by selecting an online storage node as the management node, the cyclic overlay logic (i.e., the node capacity release method) under high availability can be implemented, ensuring the robustness of the cyclic overlay logic (i.e., the node capacity release method) under the cloud storage cluster.

[0046] After obtaining the data occupancy of each bucket corresponding to the target domain, the target data amount to be released in each bucket can be determined based on the ratio of the target capacity to the data occupancy of each bucket.

[0047] As an example, the final determined proportion of the target data volume to be released in each bucket is the proportion of the data usage in each bucket, and the sum of the target data volumes to be released in each bucket is the target capacity. For instance, if the target domain corresponds to bucket A and bucket B, and the data usage ratio of bucket A to bucket B is 1:2, then the target data volumes to be released in bucket A and bucket B are determined according to the 1:2 ratio. For example, if the target capacity is 6G, the target data volume to be released in bucket A is 2G, and the target data volume to be released in bucket B is 4G.

[0048] As another embodiment, to avoid excessive overwriting of bucket data, this embodiment also needs to consider minimum data volume protection logic when determining the target data volume to be released for each bucket, so as to ensure that buckets with relatively small data volumes are not released. The following will illustrate this further. Figure 3 Examples will be provided, but will not be elaborated upon here.

[0049] Step 104: For each bucket, based on the target data volume to be released in the bucket, notify the corresponding storage node to release the written data in the order of writing time from front to back; wherein the amount of data released is less than or equal to the target data volume to be released in the bucket.

[0050] Optionally, in this embodiment, based on the index of each bucket object, the corresponding storage node can be notified to release the written data in the order of writing time from front to back, and the earliest data will be released in the end, with the total capacity of released data not less than the target data capacity of the bucket to be released.

[0051] This concludes the process. Figure 1 The process is shown below.

[0052] pass Figure 1 As can be seen from the process shown, this embodiment triggers node capacity release based on insufficient domain capacity, such as the current remaining capacity of online storage nodes in the domain, rather than relying on user-defined periodic rules. This ensures that data can be stored normally even in scenarios such as sudden offline of storage nodes causing the online available capacity (i.e., storage space) to be unable to handle data writing pressure, or a sudden increase in written data.

[0053] Furthermore, in this embodiment, the corresponding storage space is released according to the data occupancy of each bucket corresponding to the target domain (the storage space currently occupied by the data in the bucket in the storage node of the target domain), thereby achieving the purpose of releasing storage space as soon as possible and balancing the amount of data released from each bucket.

[0054] The following describes how, in step 103 above, the target capacity to be released in the target domain is determined based on the total capacity of the online storage nodes in the target domain and the current remaining capacity of the online storage nodes in the target domain:

[0055] See Figure 2 , Figure 2 A flowchart illustrating the implementation of step 102 in an embodiment of this application. Figure 2 As shown, the process may include the following steps:

[0056] Step 201: Determine the remaining capacity threshold corresponding to the target domain based on the sum of the total capacity N2 of all online storage nodes in the target domain and the preset ratio threshold.

[0057] In this embodiment, the preset ratio threshold is as described above, and will not be repeated here.

[0058] Optionally, as an embodiment, step 201 determines the remaining capacity threshold corresponding to the target domain based on the sum of the total capacity N2 of all online storage nodes in the target domain and a preset ratio threshold. There are many ways to implement this, such as calculating the product of the sum of the total capacity N2 of all online storage nodes in the target domain and the preset ratio threshold, and determining the calculation result as the remaining capacity threshold corresponding to the target domain, etc. This embodiment is not specifically limited.

[0059] Step 202: Calculate the difference between the remaining capacity threshold and the sum of the current remaining capacities N1 of the online storage nodes in the target domain; determine the difference as the target capacity to be released in the target domain.

[0060] Finally passed Figure 2 The illustrated process demonstrates how to determine the target capacity to be released in a target domain based on the total capacity and the current remaining capacity of the online storage nodes in the target domain. For example, in one embodiment, a preset percentage threshold is p%, the total capacity of all online storage nodes in the target domain is T1, the current remaining capacity of the online storage nodes in the target domain is T2, the remaining capacity threshold corresponding to the target domain is T1 * p%, and the target capacity to be released is T1 * p% - T2.

[0061] For example, if the preset percentage threshold is 10%, the total capacity of all online storage nodes in the target domain is 100GB, and the current remaining capacity of the online storage nodes in the target domain is 5GB, then the above remaining capacity threshold is 100*10%=10GB, and the target capacity to be released is 10GB-5GB=5GB.

[0062] This concludes the process. Figure 2 The process is shown below.

[0063] The following is about Figure 3 The process is described as follows:

[0064] See Figure 3 , Figure 3 This is a flowchart illustrating step 103 as provided in an embodiment of this application. Figure 3 As shown, the process may include the following steps:

[0065] Step 301: For each bucket, determine whether the bucket is a target bucket that meets the capacity release conditions based on the current data occupancy of the bucket.

[0066] As an example, when the current data usage of a bucket is greater than the capacity protection threshold corresponding to that bucket, the bucket is determined to be a target bucket that meets the capacity release conditions.

[0067] Here, the capacity protection threshold for each bucket can be determined through the following steps: For each bucket, determine the storage space occupancy ratio configured for that bucket, calculate the product of the storage space occupancy ratio of that bucket and N2, where N2 represents the sum of the total capacity of the currently online storage nodes in the target domain; and determine the calculation result as the capacity protection threshold for that bucket.

[0068] In practical implementation, the aforementioned storage space occupancy ratio refers to the proportion of storage space used by the bucket from the online storage nodes within the target domain. This ratio is related to the configured size of the bucket. For example, if the target domain corresponds to bucket A and bucket B, and the sizes of bucket A and bucket B are 1GB and 2GB respectively, then bucket A and bucket B will use the storage space of the online storage devices within the domain in a 1:2 ratio. That is, the storage space occupancy ratio of bucket A is 1 / 3, and correspondingly, the storage space occupancy ratio of bucket B is 2 / 3.

[0069] For example, the total capacity of online storage devices in the target domain is T. The target domain corresponds to bucket A and bucket B. The sizes of bucket A and bucket B are 1GB and 2GB, respectively. Bucket A and bucket B use the storage space of online storage devices in the domain in a 1:2 ratio. If the data occupied by bucket A is less than T / 3, while the data occupied by bucket B is greater than 2T / 3, then the data in bucket B can be overwritten first, and the overwriting of data in bucket A can be suspended. That is, bucket B is the target bucket that meets the capacity release condition, and bucket A is the target bucket that does not meet the capacity release condition.

[0070] Step 302: Determine the target data amount to be released for each target bucket based on the ratio of the target capacity to the data occupancy of each target bucket.

[0071] As an example, the sum of the target data to be released in each target bucket is less than or equal to the target capacity.

[0072] As an example, the proportion of the target data to be released in each target bucket is the proportion of the data occupied in each target bucket.

[0073] This concludes the process. Figure 3 The process is shown below.

[0074] pass Figure 3 The process shown implements minimum data protection logic when determining the target amount of data to be released in each bucket, thus avoiding excessive data overwriting or release in the bucket.

[0075] It should be noted that in this embodiment, if a storage node, such as the storage node with the highest number, acts as the management node, this management node may not have object index information. Therefore, as an example, the overwrite time point can be determined based on the data occupancy ratio of each bucket, and this overwrite time point can be distributed to the corresponding storage node. The storage node can then release storage space on the storage device based on the overwrite time point (e.g., deleting all data written before the overwrite time point). Here, the overwrite time point can be calculated based on the average hourly data volume of the bucket. The average hourly data volume of the bucket depends on the configured storage cycle of the bucket and the effective data written.

[0076] The methods provided in the embodiments of this application have been described above. The apparatus and electronic devices provided in the embodiments of this application are described below:

[0077] See Figure 4 , Figure 4 This is a structural diagram of an apparatus provided in an embodiment of this application. The apparatus may include a determining unit and a releasing unit.

[0078] The determining unit is used, for each domain, to determine whether the domain is a target domain for which node capacity needs to be released, based on the current remaining capacity of the online storage nodes in that domain; and,

[0079] When a domain is determined to be a target domain for which node capacity needs to be released, the target capacity to be released in the target domain is determined based on the total capacity of online storage nodes in the target domain and the current remaining capacity of online storage nodes in the target domain; and,

[0080] Based on the target capacity and the data occupancy of each bucket corresponding to the target domain, the target data amount to be released in each bucket is determined; wherein, the data occupancy of each bucket refers to the storage space of the storage node in the target domain currently occupied by the data in that bucket;

[0081] The release unit is used to, for each bucket, notify the corresponding storage node to release the written data in the order of writing time from front to back, based on the target data volume to be released in that bucket; wherein, the amount of data to be released is less than or equal to the target data volume to be released in that bucket.

[0082] Optionally, the determining unit determines whether a domain is a target domain for which node capacity needs to be released based on the current remaining capacity of the online storage nodes in that domain, including:

[0083] Determine the total remaining capacity N1 of the online storage nodes in this domain;

[0084] Determine the proportion of N1 in the total capacity N2 of all online storage nodes in this domain;

[0085] If the ratio is less than a preset ratio threshold, the domain is determined to be the target domain.

[0086] Optionally, the determining unit determines the target capacity to be released in the target domain based on the total capacity of the online storage nodes in the target domain and the current remaining capacity of the online storage nodes in the target domain, including:

[0087] The remaining capacity threshold corresponding to the target domain is determined based on the sum of the total capacity N2 of all online storage nodes in the target domain and a preset ratio threshold; wherein, the preset ratio threshold is the basis for determining the target domain;

[0088] Calculate the difference between the remaining capacity threshold and the sum of the current remaining capacities N1 of the online storage nodes in the target domain; determine the difference as the target capacity to be released in the target domain.

[0089] Optionally, the determining unit determines the remaining capacity threshold corresponding to the target domain based on the sum N2 of the total capacity of all online storage nodes in the target domain and a preset ratio threshold, including:

[0090] Calculate the product of the total capacity N2 of all online storage nodes in the target domain and a preset ratio threshold, and determine the result as the current remaining capacity threshold of the target domain.

[0091] Optionally, the determining unit determines the target data amount to be released in each bucket based on the target capacity and the data occupancy of each bucket corresponding to the target domain, including:

[0092] Obtain the data usage of each bucket corresponding to the target domain;

[0093] Based on the target capacity and the ratio of data occupancy in each bucket, the target data amount to be released in each bucket is determined; wherein, the ratio of the target data amount to be released in each bucket is the ratio of data occupancy in each bucket, and the sum of the target data amounts to be released in each bucket is the target capacity.

[0094] Optionally, the determining unit determines the target data amount to be released in each bucket based on the target capacity and the data occupancy of each bucket corresponding to the target domain, including:

[0095] Obtain the data usage of each bucket corresponding to the target domain;

[0096] For each bucket, determine whether the bucket is a target bucket that meets the capacity release conditions based on the current data occupancy of the bucket; where the current data occupancy of the bucket is greater than the capacity protection threshold corresponding to the bucket, the bucket is determined to be a target bucket that meets the capacity release conditions.

[0097] Based on the target capacity and the ratio of data occupancy in each target bucket, the target data amount to be released in each target bucket is determined.

[0098] Optionally, the capacity protection threshold for each bucket is determined through the following steps:

[0099] For each bucket, determine the storage space occupancy ratio configured for that bucket; the storage space occupancy ratio refers to the proportion of storage space used by the bucket from the online storage nodes within the target domain, and the storage space occupancy ratio is related to the size configured for that bucket;

[0100] Calculate the product of the storage space occupancy ratio of the bucket and N2, where N2 represents the sum of the total capacity of the currently online storage nodes in the target domain; determine the calculation result as the capacity protection threshold corresponding to the bucket.

[0101] Optionally, the management node is a node deployed in the cloud storage cluster for performing node capacity release; or,

[0102] The management node is a storage node selected from all storage nodes in the cloud storage cluster to perform node capacity release.

[0103] This concludes the process. Figure 4 Structural description of the device shown.

[0104] This application also provides embodiments that... Figure 4 The hardware structure of the device shown. See also Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 5 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0105] Based on the same application concept as the above method, this application embodiment also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0106] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0107] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0108] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0109] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0111] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0112] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0113] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method of node capacity release, characterized by, The method is applied to a management node in a cloud storage cluster, which also includes storage nodes. The storage nodes in the cloud storage cluster are divided into at least one domain, and each domain corresponds to at least one bucket used for storing data. The method includes: For each domain, based on the current remaining capacity of the online storage nodes in that domain, determine whether that domain is a target domain for which node capacity needs to be released; When a domain is determined to be a target domain for which node capacity needs to be released, the target capacity to be released in the target domain is determined based on the total capacity of online storage nodes in the target domain and the current remaining capacity of online storage nodes in the target domain. Based on the target capacity and the data occupancy of each bucket corresponding to the target domain, the target data amount to be released in each bucket is determined; wherein, the data occupancy of each bucket refers to the storage space of the storage node in the target domain currently occupied by the data in that bucket; For each bucket, based on the target amount of data to be released in that bucket, the corresponding storage node is notified to release the written data in the order of writing time from front to back; wherein the amount of data released is less than or equal to the target amount of data to be released in that bucket.

2. The method of claim 1, wherein, Determining whether a domain is a target domain for releasing node capacity based on the current remaining capacity of the online storage nodes in that domain includes: Determine the total remaining capacity N1 of the online storage nodes in this domain; Determine the proportion of N1 in the total capacity N2 of all online storage nodes in this domain; If the ratio is less than a preset ratio threshold, the domain is determined to be the target domain.

3. The method of claim 1, wherein, Based on the total capacity of online storage nodes in the target domain and the current remaining capacity of online storage nodes in the target domain, the target capacity to be released in the target domain is determined as follows: The remaining capacity threshold corresponding to the target domain is determined based on the sum of the total capacity N2 of all online storage nodes in the target domain and a preset ratio threshold; wherein, the preset ratio threshold is the basis for determining the target domain; Calculate the difference between the remaining capacity threshold and the sum of the current remaining capacities N1 of the online storage nodes in the target domain; determine the difference as the target capacity to be released in the target domain.

4. The method of claim 3, wherein, The remaining capacity threshold of the target domain is determined based on the sum of the total capacity N2 of all online storage nodes in the target domain and a preset ratio threshold, including: Calculate the product of the total capacity N2 of all online storage nodes in the target domain and a preset ratio threshold, and determine the result as the current remaining capacity threshold of the target domain.

5. The method of claim 1, wherein, Based on the target capacity and the data occupancy of each bucket corresponding to the target domain, the target data amount to be released in each bucket includes: Obtain the data usage of each bucket corresponding to the target domain; Based on the target capacity and the ratio of data occupancy in each bucket, the target data amount to be released in each bucket is determined; wherein, the ratio of the target data amount to be released in each bucket is the ratio of data occupancy in each bucket, and the sum of the target data amounts to be released in each bucket is the target capacity.

6. The method of claim 1, wherein, Based on the target capacity and the data occupancy of each bucket corresponding to the target domain, the target data amount to be released in each bucket includes: Obtain the data usage of each bucket corresponding to the target domain; For each bucket, determine whether the bucket is a target bucket that meets the capacity release conditions based on the current data occupancy of the bucket; where the current data occupancy of the bucket is greater than the capacity protection threshold corresponding to the bucket, the bucket is determined to be a target bucket that meets the capacity release conditions. Based on the target capacity and the ratio of data occupancy in each target bucket, the target data amount to be released in each target bucket is determined.

7. The method of claim 6, wherein, The capacity protection threshold for each bucket is determined through the following steps: For each bucket, determine the storage space occupancy ratio configured for that bucket; the storage space occupancy ratio refers to the proportion of storage space used by the bucket from the online storage nodes within the target domain, and the storage space occupancy ratio is related to the size configured for that bucket; Calculate the product of the storage space occupancy ratio of the bucket and N2, where N2 represents the sum of the total capacity of the currently online storage nodes in the target domain; determine the calculation result as the capacity protection threshold corresponding to the bucket.

8. The method of claim 1, wherein, The management node is a node deployed in the cloud storage cluster used to perform node capacity release; or, The management node is a storage node selected from all storage nodes in the cloud storage cluster to perform node capacity release.

9. A node capacity release apparatus characterized by comprising: The device is applied to a management node in a cloud storage cluster, the cloud storage cluster further comprising storage nodes, the storage nodes in the cloud storage cluster being divided into at least one domain, each domain corresponding to at least one bucket used for storing data; the device includes: The determining unit is used, for each domain, to determine whether the domain is a target domain for which node capacity needs to be released, based on the current remaining capacity of the online storage nodes in that domain; and, When a domain is determined to be a target domain for which node capacity needs to be released, the target capacity to be released in the target domain is determined based on the total capacity of online storage nodes in the target domain and the current remaining capacity of online storage nodes in the target domain. Based on the target capacity and the data occupancy of each bucket corresponding to the target domain, the target data amount to be released in each bucket is determined; wherein, the data occupancy of each bucket refers to the storage space occupied by the data in that bucket in the storage nodes of the target domain. The release unit is used to, for each bucket, notify the corresponding storage node to release the written data in the order of writing time from front to back, based on the target data volume to be released in that bucket; wherein, the amount of data to be released is less than or equal to the target data volume to be released in that bucket.

10. An electronic device, comprising: The electronic device includes: a processor and a machine-readable storage medium; The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the steps of the method according to any one of claims 1-8.

Images