A capacity detection method, device and equipment of a distributed storage system and a medium

Abstract

The application discloses a capacity detection method, device and equipment of a distributed storage system and a medium, relates to the technical field of distributed storage, and is applied to a storage device and comprises the following steps: acquiring the current stored capacity of the storage device; determining a capacity reporting mechanism of the storage device according to the current stored capacity; wherein the capacity reporting mechanism comprises a dynamic reporting mechanism and a fixed reporting mechanism; if the capacity reporting mechanism is the dynamic reporting mechanism, then a dynamic reporting period is determined according to the stored capacity change information of the storage device, and the capacity information of the storage device is collected and reported according to the dynamic reporting period; according to the current stored capacity of the storage device, the low-frequency fixed reporting mechanism and the high-frequency dynamic reporting mechanism are selected, the timeliness and the collection frequency of the capacity information reporting are taken into account; and when the dynamic reporting mechanism is selected, the capacity collection frequency can be dynamically changed, so that the capacity detection of the distributed storage system is more timely and accurate.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, and in particular to a capacity detection method, apparatus, storage device, and computer-readable storage medium for a distributed storage system. Background Technology

[0002] As is well known, distributed storage consists of multiple storage devices as nodes. Most current distributed storage systems inevitably have a storage system capacity detection mechanism to collect and detect the storage capacity of each node in the distributed storage system, reflecting the overall capacity of the entire distributed storage. The conventional method for collecting the storage capacity of each node is periodic collection. The drawback of periodic collection is that when the cluster is in a high-bandwidth write scenario and the storage capacity of the storage devices is small, the collection of the capacity of each node is often not timely, accurate, or flexible enough. The storage pool may not handle the warning threshold in a timely manner, leading to ineffective control of front-end services, which in turn causes storage devices (such as hard drives) to report "No space" issues. "No space" in storage devices causes the storage service to exit, which in turn causes cluster failure recovery problems.

[0003] Therefore, how to report the capacity usage of each storage device more timely and accurately, and avoid the problem of untimely cluster capacity alarms when the distributed storage system is experiencing large-scale traffic writes, is an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a capacity detection method, apparatus, storage device, and computer-readable storage medium for a distributed storage system, so as to report the capacity usage of each storage device more timely and accurately, and avoid the problem of untimely cluster capacity alarms when the distributed storage system is experiencing large-scale traffic writes.

[0005] To address the aforementioned technical problems, this invention provides a capacity detection method for a distributed storage system, applied to a storage device, comprising:

[0006] Obtain the current stored capacity of the storage device;

[0007] Based on the current stored capacity, determine the capacity reporting mechanism of the storage device; wherein, the capacity reporting mechanism includes a dynamic reporting mechanism and a fixed reporting mechanism;

[0008] If the capacity reporting mechanism is a fixed reporting mechanism, then the capacity information of the storage device is collected and reported according to a fixed reporting cycle;

[0009] If the capacity reporting mechanism is a dynamic reporting mechanism, then the dynamic reporting period is determined based on the changes in the stored capacity of the storage device, and the capacity information of the storage device is collected and reported according to the dynamic reporting period; wherein the dynamic reporting period is less than or equal to the fixed reporting period.

[0010] In some embodiments, the mechanism for determining the capacity reporting of the storage device based on the currently stored capacity includes:

[0011] Determine whether the current stored capacity is greater than a preset capacity percentage; wherein, the current stored capacity is the quotient of the used space of the storage device and the storage capacity;

[0012] If so, then the dynamic reporting mechanism will be determined as the capacity reporting mechanism;

[0013] If not, then the fixed reporting mechanism will be determined as the capacity reporting mechanism.

[0014] In some embodiments, before determining whether the currently stored capacity is greater than a preset capacity percentage, the method further includes:

[0015] The preset capacity percentage is determined based on the current cluster scenario of the distributed storage system in which the storage device is located; wherein, the preset capacity percentage is a capacity percentage threshold corresponding to the current cluster scenario, the current cluster scenario is any preset cluster scenario, and the preset cluster scenario includes pure business scenario and recovery scenario.

[0016] In some embodiments, after determining the dynamic reporting period based on the storage capacity change information of the storage device, and collecting and reporting the capacity information of the storage device according to the dynamic reporting period, the method further includes:

[0017] If the current cluster scenario changes from the pure business scenario to the recovery scenario, the capacity information of the storage device is collected and reported according to the dynamic reporting cycle; wherein the capacity percentage threshold corresponding to the recovery scenario is greater than the capacity percentage threshold corresponding to the pure business scenario.

[0018] In some embodiments, determining a dynamic reporting period based on the storage capacity change information of the storage device, and collecting and reporting the capacity information of the storage device according to the dynamic reporting period, includes:

[0019] Based on the stored capacity change information, determine whether the capacity change of the storage device is in an increasing state;

[0020] If it is not in the growth state, then the fixed reporting period is determined as the dynamic reporting period, and the capacity information of the storage device is collected and reported according to the dynamic reporting period;

[0021] If the storage device is in the growth state, the target growth time is calculated based on the changes in the stored capacity; wherein, the target growth time is the time it takes for the stored capacity of the storage device to increase by a preset capacity.

[0022] Determine whether the target growth time is less than the fixed reporting period;

[0023] If the time is less than the fixed reporting period, the target growth time is determined as the dynamic reporting period, and the capacity information of the storage device is collected and reported according to the dynamic reporting period.

[0024] If the reporting period is not less than the fixed reporting period, then the fixed reporting period is determined as the dynamic reporting period, and the capacity information of the storage device is collected and reported according to the dynamic reporting period.

[0025] In some embodiments, the preset capacity is 1%.

[0026] In some embodiments, the stored capacity change information includes the stored capacity collected at preset time intervals within a preset time period prior to the current time.

[0027] The present invention also provides a capacity detection device for a distributed storage system, applied to a storage device, comprising:

[0028] The acquisition module is used to acquire the current stored capacity of the storage device;

[0029] A determination module is used to determine the capacity reporting mechanism of the storage device based on the current stored capacity; wherein the capacity reporting mechanism includes a dynamic reporting mechanism and a fixed reporting mechanism;

[0030] The first reporting module is used to collect and report the capacity information of the storage device according to a fixed reporting cycle if the capacity reporting mechanism is a fixed reporting mechanism.

[0031] The second reporting module is used to determine the dynamic reporting period based on the changes in the stored capacity of the storage device if the capacity reporting mechanism is a dynamic reporting mechanism, and to collect and report the capacity information of the storage device according to the dynamic reporting period; wherein the dynamic reporting period is less than or equal to the fixed reporting period.

[0032] The present invention also provides a storage device for a distributed storage system, comprising:

[0033] Memory, used to store computer programs;

[0034] A processor is configured to implement the steps of the capacity detection method for a distributed storage system as described above when executing the computer program.

[0035] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the capacity detection method for a distributed storage system as described above.

[0036] The present invention provides a capacity detection method for a distributed storage system, applied to a storage device, comprising: obtaining the current stored capacity of the storage device; determining a capacity reporting mechanism for the storage device based on the current stored capacity; wherein the capacity reporting mechanism includes a dynamic reporting mechanism and a fixed reporting mechanism; if the capacity reporting mechanism is a fixed reporting mechanism, then the capacity information of the storage device is collected and reported according to a fixed reporting period; if the capacity reporting mechanism is a dynamic reporting mechanism, then the dynamic reporting period is determined based on the changes in the stored capacity of the storage device, and the capacity information of the storage device is collected and reported according to the dynamic reporting period; wherein the dynamic reporting period is less than or equal to the fixed reporting period.

[0037] As can be seen, this invention selects a low-frequency fixed reporting mechanism and a high-frequency dynamic reporting mechanism based on the current stored capacity of the storage device, balancing the timeliness of capacity information reporting with the collection frequency. Furthermore, when selecting the dynamic reporting mechanism, the capacity collection frequency can be dynamically adjusted according to the current stored capacity of the storage device. By changing the capacity collection frequency, the accuracy of the reported storage device capacity is optimized, thus making the capacity detection of the distributed storage system more timely. This avoids the problem of untimely cluster capacity alarms during large-scale traffic writes, reducing subsequent maintenance events caused by capacity issues in the storage cluster, enabling the storage system to operate more healthily, and further improving the competitiveness of the distributed storage system. In addition, this invention also provides a capacity detection device, storage device, and computer-readable storage medium for a distributed storage system, which also have the above-mentioned beneficial effects. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0039] Figure 1 A flowchart illustrating a capacity detection method for a distributed storage system provided in an embodiment of the present invention;

[0040] Figure 2A flowchart of another capacity detection method for a distributed storage system provided in an embodiment of the present invention;

[0041] Figure 3 This is a structural block diagram of a capacity detection device for a distributed storage system provided in an embodiment of the present invention;

[0042] Figure 4 This is a simplified structural diagram of a storage device in a distributed storage system provided in an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the specific structure of a storage device in a distributed storage system provided in an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a capacity detection method for a distributed storage system provided in an embodiment of the present invention. The method is applied to a storage device and may include:

[0047] Step 101: Obtain the current stored capacity of the storage device.

[0048] It is understood that the storage device in this embodiment can be a device that acts as a storage node in a distributed storage system. That is to say, this embodiment demonstrates the reporting of capacity information of one storage device in a distributed storage system as an example. The reporting of capacity information of other storage devices in the distributed storage system can be implemented in the same or similar way as the method provided in this embodiment, and this embodiment does not impose any restrictions on this.

[0049] In this embodiment, the current stored capacity of the storage device can be information about the current stored capacity of the storage device, that is, information related to the capacity of stored data out of the total storage capacity of the storage device. The specific content of the current stored capacity of the storage device in this embodiment can be set by the designer according to the usage scenario and user needs. For example, the current stored capacity of the storage device can be the capacity space occupied by the data currently stored in the storage device (i.e., the used space); the current stored capacity of the storage device can also be the quotient of the used space and the storage capacity, such as the percentage of the storage capacity occupied by the used space. This embodiment does not impose any restrictions on this.

[0050] Correspondingly, the specific time at which the storage device obtains its current stored capacity in this embodiment can be set by the designer according to the practical scenario and user needs. For example, the storage device can obtain its current stored capacity at a preset reporting and adjustment time interval to adjust the reporting cycle of the storage device's capacity information according to the preset reporting and adjustment time interval. The storage device can also obtain its current stored capacity at the time point corresponding to each reporting cycle (such as a fixed reporting cycle or a dynamic reporting cycle) to adjust the subsequent reporting cycle in a timely manner after the current reporting is completed. This embodiment does not impose any restrictions on this.

[0051] Correspondingly, the specific method for obtaining the current stored capacity of the storage device in this step can be set by the designer according to the practical scenario and user needs. For example, it can be implemented in the same or similar way as the storage device's stored capacity acquisition method in the existing technology. For example, the storage device can obtain the current used space and storage capacity, and multiply the quotient of the used space and storage capacity by 100% as the current stored capacity of the storage device.

[0052] Step 102: Determine the capacity reporting mechanism of the storage device based on the current stored capacity; the capacity reporting mechanism includes a dynamic reporting mechanism and a fixed reporting mechanism.

[0053] It is understood that the capacity reporting mechanism in this embodiment can be a mechanism for storage devices to report collected capacity information, that is, a configuration mechanism for periodically reporting collected capacity information at a certain reporting period. In this embodiment, the capacity reporting mechanism can include a dynamic reporting mechanism and a fixed reporting mechanism; the dynamic reporting mechanism can be a mechanism for collecting and reporting storage device capacity information using a dynamically adjustable reporting period (i.e., a dynamic reporting period); the fixed reporting mechanism can be a mechanism for collecting and reporting storage device capacity information using a fixed reporting period (i.e., a fixed reporting period).

[0054] Correspondingly, in this embodiment, by setting dynamic and fixed reporting mechanisms in the capacity reporting mechanism, the storage device can select a low-frequency fixed reporting mechanism and a high-frequency dynamic reporting mechanism according to the current stored capacity. This balances the timeliness of capacity information reporting and the collection frequency. When the stored capacity of the distributed storage system is high, such as in recovery scenarios or continuous large-scale IO (input / output) traffic, the dynamic reporting mechanism can dynamically adjust the capacity collection and reporting frequency, making the capacity detection of the distributed storage system more timely and avoiding cluster overload problems caused by untimely capacity detection. When the stored capacity of the storage device is low, the fixed reporting mechanism is used to report capacity information at a lower fixed capacity collection and reporting frequency, reducing the pressure on message processing.

[0055] It should be noted that the specific method by which the storage device determines its capacity reporting mechanism based on the current stored capacity in this embodiment can be set by the designers according to the usage scenario and user needs. For example, the storage device can determine whether the current stored capacity is greater than a preset capacity percentage; if so, the dynamic reporting mechanism is determined as the capacity reporting mechanism; if not, the fixed reporting mechanism is determined as the capacity reporting mechanism. Here, the current stored capacity is the quotient of the storage device's used space and its storage capacity. In other words, when the current stored capacity is at a high level (greater than the preset capacity percentage), the storage device can use the dynamic reporting mechanism to collect and report capacity information at a higher frequency to improve the capacity detection frequency and accuracy of the distributed storage system; when the current stored capacity is at a low level (not greater than the preset capacity percentage), the fixed reporting mechanism is used to collect and report capacity information at a lower frequency to reduce the pressure on message processing. Correspondingly, when the current stored capacity is the storage device's used space, the storage device can also determine whether the current stored capacity is greater than a preset capacity threshold; if so, the dynamic reporting mechanism is determined as the capacity reporting mechanism; if not, the fixed reporting mechanism is determined as the capacity reporting mechanism. This embodiment does not impose any limitations on this.

[0056] Correspondingly, the aforementioned preset capacity percentage can be a fixed percentage threshold. To further improve the timeliness and accuracy of capacity detection in the distributed storage system, the preset capacity percentage can also be a percentage threshold adjusted according to the current cluster scenario of the distributed storage system (i.e., the current cluster scenario). For example, before determining whether the currently stored capacity is greater than the preset capacity percentage, the storage device can determine the preset capacity percentage based on the current cluster scenario of the distributed storage system in which the storage device resides. Here, the preset capacity percentage is the capacity percentage threshold corresponding to the current cluster scenario, and the current cluster scenario is any preset cluster scenario, including pure business scenarios and recovery scenarios. The capacity percentage thresholds corresponding to each preset cluster scenario can be different, such as the capacity percentage threshold corresponding to the recovery scenario being greater than the capacity percentage threshold corresponding to the pure business scenario.

[0057] Step 103: If the capacity reporting mechanism is a fixed reporting mechanism, then the capacity information of the storage device is collected and reported according to the fixed reporting cycle.

[0058] In this embodiment, the fixed reporting period can be a fixed period for collecting and reporting the capacity information of the storage device. That is, the storage device can periodically collect and report the capacity information of the storage device according to the fixed reporting period, so that the controller of the distributed storage system can detect the capacity information of the storage device according to the fixed reporting period.

[0059] Correspondingly, the specific time setting of the fixed reporting period in this embodiment can be set by the designer according to the practical scenario and user needs. For example, it can be implemented in a way that is the same as or similar to the reporting period setting method of storage device nodes in the existing distributed storage system. For example, the fixed reporting period can be directly set as a fixed period; the fixed reporting period can also be set as a heartbeat period. For example, the heartbeat period can be a fixed period + a random number × a floating period. For example, when the storage engine of the storage device starts, the capacity status of the storage device can be initialized according to the storage engine allocator information, and the heartbeat period can be initialized accordingly. This embodiment does not impose any restrictions on this.

[0060] It should be noted that the capacity information reported by the storage device in this embodiment can be information related to the capacity of the storage device to store data. The specific content of the capacity information of the storage device in this embodiment can be set by the designer, such as using content that is the same as or similar to the capacity information reported by the storage device nodes in existing distributed storage systems. For example, the capacity information may include the currently stored capacity, or it may include the total storage capacity. This embodiment does not impose any limitations on this.

[0061] Step 104: If the capacity reporting mechanism is a dynamic reporting mechanism, then determine the dynamic reporting period based on the changes in the stored capacity of the storage device, and collect and report the capacity information of the storage device according to the dynamic reporting period; wherein, the dynamic reporting period is less than or equal to the fixed reporting period.

[0062] It is understood that the dynamic reporting period in this embodiment can be a dynamically adjustable reporting period. The stored capacity change information in this embodiment can be information related to changes in the stored capacity of the storage device. For example, the stored capacity change information can include the stored capacity of the storage device collected at preset time intervals, so that the changes in stored capacity can be reflected through multiple collected stored capacities; for example, when the preset time interval is 1 second, the storage device can determine the dynamic reporting period based on the stored capacity collected per second. Furthermore, the stored capacity change information can include the stored capacity collected at preset time intervals within a preset time period before the current time, so that the dynamic reporting period can be determined using a certain number of newly collected stored capacities, ensuring the accuracy of the dynamic reporting period.

[0063] Correspondingly, the specific method by which the storage device determines the dynamic reporting cycle based on the changes in the stored capacity in this step can be set by the designer according to the practical scenario and user needs. For example, the storage device can calculate the target growth time based on the changes in the stored capacity; determine whether the target growth time is less than the fixed reporting cycle; if it is less than the fixed reporting cycle, then the target growth time is determined as the dynamic reporting cycle; if it is not less than the fixed reporting cycle, then the fixed reporting cycle is determined as the dynamic reporting cycle. Among them, the target growth time is the time for the stored capacity of the storage device to increase by a preset capacity. If the stored capacity is the ratio of the used space to the storage capacity, the preset capacity can be a pre-set percentage of the storage capacity, such as 1% or 0.5%. Furthermore, to reduce computational load, in this step, the storage device can determine whether its capacity change is in an increasing state based on the information on changes in the stored capacity. If it is not in an increasing state, the fixed reporting period is set as the dynamic reporting period. If it is in an increasing state, the target growth time is calculated based on the information on changes in the stored capacity. It is then determined whether the target growth time is less than the fixed reporting period. If it is less than the fixed reporting period, the target growth time is set as the dynamic reporting period. If it is not less than the fixed reporting period, the fixed reporting period is set as the dynamic reporting period.

[0064] In some embodiments, in this step, the storage device can further determine whether the storage device's capacity change is in an increasing state based on the stored capacity change information. If it is not in an increasing state, the fixed reporting period is determined as the dynamic reporting period, and the storage device's capacity information is collected and reported according to the dynamic reporting period. If it is in an increasing state, the target growth time is calculated based on the stored capacity change information; it is then determined whether the target growth time is less than the current reporting period. If it is not less than the current reporting period, the current reporting period is determined as the dynamic reporting period, and the storage device's capacity information is collected and reported according to the dynamic reporting period. In other words, the storage device can continue to collect and report its capacity information according to the current reporting period. If it is less than the fixed reporting period, the target growth time is determined as the dynamic reporting period, and the storage device's capacity information is collected and reported according to the dynamic reporting period. In other words, the storage device can adjust the current dynamic reporting period and collect and report its capacity information according to the adjusted dynamic reporting period. Accordingly, the current reporting period can be the reporting period currently used by the storage device, such as a fixed reporting period or a dynamic reporting period.

[0065] In this embodiment, the present invention selects a low-frequency fixed reporting mechanism and a high-frequency dynamic reporting mechanism based on the current stored capacity of the storage device, balancing the timeliness of capacity information reporting and the collection frequency. When selecting the dynamic reporting mechanism, the capacity collection frequency can be dynamically changed according to the current stored capacity of the storage device. By changing the capacity collection frequency, the accuracy of the reported storage device capacity is optimized, thereby making the capacity detection of the distributed storage system more timely. This avoids the problem of untimely cluster capacity alarms during large-scale traffic writes in the distributed storage system, reduces subsequent maintenance events caused by capacity issues in the storage cluster, and enables the storage system to operate more healthily, thereby further improving the competitiveness of the distributed storage system.

[0066] Based on the above embodiments, this invention also provides another capacity detection method for a distributed storage system to further improve the accuracy of capacity acquisition frequency adjustment. For details, please refer to... Figure 2 , Figure 2 A flowchart illustrating another capacity detection method for a distributed storage system provided in an embodiment of the present invention. This method is applied to a storage device and may include:

[0067] Step 201: Obtain the current stored capacity of the storage device.

[0068] This step is similar to step 101, and will not be described again here.

[0069] Step 202: Determine the preset capacity percentage based on the current cluster scenario of the distributed storage system where the storage device is located.

[0070] The preset capacity percentage is the capacity percentage threshold corresponding to the current cluster scenario. The current cluster scenario is any preset cluster scenario, which includes pure business scenarios and recovery scenarios.

[0071] It is understandable that in this step, the storage device can select the corresponding capacity percentage threshold as the preset capacity percentage based on the current cluster scenario of the distributed storage system it belongs to (i.e., the current cluster scenario) in order to achieve dynamic adjustment of the preset capacity percentage.

[0072] Correspondingly, the specific settings for each preset cluster scenario and its corresponding capacity percentage threshold can be configured by the designers according to practical scenarios and user needs. For example, preset cluster scenarios can be set in the same or similar way as the cluster scenario settings in existing distributed storage systems. Preset cluster scenarios may include pure business scenarios and recovery scenarios. The capacity percentage threshold for each preset cluster scenario can be the threshold for stopping data writing for that specific scenario. For example, the capacity percentage threshold for the recovery scenario can be 95%, meaning that when this threshold is reached, the storage pool recovery business stops writing; the capacity percentage threshold for the pure business scenario can be 85%, meaning that when this threshold is reached, the front-end business stops writing.

[0073] Step 203: Determine whether the current stored capacity is greater than the preset capacity percentage; if not, proceed to step 204; if yes, proceed to step 205.

[0074] The current stored capacity is the quotient of the used space and the storage capacity of the storage device. In this step, the storage device determines the capacity reporting mechanism by judging whether the current stored capacity is greater than a preset capacity percentage. When the current stored capacity is not greater than the preset capacity percentage, the capacity reporting mechanism is determined to be a fixed reporting mechanism, and the process can proceed to step 204 to collect and report the storage device's capacity information according to the fixed reporting cycle of the fixed reporting mechanism. When the current stored capacity is greater than the preset capacity percentage, the capacity reporting mechanism is determined to be a dynamic reporting mechanism, and the process can proceed to step 205 to collect and report the storage device's capacity information according to the dynamic reporting cycle of the dynamic reporting mechanism.

[0075] Step 204: Collect and report the capacity information of the storage devices according to the fixed reporting cycle.

[0076] Understandably, this step is similar to step 103, and will not be repeated here.

[0077] Step 205: Based on the stored capacity change information, determine whether the storage device capacity change is in an increasing state; if not, proceed to step 206; if yes, proceed to step 207.

[0078] In this embodiment, the information on changes in stored capacity can include the stored capacity collected at preset time intervals within a preset time period prior to the current time.

[0079] For example, when the preset time interval is 1 second, in the dynamic reporting mechanism, the storage device can determine whether the storage device’s capacity change is in an increasing state based on the stored capacity detected every second. If the size of the stored capacity collected in the 10 seconds before the current time (i.e., the preset time period) increases sequentially according to the collection time, the storage device’s capacity change is determined to be in an increasing state; otherwise, the storage device’s capacity change is determined not to be in an increasing state.

[0080] Step 206: Determine the fixed reporting period as the dynamic reporting period, and collect and report the storage device capacity information according to the dynamic reporting period.

[0081] In this embodiment, when the storage device's capacity change is not in an increasing state or the target increase time is not less than the fixed reporting period, the fixed reporting period can be determined as the dynamic reporting period, and the storage device's capacity information can be collected and reported according to the fixed reporting period.

[0082] Step 207: Calculate the target growth time based on the information on changes in the stored capacity; where the target growth time is the time it takes for the stored capacity of the storage device to increase by a preset capacity.

[0083] Understandably, in this step, when the storage device's capacity is in an increasing state, it can calculate the time (i.e., the target growth time) for the stored capacity to increase by a preset capacity based on the information about the changes in the stored capacity. For example, based on the stored capacity collected 10 times in the 10 seconds prior to the current moment, the time for the stored capacity to increase by 1% (i.e., the preset capacity) can be calculated.

[0084] Step 208: Determine if the target growth time is less than the fixed reporting cycle; if not, proceed to step 206; if yes, proceed to step 209.

[0085] In this step, the storage device compares the target growth time with the fixed reporting period and selects the shorter of the two as the dynamic reporting period.

[0086] Correspondingly, in some embodiments, the storage device can determine whether the target growth time is less than the current reporting period (i.e., the currently used dynamic reporting period or fixed reporting period) in this step; if yes, proceed to step 206; if no, determine the current reporting period as the dynamic reporting period, and collect and report the storage device's capacity information according to the dynamic reporting period; so as to select the smaller time between the target growth time and the currently used dynamic reporting period as the subsequent dynamic reporting period.

[0087] Step 209: Determine the target growth time as the dynamic reporting cycle, and collect and report the capacity information of the storage device according to the dynamic reporting cycle.

[0088] In this step, when the target growth time is less than the fixed reporting period, the storage device can determine the target growth time as the dynamic reporting period, and periodically collect and report the storage device's capacity information according to the target growth time, so as to increase the frequency of capacity information collection and reporting of the storage device and increase the capacity detection frequency of the distributed storage system.

[0089] Furthermore, in this embodiment, after the storage device collects and reports its capacity information according to the dynamic reporting cycle, if the current cluster scenario changes from a pure business scenario to a recovery scenario, the storage device's capacity information is collected and reported again according to the dynamic reporting cycle. The capacity percentage threshold for the recovery scenario is greater than the capacity percentage threshold for the pure business scenario. In other words, during the process of reporting capacity information using the dynamic reporting mechanism, if the current cluster scenario changes from a preset cluster scenario with a lower corresponding capacity percentage threshold to a preset cluster scenario with a higher corresponding capacity percentage threshold, the dynamic reporting mechanism can continue to be used to report storage information according to the previous or updated dynamic reporting cycle, ensuring the accuracy of capacity detection in the distributed storage system. During the process of reporting capacity information using the fixed reporting mechanism, if the current cluster scenario changes, the process can return to step 201 to select the capacity reporting mechanism and report the cycle according to the preset capacity percentage corresponding to the changed current cluster scenario.

[0090] In this embodiment, the present invention determines a preset capacity percentage based on the current cluster scenario of the distributed storage system where the storage device is located. This allows the distributed storage system to dynamically adjust the water level threshold selected by the capacity reporting mechanism according to the cluster scenario of the distributed storage system. This enables the distributed storage system to not only obtain the capacity information of the storage device more accurately in various cluster scenarios, but also to minimize the pressure of processing capacity information.

[0091] Corresponding to the above method embodiments, this invention also provides a capacity detection device for a distributed storage system. The capacity detection device for a distributed storage system described below and the capacity detection method for a distributed storage system described above can be referred to in correspondence.

[0092] Please refer to Figure 3 , Figure 3 This is a structural block diagram of a capacity detection device for a distributed storage system provided in an embodiment of the present invention. The device is applied to a storage device and may include:

[0093] Module 10 is used to obtain the current stored capacity of the storage device;

[0094] The determination module 20 is used to determine the capacity reporting mechanism of the storage device based on the current stored capacity; wherein, the capacity reporting mechanism includes a dynamic reporting mechanism and a fixed reporting mechanism;

[0095] The first reporting module 30 is used to collect and report the capacity information of the storage device according to a fixed reporting cycle if the capacity reporting mechanism is a fixed reporting mechanism.

[0096] The second reporting module 40 is used to determine the dynamic reporting period based on the changes in the stored capacity of the storage device if the capacity reporting mechanism is a dynamic reporting mechanism, and to collect and report the capacity information of the storage device according to the dynamic reporting period; wherein the dynamic reporting period is less than or equal to the fixed reporting period.

[0097] In some embodiments, the determining module 20 may include:

[0098] The capacity judgment submodule is used to determine whether the current stored capacity is greater than the preset capacity percentage; where the current stored capacity is the quotient of the used space of the storage device and the storage capacity; if yes, the dynamic reporting mechanism is determined as the capacity reporting mechanism; if no, the fixed reporting mechanism is determined as the capacity reporting mechanism.

[0099] In some embodiments, the determining module 20 may further include:

[0100] The threshold determination submodule is used to determine a preset capacity percentage based on the current cluster scenario of the distributed storage system where the storage device is located. The preset capacity percentage is the capacity percentage threshold corresponding to the current cluster scenario. The current cluster scenario can be any preset cluster scenario, including pure business scenarios and recovery scenarios.

[0101] In some embodiments, the second reporting module 40 may include:

[0102] The change reporting submodule is used to determine the dynamic reporting period based on the storage capacity change information of the storage device, and to collect and report the capacity information of the storage device according to the dynamic reporting period. If the current cluster scenario changes from a pure business scenario to a recovery scenario, the capacity information of the storage device will be collected and reported according to the dynamic reporting period. The capacity percentage threshold corresponding to the recovery scenario is greater than the capacity percentage threshold corresponding to the pure business scenario.

[0103] In some embodiments, the second reporting module 40 may include:

[0104] The status judgment submodule is used to determine whether the storage device's capacity change is an increase state based on the stored capacity change information;

[0105] The first reporting submodule is used to determine the fixed reporting period as the dynamic reporting period if the device is not in an increasing state, and to collect and report the capacity information of the storage device according to the dynamic reporting period.

[0106] The dynamic calculation submodule is used to calculate the target growth time based on the changes in the stored capacity if the storage device is in a growth state; where the target growth time is the time it takes for the stored capacity of the storage device to grow to a preset capacity.

[0107] The periodic judgment submodule is used to determine whether the target growth time is less than the fixed reporting period;

[0108] The second reporting submodule is used to determine the target growth time as the dynamic reporting period if it is less than the fixed reporting period, and to collect and report the capacity information of the storage device according to the dynamic reporting period.

[0109] The third reporting submodule is used to determine the fixed reporting period as the dynamic reporting period if it is not less than the fixed reporting period, and to collect and report the capacity information of the storage device according to the dynamic reporting period.

[0110] In some embodiments, the preset capacity is 1%.

[0111] In some embodiments, the stored capacity change information includes the stored capacity collected at preset time intervals within a preset time period prior to the current time.

[0112] In this embodiment, the present invention determines whether to use a low-frequency fixed reporting mechanism or a high-frequency dynamic reporting mechanism based on the current stored capacity of the storage device by using the determining module 20. This balances the timeliness of capacity information reporting with the frequency of data collection. When selecting the dynamic reporting mechanism, the capacity collection frequency can be dynamically changed according to the current stored capacity of the storage device. By changing the capacity collection frequency, the accuracy of the reported storage device capacity is optimized, making the capacity detection of the distributed storage system more timely. This avoids the problem of untimely cluster capacity alarms during large-scale traffic writes, reduces subsequent maintenance events caused by capacity issues in the storage cluster, and enables the storage system to operate more healthily, thereby further improving the competitiveness of the distributed storage system.

[0113] Corresponding to the above method embodiments, this invention also provides a storage device for a distributed storage system. The storage device for a distributed storage system described below and the capacity detection method for a distributed storage system described above can be referred to each other.

[0114] Please refer to Figure 4 , Figure 4 This is a simplified structural diagram of a storage device in a distributed storage system provided by an embodiment of the present invention. The storage device in this distributed storage system may include:

[0115] Memory D1 is used to store computer programs;

[0116] Processor D2 is used to execute computer programs to implement the steps of the capacity detection method for the distributed storage system provided in the above method embodiments.

[0117] Accordingly, please refer to Figure 5 , Figure 5 This is a schematic diagram illustrating the specific structure of a storage device in a distributed storage system provided in an embodiment of the present invention. The storage device 310 can vary significantly due to different configurations or performance characteristics. It may include one or more Central Processing Units (CPUs) 322 (e.g., one or more processors) and a memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 342 or data 344. The memory 332 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations. Furthermore, the central processing unit 322 may be configured to communicate with the storage media 330 and execute a series of instruction operations from the storage media 330 on the storage device 310.

[0118] Storage device 310 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0119] The steps in the capacity detection method for the distributed storage system described above can be implemented by the structure of the storage devices in the distributed storage system.

[0120] Corresponding to the above method embodiments, this invention also provides a computer-readable storage medium. The computer-readable storage medium described below and the capacity detection method for a distributed storage system described above can be referred to in correspondence.

[0121] Please refer to Figure 6 , Figure 6 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of the present invention. The computer-readable storage medium 50 stores a computer program 51, which, when executed by a processor, implements the steps of the capacity detection method for a distributed storage system as provided in the above-described method embodiment.

[0122] The computer-readable storage medium 50 can be any storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0123] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus, storage device, and computer-readable storage medium disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant details can be found in the method section.

[0124] The capacity detection method, apparatus, storage device, and computer-readable storage medium of a distributed storage system provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A capacity detection method for a distributed storage system, characterized in that, Used in storage devices, including: Obtain the current stored capacity of the storage device; Determine whether the current stored capacity is greater than a preset capacity percentage; wherein the current stored capacity is the quotient of the used space of the storage device and the storage capacity; if yes, then the dynamic reporting mechanism is determined as the capacity reporting mechanism; if no, then the fixed reporting mechanism is determined as the capacity reporting mechanism. If the capacity reporting mechanism is a fixed reporting mechanism, then the capacity information of the storage device is collected and reported according to a fixed reporting period; the fixed reporting period is a fixed period + a random number × a floating period. If the capacity reporting mechanism is a dynamic reporting mechanism, then the dynamic reporting period is determined based on the changes in the stored capacity of the storage device, and the capacity information of the storage device is collected and reported according to the dynamic reporting period; wherein, the dynamic reporting period is less than or equal to the fixed reporting period. The step of determining a dynamic reporting period based on the changes in the stored capacity of the storage device, and collecting and reporting the capacity information of the storage device according to the dynamic reporting period, includes: Based on the stored capacity change information, determine whether the capacity change of the storage device is in an increasing state; If it is not in the growth state, then the fixed reporting period is determined as the dynamic reporting period, and the capacity information of the storage device is collected and reported according to the dynamic reporting period; If the storage device is in the growth state, the target growth time is calculated based on the changes in the stored capacity; wherein, the target growth time is the time it takes for the stored capacity of the storage device to increase by a preset capacity. Determine whether the target growth time is less than the fixed reporting period; If the time is less than the fixed reporting period, the target growth time is determined as the dynamic reporting period, and the capacity information of the storage device is collected and reported according to the dynamic reporting period. If the reporting period is not less than the fixed reporting period, then the fixed reporting period is determined as the dynamic reporting period, and the capacity information of the storage device is collected and reported according to the dynamic reporting period.

2. The capacity detection method for a distributed storage system according to claim 1, characterized in that, Before determining whether the currently stored capacity is greater than a preset capacity percentage, the method further includes: The preset capacity percentage is determined based on the current cluster scenario of the distributed storage system in which the storage device is located; wherein, the preset capacity percentage is a capacity percentage threshold corresponding to the current cluster scenario, the current cluster scenario is any preset cluster scenario, and the preset cluster scenario includes pure business scenario and recovery scenario.

3. The capacity detection method for a distributed storage system according to claim 2, characterized in that, After determining the dynamic reporting period based on the changes in the stored capacity of the storage device, and collecting and reporting the capacity information of the storage device according to the dynamic reporting period, the method further includes: If the current cluster scenario changes from the pure business scenario to the recovery scenario, the capacity information of the storage device is collected and reported according to the dynamic reporting cycle; wherein the capacity percentage threshold corresponding to the recovery scenario is greater than the capacity percentage threshold corresponding to the pure business scenario.

4. The capacity detection method for a distributed storage system according to claim 1, characterized in that, The preset capacity is 1%.

5. The capacity detection method for a distributed storage system according to claim 4, characterized in that, The information on changes in stored capacity includes the stored capacity collected at preset time intervals within a preset time period prior to the current time.

6. A capacity detection device for a distributed storage system, characterized in that, Used in storage devices, including: The acquisition module is used to acquire the current stored capacity of the storage device; A determination module is used to determine the capacity reporting mechanism of the storage device based on the current stored capacity; wherein the capacity reporting mechanism includes a dynamic reporting mechanism and a fixed reporting mechanism; The first reporting module is used to collect and report the capacity information of the storage device according to a fixed reporting period if the capacity reporting mechanism is a fixed reporting mechanism; the fixed reporting period is a fixed period + a random number × a floating period. The second reporting module is used to determine the dynamic reporting period based on the storage capacity change information of the storage device if the capacity reporting mechanism is a dynamic reporting mechanism, and to collect and report the capacity information of the storage device according to the dynamic reporting period; wherein the dynamic reporting period is less than or equal to the fixed reporting period. The determining module includes a capacity judgment submodule, used to determine whether the current stored capacity is greater than a preset capacity percentage; wherein the current stored capacity is the quotient of the used space and the storage capacity of the storage device; if yes, the dynamic reporting mechanism is determined as the capacity reporting mechanism; if no, the fixed reporting mechanism is determined as the capacity reporting mechanism. The second reporting module includes: The status determination submodule is used to determine whether the capacity change of the storage device is an increasing state based on the stored capacity change information. The first reporting submodule is used to determine the fixed reporting period as the dynamic reporting period if it is not in the growth state, and to collect and report the capacity information of the storage device according to the dynamic reporting period. The dynamic calculation submodule is used to calculate the target growth time based on the stored capacity change information if the growth state is in progress; wherein the target growth time is the time it takes for the stored capacity of the storage device to increase by a preset capacity. The periodicity determination submodule is used to determine whether the target growth time is less than the fixed reporting period; The second reporting submodule is used to determine the target growth time as the dynamic reporting period if it is less than the fixed reporting period, and to collect and report the capacity information of the storage device according to the dynamic reporting period. The third reporting submodule is used to determine the fixed reporting period as the dynamic reporting period if it is not less than the fixed reporting period, and to collect and report the capacity information of the storage device according to the dynamic reporting period.

7. A storage device for a distributed storage system, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the capacity detection method for a distributed storage system as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the capacity detection method for a distributed storage system as described in any one of claims 1 to 5.

Images