A distributed storage system capacity balancing triggering method, device and equipment

Abstract

The application relates to the technical field of data storage, and discloses a distributed storage system capacity balancing triggering method, device and equipment, which comprises the following steps: obtaining a notification message sent by a process manager daemon process, obtaining a latest object storage device data set according to the notification message, determining a current storage device failure ratio according to the number of storage device failures in the latest object storage device data set, determining whether to change the attribute value of an attribute parameter corresponding to a capacity balancing operation into a first attribute value according to the current storage device failure ratio and / or the comparison result between the latest object storage device data set and a previously obtained object storage device data set, so that the capacity balancing operation is executed according to the first attribute value in the future. The application can respond to the change of a cluster state in real time, timely trigger the capacity balancing operation, optimize the condition for triggering the capacity balancing operation execution, realize automatic triggering of the capacity balancing, does not need to depend on management software, and reduces the system maintenance cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, and in particular to a distributed storage system capacity balancing triggering method, device and equipment. BACKGROUND

[0002] In the related art, two methods are usually used to trigger capacity balancing: one is to actively call the capacity balancing script after the fixed scene ends, and the other is to regularly assess the state of the placement group (PG) of the cluster and trigger capacity balancing. The first method depends on the existence of management software, and for those clusters without installed management software, capacity balancing operation cannot be automatically triggered. In addition, with the development of technology and the increase of customer demand, there may be more and more scenarios that need to call the capacity balancing script, which will increase the code maintenance cost. The second method triggers capacity balancing by regularly assessing the state of the PG of the cluster, and it is a difficult problem to set a suitable time interval. If the time interval is too long, the cluster will be in an unbalanced state for a long time, and if the time interval is too short, the PG state will be assessed frequently, increasing invalid operations and wasting cluster resources. Moreover, the timing of capacity balancing may conflict with other businesses of the cluster, especially when the cluster is executing critical businesses, the execution of capacity balancing may reduce the business processing capacity of the cluster, and even bring risks. SUMMARY

[0003] Therefore, the present application provides a distributed storage system capacity balancing triggering method, device and equipment to solve the problem that capacity balancing operation cannot be efficiently and accurately triggered in the related art.

[0004] In a first aspect, the present application provides a distributed storage system capacity balancing triggering method, which is executed by a capacity balancing plug-in in a process manager, and the method comprises:

[0005] obtaining a notification message sent by a process manager daemon, the notification message being a notification message sent by the process manager daemon to the capacity balancing plug-in when the process manager daemon detects that the object storage device dataset has changed;

[0006] According to the notification message, obtaining the latest object storage device dataset;

[0007] determining the current storage device failure rate according to the number of storage device failures in the latest object storage device dataset;

[0008] According to the current storage device failure ratio, and / or according to the comparison result between the latest object storage device data set and the object storage device data set acquired last time, it is determined whether to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0009] The distributed storage system capacity balancing triggering method provided by the application has the following advantages:

[0010] In the above method steps, when the process manager daemon detects that the object storage device data set has changed, the object storage device data set change notification is sent to the capacity balancing plug-in, the object storage device data set is monitored, the change of the cluster state is responded in real time, and the capacity balancing operation is triggered in time, so as to avoid the delay problem of fixed scene triggering and timing triggering. Then, the latest object storage device data set is acquired, which facilitates the judgment of whether to trigger the capacity balancing in the later stage, and the current storage device failure ratio is determined based on the latest object storage device data set. If the current storage device failure ratio reaches a preset threshold, the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation is changed to a first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed, thereby optimizing the condition of triggering the execution of the capacity balancing operation, avoiding the invalid balancing caused by the timing capacity balancing, and avoiding the failure of the function caused by manual input of the command. Moreover, the latest object storage device data set is compared with the object storage device data set acquired last time. If the data of the two is inconsistent, the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation is changed to a first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed. Through the above judgment condition, the automatic triggering of the capacity balancing is realized, the management software is not needed to be relied on, and the system maintenance cost is reduced.

[0011] In an optional embodiment, the latest object storage device data set includes:

[0012] The number of storage pools in the current object storage device data set, the number of object storage devices in each storage pool, and the number of various states of the object storage devices.

[0013] Specifically, the latest object storage device dataset is acquired, wherein the latest object storage device dataset includes the number of storage pools in the current object storage device dataset, the number of object storage devices in each storage pool, and the number of various states of the object storage devices, and the capacity balancing triggering condition is triggered based on whether the number of storage pools in the current object storage device dataset, the number of object storage devices in each storage pool, and the number of various states of the object storage devices change, so that the capacity balancing triggering management does not need to depend on the management software, and the capacity balancing operation does not need to be performed at a fixed time, thereby avoiding that a long capacity balancing operation time interval causes the cluster to be in an unbalanced state for a long time.

[0014] In an optional embodiment, before the latest object storage device dataset is acquired according to the notification message, the method further includes:

[0015] When the capacity balancing plug-in receives the daemon call, it is judged whether the changed data is the object storage device dataset.

[0016] If the changed data is the object storage device dataset, the latest object storage device dataset is acquired according to the notification message.

[0017] Specifically, when the capacity balancing plug-in receives the daemon call, the acquired call information is first judged to determine whether the changed data is the object storage device dataset, so as to prevent the daemon from calling the capacity balancing plug-in due to changes in other factors and to avoid unnecessary capacity balancing operations and reduce consumption of system resources.

[0018] In an optional embodiment, according to the current storage device failure ratio, it is determined whether to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to a first attribute value used to indicate that the capacity balancing operation is performed, so as to subsequently perform the capacity balancing operation according to the first attribute value, including:

[0019] If the current storage device failure ratio exceeds the preset threshold, it is determined to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to the first attribute value, so as to subsequently perform the capacity balancing operation according to the first attribute value.

[0020] Specifically, real-time monitoring of the failure rate of storage devices and dynamic adjustment of capacity balancing policies based on actual conditions improve system adaptability. When the device failure rate is detected to exceed a threshold, capacity balancing is immediately performed to prevent the failure from spreading within the system. Capacity balancing helps optimize the distribution of data across storage devices, improving data access efficiency and storage space utilization. Balancing enhances the system's fault tolerance and robustness. Automated capacity balancing reduces the complexity and cost of system maintenance. Optimized storage resource allocation improves overall system performance, especially under high loads. Improved system stability and performance directly impact the user experience, reducing delays and inconveniences caused by system failures. Balancing enhances data security and reduces the likelihood of data loss. Automatically adjusting policies based on the failure rate enables the system to better adapt to changing workloads and environmental conditions.

[0021] In an optional embodiment, determining, based on a comparison result between a latest object storage device dataset and a previously acquired object storage device dataset, whether to change an attribute value of an attribute parameter corresponding to triggering a capacity balancing operation to a first attribute value for instructing to perform a capacity balancing operation, so that the capacity balancing operation is subsequently performed according to the first attribute value, includes:

[0022] Compare the number of storage pools in the latest object storage device data set with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result;

[0023] If the first comparison result is that the number of storage pools in the latest object storage device data set is not equal to the number of storage pools in the previously obtained object storage device data set, it is determined to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to the first attribute value.

[0024] Specifically, by comparing the two data sets, the system ensures that its understanding of the number of storage pools is up to date, thereby guaranteeing data consistency. When the number of storage pools changes, the system can immediately trigger a capacity balancing operation, quickly responding to the change and avoiding potential bottlenecks or overload issues. Flexible resource allocation based on changes in the storage pools enhances the overall flexibility of the system. In the above approach, automated processing of changes in the number of storage pools improves the efficiency of storage resource management. Automation reduces the time and effort required of storage administrators. It also reduces resource allocation errors caused by human misjudgment. By timely balancing storage resources, system availability and reliability are improved. The system can adapt to dynamically changing storage environments and better support business growth and change.

[0025] In an optional embodiment, determining, based on a comparison result between a latest object storage device dataset and a previously acquired object storage device dataset, whether to change an attribute value of an attribute parameter corresponding to triggering a capacity balancing operation to a first attribute value for instructing to perform a capacity balancing operation, so that the capacity balancing operation is subsequently performed according to the first attribute value, includes:

[0026] Compare the number of object storage devices in each storage pool in the latest object storage device data set with the number of object storage devices in each storage pool recorded in the object storage device data set obtained previously to obtain a second comparison result;

[0027] If the second comparison result is that the number of object storage devices in each storage pool in the latest object storage device data set is not equal to the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to the first attribute value.

[0028] Specifically, the system monitors changes in the number of object storage devices in a storage pool in real time, ensuring timely and accurate data storage. By comparing the results, the system optimizes resource allocation to ensure load balancing within each storage pool. It responds promptly to changes in the number of devices, improving system reliability. Balancing reduces the risk of data loss due to device failure or improper resource allocation. Balanced storage resource allocation improves data access efficiency and reduces data retrieval time. The system also adapts to changes in storage pool size, enhancing scalability.

[0029] In an optional embodiment, the method further includes: utilizing a daemon process of the capacity balancing plug-in to periodically query an attribute value of an attribute parameter corresponding to triggering a capacity balancing operation;

[0030] When it is determined that the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation is the first attribute value, the capacity balancing operation is performed.

[0031] In an optional implementation, after the capacity balancing operation is completed, the method further includes:

[0032] The attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

[0033] Specifically, after the capacity balancing operation is completed, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation, so as to avoid repeated execution of the capacity balancing operation, which may cause a system dead loop and damage system data.

[0034] In an optional implementation, before capacity balancing begins, the method further includes:

[0035] An attribute value of the attribute parameter is determined, and if the attribute value of the attribute parameter is a first attribute value, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

[0036] Specifically, before each capacity balancing operation is started, the attribute value of the attribute parameter needs to be initialized to the second attribute value to avoid unnecessary capacity balancing operations and occupying system resources after the attribute value of the attribute parameter is modified to the first attribute value due to other reasons.

[0037] In an optional embodiment, the method further includes: obtaining a capacity balancing measurement parameter other than the current storage device failure ratio and / or a comparison result between the latest object storage device data set and a previously obtained object storage device data set;

[0038] Based on the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, determine whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so that the capacity balancing operation is subsequently performed according to the first attribute value.

[0039] Specifically, by considering a wider range of measurement parameters, the system can more comprehensively assess the status of storage resources, thereby making more accurate capacity balancing decisions. Integrating multiple measurement parameters can reduce the bias of a single metric and improve the accuracy of capacity balancing decisions. This approach is more adaptable to complex storage environments and changing business needs. Through appropriate capacity balancing, storage performance can be optimized, improving data access speed and system throughput. By avoiding unnecessary capacity balancing operations, resource waste can be reduced, thereby reducing costs.

[0040] In an optional embodiment, the capacity balancing measurement parameters include utilization of the object storage device, data access speed, network bandwidth information between the object storage devices, and storage pool capacity information;

[0041] Determining, based on a capacity balancing measurement parameter and a capacity balancing target corresponding to the capacity balancing measurement parameter, whether to change an attribute value of an attribute parameter corresponding to triggering a capacity balancing operation to a first attribute value indicating execution of the capacity balancing operation, so that the capacity balancing operation is subsequently executed according to the first attribute value, includes:

[0042] When the utilization of one or more object storage devices is overloaded or in an idle state, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0043] and / or, when the data access speed is lower than a preset speed threshold, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for instructing to perform the capacity balancing operation;

[0044] and / or, when it is determined according to the network bandwidth information that network congestion exists, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0045] And / or, when it is determined based on the storage pool capacity information that the difference between the capacity utilization rates of all storage pools is greater than a preset capacity utilization difference threshold, it is determined that the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation will be changed to a first attribute value used to indicate the execution of the capacity balancing operation.

[0046] In an optional embodiment, the method further includes: after the capacity balancing operation is performed, periodically monitoring the result of the capacity balancing operation;

[0047] When the result does not meet the preset requirements, the capacity balancing operation strategy is adjusted, where the capacity balancing strategy is a strategy for performing the capacity balancing operation.

[0048] In a second aspect, the present invention provides a distributed storage system capacity balancing triggering device, comprising:

[0049] The instruction sending module obtains the notification message sent by the process manager daemon process. The notification message is a notification message sent by the process manager daemon process to the capacity balancing plug-in when the object storage device data set is detected to be changed;

[0050] The data acquisition module obtains the latest object storage device data set based on the notification message;

[0051] A failure ratio acquisition module determines the current storage device failure ratio based on the number of storage device failures in the latest object storage device data set;

[0052] The processing module determines, based on the current storage device failure ratio and / or based on a comparison result between the latest object storage device data set and the object storage device data set obtained last time, whether to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0053] The present invention provides a distributed storage system capacity balancing triggering device, which has the following advantages:

[0054] When the process manager daemon detects a change in the object storage device dataset, it sends a notification of the change to the capacity balancing plug-in. By monitoring the object storage device dataset, it responds to changes in cluster status in real time and promptly triggers a capacity balancing operation, avoiding the delays associated with fixed-scenario and scheduled triggering. It then obtains the latest object storage device dataset to facilitate a later determination of whether to trigger capacity balancing. The current storage device failure rate is determined based on the latest object storage device dataset. If the current storage device failure rate reaches a preset threshold, the attribute value of the attribute parameter corresponding to the trigger for the capacity balancing operation is changed to the first attribute value indicating the execution of the capacity balancing operation, and the capacity balancing operation is executed. This optimizes the conditions for triggering the capacity balancing operation, avoiding both ineffective balancing caused by scheduled capacity balancing and ineffective balancing caused by manual command input. Furthermore, the latest object storage device dataset is compared with the previously obtained object storage device dataset. If the two data are inconsistent, the attribute value of the attribute parameter corresponding to the trigger for the capacity balancing operation is changed to the first attribute value indicating the execution of the capacity balancing operation, and the capacity balancing operation is executed. This judgment condition enables automatic triggering of capacity balancing without relying on management software, reducing system maintenance costs.

[0055] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the distributed storage system capacity balancing triggering method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the distributed storage system capacity balancing triggering method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0057] In a fifth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to execute the distributed storage system capacity balancing triggering method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in related technologies, the following briefly introduces the drawings required for use in the specific embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 This is a flow chart of a method for triggering capacity balancing of a distributed storage system provided by an embodiment of the present invention;

[0060] Figure 2 1 is a flow chart of another method for triggering capacity balancing of a distributed storage system provided by an embodiment of the present invention;

[0061] Figure 3 This is a schematic diagram of the overall process of the distributed storage system capacity balancing triggering method provided by an embodiment of the present invention;

[0062] Figure 4 This is a structural block diagram of a distributed storage system capacity balancing triggering device provided by an embodiment of the present invention;

[0063] Figure 5 The figure is a schematic diagram of the hardware structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0065] Distributed storage systems are the cornerstone of cloud computing and big data services. They address the scalability, reliability, and performance challenges faced by traditional storage systems through a software-defined approach. Data storage and management are core tasks in distributed storage systems, and creating a storage pool is the first step in achieving data storage. Creating a storage pool involves distributing a large number of data objects across the physical storage devices in the storage system according to a specific algorithm. This process is typically accomplished using the Controlled Replication Under Scalable Hashing (CRUSH) algorithm.

[0066] The CRUSH algorithm is a pseudo-random data distribution algorithm that maps data objects to different PGs (Placement Groups) by calculating their hash values. These PGs are then distributed to various OSDs (Object Storage Devices). The CRUSH algorithm is designed to achieve uniform data distribution while ensuring data reliability. However, due to the pseudo-randomness of the CRUSH algorithm, the actual PG distribution may not be completely balanced, which necessitates the introduction of a capacity balancing function to further optimize data distribution.

[0067] The core goal of capacity balancing is to ensure relatively even data distribution across all disks on each node. This balance not only helps improve storage system utilization but also enhances system read and write performance. Capacity balancing plays a crucial role in various scenarios, including high-performance computing, data backup, and big data analysis.

[0068] However, in related technologies, capacity balancing is triggered by timing, which results in the cluster being in an unbalanced state for a long time due to a time interval that is too long, or the PG status being frequently evaluated due to a time interval that is too short, increasing invalid operations. When the capacity balancing operation is triggered by software, the capacity balancing script needs to be called, which increases the code maintenance cost.

[0069] To solve the above problems, an embodiment of the present invention provides an embodiment of capacity balancing triggering of a distributed storage system. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system (computer device) including, for example, a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0070] In this embodiment, a distributed storage system capacity balancing triggering method is provided, which can be used for the above-mentioned computer device. Figure 1 4 is a flow chart of a method for triggering capacity balancing of a distributed storage system according to an embodiment of the present invention.

[0071] Before introducing the steps of this method, the following are explained first:

[0072] The application scenario of this application is to use the mgr (Manager daemon) plug-in subscription to trigger capacity balancing. First, by subscribing to the changes in the cluster's object storage device data set (OSDMAP), when changes in OSDMAP are detected, the plug-in process can be called up in time to trigger the capacity balancing operation. The details are as follows:

[0073] 1. Start the Mgr service and enable the capacity balancing plug-in.

[0074] 2. Initialization parameters.

[0075] The attribute value balancer_flag=false of the initialization attribute parameter is used to mark whether capacity balancing is required; wherein false is used to indicate that the capacity balancing operation is not currently being performed.

[0076] Initialize OSDMAP and obtain the latest OSDMAP from the cluster.

[0077] Specifically, based on the latest OSDMAP, count the current number of storage pools, the number of OSDs in each pool, and the total number of OSDs in each state (for example, in (indicates that the OSD is inside the cluster) / out (indicates that the OSD has been marked as unavailable) / down (indicates that the OSD daemon has stopped running) / up (indicates that the OSD daemon is running)).

[0078] 3. Subscribe OSDMAP to the Mgr daemon. When OSDMAP changes, the Mgr daemon notifies the capacity balancing plug-in.

[0079] Then, the capacity balancing plug-in in the mgr daemon process executes the following method steps, see Figure 1 As shown, the method process includes:

[0080] Step S101: Acquire a notification message sent by the process manager daemon process.

[0081] The notification message is sent by the process manager daemon to the capacity balancing plug-in when it detects a change in the object storage device dataset.

[0082] In a specific example, when the number of storage pools in the object storage device data set changes, the process manager daemon process obtains the change in the number of storage pools and sends a notification message to the capacity balancing plug-in to inform it of the change in storage pool information and start to determine whether to trigger a capacity balancing operation.

[0083] Step S102: Obtain the latest object storage device data set according to the notification message.

[0084] The latest object storage device data set includes the number of storage pools, the number of object storage devices in each storage pool, and the number of object storage devices in each state.

[0085] In an optional example, when the capacity balancing plug-in receives a notification message sent by the process manager daemon, it first confirms whether the notification message is a notification message generated by a change in a data set of the object storage device.

[0086] After confirming that the notification message is a notification message generated by a change in the object storage device dataset, the latest object storage device dataset is obtained, that is, the number of current storage pools, the number of object storage devices in each storage pool, and the number of each state of the object storage devices are obtained, so as to subsequently compare them with the number of storage pools, the number of object storage devices in each storage pool, and the number of each state of the object storage devices obtained previously.

[0087] Step S103, determining a current storage device failure ratio according to the number of object storage device failures in the latest object storage device data set.

[0088] Specifically, based on the latest object storage device data set, a ratio of the number of storage device failures in the latest object storage device data set in the current failure domain is calculated to obtain the current storage device failure ratio.

[0089] Step S104, determining whether to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value used to indicate performing the capacity balancing operation according to the current storage device failure ratio, and / or according to the comparison result between the latest object storage device data set and the object storage device data set obtained last time, so as to subsequently perform the capacity balancing operation according to the first attribute value.

[0090] Specifically, the current storage device failure ratio changes, for example, exceeds a preset ratio threshold, or the object storage device data set is inconsistent with the object storage device data set obtained last time, which indicates that the cluster state changes. Therefore, the capacity balancing operation needs to be performed in this case. Further, the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation is changed from false in the initial state to the first attribute value, which is, for example, true in an optional example. Of course, the attribute value in the initial state is not necessarily false as introduced above, for example, it can also be set to 0. Correspondingly, the first attribute value is, for example, 1.

[0091] The distributed storage system capacity balancing triggering method provided by the embodiment can send an object storage device data set change notification to a capacity balancing plug-in when a process manager daemon detects that an object storage device data set changes, respond to changes in a cluster state in real time by monitoring the object storage device data set, and timely trigger a capacity balancing operation to avoid the delay problems of fixed scene triggering and timing triggering. Then, the latest object storage device data set is obtained to facilitate the judgment of whether to trigger the capacity balancing in the later stage, and the current storage device failure ratio is determined based on the latest object storage device data set. If the current storage device failure ratio reaches a preset threshold, the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation is changed to a first attribute value for indicating the execution of the capacity balancing operation, and the capacity balancing operation is executed. The condition for triggering the execution of the capacity balancing operation is optimized, which avoids the invalid balancing caused by the timing capacity balancing and the failure of the function caused by manual input of a command. Moreover, the latest object storage device data set is compared with the object storage device data set obtained last time. If the data of the two is inconsistent, the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation is changed to the first attribute value for indicating the execution of the capacity balancing operation, and the capacity balancing operation is executed. Through the above judgment condition, the automatic triggering of the capacity balancing is realized, the management software does not need to be relied on, and the system maintenance cost is reduced.

[0092] On the basis of the foregoing embodiment, whether to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to the first attribute value for indicating the execution of the capacity balancing operation according to the current storage device failure ratio can include the following steps:

[0093] Step a1, determining whether the current storage device failure ratio exceeds a preset threshold.

[0094] Step a2, if the current storage device failure ratio exceeds the preset threshold, determining to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to the first attribute value.

[0095] The current storage device failure ratio is the proportion of the number of storage device failures in the current failure domain in the object storage device data set.

[0096] Specifically, the preset threshold is set based on a cluster environment, and whether the proportion of the number of storage device failures in the current failure domain in the latest object storage device data set reaches the preset threshold is checked. If the proportion of the number of storage device failures in the current failure domain in the latest object storage device data set reaches the preset threshold, the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation is changed to the first attribute value.

[0097] In a specific example, for example, the preset threshold is set to 10%, the number of failures in the latest object storage device data set is 5, and the number of object storage devices in the current fault domain is 10. Then, the proportion of the number of storage device failures in the latest object storage device data set in the current fault domain (20%) exceeds the preset threshold (10%). Therefore, the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation needs to be changed to the first attribute value True.

[0098] Based on the foregoing embodiment, determining whether to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value indicating execution of the capacity balancing operation based on a comparison result between the latest object storage device dataset and the previously acquired object storage device dataset, so as to subsequently execute the capacity balancing operation based on the first attribute value, may include the following steps:

[0099] Step b1: Compare the number of storage pools in the latest object storage device data set with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result.

[0100] Step b2: If the first comparison result is that the number of storage pools in the latest object storage device data set is not equal to the number of storage pools in the previously acquired object storage device data set, determine to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to the first attribute value.

[0101] Specifically, the number of storage pools in the latest object storage device data set is compared with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result.

[0102] If the number of storage pools in the latest object storage device data set in the first comparison result is not equal to the number of storage pools in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to the first attribute value.

[0103] In a specific example, for example, if the number of storage pools in the latest object storage device data set is 90 and the number of storage pools in the previously obtained object storage device data set is 100, the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation is changed to the first attribute value True.

[0104] Based on the above embodiment, determining whether to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to the first attribute value indicating execution of the capacity balancing operation based on the comparison result between the latest object storage device dataset and the previously acquired object storage device dataset, so that the capacity balancing operation is subsequently executed according to the first attribute value, may further include the following steps:

[0105] Step c1: compare the number of object storage devices in each storage pool in the latest object storage device data set with the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time to obtain a second comparison result.

[0106] Step c2: If the second comparison result is that the number of object storage devices in each storage pool in the latest object storage device data set is not equal to the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, determine to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value.

[0107] In a specific example, for example, the number of object storage devices in each storage pool in the latest object storage device data set is 50, and the number of object storage devices in each storage pool recorded in the previously acquired object storage device data set is 55, then the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation is changed to the first attribute value True.

[0108] Specifically, in the above method steps, the latest object storage device data set is compared with the object storage device data set obtained the previous time. If the data of the two are inconsistent, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed. Through the above judgment conditions, automatic triggering of capacity balancing is achieved without relying on management software, thereby reducing system maintenance costs. The current storage device failure ratio is determined based on the latest object storage device data set. If the current storage device failure ratio reaches a preset threshold, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed. This optimizes the conditions for triggering the execution of the capacity balancing operation, avoiding both invalid balancing due to timed capacity balancing and failure of the function due to manual input of commands.

[0109] Alternatively, after the capacity balancing operation is completed, the method may further include:

[0110] The attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

[0111] Specifically, after the capacity balancing operation is performed, the second attribute value of the capacity balancing operation, that is, balancer_flag is modified to false.

[0112] Specifically, after the capacity balancing operation is completed, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation, so as to avoid repeated execution of the capacity balancing operation, which may cause a system dead loop and damage system data.

[0113] Further optionally, before the capacity balancing is started, the method further includes:

[0114] An attribute value of the attribute parameter is determined, and if the attribute value of the attribute parameter is a first attribute value, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

[0115] Specifically, before each capacity balancing operation is started, the attribute value of the attribute parameter needs to be initialized to the second attribute value to avoid unnecessary capacity balancing operations and occupying system resources after the attribute value of the attribute parameter is modified to the first attribute value due to other reasons.

[0116] In an optional example, the method may further include the following method steps, see for details: Figure 2 Shown, including:

[0117] Step S201 : Acquire capacity balancing measurement parameters other than the current storage device failure ratio and / or the comparison result between the latest object storage device data set and the previously acquired object storage device data set.

[0118] In step S202, based on the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, it is determined whether the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to a first attribute value used to indicate the execution of the capacity balancing operation, so that the capacity balancing operation is subsequently performed according to the first attribute value.

[0119] Specifically, in a specific example, the capacity balancing measurement parameters include utilization of object storage devices, data access speed, network bandwidth information between object storage devices, and storage pool capacity information.

[0120] The method of determining, based on the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, whether to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes:

[0121] When the utilization of one or more object storage devices is overloaded or in an idle state, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0122] and / or, when the data access speed is lower than a preset speed threshold, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for instructing to perform the capacity balancing operation;

[0123] and / or, when it is determined according to the network bandwidth information that there is network congestion, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value indicating to perform the capacity balancing operation;

[0124] and / or, when it is determined according to the storage pool capacity information that the difference between the capacity usage rates of all storage pools is greater than a preset capacity usage rate difference threshold, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value indicating to perform the capacity balancing operation.

[0125] In addition, after the capacity balancing operation is performed, the method can further include:

[0126] periodically monitoring the result after the capacity balancing operation is performed;

[0127] when the result does not meet a preset requirement, adjusting the capacity balancing operation strategy, wherein the capacity balancing strategy is a strategy for performing the capacity balancing operation.

[0128] The specific implementation process of the above method steps, for example, includes:

[0129] 1. Collect data

[0130] First, the following data needs to be collected:

[0131] 1) Object storage device utilization rate, such as the current load of each object storage device, including CPU usage, disk I / O, network bandwidth usage, etc.

[0132] 2) Data access speed: read and write speed and latency of each storage device.

[0133] 3) Network bandwidth information between object storage devices: bandwidth and latency of network connection between storage devices.

[0134] 4) Storage pool capacity information: total capacity and used capacity of each storage pool.

[0135] 2. Define balancing goals

[0136] According to the collected data, the following capacity balancing goals are defined:

[0137] 1) Object storage device utilization rate: avoid any device overload or idling, keep the utilization rate of all devices within a reasonable range.

[0138] 2) Optimize data access speed: improve data access speed and reduce latency, for example, increase data access speed to be higher than or equal to a preset speed threshold.

[0139] 3) Maximize bandwidth utilization: ensure that network bandwidth is fully utilized, avoid bandwidth bottlenecks caused by network congestion, etc.

[0140] 4) Maintaining balanced storage pool capacity: Ensure that the capacity utilization of each storage pool is close to that of the others, avoiding overloading one storage pool while leaving others idle. Specifically, the difference in capacity utilization across all storage pools is less than or equal to the preset capacity utilization difference threshold.

[0141] 3. When a capacity balancing operation is triggered, a balancing strategy is executed, such as the following strategies:

[0142] 1) Balance based on equipment utilization

[0143] When it is detected that the utilization of a device is too high, data is automatically migrated to a device with lower utilization.

[0144] or,

[0145] Before data is written, data distribution is predicted based on the current utilization of the device to optimize data placement.

[0146] 2) Balance based on data access speed

[0147] Continuously monitor data access speeds to identify performance bottlenecks. Use caches for frequently accessed data to reduce access to slow devices.

[0148] 3) Balancing based on bandwidth between devices

[0149] Monitor bandwidth usage between devices to ensure proper bandwidth allocation and avoid network congestion.

[0150] Distribute data across multiple storage devices to prevent a single device or network connection from becoming a bottleneck.

[0151] 4) Balancing based on storage pool capacity

[0152] Automatically expand or reduce storage pool capacity based on storage pool usage.

[0153] If a storage pool is close to full, migrate data to another storage pool.

[0154] 4. Implement balancing operations

[0155] During specific execution, for example, balancing tasks are run regularly to check device status and data distribution, and adjust data according to balancing targets. Balancing operations are triggered when device status changes or reaches a certain threshold.

[0156] Alternatively, use machine learning or predictive algorithms to make intelligent decisions based on historical and real-time monitoring data.

[0157] 5. Monitor and adjust

[0158] Continuously monitor the effectiveness of balancing operations to ensure that the balancing strategy achieves the expected goals.

[0159] Based on monitoring data, adjust balancing strategies and parameters to adapt to the changing environment.

[0160] Through the above operation process, a more intelligent and flexible capacity balancing system can be implemented to ensure the effective use of storage resources and the overall performance of the system.

[0161] In an optional example, RNN (recurrent neural network) and its variants such as LSTM and GRU network models can be used as a capacity balancing trigger prediction model. For details, see the following:

[0162] Establish a comprehensive monitoring system to collect data on key indicators such as CPU usage, memory usage, disk I / O, network traffic, etc.

[0163] The collected data is preprocessed, including outlier removal, standardization (unification of dimensions), and other operations.

[0164] The variables that have the greatest impact on the prediction target are selected from the preprocessed data as input features.

[0165] The model is trained based on the input features to obtain a capacity balancing trigger prediction model at the end of training.

[0166] The capacity balancing trigger prediction model after training is used to determine whether the cluster needs capacity balancing operations.

[0167] In this embodiment, a distributed storage system capacity balancing triggering method is provided, which can be used in the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 3 This is a flowchart of the method for triggering capacity balancing in a distributed storage system according to an embodiment of the present invention. This method flow is a relatively specific and comprehensive example of applying the method of the above embodiment of the present application. Specifically, see below, including the following steps:

[0168] Step 1: Start the cluster service and enable the capacity balancing plug-in.

[0169] Step 2: Initialize parameters.

[0170] Set the attribute parameter (balancer_flag) to false to indicate whether capacity balancing is required. Obtain the latest object storage device dataset (osdmap) from the cluster. Based on the latest object storage device dataset, count the current number of storage pools (pools), the number of object storage devices (osds) in each storage pool, and the total number of object storage devices in each state (in / out / down / up).

[0171] Step 3: When the object storage device data set changes, the daemon notifies the capacity balancing plug-in and calls the plug-in's notify function.

[0172] The notify function is a judgment function, and a method similar to that mentioned in the above embodiment is used to determine whether the judgment condition for triggering capacity balancing is called by calling the notify function.

[0173] Step 4: upon receiving the daemon process call, determine whether the changed data is an object storage device data set.

[0174] If the determination is successful, the latest object storage device data set is obtained, and the latest object storage device data set is compared with the previously obtained object storage device data set by checking statistics.

[0175] The judgment conditions are as follows:

[0176] Checks whether the number of storage pools is equal to the record. This check is used to determine whether a storage pool has been created or deleted.

[0177] Check whether the number of object storage devices in each storage pool is equal to the record. This check can determine whether the storage pool has been expanded or reduced.

[0178] If the object storage device data set is updated due to the ejection of a certain object storage device, it is determined whether the amount of failed object storage device data in the fault domain where the object storage device data is located reaches 10%.

[0179] If any of the above three situations is successfully judged, the attribute value of the attribute parameter is assigned to true.

[0180] Step 5: The plug-in daemon process periodically queries the attribute value of the attribute parameter.

[0181] When the attribute value of the attribute parameter is true, the capacity balancing operation is performed, and after the operation is completed, the attribute value of the attribute parameter is modified to false.

[0182] If only the attribute value of the information is false, wait for a period of time, for example, 60 seconds, and then query the attribute value of the attribute parameter again.

[0183] In addition, in the above-mentioned determination process, if in step 4, when receiving the daemon process call, it is determined that the changed data is not an object storage device data set, no operation is performed, that is, pass.

[0184] Similarly, if capacity balancing is not required, no operation is performed, i.e., pass.

[0185] The specific implementation process of the method embodiment has been described in detail above, so it will not be repeated here.

[0186] The distributed storage system capacity balancing triggering method provided by this embodiment includes the following steps: when the process manager daemon detects a change in an object storage device dataset, it sends an object storage device dataset change notification to the capacity balancing plug-in. By monitoring the object storage device dataset, it responds to cluster status changes in real time and promptly triggers a capacity balancing operation, thus avoiding the delay issues associated with fixed-scenario and scheduled triggering. Subsequently, the latest object storage device dataset is obtained to facilitate a later determination of whether to trigger capacity balancing. The current storage device failure rate is determined based on the latest object storage device dataset. If the current storage device failure rate reaches a preset threshold, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to a first attribute value indicating the execution of the capacity balancing operation, and the capacity balancing operation is executed. This optimizes the conditions for triggering the capacity balancing operation, avoiding both ineffective balancing due to scheduled capacity balancing and ineffectiveness due to manual command input. Furthermore, the latest object storage device dataset is compared with the previously obtained object storage device dataset. If the two data are inconsistent, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value indicating the execution of the capacity balancing operation, and the capacity balancing operation is executed. Through the above judgment conditions, automatic triggering of capacity balancing is achieved without relying on management software, thus reducing system maintenance costs.

[0187] This embodiment also provides a distributed storage system capacity balancing triggering device, which is used to implement the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0188] This embodiment provides a distributed storage system capacity balancing triggering device, such as Figure 4 Shown, including:

[0189] The instruction sending module 401 obtains a notification message sent by the process manager daemon process, where the notification message is a notification message sent by the process manager daemon process to the capacity balancing plug-in when the process manager daemon process detects a change in the object storage device data set;

[0190] The data acquisition module 402 acquires the latest object storage device data set according to the notification message;

[0191] The failure ratio acquisition module 403 determines the current storage device failure ratio according to the number of storage device failures in the latest object storage device data set;

[0192] The processing module 404 determines whether to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, based on the current storage device failure ratio and / or based on the comparison result between the latest object storage device data set and the object storage device data set obtained last time, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0193] In an optional example, the latest object storage device dataset includes:

[0194] The current object storage device dataset includes the number of storage pools, the number of object storage devices in each storage pool, and the number of object storage devices in each state.

[0195] In an optional example, the processing module 404 is further configured to determine whether the changed data is an object storage device data set when the capacity balancing plug-in receives a call from the daemon process;

[0196] The acquisition module 402 is also used to obtain the latest object storage device dataset according to the notification message if the changed data is an object storage device dataset. The latest object storage device dataset includes: the number of storage pools in the current object storage device dataset, the number of object storage devices in each storage pool, and the number of each state of the object storage device.

[0197] In an optional example, the processing module 404 is specifically used to: if the current storage device failure ratio exceeds a preset threshold, determine to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value, so as to subsequently perform the capacity balancing operation according to the first attribute value.

[0198] In an optional example, the processing module 404 is specifically configured to: compare the number of storage pools in the latest object storage device data set with the number of storage pools in the object storage device data set obtained previously to obtain a first comparison result;

[0199] If the first comparison result is that the number of storage pools in the latest object storage device data set is not equal to the number of storage pools in the previously obtained object storage device data set, it is determined to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to the first attribute value.

[0200] In an optional example, the processing module 404 is specifically configured to: compare the number of object storage devices of each storage pool in the latest object storage device data set with the number of object storage devices of each storage pool recorded in the object storage device data set acquired last time, and obtain a second comparison result;

[0201] If the second comparison result is that the number of object storage devices of each storage pool in the latest object storage device data set is not equal to the number of object storage devices of each storage pool recorded in the object storage device data set acquired last time, it is determined that the attribute value of the attribute parameter corresponding to the capacity balancing operation is changed to the first attribute value.

[0202] In an optional example, the processing module 404 is further configured to: periodically query, by using a daemon process of the capacity balancing plug-in, the attribute value of the attribute parameter corresponding to the capacity balancing operation;

[0203] When it is determined that the attribute value of the attribute parameter corresponding to the capacity balancing operation is the first attribute value, the capacity balancing operation is performed.

[0204] In an optional example, the processing module 404 is further configured to: after the capacity balancing operation is performed, the attribute value of the attribute parameter is modified to the second attribute value for stopping the execution of the capacity balancing operation.

[0205] In an optional example, the processing module 404 is further configured to: before the capacity balancing operation is performed, the attribute value of the attribute parameter is judged, and if the attribute value of the attribute parameter is the first attribute value, the attribute value of the attribute parameter is modified to the second attribute value for stopping the execution of the capacity balancing operation.

[0206] In an optional example, the obtaining module 402 is further configured to: obtain a capacity balancing measurement parameter in addition to the current storage device failure rate and / or the comparison result between the latest object storage device data set and the object storage device data set acquired last time;

[0207] The processing module 404 is further configured to: according to the capacity balancing measurement parameter and a capacity balancing target corresponding to the capacity balancing measurement parameter, determine whether to change the attribute value of the attribute parameter corresponding to the capacity balancing operation to the first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently perform the capacity balancing operation according to the first attribute value.

[0208] In an optional example, the capacity balancing measurement parameter includes the utilization rate of the object storage device, the data access speed, the network bandwidth information between the object storage devices, and the storage pool capacity information; and the processing module 404 is specifically configured to: when the utilization rate of one or more object storage devices has an overload behavior or is in an idle state, determine to change the attribute value of the attribute parameter corresponding to the capacity balancing operation to the first attribute value for indicating the execution of the capacity balancing operation.

[0209] and / or, when the data access speed is lower than a preset speed threshold, determining to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to a first attribute value indicating execution of the capacity balancing operation;

[0210] and / or, when it is determined according to the network bandwidth information that there is network congestion, determining to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to a first attribute value indicating execution of the capacity balancing operation;

[0211] and / or, when it is determined according to the storage pool capacity information that the difference between the capacity usage rates of all storage pools is greater than a preset capacity usage rate difference threshold, determining to change the attribute value of the attribute parameter corresponding to the triggering of the capacity balancing operation to a first attribute value indicating execution of the capacity balancing operation.

[0212] In an optional example, the processing module 404 is further configured to periodically monitor the result after the capacity balancing operation is executed.

[0213] When the result does not meet a preset requirement, adjusting the capacity balancing operation strategy, wherein the capacity balancing strategy is a strategy for executing the capacity balancing operation.

[0214] Further function descriptions of the above-mentioned various modules and units are the same as those of the corresponding embodiments, and will not be repeated here.

[0215] The distributed storage system capacity balancing triggering device in the embodiment is in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory executing one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0216] The embodiment of the present application also provides a computer device having the above-mentioned Figure 4 distributed storage system capacity balancing triggering device.

[0217] Please refer to Figure 5 , Figure 5 is a structural schematic diagram of a computer device provided by an optional embodiment of the present application, as shown in Figure 5As shown, the computer device includes one or more processors 10, memory 20, and interfaces 30 for external devices such as a display, a keyboard or a pointing device. One or more buses 50 can be used for communication between the above components. Various embodiments of the present disclosure are described in terms of this example computer device, a display, a keyboard, or a pointing device, or in terms of a process performed by the computer device, the display, the keyboard, or the pointing device. However, a person having ordinary skill in the art will recognize that the above-described embodiments of the present disclosure can be practiced with other computer device structures and that claims of the present disclosure should not be limited to the above-described computer device structure and process. Figure 5 The processor 10 is used as an example in the embodiments.

[0218] The processor 10 can be a central processing unit, a network processor, or a combination thereof. The processor 10 can further include a hardware integrated circuit. The hardware integrated circuit can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic device, a general array logic, or any combination thereof.

[0219] The memory 20 stores instructions that can be executed by the at least one processor 10, so that the at least one processor 10 performs the method shown in the above embodiments.

[0220] The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function, and the like. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0221] The memory 20 can include a volatile memory such as a random access memory, and can also include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk. The memory 20 can further include a combination of the above-mentioned types of memories.

[0222] The computer device further includes a communication interface 30 for communication between the computer device and other devices or communication networks.

[0223] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0224] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0225] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A distributed storage system capacity balancing triggering method, characterized in that: The method is executed by a capacity balancing plug-in in a process manager, and the method includes: Acquire a notification message sent by a process manager daemon process, where the notification message is a notification message sent by the process manager daemon process to the capacity balancing plug-in when the process manager daemon process detects a change in an object storage device data set; According to the notification message, the latest object storage device data set is obtained, wherein the latest object storage device data set includes the number of storage pools in the current object storage device data set, the number of object storage devices in each storage pool, and the number of each state of the object storage devices; determining, based on a comparison result between the latest object storage device dataset and a previously acquired object storage device dataset, whether to change an attribute value of an attribute parameter corresponding to triggering a capacity balancing operation to a first attribute value indicating execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value; The determining, based on a comparison result between the latest object storage device dataset and a previously acquired object storage device dataset, whether to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation includes: Compare the number of storage pools in the latest object storage device data set with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result; If the first comparison result is that the number of storage pools in the latest object storage device data set is not equal to the number of storage pools in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to the first attribute value.

2. The distributed storage system capacity balancing triggering method according to claim 1, characterized in that: Before obtaining the latest object storage device data set according to the notification message, the method further includes: When the capacity balancing plug-in receives a call from the daemon process, it determines whether the changed data is an object storage device data set; If the changed data is an object storage device dataset, the latest object storage device dataset is obtained according to the notification message.

3. The distributed storage system capacity balancing triggering method according to claim 1, characterized in that: The latest object storage device data set also includes: the number of storage device failures. After obtaining the latest object storage device data set according to the notification message, the method further includes: Determining a current storage device failure ratio based on the number of storage device failures in the latest object storage device data set; According to the current storage device failure ratio, it is determined whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation, so that the capacity balancing operation is subsequently executed according to the first attribute value.

4. The distributed storage system capacity balancing triggering method according to claim 3, characterized in that: The determining, based on the current storage device failure ratio, whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for instructing to perform the capacity balancing operation, so that the capacity balancing operation is subsequently performed based on the first attribute value, includes: If the current storage device failure ratio exceeds a preset threshold, it is determined to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value, so that the capacity balancing operation is subsequently performed according to the first attribute value.

5. The distributed storage system capacity balancing triggering method according to claim 1, characterized in that: Determining, based on a comparison result between the latest object storage device dataset and a previously acquired object storage device dataset, whether to change an attribute value of an attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value indicating execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes: Compare the number of object storage devices in each storage pool in the latest object storage device data set with the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time to obtain a second comparison result; If the second comparison result is that the number of object storage devices in each storage pool in the latest object storage device data set is not equal to the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, determine to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to the first attribute value.

6. The distributed storage system capacity balancing triggering method according to any one of claims 1 to 5, characterized in that: The method further comprises: Utilizing the daemon process of the capacity balancing plug-in, periodically querying the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation; When it is determined that the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is the first attribute value, the capacity balancing operation is performed.

7. The distributed storage system capacity balancing triggering method according to any one of claims 1 to 5, characterized in that: After the capacity balancing operation is completed, the method further includes: The attribute value of the attribute parameter is modified to a second attribute value for stopping execution of the capacity balancing operation.

8. The distributed storage system capacity balancing triggering method according to claim 7, characterized in that: Before the capacity balancing is started, the method further includes: Determining the attribute value of the attribute parameter; If the attribute value of the attribute parameter is the first attribute value, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

9. The distributed storage system capacity balancing triggering method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire a capacity balancing measurement parameter other than the current storage device failure ratio and / or a comparison result between the latest object storage device data set and a previously acquired object storage device data set; Based on the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, determine whether to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

10. The distributed storage system capacity balancing triggering method according to claim 9, characterized in that: The capacity balancing measurement parameters include utilization of the object storage device, data access speed, network bandwidth information between the object storage devices, and storage pool capacity information; The determining, based on the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, whether to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value indicating execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes: When the utilization of one or more object storage devices is overloaded or in an idle state, determining to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation; and / or, when the data access speed is lower than a preset speed threshold, determining to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation; and / or, when it is determined according to the network bandwidth information that network congestion exists, determining to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation; And / or, when it is determined based on the storage pool capacity information that the difference between the capacity utilization rates of all storage pools is greater than a preset capacity utilization difference threshold, it is determined that the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation will be changed to a first attribute value used to indicate the execution of the capacity balancing operation.

11. The distributed storage system capacity balancing triggering method according to claim 10, characterized in that: The method further comprises: After the capacity balancing operation is completed, periodically monitoring the result of the capacity balancing operation; When the result does not meet the preset requirement, the capacity balancing operation strategy is adjusted, wherein the capacity balancing strategy is a strategy for executing the capacity balancing operation.

12. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the distributed storage system capacity balancing triggering method according to any one of claims 1 to 11 by executing the computer instructions.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the distributed storage system capacity balancing triggering method according to any one of claims 1 to 11.

14. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer device to execute the distributed storage system capacity balancing triggering method according to any one of claims 1 to 11.

Images