Distributed storage system capacity balance triggering method, device and equipment

Abstract

The invention relates to the technical field of data storage, and discloses a distributed storage system capacity balance triggering method, device and equipment, and the method comprises the steps: obtaining a notification message sent by a daemon process of a process manager, and obtaining a latest object storage equipment data set according to the notification message; determining a current storage device fault proportion according to the number of storage device faults in the latest object storage device data set; according to the fault proportion of the current storage device and / or according to the comparison result between the latest object storage device data set and the object storage device data set obtained last time, whether the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed into the first attribute value or not is determined, so that the capacity balancing operation is completed according to the first attribute value subsequently. And executing a capacity balancing operation. The change of the cluster state is responded in real time, the capacity balancing operation is triggered in time, the condition for triggering the execution of the capacity balancing operation is optimized, the capacity balancing is automatically triggered, dependence on management software is not needed, and the system maintenance cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a distributed storage system capacity balancing triggering method, device and equipment. Background Art

[0002] In the related art, two methods are usually used to trigger capacity balancing: one is to actively call the capacity balancing script after the fixed scenario ends, and the other is to regularly evaluate the placement group (PG) status of the cluster and trigger capacity balancing. The first method relies on the existence of management software. For clusters that do not have management software installed, the capacity balancing operation cannot be automatically triggered. In addition, with the development of technology and the increase in customer demand, there may be more and more scenarios that require calling capacity balancing scripts, which will lead to an increase in code maintenance costs. The second method triggers capacity balancing by regularly evaluating the PG status of the cluster. It is a difficult problem to set a suitable time interval for this method. If the time interval is too long, the cluster will be in an unbalanced state for a long time, while if the time interval is too short, the PG status will be frequently evaluated, increasing invalid operations and wasting cluster resources. Moreover, the scheduled execution of capacity balancing may conflict with other services of the cluster. Especially when the cluster is executing critical services, the execution of capacity balancing may reduce the business processing capabilities of the cluster and even bring risks. Summary of the invention

[0003] In view of this, the present invention provides a distributed storage system capacity balancing triggering method, device and equipment to solve the problem in the related art that the capacity balancing operation cannot be triggered efficiently and accurately.

[0004] In a first aspect, the present invention provides a distributed storage system capacity balancing triggering method, the method being executed by a capacity balancing plug-in in a process manager, the method comprising:

[0005] Obtain a notification message sent by the process manager daemon, which is a notification message sent by the process manager daemon to the capacity balancing plug-in when the process manager daemon detects a change in the object storage device data set;

[0006] According to the notification message, obtain the latest object storage device data set;

[0007] Determine the current storage device failure ratio according to the number of storage device failures in the latest object storage device data set;

[0008] Based on the current storage device failure ratio and / or based on the comparison result between the latest object storage device data set and the object storage device data set obtained last time, determine whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0009] A distributed storage system capacity balancing triggering method provided by the present invention has the following advantages:

[0010] In the above method steps, when the process manager daemon detects that the object storage device data set has changed, it sends an object storage device data set change notification to the capacity balancing plug-in, monitors the object storage device data set, responds to changes in cluster status in real time, and triggers the capacity balancing operation in time, avoiding the delay problem of fixed scene triggering and timed triggering. Then, the latest object storage device data set is obtained to facilitate the later judgment of whether to trigger capacity balancing, and the current storage device failure ratio is determined based on the latest object storage device data set. If the current storage device failure ratio reaches the preset threshold, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed, which optimizes the conditions for triggering the execution of the capacity balancing operation, avoiding invalid balancing due to timed capacity balancing, and avoiding the failure of the function caused by manual input of commands. Moreover, the latest object storage device data set is compared with the object storage device data set obtained last time. If the data of the two are inconsistent, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed. Through the above judgment conditions, automatic triggering of capacity balancing is achieved without relying on management software, thus reducing system maintenance costs.

[0011] In an optional implementation, the latest object storage device data set includes:

[0012] The current object storage device data set contains the number of storage pools, the number of object storage devices in each storage pool, and the number of object storage devices in each state.

[0013] Specifically, the latest object storage device data set is obtained, wherein the latest object storage device data set includes the number of storage pools in the current object storage device data set, the number of object storage devices in each storage pool, and the number of each state of the object storage devices. The capacity balancing trigger condition of the present invention is triggered based on whether the number of storage pools in the current object storage device data set, the number of object storage devices in each storage pool, and the number of each state of the object storage devices change. There is no need to rely on management software for capacity balancing trigger management, and there is no need to perform capacity balancing operations at regular intervals, thereby avoiding the long-term imbalance of the cluster caused by the capacity balancing operation time interval being too long.

[0014] In an optional implementation, before obtaining the latest object storage device data set according to the notification message, the method further includes:

[0015] When the capacity balancing plug-in receives a call from the daemon process, it determines whether the changed data is an object storage device data set;

[0016] If the changed data is an object storage device data set, the latest object storage device data set is obtained according to the notification message.

[0017] Specifically, when the capacity balancing plug-in receives a call from the daemon process, it first judges the obtained call information to determine whether the changed data is an object storage device data set, to prevent the daemon process from calling the capacity balancing plug-in due to changes in other factors, thereby avoiding unnecessary capacity balancing operations and reducing the consumption of system resources.

[0018] In an optional implementation, determining, according to the current storage device failure ratio, whether to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation, so that the capacity balancing operation is subsequently executed according to the first attribute value, includes:

[0019] If the current storage device failure ratio exceeds a preset threshold, it is determined to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value, so that the capacity balancing operation is subsequently performed according to the first attribute value.

[0020] Specifically, the failure rate of storage devices is monitored in real time, and the capacity balancing strategy is adjusted dynamically according to the actual situation, thereby improving the adaptability of the system. When it is detected that the device failure rate exceeds the threshold, the capacity balancing operation is performed immediately to prevent the failure from spreading in the system. The capacity balancing operation helps to optimize the distribution of data among storage devices, improve data access efficiency and storage space utilization. Through the balancing operation, the fault tolerance of the system is enhanced and the robustness of the system is improved. The automated capacity balancing operation reduces the complexity and cost of system maintenance. The optimized storage resource allocation can improve the overall system performance, especially under high load conditions. The improvement of system stability and performance directly affects the user experience and reduces the delays and inconveniences caused by system failures. Through the balancing operation, the security of data is improved and the possibility of data loss is reduced. Automatic adjustment of the strategy according to the failure rate enables the system to better adapt to changing workloads and environmental conditions.

[0021] In an optional implementation, determining whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes:

[0022] Compare the number of storage pools in the latest object storage device data set with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result;

[0023] If the first comparison result is that the number of storage pools in the latest object storage device data set is not equal to the number of storage pools in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value.

[0024] Specifically, by comparing the two data sets, the system ensures that the system has the latest understanding of the number of storage pools, thereby ensuring data consistency. When the number of storage pools changes, the system can immediately trigger a capacity balancing operation to quickly respond to changes and avoid potential bottlenecks or overload problems. Flexible resource allocation based on changes in storage pools enhances the overall flexibility of the system. In the above method, automated processing of changes in the number of storage pools improves the efficiency of storage resource management. Automation reduces the time and energy requirements of storage administrators. It can also reduce resource allocation errors caused by human misjudgment. By timely balancing storage resources, the availability and reliability of the system are improved. The system can adapt to dynamically changing storage environments and better support business growth and changes.

[0025] In an optional implementation, determining whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes:

[0026] Compare the number of object storage devices in each storage pool in the latest object storage device data set with the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, to obtain a second comparison result;

[0027] If the second comparison result is that the number of object storage devices in each storage pool in the latest object storage device data set is not equal to the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value.

[0028] Specifically, the system can monitor the changes in the number of object storage devices in the storage pool in real time to ensure the real-time and accuracy of data storage. By comparing the results, the system can optimize resource allocation to ensure load balancing for each storage pool. It can respond promptly when the number of devices changes, improving the reliability of the system. Through balancing operations, the risk of data loss caused by device failure or improper resource allocation can be reduced. Balanced storage resource allocation can improve data access efficiency and reduce data retrieval time. The system can also adapt to changes in storage pool size, enhancing the scalability of the system.

[0029] In an optional implementation, the method further includes: utilizing a daemon process of the capacity balancing plug-in to periodically query an attribute value of an attribute parameter corresponding to a triggered capacity balancing operation;

[0030] When it is determined that the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation is the first attribute value, the capacity balancing operation is performed.

[0031] In an optional implementation, after the capacity balancing operation is completed, the method further includes:

[0032] The attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

[0033] Specifically, after the capacity balancing operation is completed, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation, so as to avoid repeated execution of the capacity balancing operation, which may cause a system dead loop and damage system data.

[0034] In an optional implementation, before the capacity balancing starts, the method further includes:

[0035] An attribute value of the attribute parameter is determined, and if the attribute value of the attribute parameter is a first attribute value, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

[0036] Specifically, before each capacity balancing execution begins, the attribute value of the attribute parameter needs to be initialized to the second attribute value to avoid useless capacity balancing operations and occupation of system resources after the attribute value of the attribute parameter is modified to the first attribute value due to other reasons.

[0037] In an optional embodiment, the method further includes: obtaining a capacity balancing measurement parameter other than the current storage device failure ratio and / or a comparison result between the latest object storage device data set and the object storage device data set obtained last time;

[0038] According to the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, determine whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0039] Specifically, by considering a wider range of measurement parameters, the system can more comprehensively evaluate the status of storage resources and make more accurate capacity balancing decisions. Combining multiple measurement parameters can reduce the deviation of a single indicator and improve the accuracy of capacity balancing decisions. When faced with complex storage environments and changing business needs, this approach can better adapt to these scenarios. Through reasonable capacity balancing, storage performance can be optimized, data access speed and system throughput can be improved. By avoiding unnecessary capacity balancing operations, resource waste can be reduced, thereby reducing costs.

[0040] In an optional implementation, the capacity balancing measurement parameters include utilization of the object storage device, data access speed, network bandwidth information between the object storage devices, and storage pool capacity information;

[0041] Determining, according to a capacity balancing measurement parameter and a capacity balancing target corresponding to the capacity balancing measurement parameter, whether to change an attribute value of an attribute parameter corresponding to a triggered capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation, so that the capacity balancing operation is subsequently executed according to the first attribute value, includes:

[0042] When the utilization of one or more object storage devices is overloaded or in an idle state, determine to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation;

[0043] and / or, when the data access speed is lower than a preset speed threshold, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0044] and / or, when it is determined according to the network bandwidth information that there is network congestion, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0045] And / or, when it is determined based on the storage pool capacity information that the difference between the capacity utilization rates of all storage pools is greater than a preset capacity utilization difference threshold, it is determined to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value used to indicate performing the capacity balancing operation.

[0046] In an optional implementation, the method further includes: after the capacity balancing operation is performed, periodically monitoring the result of performing the capacity balancing operation;

[0047] When the result does not meet the preset requirements, the capacity balancing operation strategy is adjusted, where the capacity balancing strategy is a strategy for executing the capacity balancing operation.

[0048] In a second aspect, the present invention provides a distributed storage system capacity balancing triggering device, comprising:

[0049] The instruction sending module obtains the notification message sent by the process manager daemon, where the notification message is a notification message sent by the process manager daemon to the capacity balancing plug-in when the object storage device data set is detected to be changed;

[0050] The data acquisition module acquires the latest object storage device data set according to the notification message;

[0051] A failure ratio acquisition module determines the current storage device failure ratio according to the number of storage device failures in the latest object storage device data set;

[0052] The processing module determines whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, based on the current storage device failure ratio and / or based on the comparison result between the latest object storage device data set and the object storage device data set obtained last time, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0053] A distributed storage system capacity balancing trigger device provided by the present invention has the following advantages:

[0054] When the process manager daemon detects that the object storage device data set has changed, it sends an object storage device data set change notification to the capacity balancing plug-in. By monitoring the object storage device data set, it responds to the change of the cluster status in real time, and triggers the capacity balancing operation in time, avoiding the delay problem of fixed scene triggering and timed triggering. Then, the latest object storage device data set is obtained to facilitate the later judgment of whether to trigger capacity balancing, and the current storage device failure ratio is determined based on the latest object storage device data set. If the current storage device failure ratio reaches the preset threshold, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is performed, which optimizes the conditions for triggering the execution of the capacity balancing operation, avoiding invalid balancing caused by timed capacity balancing, and avoiding the failure of the function caused by manual input of commands. Moreover, the latest object storage device data set is compared with the object storage device data set obtained last time. If the data of the two are inconsistent, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is performed. Through the above judgment conditions, automatic triggering of capacity balancing is realized without relying on management software, reducing system maintenance costs.

[0055] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the distributed storage system capacity balancing triggering method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the distributed storage system capacity balancing triggering method of the first aspect or any corresponding embodiment thereof.

[0057] In a fifth aspect, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the distributed storage system capacity balancing triggering method of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 It is a flow chart of a distributed storage system capacity balancing triggering method provided by an embodiment of the present invention;

[0060] Figure 2 It is a flow chart of another distributed storage system capacity balancing triggering method provided by an embodiment of the present invention;

[0061] Figure 3 It is a schematic diagram of the overall process of the distributed storage system capacity balancing triggering method provided by an embodiment of the present invention;

[0062] Figure 4 It is a structural block diagram of a distributed storage system capacity balancing triggering device provided by an embodiment of the present invention;

[0063] Figure 5 It is a schematic diagram of the hardware structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0065] Distributed storage systems are the cornerstone of cloud computing and big data services. They solve the scalability, reliability, and performance issues faced by traditional storage systems through software-defined methods. In distributed storage systems, data storage and management are core tasks, and creating a storage pool is the first step to achieve data storage. The creation of a storage pool involves distributing a large number of data objects to various physical storage devices in the storage system according to a certain algorithm. This process is usually completed by the Controlled Replication Under Scalable Hashing (CRUSH) algorithm.

[0066] The CRUSH algorithm is a pseudo-random data distribution algorithm that maps data objects to different PGs (Placement Groups) by calculating the hash values ​​of data objects, and then these PGs are distributed to various OSDs (Object Storage Devices). The design goal of the CRUSH algorithm is to achieve uniform distribution of data while ensuring data reliability. However, due to the pseudo-randomness of the CRUSH algorithm, the actual PG distribution may not be completely balanced, which requires the introduction of a capacity balancing function to further optimize data distribution.

[0067] The core goal of the capacity balancing function is to ensure that each disk on each node can achieve a relatively even data distribution. This balance not only helps to improve the utilization of the storage system, but also can improve the read and write performance of the system to a certain extent. In many scenarios such as high-performance computing, data backup and big data analysis, the capacity balancing function plays a vital role.

[0068] However, in the related technology, capacity balancing is triggered by timing, which results in the cluster being in an unbalanced state for a long time due to a time interval that is too long, or the PG status is frequently evaluated due to a time interval that is too short, increasing invalid operations. When the capacity balancing operation is triggered by software, the capacity balancing script needs to be called, which increases the code maintenance cost.

[0069] To solve the above problems, an embodiment of the present invention provides an embodiment of capacity balancing triggering of a distributed storage system. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system (computer device) including a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0070] In this embodiment, a distributed storage system capacity balancing triggering method is provided, which can be used for the above-mentioned computer device. Figure 1 The present invention is a flowchart of a distributed storage system capacity balancing triggering method according to an embodiment of the present invention.

[0071] Before introducing the steps of this method, the following are explained first:

[0072] The application scenario of this application is to use the mgr (Manager daemon) plug-in subscription to trigger capacity balancing. First, by subscribing to the changes in the object storage device data set (OSDMAP) of the cluster, when changes in OSDMAP are detected, the plug-in process can be called up in time to trigger the capacity balancing operation. The details are as follows:

[0073] 1. Enable the mgr service and start the capacity balancing plug-in.

[0074] 2. Initialization parameters.

[0075] The attribute value balancer_flag=false of the initialization attribute parameter is used to mark whether capacity balancing is required; wherein false is used to indicate that the capacity balancing operation is not currently performed.

[0076] Initialize OSDMAP and obtain the latest OSDMAP from the cluster.

[0077] Specifically, according to the latest OSDMAP, count the current number of storage pools, the number of OSDs in each pool, and the total number of OSD states (for example, in (indicates that the OSD is inside the cluster) / out (indicates that the OSD has been marked as unavailable) / down (indicates that the OSD daemon has stopped running) / up (indicates that the OSD daemon is running)).

[0078] 3. Subscribe OSDMAP to the mgr daemon. When OSDMAP changes, the mgr daemon notifies the capacity balancing plug-in.

[0079] Then, the capacity balancing plug-in in the mgr daemon process executes the following method steps, see Figure 1 As shown, the method process comprises:

[0080] Step S101, obtaining a notification message sent by the process manager daemon process.

[0081] The notification message is a notification message sent by the process manager daemon to the capacity balancing plug-in when the process manager daemon detects that the object storage device data set has changed.

[0082] In a specific example, when the number of storage pools in the object storage device data set changes, the process manager daemon obtains the change in the number of storage pools and sends a notification message to the capacity balancing plug-in that the storage pool information has changed, and begins to determine whether to trigger a capacity balancing operation.

[0083] Step S102: Obtain the latest object storage device data set according to the notification message.

[0084] The latest object storage device data set includes the number of storage pools, the number of object storage devices in each storage pool, and the number of object storage devices in each state.

[0085] In an optional example, when the capacity balancing plug-in receives a notification message sent by the process manager daemon, it first confirms whether the notification message is a notification message generated by a change in a data set of the object storage device.

[0086] After confirming that the notification message is a notification message generated by a change in the object storage device data set, obtain the latest object storage device data set, that is, obtain the number of current storage pools, the number of object storage devices in each storage pool, and the number of each state of the object storage device, so as to compare them with the number of storage pools, the number of object storage devices in each storage pool, and the number of each state of the object storage device obtained last time.

[0087] Step S103: determining the current storage device failure ratio according to the number of object storage device failures in the latest object storage device data set.

[0088] Specifically, based on the latest object storage device data set, the ratio of the number of storage device failures in the latest object storage device data set to the current failure domain is calculated to obtain the current storage device failure ratio.

[0089] Step S104, based on the current storage device failure ratio and / or based on the comparison result between the latest object storage device data set and the object storage device data set obtained last time, determine whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0090] Specifically, if the current storage device failure ratio changes, for example, exceeds a preset ratio threshold, or the object storage device data set is inconsistent with the object storage device data set obtained last time, it means that the cluster state has changed. Therefore, it is necessary to perform a capacity balancing operation in this case. Then, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed from false in the initial state to a first attribute value. In an optional example, the first attribute value is, for example, true. Of course, the attribute value in the initial state does not have to be false as introduced above, for example, it can also be set to 0. Correspondingly, the first attribute value is, for example, 1.

[0091] The distributed storage system capacity balancing triggering method provided in this embodiment, when the process manager daemon detects that the object storage device data set has changed, it sends the object storage device data set change notification to the capacity balancing plug-in, monitors the object storage device data set, responds to the change of the cluster state in real time, and triggers the capacity balancing operation in time, avoiding the delay problem of fixed scene triggering and timed triggering. Then, the latest object storage device data set is obtained to facilitate the later judgment of whether to trigger capacity balancing, and the current storage device failure ratio is determined based on the latest object storage device data set. If the current storage device failure ratio reaches the preset threshold, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed, which optimizes the conditions for triggering the execution of the capacity balancing operation, avoids invalid balancing caused by timed capacity balancing, and avoids the failure of the function caused by manual input of commands. Moreover, the latest object storage device data set is compared with the object storage device data set obtained last time. If the data of the two are inconsistent, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed. Through the above judgment conditions, automatic triggering of capacity balancing is achieved without relying on management software, thus reducing system maintenance costs.

[0092] On the basis of the foregoing embodiment, according to the current storage device failure ratio, determining whether to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to the first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, may include the following steps:

[0093] Step a1, determining whether the current storage device failure ratio exceeds a preset threshold.

[0094] Step a2: if the current storage device failure ratio exceeds a preset threshold, determine to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to the first attribute value.

[0095] The current storage device failure ratio is the ratio of the number of storage device failures in the object storage device data set to the current failure domain.

[0096] Specifically, a preset threshold is set based on the cluster environment, and it is checked whether the proportion of the number of storage device failures in the latest object storage device data set in the current fault domain has reached the preset threshold; if the proportion of the number of storage device failures in the latest object storage device data set in the current fault domain has reached the preset threshold, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value.

[0097] In a specific example, for example, the preset threshold is set to 10%, the number of failures in the latest object storage device data set is 5, and the number of object storage devices in the current fault domain is 10. Then, the proportion of the number of storage device failures in the latest object storage device data set in the current fault domain (20%) exceeds the preset threshold (10%). Therefore, the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation needs to be changed to the first attribute value True.

[0098] On the basis of the foregoing embodiment, according to the comparison result between the latest object storage device data set and the object storage device data set obtained last time, determining whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, may include the following steps:

[0099] Step b1: compare the number of storage pools in the latest object storage device data set with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result.

[0100] Step b2: if the first comparison result is that the number of storage pools in the latest object storage device data set is not equal to the number of storage pools in the object storage device data set obtained last time, determine to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value.

[0101] Specifically, the number of storage pools in the latest object storage device data set is compared with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result.

[0102] If the number of storage pools in the latest object storage device data set in the first comparison result is not equal to the number of storage pools in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value.

[0103] In a specific example, for example, if the number of storage pools in the latest object storage device data set is 90 and the number of storage pools in the last acquired object storage device data set is 100, the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation is changed to the first attribute value True.

[0104] On the basis of the above-mentioned embodiment, according to the comparison result between the latest object storage device data set and the object storage device data set obtained last time, determining whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, may further include the following steps:

[0105] Step c1, comparing the number of object storage devices in each storage pool in the latest object storage device data set with the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, to obtain a second comparison result.

[0106] Step c2: If the second comparison result is that the number of object storage devices in each storage pool in the latest object storage device data set is not equal to the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, determine to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value.

[0107] In a specific example, for example, the number of object storage devices in each storage pool in the latest object storage device data set is 50, and the number of object storage devices in each storage pool recorded in the previously acquired object storage device data set is 55, then the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation is changed to the first attribute value True.

[0108] Specifically, in the above method steps, the latest object storage device data set is compared with the object storage device data set obtained last time. If the data of the two are inconsistent, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed. Through the above judgment conditions, automatic triggering of capacity balancing is realized without relying on management software, which reduces the system maintenance cost. The current storage device failure ratio is determined based on the latest object storage device data set. If the current storage device failure ratio reaches a preset threshold, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed, which optimizes the conditions for triggering the execution of the capacity balancing operation, avoiding both invalid balancing due to timed capacity balancing and failure of the function due to manual input of commands.

[0109] Further, after the capacity balancing operation is completed, the method further includes:

[0110] The attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

[0111] Specifically, after the capacity balancing operation is performed, the second attribute value of the capacity balancing operation, that is, balancer_flag is modified to false.

[0112] Specifically, after the capacity balancing operation is completed, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation, so as to avoid repeated execution of the capacity balancing operation, which may cause a system dead loop and damage system data.

[0113] Further optionally, before the capacity balancing execution starts, the method further includes:

[0114] An attribute value of the attribute parameter is determined, and if the attribute value of the attribute parameter is a first attribute value, the attribute value of the attribute parameter is modified to a second attribute value for stopping the capacity balancing operation.

[0115] Specifically, before each capacity balancing execution begins, the attribute value of the attribute parameter needs to be initialized to the second attribute value to avoid useless capacity balancing operations and occupation of system resources after the attribute value of the attribute parameter is modified to the first attribute value due to other reasons.

[0116] In an optional example, the method may also include the following method steps, see for details Figure 2 As shown, including:

[0117] Step S201 : acquiring capacity balancing measurement parameters other than the current storage device failure ratio and / or the comparison result between the latest object storage device data set and the object storage device data set acquired last time.

[0118] Step S202, based on the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, determines whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0119] Specifically, in a specific example, the capacity balancing measurement parameters include utilization of the object storage device, data access speed, network bandwidth information between the object storage devices, and storage pool capacity information.

[0120] The method of determining, according to the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes:

[0121] When the utilization of one or more object storage devices is overloaded or in an idle state, determine to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation;

[0122] and / or, when the data access speed is lower than a preset speed threshold, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0123] and / or, when it is determined according to the network bandwidth information that there is network congestion, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0124] And / or, when it is determined based on the storage pool capacity information that the difference between the capacity utilization rates of all storage pools is greater than a preset capacity utilization difference threshold, it is determined to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value used to indicate performing the capacity balancing operation.

[0125] In addition, after the capacity balancing operation is performed, the method may further include:

[0126] Periodically monitor the results of capacity balancing operations;

[0127] When the result does not meet the preset requirements, the capacity balancing operation strategy is adjusted, where the capacity balancing strategy is a strategy for executing the capacity balancing operation.

[0128] The specific implementation process of the above method steps, for example, includes:

[0129] 1. Collect data

[0130] First, the following data needs to be collected:

[0131] 1) Object storage device utilization, such as the current load of each object storage device, including CPU usage, disk I / O, network bandwidth usage, etc.

[0132] 2) Data access speed: the read and write speed and latency of each storage device.

[0133] 3) Network bandwidth information between object storage devices: bandwidth and latency of network connections between storage devices.

[0134] 4) Storage pool capacity information: total capacity and used capacity of each storage pool.

[0135] 2. Define the balance target

[0136] Based on the collected data, the following capacity balancing objectives are defined:

[0137] 1) Utilization of object storage devices: Avoid overloading or idling any device and keep the utilization of all devices within a reasonable range.

[0138] 2) Optimize data access speed: Improve data access speed and reduce latency, such as increasing data access speed to a level higher than or equal to a preset speed threshold.

[0139] 3) Maximize bandwidth utilization: Ensure that network bandwidth is fully utilized and avoid bandwidth bottlenecks caused by network congestion and other situations.

[0140] 4) Maintaining storage pool capacity balance: Ensure that the capacity utilization of each storage pool is close to avoid overloading of one storage pool while others are idle. That is, the difference between the capacity utilization of all storage pools is less than or equal to the preset capacity utilization difference threshold.

[0141] 3. When the capacity balancing operation is triggered, the balancing strategy is executed, for example, including the following strategies:

[0142] 1) Balance based on equipment utilization

[0143] When it is detected that the utilization of a device is too high, data is automatically migrated to a device with lower utilization.

[0144] or,

[0145] Before data is written, data distribution is predicted based on the current utilization of the device to optimize data placement.

[0146] 2) Balance based on data access speed

[0147] Continuously monitor data access speed to identify performance bottlenecks. Use cache for frequently accessed data to reduce access to slow devices.

[0148] 3) Balance based on bandwidth between devices

[0149] Monitor bandwidth usage between devices to ensure proper bandwidth allocation and avoid network congestion.

[0150] Distribute data across multiple storage devices to prevent a single device or network connection from becoming a bottleneck.

[0151] 4) Balancing based on storage pool capacity

[0152] Automatically expand or reduce storage pool capacity based on storage pool usage.

[0153] If a storage pool is close to full, migrate data to other storage pools.

[0154] 4. Implement balancing operations

[0155] When executing, for example, you can periodically run balancing tasks, check device status and data distribution, and adjust data according to balancing goals. When the device status changes or reaches a certain threshold, the balancing operation is triggered.

[0156] Or, use machine learning or predictive algorithms to make intelligent decisions based on historical and real-time monitoring data.

[0157] 5. Monitor and adjust

[0158] Continuously monitor the effectiveness of balancing operations to ensure that the balancing strategy achieves the expected goals.

[0159] Based on monitoring data, adjust balancing strategies and parameters to adapt to the changing environment.

[0160] Through the above operation process, a more intelligent and flexible capacity balancing system can be implemented to ensure the effective use of storage resources and the overall performance of the system.

[0161] In an optional example, RNN (recurrent neural network) and its variants such as LSTM and GRU network models can be used as a capacity balancing trigger prediction model. For details, see the following:

[0162] Establish a comprehensive monitoring system to collect data on key indicators such as CPU usage, memory usage, disk I / O, network traffic, etc.

[0163] The collected data are preprocessed, including removing outliers, standardizing (unifying dimensions), and other operations.

[0164] The variables that have the greatest influence on the prediction target are selected from the preprocessed data as input features.

[0165] The model is trained based on the input features to obtain a capacity balancing trigger prediction model at the end of the training.

[0166] The capacity balancing trigger prediction model after training is used to determine whether the cluster needs to perform capacity balancing operations.

[0167] In this embodiment, a distributed storage system capacity balancing triggering method is provided, which can be used for the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 3 This is an overall flow chart of the distributed storage system capacity balancing triggering method according to an embodiment of the present invention. The method flow chart introduces a relatively specific and comprehensive example of applying the method of the above embodiment of the present application. Specifically, see as follows, including the following steps:

[0168] Step 1: Start the cluster service and enable the capacity balancing plug-in.

[0169] Step 2, initialize parameters.

[0170] Assign the attribute value (balancer_flag) of the attribute parameter to false to mark whether capacity balancing is required. Obtain the latest object storage device data set (osdmap) from the cluster. Based on the latest object storage device data set, count the number of current storage pools (pool), the number of object storage devices (osd) in each storage pool, and the total number of object storage devices in each state (in / out / down / up).

[0171] Step 3: When the object storage device data set changes, the daemon notifies the capacity balancing plug-in and calls the notify function of the plug-in.

[0172] The notify function is a judgment function, and a method similar to that mentioned in the above embodiment is used to judge whether the judgment condition of capacity balancing is triggered by calling the notify function.

[0173] Step 4: upon receiving the daemon call, determine whether the changed data is an object storage device data set.

[0174] If the determination is successful, the latest object storage device data set is obtained, and the latest object storage device data set is compared with the object storage device data set obtained last time by checking statistics.

[0175] The judgment conditions are as follows:

[0176] Check whether the number of storage pools is equal to the record. This check is used to determine whether a storage pool is created or deleted.

[0177] Check whether the number of object storage devices in each storage pool is equal to the record. This check can determine whether the storage pool has been expanded or reduced.

[0178] If the object storage device data set is updated due to the ejection of a certain object storage device, it is determined whether the amount of failed object storage device data in the fault domain where the object storage device data is located reaches 10%.

[0179] If any of the above three situations is successfully judged, the attribute value of the attribute parameter is assigned to true.

[0180] Step 5: The plug-in daemon periodically queries the attribute value of the attribute parameter.

[0181] When the attribute value of the attribute parameter is true, the capacity balancing operation is performed, and after the operation is completed, the attribute value of the attribute parameter is modified to false.

[0182] If only the attribute value of the information is false, wait for a period of time, for example, 60 seconds before querying the attribute value of the attribute parameter again.

[0183] In addition, in the above-mentioned determination process, if in step 4, when receiving the daemon process call, it is determined that the changed data is not an object storage device data set, no operation is performed, that is, pass.

[0184] Similarly, if there is no need to perform capacity balancing operations, no operation is performed, that is, pass.

[0185] The specific implementation process of the method embodiment has been described in detail in the previous text, so it will not be repeated here.

[0186] The distributed storage system capacity balancing triggering method provided in this embodiment, when the process manager daemon detects that the object storage device data set has changed, it sends the object storage device data set change notification to the capacity balancing plug-in, monitors the object storage device data set, responds to the change of the cluster state in real time, and triggers the capacity balancing operation in time, avoiding the delay problem of fixed scene triggering and timed triggering. Then, the latest object storage device data set is obtained to facilitate the later judgment of whether to trigger capacity balancing, and the current storage device failure ratio is determined based on the latest object storage device data set. If the current storage device failure ratio reaches the preset threshold, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed, which optimizes the conditions for triggering the execution of the capacity balancing operation, avoids invalid balancing caused by timed capacity balancing, and avoids the failure of the function caused by manual input of commands. Moreover, the latest object storage device data set is compared with the object storage device data set obtained last time. If the data of the two are inconsistent, the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is changed to the first attribute value used to indicate the execution of the capacity balancing operation, and the capacity balancing operation is executed. Through the above judgment conditions, automatic triggering of capacity balancing is achieved without relying on management software, thus reducing system maintenance costs.

[0187] In this embodiment, a distributed storage system capacity balancing trigger device is also provided, which is used to implement the above embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0188] This embodiment provides a distributed storage system capacity balancing trigger device, such as Figure 4 As shown, including:

[0189] The instruction sending module 401 obtains a notification message sent by the process manager daemon, where the notification message is a notification message sent by the process manager daemon to the capacity balancing plug-in when the process manager daemon detects a change in a data set of an object storage device;

[0190] The data acquisition module 402 acquires the latest object storage device data set according to the notification message;

[0191] A failure ratio acquisition module 403 determines the current storage device failure ratio according to the number of storage device failures in the latest object storage device data set;

[0192] Processing module 404 determines whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, based on the current storage device failure ratio and / or based on the comparison result between the latest object storage device data set and the object storage device data set obtained last time, so as to subsequently execute the capacity balancing operation according to the first attribute value.

[0193] In an optional example, the latest object storage device data set includes:

[0194] The current object storage device data set contains the number of storage pools, the number of object storage devices in each storage pool, and the number of object storage devices in each state.

[0195] In an optional example, the processing module 404 is further configured to determine whether the changed data is an object storage device data set when the capacity balancing plug-in receives a call from the daemon process;

[0196] The acquisition module 402 is also used to obtain the latest object storage device dataset according to the notification message if the changed data is an object storage device dataset. The latest object storage device dataset includes: the number of storage pools in the current object storage device dataset, the number of object storage devices in each storage pool, and the number of each state of the object storage device.

[0197] In an optional example, the processing module 404 is specifically used to: if the current storage device failure ratio exceeds a preset threshold, determine to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value, so as to subsequently perform the capacity balancing operation according to the first attribute value.

[0198] In an optional example, the processing module 404 is specifically used to: compare the number of storage pools in the latest object storage device data set with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result;

[0199] If the first comparison result is that the number of storage pools in the latest object storage device data set is not equal to the number of storage pools in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value.

[0200] In an optional example, the processing module 404 is specifically used to: compare the number of object storage devices in each storage pool in the latest object storage device data set with the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, to obtain a second comparison result;

[0201] If the second comparison result is that the number of object storage devices in each storage pool in the latest object storage device data set is not equal to the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to the first attribute value.

[0202] In an optional example, the processing module 404 is further used to utilize the daemon process of the capacity balancing plug-in to periodically query the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation;

[0203] When it is determined that the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation is the first attribute value, the capacity balancing operation is performed.

[0204] In an optional example, the processing module 404 is further configured to modify the attribute value of the attribute parameter to a second attribute value for stopping the execution of the capacity balancing operation after the execution of the capacity balancing operation is completed.

[0205] In an optional example, the processing module 404 is further configured to determine the attribute value of the attribute parameter before the capacity balancing operation starts, and if the attribute value of the attribute parameter is the first attribute value, modify the attribute value of the attribute parameter to the second attribute value for stopping the capacity balancing operation.

[0206] In an optional example, the acquisition module 402 is further used to acquire capacity balancing measurement parameters other than the current storage device failure ratio and / or the comparison result between the latest object storage device data set and the object storage device data set acquired last time;

[0207] The processing module 404 is also used to determine whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation based on the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, so as to subsequently execute the capacity balancing operation based on the first attribute value.

[0208] In an optional example, the capacity balancing measurement parameter includes the utilization rate of the object storage device, the data access speed, the network bandwidth information between the object storage devices, and the storage pool capacity information; the processing module 404 is specifically used to: when the utilization rate of one or more object storage devices is overloaded or in an idle state, determine to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation;

[0209] and / or, when the data access speed is lower than a preset speed threshold, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0210] and / or, when it is determined according to the network bandwidth information that there is network congestion, determining to change the attribute value of the attribute parameter corresponding to triggering the capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation;

[0211] And / or, when it is determined based on the storage pool capacity information that the difference between the capacity utilization rates of all storage pools is greater than a preset capacity utilization difference threshold, it is determined to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation.

[0212] In an optional example, the processing module 404 is further configured to periodically monitor the result of executing the capacity balancing operation after the capacity balancing operation is executed;

[0213] When the result does not meet the preset requirements, the capacity balancing operation strategy is adjusted, where the capacity balancing strategy is a strategy for executing the capacity balancing operation.

[0214] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0215] The distributed storage system capacity balancing trigger device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0216] The embodiment of the present invention also provides a computer device having the above Figure 4 A distributed storage system capacity balancing trigger device is shown.

[0217] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0218] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware integrated circuit. The hardware integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0219] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0220] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0221] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0222] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0223] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0224] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0225] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A distributed storage system capacity balancing triggering method, characterized in that: The method is executed by a capacity balancing plug-in in a process manager, and the method includes: Acquire a notification message sent by a process manager daemon, where the notification message is a notification message sent by the process manager daemon to the capacity balancing plug-in when the process manager daemon detects a change in a data set of an object storage device; According to the notification message, obtain the latest object storage device data set; Determine the current storage device failure ratio according to the number of storage device failures in the latest object storage device data set; Based on the current storage device failure ratio, and / or based on the comparison result between the latest object storage device data set and the object storage device data set obtained last time, determine whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

2. The distributed storage system capacity balancing triggering method according to claim 1, characterized in that: The latest object storage device dataset includes: The current object storage device data set contains the number of storage pools, the number of object storage devices in each storage pool, and the number of object storage devices in each state.

3. The distributed storage system capacity balancing triggering method according to claim 1, characterized in that: Before obtaining the latest object storage device data set according to the notification message, the method further includes: When the capacity balancing plug-in receives a call from the daemon process, it determines whether the changed data is an object storage device data set; If the changed data is an object storage device data set, the latest object storage device data set is obtained according to the notification message.

4. The distributed storage system capacity balancing triggering method according to claim 2, characterized in that: Determining, according to the current storage device failure ratio, whether to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes: If the current storage device failure ratio exceeds a preset threshold, it is determined to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to the first attribute value, so that the capacity balancing operation is subsequently performed according to the first attribute value.

5. The distributed storage system capacity balancing triggering method according to claim 2, characterized in that: Determining, according to a comparison result between the latest object storage device data set and the object storage device data set obtained last time, whether to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes: Compare the number of storage pools in the latest object storage device data set with the number of storage pools in the object storage device data set obtained last time to obtain a first comparison result; If the first comparison result is that the number of storage pools in the latest object storage device data set is not equal to the number of storage pools in the object storage device data set obtained last time, it is determined to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to the first attribute value.

6. The distributed storage system capacity balancing triggering method according to claim 2, characterized in that: Determining, according to a comparison result between the latest object storage device data set and the object storage device data set obtained last time, whether to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value for indicating the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes: Compare the number of object storage devices in each storage pool in the latest object storage device data set with the number of object storage devices in each storage pool recorded in the object storage device data set obtained last time, to obtain a second comparison result; If the second comparison result is that the number of object storage devices in each storage pool in the latest object storage device data set is not equal to the number of object storage devices recorded in each storage pool in the object storage device data set obtained last time, determine to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to the first attribute value.

7. The distributed storage system capacity balancing triggering method according to any one of claims 4 to 6, characterized in that: The method further comprises: Using the daemon process of the capacity balancing plug-in, periodically querying the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation; When it is determined that the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation is the first attribute value, the capacity balancing operation is performed.

8. The distributed storage system capacity balancing triggering method according to any one of claims 1 to 6, characterized in that: After the capacity balancing operation is completed, the method further includes: The attribute value of the attribute parameter is modified to a second attribute value for stopping execution of the capacity balancing operation.

9. The distributed storage system capacity balancing triggering method according to claim 8, characterized in that: Before the capacity balancing is started, the method further includes: Determining the attribute value of the attribute parameter; If the attribute value of the attribute parameter is a first attribute value, the attribute value of the attribute parameter is modified to a second attribute value for stopping execution of the capacity balancing operation.

10. The distributed storage system capacity balancing triggering method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a capacity balancing measurement parameter other than the current storage device failure ratio and / or a comparison result between the latest object storage device data set and a previously acquired object storage device data set; Based on the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, determine whether to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value.

11. The distributed storage system capacity balancing triggering method according to claim 10, characterized in that: The capacity balancing measurement parameters include utilization of the object storage device, data access speed, network bandwidth information between the object storage devices, and storage pool capacity information; The determining, according to the capacity balancing measurement parameter and the capacity balancing target corresponding to the capacity balancing measurement parameter, whether to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation, so as to subsequently execute the capacity balancing operation according to the first attribute value, includes: When the utilization rate of one or more object storage devices is overloaded or in an idle state, determine to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation; and / or, when the data access speed is lower than a preset speed threshold, determining to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation; and / or, when it is determined according to the network bandwidth information that there is network congestion, determining to change the attribute value of the attribute parameter corresponding to the triggering capacity balancing operation to a first attribute value for indicating execution of the capacity balancing operation; And / or, when it is determined based on the storage pool capacity information that the difference between the capacity utilization rates of all storage pools is greater than a preset capacity utilization rate difference threshold, it is determined to change the attribute value of the attribute parameter corresponding to the triggered capacity balancing operation to a first attribute value used to indicate the execution of the capacity balancing operation.

12. The distributed storage system capacity balancing triggering method according to claim 11, characterized in that: The method further comprises: After the capacity balancing operation is performed, periodically monitoring the result of the capacity balancing operation; When the result does not meet the preset requirement, the capacity balancing operation strategy is adjusted, wherein the capacity balancing strategy is a strategy for executing the capacity balancing operation.

13. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the distributed storage system capacity balancing triggering method according to any one of claims 1 to 12 by executing the computer instructions.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the distributed storage system capacity balancing triggering method according to any one of claims 1 to 12.

15. A computer program product, characterized in that It comprises computer instructions, and the computer instructions are used to enable a computer device to execute the distributed storage system capacity balancing triggering method according to any one of claims 1 to 12.

Images