Background IO control method and device, equipment, storage medium and program product

Abstract

The application relates to a background IO control method, device, equipment, storage medium and program product. The method comprises the following steps: acquiring a system time delay of a storage cluster; adjusting a flow control threshold value of a background IO according to a size relationship between the system time delay and a time delay threshold value range, so as to control the quantity of background IO requests; wherein the flow control threshold value is related to the cluster performance of the storage cluster; the cluster performance comprises a cluster throughput and / or an input / output per second (IOPS) of the cluster. The method can realize efficient utilization of cluster resources.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a background IO control method, apparatus, device, storage medium, and program product. Background Art

[0002] As a new storage method, object storage is widely used in various cloud services due to its advantages such as high scalability, high availability and low cost.

[0003] The current object storage backend I / O, which relies on thread control, can no longer meet current needs and cannot effectively control latency. Therefore, maximizing cluster I / O utilization and reducing latency while meeting user needs has become an urgent issue. Summary of the Invention

[0004] Based on this, it is necessary to provide a background IO control method, device, equipment, storage medium and program product to address the above technical problems, which can achieve efficient utilization of cluster resources.

[0005] In a first aspect, the present application provides a background IO control method, the method comprising:

[0006] Get the system latency of the storage cluster;

[0007] Based on the relationship between system latency and the latency threshold range, the background IO flow control threshold is adjusted to control the background IO request volume. The flow control threshold is related to the cluster performance of the storage cluster. Cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0008] In one embodiment, adjusting the flow control threshold of the background IO based on the relationship between the system delay and the delay threshold range includes:

[0009] Based on the relationship between system latency and latency threshold range, as well as the peak and valley data of the storage cluster's foreground input and output IO, the background IO flow control threshold is adjusted to control the background IO request volume. The peak and valley data is determined based on the foreground IO request volume within a historical period.

[0010] In one embodiment, based on the relationship between system latency and latency threshold range, as well as peak and valley data of foreground input and output IO of the storage cluster, the flow control threshold of the background IO is adjusted, including:

[0011] Based on the peak and valley data of the storage cluster's front-end IO, predict the peak and valley conditions of the storage cluster in the next period; based on the size relationship between the system latency and the latency threshold range, as well as the peak and valley conditions, adjust the flow control threshold of the background IO.

[0012] In one embodiment, the flow control threshold of the background IO is adjusted based on the relationship between the system delay and the delay threshold range, as well as the peak and valley conditions, including:

[0013] When the system latency is greater than the upper latency threshold, the background IO flow control threshold is gradually lowered based on peak and valley conditions until the storage cluster system latency is within the latency threshold. When the system latency is less than the lower latency threshold, the background IO flow control threshold is gradually raised based on peak and valley conditions until the storage cluster system latency is within the latency threshold.

[0014] In one embodiment, the method includes:

[0015] Determine the node performance of a single node in the storage cluster; determine the cluster performance of the storage cluster based on the node performance, the erasure code conversion ratio of the storage cluster, and the number of nodes in the storage cluster.

[0016] In one embodiment, determining node performance of a single node in a storage cluster includes:

[0017] Determine the node performance of a single node based on the single disk performance and the number of disks on the single node in the storage cluster.

[0018] In a second aspect, the present application further provides a background IO control device, which includes:

[0019] The acquisition module is used to obtain the system latency of the storage cluster;

[0020] The control module is used to adjust the background IO flow control threshold based on the relationship between the system latency and the latency threshold range to control the background IO request volume. The flow control threshold is related to the cluster performance of the storage cluster. Cluster performance includes cluster throughput and / or cluster input and output per second (IOPS).

[0021] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0022] Get the system latency of the storage cluster;

[0023] Based on the relationship between system latency and the latency threshold range, the background IO flow control threshold is adjusted to control the background IO request volume. The flow control threshold is related to the cluster performance of the storage cluster. Cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0024] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0025] Get the system latency of the storage cluster;

[0026] Based on the relationship between system latency and the latency threshold range, the background IO flow control threshold is adjusted to control the background IO request volume. The flow control threshold is related to the cluster performance of the storage cluster. Cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0027] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0028] Get the system latency of the storage cluster;

[0029] Based on the relationship between system latency and the latency threshold range, the background IO flow control threshold is adjusted to control the background IO request volume. The flow control threshold is related to the cluster performance of the storage cluster. Cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0030] The above-mentioned background IO control method, device, equipment, storage medium and program product provide a basis for judging the cluster performance of the storage cluster by obtaining the system latency of the storage cluster; further, based on the size relationship between the system latency and the latency threshold range, the cluster performance of the storage cluster can be determined; further, based on the size relationship between the system latency and the latency threshold range, the flow control threshold of the background IO can be adjusted, ultimately achieving control of the background IO request volume. This solution provides a basis for determining the cluster performance of the storage cluster by introducing the latency threshold range; further, by determining the size relationship between the system latency and the latency threshold range, it lays the foundation for determining the adjustment direction of the storage cluster; finally, by introducing the flow control threshold, dynamic adjustment of the flow control threshold of the background IO is achieved, ultimately achieving efficient utilization of cluster resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 This is an architectural diagram of a background IO control method provided in an embodiment of the present application;

[0033] Figure 2 This is a functional structure diagram of a StateCenter module provided in an embodiment of the present application;

[0034] Figure 3 A process environment diagram of a background IO control method provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of a process for determining cluster performance provided in an embodiment of the present application;

[0036] Figure 5 This is a flow chart of another background IO control method provided in an embodiment of the present application;

[0037] Figure 6 This is a structural block diagram of a background IO control device provided in an embodiment of the present application;

[0038] Figure 7 This is a structural block diagram of another background IO control device provided in an embodiment of the present application;

[0039] Figure 8 This is a diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0041] As a new storage method, object storage is widely used in various cloud services due to its advantages such as high scalability, high availability and low cost. Figure 1 As shown, the background IO control method in the embodiment of the present application can be implemented by the StateCenter module and the flow control module; wherein the StateCenter module includes functions such as delay acquisition, throughput control, IOPS control, flow prediction, cluster capacity prediction, control strategy and fuse, as shown in FIG. Figure 2 As shown; the flow control module is responsible for the background IO requests of all gateways.

[0042] Based on this, in an exemplary embodiment, as Figure 3 As shown, a background IO control method is provided, which is described by taking the application of the method to the StateCenter module as an example. The method may include the following steps:

[0043] S301: Obtain the system latency of the storage cluster.

[0044] Among them, the storage cluster can be composed of several general-purpose storage devices, which are formed into a whole through cluster technology to store data; system latency can represent the time consumed from the initiation to the completion of an IO request, which can be affected by the storage medium, network conditions, and load conditions.

[0045] It's important to note that system latency in a storage cluster can occur across all data exchange nodes within the cluster's hierarchy. Each data exchange node performs data encapsulation, decapsulation, data rearrangement, queuing, access control, resource application / release, error correction, and retransmission, all of which can contribute to latency.

[0046] For example, a performance testing tool or a network testing tool can be used to collect the system latency of the storage cluster multiple times, and the results of the multiple collections can be averaged to improve the accuracy of the data; thus, the system latency of the storage cluster can be directly obtained.

[0047] S302: Adjust the background IO flow control threshold according to the size relationship between the system delay and the delay threshold range to control the background IO request amount.

[0048] Among them, the latency threshold range includes an upper latency threshold and a lower latency threshold; background IO can refer to input and output operations in the computer system that do not directly interact with the user but are performed in the background, such as disk IO; correspondingly, foreground IO can refer to input and output operations that directly interact with the user interface, such as keyboard input, screen display, etc.; the flow control threshold can represent the critical value in the flow control mechanism, which can be used to limit or regulate network traffic. For example, if the flow of background IO exceeds the flow control threshold of the background IO, the flow of the background IO will be controlled, such as reducing the number of background IO requests; the flow control threshold is related to the cluster performance of the storage cluster; cluster performance can include cluster throughput and / or the cluster's input and output volume per second IOPS.

[0049] It's important to note that system latency is highly correlated with IOPS. Generally speaking, as IOPS increases, latency also increases. When IOPS reaches a certain limit, reaching the storage cluster's processing capacity, IOPS stops increasing, only latency increases. Therefore, based on system latency, you can adjust the flow control threshold associated with cluster IOPS.

[0050] Optionally, the adjustment range of the flow control threshold of the background IO can be determined based on the historical flow control threshold and the size relationship between the system delay and the delay threshold range. The embodiment of the present application does not specifically limit the adjustment strategy.

[0051] For example, the processing capacity of the storage cluster can be determined based on the size relationship between the system delay and the delay threshold range; further, the flow control threshold of the background IO can be adjusted according to the processing capacity of the storage cluster. For example, when the processing capacity of the storage cluster is insufficient, the flow control threshold of the background IO can be appropriately lowered to reduce the request volume of the background IO, and more processing capacity of the storage cluster can be allocated to the front-end IO, giving users a better user experience.

[0052] The above-mentioned background IO control method provides a basis for determining the cluster performance of the storage cluster by obtaining the system latency of the storage cluster. Furthermore, based on the size relationship between the system latency and the latency threshold range, the cluster performance of the storage cluster can be determined. Furthermore, the flow control threshold of the background IO can be adjusted according to the size relationship between the system latency and the latency threshold range, ultimately achieving control of the background IO request volume. This solution provides a basis for determining the cluster performance of the storage cluster by introducing the latency threshold range. Furthermore, by determining the size relationship between the system latency and the latency threshold range, it lays the foundation for determining the adjustment direction of the storage cluster. Finally, by introducing the flow control threshold, dynamic adjustment of the flow control threshold of the background IO is achieved, ultimately achieving efficient utilization of cluster resources.

[0053] Based on the above embodiment, the present embodiment provides a detailed explanation of the above embodiment S302. Specifically, the process of adjusting the flow control threshold in the present embodiment includes adjusting the flow control threshold of the background IO based on the relationship between the system latency and the latency threshold range, as well as the peak and valley data of the front-end input and output IO of the storage cluster, to achieve control of the back-end IO request volume.

[0054] Among them, foreground IO can represent input and output operations that directly interact with the user interface, such as keyboard input, screen display, etc.; peak and valley data can include peak data and valley data, which are determined based on the amount of foreground IO requests in a historical period.

[0055] Optionally, the foreground IO data of the storage cluster in the historical period can be determined by collecting the cluster throughput and / or cluster IOPS of the storage cluster in the historical period; further, the peak and valley data of the foreground IO of the storage cluster can be determined based on the foreground IO data of the storage cluster in the historical period. For example, if the historical period is the previous 7 days, the peak data can be determined every 10 minutes in the historical period (the interval time can be determined according to actual conditions. Generally, the smaller the time interval, the more accurate the peak and valley data of the foreground IO of the storage cluster). For example, during the period from 13:00 to 13:10 on each day of the previous 7 days, the peaks are 1GB, 1.1GB, 3GB, 2GB, 1.2GB, 1GB, and 1.4GB respectively. Then, the peak during the historical period from 13:00 to 13:10 of the previous 7 days is 3GB, and the valley is 1GB.

[0056] One possible implementation method is to predict the peak and valley conditions of the storage cluster in the next period based on the peak and valley data of the storage cluster's foreground IO; and adjust the flow control threshold of the background IO based on the size relationship between the system delay and the delay threshold range, as well as the peak and valley conditions.

[0057] Exemplarily, based on the peak and valley data of the foreground IO of the storage cluster, the trend of the peak and valley data in the historical period can be analyzed through time series analysis technology, and features related to the peaks and valleys, such as peak height, valley depth, time interval between peaks and valleys, etc., can be extracted, and the peak and valley conditions of the storage cluster in the next period can be determined through a prediction model (such as a long short-term memory network model); further, based on the peak and valley conditions of the storage cluster in the next period, the foreground IO request volume corresponding to the peak of the storage cluster in the next period, and the foreground IO request volume corresponding to the valley of the storage cluster in the next period can be determined; further, before the peak and / or valley arrives in the next period, and according to the size relationship between the system delay and the delay threshold range, the flow control threshold of the background IO can be adjusted in advance to cope with the predicted peak and / or valley.

[0058] Optionally, when the system latency is greater than the upper latency threshold, the flow control threshold of the background IO is gradually lowered according to the peak and valley conditions until the system latency of the storage cluster after the lowering process is within the latency threshold range; when the system latency is less than the lower latency threshold, the flow control threshold of the background IO is gradually increased according to the peak and valley conditions until the system latency of the storage cluster after the increased processing is within the latency threshold range.

[0059] For example, if the system latency of the current storage cluster is greater than the upper latency threshold (such as exceeding 5% of the upper latency threshold), it indicates that the processing speed of the current storage cluster is slow, which may cause a bad access experience for front-end users. In this case, more processing capacity of the storage cluster can be allocated to the front-end IO, that is, by reducing the flow control threshold of the background IO, the processing capacity allocated to the background IO is reduced; wherein, the process of adjusting the flow control threshold of the background IO is a gradual adjustment, which can be combined with the peak and valley conditions. If the peak is approaching, the pace of adjusting the flow control threshold can be appropriately increased. If the trough is approaching, the flow control threshold can be adjusted smoothly to ensure the stability of the storage cluster.

[0060] Accordingly, if the current storage cluster system latency is less than the lower latency threshold (e.g., less than 5% of the lower latency threshold), this indicates that the storage cluster has a high level of idle I / O. You can appropriately increase backend I / O by raising the flow control threshold for backend I / O, allocating more processing power to backend I / O and fully utilizing the cluster's I / O. The flow control threshold for backend I / O can be adjusted based on peak and valley conditions. If a peak is approaching, the flow control threshold can be slowly increased to ensure that the processing speed of frontend I / O is not affected. If a valley is approaching, the flow control threshold can be adjusted more quickly to maximize the storage cluster's performance.

[0061] In the embodiment of the present application, by introducing the delay threshold range, a basis is provided for determining the cluster performance of the storage cluster. Furthermore, by adjusting the flow control threshold of the background IO, a foundation is laid for ensuring user experience and fully utilizing the storage cluster.

[0062] Based on the above embodiments, the embodiments of the present application relate to a process of determining the cluster performance of a storage cluster, such as Figure 4 As shown, the specific steps include:

[0063] S401: Determine the node performance of a single node in a storage cluster.

[0064] It should be noted that the storage cluster includes several general storage devices, each of which is a single node in the storage cluster.

[0065] Node performance may include node throughput and node IOPS.

[0066] One achievable approach is to determine the node performance of a single node based on the performance of a single disk and the number of disks of the single node in the storage cluster.

[0067] The performance of a single disk can include single-disk throughput and single-disk IOPS.

[0068] For example, if a single node has 36 disk slots, the single disk throughput of a single disk is 30MB, and the single disk IOPS is 30IOPS, then the node throughput of the single node can be 30MB×36=1080MB, and the node IOPS can be 30IOPS×36=1080IOPS.

[0069] Another possible implementation method is to use professional benchmarking tools to perform performance tests on individual nodes in the storage cluster, such as processor performance tests and disk performance tests.

[0070] S402 : Determine the cluster performance of the storage cluster based on the node performance, the erasure code conversion ratio of the storage cluster, and the number of nodes in the storage cluster.

[0071] Among them, the erasure code conversion ratio can express the proportional relationship between the original data volume and the actual storage data volume in the storage cluster (including the original data blocks and redundant check blocks), which can reflect the utilization efficiency of the storage space.

[0072] In the above example, there are 11 storage devices in the storage cluster, and the EC is 8+3. The erasure code conversion ratio of this storage cluster is 8 / (8+3), or 8 / 11. Consequently, the cluster throughput of this storage cluster can be 11×1080MB×8 / 11=8640MB, and the cluster IOPS can be 11×1080IOPS×8 / 11=8640IOPS.

[0073] In the embodiment of the present application, by introducing the erasure code conversion ratio of the storage cluster, a method is provided for accurately determining the cluster performance of the storage cluster.

[0074] Based on the above embodiment, this embodiment provides an optional example of a background IO control method. Figure 5 As shown, the specific implementation process is as follows:

[0075] S501: Obtain the system latency of the storage cluster.

[0076] S502 : Predicting the peak and valley conditions of the storage cluster in the next period based on the peak and valley data of the foreground IO of the storage cluster.

[0077] Among them, the peak and valley data are determined based on the front-end IO request volume in the historical period.

[0078] S503, determine whether the system delay is greater than the upper delay threshold, if so, execute S504; if not, execute S505.

[0079] S504: gradually lowering the flow control threshold of the background IO according to the peak and valley conditions until the system latency of the storage cluster after the lowering process is within the latency threshold range.

[0080] The flow control threshold is related to the cluster performance of the storage cluster; the cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0081] S505 , gradually increasing the flow control threshold of the background IO according to the peak and valley conditions until the system latency of the storage cluster after the increase is within the latency threshold range, so as to control the amount of background IO requests.

[0082] Optionally, determine the node performance of a single node in the storage cluster; and determine the cluster performance of the storage cluster based on the node performance, an erasure code conversion ratio of the storage cluster, and the number of nodes in the storage cluster.

[0083] The node performance of a single node may be determined based on the single disk performance and the number of disks of the single node in the storage cluster.

[0084] The specific process of the above S501-S505 can refer to the description of the above method embodiment. The implementation principle and technical effects are similar and will not be repeated here.

[0085] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0086] Based on the same inventive concept, embodiments of the present application also provide a background IO control device for implementing the background IO control method described above. The implementation solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more background IO control device embodiments provided below can be found in the above-mentioned limitations of the background IO control method and will not be repeated here.

[0087] In an exemplary embodiment, Figure 6 As shown, a background IO control device 1 is provided, comprising: an acquisition module 10 and a control module 20, wherein:

[0088] The acquisition module 10 is configured to acquire the system latency of the storage cluster.

[0089] Control module 20 is used to adjust the background IO flow control threshold based on the size relationship between the system latency and the latency threshold range to control the background IO request volume; wherein the flow control threshold is related to the cluster performance of the storage cluster; cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0090] In one embodiment, the control module 20 is specifically configured to:

[0091] Based on the relationship between system latency and latency threshold range, as well as the peak and valley data of the storage cluster's foreground input and output IO, the background IO flow control threshold is adjusted to control the background IO request volume. The peak and valley data is determined based on the foreground IO request volume within a historical period.

[0092] In one embodiment, Figure 7 As shown, the control module 20 specifically further includes:

[0093] The prediction unit 21 is configured to predict the peak and valley conditions of the storage cluster in the next period based on the peak and valley data of the foreground IO of the storage cluster.

[0094] The adjustment unit 22 is used to adjust the flow control threshold of the background IO according to the size relationship between the system delay and the delay threshold range, as well as the peak and valley conditions.

[0095] In one embodiment, the adjustment unit 22 is further configured to:

[0096] When the system latency is greater than the upper latency threshold, the background IO flow control threshold is gradually lowered based on peak and valley conditions until the storage cluster system latency is within the latency threshold. When the system latency is less than the lower latency threshold, the background IO flow control threshold is gradually raised based on peak and valley conditions until the storage cluster system latency is within the latency threshold.

[0097] In one embodiment, the background IO control device 1 specifically further includes:

[0098] The performance determination module is used to determine the node performance of a single node in the storage cluster; the cluster performance of the storage cluster is determined based on the node performance, the erasure code conversion ratio of the storage cluster, and the number of nodes in the storage cluster.

[0099] In one embodiment, the performance determination module is further configured to:

[0100] Determine the node performance of a single node based on the single disk performance and the number of disks on the single node in the storage cluster.

[0101] Each module in the background IO control device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0102] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store cluster data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a background IO control method is implemented.

[0103] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0104] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0105] Get the system latency of the storage cluster;

[0106] Based on the relationship between system latency and the latency threshold range, the background IO flow control threshold is adjusted to control the background IO request volume. The flow control threshold is related to the cluster performance of the storage cluster. Cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0107] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0108] Based on the relationship between system latency and latency threshold range, as well as the peak and valley data of the storage cluster's foreground input and output IO, the background IO flow control threshold is adjusted to control the background IO request volume. The peak and valley data is determined based on the foreground IO request volume within a historical period.

[0109] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0110] Based on the peak and valley data of the storage cluster's front-end IO, predict the peak and valley conditions of the storage cluster in the next period; based on the size relationship between the system latency and the latency threshold range, as well as the peak and valley conditions, adjust the flow control threshold of the background IO.

[0111] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0112] When the system latency is greater than the upper latency threshold, the background IO flow control threshold is gradually lowered based on peak and valley conditions until the storage cluster system latency is within the latency threshold. When the system latency is less than the lower latency threshold, the background IO flow control threshold is gradually raised based on peak and valley conditions until the storage cluster system latency is within the latency threshold.

[0113] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0114] Determine the node performance of a single node in the storage cluster; determine the cluster performance of the storage cluster based on the node performance, the erasure code conversion ratio of the storage cluster, and the number of nodes in the storage cluster.

[0115] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0116] Determine the node performance of a single node based on the single disk performance and the number of disks on the single node in the storage cluster.

[0117] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0118] Get the system latency of the storage cluster;

[0119] Based on the relationship between system latency and the latency threshold range, the background IO flow control threshold is adjusted to control the background IO request volume. The flow control threshold is related to the cluster performance of the storage cluster. Cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0120] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0121] Based on the relationship between system latency and latency threshold range, as well as the peak and valley data of the storage cluster's foreground input and output IO, the background IO flow control threshold is adjusted to control the background IO request volume. The peak and valley data is determined based on the foreground IO request volume within a historical period.

[0122] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0123] Based on the peak and valley data of the storage cluster's front-end IO, predict the peak and valley conditions of the storage cluster in the next period; based on the size relationship between the system latency and the latency threshold range, as well as the peak and valley conditions, adjust the flow control threshold of the background IO.

[0124] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0125] When the system latency is greater than the upper latency threshold, the background IO flow control threshold is gradually lowered based on peak and valley conditions until the storage cluster system latency is within the latency threshold. When the system latency is less than the lower latency threshold, the background IO flow control threshold is gradually raised based on peak and valley conditions until the storage cluster system latency is within the latency threshold.

[0126] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0127] Determine the node performance of a single node in the storage cluster; determine the cluster performance of the storage cluster based on the node performance, the erasure code conversion ratio of the storage cluster, and the number of nodes in the storage cluster.

[0128] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0129] Determine the node performance of a single node based on the single disk performance and the number of disks on the single node in the storage cluster.

[0130] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0131] Get the system latency of the storage cluster;

[0132] Based on the relationship between system latency and the latency threshold range, the background IO flow control threshold is adjusted to control the background IO request volume. The flow control threshold is related to the cluster performance of the storage cluster. Cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

[0133] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0134] Based on the relationship between system latency and latency threshold range, as well as the peak and valley data of the storage cluster's foreground input and output IO, the background IO flow control threshold is adjusted to control the background IO request volume. The peak and valley data is determined based on the foreground IO request volume within a historical period.

[0135] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0136] Based on the peak and valley data of the storage cluster's front-end IO, predict the peak and valley conditions of the storage cluster in the next period; based on the size relationship between the system latency and the latency threshold range, as well as the peak and valley conditions, adjust the flow control threshold of the background IO.

[0137] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0138] When the system latency is greater than the upper latency threshold, the background IO flow control threshold is gradually lowered based on peak and valley conditions until the storage cluster system latency is within the latency threshold. When the system latency is less than the lower latency threshold, the background IO flow control threshold is gradually raised based on peak and valley conditions until the storage cluster system latency is within the latency threshold.

[0139] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0140] Determine the node performance of a single node in the storage cluster; determine the cluster performance of the storage cluster based on the node performance, the erasure code conversion ratio of the storage cluster, and the number of nodes in the storage cluster.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0142] Determine the node performance of a single node based on the single disk performance and the number of disks on the single node in the storage cluster.

[0143] It should be noted that the information (including but not limited to storage cluster information, etc.) and data (including but not limited to data used for analysis, storage, display, etc.) involved in this application are all information and data fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0144] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0145] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A background IO control method, characterized in that: The method comprises: Get the system latency of the storage cluster; Based on the size relationship between the system latency and the latency threshold range, as well as the peak and valley data of the foreground input and output IO of the storage cluster, the flow control threshold of the background IO is adjusted to control the amount of background IO requests; wherein, the peak and valley data is determined based on the amount of foreground IO requests in a historical period; the flow control threshold is related to the cluster performance of the storage cluster; the cluster performance includes cluster throughput and / or cluster input and output volume per second (IOPS).

2. The method according to claim 1, characterized in that The adjusting of the background IO flow control threshold according to the size relationship between the system delay and the delay threshold range, and the peak and valley data of the front-end input and output IO of the storage cluster, includes: Predicting the peak and valley conditions of the storage cluster in the next period based on the peak and valley data of the foreground IO of the storage cluster; The flow control threshold of the background IO is adjusted according to the size relationship between the system delay and the delay threshold range, as well as the peak and valley conditions.

3. The method according to claim 2, characterized in that The delay threshold range includes an upper delay threshold and a lower delay threshold; The adjusting of the background IO flow control threshold according to the relationship between the system delay and the delay threshold range, and the peak and valley conditions, includes: When the system latency is greater than the upper latency threshold, gradually lower the background IO flow control threshold according to the peak and valley conditions until the system latency of the storage cluster after the adjustment falls within the latency threshold range; When the system latency is less than the lower latency threshold, the flow control threshold of the background IO is gradually increased according to the peak and valley conditions until the system latency of the storage cluster after the increase is within the latency threshold range.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Determining node performance of a single node in the storage cluster; The cluster performance of the storage cluster is determined according to the node performance, the erasure code conversion ratio of the storage cluster, and the number of nodes in the storage cluster.

5. The method according to claim 4, characterized in that The determining the node performance of a single node in the storage cluster includes: The node performance of the single node is determined based on the single disk performance and the number of disks of the single node in the storage cluster.

6. A background IO control device, characterized in that: The device comprises: The acquisition module is used to obtain the system latency of the storage cluster; A control module is configured to adjust a flow control threshold for background IO based on the relationship between the system latency and the latency threshold range, as well as peak and valley data of the foreground input and output (IO) of the storage cluster, to control the amount of background IO requests. The peak and valley data is determined based on the amount of foreground IO requests within a historical period. The flow control threshold is related to the cluster performance of the storage cluster, which includes cluster throughput and / or cluster input and output (IOPS) per second.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images