A method and apparatus for flattened speed control

Abstract

The embodiment of the present application provides a kind of flattening speed control method and device, can be obtained by each storage node executing flattening task and the business throughput generated by service throughput, when the flattening throughput and service throughput satisfy preset speed reduction condition, control storage node reduces the rate of execution flattening task.In the embodiment of the present application, the flattening throughput and service throughput on the storage node are obtained by management node, determine the processing service resource condition on each node, and then the rate of execution flattening task of storage node can be controlled according to service resource condition.That is, when there are more background flattening tasks, the flattening task speed can be dynamically adjusted, and the influence on normal service performance can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed block storage, in particular to a flattening speed control method and device. BACKGROUND

[0002] There can be no required data in part or all of the storage space of a storage volume in a storage node, and part of the data stored in other volumes referenced by the storage volume is required. The process of copying the data of the other volumes referenced by the storage volume to the storage space of the storage volume itself is called flattening, also known as data isolation. After flattening is completed, the storage volume itself no longer references the data of other volumes, and the data can be read from the storage space of the storage volume itself.

[0003] In related technologies, there are generally two implementation methods for flattening technology: one is to directly copy the data of other volumes referenced by a storage volume to the storage space of the storage volume when the data of the other volumes is read, and the other is to copy the data of other volumes referenced by a storage volume to the storage space of the storage volume asynchronously through a background flattening task. However, when there are many flattening tasks in the background, due to the limitation of disk resources, the disk pressure is too large, which affects the rate of the flattening task and further affects the normal business performance. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a flattening speed control method and device to dynamically adjust the flattening task rate when there are many flattening tasks in the background. The specific technical solutions are as follows:

[0005] In a first aspect of the embodiments of the present application, a flattening speed control method is provided, which is applied to a management node in a distributed block storage cluster, and the management node is used to manage storage nodes in the distributed block storage cluster. The method comprises the following steps:

[0006] Respectively acquiring flattening throughputs generated by executing flattening tasks by each storage node and business throughputs generated by executing business by the each storage node;

[0007] If the flattening throughputs and the business throughputs satisfy a speed reduction condition, then the speed of executing the flattening tasks by at least one storage node is controlled to be reduced. The reduction amplitude of the speed is negatively related to the flattening throughputs, and the reduction amplitude of the speed is positively related to the business throughputs.

[0008] In a possible embodiment, if the flattening throughputs and the business throughputs satisfy the speed reduction condition, then the speed of executing the flattening tasks by at least one storage node is controlled to be reduced, which comprises the following steps:

[0009] if the total amount of the flat throughputs in the throughput group meets a speed reduction condition, reducing the speed of the flat tasks performed by the storage nodes of the flat throughputs in the throughput group, wherein the throughput group comprises at least one of the flat throughputs.

[0010] In a possible embodiment, the throughput group comprises:

[0011] a flat throughput generated by a flat task; or,

[0012] flat throughputs generated by flat tasks performed on the same storage node; or,

[0013] flat throughputs generated by flat tasks performed on a plurality of the storage nodes.

[0014] In a possible embodiment, if the total amount of the flat throughputs in the throughput group meets a speed reduction condition, reducing the speed of the flat tasks performed by the storage nodes of the flat throughputs in the throughput group, comprises:

[0015] if the total amount of the flat throughputs generated by a flat task in the throughput group exceeds a throughput threshold of the flat task, sending an instruction to the storage node where the flat task is located to reduce the speed of the flat task; or,

[0016] if the total amount of the flat throughputs generated by flat tasks performed on the same storage node in the throughput group exceeds a throughput threshold of the flat tasks of the storage node, sending an instruction to the storage node to reduce the speed of the flat tasks; or,

[0017] if the total amount of the flat throughputs generated by flat tasks performed on a plurality of the storage nodes in the throughput group exceeds a throughput threshold of the flat tasks of the plurality of the storage nodes, sending an instruction to the plurality of the storage nodes to reduce the speed of the flat tasks.

[0018] In a possible embodiment, the controlling at least one of the storage nodes to reduce the speed of performing the flat tasks comprises:

[0019] determining a first ratio, wherein the first ratio is a ratio of the flat throughput of a volume in the storage node to the service throughput;

[0020] controlling at least one of the storage nodes to reduce the speed of performing the flat tasks according to the first ratio, wherein the reduction range of the speed is negatively related to the first ratio.

[0021] In a possible embodiment, the controlling at least one of the storage nodes to reduce the speed of performing the flat tasks according to the first ratio comprises:

[0022] if the first ratio is greater than a preset ratio threshold, controlling at least one of the storage nodes to reduce a rate of executing the flattening task by a first magnitude;

[0023] if the first ratio is not greater than the preset ratio threshold, controlling at least one of the storage nodes to reduce a rate of executing the flattening task by a second magnitude, wherein the second magnitude is greater than the first magnitude.

[0024] In a second aspect of the embodiments of the present application, a flattening speed control device is provided, which is applied to a management node in a distributed block storage cluster, and the management node is used to manage storage nodes in the distributed block storage cluster; the device comprises:

[0025] a first obtaining module, configured to respectively obtain flattening throughputs generated by the storage nodes executing flattening tasks and service throughputs generated by the storage nodes executing services;

[0026] a control module, configured to, if the flattening throughputs and the service throughputs satisfy a speed reduction condition, control at least one of the storage nodes to reduce a rate of executing the flattening task; wherein a reduction magnitude of the rate is negatively related to the flattening throughputs, and the reduction magnitude of the rate is positively related to the service throughputs.

[0027] In a possible embodiment, the control module is specifically configured to, if a total amount of the flattening throughputs in a throughput group satisfies a speed reduction condition, reduce a rate of the storage nodes executing the flattening task for the flattening throughputs in the throughput group, wherein the throughput group comprises at least one of the flattening throughputs.

[0028] The throughput group comprises:

[0029] a flattening throughput generated by one flattening task; or,

[0030] flattening throughputs generated by each of the flattening tasks on the same storage node; or,

[0031] flattening throughputs generated by each of the flattening tasks on a plurality of the storage nodes;

[0032] The if the total amount of the flattening throughputs in the throughput group satisfies the speed reduction condition, reducing the rate of the storage nodes executing the flattening task for the flattening throughputs in the throughput group, comprises:

[0033] if a total amount of flattening throughputs generated by one flattening task in a throughput group exceeds a throughput threshold of the flattening task, sending an instruction of reducing a rate of the flattening task to a storage node where the flattening task is located; or,

[0034] if the total amount of flat throughput generated by each of the flat tasks executed on the same storage node in the throughput group exceeds the throughput threshold of the flat tasks of the storage node, sending an instruction to the storage node to reduce the rate of the flat tasks; or,

[0035] if the total amount of flat throughput generated by each of the flat tasks executed on the plurality of storage nodes in the throughput group exceeds the throughput threshold of the flat tasks of the plurality of storage nodes, sending an instruction to the plurality of storage nodes to reduce the rate of the flat tasks;

[0036] The control module is specifically configured to determine a first ratio, wherein the first ratio is a ratio of the flat throughput of the volume in the storage node to the service throughput.

[0037] According to the first ratio, the rate of executing the flat tasks of at least one of the storage nodes is controlled to be reduced, wherein the reduction range of the rate is negatively correlated with the first ratio.

[0038] If the first ratio is greater than a preset ratio threshold, the rate of executing the flat tasks of at least one of the storage nodes is controlled to be reduced by a first range.

[0039] If the first ratio is not greater than the preset ratio threshold, the rate of executing the flat tasks of at least one of the storage nodes is controlled to be reduced by a second range, wherein the second range is greater than the first range.

[0040] In a third aspect of the embodiments of the present application, an electronic device is provided, and the electronic device comprises:

[0041] a memory for storing a computer program;

[0042] a processor for executing the program stored on the memory, and the method steps of any of the above first aspect.

[0043] In a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method steps of any of the above first aspect.

[0044] The embodiments of the present application have the following beneficial effects:

[0045] The flat speed control method and device provided by the embodiments of the present application can obtain the flat throughput generated by each storage node performing a flattening task and the service throughput generated by performing a service, and control the storage node to reduce the rate of performing the flattening task when the flat throughput and the service throughput meet a preset speed reduction condition. In the embodiments of the present application, the management node obtains the flat throughput and the service throughput on the storage node, determines the service resource condition on each node, and then controls the rate of the storage node performing the flattening task according to the service resource condition. That is, the flattening task speed can be dynamically adjusted when there are many background flattening tasks, thereby avoiding the influence on the normal service performance.

[0046] Of course, implementing any product or method of the present application does not necessarily require all the advantages described above to be achieved at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can also be obtained by those skilled in the art based on these drawings.

[0048] Figure 1 A flowchart of a flattening speed control method provided by the embodiments of the present application;

[0049] Figure 2a A schematic diagram of a distributed block storage cluster provided by the embodiments of the present application;

[0050] Figure 2b A process diagram of flattening a volume in a distributed block storage of a distributed block storage cluster provided by the embodiments of the present application;

[0051] Figure 3 A flattening task throughput information reporting diagram on a storage node provided by the embodiments of the present application;

[0052] Figure 4 A structural diagram of a flattening speed control method and device provided by the embodiments of the present application;

[0053] Figure 5 A structural diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0054] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art based on the present application are within the scope of protection of the present application.

[0055] To more clearly illustrate the flattening speed control method provided by the embodiments of the present application, a possible application scenario of a flattening speed control method provided by the embodiments of the present application will be exemplarily described below. It can be understood that the following example is only a possible application scenario of the flattening speed control method provided by the embodiments of the present application, and the flattening speed control method provided by the embodiments of the present application can also be applied to other possible application scenarios in other possible application scenarios, and the following example does not limit this.

[0056] As shown in Figure 2a , a distributed block storage cluster includes a management node and a storage node. The management node manages all storage nodes and storage volumes (hereinafter referred to as volumes) in the cluster. The storage node carries the volumes and provides block storage services.

[0057] The process of flattening the volumes in the distributed block storage is shown in Figure 2b . The management node includes a flattening task control module, which manages all flattening tasks in the cluster. Assume that the management node A manages a cluster that includes two storage nodes, storage node 1 and storage node 2. The storage nodes have three volumes, volume 1, volume 2, and volume 3. Volume 1 and volume 2 are on storage node 1, and volume 3 is on storage node 2. Volume 2 and volume 3 reference the data of volume 1, that is, all or part of the data of volume 2 and volume 3 is the same as that of volume 1, but volume 2 and volume 3 do not have this data themselves, but obtain it from volume 1 when reading. That is, volume 2 and volume 3 reference all or part of the data of volume 1. When volume 2 and volume 3 need to be flattened, the flattening task control module will issue a flattening task to the storage nodes of volume 2 and volume 3, respectively. After receiving the flattening task, the storage nodes start data copying. Taking volume 2 as an example, the data copying process is: querying the data of volume 2 referencing volume 1, reading the data of volume 1, and writing the data to the storage space of volume 2. The flattening process of volume 3 is similar, and will not be described in detail here.

[0058] In a distributed block storage system, there are a large number of volumes, and there can be a large number of volumes that need to be flattened. If these volumes are flattened at the same time, it will bring a large disk read-write pressure and network transmission pressure to the entire cluster. Moreover, if the data of a volume is referenced by multiple volumes, the volumes referencing the data of the volume will bring a large read pressure to the volume when flattening at the same time, which can cause the performance of the volume to decrease sharply, and even the volume cannot be used. Therefore, the flattening speed of all clusters needs to be controlled to ensure the quality of the block storage service of the cluster.

[0059] Based on this, the application provides a flattening speed control method, applied to a management node in a distributed block storage cluster, the management node being used to manage storage nodes in the distributed storage cluster; as shown in Figure 1 The method comprises the following steps:

[0060] S101, respectively acquiring a flattening throughput generated by each storage node executing a flattening task and a service throughput generated by each storage node executing a service.

[0061] S102, if the flattening throughputs and the service throughputs satisfy a speed reduction condition, controlling at least one storage node to reduce a speed of executing the flattening task; wherein a reduction amplitude of the speed is negatively related to the flattening throughput, and the reduction amplitude of the speed is positively related to the service throughput.

[0062] In this embodiment, by acquiring the flattening throughput generated by each storage node executing a flattening task and the service throughput generated by each storage node executing a service, when the flattening throughput and the service throughput satisfy a preset speed reduction condition, the speed of the storage node executing the flattening task is controlled to be reduced. In the embodiment of the application, the management node acquires the flattening throughput and the service throughput on the storage node, determines the processing service resource condition of each node, and then controls the speed of the storage node executing the flattening task according to the service resource condition. That is, when there are a large number of background flattening tasks, the flattening task speed can be dynamically adjusted, thereby avoiding the influence on the normal service performance.

[0063] For example, it is assumed that there are two storage nodes in cluster 1 managed by management node A, which are respectively storage node 1 and storage node 2. As shown in Figure 3The diagram illustrates the throughput information reported by storage nodes for flattening tasks. Both storage nodes have flattening tasks currently executing. Storage node 1 is executing Flattening Task 1, Flattening Task 2, and Flattening Task 3; Storage node 2 is executing Flattening Task 4 and Flattening Task 5. The flattening task data throughput aggregation module on each storage node periodically calculates the flattening throughput of each flattening task on that node and the flattening throughput of the flattening tasks on that storage node, and periodically reports this to the flattening task management module on the management node, so that the flattening task management module can control the flattening speed.

[0064] Block storage volumes need to provide services externally. Flattening tasks require disk read / write operations, which may impact volume operations. Therefore, when volume workload is high, the flattening speed should be reduced, while when workload is low, disk performance can be fully utilized to complete the flattening task as quickly as possible. Thus, the flattening task rate needs to be dynamically adjusted based on volume workload. Volume workload is determined by both volume throughput and flattening throughput. A maximum data throughput is pre-set for each volume on the storage node. When the volume throughput plus the volume flattening throughput exceeds the volume's maximum data throughput, it indicates that the volume workload is too high.

[0065] The steps S101-S102 described above will be explained below:

[0066] In S101, the management node obtains the flattening throughput generated by each storage node in the cluster it manages during the execution of flattening tasks. This can be achieved by directly retrieving the data information corresponding to the flattening throughput generated by each storage node's flattening task from the management node's flattening task management module, and then parsing this data information to obtain the corresponding flattening throughput. The cluster includes at least one storage node. The management node has a flattening task management module, and the storage nodes have flattening task data throughput aggregation modules. In one possible implementation, the flattening task data throughput aggregation module on each storage node calculates the flattening throughput of each flattening task on that node and the flattening throughput of the flattening tasks on that storage node, generates corresponding data information, and then sends the data information to the management node's flattening task management module.

[0067] Service throughput refers to the data throughput generated by the current business operations on the volumes of the storage node. The management node obtains service throughput in a similar way to it, and will not be elaborated upon here.

[0068] In another possible implementation, the flattening task data throughput aggregation module on each storage node can calculate the flattening throughput of each flattening task on the local node and the flattening throughput of the flattening task on the local storage node, generate corresponding data information, and then send the data information to the intermediate device. The flattening task management module of the management node obtains the data information from the intermediate device, and obtains the flattening throughput uploaded by each storage node by parsing the data information.

[0069] In another possible implementation, a flattened task data throughput aggregation module on each storage node can calculate the flattened throughput of each flattened task on its local node, generate data information, and then send the data information to the flattened task management module of the management node. The flattened task management module then parses the data to obtain the flattened throughput of each flattened task on that storage node and calculates the flattened throughput of the flattened tasks on that storage node. Any method that can obtain the flattened throughput uploaded by each storage node can be applied to this application; this application does not limit how the flattened throughput uploaded by each storage node is obtained.

[0070] For example, taking cluster 1 managed by the aforementioned management node A as an example, management node A obtains the flattening throughput of flattening task 1, flattening task 2, flattening task 3, flattening task 4 and flattening task 5 in storage node 1 and storage node 2.

[0071] In S102, the deceleration conditions are set by those skilled in the art based on the application scenario, and the preset conditions may be different for different application scenarios.

[0072] Flattening throughput and service throughput refer to the volumes performing flattening tasks on storage nodes. Flattening throughput is the data throughput generated by the volume during the current flattening task, while service throughput is the data throughput generated by the volume during current service operations. Additionally, each volume has a pre-set maximum throughput. If the sum of the flattening throughput and service throughput of a volume on a storage node exceeds the maximum throughput, it indicates that the volume is under significant service load, and the rate at which the flattening task is performed on the storage node containing that volume needs to be adjusted.

[0073] For example, suppose storage node 3 has two volumes: Volume 1 and Volume 2. The current business being performed on Volume 2 is business 1, which requires data from Volume 1. Volume 2 then uses flattening task 1 to copy the required data from Volume 1 to Volume 2. Flattening task 1 reads data from Volume 1 and writes it to the storage space of Volume 2. That is, the flattening task currently being performed on Volume 2 is flattening task 1. Therefore, the flattening throughput of Volume 2 is the data throughput 1 generated by flattening task 1, and the business throughput of Volume 2 is the data throughput 2 generated by business 1. Assuming that data throughput 1 and data throughput 2 meet a preset rate-reduction condition, the storage node 3 corresponding to Volume 2 is controlled to reduce the rate at which the flattening task is executed.

[0074] In one possible implementation, to control the rate of flattening tasks in a targeted manner, the flattening throughput received by the management node can be grouped, and then the rate of flattening tasks can be controlled according to the grouping, thereby improving control efficiency. Specifically, S102 can be as follows:

[0075] S1021, if the total amount of each flat throughput in the throughput group meets the preset speed reduction condition, reduce the rate at which the storage nodes of each flat throughput in the throughput group perform flattening tasks, wherein the throughput group includes at least one flat throughput.

[0076] In one possible implementation, depending on the actual application scenario, the throughput group may specifically include: the flat throughput generated by a single flattening task; or, the flat throughput generated by each flattening task executed on the same storage node; or, the flat throughput generated by each flattening task executed on multiple storage nodes. The flat throughput generated by each flattening task executed on the same storage node can be the flat throughput generated by all flattening tasks executed on that storage node, or it can be the flat throughput generated by some flattening tasks executed on that storage node. The flat throughput generated by each flattening task executed on multiple storage nodes can be the flat throughput generated by all flattening tasks executed on all storage nodes in a cluster, or it can be the flat throughput generated by some flattening tasks executed on some storage nodes in a cluster; this application embodiment does not impose further limitations.

[0077] Taking cluster 1 managed by the aforementioned management node A as an example, the flattening tasks being executed on storage node 1 include: flattening task 1, flattening task 2, and flattening task 3; the flattening tasks being executed on storage node 2 include: flattening task 4 and flattening task 5.

[0078] Assume the throughput group includes: the flat throughput 1 of flattened task 1; or the flat throughput 2 of flattened task 2; or the flat throughput 3 of flattened task 3; or the flat throughput 4 of flattened task 4; or the flat throughput 5 of flattened task 5. For example, if the throughput group includes the flat throughput 1 of flattened task 1, then the total flat throughput of each task in the throughput group is the flat throughput 1. If the flat throughput 1 meets the preset rate reduction condition, then the rate of flattened task 1 is reduced. Assume the throughput group includes the flat throughput 2 of flattened task 2, then the total flat throughput of each task in the throughput group is the flat throughput 2. If the flat throughput 2 meets the preset rate reduction condition, then the rate of flattened task 2 is reduced. The process for flattened tasks 3, 4, and 5 is similar and will not be elaborated here.

[0079] Assume the throughput group includes: the flat throughput 1 of flattened task 1, the flat throughput 2 of flattened task 2, and the flat throughput 3 of flattened task 3 in storage node 1; or the flat throughput 4 of flattened task 4 and the flat throughput 5 of flattened task 5 in storage node 2. For example, if the throughput group includes the flat throughput 1 of flattened task 1, the flat throughput 2 of flattened task 2, and the flat throughput 3 of flattened task 3 in storage node 1, then the total flat throughput of each flattened task in the throughput group is the sum of flat throughput 1, flat throughput 2, and flat throughput 3. If the sum of flat throughput 1, flat throughput 2, and flat throughput 3 satisfies a preset rate-reduction condition, then the rate of the flattened tasks on storage node 1 is reduced. How to reduce the rate of the flattened tasks on the storage node will be explained in detail below.

[0080] Assume the throughput group includes: the flat throughput of flattened task 1 (1), flat throughput of flattened task 2 (2), flat throughput of flattened task 3 (3) in storage node 1, and the flat throughput of flattened task 4 (4) and flat throughput of flattened task 5 (5) in storage node 2. Then the total flat throughput of each task in the throughput group is the sum of the flat throughputs of all tasks in cluster 1, i.e., the sum of flat throughput 1, flat throughput 2, flat throughput 3, flat throughput 4, and flat throughput 5. If the sum of flat throughput 1, flat throughput 2, flat throughput 3, flat throughput 4, and flat throughput 5 meets a preset rate-reduction condition, then the rate of the flattened tasks on storage node 1 is reduced. How to reduce the rate of the flattened tasks on the storage nodes will be explained in detail below.

[0081] In this embodiment, the flat throughput is grouped according to different application scenarios. Based on the throughput group, the total flat throughput is determined to control the rate of flattened tasks in the storage node. This allows for targeted control of the rate of flattened tasks in the storage node, thereby improving the efficiency of controlling the rate of flattened tasks.

[0082] In one possible embodiment, to improve the efficiency of controlling the flattening task rate, step S1021 above can specifically be:

[0083] S1021a, if the total flat throughput generated by a flattening task in the throughput group exceeds the throughput threshold of the flattening task, an instruction to reduce the flattening task rate is sent to the storage node where the flattening task is located.

[0084] In this step, since the total flat throughput is the flat throughput generated by the flattening task, when the total flat throughput exceeds the throughput threshold, an instruction to reduce the flattening task rate can be sent directly to the storage node where the flattening task is located, so that the storage node reduces the rate of the flattening task after receiving the instruction.

[0085] Taking cluster 1 managed by the aforementioned management node A as an example, assuming that the throughput group includes the flat throughput 2 of flattened task 2, the total flat throughput of each flattened task in the throughput group is the flat throughput 2. If the flat throughput 2 meets the preset speed reduction condition, then an instruction 2 to reduce the rate of flattened task 2 is sent to storage node 1, so that storage node 1 reduces the rate of flattened task 2 when it receives instruction 2.

[0086] or,

[0087] S1021b: If the total flattening throughput generated by each flattening task executed on the same storage node in the throughput group exceeds the throughput threshold of the flattening task of that storage node, an instruction to reduce the rate of the flattening task is sent to the storage node.

[0088] In this step, since the total flattening throughput is the sum of the flattening throughput generated by all flattening tasks on the same node, when the total flattening throughput exceeds the throughput threshold, an instruction to reduce the flattening task rate is sent to the storage node where the flattening task resides. Upon receiving the instruction, the storage node reduces the rate of the flattening tasks on that node. Specifically, the rate can be reduced by the same amount for all flattening tasks on the node, or the rate of the flattening task with the higher rate ranking can be reduced, or the rate of the flattening task with lower priority can be reduced. Other methods for reducing the rate can also be applied to the embodiments of this application.

[0089] Taking cluster 1 managed by the aforementioned management node A as an example, assuming the throughput group includes the flat throughput 1 of flattened task 1, the flat throughput 2 of flattened task 2, and the flat throughput 3 of flattened task 3 in storage node 1, then the total flat throughput of each flattened task in the throughput group is the sum of flat throughput 1, flat throughput 2, and flat throughput 3. If the sum of flat throughput 1, flat throughput 2, and flat throughput 3 meets the preset rate reduction condition, then an instruction 'a' to reduce the rate of the flattened tasks is sent to storage node 1, so that storage node 1 reduces the rate of the flattened tasks on the storage node upon receiving instruction 'a'. Specifically, the rates of flattened task 1, flattened task 2, and flattened task 3 can all be reduced by 5%, or the rates of flattened task 1, flattened task 2, and flattened task 3 can be reduced to 300MB / s.

[0090] or,

[0091] S1021c, if the total flattening throughput generated by each of the multiple storage nodes in the throughput group executing the flattening task exceeds the throughput threshold of the flattening task of the multiple storage nodes, an instruction to reduce the rate of the flattening task is sent to the multiple storage nodes.

[0092] In this step, since the total flattening throughput is the sum of the flattening throughput generated by all flattening tasks on multiple storage nodes in the same cluster, when the total flattening throughput exceeds the throughput threshold, an instruction to reduce the flattening task rate is sent to all or some storage nodes in the cluster. Upon receiving the instruction, the storage nodes reduce the rate of the flattening tasks on that storage node. Specifically, the flattening throughput of all flattening tasks on the storage node is reduced by a certain percentage (e.g., a 5% reduction), or the flattening throughput of all flattening tasks on the storage node is reduced to a certain value (e.g., reduced to 500MB / s). Other methods for reducing the rate can also be applied to the embodiments of this application.

[0093] Taking cluster 1 managed by the aforementioned management node A as an example, assume that the throughput group includes: the flat throughput 1 of flattened task 1, the flat throughput 2 of flattened task 2, the flat throughput 3 of flattened task 3 in storage node 1, and the flat throughput 4 of flattened task 4 and the flat throughput 5 of flattened task 5 in storage node 2. Then the total flat throughput of each flattened task in the throughput group is the sum of all flattened throughputs in cluster 1, that is, the sum of flat throughput 1, flat throughput 2, flat throughput 3, flat throughput 4, and flat throughput 5. If the sum of flat throughput 1, flat throughput 2, flat throughput 3, flat throughput 4, and flat throughput 5 meets the preset rate reduction condition, then an instruction A to reduce the rate of flattened tasks is sent to storage node 1 and storage node 2. When storage node 1 and storage node 2 receive instruction A, storage node 1 reduces the rate of flattened tasks on that storage node, and storage node 2 reduces the rate of flattened tasks on that storage node. Specifically, the rates of flattening tasks 1, 2, 3, 4, and 5 can be sorted, and the rates of the top three flattening tasks can be reduced by 5%, or the rates of the top three flattening tasks can be reduced to 300MB / s.

[0094] In this embodiment, the flattening throughput of the distributed block storage cluster is divided into three types. The rate of a single flattening task is controlled in a targeted manner to ensure that the rate of a single flattening task does not exceed a preset threshold, thus avoiding impact on volume service performance. The rate of flattening tasks on a single storage node is also controlled in a targeted manner to ensure that the rate of flattening tasks on the storage node does not exceed a preset threshold, keeping the disk pressure on the storage node within a limited range. Finally, the rate of flattening tasks on all storage nodes within a single cluster is controlled in a targeted manner to ensure that the rate of all flattening tasks in the entire cluster does not exceed a preset threshold, preventing excessive disk pressure on the cluster and avoiding impact on cluster stability and service quality.

[0095] In one possible embodiment, in order to dynamically adjust the rate of the flattening task according to the volume's workload, controlling at least one storage node to reduce the rate of executing the flattening task specifically includes:

[0096] S201, determine the first ratio; wherein the first ratio is the ratio of the volume flattening throughput to the service throughput in the storage node.

[0097] S202, based on the first ratio, control at least one of the storage nodes to reduce the rate at which the flattening task is executed, wherein the reduction in rate is negatively correlated with the first ratio.

[0098] In this step, the first ratio is the ratio of volume flattening throughput to service throughput in the storage node. A larger ratio indicates a smaller proportion of service throughput in the storage node, meaning the volume is handling a smaller volume of service. Therefore, reducing the flattening task rate has little impact on service processing speed, and the flattening task rate is reduced at a low level. Conversely, a smaller ratio indicates a larger proportion of service throughput in the storage node, meaning the volume is handling a larger volume of service. To avoid affecting service processing speed, the flattening task rate needs to be significantly reduced.

[0099] In one possible embodiment, the above-mentioned S202 can specifically be:

[0100] S202a, if the first ratio is greater than a preset ratio threshold, then control at least one storage node to reduce the rate at which it performs the flattening task by a first magnitude.

[0101] In this step, the preset ratio threshold is set by those skilled in the art based on the actual application scenario. The preset ratio threshold is the minimum ratio of the volume's flattening throughput to the service throughput in the storage node where the rate of the flattening task needs to be adjusted. The preset ratio threshold corresponds to a preset reduction in the flattening task rate. When the first ratio is greater than the preset ratio threshold, it indicates that the volume's flattening throughput ratio is greater than the aforementioned minimum ratio, and the rate of the flattening task is reduced by a margin less than the preset rate.

[0102] S202b, if the first ratio is not greater than a preset ratio threshold, then control at least one storage node to reduce the rate at which it performs the flattening task by a second magnitude, wherein the second magnitude is greater than the first magnitude.

[0103] In this step, when the first ratio is greater than the preset ratio threshold, it means that the flattening throughput ratio of the volume is less than the minimum ratio mentioned above. At this time, reducing the speed of the flattening task by the preset range will not have an impact on alleviating the business pressure. In this case, it is necessary to reduce the rate of the flattening task by a range greater than the preset range.

[0104] In one possible implementation, the second magnitude can be the rate at which the volume performs the flattening task. That is, when the first ratio is not greater than a preset ratio threshold, the rate of the flattening task can be reduced according to the rate at which the volume performs the flattening task. In other words, the storage node can be directly controlled to reduce the rate of the flattening task to 0MB / s and pause the flattening task.

[0105] In this embodiment, a control method dynamically adjusts the flattening speed based on the service pressure of a single volume. This method allows the flattening speed to be reduced when the service pressure on a volume is high, and to be restored when the service pressure is low. This prevents the volume's service performance from being unable to improve during flattening, while also enabling flattening to be completed as quickly as possible when the service pressure on the volume is low.

[0106] For distributed block storage clusters, the probability of service failure is relatively high when the overall data throughput of the cluster reaches its limit. Therefore, when the business pressure of the cluster is high, it is necessary to control the data throughput of flattened tasks in the cluster to prevent the overall data throughput of the cluster from exceeding the limit, which would cause widespread performance degradation or service failure of block storage services in the cluster.

[0107] In one possible implementation, the flattening speed can be dynamically adjusted according to the cluster's workload, specifically as follows:

[0108] S301, respectively obtain the flattening throughput generated by each storage node executing the flattening task and the service throughput generated by each storage node executing the service.

[0109] S302, if the flat throughput and service throughput in the cluster meet the preset speed reduction conditions, then control the storage nodes in the cluster to reduce the rate at which they execute flattening tasks.

[0110] In this step, the storage nodes will periodically report the total data throughput of the business on their respective nodes to the management node, and will also periodically report the data throughput of all flattening tasks on their respective nodes. The management node will aggregate the business throughput and flattening throughput of all businesses in the cluster, and then determine the business pressure of the entire cluster based on the aggregated business throughput and flattening throughput, and then dynamically adjust the flattening rate based on the business pressure of the cluster.

[0111] Specifically, a threshold for the maximum data throughput of the entire cluster can be preset as the first threshold. When the sum of the total business throughput and the flattening throughput of the cluster exceeds the first threshold, it indicates that the cluster is under significant business pressure and the flattening task rate of the cluster needs to be adjusted. The specific adjustment scheme is similar to the aforementioned step S1021C, and the flattening task rate of the storage nodes in the cluster can be adjusted based on the adjustment method in step S1021C.

[0112] In this embodiment, the flattening speed is dynamically adjusted according to the overall business pressure of the cluster. This reduces the impact of flattening tasks on business performance when the business pressure of the cluster increases significantly, while also avoiding excessive overall disk pressure on the cluster from having a significant impact on business stability and service quality.

[0113] See Figure 4 , Figure 4 The diagram shown is a structural schematic of a flattened speed control device provided in an embodiment of this application. It is applied to a management node in a distributed block storage cluster. The management node manages the storage nodes in the distributed block storage cluster. The device includes:

[0114] The first acquisition module 401 is used to acquire the flattening throughput generated by each storage node executing the flattening task and the business throughput generated by each storage node executing business.

[0115] The control module 402 is used to control at least one storage node to reduce the rate at which the flattening task is executed if the flattening throughput and the service throughput meet the rate reduction conditions; wherein the rate reduction is negatively correlated with the flattening throughput and positively correlated with the service throughput.

[0116] In one possible embodiment, the control module 402 is specifically configured to reduce the rate at which the storage node of each of the flat throughputs in the throughput group performs the flattening task if the total amount of each of the flat throughputs in the throughput group meets the rate reduction condition, wherein the throughput group includes at least one of the flat throughputs.

[0117] In one possible embodiment, the throughput group includes:

[0118] A flattened task produces a flattened throughput; or,

[0119] The flattening throughput generated by each flattening task executed on the same storage node; or,

[0120] The flat throughput generated by each of the aforementioned storage nodes executing a flattening task;

[0121] If the total flat throughput of each of the throughput groups meets the rate-reduction condition, the rate at which the storage nodes of each of the flat throughput groups execute the flattening task is reduced, including:

[0122] If the total flattened throughput generated by a flattening task in the throughput group exceeds the throughput threshold of that flattening task, an instruction to reduce the rate of the flattening task is sent to the storage node where the flattening task resides; or,

[0123] If the total flattening throughput generated by all flattening tasks on the same storage node in the throughput group exceeds the throughput threshold of the flattening tasks on that storage node, an instruction to reduce the rate of the flattening tasks is sent to the storage node; or,

[0124] If the total flattening throughput generated by each of the multiple storage nodes in the throughput group executing flattening tasks exceeds the throughput threshold of the flattening tasks of the multiple storage nodes, an instruction to reduce the rate of the flattening tasks is sent to the multiple storage nodes.

[0125] The control module 402 is specifically used to determine a first ratio; wherein the first ratio is the ratio of the volume flattening throughput to the service throughput in the storage node;

[0126] Based on the first ratio, at least one of the storage nodes is controlled to reduce the rate at which the flattening task is executed, wherein the rate reduction is negatively correlated with the first ratio.

[0127] If the first ratio is greater than a preset ratio threshold, then control at least one of the storage nodes to reduce the rate at which the flattening task is executed by a first magnitude.

[0128] If the first ratio is not greater than the preset ratio threshold, then at least one of the storage nodes will be controlled to reduce the rate at which the flattening task is performed by a second magnitude, wherein the second magnitude is greater than the first magnitude.

[0129] In this embodiment, by acquiring the flattening throughput generated by each storage node executing flattening tasks, and when the flattening throughput meets a preset rate-reduction condition, the storage nodes are controlled to reduce the rate at which they execute flattening tasks. In this embodiment, the management node acquires the flattening throughput generated by the flattening tasks executed on the storage nodes, determines the throughput of the flattening tasks on each node, and then controls the rate at which the storage nodes execute flattening tasks based on the flattening throughput. That is, when there are many flattening tasks in the background, the speed of the flattening tasks can be dynamically adjusted, thereby avoiding the impact on normal business performance.

[0130] This application also provides an electronic device, such as... Figure 5 As shown, it includes:

[0131] Memory 501 is used to store computer programs;

[0132] When processor 502 executes the program stored in memory 501, it performs the following steps:

[0133] The flattening throughput generated by each storage node executing the flattening task and the service throughput generated by each storage node executing the service are obtained respectively.

[0134] If each of the flattening throughputs and each of the service throughputs meets the rate reduction condition, then at least one of the storage nodes is controlled to reduce the rate at which the flattening task is executed; wherein the rate reduction is negatively correlated with the flattening throughput and positively correlated with the service throughput.

[0135] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 502, communication interface, and memory 501 communicating with each other via the communication bus.

[0136] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0137] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0138] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0139] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0140] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described flattening speed control method.

[0141] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the flattened speed control methods described above.

[0142] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0143] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0144] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for devices, electronic devices, and computer-readable storage media, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments.

[0145] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A flattened speed control method, characterized in that, A management node applied in a distributed block storage cluster, the management node being used to manage the storage nodes in the distributed block storage cluster; the method includes: The flattening throughput generated by each storage node executing the flattening task and the service throughput generated by each storage node executing services are obtained respectively; wherein, the flattening task refers to the process in which a storage volume in a storage node references data from other volumes and copies it to the storage space of the storage volume itself; If the total flat throughput generated by each of the multiple storage nodes in the throughput group executing flattening tasks exceeds the throughput threshold of the flattening tasks of the multiple storage nodes, a first ratio is determined; based on the first ratio, at least one of the storage nodes is controlled to reduce the rate at which the flattening tasks are executed; wherein, the throughput group includes: the flat throughput generated by each of the multiple storage nodes executing flattening tasks, the first ratio is the ratio of the flat throughput of the volume in the storage node to the service throughput; the rate reduction is negatively correlated with the flat throughput, the rate reduction is positively correlated with the service throughput, and the rate reduction is negatively correlated with the first ratio.

2. The method according to claim 1, characterized in that, The step of controlling multiple storage nodes to reduce the execution rate of the flattening task according to the first ratio includes: If the first ratio is greater than a preset ratio threshold, then the multiple storage nodes will be controlled to reduce the rate at which they perform the flattening task by a first magnitude. If the first ratio is not greater than the preset ratio threshold, then the multiple storage nodes are controlled to reduce the rate at which they perform the flattening task by a second magnitude, wherein the second magnitude is greater than the first magnitude.

3. A flattened speed control device, characterized in that, A management node applied in a distributed block storage cluster, the management node being used to manage the storage nodes in the distributed block storage cluster; the apparatus includes: The first acquisition module is used to acquire the flattening throughput generated by each storage node performing the flattening task and the service throughput generated by each storage node performing the service; wherein, the flattening task refers to the process in which a storage volume in a storage node references data from other volumes and copies it to the storage space of the storage volume itself; A control module is configured to determine a first ratio if the total flattening throughput generated by each of the multiple storage nodes executing flattening tasks in a throughput group exceeds the throughput threshold of the flattening tasks of the multiple storage nodes; and, based on the first ratio, control at least one of the storage nodes to reduce the rate at which the flattening tasks are executed; wherein the throughput group includes: the flattening throughput generated by each of the multiple storage nodes executing flattening tasks, the first ratio is the ratio of the flattening throughput of volumes in the storage nodes to the service throughput; the rate reduction is negatively correlated with the flattening throughput, the rate reduction is positively correlated with the service throughput, and the rate reduction is negatively correlated with the first ratio.

4. The apparatus according to claim 3, characterized in that, The control module is specifically configured to, if the first ratio is greater than a preset ratio threshold, control the multiple storage nodes to reduce the rate at which they execute the flattening task by a first magnitude. If the first ratio is not greater than the preset ratio threshold, then the multiple storage nodes are controlled to reduce the rate at which they perform the flattening task by a second magnitude, wherein the second magnitude is greater than the first magnitude.

5. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-2.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-2.

Images