Hierarchical current limiting method and device, equipment, storage medium and product

Abstract

The invention discloses a hierarchical flow limiting method and device, equipment, a storage medium and a product, and relates to the technical field of flow limiting. A flow control unit is arranged on a target data layer to construct a multi-level flow control structure; distributing the flow control quota of the upper-layer flow control unit to the associated lower-layer flow control unit; and carrying out layer-by-layer current limiting on an input IO request based on each flow control unit. According to the method, a multi-level flow control structure is constructed on the basis of the data organization structure, flow limiting is carried out layer by layer when the IO request is input, flow limiting with lower fine granularity can be achieved, and the flow limiting requirement of a high-performance scene is met.

Description

Technical Field

[0001] This invention relates to the field of current limiting technology, and in particular to a graded current limiting method, apparatus, equipment, storage medium, and product. Background Technology

[0002] Rate limiting is an important stability guarantee mechanism in high-concurrency systems, preventing system crashes caused by frequent access. Existing rate limiting methods include single-machine rate limiting and distributed rate limiting, but multi-level rate limiting methods are lacking.

[0003] For example, taking a unified key-value storage system as an example, the system consists of multiple management nodes and data nodes. Existing technologies only consider rate limiting at the data node level when performing rate limiting. However, in a unified key-value storage system, a data node can start multiple instances, an instance can contain multiple tables, and a table can be divided into multiple partitions. Therefore, if only rate limiting at the data node level is considered, it is possible that the IO requests at the data node level meet the rate limiting requirements, while the access at the table or partition level is overloaded. In high-performance scenarios, the rate limiting effect is not ideal.

[0004] Therefore, there is a need to provide a multi-level rate limiting method that can take into account the data organization format. Summary of the Invention

[0005] The purpose of this invention is to provide a hierarchical rate limiting method, apparatus, device, storage medium, and product that can achieve multi-level rate limiting based on data organization architecture, meeting the rate limiting requirements of high-performance scenarios.

[0006] To achieve the above objectives, embodiments of the present invention provide a tiered current limiting method, comprising: Flow control units are set up in the target data layer to construct a multi-level flow control structure; The flow control quotas of the upper-level flow control unit are allocated to the associated lower-level flow control units; Each of the aforementioned flow control units performs level-by-level flow limiting on the input I / O requests.

[0007] As an improvement to the above scheme, when the flow control unit is in the enabled state, the flow limiting of input IO requests based on each flow control unit includes: Calculate the remaining flow control quota based on the flow control quota and the current timestamp; Release IO requests in the pending queue according to the remaining flow control quota, and update the remaining flow control quota; When the updated remaining flow control quota is greater than 0, allow the input IO request; When the updated remaining flow control quota is not greater than 0, the input IO request is added to the pending queue.

[0008] As an improvement to the above solution, when the flow control unit is in the on state, the flow control unit is further used to: When there are IO requests in the pending queue, the remaining flow control quota is updated periodically; When the updated remaining flow control quota is greater than 0, the IO requests in the suspended queue are allowed to proceed according to the updated remaining flow control quota.

[0009] As an improvement to the above scheme, when the upper-layer flow control unit is a surface layer, the allocation of flow control quotas from the upper-layer flow control unit to the associated lower-layer flow control unit includes: The flow control quotas of the upper-level flow control units are evenly distributed to the associated lower-level flow control units as the initial flow control quotas; After the dynamic allocation condition is triggered, the allocable flow control quota is allocated according to the proportion of each flow control unit to be adjusted and then superimposed on the initial flow control quota. The allocatable flow control quota is obtained by subtracting the initial flow control quota of the flow control unit to be adjusted and the actual value of the flow control parameters of the non-flow control units from the flow control quota of the upper-level flow control unit; the flow control unit to be adjusted is the lower-level flow control unit whose actual value of the flow control parameters is greater than the initial flow control quota.

[0010] As an improvement to the above solution, when the target data layer is a lower-level data layer and is configured with multi-threading, setting a flow control unit in the target data layer includes: In the flow control unit of the underlying data layer, a sub-flow control unit is set up for each thread.

[0011] As an improvement to the above scheme, the sub-flow control unit is used for: Upon receiving an I / O request, determine whether the I / O request falls within the scope of the sampling inspection: If so, rate-limit the I / O requests; If not, allow the IO request directly.

[0012] As an improvement to the above solution, the step of determining whether an IO request is within the sampling scope includes: Determine if the ID of the IO request is within the target range: If so, the IO request is determined to be within the sampling inspection range; If not, the IO request is determined to be outside the scope of the sampling inspection.

[0013] As an improvement to the above scheme, the target data layer includes one or more of the instance layer, table layer, and partition layer.

[0014] To achieve the above objectives, embodiments of the present invention also provide a graded current limiting device, comprising: A multi-level flow control module is used to set up flow control units at the target data layer to build a multi-level flow control structure; The quota module is used to allocate flow control quotas from upper-level flow control units to associated lower-level flow control units. The rate limiting module is used to perform level-by-level rate limiting on the input IO requests based on each of the flow control units.

[0015] To achieve the above objectives, embodiments of the present invention also provide a tiered current limiting device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the tiered current limiting method as described in any of the above embodiments.

[0016] To achieve the above objectives, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the graded rate limiting method as described in any of the above embodiments.

[0017] To achieve the above objectives, embodiments of the present invention also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the graded rate limiting method as described in any of the above embodiments.

[0018] Compared with existing technologies, the hierarchical rate limiting method, apparatus, device, storage medium, and product of this invention establishes a flow control unit at the target data layer to construct a multi-level flow control structure; allocates flow control quotas from upper-level flow control units to associated lower-level flow control units; and performs tiered rate limiting on input IO requests based on each of the flow control units. This invention constructs a multi-level flow control structure based on the data organization architecture and performs tiered rate limiting when IO requests are input, enabling finer-grained rate limiting and meeting the rate limiting requirements of high-performance scenarios. Attached Figure Description

[0019] Figure 1 This is a flowchart of a graded current limiting method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of step-by-step current limiting provided in an embodiment of the present invention; Figure 3 This is a flowchart of the flow control unit provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a partitioned flow control unit provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of sampling flow control according to an embodiment of the present invention; Figure 6This is a schematic diagram of a graded current limiting device provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a graded current limiting device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It is understood that a unified KV storage system (KV Storage System / Key-Value Storage System) is a strongly consistent distributed storage system that provides services through a table interface. A unified KV storage system consists of multiple management nodes and data nodes. Multiple KV instances are started on a single data node to provide data read / write (IO) services. One KV instance is started on one data disk, i.e., a single-disk, single-instance mode. Internally, tables are divided into multiple partitions according to range or hash methods. Each partition manages a portion of the table data and provides data read / write services. For ease of explanation, the following explanation uses a unified KV storage system to illustrate the hierarchical rate limiting method of this invention. It is worth noting that this invention is also applicable to other scenarios and is not limited here.

[0022] One embodiment of the present invention provides a graded current limiting method, see [link to relevant documentation]. Figure 1 The flowchart of the graded current limiting method shown includes steps S11 to S13: S11. Set up flow control units in the target data layer to construct a multi-level flow control structure; S12. Allocate the flow control quota of the upper-level flow control unit to the associated lower-level flow control unit; S13. Based on each of the flow control units, the input IO requests are rate-limited layer by layer.

[0023] It's worth noting that the target data layer can be understood as the data layer that performs flow control, and the target data layer can be customized. Taking a unified key-value storage system as an example, the data layer includes an instance layer, a table layer, and a partition layer. Therefore, the target data layer can consist of an instance layer and a table layer, or it can consist of an instance layer, a table layer, and a partition layer, etc., without any limitation here.

[0024] For example, in step S11, flow control units can be configured on all data layers and the flow control unit of the target data layer can be set to the enabled state, or flow control units can be configured and enabled only on the target data layer; this is not limited here. Further, the provision of flow control units in the target data layer as described in this invention can be understood as setting one flow control unit for each data unit on each target data layer. For example, assuming instance 1 contains three tables: Table A, Table B, and Table C, where Table A contains three partitions: Table A / P1, Table A / P2, and Table A / P3, when the target data layer includes an instance layer, a table layer, and a partition layer, then one flow control unit is set for instance 1 at the instance layer, and one flow control unit is set for each table (a data unit in the table layer) at the table layer, i.e., one flow control unit is set for each of Table A, Table B, and Table C. Similarly, one flow control unit is set for each of Table A / P1, Table A / P2, and Table A / P3 at the partition layer, thereby forming a multi-level flow control structure.

[0025] It is worth noting that in some implementations, multiple flow control units can be pre-configured. For example, flow control units can be configured for each data unit in each data layer. Then, the state of each flow control unit is set to on or off according to the specific flow control scheme. Thus, by setting the state of the flow control units, this embodiment of the invention can facilitate flexible adjustment of the flow control scheme and improve the flexibility of constructing a multi-level flow control structure.

[0026] Furthermore, in this invention, flow control parameters can be understood as indicators for setting limits. These parameters can be one or more of the following: read IOPS (Input / Output Operations Per Second), read BPS (Bits Per Second), write IOPS, write BPS, total read / write IOPS, and total read / write BPS. Furthermore, the flow control parameters for each flow control unit can be customized; that is, the flow control parameters for different flow control units can be the same or different. A flow control unit can be set with one or multiple flow control parameters, without limitation.

[0027] Furthermore, flow control quotas can be understood as thresholds for flow control parameters, serving as the basis for flow control units to determine flow limiting. It is worth noting that in the multi-level flow control of this invention, upper-level flow control units are associated with specific lower-level flow control units. For example, in the above embodiment, Table A, Table B, and Table C are lower-level data layers of Instance 1, and there is an association between upper and lower-level data layers. Therefore, the flow control unit of Instance 1 is also associated with the respective flow control units of Table A, Table B, and Table C. Similarly, Table A / P1, Table A / P2, and Table A / P3 are lower-level data layers of Table A, and there is an association between them. Therefore, the flow control unit of Table A is also associated with the respective flow control units of Table A / P1, Table A / P2, and Table A / P3. The association between upper and lower-level flow control units is reflected in the data relationships and flow limiting processes between flow control quotas. It is worth noting that for a given flow control parameter, the flow control quota of a lower-level flow control unit is allocated from the flow control quota of the upper-level flow control unit. The sum of the flow control quotas of the lower-level flow control units is less than or equal to the flow control quota of the upper-level flow control unit. The allocation method for the flow control quota can be average allocation, dynamic allocation, or other methods, which are not limited here. Taking the above embodiment as an example, if the read IOPS quota of the flow control unit of Table A is 3000, it can be evenly allocated to the lower-level flow control units. That is, the read IOPS quota of the flow control units of Table A / P1, Table A / P2, and Table A / P3 is 1000 each. It can be understood that the flow control quota of a flow control unit is used to limit the flow of the corresponding data unit. For example, the flow control unit of Table A will limit the flow of Table A according to its read IOPS quota. Understandably, in a multi-level flow control structure, the entire IO process is similar to entering multiple levels of pipes with different "flow rates", and the upper limit of the "flow rate" is limited by the level that reaches the upper limit first in the process.

[0028] Furthermore, the input I / O requests are rate-limited level by level by the flow control units in the multi-level flow control structure. For example, see [link to example]. Figure 2 This diagram illustrates the hierarchical rate limiting mechanism corresponding to the above embodiments. As shown in the diagram, if an IO request wants to perform a read / write operation on Table A / P1, and flow control units are set up for Instance 1, Table A, and Table A / P1, then the input IO request must first pass through the flow control unit of Instance 1. Only after the flow control unit of Instance 1 allows the request can it be input to the flow control unit of Table A. Only after the flow control unit of Table A allows the request can it be input to the flow control unit of Table A / P1. Finally, only after the flow control unit of Table A / P1 allows the request can a read / write operation be performed on Table A / P1.

[0029] Compared with existing technologies, the hierarchical rate limiting method of this invention sets up a flow control unit at the target data layer to construct a multi-level flow control structure; allocates the flow control quota of the upper-level flow control unit to the associated lower-level flow control unit; and performs step-by-step rate limiting on the input IO requests based on each of the flow control units. This invention constructs a multi-level flow control structure based on the data organization architecture and performs step-by-step rate limiting when IO requests are input, enabling finer-grained rate limiting and meeting the rate limiting requirements of high-performance scenarios.

[0030] In one alternative implementation, the target data layer includes one or more of an instance layer, a table layer, and a partition layer.

[0031] For example, assuming the target data layer includes an instance layer, a table layer, and a partition layer, it can be understood that the flow control unit of the instance layer is used to perform flow control on the entire instance, limiting the upper limit of CPU and memory resource usage, thereby protecting the disk load and network load of the instance and ensuring the stable operation of data nodes; the flow control unit of the table layer is used to perform flow control on the entire table; and the flow control unit of the partition layer is the lowest-level flow control unit, used to perform flow control on partition read and write operations, etc.

[0032] In one optional implementation, when the flow control unit is enabled, the flow limiting of input I / O requests based on each of the flow control units includes: Calculate the remaining flow control quota based on the flow control quota and the current timestamp; Release IO requests in the pending queue according to the remaining flow control quota, and update the remaining flow control quota; When the updated remaining flow control quota is greater than 0, allow the input IO request; When the updated remaining flow control quota is not greater than 0, the input IO request is added to the pending queue.

[0033] It is worth noting that in the multi-level flow control structure of the present invention, if the flow control unit is in the closed state, the IO request is directly allowed. The flow limiting method described in the embodiments of the present invention is executed only when the flow control unit is in the open state.

[0034] Specifically, when the flow control unit is enabled, upon receiving a new IO request, the remaining flow control quota is calculated. For example, a time period is determined based on the start of the time slice and the timestamp, and the IO request statistics within the time period are obtained. The remaining flow control quota is obtained by subtracting the IO request statistics from the current time slice's flow control quota. Further, IO requests in the pending queue are allowed according to the remaining flow control quota. Specifically, if the remaining flow control quota is greater than 0, the IO requests in the pending queue are allowed, and the number of allowed IO requests in the pending queue cannot exceed the remaining flow control quota. After allowing the IO requests in the pending queue, the remaining flow control quota is updated. If the remaining flow control quota is greater than 0, the incoming IO request is allowed; if the remaining flow control quota is 0, the new IO request is added to the pending queue.

[0035] The following example illustrates this. Suppose that after receiving a new IO request, the remaining flow control quota is calculated to be 100. Then, the IO requests in the pending queue are allowed to proceed, and the new remaining flow control quota is calculated. For example, if the number of IO requests in the pending queue is 36, the remaining flow control quota after allowing the IO requests in the pending queue is 64 > 0, so the incoming IO request is allowed. If the number of IO requests in the pending queue is 120, the first 100 IO requests in the pending queue are allowed, and then the remaining flow control quota is updated to 0. Since the remaining flow control quota is not greater than 0, the incoming IO request is added to the pending queue.

[0036] In an optional implementation, when the flow control unit is in the enabled state, the flow control unit is further configured to: When there are IO requests in the pending queue, the remaining flow control quota is updated periodically; When the updated remaining flow control quota is greater than 0, the IO requests in the suspended queue are allowed to proceed according to the updated remaining flow control quota.

[0037] Understandably, during flow control, the flow control unit adds IO requests that do not meet the release conditions to a suspend queue. Therefore, in this embodiment of the invention, when there are IO requests in the suspend queue, the remaining flow control quota needs to be calculated according to a preset period. When the remaining flow control quota is greater than 0, the IO requests in the suspend queue are released according to the remaining flow control quota. For example, when the number of IO requests in the suspend queue is greater than the remaining flow control quota, only IO requests equal to the number of remaining flow control quotas are released. When the number of IO requests in the suspend queue is less than or equal to the remaining flow control quota, all IO requests in the suspend queue are released.

[0038] To facilitate understanding, the workflow of the flow control unit is explained below with reference to the accompanying diagram. See also... Figure 3Upon receiving an IO request, the flow control unit first determines whether it is in an enabled state. If not, it directly allows the IO request. If it is enabled, it allows the IO request in the pending queue and then checks if the remaining flow control quota is 0. If the remaining flow control quota is 0, it adds the IO request to the pending queue. If the remaining flow control quota is not 0, it allows the IO request. Furthermore, when there is an IO request in the pending queue, a timer trigger periodically cleans up the pending queue. Specifically, it retrieves an IO request from the pending queue and checks if the remaining flow control quota is 0. If the remaining flow control quota is 0, it waits for the next timer trigger to clean up. If the remaining flow control quota is not 0, it allows the IO request and returns to the step of "retrieving an IO request from the pending queue" to repeat the above steps until the remaining flow control quota is 0 or there are no IO requests in the pending queue.

[0039] In one optional implementation, when the upper-layer flow control unit is a surface layer, the allocation of flow control quotas from the upper-layer flow control unit to the associated lower-layer flow control unit includes: The flow control quotas of the upper-level flow control units are evenly distributed to the associated lower-level flow control units as the initial flow control quotas; After the dynamic allocation condition is triggered, the allocable flow control quota is allocated according to the proportion of each flow control unit to be adjusted and then superimposed on the initial flow control quota. The allocatable flow control quota is obtained by subtracting the initial flow control quota of the flow control unit to be adjusted and the actual value of the flow control parameters of the non-flow control units from the flow control quota of the upper-level flow control unit; the flow control unit to be adjusted is the lower-level flow control unit whose actual value of the flow control parameters is greater than the initial flow control quota.

[0040] Understandably, the common way to allocate flow control quotas is to distribute them equally. While equal distribution can ensure that each lower-level flow control unit has a certain flow control capability, it is difficult to adapt to dynamically changing load scenarios and cannot make full use of the total flow control quota of the upper-level flow control units, thus lacking flexibility.

[0041] To address the aforementioned technical problems, this invention provides a method for dynamically allocating flow control quotas, used to dynamically allocate flow control quotas from table flow control units (flow control units of a table) to their underlying partition flow control units (flow control units of partitions). Specifically, firstly, the flow control quotas of the table flow control units are evenly distributed to each partition flow control unit. That is, assuming the flow control quota of the table flow control unit is L and the number of partitions is n, the initial flow control quota for each partition flow control unit is n / L. Further, after triggering the dynamic allocation condition, partition flow control units whose actual flow control parameter values ​​are greater than their initial flow control quotas are designated as flow control units to be adjusted, and partition flow control units whose actual flow control parameter values ​​are not greater than their initial flow control quotas are designated as flow control units not to be adjusted. Then, the flow control quotas of the non-adjustable flow control units remain unchanged, and the flow control quotas of the flow control units to be adjusted are dynamically adjusted according to the available flow control quotas.

[0042] Specifically, in this embodiment of the invention, a central node (process or thread) needs to be set up to periodically collect the actual values ​​of flow control parameters of each partition and dynamically allocate flow control quotas. For example, the central node can be placed on the Master node of a unified KV storage system.

[0043] For ease of explanation, assume the flow control parameter is IOPS, the table's IOPS quota is L, the table has n partitions, each partition's flow control unit has an upper limit of IOPS of Li and a lower limit of r, the actual IOPS value is Qi, and the table's status is denoted as Status, set to either no flow control or flow control in progress. The dynamic allocation method for flow control quotas includes the following three steps: 1. Initialize; 2. When the dynamic allocation condition is triggered, dynamically allocate the table's IOPS quota to Li; 3. If the dynamic allocation condition is released, restore the initialization state.

[0044] First, initialization is performed by setting the IOPS upper limit Li of each partition flow control unit to L and the status Status of the table flow control unit to not flow controlled. Then, the lower limit r of IOPS is set to L / n to ensure that each partition flow control unit is allocated at least 1 / n of the flow control quota of the table flow control unit.

[0045] Furthermore, when the dynamic allocation condition is triggered, the IOPS quota of the table is dynamically allocated to Li, and Status is set to {Flow Control in Progress}. For example, the dynamic allocation condition is triggered when any of the following conditions are met: (1) Status = {Flow Control Not in Progress}, and the sum of the actual IOPS values ​​of each partition flow control unit is greater than or equal to a first preset threshold, wherein the first preset threshold can be 95%*L; (2) Status = {Flow Control in Progress}, and the sum of the actual IOPS values ​​of each partition flow control unit is greater than or equal to a second preset threshold, wherein the second preset threshold can be 70%*L.

[0046] Furthermore, dynamically allocating the IOPS quota in the table to Li specifically means: if Qi ≤ r, that is, the actual IOPS value of the partition flow control unit is less than or equal to the lower limit of IOPS, then the partition flow control unit is recorded as a non-adjustable flow control unit, that is, no flow control quota needs to be adjusted; if Qi > r, then the partition flow control unit is recorded as a flow control unit to be adjusted, and its flow control quota is adjusted. Specifically, when adjusting the flow control quota, the allocable IOPS quota is calculated according to the following formula: (1) in, Indicates the allocatable IOPS quota; This indicates the table's IOPS quota; Indicates the first Actual IOPS values ​​for each partition flow control unit; Indicates the number of partitions in the table; This indicates the lower limit of IOPS.

[0047] Furthermore, the upper limit of IOPS for the flow control unit to be adjusted is calculated according to the following formula: (2) in, Indicates the first The IOPS limit of each partition flow control unit should be adjusted according to equation (2) only for flow control units that need to be adjusted. Indicates the lower limit of IOPS; Indicates the allocatable flow control quota; Indicates the first Actual IOPS values ​​for each partition flow control unit; This represents the sum of the actual IOPS values ​​of the flow control unit to be adjusted, i.e. .

[0048] As can be seen from the above formula, in this embodiment of the invention, the allocable IOPS quota is calculated by summing the remaining flow control quota (initial IOPS quota minus the actual IOPS value) of each non-adjustable flow control unit, and then the allocable IOPS quota is determined based on the proportion of the actual IOPS value of each flow control unit to the total flow control quota. The ratio is used to allocate dynamically adjusted amounts from the allocable IOPS quota to each flow control unit to be adjusted. The amount will be adjusted dynamically. The lower limit of IOPS is superimposed as the upper limit of IOPS for each flow control unit to be adjusted.

[0049] It is worth noting that, in the above example, in order to more clearly illustrate that the embodiments of the present invention can both prioritize ensuring that each partition flow control unit is allocated at least 1 / n of the table's IOPS quota, thereby guaranteeing the basic flow control capability of each partition flow control unit, and can also dynamically allocate the allocable IOPS quota to make full use of the table's total IOPS quota, this example uses two indicators, IOPS upper limit and IOPS lower limit, to describe these two indicators. Therefore, the following supplementary explanation should be given regarding these two indicators: the initial flow control quota of each partition flow control unit is equal to the IOPS lower limit. For flow control units that are not to be adjusted, the flow control quota = initial flow control quota = IOPS lower limit. For flow control units to be adjusted, the adjusted IOPS upper limit is the dynamically adjusted flow control quota.

[0050] Furthermore, to facilitate understanding of the above dynamic allocation method for flow control quotas, an example will be used to illustrate it below.

[0051] Assuming the table's IOPS quota is 4000 and the table has 4 partitions, the lower limit for IOPS for each partition's flow control unit is 1000. After initializing the table's state, assuming the actual IOPS values ​​of the 4 partition flow control units are 500, 100, 1800, and 1500 respectively, the table's actual IOPS value is 3900, triggering the dynamic allocation condition. Since the actual IOPS values ​​of the first two partition flow control units are less than the lower limit, i.e., less than the initial flow control quota, no adjustment of the flow control quota is needed. Furthermore, for the flow control units to be adjusted, the sum of the actual IOPS values ​​of the units to be adjusted is calculated, i.e. Then calculate the allocable IOPS quota, i.e. Finally, the dynamically adjusted IOPS quotas for the third and fourth partition flow control units are calculated, which is also the dynamically adjusted IOPS upper limit. , .

[0052] Furthermore, multiple dynamic adjustments can be made according to the above steps until the condition for releasing dynamic allocation is met, at which point the initialization state is restored. For example, the condition for releasing dynamic allocation may be: the sum of the actual IOPS values ​​of each partition flow control unit is less than a third preset threshold, where the third preset threshold may be 70%*L.

[0053] Compared with the prior art, the embodiments of the present invention adopt a combination of average allocation and dynamic adjustment, which can stabilize the actual value of the flow control parameters of a single table near the flow control quota of the table flow control unit, thereby making full use of the total flow control quota of the table; in addition, the embodiments of the present invention can prioritize ensuring that each partition has at least 1 / n of the table's flow control quota, ensuring the basic flow control capability of each partition, and ultimately achieving dynamic balance of the flow control capability of each partition.

[0054] In an optional implementation, when the target data layer is a lower-level data layer and is configured with multithreading, setting the flow control unit in the target data layer includes: In the flow control unit of the underlying data layer, a sub-flow control unit is set up for each thread.

[0055] It's worth noting that in a unified key-value (KV) storage system, the underlying data layer is the partition layer. Understandably, to fully utilize the instance resources of the unified KV storage system, such as CPU (Central Processing Unit), memory, disk, and network, these resources are typically evenly distributed among each thread in a partition for data processing. For example, when a partition is configured with 8 IO threads, IO requests entering the partition will be evenly distributed across the 8 threads for processing. In this technical context, when multiple threads access the same flow control unit, lock contention can occur. When IO capacity is high, lock contention in the flow control unit can become an IO bottleneck.

[0056] See Figure 4 To address the aforementioned technical problems, in this embodiment of the invention, a plurality of sub-flow control units are set on the partition flow control unit. The number of sub-flow control units is equal to the total number of threads in the partition. Specific flow limiting tasks, such as checking remaining flow control quotas and allowing I / O requests, are distributed to each sub-flow control unit. The partition flow control unit provides an external interface to receive the flow control quotas set by the user for the partition and distributes these quotas evenly among the sub-flow control units. The sub-flow control units are not exposed externally. Furthermore, after an I / O request enters the partition flow control unit, it is flow-limited by the sub-flow control unit of that thread based on its thread ID.

[0057] Compared with the prior art, the embodiments of the present invention distribute the rate limiting work of the underlying data layer to each sub-flow control unit, which enables each thread to perform rate limiting independently, avoiding lock conflicts caused by multiple threads accessing the same flow control unit, thereby avoiding IO capacity bottlenecks caused by setting flow control units.

[0058] In one alternative implementation, the sub-flow control unit is used for: Upon receiving an I / O request, determine whether the I / O request falls within the scope of the sampling inspection: If so, rate-limit the I / O requests; If not, allow the IO request directly.

[0059] It is worth noting that in existing technologies, a complete rate limiting process is executed for each IO request, including obtaining the timestamp, calculating the remaining flow control quota, and determining whether to allow the request. In low-speed scenarios, the overhead of this method is not significant. However, in high-performance scenarios, such as when a unified KV storage system uses high-performance disks like NVMe SSDs (Non-Volatile Memory Express Solid State Drives), the latency of its backend processing is very low, which amplifies the overhead of rate limiting for a single IO request. If each IO request goes through the rate limiting process, it will result in an overall performance loss of about 6%.

[0060] To further reduce the performance overhead of rate limiting, this embodiment of the invention employs a sampling rate limiting strategy. Specifically, a sampling ratio is set for each IO thread, for example, 80%. Then, the sub-flow control unit of the IO thread only performs flow control on IO requests within this ratio. That is, after receiving an IO request, the sub-flow control unit first determines whether the IO request is within the sampling range. If it is, rate limiting is applied to the IO request, i.e., a rate limiting process is executed to determine whether to allow the IO request; if the IO request is not within the sampling range, the IO request is directly allowed.

[0061] Compared with the prior art, the embodiments of the present invention can avoid the performance loss caused by rate limiting one by one by sampling the IO requests entering the sub-flow control unit and reduce the latency in high-performance scenarios by sampling the IO requests entering the sub-flow control unit.

[0062] In one optional implementation, determining whether an I / O request is within the sampling scope includes: Determine if the ID of the IO request is within the target range: If so, the IO request is determined to be within the sampling inspection range; If not, the IO request is determined to be outside the scope of the sampling inspection.

[0063] It is worth noting that, since concurrency can occur on each thread, this embodiment of the invention does not sample IO requests one by one, but rather samples a batch of consecutive IO requests each time, for example, the first 800 out of every 1000 IO requests. Because the IO concurrency is far less than 800, it can be ensured that for a period of time during the 800 consecutive flow control periods, IO requests are fully under flow control. For example, see [link to example]. Figure 5 This is a schematic diagram of sampling flow control in an embodiment of the present invention. In the diagram, the target range is [0, 800]. Based on whether the last three digits of the ID of the IO request fall within the target range, it can be determined whether the current IO request is in the sampled batch. If it is in the sampled batch, the sub-flow control unit will perform flow control on the IO request. If it is not in the sampled batch, it will be allowed directly.

[0064] Compared with existing technologies, the embodiments of the present invention can effectively balance the efficiency and effectiveness of flow control by continuously sampling flow control of IO requests, and can meet the low latency requirements of high-performance scenarios.

[0065] Compared with existing technologies, the hierarchical flow limiting device of this invention sets up a flow control unit at the target data layer to construct a multi-level flow control structure; it allocates the flow control quota of the upper-level flow control unit to the associated lower-level flow control unit; and it performs step-by-step flow limiting on the input IO requests based on each of the flow control units. This invention constructs a multi-level flow control structure based on the data organization architecture and performs step-by-step flow limiting when IO requests are input, enabling finer-grained flow limiting and meeting the flow limiting requirements of high-performance scenarios.

[0066] See Figure 6 This invention also provides a graded current limiting device, comprising: The multi-level flow control module 21 is used to set up flow control units in the target data layer to construct a multi-level flow control structure; Quota module 22 is used to allocate flow control quotas from upper-level flow control units to associated lower-level flow control units; The rate limiting module 23 is used to perform level-by-level rate limiting on the input IO requests based on each of the flow control units.

[0067] It is worth noting that the specific working process of the graded current limiting device can be referred to the working process of the graded current limiting method described in the above embodiments, and will not be repeated here.

[0068] In one optional implementation, when the flow control unit is in the enabled state, the flow control unit is specifically used for: Calculate the remaining flow control quota based on the flow control quota and the current timestamp; Release IO requests in the pending queue according to the remaining flow control quota, and update the remaining flow control quota; When the updated remaining flow control quota is greater than 0, allow the input IO request; When the updated remaining flow control quota is not greater than 0, the input IO request is added to the pending queue.

[0069] In an optional implementation, when the flow control unit is in the enabled state, the flow control unit is further configured to: When there are IO requests in the pending queue, the remaining flow control quota is updated periodically; When the updated remaining flow control quota is greater than 0, the IO requests in the suspended queue are allowed to proceed according to the updated remaining flow control quota.

[0070] In an optional implementation, when the upper-layer flow control unit is a surface layer, the quota module 22 is specifically used for: The flow control quotas of the upper-level flow control units are evenly distributed to the associated lower-level flow control units as the initial flow control quotas; After the dynamic allocation condition is triggered, the allocable flow control quota is allocated according to the proportion of each flow control unit to be adjusted and then superimposed on the initial flow control quota. The allocatable flow control quota is obtained by subtracting the initial flow control quota of the flow control unit to be adjusted and the actual value of the flow control parameters of the non-flow control units from the flow control quota of the upper-level flow control unit; the flow control unit to be adjusted is the lower-level flow control unit whose actual value of the flow control parameters is greater than the initial flow control quota.

[0071] In an optional implementation, when the target data layer is a lower-level data layer and is configured with multi-threading, the multi-level flow control module 21 is specifically used for: In the flow control unit of the underlying data layer, a sub-flow control unit is set up for each thread.

[0072] In one alternative implementation, the sub-flow control unit is used for: Upon receiving an I / O request, determine whether the I / O request falls within the scope of the sampling inspection: If so, rate-limit the I / O requests; If not, allow the IO request directly.

[0073] In one optional implementation, determining whether an I / O request is within the sampling scope includes: Determine if the ID of the IO request is within the target range: If so, the IO request is determined to be within the sampling inspection range; If not, the IO request is determined to be outside the scope of the sampling inspection.

[0074] In one alternative implementation, the target data layer includes one or more of an instance layer, a table layer, and a partition layer.

[0075] Compared with existing technologies, the hierarchical flow limiting device of this invention sets up a flow control unit at the target data layer to construct a multi-level flow control structure; it allocates the flow control quota of the upper-level flow control unit to the associated lower-level flow control unit; and it performs step-by-step flow limiting on the input IO requests based on each of the flow control units. This invention constructs a multi-level flow control structure based on the data organization architecture and performs step-by-step flow limiting when IO requests are input, enabling finer-grained flow limiting and meeting the flow limiting requirements of high-performance scenarios.

[0076] See Figure 7 , Figure 7This is a schematic diagram of a tiered current limiting device provided in an embodiment of the present invention. The tiered current limiting device includes a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31. When the processor 31 executes the computer program, it implements the steps as described in the above-described tiered current limiting method embodiment, for example... Figure 1 The steps S11 to S13 described above; or, when the processor 31 executes the computer program, it implements the functions of each module in the above-described device embodiments.

[0077] For example, the computer program can be divided into one or more modules, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the graded current limiting device. For example, the computer program can be divided into multiple modules, each with the following specific functions: The multi-level flow control module 21 is used to set up flow control units in the target data layer to construct a multi-level flow control structure; Quota module 22 is used to allocate flow control quotas from upper-level flow control units to associated lower-level flow control units; The rate limiting module 23 is used to perform level-by-level rate limiting on the input IO requests based on each of the flow control units.

[0078] The specific working process of each module can be referred to the working process of the graded current limiting device described in the above embodiments, and will not be repeated here.

[0079] The tiered rate limiting device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The tiered rate limiting device may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will understand that the tiered rate limiting device may also include input / output devices, network access devices, buses, etc.

[0080] The processor 31 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 31 is the control center of the graded current limiting device, connecting various parts of the device via various interfaces and lines.

[0081] The memory 32 can be used to store the computer program and / or modules. The processor 31 implements various functions of the graded current limiting device by running or executing the computer program and / or modules stored in the memory 32 and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory 32 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0082] If the integrated module of the graded current limiting device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 31, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0083] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the graded rate limiting method as described in any of the above embodiments.

[0084] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A graded current limiting method, characterized in that, include: Flow control units are set up in the target data layer to construct a multi-level flow control structure; The flow control quotas of the upper-level flow control unit are allocated to the associated lower-level flow control units; Each of the aforementioned flow control units performs level-by-level flow limiting on the input I / O requests.

2. The graded current limiting method as described in claim 1, characterized in that, When the flow control unit is enabled, the flow limiting of input I / O requests based on each flow control unit includes: Calculate the remaining flow control quota based on the flow control quota and the current timestamp; Release IO requests in the pending queue according to the remaining flow control quota, and update the remaining flow control quota; When the updated remaining flow control quota is greater than 0, allow the input IO request; When the updated remaining flow control quota is not greater than 0, the input IO request is added to the pending queue.

3. The graded current limiting method as described in claim 1, characterized in that, When the flow control unit is in the enabled state, the flow control unit is also used for: When there are IO requests in the pending queue, the remaining flow control quota is updated periodically; When the updated remaining flow control quota is greater than 0, the IO requests in the suspended queue are allowed to proceed according to the updated remaining flow control quota.

4. The graded current limiting method as described in claim 1, characterized in that, When the upper-layer flow control unit is a surface layer, the step of allocating the flow control quota of the upper-layer flow control unit to the associated lower-layer flow control unit includes: The flow control quotas of the upper-level flow control units are evenly distributed to the associated lower-level flow control units as the initial flow control quotas; After the dynamic allocation condition is triggered, the allocable flow control quota is allocated according to the proportion of each flow control unit to be adjusted and then superimposed on the initial flow control quota. The allocatable flow control quota is obtained by subtracting the initial flow control quota of the flow control unit to be adjusted and the actual value of the flow control parameters of the non-flow control units from the flow control quota of the upper-level flow control unit; the flow control unit to be adjusted is the lower-level flow control unit whose actual value of the flow control parameters is greater than the initial flow control quota.

5. The graded current limiting method as described in claim 1, characterized in that, When the target data layer is a lower-level data layer and is configured with multi-threading, setting the flow control unit in the target data layer includes: In the flow control unit of the underlying data layer, a sub-flow control unit is set up for each thread.

6. The graded current limiting method as described in claim 5, characterized in that, The sub-flow control unit is used for: Upon receiving an I / O request, determine whether the I / O request falls within the scope of the sampling inspection: If so, rate-limit the I / O requests; If not, allow the IO request directly.

7. The graded current limiting method as described in claim 6, characterized in that, The determination of whether an IO request is within the sampling scope includes: Determine if the ID of the IO request is within the target range: If so, the IO request is determined to be within the sampling inspection range; If not, the IO request is determined to be outside the scope of the sampling inspection.

8. The graded current limiting method as described in claim 1, characterized in that, The target data layer includes one or more of the instance layer, table layer, and partition layer.

9. A graded current limiting device, characterized in that, include: A multi-level flow control module is used to set up flow control units at the target data layer to build a multi-level flow control structure; The quota module is used to allocate flow control quotas from upper-level flow control units to associated lower-level flow control units. The rate limiting module is used to perform level-by-level rate limiting on the input IO requests based on each of the flow control units.

10. A graded current limiting device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the graded current limiting method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the graded rate limiting method as described in any one of claims 1 to 8.

12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the graded rate limiting method as described in any one of claims 1 to 8.

Images