Network cluster and tenant aware hierarchical processing system for distributed quality of service

Abstract

The present invention relates to a quality of service (QoS) of a service. The present invention provides a QoS processing system for one or more services, the QoS processing system comprising a plurality of QoS processing nodes, the plurality of QoS processing nodes being arranged in a hierarchical tree structure having at least two hierarchical levels. The highest hierarchical level includes one or more root nodes, each root node associated with a set of service instances of the service. The lowest hierarchy level comprises a plurality of leaf nodes, each leaf node being associated with one service instance, each leaf node being derived from at least one root node, and the service instance of the set of each root node being associated with the leaf node derived from the root node. Each leaf node may apply a local QoS policy to the associated service instance of each leaf node, and each root node may apply a first common QoS policy to the set of service instances of each root node.

Description

Technical Field

[0001] The present invention relates to a quality of service (QoS) strategy for a service. The present invention provides a QoS processing system for one or more services. The QoS processing system is based on a hierarchical tree structure of QoS processing nodes. The QoS processing system can be used to provide distributed QoS on service instances of one or more services in a network or for different tenants. Background Art

[0002] With the use of microservices, applications can be horizontally scaled simply by adding more server instances to the production cluster. However, horizontal scaling brings many challenges to the service infrastructure, such as how to apply QoS policies (e.g., large-scale policies) to these instances without affecting the overall service performance.

[0003] Currently, QoS schemes usually rely on distributed storage to make collective QoS decisions based on the aggregated information of all active nodes. This approach can also perform well in large-scale multi-region applications, but it is not suitable for multi-tenant service deployments, where QoS policies need to be applied at the tenant and global levels. When all these policies share distributed storage for decision making, the noisy neighbor effect needs to be addressed. In large service deployments, multiple distributed storage systems are used to mitigate this problem. Therefore, this model has its own costs and management overhead.

[0004] Most traditional rate limiter (QoS parameter) schemes are specifically designed to rate limit queries for a specific service or service instance. A traditional approach is to deploy multiple instances of the rate limiter scheme (e.g., based on a distributed architecture) to distribute the rate limiter load. However, managing such a deployment in a large-scale shared cloud system is quite cumbersome. Therefore, the main disadvantages of traditional rate limiter schemes are: (1) dependence on third-party distributed storage solutions; (2) scalability; (3) applicability to multi-tenant QoS use cases, such as rate limiting for OBS; and (4) maintenance, as most schemes require maintaining a series of different components for QoS processing. Summary of the invention

[0005] The purpose of the present invention is to design a QoS processing system that overcomes the above-mentioned drawbacks. For example, a QoS processing system is needed that can handle ultra-large-scale multi-tenant workloads without using multiple QoS service deployments.

[0006] These and other objects are achieved by the present invention as described in the independent claims. Advantageous implementations are further described in the dependent claims.

[0007] A first aspect of the present invention provides a QoS processing system for one or more services, the QoS processing system comprising: a plurality of QoS processing nodes arranged in a hierarchical tree structure comprising at least two hierarchical levels; the highest hierarchical level of the at least two hierarchical levels comprises one or more root nodes of the tree structure of the QoS processing nodes, each root node being associated with a set of service instances of the one or more services; the lowest hierarchical level of the at least two hierarchical levels comprises a plurality of leaf nodes of the tree structure of the QoS processing nodes, each leaf node being associated with a service instance of the one or more services, each leaf node being derived from at least one root node, and the service instances in the service instance set of each root node are associated with the leaf nodes derived from the root node; each leaf node is used to apply a local QoS policy to the associated service instance of each leaf node; and each root node is used to apply a first common QoS policy to the set of service instances of each root node.

[0008] Depending on how the hierarchical tree structure specifically designs the processing nodes and the connections between the processing nodes, each single root node can be associated with multiple leaf nodes and thus multiple service instances (because each leaf node is associated with one service instance).

[0009] Using the hierarchical tree structure of the QoS processing system of the first aspect to provide local and / or public QoS policies to service instances of the one or more services provides several advantages. The QoS processing system of the present invention does not rely on a third-party distributed storage solution. In addition, due to the hierarchical approach, the QoS processing system has good scalability. Hierarchical levels can be added and deleted as needed. The QoS processing system is also suitable for multi-tenant QoS use cases, such as rate limiting for object storage (OBS). The QoS processing system is easy to maintain because it does not require maintaining a series of different components for QoS processing. The QoS processing system can handle ultra-large-scale multi-tenant workloads without using multiple QoS service deployments.

[0010] In an implementation of the first aspect, the hierarchical tree structure includes at least three hierarchical levels, and the QoS processing system also includes: at least one intermediate hierarchical level, a plurality of intermediate nodes arranged between the highest hierarchical level and the lowest hierarchical level and including the QoS processing node, each intermediate node is derived from at least one root node, each intermediate node is associated with a subset of service instances of the one or more services, and the service instances in the service instance subset of each intermediate node are associated with leaf nodes derived from the intermediate node; each intermediate node is used to apply a second common QoS policy to the subset of service instances of each intermediate node.

[0011] The hierarchical tree structure may also include 4 or 5 hierarchical levels, each of which may include multiple intermediate nodes. The number of hierarchical levels is unlimited, making the QoS processing system highly scalable. Different intermediate hierarchical levels may be associated with hosts, clusters, AZs, etc. Depending on the number of hierarchical levels, more than two common QoS policies may be applied. At each higher hierarchical level, a common QoS policy for a larger set of service instances may be applied.

[0012] In an implementation of the first aspect, each specific QoS processing node in the QoS processing nodes is used to: collect at least one QoS indicator of at least one service instance from each child node in the QoS processing node, each child node being directly derived from the specific QoS processing node; aggregate the collected QoS indicators; and provide the aggregated QoS indicators to each parent node in the QoS processing nodes, the specific QoS processing node being directly derived from each parent node.

[0013] In this way, relevant QoS indicators can be distributed to all QoS processing nodes at all hierarchical levels. The root node can receive all QoS indicators and make QoS policy decisions based on these QoS indicators.

[0014] In an implementation of the first aspect, each specific QoS processing node among the QoS processing nodes is used to: receive the QoS configuration of at least one service instance from each parent node among the QoS processing nodes, and the specific QoS processing node is directly derived from each parent node; and / or provide the received QoS configuration and / or the QoS configuration of at least one service instance of itself to each child node of the QoS processing node, and each child node is directly derived from the specific QoS processing node.

[0015] Thus, each QoS processing node at each hierarchical level may receive a QoS configuration and may apply a local or common QoS policy, for example, based on the QoS configuration.

[0016] In an implementation of the first aspect, each root node is used to distribute at least one rate limiting parameter of at least one service instance to each QoS processing node derived from the root node.

[0017] Therefore, a rate limiting scheme can be implemented by the QoS processing system.

[0018] In an implementation of the first aspect, each QoS policy, including each local QoS policy, each first public QoS policy and / or each second public QoS policy, is applied based on at least one of the following: one or more QoS indicators among the QoS indicators of the one or more service instances to which the QoS policy is applied; one or more QoS configurations among the QoS configurations of the one or more service instances to which the QoS policy is applied; one or more rate limiting parameters among the rate limiting parameters of the one or more service instances to which the QoS policy is applied.

[0019] QoS indicators and QoS configurations may be propagated and distributed through the hierarchical levels as described above, and QoS policies may be applied accordingly in a hierarchical manner.

[0020] In an implementation manner of the first aspect, the local QoS policy, the first public QoS policy and / or the second public QoS policy include a rate limiting policy.

[0021] In an implementation of the first aspect, the QoS processing system further includes a control node, and the control node is used to allocate and / or reallocate one or more QoS processing nodes among the multiple QoS processing nodes in the hierarchical tree structure.

[0022] Therefore, the control node may select the optimal QoS processing node based on using some suitable algorithms.

[0023] In an implementation manner of the first aspect, the control node is a QoS processing node among the QoS processing nodes in the hierarchical tree structure.

[0024] In an implementation of the first aspect, the control node is used to perform the allocation and / or the reallocation based on at least one of the following: the load on the currently allocated QoS processing node, the QoS policy applied by the currently allocated QoS processing node, the communication cost between the currently allocated QoS processing nodes, the topology of the service instance, and the QoS configuration of each service instance in the service instance.

[0025] In an implementation of the first aspect, each QoS processing node in the hierarchical tree structure knows its position in the tree structure and / or includes information describing the entire hierarchical tree structure.

[0026] In an implementation of the first aspect, the QoS processing nodes in the hierarchical tree structure are used to execute a consensus protocol to determine which QoS processing node is responsible for which QoS policy and / or share one or more QoS policies using the hierarchical tree structure.

[0027] In an implementation manner of the first aspect, the control node is further used to report the allocation and / or reallocation of the one or more QoS processing nodes to all the QoS processing nodes.

[0028] The QoS processing nodes may further apply their local and / or public QoS policies based on their position in the tree structure, and / or based on the results of the consensus protocol, and / or reports of allocations and / or reallocations.

[0029] In an implementation of the first aspect, the hierarchical tree structure includes four hierarchical levels, wherein: each QoS processing node of the lowest hierarchical level is associated with a service instance of one or more services of a host among multiple hosts, and each host executes one or more service instances of the one or more services; each QoS processing node of the first intermediate hierarchical level above the lowest hierarchical level is associated with all the service instances of a host; each QoS processing node of the second intermediate hierarchical level above the first intermediate hierarchical level is associated with all the service instances of all hosts in a host cluster; and each QoS processing node of the highest hierarchical level above the second intermediate hierarchical level is associated with all the service instances of all clusters in a cluster area.

[0030] A second aspect of the present invention provides a method for QoS processing of one or more services, the method comprising: allocating or reallocating multiple QoS processing nodes in a hierarchical tree structure comprising at least two hierarchical levels; the highest hierarchical level of the at least two hierarchical levels comprises one or more root nodes of the tree structure of the QoS processing nodes, each root node being associated with a set of service instances of the one or more services; the lowest hierarchical level of the at least two hierarchical levels comprises multiple leaf nodes of the tree structure of the QoS processing nodes, each leaf node being associated with a service instance of the one or more services, each leaf node being derived from at least one root node, and the service instances in the service instance set of each root node are associated with the leaf nodes derived from the root nodes; each leaf node applies a local QoS policy to the service instance associated with the leaf node; and each root node applies a first public QoS policy to the set of service instances associated with the root node.

[0031] In an implementation of the second aspect, the hierarchical tree structure includes at least three hierarchical levels, and the method also includes: allocating or reallocating multiple intermediate nodes of the QoS processing node in at least one intermediate hierarchical level arranged between the highest hierarchical level and the lowest hierarchical level, each intermediate node is derived from at least one root node, each intermediate node is associated with a subset of service instances of the one or more services, and the service instances in the service instance subset of each intermediate node are associated with leaf nodes derived from the intermediate node; each intermediate node is used to apply a second common QoS policy to the subset of service instances of each intermediate node.

[0032] In an implementation of the second aspect, the method includes: utilizing each specific QoS processing node among the QoS processing nodes, collecting at least one QoS indicator of at least one service instance from each child node of the QoS processing node, each child node being directly derived from the specific QoS processing node; aggregating the collected QoS indicators; and providing the aggregated QoS indicators to each parent node among the QoS processing nodes, the specific QoS processing node being directly derived from each parent node.

[0033] In an implementation of the second aspect, the method includes: using each specific QoS processing node among the QoS processing nodes to receive the QoS configuration of at least one service instance from each parent node among the QoS processing nodes, and the specific QoS processing node is directly derived from each parent node; and / or providing the received QoS configuration and / or the QoS configuration of at least one service instance of its own to each child node of the QoS processing node, and each child node is directly derived from the specific QoS processing node.

[0034] In an implementation of the second aspect, each root node distributes at least one rate limiting parameter of at least one service instance to each QoS processing node derived from the root node.

[0035] In an implementation of the second aspect, each QoS policy, including each local QoS policy, each first public QoS policy and / or each second public QoS policy, is applied based on at least one of the following: one or more QoS indicators among the QoS indicators of the one or more service instances to which the QoS policy is applied; one or more QoS configurations among the QoS configurations of the one or more service instances to which the QoS policy is applied; one or more rate limiting parameters among the rate limiting parameters of the one or more service instances to which the QoS policy is applied.

[0036] In an implementation manner of the second aspect, the local QoS policy, the first public QoS policy and / or the second public QoS policy include a rate limiting policy.

[0037] In an implementation manner of the second aspect, the method further includes: using a control node to allocate and / or reallocate one or more QoS processing nodes among the multiple QoS processing nodes in the hierarchical tree structure.

[0038] In an implementation manner of the second aspect, the control node is a QoS processing node among the QoS processing nodes in the hierarchical tree structure.

[0039] In an implementation of the second aspect, the control node performs the allocation and / or the reallocation based on at least one of the following: the load on the currently allocated QoS processing node, the QoS policy applied by the currently allocated QoS processing node, the communication cost between the currently allocated QoS processing nodes, the topology of the service instance, and the QoS configuration of each service instance in the service instance.

[0040] In an implementation of the second aspect, each QoS processing node in the hierarchical tree structure knows its position in the tree structure and / or includes information describing the entire hierarchical tree structure.

[0041] In an implementation of the second aspect, the QoS processing nodes in the hierarchical tree structure execute a consensus protocol to determine which QoS processing node is responsible for which QoS policy and / or share one or more QoS policies using the hierarchical tree structure.

[0042] In an implementation manner of the second aspect, the control node reports the allocation and / or reallocation of the one or more QoS processing nodes to all the QoS processing nodes.

[0043] In an implementation of the second aspect, the hierarchical tree structure includes four hierarchical levels, wherein: each QoS processing node of the lowest hierarchical level is associated with a service instance of one or more services of a host among multiple hosts, and each host executes one or more service instances of the one or more services; each QoS processing node of the first intermediate hierarchical level above the lowest hierarchical level is associated with all the service instances of a host; each QoS processing node of the second intermediate hierarchical level above the first intermediate hierarchical level is associated with all the service instances of all hosts in a host cluster; and each QoS processing node of the highest hierarchical level above the second intermediate hierarchical level is associated with all the service instances of all clusters in a cluster area.

[0044] The method described in the second aspect and its implementation method achieves the same advantages as the QoS processing system described in the first aspect and its corresponding implementation method.

[0045] A third aspect of the present invention provides a computer program comprising instructions, and when the program is executed by a computer, the instructions cause the computer to perform the method according to the second aspect or any implementation thereof.

[0046] A fourth aspect of the present invention provides a non-transitory storage medium, which stores executable program code. When the executable program code is executed by a processor, it performs the method according to the second aspect or any one of its implementations.

[0047] In summary, the present invention proposes a hierarchical QoS processing system using QoS processing nodes in a hierarchical tree architecture, and uses aggregation at different hierarchical levels to make QoS policy decisions. In addition, the present invention proposes an algorithm for allocating or reallocating QoS processing nodes to process QoS policies for different service instances. Various factors (e.g., the load on the QoS processing nodes, the QoS policies currently processed by the QoS processing nodes, and the communication cost with a given QoS processing node) are considered to determine which QoS processing nodes should be responsible for processing specific QoS policies for services belonging to different tenants. The present invention also proposes an aggregation pattern in each hierarchical level of the tree structure to better utilize resources.

[0048] It should be noted that all devices, elements, units and devices described in this application can be implemented in software or hardware elements or any combination thereof. All steps performed by various entities described in this application and the functions described to be performed by various entities are intended to indicate that the corresponding entities are used to perform the corresponding steps and functions. Although in the description of the following specific embodiments, the specific functions or steps performed by the external entity are not reflected in the description of the specific detailed elements of the entity performing the specific steps or functions, it should be clear to the technician that these methods and functions can be implemented by corresponding software or hardware elements or any combination thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In conjunction with the accompanying drawings, the following description of specific embodiments will illustrate the above aspects and their implementation methods. In the drawings:

[0050] Figure 1 The QoS processing system provided by the present invention has at least two hierarchical levels.

[0051] Figure 2 The QoS processing system with five hierarchical levels provided by the present invention is shown.

[0052] Figure 3 The QoS processing system with four levels provided by the present invention is shown.

[0053] Figure 4 Shows Figure 3 Local QoS policy processing at a service instance in the QoS processing system.

[0054] Figure 5 Shows Figure 3 Host-level QoS policy processing in the QoS processing system.

[0055] Figure 6 Shows Figure 3 Cluster-level QoS policy processing in the QoS processing system.

[0056] Figure 7 Shows Figure 3 Regional-level QoS policy processing in the QoS processing system.

[0057] Figure 8 The method for QoS processing of one or more services provided by the present invention is shown. DETAILED DESCRIPTION

[0058] Figure 1The QoS processing system 100 provided by the present invention is shown. The QoS processing system 100 is used to process the QoS of one or more services, specifically, by applying one or more QoS policies to service instances 105 of one or more services. The QoS processing system 100 includes a plurality of QoS processing nodes 101, 102, and the plurality of QoS processing nodes 101, 102 are arranged in a hierarchical tree structure including at least two hierarchical levels 103, 104. Figure 1 In FIG. 1 , two hierarchical levels 103 and 104 are precisely shown. However, this is only an example, and the hierarchical tree structure may include three hierarchical levels, or four hierarchical levels (eg, Figures 3 to 7 ), or five hierarchical levels (as shown in Figure 2 ), or even more hierarchical levels.

[0059] like Figure 1 As shown, the highest hierarchical level 103 of at least two hierarchical levels 103, 104 includes one or more root nodes 101 of a hierarchical tree structure of QoS processing nodes 101, 102. Each root node 101 is associated with a set of service instances 105 of one or more services. In addition, the lowest hierarchical level 104 of at least two hierarchical levels 103, 104 includes a plurality of leaf nodes 102 of the tree structure of QoS processing nodes 101, 102. Each leaf node 102 is associated with a service instance 105 of one or more services, each leaf node 102 is derived from at least one root node 101, and a service instance 105 in the set of service instances 105 of each root node 101 is associated with a leaf node 102 derived from the root node 101.

[0060] In the hierarchical tree structure of the QoS processing system, each leaf node 102 is used to apply a local QoS policy to the associated service instance 105 of each leaf node 102. In addition, each root node 101 is used to apply a first common QoS policy to the set of service instances 105 of each root node 101. Therefore, the local QoS policy is only applied to the associated service instance 105, while the common QoS policy is applied to multiple service instances 105 in a common manner.

[0061] Each QoS processing node 101, 102 may include a processor or processing circuit (not shown) for executing, performing or initiating various operations of the QoS processing nodes 101, 102 described herein. The processing circuit may include hardware and / or the processing circuit may be controlled by software. The hardware may include analog circuits or digital circuits, or both analog circuits and digital circuits. The digital circuit may include components such as an application-specific integrated circuit (ASIC), a field-programmable array (FPGA), a digital signal processor (DSP) or a multi-purpose processor. Each QoS processing node 101, 102 may also include a memory circuit that stores one or more instructions that can be executed by a processor or processing circuit (specifically, executed under the control of software). For example, the memory circuit may include a non-transitory storage medium storing executable software code, which enables various operations of the QoS processing nodes 101, 102 to be executed when executed by a processor or processing circuit. In one embodiment, the processing circuit includes one or more processors and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code that, when executed by the one or more processors, causes the corresponding QoS processing node 101, 102 to perform, perform, or initiate the operations or methods described herein.

[0062] According to the present invention, two main principles can be used. First, multi-level hierarchical QoS processing nodes 101, 102. QoS processing nodes 101, 102 can be used for rate limiter processing. Second, dynamic sharding can be implemented to achieve shared-nothing (SN) in, for example, multi-tenant deployments.

[0063] For example, the QoS processing system 100 may be a rate limiter system having a series of QoS processing nodes 101, 102 connected in a hierarchical order (i.e., in a hierarchical tree structure). The QoS processing nodes 101, 102 at each hierarchical level 103, 104 may communicate only with their corresponding parent nodes (the parent node is a QoS processing node arranged at a higher hierarchical level, from which the QoS processing nodes 101, 102 are derived) or their corresponding child nodes (the child node is a QoS processing node at a lower hierarchical level, from which the QoS processing nodes 101, 102 are derived), which communication may be generally referred to as "north-south traffic". Adjacent QoS processing nodes 101, 102 at the same hierarchical level 103, 104 may not interact with each other to make any QoS policy decisions. Instead, they may communicate with their parent nodes or child nodes, respectively.

[0064] Each QoS processing node 101, 102 in each hierarchical level 103, 104 of the hierarchical tree structure may perform one or more of the following operations:

[0065] Collect relevant QoS metrics from one or more of its child nodes.

[0066] The collected QoS metrics are aggregated, for example, based on one or more QoS policies (eg, rate limiter policies).

[0067] For example, as configured by the QoS policy, the aggregated QoS indicator is reported to one or more of its parent nodes.

[0068] Polls one or more QoS configurations from its parent node or nodes.

[0069] Prepares a subset of one or more QoS configurations to be polled and pushes the subset to one or more of its child nodes.

[0070] The leaf nodes 102 of the hierarchical tree structure are arranged at the service instances 105, where local QoS policies can be applied. QoS policies (such as rate limiter policies) can be determined, for example, by the customer at the root node 101. These QoS policies (such as rate limiter policies) can be propagated down to the leaf nodes 102 (associated with the service instances 105) at regular intervals to make QoS decisions locally. Each QoS processing node 101, 102 in the hierarchical tree structure can know its position (in the hierarchical structure) and can always have a local copy of the hierarchical tree structure. When the hierarchical tree structure is modified, the modification can be reported to all QoS processing nodes 101, 102 on all hierarchical levels 103, 104.

[0071] When one or more service instances 105 are started or stopped, the information can be propagated upward from one or more leaf nodes to one or more root nodes 101, for example, through intermediate QoS processing nodes in a hierarchical tree structure. Then, the one or more root nodes 101 can identify the QoS policy for a given one or more services or service instances 105, and the QoS policy can be propagated downward to the QoS processing nodes 101, 102 in each hierarchical level 103, 104 including the leaf node 102. For example, the scheduler in each hierarchical level 103, 104 can determine who (e.g., which QoS processing node 101, 102) is responsible for the given service or service instance 105, and can assign the specific QoS processing node 101, 102 to the given service or service instance 105 for QoS policy processing, i.e., applying the QoS policy. Next, the QoS processing nodes 101, 102 in each hierarchical level 103, 104 can be updated with a new list of parent nodes with which they must interact. This can be determined by the scheduler using the service list that the QoS processing node 101, 102 is responsible for and the QoS policy being configured for the service. After updating the node list, QoS processing may be distributed across multiple QoS processing nodes 101 , 102 using sharding.

[0072] The QoS processing system 100 may include at least three hierarchical levels 103, 104. For example, Figure 2 An example of a QoS processing system 100 provided by the present invention having five hierarchical levels 103 , 204 , 205 , 206 , 104 is shown.

[0073] Therefore, the QoS processing system 100 may also include at least one intermediate hierarchy level 204, 205, 206 (e.g., Figure 2 104, 105, 106), the at least one intermediate hierarchical level 204, 205, 206 being arranged between the highest hierarchical level 103 and the lowest hierarchical level 104, and comprising a plurality of intermediate nodes 201, 202, 203 of the QoS processing nodes 101, 102, 202, 202, 203. Each intermediate node 201, 202, 203 is derived from at least one root node 101. Each intermediate node 201, 202, 203 is also associated with a subset of service instances 105 of one or more services, and the service instances 105 of the subset of service instances 105 of each intermediate node 201, 202, 203 are associated with the leaf nodes 102 derived from the intermediate nodes 201, 202, 203. Each intermediate node 201, 202, 203 is used to apply a second common QoS policy to the subset of service instances 105 of each intermediate node 201, 202, 203.

[0074] As in Figure 2As can be seen in the figure, QoS processing nodes 101, 102, 201, 202, 203 can be deployed at instances 105 (leaf nodes 102 at hierarchical level 104), hosts (intermediate nodes 203 at hierarchical level 206), clusters (intermediate nodes 202 at hierarchical level 205), availability zones (AZs) (intermediate nodes 204 at hierarchical level 204), and regions (root nodes 101 at the highest hierarchical level 103). One or more instance-level QoS processing nodes 102 can run as part of the application itself rather than as separate binaries. Depending on the configured QoS policies, specific assigned nodes 101, 201, 202, 203 can participate in the aggregation and processing of QoS indicators and policies for a given service.

[0075] Figures 3 to 7 Another example of a QoS processing system 100 with four hierarchical levels 103, 205, 206, 104 is shown. In this example, five cloud services (DB service 1, OBS service 2, analysis service 3, DNS service 4, and alarm service 5) are shared by the QoS processing system 100 deployed for a single region. Figure 3 It can be seen that there is a QoS processing node 102 (leaf node, which can be implemented as an embedded library) inside each service instance 105. In addition, each host has its own intermediate QoS processing node 203 for local QoS processing. At the cluster level, there is a set of intermediate QoS processing nodes 202 deployed for QoS processing. The number of QoS processing nodes 202 in each cluster can vary based on the workload (for example, the number of services and QoS policies to be processed within the cluster). At the regional level, there is another set of QoS processing root nodes 101 for performing regional-level QoS processing.

[0076] The local QoS policy processing at the service instance 105 at the lowest hierarchical level 104 may be referred to as follows Figure 4 For example, in this example, the Domain Name System (DNS) service only has a single instance 105 in the entire cloud. This means that QoS processing can be done locally within the service instance 105 itself without any distributed computing. The QoS processing system 100 can know the service topology to determine the QoS processing node 102 to perform processing.

[0077] Similarly, the alarm service can perform QoS processing locally even though it has more than one instance 105 in the cluster. This is because when the QoS policy is local (e.g., it does not require any distributed computing), only local QoS processing nodes 102 can be used. In this example, the alarm service has all the policies required to process QoS locally, so it is considered to be processed locally.

[0078] Host-level QoS policy processing can be referred to as follows Figure 5 The host level is the next hierarchical level 205 in the hierarchical tree structure for QoS processing. The host level QoS processing node 203 can be responsible for QoS processing of services in the same host. Alternatively, it can be responsible for the QoS policy configured for a specific host. The OBS service can have a policy configured for a specific host, so it can be processed locally by the host level QoS processing node 203 using data from the service instance QoS processing node 102.

[0079] Cluster-level QoS policy processing can be referred to as follows Figure 6 The next hierarchical level 205 of QoS processing is the cluster level. At the cluster level, the QoS processing system 100 can use more than one QoS processing node 202, and the workload can be sharded across these intermediate QoS processing nodes 202. The sharding algorithm can be run at the cluster level and can ensure that the intermediate QoS processing nodes 202 are not overloaded and that noisy neighbor situations exist. The sharding algorithm used can be entirely determined by the application and its deployment. Examples of available algorithms are as follows:

[0080] Service sticky sharding algorithm: All QoS policies for a given service are always processed in a single QoS processing node 202 .

[0081] Mixed slicing algorithm: The load is mixed on the QoS processing node 202 based on policies, services and even topology.

[0082] Random sharding algorithm: randomly assigning service instances 105 on cluster QoS processing nodes 202 .

[0083] In this example, the database (DB) service spans a single cluster, so any QoS policy applied on the entire DB service will be processed at the cluster level QoS processing node 202. For example, when the total number of active DB connections from a customer is greater than 100, new DB connections from a customer may be blocked. In order to apply such a QoS policy, the QoS processing system 100 may need to collect the number of DB connections for each customer from all active DB instances and aggregate these values ​​to determine whether the connection needs to be blocked. Afterwards, the decision should be published to the QoS processing node that is making the request. For the hierarchical model of this example, the aggregation of the first level indicators will occur at the host level, and then the results will be forwarded to the cluster QoS processing node 202. At the cluster level, these aggregated QoS indicators can be aggregated again to find the final value.

[0084] Reference below Figure 7 Describe zone level QoS policy processing. Similar to cluster level QoS processing, there may be a set of zone level QoS processing nodes 101 for zone level policy processing. The zone level QoS processing nodes 101 of the hierarchy level 103 may be sharded across different clusters based on various factors (e.g., service topology, configured policies, network topology, etc.).

[0085] As can be seen, the OBS service can span multiple clusters within a single region, and applying any policy to the entire OBS service may require policy management at the regional level. For example, when the overall utilization of the OBS CPU instance is greater than 70%, queries from customers may be blocked. In order to apply such a QoS policy, each service may have to report its CPU utilization to the next level QoS processing node. At each hierarchical level, data can be aggregated, and the aggregated data can be forwarded all the way to the regional level QoS processing node 101. At the regional level, final aggregation may occur and a QoS policy decision may be made. The decision can then be pushed down to the service QoS processing node to properly handle the query.

[0086] Figure 8 The method 800 provided by the present invention is shown. The method 800 is used for QoS processing of one or more services and can be performed by Figures 1 to 7 The QoS processing system 100 is shown to perform.

[0087] The method 800 comprises a step 801 of allocating (801) or reallocating a plurality of QoS processing nodes 101, 102, 201, 202, 203 in a hierarchical tree structure comprising at least two hierarchical levels 103, 104, 204, 205, 206. The highest hierarchical level 103 of the at least two hierarchical levels comprises one or more root nodes 101 of the hierarchical tree structure of QoS processing nodes, each root node 101 being associated with a set of service instances 105 of one or more services. The lowest hierarchical level 104 of the at least two hierarchical levels comprises a plurality of leaf nodes 102 of the hierarchical tree structure of QoS processing nodes, each leaf node 102 being associated with a service instance 105 of one or more services, each leaf node 102 being derived from at least one root node 101, and a service instance 105 of the set of service instances 105 of each root node 101 being associated with a leaf node 102 derived from the root node 101. The method 800 further includes step 802 : each leaf node 102 applies the local QoS policy to the service instance 105 associated with the leaf node 102 . The method 800 further includes step 803 : each root node 101 applies the first public QoS policy to the set of service instances 105 associated with the root node 101 .

[0088] The scheme of the present invention provides at least the following advantages. Scalability, compared to existing rate limiting schemes, which cannot scale at all or cannot perform QoS across regions, or even across AZs. The present invention solves this problem by processing QoS (e.g., rate limiting) in a hierarchical model. Multi-tenant rate limiting support because QoS processing nodes (e.g., rate limiting nodes) are allocated between service instances 105 based on QoS policies and service instance topology. This can ensure that QoS processing does not affect each other. It can also ensure that there are no noisy neighbor situations when using this scheme.

[0089] The solution of the present invention is suitable for large-scale microservice deployments, in which case one wants to apply rate limiting policies on a large number of instances. The solution of the present invention is also most useful for large-scale applications that are shared among multiple users or tenants who want to use the service in isolation mode. The solution of the present invention is also very useful for some large-scale deployments.

[0090] Through a single hierarchical QoS processing scheme, QoS processing can be performed across all of these instances without affecting each other. Each service instance 105 is associated with a leaf node 102 of the hierarchical tree structure. The total number of hierarchical levels 103, 104, 204, 205, 206 and the number of QoS processing nodes 101, 102, 201, 202, 203 in each hierarchical level can be entirely based on the type of QoS policy and the scale of one or more applications.

[0091] The invention has been described in conjunction with various embodiments as examples and implementations. However, from a study of the drawings, the invention and the independent claims, other variations will be understood and implemented by those skilled in the art in implementing the claimed subject matter. In the claims and in the specification, the word "comprising" does not exclude other elements or steps, and "a" does not exclude a plurality. A single element or other unit may fulfil the functions of several entities or items described in the claims. The enumeration of certain measures in mutually different dependent claims does not imply that a combination of these measures cannot be used in an advantageous implementation.

Claims

1. A quality of service (QoS) processing system (100) for one or more services, characterized in that: The QoS processing system (100) comprises: a plurality of QoS processing nodes (101, 102, 201, 202, 203) arranged in a hierarchical tree structure including at least two hierarchical levels (103, 104, 204, 205, 206); a highest hierarchical level (103) of the at least two hierarchical levels (103, 104, 204, 205, 206) comprising one or more root nodes (101) of the hierarchical tree structure of the QoS processing nodes (101, 102, 201, 202, 203), each root node (101) being associated with a set of service instances (105) of the one or more services; The lowest hierarchical level (104) of the at least two hierarchical levels (103, 104, 204, 205, 206) comprises a plurality of leaf nodes (102) of the hierarchical tree structure of the QoS processing nodes (101, 102, 201, 202, 203), each leaf node (102) being associated with a service instance (105) of the one or more services, each leaf node (102) being derived from at least one root node (101), and the service instances (105) in the set of service instances (105) of each root node (101) being associated with the leaf nodes (102) derived from the root node (101); Each leaf node (102) is used to apply a local QoS policy to the associated service instance (105) of each leaf node (102); Each root node (101) is used to apply a first common QoS policy to the set of service instances (105) of each root node (101).

2. The QoS processing system (100) according to claim 1, characterized in that: The hierarchical tree structure includes at least three hierarchical levels (103, 104, 204, 205, 206), and the QoS processing system (100) further includes: at least one intermediate hierarchical level (204, 205, 206), a plurality of intermediate nodes (201, 202, 203) arranged between the highest hierarchical level (103) and the lowest hierarchical level (104) and comprising the QoS processing nodes (101, 102, 201, 202, 203), each intermediate node (201, 202, 203) being derived from at least one root node (101), each intermediate node (201, 202, 203) being associated with a subset of service instances (105) of the one or more services, and the service instances (105) in the subset of service instances (105) of each intermediate node (201, 202, 203) being associated with leaf nodes (102) derived from the intermediate nodes (201, 202, 203); Each intermediate node (201, 202, 203) is used to apply a second common QoS policy to a subset of the service instances (105) of each intermediate node (201, 202, 203).

3. The QoS processing system (100) according to claim 1 or 2, characterized in that: Each specific QoS processing node (101, 102, 201, 202, 203) among the QoS processing nodes is used for: collecting at least one QoS indicator of at least one service instance (105) from each child node (102, 201, 202, 203) in the QoS processing node, each child node (102, 201, 202, 203) being directly derived from the specific QoS processing node (101, 102, 201, 202, 203); Aggregating the collected QoS indicators; The aggregated QoS indicator is provided to each parent node (101, 201, 202, 203) of the QoS processing nodes, and the specific QoS processing node (101, 102, 201, 202, 203) is directly derived from each parent node (101, 201, 202, 203).

4. The QoS processing system (100) according to any one of claims 1 to 3, characterized in that: Each specific QoS processing node (101, 102, 201, 202, 203) among the QoS processing nodes is used for: Receiving a QoS configuration of at least one service instance (105) from each parent node (101, 201, 202, 203) of the QoS processing nodes, the particular QoS processing node (101, 102, 201, 202, 203) being directly derived from each parent node (101, 201, 202, 203); and / or The received QoS configuration and / or the QoS configuration of at least one service instance (105) of the QoS processing node is provided to each child node (102, 201, 202, 203) of the QoS processing node, and each child node (102, 201, 202, 203) is directly derived from the specific QoS processing node (101, 102, 201, 202, 203).

5. The QoS processing system (100) according to any one of claims 1 to 4, characterized in that: Each root node (101) is used to distribute at least one rate limiting parameter of at least one service instance (105) to each QoS processing node (101, 102, 201, 202, 203) derived from the root node (101).

6. The QoS processing system (100) according to any one of claims 3 to 5, characterized in that: Each QoS policy, including each local QoS policy, each first public QoS policy and / or each second public QoS policy, is applied based on at least one of the following: one or more QoS indicators among the QoS indicators of the one or more service instances (105) to which the QoS policy is applied; one or more QoS configurations of the one or more service instances (105) to which the QoS policy is applied; One or more of the rate limiting parameters of the one or more service instances (105) to which the QoS policy is applied.

7. The QoS processing system (100) according to any one of claims 1 to 6, characterized in that: The local QoS policy, the first public QoS policy and / or the second public QoS policy include a rate limiting policy.

8. The QoS processing system (100) according to any one of claims 1 to 7, characterized in that: It also includes a control node, which is used to allocate and / or reallocate one or more QoS processing nodes among the plurality of QoS processing nodes (101, 102, 201, 202, 203) in the hierarchical tree structure.

9. The QoS processing system (100) according to claim 8, characterized in that: The control node is a QoS processing node among the QoS processing nodes (101, 102, 201, 202, 203) in the hierarchical tree structure.

10. The QoS processing system (100) according to claim 8 or 9, characterized in that: The control node is used to perform the allocation and / or the reallocation based on at least one of the following: the load on the currently allocated QoS processing nodes (101, 102, 201, 202, 203), the QoS policy applied by the currently allocated QoS processing nodes (101, 102, 201, 202, 203), the communication cost between the currently allocated QoS processing nodes (101, 102, 201, 202, 203), the topology of the service instance (105) and the QoS configuration of each service instance in the service instance (105).

11. The QoS processing system (100) according to any one of claims 1 to 10, characterized in that: Each QoS processing node (101, 102, 201, 202, 203) in the hierarchical tree structure knows its position in the hierarchical tree structure and / or includes information describing the entire hierarchical tree structure.

12. The QoS processing system (100) according to claim 11, characterized in that: The QoS processing nodes (101, 102, 201, 202, 203) in the hierarchical tree structure are used to execute a consensus protocol to determine which QoS processing node (101, 102, 201, 202, 203) is responsible for which QoS policy and / or share one or more QoS policies using the hierarchical tree structure.

13. The QoS processing system (100) according to any one of claims 8 to 12, characterized in that: The control node is further used to report the allocation and / or reallocation of the one or more QoS processing nodes (101, 102, 201, 202, 203) to all the QoS processing nodes (101, 102, 201, 202, 203).

14. The QoS processing system (100) according to any one of claims 1 to 13, characterized in that: The hierarchical tree structure includes four hierarchical levels (103, 104, 205, 206), wherein: Each QoS processing node (102) of the lowest hierarchical level (104) is associated with a service instance (105) of one or more services of a host among a plurality of hosts, each host executing one or more service instances (105) of the one or more services; Each QoS processing node (203) of a first intermediate hierarchical level (206) above said lowest hierarchical level (104) is associated with all said service instances (105) of a host; Each QoS processing node (202) of a second intermediate hierarchical level (205) above the first intermediate hierarchical level (206) is associated with all of the service instances (105) of all hosts in a host cluster; Each QoS processing node (101) of the highest hierarchical level (103) above the second intermediate hierarchical level (205) is associated with all of the service instances (105) of all clusters in a cluster region.

15. A method (800) for quality of service (QoS) processing of one or more services, characterized in that: The method (800) comprises: Allocating (801) or reallocating a plurality of QoS processing nodes (101, 102, 201, 202, 203) in a hierarchical tree structure comprising at least two hierarchical levels (103, 104, 204, 205, 206); a highest hierarchical level (103) of the at least two hierarchical levels (103, 104, 204, 205, 206) comprising one or more root nodes (101) of the hierarchical tree structure of the QoS processing nodes (101, 102, 201, 202, 203), each root node (101) being associated with a set of service instances (105) of the one or more services; The lowest hierarchical level (104) of the at least two hierarchical levels (103, 104, 204, 205, 206) comprises a plurality of leaf nodes (102) of the hierarchical tree structure of the QoS processing nodes (101, 102, 201, 202, 203), each leaf node (102) being associated with a service instance (105) of the one or more services, each leaf node (102) being derived from at least one root node (101), and the service instances (105) in the set of service instances (105) of each root node (101) being associated with the leaf nodes (102) derived from the root node (101); Each leaf node (102) applies (802) a local QoS policy to the service instance (105) associated with the leaf node (102); Each root node (101) applies (803) a first common QoS policy to the set of service instances (105) associated with the root node (101).

16. A computer program comprising instructions, characterized in that When the instructions are executed by a computer, the computer is caused to perform the method (800) according to claim 15.

Images