Auto-scaling query engine for enterprise-level big data workloads

Abstract

Aspects of the present invention disclose a method, computer program product, and system for autoscaling a query engine. The method includes one or more processors monitoring query traffic in the query engine. The method further includes the one or more processors classifying queries into a plurality of service classes based on the level of query complexity. The method further includes the one or more processors comparing the query traffic of each service class to a concurrency threshold for a maximum number of queries of the service class that are allowed to be processed simultaneously. The method further includes the one or more processors instructing autoscaling of a cluster of worker nodes, the autoscaling being to change the number of available worker nodes in the cluster based on a comparison of the query traffic over a specified time period to a specified upscaling threshold and a specified downscaling threshold.

Description

[Technical field]

[0001] The present invention relates generally to the field of query engines, and more particularly to autoscaling of query engines. [Background technology]

[0002] Modern cloud technologies provide great opportunities for data analytics. The trend to separate compute and storage means that you can scale your compute engine to meet the needs of your workload. For example, cloud object stores offered by all major cloud providers have the flexibility to store both structured and unstructured data at scale, with true elastic scaling and very high degrees of durability and reliability. Separating compute from storage and running compute on containerized technologies such as Kubernetes® (Kubernetes is a trademark of the Linux Foundation) means that you can quickly scale clusters horizontally up or down depending on the demands of your workload.

[0003] Query engine technologies such as structured query language (SQL) engines on Hadoop® (Hadoop is a trademark of the Apache Software Foundation) or object store technologies have the ability to directly reference data in external storage and provide the ability to analyze the data where it resides using existing complex SQL and business intelligence tools. Additionally, these technologies provide the ability to scale up or down the number of SQL engine computing nodes to meet the varying demands of the business across competing workloads.

[0004] Modern massively parallel processing (MPP) SQL engines typically consist of a head node and a cluster of multiple “n” worker nodes, where n can be hundreds or even thousands of nodes. One of the key challenges that surface in this elastic paradigm is the ability to scale a big data SQL engine cluster in an orderly manner so that the cluster remains responsive without being overly sensitive to changes in workload patterns. Scaling an SQL engine to accommodate a variety of complex workloads based on simple host-based metrics such as central processing unit (CPU) or memory usage can be problematic. For example, CPU utilization can be unpredictable and even the execution of a single complex SQL query can change very rapidly depending on which stage of that query is currently in progress. Scaling a cluster in such a simplistic manner can result in thrashing, where the number of worker nodes fluctuates erratically due to unpredictable CPU metrics. Thrashing can have a detrimental effect on the stability of the entire cluster and, in extreme cases, can make autoscaling unusable.

[0005] Another challenge is the ability to drain a worker node of active queries quickly, but without interrupting ongoing queries, when downscaling a cluster. For example, at any given time, one or two long-running queries may be in progress on a worker node. Once a query fragment starts on a given worker, it is bound to that worker and cannot be moved to other workers. SQL query access plans are inherently complex and contain numerous stages involving operations such as hash joins, nested loop joins, and sort operations. When a complex query is distributed to available worker nodes, each worker’s participation in the query execution is not gated solely by disk input / output (IO). In other words, transferring a block of data read from storage to another worker does not eliminate the worker’s participation in the remaining parts of the query, because the worker may be participating in streamed operations of shuffle or hash join across the cluster. Summary of the Invention

[0006] Aspects of the present invention disclose methods, computer program products, and systems for autoscaling a query engine. The method includes one or more processors monitoring query traffic at the query engine. The method further includes one or more processors classifying queries of the query traffic by a plurality of service classes based on a level of query complexity. The method further includes one or more processors comparing the query traffic of each service class to a concurrency threshold of a maximum number of queries of the service class that are allowed to be processed simultaneously. The method further includes one or more processors instructing autoscaling of a cluster of worker nodes, the autoscaling being to change a number of worker nodes of available worker nodes in the cluster based on a comparison of the query traffic over a specified time period to a specified upscaling threshold and a specified downscaling threshold. The autoscaling adds or removes worker nodes from the cluster of worker nodes that process queries of the query engine.

[0007] Thus, embodiments of the present invention may provide the advantage of being able to scale a cluster of worker nodes up or down depending on the needs of workloads of various complexities.

[0008] In other embodiments, the specified upscaling threshold and the specified downscaling threshold may each be a specified threshold ratio of the number of queries in the query traffic compared to a concurrency threshold, such that the workflow of a service class may be determined relative to the capacity of that class.

[0009] In other embodiments, the method may include one or more processors evaluating a number of worker nodes to add or remove based on an aggregate comparison of query traffic across all service classes at a given time. Evaluating the number of worker nodes to add or remove may include, for each service class that violates a specified upscaling threshold or a specified downscaling threshold, basing the number of worker nodes to add or remove on a current proportion of worker nodes assigned to the service class and an increase or decrease in required capacity based on the comparison.

[0010] In other embodiments, the method may include one or more processors adjusting a concurrency threshold for one or more service classes based on the number of new worker nodes in the cluster after the automatic scaling, thereby automatically updating the threshold after the scaling occurs. In the case of upscaling, the query traffic may be determined by queued queries waiting in the queue for each service class because the concurrency threshold for that service class has been reached, and the specified upscaling threshold may be violated if the ratio of the number of queries waiting in the queue for the service class compared to the concurrency threshold for the service class exceeds the specified ratio for a specified period of time. In the case of downscaling, the query traffic may be determined by current active queries for each service class, and the specified downscaling threshold may be violated if the ratio of the number of active queries for the service class compared to the concurrency threshold for the service class is less than the specified ratio for a specified period of time.

[0011] In an additional embodiment, the method may include one or more processors providing a plurality of node groups, each including a subset of available worker nodes in the cluster. The node groups are configured with respect to expected duration of a query. The method may further include one or more processors mapping a service class of a query to a node group, the mapping being performed to assign queries of the service class to worker nodes of the mapped node group. Autoscaling by removing some worker nodes may include selecting some worker nodes to be drained before removal according to node group, the worker nodes being selected from the node group configured for queries of the smallest possible expected duration.

[0012] Another aspect of the present invention discloses a method, computer program product, and system for autoscaling a query engine. The method includes one or more processors classifying queries of a query traffic based on a service class based on a respective level of complexity of the query. The method further includes one or more processors autoscaling the worker nodes by adding or removing some worker nodes from available worker nodes in a cluster based on the query traffic in the query engine. The method further includes one or more processors providing a plurality of node groups, each including a subset of available worker nodes in the cluster, the node groups being configured with respect to expected duration of the query. The method further includes one or more processors mapping each service class of the query to a node group according to an affinity between the service class and the node group. Furthermore, the autoscaling by removing some worker nodes further includes one or more processors selecting some worker nodes to be drained before removal according to the node group, the worker nodes being selected from the node group configured for queries of the smallest possible expected duration.

[0013] Thus, embodiments of the present invention may provide the advantage of minimizing the impact of cluster downscaling on ongoing queries by removing non-busy nodes with long running queries.

[0014] In another embodiment, the method may include the one or more processors ordering the node groups such that node groups for queries with shorter expected durations are drained first. Further, the node groups are dynamic and adjust in response to autoscaling of worker nodes.

[0015] In an additional embodiment, the method may include the one or more processors comparing query traffic in the form of active queries for each service class to a concurrency threshold for a maximum number of queries for the service class that are processed simultaneously. The method further includes the one or more processors autoscaling by removing some worker nodes from available worker nodes in the cluster based on a result of the comparison violating a specified downscaling threshold and the violation being maintained for a specified period of time.

[0016] The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed at the end of the specification.

[0017] Preferred embodiments of the invention will now be described, by way of example only, with reference to the following drawings, in which: [Brief description of the drawings]

[0018] [Figure 1] 1 is a schematic diagram of an exemplary embodiment of a system in which the present invention may be implemented, in accordance with an embodiment of the present invention. [Diagram 2] 4 is a flow diagram of an exemplary embodiment of a method according to an embodiment of the present invention. [Diagram 3] 4 is a flow diagram of an exemplary embodiment of an upscaling method according to an embodiment of the present invention. [Figure 4] 4 is a flow diagram of an exemplary embodiment of a downscaling method according to an embodiment of the present invention. [Figure 5A] 1 is a schematic diagram of an exemplary embodiment in accordance with an embodiment of the present invention. [Figure 5B] 1 is a schematic diagram of an exemplary embodiment in accordance with an embodiment of the present invention. [Figure 5C] 1 is a schematic diagram of an exemplary embodiment in accordance with an embodiment of the present invention. [Figure 6] FIG. 1 is a block diagram of an exemplary embodiment of a system in accordance with an embodiment of the present invention. [Figure 7]FIG. 2 is a block diagram of one embodiment of a computer system or cloud server on which the present invention may be implemented, in accordance with an embodiment of the present invention. [Figure 8] 1 is a schematic diagram of a cloud computing environment in which the present invention may be implemented, according to an embodiment of the present invention. [Figure 9] 1 is a diagram of an abstraction model layer of a cloud computing environment in which the present invention may be implemented, according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] It will be understood that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous features.

[0020] The described systems and methods provide for autoscaling of a query engine for enterprise-level big data workloads, including upscaling by adding worker nodes to a cluster of worker nodes that process queries for the query engine, or downscaling by removing worker nodes.

[0021] An embodiment of the present invention may be operable to monitor query traffic at a query engine in the form of queued and active queries, where queries are classified into service classes based on the level of query complexity. The query traffic for each service class is compared to a concurrency threshold of the maximum number of queries of that service class that are allowed to be processed simultaneously to determine whether the threshold has been violated, which would require upscaling or downscaling the number of worker nodes to accommodate the query traffic.

[0022] An embodiment of the present invention may then direct autoscaling of the cluster to add or remove a number of worker nodes from the available worker nodes in the cluster based on the result of the comparison indicating a violation of a specified upscaling threshold or a specified downscaling threshold and the violation is maintained for a specified period of time. Scaling the number of worker nodes in the query engine is performed based on the distribution of queries to the worker nodes according to the service classes of the queries, and instantiation of nodes or removal of nodes from the query engine is based on the demand of each query service class.

[0023] Additionally, during a downscale operation, embodiments of the present invention recognize that it may be necessary to first select worker nodes to remove from the group of worker nodes having the lowest cost-complexity banding to ensure rapid completion of the downscale operation without impacting ongoing or running queries.

[0024] Other embodiments of the described method and system provide a plurality of node groups, each including a subset of available worker nodes in a cluster, where the node groups are configured according to the expected duration of a query. Each service class of a query is mapped to a node group to provide affinity between the service class and the node group. When autoscaling with downscaling is required, embodiments of the invention can drain a worker node before removing it. Thus, the worker node to be drained before removal is selected according to the node group, where the worker node is selected from the node group configured for queries with the smallest possible expected duration.

[0025] In addition to the basic mechanism of scaling up or down the computing power of the query engine, embodiments of the present invention recognize the need to accommodate the complexity of queries as determined by the cost-based optimizer of the query engine. By determining the complexity of queries and then assigning the queries to a set of service classes of various complexity bandings, a cluster can be scaled in a timely and orderly manner. For a scale-down operation, embodiments of the present invention recognize the need to first select workers for removal from a group of workers with the lowest cost complexity banding to ensure rapid and orderly removal of candidate workers. In various embodiments, automatic upscale and downscale operations can be triggered as determined by queue lengths and response times.

[0026] Embodiments of the present invention can perform scaling in response to the needs of typical end-user workloads of various complexities. Further embodiments of the present invention can achieve elastic and automatic scaling in response to end-user queries based on the cost-segmented profile of the workload by mapping queries to groups of worker nodes based on their cost profile and then first selecting workers to remove from the worker node group associated with the least cost queries (and therefore execution time).

[0027] These new capabilities enable the query engine to act automatically to scale quickly and in an orderly manner in response to reliable predictions of overall workload needs derived from a sophisticated and accurate cost-based optimizer and workload management component.

[0028] Referring to FIG. 1, a block diagram 101 illustrates an exemplary embodiment of a query engine 100 for enterprise-level big data workloads, in accordance with an embodiment of the present invention.

[0029] Modern massively parallel processing (MPP) query engines, such as SQL engines (e.g., query engine 100), typically consist of a head node 110 and a cluster of multiple “n” worker nodes 150 (where n can be hundreds or even thousands of nodes). The worker nodes 150 essentially map to containers that are scheduled onto host machines (virtual machines or bare metal), with typically one or up to a few worker nodes on each host machine. When a query is submitted to engine 100, the query is scheduled for execution on available worker nodes 150, and each worker node 150 executes a fragment, also known as a runtime section, of the compiled query. In some implementations, the list of nodes is called a “dynamic node group.” The list is not static, as it changes depending on the number of worker nodes 150 present in the cluster at the time the query is submitted, and if any are explicitly excluded (e.g., if a hardware failure is detected on the host machine).

[0030] The head node 110 may run on a physical or virtual machine and may include at least one processor 111, hardware modules, or circuitry for performing the functions of the described components, which may be software units executed by the at least one processor. Multiple processors executing parallel processing threads may be provided, allowing parallel processing of some or all of the functions of the components. The memory 112 may be configured to provide computer instructions 113 to the at least one processor 111 for performing the functions of the components.

[0031] Query engine 100 may include a query optimizer component 114 responsible for selecting an optimal query access plan, and may include a query cost analysis component 115 for providing an associated execution time cost estimate for a query. In an exemplary embodiment, cost is a conceptual unit of measure (called a timeron), but in practice cost represents a combination of execution time duration and overall cluster resources required to complete the query.

[0032] In conjunction with the query optimizer component 114, the query engine 100 may also feature a workload management (WLM) component 120 to monitor and control statements executed on the cluster to efficiently utilize cluster resources and ensure that the cluster is not over- or under-utilized. Roughly speaking, a certain number of concurrent queries can be run to ensure that cluster resources are not overly saturated. Once a threshold is exceeded, incoming work is queued (e.g., shown as queue 141, queue 142, and queue 143) until some of the previous queries have completed.

[0033] The WLM component 120 may include a service class mapping component 121 that maps queries to logical concepts called WLM service classes based on their respective costs and other attributes (e.g., user and group). The WLM component 120 may segment queries into service classes that represent different complexity bandings based on the execution time cost calculated by the optimizer component 114. The bandings may be calibrated based on the period of time the cluster is monitored to understand the relative resource usage of queries across different cost profiles.

[0034] For example, queries with a timeron cost less than 150,000 may be considered “trivial,” queries with a timeron cost between 150,001 and 1 million may be considered “simple,” queries with a timeron cost between 1 million and 6 million may be considered “medium,” and queries with a timeron cost greater than 6 million may be considered “complex.” Thus, when a query is submitted to query engine 100, it is mapped to trivial, simple, medium, and complex service classes based on the cost estimated by optimizer component 114.

[0035] Based on the information collected from the calibration phase, the query engine administrator may set thresholds for the number of concurrent queries that can run in each of these service classes in the concurrency threshold setting and tuning component 122 of the WLM component 120. In an exemplary embodiment, the thresholds set for each band are typically selected to equalize the overall resource usage of the service classes (e.g., simple, medium, and complex queries). For example, in a given cluster, processing may translate to a concurrency threshold of two queries in the complex band, five queries in the medium band, and ten queries in the simple band. Trivial queries may be considered to have such little resource demand that the queries are left unmanaged. Once the concurrency threshold for a service class is reached, subsequent queries are managed in a first-in, first-out (FIFO) queue (i.e., queues 141-143) for the associated service class. Thus, administrators can ensure that there are sufficient resources available for each service class, and thus act to protect the overall stability of the cluster by avoiding situations where resources are over-committed and have adverse stability effects such as swapping.

[0036] The described method and system extends the functionality of the query engine 100 to provide an auto-scaler component 130 that activates an up- or down-scaling operation of the number of worker nodes 150 depending on the query traffic of a service class. The query traffic is the queue length when considering upscaling, or the number of current active queries when considering downscaling. The query traffic is compared to the maximum number of queues being processed simultaneously per service class. An embodiment of the invention can monitor the queue length (of queues 141-143) and set an auto-upscaling threshold based on a comparison of the queue length of a service class over a specified time period to a concurrency threshold for that service class. Further embodiments of the invention can monitor the number of active queries and set an auto-downscaling threshold based on a comparison of the number of active queries of a service class over a specified time period to a concurrency threshold for that service class.

[0037] The described methods and systems also introduce the ability to have multiple dynamic node groups, where each dynamic node group contains a subset of available worker nodes 150 in the cluster specified as a percentage of the total available worker nodes 150, and where workload service classes are mapped to dynamic node groups to provide a degree of affinity between the service classes and the node groups.

[0038] 2, a flow diagram illustrates a method 200 that is an exemplary embodiment of one aspect of the described method performed by query engine 100, in accordance with an embodiment of the present invention. In various embodiments, query engine 100 (shown in block diagram 101 of FIG. 1) may perform the processing of method 200. Additionally, the method of method 200 may include initial pre-processing setting of some parameters used in the method.

[0039] In step 201, method 200 sets a service class for a query based on the complexity of the query. In one embodiment, query engine 100 may utilize a cost-based optimizer component to calculate the runtime cost of a query based on a combination of runtime duration and cluster resources required for the query. Query engine 100 may segment or classify queries into service classes representing different complexity bandings based on runtime cost, where the bandings are calibrated based on the period of time the cluster is monitored. In step 202, method 200 sets a concurrency threshold for each service class as the maximum number of queries of that service class that are allowed to be processed by the cluster simultaneously.

[0040] In step 203, method 200 sets service class upscaling and downscaling thresholds. In various embodiments, method 200 can set the specified upscaling and downscaling thresholds to be a specified threshold ratio of the number of queries in the query traffic compared to the concurrency threshold, respectively. The query traffic can differ when determining whether upscaling or downscaling is required. In the described embodiment, the query traffic is measured by currently queued queries of the service class for purposes of upscaling, and the query traffic is measured by currently active queries for purposes of downscaling.

[0041] In step 204, method 200 can configure dynamic node groups, each including a subset of available worker nodes in the cluster and configured with respect to expected duration of queries. For example, method 200 can configure dynamic node groups for short queries, average duration queries, and long duration queries. The service classes of the queries are mapped to node groups to assign queries of the service class to worker nodes in the mapped node groups.

[0042] Once the parameters are initialized, method 200 is performed (at step 205) during operation of query engine 100 to monitor query traffic for each service class. To automatically downscale the number of active worker nodes in query engine 100, method 200 may monitor query traffic in the form of active queries for each service class at step 205. To automatically upscale the number of active worker nodes in query engine 100, method 200 may monitor query traffic in the form of queued queries for each service class.

[0043] In step 206, method 200 compares the query traffic for each service class to the concurrency threshold for that service class. Then, in step 207, method 200 determines whether the comparison results in a violation of a specified downscaling threshold or a specified upscaling threshold. A specified upscaling threshold is violated if the ratio of the number of queries waiting in a queue for a service class compared to the concurrency threshold for the service class for a specified time period exceeds the specified ratio. A specified downscaling threshold is violated if the ratio of the number of active queries for a service class compared to the concurrency threshold for the service class for a specified time period is less than the specified ratio.

[0044] In step 208, method 200 may evaluate the number of nodes that need to be added or removed across all service classes. In an example embodiment, method 200 may perform step 208 by aggregating service classes that are violating a defined upscaling threshold or a defined downscaling threshold, basing the number of worker nodes to be added or removed for each service class on the current percentage of worker nodes allocated to that service class and the increase or decrease in required capacity based on the comparison.

[0045] If downscaling is required, method 200 may select worker nodes to drain in the order of the defined dynamic node groups (step 209). For example, method 200 may select nodes that are processing lower cost queries so that the nodes may be drained more efficiently. In an exemplary embodiment, method 200 may perform the draining process without affecting or disrupting existing queries running on these worker nodes. Method 200 may select candidate nodes to remove during scale down first from node groups that are assigned short running queries, then from node groups that are assigned queries of medium duration and complexity, and finally, only as a last resort, from node groups with long running queries. Thus, candidate nodes drain active queries quickly, thereby freeing up worker nodes to allow the downscaling operation to be completed quickly and transparently to end users.

[0046] Once draining of the required number of worker nodes is complete, if required for downscaling, method 200 may auto-scale to add or remove the required number of worker nodes (step 210). In both upscaling and downscaling scenarios, method 200 may adjust (at step 211) the service class concurrency threshold based on the number of upscaled or downscaled worker nodes. Method 200 may then feed this adjustment back to the concurrency threshold setting (at step 202) for further monitoring and autoscaling of query engine 100. In both upscaling and downscaling scenarios, method 200 may adjust (at step 212) the dynamic node group based on the number of upscaled or downscaled worker nodes. Method 200 may then feed this adjustment back to the dynamic node group configuration (at step 204) for further monitoring and autoscaling of query engine 100.

[0047] 3, a flow diagram illustrates a method 300 that is an exemplary embodiment of a method for automatically upscaling a cluster of nodes in query engine 100, in accordance with an embodiment of the present invention. In various embodiments, query engine 100 (shown in block diagram 101 of FIG. 1) may perform the processing of method 300.

[0048] In step 301, method 300 begins, then in step 302, method 300 monitors queued queries in a cluster of query engines 100. Then, in operation 310 of method 300, for each service class, method 300 performs operations of steps 311-313 in parallel. In step 311, the method checks the queued queries for the service class. In response to determining (in step 312) that the ratio of queued queries compared to a concurrency threshold for the service class for a queued time longer than a specified response time exceeds a threshold ratio, method 300 evaluates (in step 313) the number of delta nodes to add to the service class.

[0049] Further, in step 303, method 300 evaluates the total number of delta nodes to be added across classes. Then (in step 304), method 300 upscales the cluster to add the delta number of worker nodes. In step 305, method 300 adjusts the configured service class concurrency thresholds to accommodate the new worker nodes. Additionally, method 200 may adjust (in step 306) the dynamic node groups of the service classes. Then, method 300 loops to step 302 to monitor queued queries in the upscaled cluster.

[0050] 4, a flow diagram illustrates a method 400 that is an exemplary embodiment of a method for automatically downscaling a cluster of nodes in query engine 100, according to an embodiment of the present invention. In various embodiments, query engine 100 (shown in block diagram 101 of FIG. 1) may perform the processing of method 400.

[0051] In step 401, method 400 begins, then in step 402, method 400 monitors active queries in nodes of a cluster of query engine 100. Then, in operation 410 of method 400, for each service class, method 400 performs steps 411-413 in parallel. In step 411, method 400 checks the active queries for the service class. In response to determining (in step 412) that the ratio of active queries to a concurrency threshold is less than a threshold ratio for a time period greater than a specified low activity time, then (in step 413) method 400 evaluates the number of delta nodes to remove for the service class.

[0052] Additionally, in step 403, method 400 evaluates a total number of delta nodes to be removed across the class. In various embodiments, the nodes need to be drained prior to removal. Method 400 includes selecting which worker nodes to drain in the order of the defined dynamic node group associated with the service class (step 404). Once the required number of nodes have been drained, method 400 may then downscale the cluster to remove the delta number of worker nodes (step 405).

[0053] At step 406, method 400 may adjust a configured service class active percentage threshold to accommodate the removed worker node. Additionally, at step 407, method 400 may also adjust the dynamic node group of the service class. Method 400 then loops to step 402 to monitor queued queries in the downscaled cluster.

[0054] Figure 5A shows a schematic diagram 500 illustrating an exemplary embodiment of the described method according to an embodiment of the present invention. Figure 5B shows a schematic diagram 590 illustrating an exemplary embodiment of the described method according to an embodiment of the present invention. Figure 5C shows a schematic diagram 598 illustrating an exemplary embodiment of the described method according to an embodiment of the present invention. In the illustrated embodiment, Figure 5A shows an upscaling scenario, Figure 5B shows dynamic node group association to service classes, and Figure 5C shows a downscaling scenario.

[0055] 5A and 5B illustrate workload management component 120 receiving and processing multiple queries (i.e., query 501, query 502, query 503, and query 504) for execution by a cluster of worker nodes 570 including node 571, node 572, node 573, node 574, node 575, node 576, node 577, node 578, and node 579 (i.e., nodes 571-579). In the illustrated example, multiple service classes are configured for queries of different cost classes. In the illustrated embodiment, workload management component 120 includes a trivial service class 510, a simple service class 520, a medium service class 530, and a complex service class 540. Each service class has an established concurrency threshold for the number of concurrent queries that can be processed by each class. In the illustrated embodiment, the concurrency threshold for trivial queries is unlimited 511, for simple queries it is 10 (concurrency threshold 521), for medium queries it is 5 (concurrency threshold 531), and for complex queries it is 2 (concurrency threshold 541).

[0056] The workload management component 120 includes queues 512, 522, 532, 542 for queries of each service class waiting to execute on a cluster of worker nodes 570. The illustrated example also includes a node group mapping component 550, which is further described with respect to FIG. 5B. The cluster of worker nodes 570 may include several worker nodes (i.e., nodes 571-579). Additionally, active queries 560 shows several active queries (i.e., trivial active queries 514, simple active queries 524, medium active queries 534, and complex active queries 544) per service class of queries that are executing across a sub-group of nodes 571-579.

[0057] Referring to FIG. 5A, a schematic diagram 500 illustrates autoscaling the number of worker nodes according to an exemplary embodiment of the present invention. In one example, an administrator may set a queue threshold amount 551, which in this example is the percentage number of queued queries for each service class compared to the concurrency threshold for each service class. In this example, the cluster administrator sets the percentage change to a value of 50% and the response time to 5 minutes. The administrator may also set an active threshold amount 552, which in this example is the percentage number of active queries for each service class compared to the concurrency threshold for each service class.

[0058] The example shown in Figure 5A includes 16 trivial active queries 514. The trivial query concurrency threshold 511 is unlimited, so the concurrency threshold does not trigger upscaling of the node. Additionally, Figure 5A shows 9 simple active queries 524, and the simple query concurrency threshold 521 is 10, so the concurrency threshold does not trigger upscaling of the node.

[0059] In a further embodiment, the concurrency threshold for the medium query service class is 5 (concurrency threshold 531), and Figure 5A shows 5 medium active queries 534 at full capacity. Additionally, Figure 5A includes 2 queued medium queries 533, but this does not trigger upscaling of the node because 2 is less than the 50% queue threshold amount 551.

[0060] In another embodiment, the concurrency threshold for complex query service class 4 is set to a concurrency threshold of 2 (e.g., concurrency threshold 541) and two complex active queries 544 are already running on the cluster of worker nodes 570. One new complex query 543 is submitted to the cluster of worker nodes 570 and is queued 542 by the workload management component 120 for more than five minutes. Thus, the autoscaler component 130 activates an upscale of the nodes to expand the cluster of worker nodes 570.

[0061] The number of nodes to be upscaled may be based on the same percentage as the queue threshold amount 551, and thus may be calculated as 50% of the portion of the nodes in the cluster that process complex queries. In this embodiment, the number of nodes is divided into thirds to process simple, medium, and complex queries, and trivial queries are not limited across the cluster. In the exemplary embodiment of FIG. 5A, schematic 500 currently includes 15 worker nodes (i.e., nodes 571-577), five of which process complex queries. A 50% upscaling of the five nodes is required, which would require three additional nodes 581, 582, 583. Cluster 584 before scale up is shown as nodes 571-577, and cluster 585 after scale up is shown as nodes 571-579 and nodes 581-583.

[0062] The autoscaler component 130 may calculate the delta number of worker nodes required based on the following formula: D = ceiling(N*(IF(qS*100 / tS)>=p THENqS*100 / tS / 300 ELSE 0 ENDIF + IF(qM*100 / tM)>= p THEN qM*100 / tM / 300 ELSE 0ENDIF + IF(qC*100 / tC)>=p THEN qC*100 / tC / 300 ELSE 0 ENDIF)

[0063] Where: D is the delta number of worker nodes, tS is the concurrency threshold for the number of simple queries, tM is the concurrency threshold for the number of medium queries, tC is the concurrency threshold for the number of complex queries, qS is the number of queued simple queries, qM is the number of queued medium queries, qC is the number of queued complex queries, N is the number of worker nodes currently in the cluster, p is the autoscale percentage trigger.

[0064] Note that the divisor of 300 (3 x 100) is based on the assumption that the calibration phase resulted in an even allocation of cluster resources between the three bands of simple, medium, and complex queries. This can be varied for each allocation.

[0065] So, in the above case, D is calculated as follows: D = ceiling(15* (1*100 / 2 / 300)) = 3

[0066] Therefore, the cluster is expanded by 3 workers, increasing the cluster size from 15 workers to 18 workers.

[0067] The medium query service class has a concurrency threshold of 5 (concurrency threshold 531), and if the number of medium active queries 534 drops to 2 and there are no queued medium queries 533, and this condition persists for 5 minutes, the autoscaler component 130 will trigger a proportionate reduction in the cluster size (i.e., downscale the cluster to free up the calculated number of worker nodes).

[0068] Additionally, an administrator may configure a set of dynamic node groups. In this embodiment, a first node group may be named "long" for queries that have a long execution time, which is bound to a first percentage of the nodes in the cluster. A second node group may be named "average" for queries that have an average execution time, which is bound to a second percentage of the different nodes in the cluster. A third node group may be named "short" for queries that have a short execution time, which is bound to all the nodes in the cluster. An administrator may then create a mapping from service classes to node groups that maps the trivial and simple query service classes to the "short" node group, the medium service class to the "average" node group, and the complex service class to the "long" node group. The mapping provides an association of service classes to node groups.

[0069] Whenever the autoscaler component initiates a scale down operation, candidate nodes for draining are selected from a group of nodes that are initially the "short" node group, then from the "average" node group, and finally (for example, only as a last resort) from the "long" node group. Thus, you can specify a fairly aggressive or short grace period for draining down, since any queries running in the "short" node group are only fleeting and will be drained down immediately. As nodes are drained and released from the cluster, the specified percentage of node group allocation is maintained by dynamically adjusting the list of nodes on which query fragments are scheduled when new queries are sent to the query engine.

[0070] 5B illustrates the node group mapping component 550 and the service class to node group mapping 595, which provides affinity between query complexity and specific worker nodes. In addition to assigning each query to its respective service class based on optimizer cost and applying associated concurrency thresholds, the workload management component 120 also maps 595 queries to appropriate node groups where query fragments of the service class queries will be scheduled.

[0071] In this example, queries of trivial service class 510 and simple service class 520 may be mapped to a node group for "short" execution time queries 593 that includes all worker nodes in the cluster, queries of medium service class 530 complexity may be mapped to a node group for "average" duration queries 592 bound to 35% of the worker nodes in the cluster, and queries of complex service class 540 may be mapped to a node group for "long" execution time queries 591 bound to a different set including 35% of the worker nodes.

[0072] In response to initiating an operation to scale down the cluster, candidate worker nodes for eviction are initially selected from those bound only to the short node group. In the exemplary configuration shown in FIG. 5B, this selection is 30% of worker nodes 594 in the cluster that are not bound to either the average 592 or the long node group (long running queries 591), meaning that the cluster can be downscaled by 30% of its original size and only short running queries (mapped based on query complexity) need to be drained. Thus, the draining down should be fast so that nodes can be freed up quickly without impacting running queries. Alternatively, if the cluster needs to be downscaled by 50%, workers in the average node group should also be selected, which would increase the draining down duration for those nodes, perhaps requiring a grace period before taking more evasive action to complete the eviction. As the cluster is scaled, the concurrency threshold is also adjusted proportionally to reflect the current overall computing capacity. The new concurrency threshold is applied to any queries submitted from that point on.

[0073] Putting all this together in Figure 5C, we see that the number of queries (complex active queries 544) executing the complex service class is now only two, which is less than the concurrency threshold 541 of three for this service class. In the scenario where the number of queries (complex active queries 544) executing the complex service class remains at two for more than five minutes, the autoscaler component triggers a downscale operation to reduce the number of workers by the calculated delta number of three workers (i.e., effectively the reverse of the scale up actuated in Figure 5A). Once the scale down operation is complete, the threshold for the complex query service class is returned to two (concurrency threshold 541).

[0074] 6, a block diagram illustrates an exemplary embodiment of a query engine 100 having additional components of a workload management component 120 for directing an autoscaler component 130 for autoscaling the number of worker nodes of a cluster of worker nodes processing queries of the query engine 100, in accordance with an embodiment of the present invention. The workload management component 120 may include the following components:

[0075] The query traffic monitoring component 601 monitors the query traffic in the query engine, and queries are classified into multiple service classes based on the level of query complexity by the service class mapping component 121. The query traffic monitoring component 601 may include a query queue monitoring component 602 for monitoring a queue of queries for a service class waiting to be processed, which may be used to upscale the number of worker nodes, and an active query monitoring component 603 for monitoring active queries being processed for a service class, which may be used to downscale the number of worker nodes. The workload management component 120 also includes a concurrency threshold setting and tuning component 122 for setting a maximum number of queries for a class that are allowed to be active at the same time and adjusting the maximum based on changes in the number of worker nodes due to autoscaling.

[0076] Additionally, the workload management component 120 includes a concurrency comparison component 604 for comparing the query traffic of each service class to a concurrency threshold to determine whether an upscaling or downscaling threshold for the service class is violated. The concurrency comparison component 604 may include an upscaling threshold component 605 for defining an upscaling threshold as a specified ratio of the number of queries waiting in a queue for a service class compared to the concurrency threshold for the service class for a specified time period. If the ratio exceeds the specified ratio for the specified time period, the upscaling threshold is violated. The concurrency comparison component 604 may include a downscaling threshold component 606 for defining a downscaling threshold as a specified ratio of the number of active queries for a service class compared to the concurrency threshold for the service class for a specified time period. If the ratio is less than the specified ratio for the specified time period, the downscaling threshold is violated.

[0077] The workload management component 120 also includes a delta node evaluation component 607 to evaluate the number of worker nodes to add or remove based on an aggregation of a comparison of query traffic across all service classes at a given time. The delta node evaluation component 607 bases the number of worker nodes to add or remove for each service class that is violating a specified upscaling threshold or a specified downscaling threshold on the current percentage of worker nodes assigned to that service class and the increase or decrease in required capacity based on the comparison. The query class aggregation component 608 can aggregate the requests of the service classes.

[0078] Further, the workload management component 120 includes an autoscaling instruction component 620 for instructing autoscaling of the cluster to add or remove a number of worker nodes from the available worker nodes in the cluster based on the result of the comparison being a violation of a specified upscaling threshold or a specified downscaling threshold and the violation is maintained for a specified period of time. The autoscaling component that is instructed may be an integrated component or may be a third-party autoscaling component that is instructed using customized metrics. For example, the Kubernetes autoscaler includes the ability to utilize custom metrics that may be calculated based on the instruction.

[0079] The workload management component 120 may include a node group component 610 for providing a number of node groups, each including a subset of available worker nodes in the cluster, where the node groups are configured with respect to expected duration of a query. The workload management component 120 may also include a service class mapping component 612 for mapping a service class of a query to a node group to assign queries of the service class to worker nodes of the mapped node group. The workload management component 120 may utilize the node group component 610 and the service class mapping component 612 in conjunction with a node drain selection component 613 for selecting a number of worker nodes to be drained prior to removal during downscaling according to the node group, where the worker nodes are selected from a node group configured for queries of the smallest possible expected duration. The process of the node drain selection component 613 selecting a number of worker nodes to be drained may include ordering the node groups such that node groups for queries of shorter expected duration are drained first. In an example embodiment, the node group component 610 dynamically adjusts node groups in response to auto-scaling of worker nodes.

[0080] 7 illustrates a block diagram of components of a head node computing device of the query engine 100 of FIG. 1 in accordance with an embodiment of the present invention. It should be appreciated that FIG. 7 is intended as an illustration of one implementation only and is not intended to imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

[0081] A computing device may include one or more processors 702, one or more computer-readable random access memories (RAMs) 704, one or more computer-readable read-only memories (ROMs) 706, one or more computer-readable storage media 708, device drivers 712, read / write drives or interfaces 714, and network adapters or interfaces 716, all interconnected through a communications fabric 718. The communications fabric 718 may be implemented with any architecture designed to pass data and / or control information between processors (e.g., microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components in the system.

[0082] One or more operating systems 710 and application programs 711, such as applications corresponding to the processing steps of methods 200, 300, and 400, are stored in one or more of the computer readable storage media 708 for execution by one or more of the processors 702 through one or more of the respective RAMs 704 (which typically include cache memory). In the illustrated embodiment, each of the computer readable storage media 708 can be an internal hard drive magnetic disk storage device, a CD-ROM, a DVD, a memory stick, a magnetic tape, a magnetic disk, an optical disk, a semiconductor storage device, such as a RAM, a ROM, an EPROM, a flash memory, or any other computer readable storage medium capable of storing computer programs and digital information according to embodiments of the present invention.

[0083] The computing device may also include a R / W drive or interface 714 for reading from and writing to one or more portable computer readable storage media 726. Application programs 711 on the computing device may be stored on one or more of the portable computer readable storage media 726 and read via the respective R / W drive or interface 714 and loaded into the respective computer readable storage media 708.

[0084] The computing device may also include a network adapter or interface 716, such as a TCP / IP adapter card or a wireless communication adapter. Application programs 711 on the computing device may be downloaded to the computing device from an external computer or external storage device via a network (e.g., the Internet, a local area network, or other wide area network or wireless network) and the network adapter or interface 716. The programs may be loaded from the network adapter or interface 716 into the computer-readable storage medium 708. The network may include copper wire, fiber optic, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0085] The computing device may also include a display screen 720, a keyboard or keypad 722, and a computer mouse or touchpad 724. The device driver 712 interfaces to the display screen 720 for imaging, to the keyboard or keypad 722, to the computer mouse or touchpad 724, or to the display screen 720 for alphanumeric entry and pressure sensing of user selections, or a combination thereof. The device driver 712, the R / W drive or interface 714, and the network adapter or interface 716 may include hardware and software stored in the computer readable storage medium 708 or ROM 706, or both.

[0086] Cloud Computing

[0087] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings recited herein is not limited to a cloud computing environment. Rather, embodiments of the invention may be implemented in conjunction with any other type of computing environment now known or later developed.

[0088] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with a service provider. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0089] Its features are as follows.

[0090] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, such as server time and network storage, automatically as needed, without the need for human interaction with the provider of the service.

[0091] Broad Network Access: Capabilities are available over the network and are accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., cell phones, laptops, and PDAs).

[0092] Resource Pooling: Pooling provider computing resources to serve multiple consumers using a multi-tenant model where various physical and virtual resources are dynamically allocated and reallocated according to demand. Consumers generally have no control over or knowledge of the exact location of the resources provided to them, although there is a sense of location independence in that the location may be identifiable at a higher level of abstraction (e.g. country, state, or data center).

[0093] Rapid Elasticity: Capacity can be rapidly and elastically provisioned, sometimes automatically, to quickly scale out and rapidly release to quickly scale in. To the consumer, provisionable capacity often appears to be unlimited and can be purchased in any quantity at any time.

[0094] Metered Services: Cloud systems leverage metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts) to automatically control and optimize resource usage. Resource usage can be monitored, controlled, and reported to provide transparency to both providers and consumers of the services utilized.

[0095] The service model is as follows:

[0096] Software as a Service (SaaS): The capability offered to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer has no management or control over the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application features, with the possible exception of limited user-specific application configuration settings.

[0097] Platform as a Service (PaaS): The capability offered to a consumer is to deploy applications that the consumer creates or acquires, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does control the deployed applications and possibly the application hosting environment configuration.

[0098] Infrastructure as a Service (IaaS): The ability offered to consumers is to provision processing, storage, network, and other basic computing resources onto which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does control the operating systems, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0099] The deployment model is as follows:

[0100] Private Cloud: The cloud infrastructure is operated exclusively for an organization. It can be managed by the organization or a third party and can be on-premise or off-premise.

[0101] Community Cloud: The cloud infrastructure is shared by several organizations to support a specific community with common concerns (e.g., mission, security requirements, policies, and compliance considerations). It may be managed by the organization or a third party and may exist on-premise or off-premise.

[0102] Public Cloud: The cloud infrastructure is made available to the general public or large industry organizations and is owned by an organization that sells cloud services.

[0103] Hybrid Cloud: A cloud infrastructure is a composite of two or more clouds (private, community, or public) that remain unique entities but are joined by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0104] Cloud computing environments are service-oriented with an emphasis on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0105] Referring now to FIG. 8, an exemplary cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as, for example, a personal digital assistant (PDA) or cell phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof, may communicate. The nodes 10 may communicate with each other. They may be physically or virtually grouped (not shown), for example, in one or more networks, such as the private, community, public, or hybrid clouds described above, or combinations thereof. This enables the cloud computing environment 50 to provide infrastructure-as-a-service, platform-as-a-service, and / or software-as-a-service services without the need for cloud consumers to maintain resources on their local computing devices. It should be understood that the types of computing devices 54A-N illustrated in FIG. 8 are intended to be exemplary only, and that the computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network addressable connection (e.g., using a web browser).

[0106] 9, there is shown a set of functional abstraction layers provided by cloud computing environment 50 (FIG. 8). It should be understood in advance that the components, layers, and functions shown in FIG. 9 are intended to be exemplary only, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0107] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframes 61, RISC (reduced instruction set computer) architecture based servers 62, servers 63, blade servers 64, storage devices 65, and networks and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0108] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities may be provided: virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.

[0109] In one example, the management layer 80 may provide the following functions: Resource provisioning 81 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and accounting or billing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification of cloud consumers and tasks as well as protection for data and other resources. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 provides allocation and management of cloud computing resources such that requested service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides advance arrangement and procurement of cloud computing resources anticipated to be required in the future according to SLAs.

[0110] The workload layer 90 provides examples of functionality for which a cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instructional delivery 93, data analytics processing 94, transaction processing 95, and query processing 96.

[0111] The computer program product of the present invention includes one or more computer readable hardware storage devices having computer readable program code stored thereon, said program code being executable by one or more processors to implement the methods of the present invention.

[0112] The computer system of the present invention includes one or more processors, one or more memories, and one or more computer-readable hardware storage devices, the one or more hardware storage devices including program code executable by the one or more processors via the one or more memories to implement the methods of the present invention.

[0113] The description of various embodiments of the present invention is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used in this specification are selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0114] Improvements and modifications can be made to the above without departing from the scope of the present invention.

[0115] The programs described herein are identified based on the applications in which they are implemented in specific embodiments of the invention. However, it should be understood that any specific program names herein are used for convenience only, and thus the invention should not be limited to use in any particular application identified and / or implied by such names.

[0116] The present invention may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to carry out aspects of the present invention.

[0117] A computer readable storage medium may be any tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or groove ridge structures having instructions recorded thereon, and any suitable combination thereof. A computer-readable storage medium, as used herein, should not be construed to be a transitory signal per se, such as, for example, radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over a wire.

[0118] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to the respective computing / processing device or to an external computer or storage device over a network, such as, for example, the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface of each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0119] The computer readable program instructions for carrying out the operations of the present invention may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or object oriented programming languages ​​such as Smalltalk®, C++, and procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the last scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or a connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing state information of the computer readable program instructions to execute the computer readable program instructions to perform aspects of the present invention.

[0120] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0121] These computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine such that, when the instructions are executed via the processor of the computer or other programmable data processing apparatus, means are generated for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium capable of directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium on which the instructions are stored constitutes an article of manufacture including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0122] Furthermore, computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps, thereby generating a computer-implemented process that, when executed on the computer, other programmable apparatus, or other device, implements the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0123] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of instructions that includes one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions described in the blocks may occur out of the order shown. For example, depending on the functionality involved, two blocks shown in succession may in fact be realized as one step and may be executed simultaneously, substantially simultaneously, partially or fully in a temporally overlapping manner, or the blocks may possibly be executed in the reverse order. It will also be noted that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or executes a combination of dedicated hardware and computer instructions.

[0124] The description of various embodiments of the present invention has been presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the present invention. The terms used in this specification are selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A method for auto-scaling a query engine, comprising: monitoring, by one or more processors, query traffic in the query engine; classifying, by one or more processors, queries of the query traffic into a plurality of service classes based on a level of query complexity; comparing, by one or more processors, the query traffic of each service class to a concurrency threshold for a maximum number of queries of said service class that are allowed to be processed simultaneously; directing, by one or more processors, autoscaling of a cluster of worker nodes, the autoscaling to change a number of worker nodes available in the cluster based on the comparison of the query traffic over a specified time period to a specified upscaling threshold and a specified downscaling threshold, the autoscaling adding or removing some worker nodes from a cluster of worker nodes that processes queries for the query engine; A method comprising:

2. The method of claim 1 , wherein the specified upscaling threshold and the specified downscaling threshold are each respective specified threshold ratios of a number of queries in the query traffic compared to the concurrency threshold.

3. evaluating, by one or more processors, a number of worker nodes to modify based on an aggregation of the comparison of query traffic across all service classes at a given time. The method of claim 1 further comprising:

4. Evaluating the number of worker nodes to change is determining, by one or more processors, for each service class that violates the defined upscaling threshold, the number of worker nodes to modify based on a current proportion of worker nodes assigned to the service class and a required increase in capacity based on the comparison; The method of claim 3 further comprising:

5. Evaluating the number of worker nodes to change is determining, by one or more processors, for each service class that violates the defined downscaling threshold, the number of worker nodes to modify based on a current proportion of worker nodes assigned to the service class and a reduction in required capacity based on the comparison; The method of claim 3 further comprising:

6. adjusting, by one or more processors, the concurrency thresholds of one or more service classes based on the number of new worker nodes in the cluster after autoscaling. The method of claim 1 further comprising:

7. determining, by one or more processors, in a queue for each service class, the query traffic based on queued queries that are waiting due to reaching the concurrency threshold for the service class; Further comprising: the specified upscaling threshold is violated if the ratio of the number of queries waiting in the queue for the service class compared to the concurrency threshold for the service class exceeds the specified ratio for a specified period of time; The method of claim 2.

8. determining, by one or more processors, the query traffic based on currently active queries for each service class; Further comprising: the specified downscaling threshold is violated if the ratio of the number of active queries for the service class compared to the concurrency threshold for the service class is less than a specified ratio for a specified period of time; The method of claim 2.

9. providing, by one or more processors, a plurality of node groups, each including a subset of the available worker nodes in the cluster, the node groups configured with respect to an expected duration of a query; mapping, by one or more processors, a service class of a query to a node group, the mapping being performed to assign queries of a service class to worker nodes of the mapped node group; The method of claim 1 further comprising:

10. Autoscaling, by one or more processors, to remove some worker nodes by selecting some worker nodes to be drained before removal according to node groups, the worker nodes being selected from node groups configured for queries of the smallest possible expected duration. The method of claim 9 further comprising:

11. 10. The method of claim 9, wherein the node groups are dynamic and adjust in response to autoscaling of the worker nodes.

12. providing, by one or more processors, a plurality of node groups, each including a subset of the available worker nodes in the cluster, the node groups configured with respect to an expected duration of a query; mapping, by one or more processors, each service class of the query to a node group according to an affinity between the service class and the node group; Autoscaling by removing some worker nodes includes selecting, by one or more processors, a number of worker nodes to be drained prior to removal according to node group, the worker nodes being selected from node groups configured for queries of minimum possible expected duration; The method of claim 1 further comprising:

13. Selecting some worker nodes to be drained is ordering, by the one or more processors, the groups of nodes such that groups of nodes for queries of shorter expected duration are drained first; The method of claim 12 further comprising:

14. The method of claim 12 , wherein the node groups are dynamic and adjust in response to autoscaling of the worker nodes.

15. comparing, by one or more processors, query traffic in the form of active queries for each service class with a concurrency threshold for a maximum number of queries of said service class that are processed simultaneously; autoscaling, by one or more processors, by removing some worker nodes from the available worker nodes in the cluster based on the result of the comparison violating a defined downscaling threshold and the violation being maintained for a defined period of time; The method of claim 12 further comprising:

16. 16. The method of claim 15, wherein the specified downscaling threshold is violated if a ratio of a number of active queries for the service class compared to the concurrency threshold for the service class is less than a specified ratio for a specified period of time.

17. 1. A method for auto-scaling a query engine, comprising: classifying, by one or more processors, queries of the query traffic based on a service class that is based on a respective level of query complexity; autoscaling the worker nodes by adding or removing, by the one or more processors, a number of worker nodes from available worker nodes in the cluster based on query traffic in the query engine; providing, by one or more processors, a plurality of node groups, each including a subset of the available worker nodes in the cluster, the node groups configured with respect to an expected duration of a query; mapping, by one or more processors, each service class of the query to a node group according to an affinity between the service class and the node group; Autoscaling by removing some worker nodes includes selecting, by one or more processors, a number of worker nodes to be drained prior to removal according to node group, the worker nodes being selected from node groups configured for queries of minimum possible expected duration; The method further comprising:

18. Selecting some worker nodes to be drained is ordering, by the one or more processors, the groups of nodes such that groups of nodes for queries of shorter expected duration are drained first; 20. The method of claim 17, further comprising:

19. 20. The method of claim 17, wherein the node groups are dynamic and adjust in response to autoscaling of the worker nodes.

20. comparing, by one or more processors, query traffic in the form of active queries for each service class with a concurrency threshold for a maximum number of queries of said service class that are processed simultaneously; autoscaling, by one or more processors, by removing some worker nodes from the available worker nodes in the cluster based on the result of the comparison violating a defined downscaling threshold and the violation being maintained for a defined period of time; 20. The method of claim 17, further comprising:

21. 21. The method of claim 20, wherein the specified downscaling threshold is violated if a ratio of a number of active queries for the service class compared to the concurrency threshold for the service class for a specified period of time is less than a specified ratio.

22. 1. A computer system for auto-scaling a query engine, comprising: one or more computer processors; one or more computer readable storage media; program instructions stored on the computer-readable storage medium for execution by at least one of the one or more processors; the program instructions comprising: program instructions for monitoring query traffic in the query engine; program instructions for classifying queries of the query traffic into a plurality of service classes based on a level of query complexity; program instructions for comparing the query traffic of each service class to a concurrency threshold for a maximum number of queries of said service class that are allowed to be processed simultaneously; program instructions for directing autoscaling of a cluster of worker nodes, the autoscaling being for changing a number of worker nodes available in the cluster based on the comparison of the query traffic over a specified time period to a specified upscaling threshold and a specified downscaling threshold, the autoscaling adding or removing worker nodes from a cluster of worker nodes that processes queries for the query engine; 1. A computer system comprising:

23. 23. The computer system of claim 22, wherein the specified upscaling threshold and the specified downscaling threshold are each respective specified threshold ratios of a number of queries in the query traffic compared to the concurrency threshold.

24. stored on the computer-readable storage medium for execution by at least one of the one or more processors; Evaluating the number of worker nodes to change based on an aggregate of said comparison of query traffic across all service classes at a given time.

23. The computer system of claim 22, further comprising program instructions for executing:

25. stored on the computer-readable storage medium for execution by at least one of the one or more processors; adjusting the concurrency thresholds of one or more service classes based on the number of new worker nodes in the cluster after autoscaling.

23. The computer system of claim 22, further comprising program instructions for executing:

26. 1. A computer system for auto-scaling a query engine, comprising: one or more computer processors; one or more computer readable storage media; program instructions stored on the computer-readable storage medium for execution by at least one of the one or more processors; the program instructions comprising: program instructions for classifying, by one or more processors, queries of the query traffic according to a service class based on respective levels of query complexity; program instructions for autoscaling worker nodes by adding or removing a number of worker nodes from available worker nodes in a cluster based on query traffic in a query engine; program instructions for providing a plurality of node groups, each including a subset of the available worker nodes in the cluster, the node groups configured with respect to an expected duration of a query; program instructions for mapping each service class of the query to a node group according to an affinity between the service class and the node group; Autoscaling by removing some worker nodes includes Program instructions for selecting a number of worker nodes to be drained before removal according to node group, the worker nodes being selected from a node group configured for queries of the smallest possible expected duration, the selecting program instructions The computer system further comprising:

27. The program instructions for selecting a number of worker nodes to be drained include: ordering the node groups such that the node groups for queries of shorter expected duration are drained first; 27. The computer system of claim 26, further comprising program instructions for executing:

28. 27. The computer system of claim 26, wherein the node groups are dynamic and adjust in response to autoscaling of the worker nodes.

29. stored on the computer-readable storage medium for execution by at least one of the one or more processors; comparing query traffic in the form of active queries for each service class with a concurrency threshold for a maximum number of queries of said service class that are processed simultaneously; autoscaling by removing some worker nodes from the available worker nodes in the cluster based on the result of the comparison violating a defined downscaling threshold and the violation being maintained for a defined period of time; 27. The computer system of claim 26, further comprising program instructions for executing:

30. 1. A computer program for auto-scaling a query engine, comprising: On the computer, program instructions for monitoring query traffic in the query engine; program instructions for classifying queries of the query traffic into a plurality of service classes based on a level of query complexity; program instructions for comparing the query traffic of each service class to a concurrency threshold for a maximum number of queries of said service class that are allowed to be processed simultaneously; program instructions for directing autoscaling of a cluster of worker nodes, the autoscaling being for changing a number of worker nodes available in the cluster based on the comparison of the query traffic over a specified time period to a specified upscaling threshold and a specified downscaling threshold, the autoscaling adding or removing worker nodes from a cluster of worker nodes that processes queries for the query engine; A computer program that executes the following:

31. A computer program comprising program code means adapted to perform the method according to any of claims 1 to 21 when said program is run on a computer.

Images