Techniques for modifying cluster computing environments

Abstract

The present invention provides a system, device, and method for intelligently coordinating a set of worker nodes within a computing cluster. [Solution] A computing device or service monitors performance metrics of a set of worker nodes in a computing cluster, and when it detects a performance metric below a threshold, it performs a first adjustment to the number of nodes in the cluster, acquires training data at least in part on the first adjustment and uses it together with supervised learning techniques to train a machine learning model to predict future performance changes in the cluster, provides the machine learning model with subsequent performance metrics and / or cluster metadata to obtain an output showing the predicted performance changes, and performs an additional adjustment to the number of worker nodes at least in part on this output.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Patent Application No. 17 / 094,715, filed on November 10, 2020, entitled "Techniques for Modifying Cluster Computing Environments", the disclosure of which is hereby incorporated by reference in its entirety for all purposes.

Background Art

[0002] Background Distributed computing systems are becoming increasingly popular. These systems may include a computing cluster of connected nodes (e.g., computers, servers, virtual machines, etc.) that cooperate and collaborate to process various requirements (e.g., requiring storage and / or retrieval of data in a system maintaining a database). As the number of tasks increases or decreases, the number of connected nodes may not reach an optimal state. For example, when the number of tasks decreases, the number of nodes may be more than required for the pending tasks, and some nodes may be left idle. Conversely, when the number of tasks increases, the number of nodes may be less than what is required to efficiently process the pending tasks, thus introducing a longer waiting time to execute the pending tasks. Improvements may be made to the way conventional systems manage the number of nodes in a computing cluster. Embodiments of the present disclosure address these and other problems, individually as well as collectively.

Summary of the Invention

[0003] Summary Techniques are provided for adjusting the number of nodes in a computing cluster in response to actual and / or predicted changes in one or more performance metrics of the computing cluster (e.g., methods, systems, non-temporary computer-readable media for storing code or instructions executable by one or more processors). This specification describes various embodiments, including methods, systems, and non-temporary computer-readable storage media for storing programs, code, or instructions executable by one or more processors.

[0004] One embodiment relates to a method for adjusting the number of compute nodes in a compute cluster based at least in part on actual and / or predicted changes in one or more performance metrics of the compute cluster. The method may include a compute service monitoring one or more performance metrics of a set of worker nodes in the compute cluster. The method may further include the compute service detecting that a performance metric falls below a performance threshold. In response to detecting that a performance metric is below a performance threshold, the method may further include the compute service performing a first adjustment to the number of worker nodes in the set of worker nodes in the compute cluster. The method may further include the compute service obtaining training data for a machine learning model based at least in part on performing the first adjustment. The method may further include the compute service using the training data and a supervised machine learning algorithm to train a machine learning model. The method may also include the compute service obtaining an output showing predicted performance changes in the compute cluster. The method may further include the following: In some embodiments, the output is obtained at least in part on providing one or more subsequent performance metrics of the computing cluster as input to a machine learning model. The method may further include the computing service performing a second adjustment to a set of worker nodes at least in part on the output showing the expected performance changes in the computing cluster.

[0005] In some embodiments, coordinating a set of worker nodes further includes the computing service generating a scaling task, which is executed by a computing process, and the computing process updates metadata associated with the computing cluster upon completion of the scaling task.

[0006] In some embodiments, the output showing the predicted performance change indicates how many worker nodes will be used to compute the task at a later time, which occurs within a predetermined period in the future.

[0007] In some embodiments, performing a first adjustment or a second adjustment includes increasing or decreasing the number of worker node sets.

[0008] In some embodiments, performing the first adjustment involves provisioning a certain number of additional worker nodes to the set of worker nodes in the compute cluster. In some embodiments, the method may further include determining that provisioning a certain number of additional worker nodes resulted in subsequent performance metrics exceeding a performance threshold, and training data is generated in response to determining that a certain number of additional worker nodes resulted in subsequent performance metrics.

[0009] In some embodiments, the training data includes one or more performance metrics, subsequent performance metrics, and the number of additional worker nodes provisioned during the first period. In some embodiments, one or more performance metrics include at least one of the following: the number of pending queries, the number of pending tasks, latency measurements, processing utilization, or memory utilization.

[0010] Another embodiment relates to a computing device. The computing device may include a computer-readable medium for storing non-temporary computer executable program instructions. The computing device may further include a processing device communicatively coupled to the computer-readable medium for executing the non-temporary computer executable program instructions. The computing device is made to perform an operation by executing the non-temporary computer executable program instructions using the processing device. These operations may include monitoring one or more performance metrics of a set of worker nodes in a computing cluster. In some embodiments, at least one of these metrics may be measured and / or collected by the worker nodes, but the metric (e.g., throughput metric) is related to an external service or computing device (e.g., object storage service). The operation may further include detecting that the performance metric falls below a performance threshold. The operation may further include performing a first adjustment to the number of worker nodes in the set of worker nodes in the computing cluster in response to detecting that the performance metric has fallen below a performance threshold. The operation may further include obtaining training data for a machine learning model based at least in part on performing the first adjustment. The operation may further include training a machine learning model using training data and supervised machine learning algorithms. The operation may further include obtaining an output showing the expected performance changes in the computing cluster. In some embodiments, the output is obtained at least in part on providing one or more subsequent performance metrics of the computing cluster as input to the machine learning model. The operation may further include performing a second adjustment on a set of worker nodes at least in part on the output showing the expected performance changes in the computing cluster.

[0011] Another embodiment relates to a non-temporary computer-readable storage medium that, when executed by a processing device of a computing device, stores computer executable program instructions that cause the computing device to perform an action. These actions may include monitoring one or more performance metrics of a set of worker nodes in a computing cluster. The actions may further include detecting that the performance metric falls below a performance threshold. The actions may further include performing a first adjustment to the number of worker nodes in the set of worker nodes in the computing cluster in response to detecting that the performance metric has fallen below a performance threshold. The actions may further include obtaining training data for a machine learning model based at least in part on performing the first adjustment. The actions may further include training the machine learning model using the training data and a supervised machine learning algorithm. The actions may further include obtaining an output that shows the expected performance change in the computing cluster. In some embodiments, the output is obtained based at least in part on providing one or more subsequent performance metrics of the computing cluster as input to the machine learning model. The operation may further include performing a second adjustment to a set of worker nodes, at least in part, based on an output indicating the expected performance changes in the computing cluster.

[0012] The foregoing, along with other features and embodiments, will become more apparent with reference to the following specification, claims, and accompanying drawings.

[0013] Brief explanation of the drawing Various embodiments of this disclosure will be described with reference to the drawings. [Brief explanation of the drawing]

[0014] [Figure 1]This figure shows a flow for adjusting the number of nodes in a compute cluster based at least in part on actual and / or predicted changes in the performance of the compute cluster, according to at least one embodiment. [Figure 2] This figure shows a flow for training a machine learning model to predict changes in the performance of a computing cluster, according to at least one embodiment. [Figure 3] This figure shows a timeline illustrating an exemplary use case for performing coordination on a computing cluster using at least one embodiment. [Figure 4] This figure shows the components of a computing system according to at least one embodiment. [Figure 5] This is a schematic diagram of an exemplary computer architecture for an autoscaling engine, including multiple modules that may perform functions according to at least one embodiment. [Figure 6] This flowchart shows an example of a method for adjusting the number of nodes in a compute cluster based at least in part on actual and / or predicted changes in the performance of the compute cluster, according to at least one embodiment. [Figure 7] This is a block diagram showing one pattern for realizing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 8] This block diagram shows another pattern for realizing cloud infrastructure as a service system, according to at least one embodiment. [Figure 9] This block diagram shows another pattern for realizing cloud infrastructure as a service system, according to at least one embodiment. [Figure 10] This block diagram shows another pattern for realizing cloud infrastructure as a service system, according to at least one embodiment. [Figure 11] A block diagram showing an exemplary computer system according to at least one embodiment. [Modes for carrying out the invention]

[0015] Detailed explanation In the following description, certain details are included for illustrative purposes to facilitate a full understanding of the particular embodiment. However, it will be apparent that various embodiments may be carried out without these specific details. The figures and descriptions are not intended to be limiting. The term “exemplary” is used here to mean “provided as an example, case, or illustration.” Any embodiment or design described herein as “exemplary” should not necessarily be construed as being preferable or advantageous over other embodiments or designs.

[0016] This disclosure relates to systems and techniques for adjusting the number of compute nodes in a compute cluster based at least in part on actual and / or predicted changes in cluster performance. A “compute cluster” may include any preferred number of nodes that operate in a cooperative manner. In some embodiments, the compute cluster may be provided in a cloud computing environment. As used herein, a “compute node” (also referred to as a “worker node” and / or “node”) may include servers, compute devices, virtual machines, or any preferred physical or virtual computing resources configured to operate as part of the compute cluster. For example, a compute cluster may include one or more master nodes and one or more worker nodes, both of which are examples of compute nodes. In some embodiments, the master node performs any preferred operation related to task assignment, load balancing, node provisioning, node removal, or any preferred operation related to managing the compute cluster, corresponding to one or more worker nodes. The worker nodes are configured to perform operations corresponding to tasks assigned to them by one or more master nodes. In a non-limiting example, a worker node may perform data storage and / or data retrieval tasks associated with a database, at the direction of a master node that assigns a particular storage / retrieval task to the worker node.

[0017] In some embodiments, an auto-scaling engine (e.g., operating on a manager node of a computing cluster or a computing device) may monitor the performance of a computing cluster with respect to one or more performance metrics and / or performance thresholds. By way of example, each computing node within the computing cluster may be configured to perform operations such that it reports one or more performance metrics (e.g., number of queued queries, number of queued tasks, latency measurements, processing utilization, memory utilization, etc.). These performance metrics can be collected and stored within a centralized data store accessible to the auto-scaling engine. The auto-scaling engine monitors these performance metrics to identify when the performance of the computing cluster fails to meet performance requirements (e.g., one performance metric falls below or exceeds one or more predetermined performance thresholds). In response to detection of this scenario, the auto-scaling engine may adjust the number of computing nodes within the cluster. As a non-limiting example, if a performance metric indicates that the latency of the system has exceeded a predetermined performance requirement (e.g., a predetermined performance requirement related to the latency of task completion), the auto-scaling engine may perform operations to increase the number of computing nodes within the cluster in an attempt to reduce the latency associated with task execution. As another example, if a performance metric indicates that the number of idle computing nodes exceeds a predetermined threshold (e.g., 1, 3, 4, 20, etc.), the auto-scaling engine may perform operations to reduce the number of computing nodes within the cluster.

[0018] First, adjustments to the cluster resulting from the cluster's actual performance may be made according to a predetermined scheme. For example, the system may be configured to increase or decrease the number of compute nodes in the cluster by a predetermined default amount (e.g., 1, 5, 10, etc.) depending on the manner in which the cluster's actual performance does not meet performance requirements (e.g., identified from one or more performance metrics provided by any suitable number of compute nodes in the cluster). After adjustments have been made (e.g., an increase has been made), the performance metrics of the compute nodes may be monitored to identify changes in performance. If the adjustments result in further degradation of the cluster's performance (e.g., increased latency), the autoscaling engine may attempt another adjustment (e.g., a reverse adjustment, such as reducing the number of compute nodes) to improve the cluster's performance. If the adjustments bring the cluster's performance closer to the performance requirements, additional adjustments of a similar nature (e.g., adding an additional set of nodes) may be performed. When the autoscaling engine detects that the cluster meets performance requirements (e.g., latency metrics are below a given latency threshold, and the number of idle compute nodes is below a given idle threshold), the autoscaling engine may store any suitable data applicable to the period between when a performance degradation was identified and when the performance degradation was corrected. In other words, any performance metrics (e.g., latency metrics, number of pending tasks, number of active queries, etc.) and / or cluster metadata (e.g., number of manager nodes, number of worker nodes, central processing units for worker / manager nodes, memory allocation for worker / manager nodes, etc.) collected within the threshold period before or after a performance degradation is detected or rectified in response to the detected degradation may be stored for subsequent use (e.g., as an instance of training data), along with an indicator showing whether the rectification was successful or unsuccessful.

[0019] In some embodiments, the automatic scaling engine may be configured to train a machine learning model using the collected training data and supervised learning techniques. As used herein, a "machine learning model" is a mathematical function that maps inputs to outputs. In some embodiments, the machine learning model may be generated at least in part based on utilizing a supervised learning algorithm with a training data set that includes exemplary inputs with predetermined / labeled outputs. Using previously collected training data (e.g., training data collected in the manner discussed above) and a supervised learning algorithm (e.g., linear regression), the machine learning model is trained to predict performance changes for a cluster. As an example, using the above collected training data, the machine learning model may take as input a performance metric and / or cluster metadata and be trained to identify how many executors (e.g., worker nodes) will be needed at a later time (e.g., in the next 15 minutes).

[0020] The disclosed technology offers improvements over conventional systems. For example, conventional systems may utilize a fixed schedule to control increases and decreases in a compute cluster (e.g., increasing the compute cluster to x nodes at 9am and decreasing it to y nodes at 10am). However, utilizing a schedule does not allow the cluster to respond to actual or potential performance changes, and therefore the number of compute nodes may not reach the optimal state. As another example, some conventional systems increase the number of nodes in a compute cluster based on responding to incoming workloads. However, since node provisioning can take time (e.g., 10 minutes, 15 minutes, etc.), a system that modifies a compute cluster to respond to an incoming workload may still experience suboptimal performance while the new compute nodes are being prepared. Furthermore, conventional systems do not utilize the types of performance metrics used herein to identify cluster performance. As an example, the compute worker nodes disclosed herein may utilize a distributed data processing engine (e.g., Apache Spark®) to provide performance metrics that can determine cluster changes. By utilizing these metrics, the disclosed systems and methods provide a more accurate assessment of both the actual and predicted performance of the computing cluster than was previously achievable.

[0021] Moving to Figure 1, this illustrates a flow 100 for adjusting the number of nodes in a computing cluster based at least in part on actual and / or predicted changes in the performance of the computing cluster, according to at least one embodiment. The cloud computing system 100 may include a cloud computing environment, which includes a computing cluster 102. The operation of flow 100 may be performed by any suitable computing node of the computing cluster 102. The computing cluster 102 may be a distributed system configured to perform operations in a cooperative manner using any suitable number of executors (e.g., execution engines running on each computing node, configured to perform operations corresponding to tasks assigned to the nodes). The computing nodes of the computing cluster 102 may individually be any suitable physical device or virtual computing resource, such as a server computer, desktop computer, laptop computer, tablet computer, or virtual machine. As a non-limiting example, the computing cluster 102 may be a cloud computing environment, which includes many computing instances running each virtual machine and / or computing device.

[0022] The auto-scaling engine 104 may be configured to run on any suitable computing node of the computing cluster 102, or as a separate service and / or device from the computing cluster 102. In some embodiments, the computing cluster 102 may include any suitable number of manager and / or worker nodes. The manager node may be configured to receive task requests (e.g., from client devices not shown) and assign the execution of the corresponding tasks to the worker nodes of the computing cluster 102. Each worker node may operate an executor module (e.g., a software or hardware module such as the execution module 478 in Figure 4) configured to perform actions in response to receiving task data from the manager node. In some embodiments, each worker node and / or manager node of the computing cluster 102 may provide each node with the current performance of each node and / or the computing cluster The computing cluster 102 may be configured to run a distributed data processing engine that provides performance metrics indicating the current performance of 102. In some embodiments, the nodes may provide these performance metrics directly to the metrics data store 106, or the metrics may be collected by a data collector (e.g., one of the nodes in the computing cluster 102) and then stored in the metrics data store 106 by the data collector. For illustrative purposes, it is assumed that the computing cluster 102 provides metrics to the metrics data store before the start of flow 100, and that the metrics continue to be provided periodically by the nodes of the computing cluster 102 during flow 100. As an example, each node may be programmed to provide metrics in response to changes in metrics or at any convenient time, according to a predetermined periodicity, frequency, and schedule.

[0023] Flow 100 may start at 108, and the autoscaling engine 104 monitors the performance of a set of worker nodes in the compute cluster 102. For example, the autoscaling engine 104 may retrieve one or more performance metrics 112 from the metric data store 106 (e.g., at any preferred time, according to a predetermined schedule, periodicity, frequency, change, etc.). The metrics may be related to one or more worker nodes in the compute cluster 102 and / or the compute cluster 102 as a whole. For example, one or more performance metrics may include any preferred combination of the following: the number of pending tasks (e.g., tasks waiting to be executed), the number of pending queries (total number of queries, also called task requests) that do not have tasks currently running or completed, the average / minimum / maximum memory utilization of the physical memory of one or more worker nodes, the number of tasks completed in the last x minutes, the current number of executors (e.g., worker nodes), the average / minimum / maximum central processing unit (CPU) utilization of worker nodes across available cores, average task completion time, etc. A table of exemplary performance metrics is provided in relation to Figure 5.

[0024] In 110, the autoscaling engine 104 may detect an indication that the performance of the compute cluster 102 has not met performance requirements (e.g., a performance metric is above or below a performance threshold). In a non-limiting example, the autoscaling engine 104 may be configured to identify the current number of executors (Ec), the current task completion rate (Tcr), and the number of pending tasks (Pt) from the performance metrics. In some embodiments, the autoscaling engine 104 may determine the number of additional worker nodes required to complete the remaining pending tasks (e.g., Ec = Pt / Tcr). The autoscaling engine 104 may be configured with one or more performance thresholds and one or more rules to adjust the compute cluster 102 according to the current performance metrics and one or more performance thresholds. In the example in progress, if the number of additional worker nodes is greater than a given threshold (e.g., 0, 5, 7, etc.), the autoscaling engine 104 may be configured to provision additional worker nodes to the compute cluster 102. In some embodiments, the number to be provisioned may be predefined by one or more rules in the rule set (e.g., add 5, add 10, etc.), or the number to be provisioned may be defined within the rule set as corresponding to a calculation to be performed by the auto-scaling engine 104 (e.g., Ec = Pt / Tcr). Thus, in this example, the auto-scaling engine 104 may provision a default number of additional worker nodes, or potentially a specific number of additional worker nodes determined from the formula Ec = Pt / Tcr, according to the rule set. In some embodiments, the specific number of additional worker nodes may be determined using the formula Ec = w × (Pt / Tcr), where w is a fixed or dynamic weight value.

[0025] As another non-restrictive example, a rule set may specify that if the average task completion time exceeds a predetermined latency threshold, a fixed number of worker nodes (e.g., 2, 5, 10, etc.) or a number of working nodes calculated using a predetermined formula should be added to the compute cluster 102. As yet another non-restrictive example, a rule set may specify that if the number of pending tasks, the CPU utilization of worker nodes, the memory utilization of worker nodes, etc., exceed predetermined performance thresholds (e.g., more than 10 pending tasks, or CPU and / or memory utilization exceeding 90%), a fixed number of worker nodes (e.g., 2, 5, 10, etc.) or a number of working nodes calculated using a predetermined formula should be added to the compute cluster 102. As yet another example, a rule set may specify that if predetermined thresholds (e.g., a predetermined latency threshold, a predetermined CPU utilization threshold, etc.) are exceeded, a parallel compute cluster may be created using a predetermined number of nodes. The parallel compute cluster may run separately from the original compute cluster, and at least some of the tasks assigned to the original compute cluster may be transferred to the parallel compute cluster. In some embodiments, any suitable number of computing clusters may be generated in this manner. It should be noted that the number and complexity of the specific rules in the rule set that identify a particular action to be performed in response to a particular stimulus may vary.

[0026] In 114, the autoscaling engine 104 may perform any preferred action to adjust the number of worker nodes in the compute cluster 102. For example, the autoscaling engine 104 may generate a separate process to adjust the number of nodes in the compute cluster 102 (e.g., to add and / or remove a specific number of worker and / or manager nodes). In some embodiments, the autoscaling engine 104 may execute this process immediately or add it to a pending queue for later execution. This process may be configured to perform any preferred action to provision (e.g., add) one or more additional nodes (e.g., worker and / or manager nodes) in the compute cluster 102, or to remove one or more nodes in the compute cluster 102, as directed by the autoscaling engine 104. In a non-limiting example, this process may perform an action to add a worker node 116 to the compute cluster 102.

[0027] In 118, the autoscaling engine 104 may be configured to generate training data based at least in part on the adjustments performed in 114. For example, the autoscaling engine 104 may collect one or more performance metrics (also called "cluster metadata") that occur during a period before adjustments are made, during adjustments, and / or during a period after adjustments are made. In some embodiments, one or more performance metrics may be collected from a predetermined period before adjustments are made (e.g., 5 minutes, 2 minutes, etc.), and / or one or more performance metrics may be collected based at least in part on the first occurrence of computing cluster 102 failing to meet a particular performance threshold. Similarly, if one or more performance metrics are collected after adjustments, a particular performance metric may be collected at least in part on a predetermined period after adjustments are made (e.g., 3 minutes, 6 minutes), and / or at least in part on identifying the time when the performance metrics of computing cluster 102 indicated that the performance degradation had been corrected (e.g., the performance metrics indicated that computing cluster 102 was meeting the performance threshold after the adjustments were made). As a non-specific example, the original performance degradation occurs starting 5 minutes before the adjustment (for example, Performance metrics of compute cluster 102 that occurred over the time period during which performance metrics indicated that latency (exceeding a predetermined threshold) was corrected (e.g., latency met or fell below a predetermined threshold) and / or over a predetermined period after the adjustment was completed (e.g., 5 minutes after a compute node was added / removed).

[0028] In some embodiments, the auto-scaling engine 104 may generate a sample training dataset containing the collected metrics, the number of computing nodes added / removed, and a label (e.g., successful / unsuccessful) indicating whether the adjustment corrected performance degradation. The sample training dataset may be added to a training dataset stored in memory (e.g., training dataset 120), which is later used to train a machine learning model (e.g., machine learning model 122) to identify adjustments (e.g., output 124) that should be made to the computing cluster 102 based at least partially on predictions of performance metrics over time (e.g., how many computing nodes will be needed in the next 15 minutes). The machine learning model 122 may be trained at least partially on the flow 200 described in relation to Figure 2.

[0029] In 126, the autoscaling engine 104 may predict that the performance of the computing cluster 102 will not meet a second performance requirement. For example, the autoscaling engine 104 may collect current performance metrics and provide these metrics as input to a machine learning model 122. The machine learning model 122 may generate an output 124 indicating whether changes to the computing cluster 102 are needed. In some embodiments, the output 124 may indicate one or more performance requirements that are expected to decline (e.g., latency exceeds a predetermined latency threshold at 15 minutes, number of pending tasks exceeds a predetermined threshold), and / or the number of computing nodes (worker and / or manager nodes) required to handle the expected pending tasks (e.g., the number of pending tasks expected to exist in the next 15 minutes), and / or the number of nodes (worker and / or manager nodes) that should be added to and / or removed from the computing cluster 102. Given the number of computing nodes required to handle the predicted pending tasks, the auto-scaling engine 104 may be configured to compare this number to the current number of nodes in the computing cluster 102 to determine how many more or fewer computing nodes 102 are needed.

[0030] In 128, the autoscaling engine 104 may adjust the compute cluster 102 based at least in part on a prediction (e.g., output 124) provided by the machine learning model 122. For example, the autoscaling engine 104 may generate a separate process to adjust the number of nodes in the compute cluster 102 (e.g., to add and / or remove a specific number of worker and / or manager nodes) based at least in part on output 124 (e.g., an output indicating that additional worker nodes are needed). In some embodiments, the autoscaling engine 104 may execute this process immediately or add it to a pending queue for later execution. This process may be configured to perform any preferred action to provision (e.g., add) one or more additional nodes (e.g., worker and / or manager nodes) to the compute cluster 102, or to remove one or more nodes from the compute cluster 102, as directed by the autoscaling engine 104. In a non-limiting example, this process may perform an action to add a worker node 130 to the compute cluster 102.

[0031] Figure 2 shows a flow 200 for training a machine learning model to predict changes in the performance of a computing cluster, according to at least one embodiment. In some embodiments, the operation of flow 200 may be performed by an autoscaling engine, which will be described later with respect to Figures 4 and 5.

[0032] In some embodiments, the machine learning model 202 may be configured to accept a training dataset 204 as input. The training dataset 204 may be retrieved by the machine learning model 202 from a data store (e.g., the metrics data store 106 in Figure 1). The training dataset 204 (also referred to as the “training dataset” or “training data”) may include any suitable number of positive and / or negative examples. An exemplary instance of the training dataset may include performance metrics and / or cluster metadata collected within a threshold period of cluster tuning (e.g., a predetermined period before or after cluster tuning), along with an indication of whether the changes corrected a performance degradation. Thus, a positive example may be considered an example where the performance metrics and / or cluster metadata corrected a performance degradation in the cluster tuning, and a negative example may be considered an example where the performance metrics and / or cluster metadata did not correct a performance degradation in the cluster tuning. The machine learning model 202 may be trained with the training dataset 204 to identify specific tunings that should be made to the compute cluster based on the current performance metrics and / or cluster metadata. In some embodiments, the operation of flow 200 may be performed any number of preferred times to train any number of preferred machine learning models and detect their respective cluster adjustments for different computing clusters. For example, machine learning model 202 may be trained to identify adjustments for a particular computing cluster based on the historical cluster changes of that particular computing cluster.

[0033] Once trained, the machine learning model 202 may be used to generate outputs 206. In some embodiments, a portion of the training dataset 204 (e.g., two-thirds) may be used for training purposes, and another portion of the training dataset 204 (e.g., one-third) may be used for validation. For example, the machine learning model 202 may be trained using two-thirds of the examples in the training dataset 204. Once trained, each of the remaining one-third of the examples in the training dataset 204 may be individually provided as input to the machine learning model 202 to generate outputs 206 (e.g., individual instances of the outputs corresponding to each input). Outputs 206 may be compared to each indicator in the training dataset 204 to determine whether the machine learning model 202 has produced outputs that match each indicator in the training dataset (e.g., indicating whether the individual adjustments corrected or did not correct performance degradation). In some embodiments, the machine learning model 202 may not be expanded until it produces outputs that exceed a predetermined accuracy threshold (e.g., 90% or more of the time, the model produces outputs that match each indicator in the training dataset 204). In some embodiments, additional training datasets may be collected and used to retrain the collected machine learning model 202 until it exceeds a predetermined accuracy threshold. Once the machine learning model 202 produces output with sufficient accuracy, it may be deployed to any suitable number of computing resources and used for subsequent cluster tuning.

[0034] When the machine learning model 202 is used to determine cluster tuning, the output 206 provided by the machine learning model 202 may be used in conjunction with a feedback procedure 208 to improve the accuracy of the machine learning model 202. In some embodiments, the feedback procedure 208 may include collecting data (e.g., performance metrics and / or cluster metadata) within a predetermined period before or after tuning (e.g., tuning identified by the machine learning model 202) is performed. By analyzing the output 206 and the collected data using a predetermined set of rules, the machine learning model It may be determined whether the adjustments identified by 202 (e.g., output 206) resulted in a correction of performance degradation. Another example is output 206 and / or any preferred portion of the collected data for one or more users and / or one or more remote systems (e.g., cluster administrator devices, crowdsourced data collection systems, etc.). A user of the crowdsourced system (e.g., a user with cluster management experience) may provide input indicating whether the adjustments identified by the machine learning model 202 (e.g., output 206) resulted in a correction of performance degradation of the compute cluster.

[0035] The feedback obtained through the execution of the feedback procedure 208 (e.g., indicating whether the adjustments identified by the machine learning model 202 (e.g., output 206) resulted in a correction of performance degradation in the computing cluster) may be provided to the machine learning model 202 in 210 to improve the accuracy of the machine learning model 202 by expanding the training dataset from which the machine learning model 202 is ultimately retrained. Thus, over time, the machine learning model 202 may become increasingly accurate in identifying adjustments to the computing cluster to correct various performance degradations.

[0036] Figure 3 shows a timeline 300 to illustrate an exemplary use case for performing adjustments to a computing cluster by at least one embodiment. The operations described in relation to timeline 300 may be performed by the auto-scaling engine 104 of Figure 1.

[0037] At time T1, the computing cluster 302 may include any suitable number of computing nodes. For example, the computing cluster 302 may include one manager node and two worker nodes. Each node of the computing cluster 302 may provide one or more performance metrics indicating its current performance and / or the performance of the computing cluster 302 as a whole. These performance metrics may be provided at any suitable time along the timeline 300 according to a predetermined schedule, periodicity, frequency, and / or based at least in part on changes in the performance metrics.

[0038] At time T2, the autoscaling engine 104 may determine that a performance metric indicates a performance degradation has occurred (e.g., the performance metric has exceeded or fallen below a predetermined performance threshold). This determination may follow a predetermined set of rules configured for access by the autoscaling engine 104. As a non-limiting example, the autoscaling engine 104 may determine at time T2 that a latency metric (e.g., average task completion time) exceeds a predetermined latency threshold. In response to the determination made at time T2, the autoscaling engine 104 may identify that an additional compute node (e.g., worker node 304) should be provisioned and added to the compute cluster 302.

[0039] At time T3, the auto-scaling engine 104 may perform any preferred action to provision a worker node 304 and add the worker node 304 to the compute cluster 302.

[0040] At time T4, the autoscaling engine 104 may add one or more performance metrics corresponding to the adjustments to the training dataset 306. For example, the autoscaling engine 104 may adjust the performance degradation between time T2 and time T4 (for example, for a predetermined period after the adjustments made at time T3) and then adjust the performance degradation. A sample training dataset may be generated that includes performance metrics collected at a time (such as a time defined by a predetermined schedule), the number of nodes added to / removed from the computing cluster (for example, one in this case), and a label indicating whether the adjustment corrected the performance degradation. The autoscaling engine 104 may then add the sample training dataset to the training dataset 306.

[0041] At time T5, the autoscaling engine 104 may determine that a performance metric indicates another performance degradation has occurred (for example, that a performance metric has exceeded or fallen below a predetermined performance threshold). This determination may again follow a predetermined set of rules configured for access by the autoscaling engine 104. In a non-limiting example, the autoscaling engine 104 may determine at time T5 that a latency metric has again exceeded a predetermined latency threshold. In response to the determination made at time T2, the autoscaling engine 104 may identify that an additional compute node (for example, a worker node 308) should be provisioned and added to the compute cluster 302.

[0042] At time T6, the auto-scaling engine 104 may perform any preferred action to provision a worker node 308 and add the worker node 308 to the compute cluster 302.

[0043] At time T7, the autoscaling engine 104 may add one or more performance metrics corresponding to the adjustments made at time T6 to the training dataset 306. For example, the autoscaling engine 104 may generate a sample training dataset that includes performance metrics collected between time T5 and time T7 (e.g., a predetermined period after the adjustments made at time T5, the time at which the autoscaling engine 104 determines that the performance degradation has been corrected, a time defined by a predetermined schedule, etc.), the number of nodes added / removed from the compute cluster (e.g., one in this case), and a label indicating whether the adjustments corrected the performance degradation. The autoscaling engine 104 may then add the sample training dataset to the training dataset 306.

[0044] At time T8, the auto-scaling engine 104 may run any suitable supervised machine learning algorithm on the training dataset 306 to train the machine learning model 310. The machine learning model 310 may be trained to identify subsequent adjustments to be made to the computing cluster, at least in part on inputs including a set of current performance metrics of the computing cluster 102.

[0045] At time T9, the auto-scaling engine 104 may provide a set of current performance metrics as input to the machine learning model 310. In response, the machine learning model 310 may generate an output 312 indicating that additional worker nodes are needed. This output 312 may be generated at least in part by the machine learning model 310 and may predict that a performance metric degradation will occur in a subsequent period (e.g., the next 15 minutes), that x computing nodes (e.g., one) will be needed to avoid the performance degradation in the subsequent period, and so on.

[0046] At time T10, the auto-scaling engine 104 provisions worker node 314 and adds worker node 314 to compute cluster 302. You may perform any suitable action.

[0047] At time T11, the autoscaling engine 104 may add one or more performance metrics corresponding to the adjustments made at time T10 to the training dataset 306. For example, the autoscaling engine 104 may generate a sample training dataset that includes performance metrics collected between time T9 and time T11 (e.g., a predetermined period after the adjustments made at time T10, a time at which the autoscaling engine 104 determines that performance degradation has been avoided, a time defined by a predetermined schedule, etc.), the number of nodes added / removed from the compute cluster (e.g., one in this case), and a label indicating whether the adjustments were successful (e.g., whether performance degradation was avoided). The autoscaling engine 104 may then add the sample training dataset to the training dataset 306.

[0048] At time T12, the auto-scaling engine 104 may provide a set of current performance metrics as input to the machine learning model 310. In response, the machine learning model 310 may generate an output 316 indicating that several (e.g., three) worker nodes of the computing cluster 302 will be idle during a subsequent period (e.g., the next 15 minutes). This output 316 may be generated at least in part on the machine learning model 310 predicting that a performance metric degradation will occur during a subsequent period (e.g., the next 15 minutes), and that x (e.g., one) computing nodes should be removed to avoid performance degradation during that subsequent period (e.g., to avoid idle computing nodes).

[0049] At time T13, the auto-scaling engine 104 may perform any preferred action to remove one or more nodes (for example, three worker nodes) from the computing cluster 302.

[0050] At time T14, the autoscaling engine 104 may add one or more performance metrics corresponding to the adjustments made at time T13 to the training dataset 306. For example, the autoscaling engine 104 may generate a sample training dataset that includes performance metrics collected between time T12 and time T14 (e.g., a predetermined period after the adjustments made at time T10, a time at which the autoscaling engine 104 determines that performance degradation has been avoided, a time defined by a predetermined schedule, etc.), the number of nodes added / removed from the compute cluster (e.g., one in this case), and a label indicating whether the adjustments were successful (e.g., whether performance degradation was avoided). The autoscaling engine 104 may then add the sample training dataset to the training dataset 306.

[0051] Figure 4 shows components of a computing system 400 according to at least one embodiment. In system 400, one or more users 402 may use user devices (e.g., one user device from a collection of user devices 404) to provide various computing requests to a distributed system 406 via a network 408. In some examples, the network 408 may include any one or a combination of many different types of networks, such as cable networks, the internet, wireless networks, cellular networks, and other private and / or public networks. The distributed system 406 (e.g., a computing cluster) may be an example of a computing cluster (e.g., computing cluster 102 in Figure 1). In some embodiments, the distributed system 406 may have any preferred number of management nodes (e.g., manager nodes 410) and any preferred number of w This may include user nodes (e.g., worker node 412) and, for brevity, will be collectively referred to as “computing nodes” or “nodes.” For example, a user may access a user interface accessible through an application 414 running on a user device 404. In some cases, the application 414 running on the user device 404 may be hosted, managed, and / or provided by computing resources of the distributed system 406 (e.g., manager node 410 and / or separate computing resources of the distributed system 406 not shown in Figure 4).

[0052] In some embodiments, application 414 may enable user 402 to interact with distributed system 406. For example, the user may request data using an interface provided by application 414. The distributed system 406 may then be configured to process the request and return the requested data to user device 404 via application 414. Application 414 may also be configured to receive, process, and / or display data provided by the distributed system 406.

[0053] The user device 404 may be any suitable type of computing device, but is not limited to, a mobile phone, handheld scanner, touchscreen device, smartphone, personal digital assistant (PDA), laptop computer, desktop computer, thin client device, tablet PC, or e-book reader. In some examples, the user device 404 may communicate with the distributed system 406 via the network 408 or other network connection.

[0054] In one exemplary configuration, the user device 404 may include at least one memory 416 and one or more processing units (or processors) 418. The processor 418 may be implemented in hardware, computer-executable instructions, firmware, or a combination thereof, as appropriate. An example of a computer-executable instruction or firmware implementation of the processor 418 may include computer-executable or machine-executable instructions written in any preferred programming language to perform the various functions described.

[0055] Memory 416 may store program instructions that can be loaded and executed on the processor 418, as well as data generated during the execution of these programs. Depending on the configuration and type of the user computing device, memory 416 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). User device 404 may also include, but not limited to, magnetic storage devices, optical disks, and / or tape storage devices, as well as additional removable and / or non-removable storage devices. Disk drives and their associated computer-readable media may provide non-volatile storage for computer-readable instructions, data structures, program modules, and other data for the computing device. In some implementations, memory 416 may include several different types of memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), or ROM.

[0056] More specifically, the contents of memory 416 may include the operating system 420, one or more data stores 422, and one or more application programs, modules, or services provided through an application 414 (e.g., a browser application, a video player application, etc.). The application 414 may be configured to receive, store, and / or display network pages or other interfaces for interacting with the distributed system 406. In addition, memory 416 may, but is not limited to, user IDs, passwords, and / or The system may store access certificates and / or other user information, such as other user information. In some examples, user information may include, but is not limited to, information used to authenticate account access requests, such as device ID, cookies, IP address, and location.

[0057] The user device 404 may also include a communication connection 418 that enables the user device 404 to communicate with a stored database, another computing device or server (e.g., a distributed system 406), a user terminal, and / or other devices on the network 408. The user device 404 may also include I / O devices 420 such as a keyboard, mouse, pen, voice input device, touch input device, display, speaker, and printer.

[0058] In some aspects, the manager node 410 and worker nodes 412 (collectively referred to as “nodes”) of the distributed system 406 may also be any suitable type of computing device, without limitation, such as mobile phones, smartphones, personal digital assistants (PDAs), laptop computers, desktop computers, server computers, thin client devices, and tablet PCs. Furthermore, it should be noted that in some embodiments, the nodes of the distributed system 406 are run by one or more virtual machines implemented in a hosted computing environment. The hosted computing environment may include one or more rapidly provisioned and released computing resources, which may include computing devices, networking devices, and / or storage devices. The hosted computing environment may also be referred to as a cloud computing environment. In some examples, the nodes of the distributed system 406 may communicate with user devices 404 via network 408 or other network connections. The nodes of the distributed system 406 may include one or more servers, possibly configured in a cluster, as a server farm or as individual servers that are not related to one another. These servers may be configured to implement the functions described herein as part of an integrated distributed computing environment.

[0059] In one exemplary configuration, the manager node 410 and the worker node 412 may each include at least one memory 428 and memory 448, and one or more processing units (or processors 430 and 450, respectively). The processors 430 and 450 may be implemented as hardware, computer-executable instructions, firmware, or a combination thereof, as appropriate. The computer-executable instruction or firmware implementations of the processors 430 and 450 may include computer-executable or machine-executable instructions written in any preferred programming language to perform the various functions described.

[0060] Memory 428 and memory 448 may store program instructions that can be loaded and executed on processors 430 and 450, respectively, as well as data generated during the execution of these programs. Depending on the node configuration and type, memory 428 and memory 448 may be volatile (such as RAM) and / or non-volatile (such as ROM, flash memory), respectively. Manager node 410 and worker node 412 may each include additional storage 432 and 452, respectively, which may include removable and / or non-removable storage. Additional storage 732 may include, but is not limited to, magnetic storage, optical disks, and / or tape storage. Disk drives and computer-readable media associated with them may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computing devices. In some implementations, memory 428 and memory 448 are SRAM, DRAM It may include multiple different types of memory, such as PMEM, NVMe, or ROM.

[0061] Memory 428, Memory 448, Additional Storage 432, and Additional Storage 452 are all examples of computer-readable storage media, whether removable or non-removable. For example, computer-readable storage media may include volatile or non-volatile removable or non-removable media implemented in any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Memory 428, Memory 448, Additional Storage 432, Additional Storage 452, Centralized Datastore 480, and Metric Datastore 490 are all examples of computer storage media. Additional types of computer storage media that may reside in the nodes and / or datastores of the distributed system 406 may include, but are not limited to, PRAM, SRAM, DRAM, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that may be used to store desired information and may be accessed by the distributed system 406. Any of the above combinations should also be included within the scope of computer-readable media.

[0062] Alternatively, a computer-readable communication medium may include computer-readable instructions, program modules, or other data transmitted in data signals such as carrier waves or other transmissions. However, as used herein, a computer-readable storage medium does not include a computer-readable communication medium.

[0063] The manager node 410 and worker node 412 may also include communication connections 434 and 454, respectively, which enable each node to communicate with stored databases (e.g., centralized data store 480, metric data store 490, etc.), other computing devices or servers, user terminals, and / or other devices on the network 408. The manager node 410 and worker node 412 may also include I / O devices 436 and 456, respectively, which may be keyboards, mice, pens, voice input devices, touch input devices, displays, speakers, printers, etc.

[0064] More specifically, the contents of memory 428 and memory 458 may include operating systems 440 and 460, one or more caches 442 and 462, and / or one or more application programs, modules, or services for implementing the features disclosed herein. For example, memory 428 of manager node 410 may include a management module 472 and an auto-scaling engine 474 (e.g., the auto-scaling engine 104 example in Figure 1). Although the management module 472 and the auto-scaling engine 474 are described as separate modules, in some embodiments both functions may be provided by a single software and / or hardware module that operates on the manager node and / or at least partially on different computing resources of the distributed system 406 (e.g., a computing device (not shown) associated with the distributed system 406 and configured to perform any preferred combination of the functions described herein with respect to Figures 1 to 3).

[0065] In some embodiments, the management module 472 is configured using software and / or hardware to perform operations associated with managing the worker node 412. For example, the management module 472 is executed by the processor 430. The instructions may include commands to cause the manager node 410 to provision a new node to the compute cluster (e.g., compute cluster 102 in Figure 1), remove a node from the compute cluster, load balance, assign tasks, provide one or more performance metrics of the manager node 410 (e.g., to the metric data store 490), or perform any preferred action related to any preferred action related to the management of the worker nodes 412. In some embodiments, the management module 472 includes a distributed data processing engine (e.g., Apache Spark®) that provides various performance metrics that can be used to determine cluster tuning. In some embodiments, the autoscaling engine 474 is configured to perform any preferred action related to determining when the number of worker nodes 412 should be increased and / or decreased, based at least in part on actual and / or predicted performance metrics as well as cluster metadata. In some embodiments, the autoscaling engine 474 may be configured to communicate with the management module 472, the metric data store 490, the worker nodes 412, or any preferred computing resources of the distributed system 406. The specific functions provided by the automatic scaling engine 474 will be discussed in more detail with respect to Figure 5.

[0066] In some embodiments, the execution module 478 performs any preferred actions for executing tasks assigned to the worker node 412 by the manager node 410. In some embodiments, the execution module 478 (also referred to herein as the “executor”) may provide performance metrics associated with the worker node 412. As a non-limiting example, the execution module 478 may include a distributed data processing engine (e.g., Apache Spark®) that provides a variety of performance metrics which may determine cluster tuning. For example, the execution module 478 may be configured to provide any preferred combination of one or more performance metrics in terms of changes occurring with respect to one or more performance metrics according to any preferred predetermined schedule, periodicity, and frequency, and / or according to a set of rules for providing the performance metrics.

[0067] The following table lists some exemplary performance metrics that may be provided by the management module 472, the auto-scaling engine 474, the execution module 478, or any preferred combination of the above.

[0068] [Table 1-1]

[0069] [Table 1-2]

[0070] [Table 1-3]

[0071] [Table 1-4]

[0072] [Table 1-5]

[0073] Figure 5 is a schematic diagram of an exemplary computer architecture for an auto-scaling engine 500 (e.g., the auto-scaling engine 474 in Figure 4), which includes a plurality of modules 502 that may perform functions according to at least one embodiment. Modules 502 may be software modules, hardware modules, or a combination thereof. If module 502 is a software module, module 502 may be embodied on a computer-readable medium and processed by a processor in any of the computer systems described herein. Any module or data store described herein may, in some embodiments, be required to perform the corresponding calculations. It should be noted that this may also be a service responsible for managing data of the type described. Module 502 may exist as part of a management node 410 (or a separate computing resource) operating within the distributed system 406 in Figure 4, or the module may exist as a separate module or service outside the distributed system 406.

[0074] In the embodiment shown in Figure 5, a metric data store 504 (for example, the metric data store 490 in Figure 4) is shown, but content data can be maintained, derived, or otherwise accessed from various data stores that are remote or local to the auto-scaling engine 500 in order to achieve the functions described herein. In at least one embodiment, the data stores described herein may operate as part of a distributed system 406 or may be physically separate from the distributed system 406. As shown in Figure 5, the auto-scaling engine 500 includes various modules such as a data processing module 512, a model management module 514, a responsive scaling module 516, a predictive scaling module 518, a feedback engine 520, and an output module 522. Some functions of module 502 are described below. However, for the benefit of the reader, a brief and non-limiting description of each module is provided in the following paragraphs.

[0075] In at least one embodiment, the auto-scaling engine 500 includes a data processing module 512. Generally, the data processing module 512 may be used to receive any preferred information relating to any example provided herein. The data processing module 512 may include any number of preferred application programming interfaces from which the functions of the auto-scaling engine 500 may be invoked.

[0076] In some embodiments, the data processing module 512 may be configured to monitor one or more performance metrics of the computing resources (e.g., computing devices, virtual machines, etc.) of a distributed system (e.g., distributed system 406). The data processing module 512 may retrieve one or more performance metrics from a data store, a metrics data store 504, configured to store such information. One or more performance metrics may be those initially provided by the management module 472 and / or executor module 478 in Figure 4. The data processing module 512 may be configured with any number of preferred performance thresholds that define performance requirements. For example, the data processing module 512 may be configured with a predetermined latency threshold that indicates the amount of time it takes for a query to complete (e.g., on average for every query, etc.). The data processing module 512 may be configured to determine when the distributed system 406 does not meet the performance thresholds. For example, the data processing module 512 may determine when the average query completion time (e.g., avgCompletionTime listed in the table above) exceeds a predetermined threshold (e.g., a predetermined latency threshold). It is possible. As another non-limiting example, the data processing module 512 may be configured to determine whether there are idle executors (conditions constituting performance degradation) based at least in part on determining the number of executors (e.g., numExecutors), the number of pending tasks (e.g., PendingTasks, tasks waiting to be executed), and the number of active tasks (e.g., Active Tasks). If the number of pending tasks is zero and the number of executors exceeds the number of active tasks, at least one executor may be considered idle. The data processing module 512 may be configured to invoke the functions of any other module of module 502 when it detects performance degradation, or at any preferred time.

[0077] In some embodiments, the model management module 514 is a compute class A model (e.g., machine learning model 202 in Figure 2) may be configured to train a model to predict performance changes within the cluster. In at least one embodiment, the output provided by the model may identify (and / or be used to identify) the total number of worker nodes and / or additional numbers required at a point in the future (e.g., 10 minutes in the future, 15 minutes in the future, etc.). The model management module 814 may retrieve training data from the metric data store 504 (a data store configured to store one or more performance metrics and / or cluster metadata provided by the management node 410 and / or worker node 412 in Figure 4). In some embodiments, the training data (e.g., the training dataset 204 in Figure 2) may be pre-stored in the metric data store 504, or the auto-scaling engine 500 may receive the training dataset from any preferred source (e.g., by the data processing module 512) and store it in the metric data store 504 for later use. The model management module 514 may be configured to train and / or retrain the machine learning model described above using the training set and any suitable supervised, unsupervised, or semi-supervised learning algorithm. For example, the model management module 514 may be configured to perform the operations described above in relation to Figure 2 with respect to training / retraining the machine learning model 202, maintaining the training dataset 204, and / or adding to the training dataset 204. Once the machine learning model has been trained by the model management module 514, the machine learning model may be used to determine cluster tuning based at least in part on subsequent performance metrics / cluster metadata provided as input.

[0078] In some embodiments, the responsive scaling module 516 may be configured with a set of rules defining any number of preferred conditions, so that the detection of a given condition causes the responsive scaling module 516 to adjust the number of compute nodes (e.g., the number of worker nodes 412) upward or downward in response to an action defined in the set of rules. For example, the responsive scaling metric may detect that one or more worker nodes are idle, and according to at least one rule in the set of rules, the responsive scaling metric may determine the number / features of nodes that should be removed from the compute cluster (e.g., distributed system 406). In some embodiments, the responsive scaling module 516 may be configured to instruct the output module 522 to perform actions to add and / or remove some worker nodes (and / or specific worker nodes) from the compute cluster.

[0079] In some embodiments, the predictive scaling module 518 may be configured to use a model trained by the model management module 514 to predict the number of compute nodes that are expected to be needed at some point in the future (e.g., 15 minutes from the current time). The predictive scaling module 518 may retrieve one or more performance metrics from the metric data store 504 (e.g., the most recent metric collected / received by the metric data store 504, metrics collected / received within a period such as the most recent x minutes). Once retrieved, the metrics may be provided by the predictive scaling module 518 to the machine learning model 202 to generate an output (e.g., output 206 in Figure 2). In some embodiments, the output may indicate the number of compute nodes to be added / removed from the cluster, and / or the output may indicate the total number of compute nodes that will be needed at some point in the future (from which the number of compute nodes to be added / removed may be derived). In some embodiments, the predictive scaling module 518 may be configured to instruct the output module 522 to perform actions to add and / or remove some worker nodes (and / or specific worker nodes) from the compute cluster.

[0080] The feedback engine 520 may be configured to generate feedback data. The feedback engine 520 may determine whether tuning the compute cluster (for example, according to the rules of a rule set enforced by the responsive scaling module 516 and / or according to the output provided by a machine learning model) corrected a previously detected performance degradation. As a simple example, the feedback engine 520 may be configured to determine that a performance degradation has occurred (e.g., a certain number of idle compute nodes have been detected), and some compute nodes have been removed from the cluster according to either a rule set utilized by the responsive scaling module 516 and / or an output provided by a machine learning model obtained by the predictive scaling module 518. The feedback engine 520 may be configured to collect performance metrics provided over a predetermined period, starting before the tuning was performed and including the time the tuning to the compute cluster was performed. The feedback engine 520 may be configured to label those metrics as correcting or not correcting a performance degradation (e.g., a particular performance degradation). The feedback engine 520 may store the labeled metrics as additional instances of the training data in the metric data store. Additional instances of the training data may be used later by the model management module 514 to retrain or update the machine learning model. The feedback engine 520 may generally perform any preferred action corresponding to the feedback procedure 208 described in relation to Figure 2.

[0081] According to some embodiments, the output module 522 performs any preferred action to adjust the number of computing nodes in a computing cluster (e.g., a distributed system 406). For example, the output module 522 may generate a computing process (referred to as a “scaling task”) which may perform any preferred action to provision new computing nodes to the computing cluster or to remove previously provisioned computing nodes from the computing cluster.

[0082] Figure 6 shows a flowchart illustrating an example of method 600 for adjusting the number of nodes in a computing cluster based at least in part on actual and / or predicted changes in the performance of the computing cluster, according to at least one embodiment. In some embodiments, the operation of method 600 may be performed by an autoscaling engine 500 of Figure 5, which operates as a computing service or runs on computing devices within the distributed system 406 in Figure 4, or operates as a separate device and / or service from the distributed system 406, with access to the performance metrics of the distributed system 406.

[0083] Method 600 may begin in block 602 and monitor one or more performance metrics of a set of worker nodes in a computing cluster (for example, by the data processing module 512 in Figure 5). Monitoring may include taking one or more performance metrics according to any preferred predetermined schedule, periodicity, frequency, etc., based at least in part on changes in the performance metrics.

[0084] In 604, the autoscaling engine 500 (or a component of the autoscaling service such as the data processing module 512) may detect a performance degradation in the computing cluster (e.g., a performance metric falling below a performance threshold). The autoscaling engine may then select any suitable performance It may be configured with any suitable thresholds (e.g., range, lower limit, upper limit, tolerance, etc.) corresponding to the performance metric (or combination of performance metrics).

[0085] In 606, in response to detecting that the performance metric is below a performance threshold, the auto-scaling engine 500 performs a first adjustment to the number of worker nodes in the set of worker nodes in the compute cluster. For example, the responsive scaling module 516 may identify the first adjustment (e.g., the number by which the set of worker nodes should be increased or decreased) according to a predetermined set of rules. The responsive scaling module 516 may then instruct the output module 522 to perform any preferred action to perform the first adjustment to the set of worker nodes in the compute cluster.

[0086] In 608, the auto-scaling engine 500 acquires training data for a machine learning model (e.g., machine learning model 202) at least in part on the basis of performing a first adjustment. For example, the feedback engine 520 may collect performance metrics applicable to the first adjustment (e.g., performance metrics before, during, and / or after the first adjustment) and use them to determine whether the first adjustment corrected performance degradation. The feedback engine 520 may label the performance metrics in its determination and provide aggregate data as instances of training data which may be used to retrain / update the machine learning model.

[0087] In 610, the machine learning model may be trained using training data and a supervised machine learning algorithm (for example, by the model management module 514).

[0088] In 612, the autoscaling engine 500 (for example, or a component of the autoscaling engine 500 such as the predictive scaling module 518) may obtain an output showing the predicted performance changes in the computing cluster. In some embodiments, this output is obtained at least in part on the basis of providing one or more subsequent performance metrics of the computing cluster as input to a machine learning model. One or more subsequent performance metrics may be received by the data processing module 512 and passed to the predictive scaling module 518, or the predictive scaling module 518 may retrieve the metrics from the metric data store 504 in Figure 5.

[0089] In 614, the auto-scaling engine 500 may perform a second adjustment to the set of worker nodes based at least in part on an output indicating the expected performance changes in the compute cluster. For example, the predictive scaling module 518 may identify the second adjustment (e.g., the number by which the set of worker nodes should be increased or decreased) according to an output provided by the model. In some embodiments, this output indicates the total number of compute nodes required and / or the number by which compute nodes should be increased or decreased. The predictive scaling module 518 then instructs the output module 522 to perform any preferred action to perform the first adjustment to the set of worker nodes in the compute cluster.

[0090] As mentioned above, Infrastructure as a Service (IaaS) is one specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the internet). In the IaaS model, the cloud computing provider provides infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., It may host the hypervisor layer, etc. In some cases, the IaaS provider may also supply various services (e.g., billing, monitoring, logging, security, load balancing, and clustering) to accompany those infrastructure components. Therefore, since these services may be policy-driven, the IaaS user may be able to implement policies to drive load balancing in order to maintain application availability and performance.

[0091] In some cases, IaaS customers may access resources and services over a wide area network (WAN), such as the internet, and install the rest of their application stack using the cloud provider's services. For example, a user might log into an IaaS platform, create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and even install enterprise software on those VMs. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0092] In most cases, the cloud computing model will require the participation of a cloud provider. This cloud provider may be, but does not have to be, a third-party service specializing in providing IaaS (e.g., offering, renting, or selling). Entities may also choose to deploy a private cloud and become their own provider of infrastructure services.

[0093] In some examples, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is often managed by the cloud provider under the hypervisor layer (e.g., servers, storage, network hardware, and virtualization). Thus, the customer may be responsible for handling (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, which may be launched on demand, etc.).

[0094] In some examples, IaaS provisioning may refer to acquiring computers or virtual hosts for use and even installing the necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0095] In some cases, IaaS provisioning presents two distinct challenges. First, there's the initial challenge of provisioning an initial set of infrastructure before anything is operational. Second, once everything is provisioned, there's the challenge of evolving the existing infrastructure (e.g., adding new services, modifying services, removing services, etc.). In some cases, these two challenges can be addressed by allowing the infrastructure configuration to be defined declaratively. In other words, the infrastructure (e.g., what components are needed and how they interact) can be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they work together) can be described declaratively. In some examples, once the topology is defined, it can generate workflows to create and / or manage the different components described in the configuration files.

[0096] In some examples, the infrastructure may have many interconnected elements. For example, there may be one or more virtual private clouds (VPCs), also known as core networks (e.g., a potentially on-demand pool of configurable and / or shared computing resources). In some examples, there may also be one or more security group rules and one or more virtual machines (VMs) provisioned to define how network security is set up. Other infrastructure elements such as load balancers and databases may also be provisioned. As more and more infrastructure elements are desired and / or added, the infrastructure can evolve incrementally.

[0097] In some cases, sequential deployment techniques may be employed to enable the deployment of infrastructure code across various virtual computing environments. In addition, the techniques described may enable infrastructure management within these environments. In some cases, a service team may write code that is to be deployed to one or more, but often many, different production environments (e.g., across various different geographical locations, sometimes spanning the entire world). However, in some cases, the infrastructure to which the code is deployed must be set up first. In some cases, provisioning may be done manually, resources may be provisioned using provisioning tools, and / or the code may be deployed using deployment tools once the infrastructure is provisioned.

[0098] Figure 7 is a block diagram 700 showing an exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 702 may be communicably coupled to a secure host tenancy 704 which may include a virtual cloud network (VCN) 706 and a secure host subnet 708. In some examples, the service operator 702 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhone®, cellular phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display), running software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, and the Internet, email, short message service (SMS), Blackberry®, or other enabled communication protocols. Alternatively, client computing devices may include general-purpose personal computers, such as personal computers and / or laptop computers, running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. Client computing devices may also include workstation computers running any of the various commercially available UNIX® or UNIX-like operating systems, including but not limited to various GNU / Linux operating systems such as Google® Chrome OS.Alternatively, or in addition, the client computing device may be any other electronic device, such as a thin client computer, an internet-enabled game system (e.g., a Microsoft Xbox game console with or without a Kinect® gesture input device), and / or a personal messaging device, that can communicate via a network that can access the VCN706 and / or the Internet.

[0099] VCN706 may include a Local Peering Gateway (LPG) 710, which can then connect to Secure Shell (SSH) VCNs via LPG710, which is included in SSH VCN712. SSH VCN712 may be communicatively coupled to 712. SSH VCN712 may include SSH subnet 714, and SSH VCN712 may be communicatively coupled to control plane VCN716 via LPG710, which is included in control plane VCN716. SSH VCN712 may also be communicatively coupled to data plane VCN718 via LPG710. Control plane VCN716 and data plane VCN718 may be included in a service tenancy 719, which may be owned and / or operated by the IaaS provider.

[0100] The control plane VCN 716 may include a control plane demilitary zone (DMZ) layer 720 that functions as a peripheral network (e.g., a portion of the corporate network between the corporate intranet and the external network). DMZ-based servers may have limited responsibilities and help keep security breaches contained. Furthermore, the DMZ layer 720 may include one or more load balancer (LB) subnets 722, a control plane application layer 724 which may include application subnets 726, and a control plane data layer 728 which may include database (DB) subnets 730 (e.g., a front-end DB subnet and / or a back-end DB subnet). The LB subnet 722 included in the control plane DMZ layer 720 may be communicatively coupled to the application subnet 726 included in the control plane application layer 724 and to an internet gateway 734 which may be included in the control plane VCN 716, and the application subnet 726 may be communicatively coupled to the DB subnet 730 included in the control plane data layer 728, a service gateway 736, and a network address translation (NAT) gateway 738. The control plane VCN716 may include a service gateway 736 and a NAT gateway 738.

[0101] The control plane VCN 716 may include a data plane mirror application layer 740 which may include an application subnet 726. The application subnet 726 included in the data plane mirror application layer 740 may include a virtual network interface controller (VNIC) 742 which may run a compute instance 744. The compute instance 744 may communicatively connect the application subnet 726 of the data plane mirror application layer 740 to the application subnet 726 which may be included in the data plane application layer 746.

[0102] The data plane VCN718 may include a data plane application layer 746, a data plane DMZ layer 748, and a data plane data layer 750. The data plane DMZ layer 748 may include an LB subnet 722 that can be communicatively coupled to the application subnet 726 of the data plane application layer 746 and the internet gateway 734 of the data plane VCN718. The application subnet 726 may be communicatively coupled to the service gateway 736 of the data plane VCN718 and the NAT gateway 738 of the data plane VCN718. The data plane data layer 750 may also include a DB subnet 730 that can be communicatively coupled to the application subnet 726 of the data plane application layer 746.

[0103] The Internet gateway 734 of the control plane VCN716 and data plane VCN718 can be communicatively coupled to a metadata management service 752, which can be communicatively coupled to the public internet 754. The public internet 754 can be communicatively coupled to the NAT gateway 738 of the control plane VCN716 and data plane VCN718. The service gateway 736 of the control plane VCN716 and data plane VCN718 can be communicatively coupled to a cloud service 756.

[0104] In some examples, the service gateway 736 of the control plane VCN716 or data plan VCN718 makes application programming interface (API) calls to cloud services 756 without going through the public internet 754. It is possible that the API call from service gateway 736 to cloud service 756 may be one-way, where service gateway 736 can make an API call to cloud service 756, and cloud service 756 can send the requested data to service gateway 736. However, cloud service 756 does not have to initiate an API call to service gateway 736.

[0105] In some examples, a secure host tenancy 704 may be directly connected to a service tenancy 719, which may otherwise be isolated. A secure host subnet 708 may communicate with an SSH subnet 714 via an LPG 710, which may otherwise enable bidirectional communication through an isolated system. Connecting the secure host subnet 708 to the SSH subnet 714 may give the secure host subnet 708 access to other entities within the service tenancy 719.

[0106] The control plane VCN716 may allow users of service tenancy 719 to set up or otherwise provision desired resources. Desired resources provisioned within the control plane VCN716 may be deployed or otherwise used within the data plane VCN718. In some examples, the control plane VCN716 may be isolated from the data plane VCN718, and the data plane mirror application layer 740 of the control plane VCN716 may communicate with the data plane application layer 746 of the data plane VCN718 via a VNIC 742 which may be included in the data plane mirror application layer 740 and the data plane application layer 746.

[0107] In some examples, a user or customer of the system may perform requests, such as create, read, update, or delete (CRUD) operations, via the public internet 754, which can communicate requests to the metadata management service 752. The metadata management service 752 may communicate requests to the control plane VCN 716 via the internet gateway 734. This request may be received by the LB subnet 722, which is included in the control plane DMZ layer 720. The LB subnet 722 may determine that the request is valid, and in response to this determination, the LB subnet 722 may send the request to the application subnet 726, which is included in the control plane application layer 724. If the request is validated and requires a call to the public internet 754, the call to the public internet 754 may be sent to the NAT gateway 738, which can make calls to the public internet 754. Memory that may be desired to be stored by the request may be stored in the DB subnet 730.

[0108] In some examples, the data plane mirror application layer 740 can facilitate direct communication between the control plane VCN716 and the data plane VCN718. For example, it may be desirable that configuration changes, updates, or other appropriate modifications be applied to the resources contained in the data plane VCN718. Through VNIC742, the control plane VCN716 can communicate directly with the resources contained in the data plane VCN718, thereby enabling it to perform configuration changes, updates, or other preferred modifications.

[0109] In some embodiments, the control plane VCN716 and data plane VCN718 may be included in the service tenancy 719. In this case, the system user or customer does not have to own or operate either the control plane VCN716 or the data plane VCN718. Instead, the IaaS provider may own or operate the control plane VCN716 and the data plane VCN718, and both may be included in the service tenancy 719. This embodiment may enable network isolation that can prevent a user or customer from interacting with resources of other users or other customers. Furthermore, this embodiment may enable system users or customers to store databases privately without having to rely on the public internet, which may not have the desired level of security for storage.

[0110] In another embodiment, the LB subnet 722 included in the control plane VCN 716 may be configured to receive signals from the service gateway 736. In this embodiment, the control plane VCN 716 and the data plane VCN 718 may be configured to be invoked by the IaaS provider's customer without calling the public internet 754. The IaaS provider's customer may desire this embodiment because the database used by the customer may be controlled by the IaaS provider and stored on a service tenancy 719 which may be isolated from the public internet 754.

[0111] Figure 8 is a block diagram 800 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 802 (e.g., service operator 702 in Figure 7) may be communicatively coupled to a secure host tenancy 804 (e.g., secure host tenancy 704 in Figure 7), which may include a virtual cloud network (VCN) 806 (e.g., VCN706 in Figure 7) and a secure host subnet 808 (e.g., secure host subnet 708 in Figure 7). The VCN 806 may include a local peering gateway (LPG) 810 (e.g., LPG710 in Figure 7), which may be communicatively coupled to a secure shell (SSH) VCN 812 (e.g., SSH VCN712 in Figure 7) via an LPG 710 contained in an SSH VCN 812. SSH VCN812 may include SSH subnet 814 (for example, SSH subnet 714 in Figure 7), and SSH VCN812 may be communicably coupled to control plane VCN816 (for example, control plane VCN716 in Figure 7) via LPG810 included in control plane VCN816. Control plane VCN816 may be included in service tenancy 819 (for example, service tenancy 719 in Figure 7), and data plane VCN818 (for example, data plane VCN718 in Figure 7) may be included in customer tenancy 821, which may be owned or operated by a user or customer of the system.

[0112] The control plane VCN816 may include a control plane DMZ layer 820 (e.g., control plane DMZ layer 720 in Figure 7) which may include an LB subnet 822 (e.g., LB subnet 722 in Figure 7), a control plane application layer 824 (e.g., control plane application layer 724 in Figure 7) which may include an application subnet 826 (e.g., application subnet 726 in Figure 7), and a control plane data layer 828 (e.g., control plane data layer 728 in Figure 7) which may include a database (DB) subnet 830 (e.g., similar to DB subnet 730 in Figure 7). The LB subnet 822 included in the control plane DMZ layer 820 may be communicatively coupled to the application subnet 826 included in the control plane application layer 824 and to an internet gateway 834 (e.g., internet gateway 734 in Figure 7) which may be included in the control plane VCN 816. The application subnet 826 may be communicatively coupled to the DB subnet 830 included in the control plane data layer 828, to a service gateway 836 (e.g., service gateway in Figure 7) and to a network address translation (NAT) gateway 838 (e.g., NAT gateway 738 in Figure 7). The control plane VCN 816 may include the service gateway 836 and the NAT gateway 838.

[0113] The control plane VCN 816 may include a data plane mirror application layer 840 (e.g., data plane mirror application layer 740 in Figure 7) which may include an application subnet 826. The application subnet 826 included in the data plane mirror application layer 840 may include a virtual network interface controller (VNIC) 842 (e.g., VNIC 742) which may run compute instance 844 (e.g., similar to compute instance 744 in Figure 7). This may include: Computation instance 844 can facilitate communication between the application subnet 826 of the data plane mirror application layer 840 and an application subnet 826 that may be included in the data plane application layer 846 (for example, the data plane application layer 746 in Figure 7) via the VNIC 842 included in the data plane mirror application layer 840 and the VNIC 842 included in the data plane application layer 846.

[0114] The Internet gateway 834 included in the control plane VCN816 can be communicatively coupled to a metadata management service 852 (e.g., metadata management service 752 in Figure 7), which can be communicatively coupled to the public internet 854 (e.g., public internet 754 in Figure 7). The public internet 854 can be communicatively coupled to a NAT gateway 838 included in the control plane VCN816. The service gateway 836 included in the control plane VCN816 can be communicatively coupled to a cloud service 856 (e.g., cloud service 756 in Figure 7).

[0115] In some examples, the data plane VCN818 may be included in a customer tenancy 821. In this case, the IaaS provider may provide a control plane VCN816 per customer, and the IaaS provider may set up a unique compute instance 844 included in a service tenancy 819 for each customer. Each compute instance 844 may enable communication between the control plane VCN816 included in the service tenancy 819 and the data plane VCN818 included in the customer tenancy 821. The compute instance 844 may enable resources provisioned in the control plane VCN816 included in the service tenancy 819 to be deployed or otherwise used in the data plane VCN818 included in the customer tenancy 821.

[0116] In another example, an IaaS provider's customer may have a database residing in customer tenancy 821. In this example, the control plane VCN 816 may include a data plane mirror application layer 840, which may include an application subnet 826. The data plane mirror application layer 840 may reside within the data plane VCN 818, but does not have to reside within the data plane VCN 818. That is, the data plane mirror application layer 840 may have access to customer tenancy 821, but does not have to reside within the data plane VCN 818, or be owned or operated by the IaaS provider's customer. The data plane mirror application layer 840 may be configured to make calls to the data plane VCN 818, but does not have to be configured to make calls to any entity contained in the control plane VCN 816. The customer may wish to deploy or otherwise use resources in the data plane VCN818 provisioned within the control plane VCN816, and the data plane mirror application layer 840 can facilitate the customer's desired deployment or other use of resources.

[0117] In some embodiments, a customer of the IaaS provider may apply filters to the data plane VCN818. In this embodiment, the customer may determine what the data plane VCN818 can access, and may restrict access from the data plane VCN818 to the public internet 854. The IaaS provider may not be able to filter or otherwise control the data plane VCN818's access to any external network or database. Applying filters and controls by the customer on the data plane VCN818 included in the customer tenancy 821 can help isolate the data plane VCN818 from other customers and the public internet 854.

[0118] In some embodiments, the cloud service 856 may be invoked by the service gateway 836 to access services that may not reside on the public internet 854, the control plane VCN 816, or the data plane VCN 818. The connection between the cloud service 856 and the control plane VCN 816 or data plane VCN 818 may not be live or continuous. The cloud service 856 may reside on different networks owned or operated by the IaaS provider. The cloud service 856 may be configured to receive calls from the service gateway 836 and may be configured not to receive calls from the public internet 854. Some cloud services 856 may be isolated from other cloud services 856, and the control plane VCN 816 may be isolated from cloud services 856 that may not be in the same region as the control plane VCN 816. For example, the control plane VCN 816 may be located in "region 1", and the cloud service "deployment 7" may be located in regions 1 and "region 2". If a call to deployment 7 is made by a service gateway 836 included in the control plane VCN816 located in region 1, the call may be transmitted to deployment 7 in region 1. In this example, the control plane VCN816, or deployment 7 in region 1, does not have to be communicatively coupled to, or otherwise does not communicate with, deployment 7 in region 2.

[0119] Figure 9 is a block diagram 900 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 902 (e.g., service operator 702 in Figure 7) may be communicatively coupled to a secure host tenancy 904 (e.g., secure host tenancy 704 in Figure 7), which may include a virtual cloud network (VCN) 906 (e.g., VCN706 in Figure 7) and a secure host subnet 908 (e.g., secure host subnet 708 in Figure 7). VCN906 may include an LPG 910 (e.g., LPG710 in Figure 7), which may be communicatively coupled to an SSH VCN912 (e.g., SSH VCN712 in Figure 7) via an LPG910 contained within the SSH VCN912. SSH VCN912 may include SSH subnet 914 (for example, SSH subnet 714 in Figure 7), SSH VCN912 may be communicatively coupled to control plane VCN916 (for example, control plane VCN716 in Figure 7) via LPG910 included in control plane VCN916, and may be communicatively coupled to data plane VCN918 (for example, data plane 718 in Figure 7) via LPG910 included in data plane VCN918. Control plane VCN916 and data plane VCN918 may be included in service tenancy 919 (for example, service tenancy 719 in Figure 7).

[0120] The control plane VCN916 may include a control plane DMZ layer 920 (e.g., control plane DMZ layer 720 in Figure 7) which may include a load balancer (LB) subnet 922 (e.g., LB subnet 722 in Figure 7), a control plane application layer 924 (e.g., control plane application layer 724 in Figure 7) which may include an application subnet 926 (e.g., similar to application subnet 726 in Figure 7), and a control plane data layer 928 (e.g., control plane data layer 728 in Figure 7) which may include a DB subnet 930. The LB subnet 922 included in the control plane DMZ layer 920 may be communicatively coupled to the application subnet 926 included in the control plane application layer 924 and to an internet gateway 934 (e.g., internet gateway 734 in Figure 7) which may be included in the control plane VCN 916. The application subnet 926 may be communicatively coupled to the DB subnet 930 included in the control plane data layer 928, as well as to a service gateway 936 (e.g., service gateway in Figure 7) and a network address translation (NAT) gateway 938 (e.g., NAT gateway 738 in Figure 7). The control plane VCN 916 may include the service gateway 936 and the NAT gateway 938.

[0121] The data plane VCN918 may include a data plane application layer 946 (e.g., data plane application layer 746 in Figure 7), a data plane DMZ layer 948 (e.g., data plane DMZ layer 748 in Figure 7), and a data plane data layer 950 (e.g., data plane data layer 750 in Figure 7). The data plane DMZ layer 948 may include an LB subnet 922 that can be communicatively coupled to the trusted application subnet 960 and untrusted application subnet 962 of the data plane application layer 946, and to the internet gateway 934 included in the data plane VCN918. The trusted application subnet 960 may be communicatively coupled to the service gateway 936 included in the data plane VCN918, the NAT gateway 938 included in the data plane VCN918, and the DB subnet 930 included in the data plane data layer 950. The untrusted application subnet 962 may be communicatively coupled to the service gateway 936 included in the data plane VCN918 and to the DB subnet 930 included in the data plane data layer 950. The data plane data layer 950 may include a DB subnet 930 that can be communicatively coupled to a service gateway 936 included in the data plane VCN 918.

[0122] An untrusted application subnet 962 may include one or more primary VNICs 964(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 966(1)-(N). Each tenant VM 966(1)-(N) may be communicatively coupled to each application subnet 967(1)-(N) that may be included in each container egress VCN 968(1)-(N) that may be included in each customer tenancy 970(1)-(N). Each secondary VNIC 972(1)-(N) can facilitate communication between the untrusted application subnet 962 included in the data plane VCN 918 and the application subnets included in the container egress VCN 968(1)-(N). Each container egress VCN 968(1)-(N) may include a NAT gateway 938 that can be communicatively coupled to the public internet 954 (e.g., public internet 754 in Figure 7).

[0123] The Internet gateway 934, included in the control plane VCN916 and the data plane VCN918, can be communicatively coupled to a metadata management service 952 (e.g., the metadata management system 752 in Figure 7), which can be communicatively coupled to the public internet 954. The public internet 954 can be communicatively coupled to a NAT gateway 938, included in the control plane VCN916 and the data plane VCN918. The service gateway 936, included in the control plane VCN916 and the data plane VCN918, can be communicatively coupled to a cloud service 956.

[0124] In some embodiments, the data plane VCN918 may be integrated with the customer tenancy 970. This integration may be useful or desirable for the IaaS provider's customer in several cases, such as when they may want support when executing code. The customer may provide code to be executed that may be disruptive, communicate with other customer resources, or otherwise cause undesirable effects. In response, the IaaS provider may decide whether to execute the code provided to the IaaS provider by the customer.

[0125] In some examples, an IaaS provider's customer might request temporary network access to the IaaS provider and that a certain function be attached to a data plane tier application 946. The code for executing the function may run in VM966(1)~(N) and may not be configured to run anywhere else on the data plane VCN918. Each VM966(1)~(N) is located in one customer tenancy 970. They may be connected. Each container 971(1)~(N) contained within VM966(1)~(N) may be configured to execute code. In this case, double isolation may be possible (for example, containers 971(1)~(N) may execute code and containers 971(1)~(N) may be contained within at least VM966(1)~(N) contained within the untrusted app subnet 962), which may help prevent incorrect or otherwise undesirable code from damaging the IaaS provider's network or the network of a different customer. Containers 971(1)~(N) may be communicably coupled to customer tenancy 970 and may be configured to send or receive data to or from customer tenancy 970. Containers 971(1)~(N) do not have to be configured to send or receive data to or from any other entity in the data plane VCN918. Once code execution is complete, the IaaS provider may kill or otherwise discard containers 971(1)~(N).

[0126] In some embodiments, a trusted application subnet 960 may execute code that may be owned or operated by the IaaS provider. In this embodiment, the trusted application subnet 960 may be communicatively coupled to a DB subnet 930 and configured to perform CRUD operations within the DB subnet 930. An untrusted application subnet 962 may be communicatively coupled to a DB subnet 930, but in this embodiment, the untrusted application subnet may be configured to perform read operations within the DB subnet 930. Containers 971(1)~(N) that may be contained in each customer's VM966(1)~(N) and execute code from that customer do not have to be communicatively coupled to the DB subnet 930.

[0127] In other embodiments, the control plane VCN916 and the data plane VCN918 may not be directly communicatively coupled. In this embodiment, there may be no direct communication between the control plane VCN916 and the data plane VCN918. However, communication may occur indirectly through at least one method. An LPG910 that facilitates communication between the control plane VCN916 and the data plane VCN918 may be established by the IaaS provider. In another example, the control plane VCN916 or the data plane VCN918 may make a call to a cloud service 956 via a service gateway 936. For example, a call from the control plane VCN916 to the cloud service 956 may include a request for a service that can communicate with the data plane VCN918.

[0128] Figure 10 is a block diagram 1000 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1002 (e.g., service operator 702 in Figure 7) may be communicatively coupled to a secure host tenancy 1004 (e.g., secure host tenancy 704 in Figure 7), which may include a virtual cloud network (VCN) 1006 (e.g., VCN 706 in Figure 7) and a secure host subnet 1008 (e.g., secure host subnet 708 in Figure 7). VCN 1006 may include an LPG 1010 (e.g., LPG 710 in Figure 7), which may be communicatively coupled to an SSH VCN 1012 (e.g., SSH VCN 712 in Figure 7) via an LPG 1010 contained within the SSH VCN 1012. SSH VCN1012 may include SSH subnet 1014 (for example, SSH subnet 714 in Figure 7), and SSH VCN1012 may be communicatively coupled to control plane VCN1016 (for example, control plane VCN716 in Figure 7) via LPG1010 included in control plane VCN1016, and may be communicatively coupled to data plane VCN1018 (for example, data plane 718 in Figure 7) via LPG1010 included in data plane VCN1018. Control plane VCN1016 and data plane VCN1018 may be included in service tenancy 1019 (for example, service tenancy 719 in Figure 7).

[0129] The control plane VCN1016 may include a control plane DMZ layer 1020 (e.g., control plane DMZ layer 720 in Figure 7) which may include an LB subnet 1022 (e.g., LB subnet 722 in Figure 7), a control plane application layer 1024 (e.g., control plane application layer 724 in Figure 7) which may include an application subnet 1026 (e.g., application subnet 726 in Figure 7), and a control plane data layer 1028 (e.g., control plane data layer 728 in Figure 7) which may include a DB subnet 1030 (e.g., DB subnet 930 in Figure 9). The LB subnet 1022 included in the control plane DMZ layer 1020 may be communicatively coupled to the application subnet 1026 included in the control plane application layer 1024 and to an internet gateway 1034 (e.g., internet gateway 734 in Figure 7) which may be included in the control plane VCN 1016. The application subnet 1026 may be communicatively coupled to the DB subnet 1030 included in the control plane data layer 1028 and to a service gateway 1036 (e.g., service gateway in Figure 7) and a network address translation (NAT) gateway 1038 (e.g., NAT gateway 738 in Figure 7). The control plane VCN 1016 may include the service gateway 1036 and the NAT gateway 1038.

[0130] The data plane VCN 1018 may include a data plane application layer 1046 (e.g., data plane application layer 746 in Figure 7), a data plane DMZ layer 1048 (e.g., data plane DMZ layer 748 in Figure 7), and a data plane data layer 1050 (e.g., data plane data layer 750 in Figure 7). The data plane DMZ layer 1048 may include a trusted application subnet 1060 (e.g., trusted application subnet 960 in Figure 9) and an untrusted application subnet 1062 (e.g., untrusted application subnet 962 in Figure 9) of the data plane application layer 1046, and an LB subnet 1022 that can be communicatively coupled to the internet gateway 1034 included in the data plane VCN 1018. The trusted application subnet 1060 may be communicatively coupled to the service gateway 1036 included in the data plane VCN 1018, the NAT gateway 1038 included in the data plane VCN 1018, and the DB subnet 1030 included in the data plane data layer 1050. An untrusted application subnet 1062 may be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018 and to a DB subnet 1030 included in the data plane data layer 1050. The data plane data layer 1050 may include a DB subnet 1030 that can be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018.

[0131] An untrusted application subnet 1062 may include primary VNICs 1064(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1066(1)-(N) residing within the untrusted application subnet 1062. Each tenant VM 1066(1)-(N) may execute code in its respective container 1067(1)-(N) and can be communicatively coupled to an application subnet 1026 that may be included in a data plane application layer 1046 that may be included in a container egress VCN 1068. Each secondary VNIC 1072(1)-(N) can facilitate communication between the untrusted application subnet 1062 included in the data plane VCN 1018 and the application subnet included in the container egress VCN 1068. The container egress VCN may include a NAT gateway 1038 that can be communicatively coupled to the public internet 1054 (e.g., public internet 754 in Figure 7).

[0132] The Internet gateway 1034, which is included in the control plane VCN1016 and the data plane VCN1018, can be communicably coupled to a metadata management service 1052 (e.g., the metadata management system 752 in Figure 7), which can be communicably coupled to the public internet 1054. The public internet 1054 is the control plane The NAT gateway 1038, included in VCN1016 and included in data plane VCN1018, can be communicatively coupled to it. The service gateway 1036, included in control plane VCN1016 and included in data plane VCN1018, can be communicatively coupled to cloud service 1056.

[0133] In some examples, the pattern shown by the architecture of block diagram 1000 in Figure 10 may be considered an exception to the pattern shown by the architecture of block diagram 900 in Figure 9, which may be desirable for the IaaS provider's customers when the IaaS provider cannot communicate directly with the customers (e.g., in a disconnected region). Each container 1067(1)~(N) contained within VM1066(1)~(N) for each customer may be accessible by the customer in real time. Each container 1067(1)~(N) may be configured to make calls to each secondary VNIC 1072(1)~(N) contained within the application subnet 1026 of the data plane application layer 1046, which may be contained within the container egress VCN 1068. The secondary VNICs 1072(1)~(N) may send calls to the NAT gateway 1038, which may send calls to the public internet 1054. In this example, containers 1067(1)-(N), which may be accessed by customers in real time, may be isolated from the control plane VCN1016 and from other entities included in the data plane VCN1018. Containers 1067(1)-(N) may also be isolated from resources from other customers.

[0134] In another example, a customer may use containers 1067(1)-(N) to invoke cloud service 1056. In this example, the customer may execute code within containers 1067(1)-(N) to request a service from cloud service 1056. Containers 1067(1)-(N) may send this request to secondary VNICs 1072(1)-(N), which may send the request to the NAT gateway, which may send the request to the public internet 1054. The public internet 1054 may send the request to LB subnet 1022, which is included in control plane VCN 1016, via internet gateway 1034. In response to determining that the request is valid, the LB subnet may send the request to application subnet 1026, which may send the request to cloud service 1056 via service gateway 1036.

[0135] Please understand that the illustrated IaaS architectures 700, 800, 900, and 1000 may have components other than those shown. Furthermore, the illustrated embodiments are only some examples of cloud infrastructure systems that may incorporate embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than those shown, may combine two or more components, or may have different configurations or arrangements of components.

[0136] In some embodiments, the IaaS system described herein may include a set of applications, middleware, and database service offerings that are self-service, subscription-based, flexibly scalable, reliable, highly available, and delivered to customers in a secure manner. An example of such an IaaS system is Oracle Cloud Infrastructure (OCI) provided by the Assignee.

[0137] Figure 11 shows an exemplary computer system 1100 in which various embodiments may be implemented. System 1100 may be used to implement any of the computer systems described above. As shown in the figure, computer system 1100 is a processing unit that communicates with several peripheral subsystems via bus subsystem 1102. This includes a nit 1104. These peripheral subsystems may also include a processing acceleration unit 1106, an I / O subsystem 1108, a storage subsystem 1118, and a communication subsystem 1124. The storage subsystem 1118 includes a tangible computer-readable storage medium 1122 and system memory 1110.

[0138] The bus subsystem 1102 provides a mechanism for various components and subsystems of the computer system 1100 to communicate with each other as intended. Although the bus subsystem 1102 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 1102 may be one of several types of bus structures, including a memory bus or memory controller, peripheral bus and local bus, using any of the various bus architectures. For example, such architectures may include the Industry Standard Architecture (ISA) bus, the Microchannel Architecture (MCA) bus, the Extended ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus, which can be implemented as a mezzanine bus manufactured according to the IEEE P1386.1 standard.

[0139] The processing unit 1104 can be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers) and controls the operation of the computer system 1100. One or more processors may be included in the processing unit 1104. These processors may include single-core processors or multi-core processors. In certain embodiments, the processing unit 1104 may be implemented as one or more independent processing units 1132 and / or 1134, each containing a single-core or multi-core processor. In other embodiments, the processing unit 1104 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors onto a single chip.

[0140] In various embodiments, the processing unit 1104 can execute various programs in response to program code and can maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed can reside in the processor 1104 and / or the storage subsystem 1118. Through suitable programming, the processor 1104 can provide the various functionalities described above. The computer system 1100 may further include a processing acceleration unit 1106, which may include a digital signal processor (DSP), a special-purpose processor, and the like.

[0141] The I / O subsystem 1108 may include user interface input devices and user interface output devices. User interface input devices may include pointing devices such as keyboards, mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion sensing and / or gesture recognition devices such as Microsoft Kinect® motion sensors, which enable users to control and interact with input devices such as Microsoft Xbox® 360 game controllers through a natural user interface using gesture and speech commands. User interface input devices may detect eye movements from the user (e.g., blinking while taking a picture and / or making a menu selection) and use eye gestures to input devices (e.g., Google Google Glass® blink detector and other applications that convert input to Glass®. The user interface input device may also include an eye gesture recognition device. In addition, the user interface input device may include a voice recognition sensing device that enables the user to interact with a voice recognition system (e.g., Siri® Navigator) via voice commands.

[0142] User interface input devices may include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphic tablets, as well as auditory / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. In addition, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards and digital musical instruments.

[0143] User interface output devices may include non-visual displays such as display subsystems, indicator lights, or audio output devices. Display subsystems may include flat panel devices such as those using cathode ray tubes (CRTs), liquid crystal displays (LCDs), or plasma displays, projection devices, touchscreens, etc. Generally, the use of the term “output device” is intended to include all conceivable types of devices and mechanisms for outputting information from the computer system 1100 to a user or another computer. For example, user interface output devices may include, but are not limited to, a variety of display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0144] The computer system 1100 may also include a storage subsystem 1118 containing software elements, which are currently shown as being located in the system memory 1110. The system memory 1110 may store program instructions that can be loaded and executed on the processing unit 1104, as well as data generated during the execution of these programs.

[0145] Depending on the configuration and type of the computer system 1100, the system memory 1110 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically includes data and / or program modules that are immediately accessible to the processing unit 1104 and / or currently being operated and executed by the processing unit 1104. In some implementations, the system memory 1110 may include several different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS) containing basic routines that help transfer information between elements within the computer system 1100, such as during startup, may typically be stored in ROM. As an example, but not an limitation, the system memory 1110 also includes application programs 1112, program data 1114, and an operating system 1116, which may include client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), etc. For example, operating system 1116 is compatible with various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux®. (Trademark) Operating System, various commercially available UNIX (registered trademark) or UNIX-like operating systems This includes, but is not limited to, various GNU / Linux operating systems, Google Chrome® OS, and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS.

[0146] The storage subsystem 1118 may also provide a tangible, computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by a processor, provides the functionality described above may be stored in the storage subsystem 1118. These software modules or instructions may be executed by the processing unit 1104. The storage subsystem 1118 may also provide a repository for storing data used in accordance with this disclosure.

[0147] The storage subsystem 1100 may also include a computer-readable storage medium reader 1120 which may be further connected to the computer-readable storage medium 1122. Together with the system memory 1110, and optionally in combination with the system memory 1110, the computer-readable storage medium 1122 may comprehensively represent a combination of a storage medium, remote, local, fixed, and / or removable storage device, for temporarily and / or more permanently storing, storing, transmitting, and retrieving computer-readable information.

[0148] The computer-readable storage medium 1122 containing code or a portion of code may also include, but is not limited to, any suitable medium known or used in the art, including storage and communication media, such as volatile and non-volatile, removable and non-removable media, implemented by any method or technique for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or other tangible computer-readable media. This may also include intangible computer-readable media such as any other media that can be used to transmit data signals, data transmissions or desired information and can be accessed by the computing system 1100.

[0149] For example, the computer-readable storage medium 1122 is a hard disk drive that reads and writes to a non-removable non-volatile magnetic medium, a magnetic disk drive that reads and writes to a removable non-volatile magnetic disk, a CD-ROM, a DVD, and a Blu-ray (registered trademark). The computer-readable storage medium 1122 may include, but is not limited to, optical disc drives that read from and write to removable non-volatile optical discs such as disks, or other optical media. The computer-readable storage medium 1122 may also include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, digital videotapes, etc. The computer-readable storage medium 1122 may also include flash memory-based SSDs, enterprise flash drives, solid-state drives (SSDs) based on non-volatile memory such as solid-state ROM, SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. Disk drives and computer-readable media associated with them may contain computer-readable instructions, data structures, program modules, and other The data may be provided to the computer system 1100 as non-volatile storage.

[0150] The communication subsystem 1124 provides an interface to other computer systems and networks. The communication subsystem 1124 acts as an interface for sending and receiving data between other systems and the computer system 1100. For example, the communication subsystem 1124 may enable the computer system 1100 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1124 may include radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (using, for example, cellular telephone technology, 3G, 4G, or EDGE (High Speed ​​Data Rate for Global Evolution)), a Global Positioning System (GPS) receiver component, and / or other components. In some embodiments, the communication subsystem 1124 can provide wired network connectivity (e.g., Ethernet®) in addition to, or instead of, the wireless interface.

[0151] In some embodiments, the communication subsystem 1124 may also receive input communications in the form of structured and / or unstructured data feeds 1126, event streams 1128, event updates 1130, etc., on behalf of one or more users, which may use a computer system 1100.

[0152] For example, the communication subsystem 1124 handles Twitter® feeds, Facebook ( (Registered Trademark) Updates, Web feeds such as Rich Site Summary (RSS) feeds, and It may also be configured to receive data feeds 1126 in real time from users of social networks and / or other communication services, such as real-time updates from one or more third-party sources.

[0153] In addition, the communication subsystem 1124 may also be configured to receive data in the form of a continuous data stream, which may include an event stream 1128 and / or event update 1130 of real-time events, which may be inherently continuous or infinite, without an explicit termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial stock market boards, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0154] The communication subsystem 1124 may also be configured to output structured and / or unstructured data feeds 1126, event streams 1128, event updates 1130, etc., to one or more databases which may communicate with one or more streaming data source computers coupled to the computer system 1100.

[0155] The computer system 1100 can be one of various types, including handheld portable devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or any other data processing systems.

[0156] Due to the ever-changing nature of computers and networks, the description of computer system 1100 shown in the diagram is intended merely as a specific example. Many other configurations are possible, having more or fewer components than the system depicted in the diagram. Customized hardware may also be used, and / or certain elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Furthermore, connections to other computing devices, such as network input / output devices, may be employed. Based on the disclosures and teachings provided herein, those skilled in the art will understand other aspects and / or methods for implementing various embodiments.

[0157] While specific embodiments have been described, various modifications, changes, alternative configurations, and equivalents are also included within the scope of this disclosure. The embodiments are not limited to operation within a particular data processing environment, but can freely operate within multiple data processing environments. Furthermore, while the embodiments are described using a specific set of transactions and steps, it should be apparent to those skilled in the art that the scope of this disclosure is not limited to the described set of transactions and steps. The various features and aspects of the embodiments described above may be used individually or in combination.

[0158] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments may be implemented using hardware alone, software alone, or a combination thereof. Various processes described herein may be implemented on the same processor or different processors of any combination thereof. Thus, where a component or module is described as being configured to perform a particular operation, such configuration may be achieved, for example, by designing an electronic circuit that performs the operation, by programming a programmable electronic circuit (such as a microprocessor) to perform the operation, or by any combination thereof. Processes may communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0159] Therefore, the specification and drawings should be considered illustrative rather than restrictive. However, it will be clear that additions, reductions, deletions, and other modifications and changes may be made without departing from the broader spirit and scope set forth in the claims. Thus, while specific embodiments of the disclosure have been described, these are not intended to be limiting. Various modifications and equivalents are within the scope of the claims.

[0160] The use of the words “a,” “an,” and “the,” as well as similar referents, in the context of describing the disclosed embodiments (particularly in the context of the claims), should be interpreted as encompassing both singular and plural forms unless otherwise indicated herein or unless the context clearly contradicts this interpretation. The words “comprising,” “having,” “including,” and “containing” are non-restrictive unless otherwise specified (i.e., “including but also containing”). The term “connected” should be interpreted as meaning “not limited to.” The phrase “connected” should be interpreted as being partially or entirely included, attached, or combined with something, even if something is intervening. Unless otherwise indicated herein, the descriptions of value ranges are intended merely as a concise way of referring individually to each distinct value that falls within that range, and each distinct value is incorporated herein as if it were individually described herein. All methods described herein may be carried out in any preferred order unless otherwise indicated herein or unless it is clearly inconsistent with the context. Any and all examples or illustrative words (e.g., “etc.”) provided herein are intended merely to better illustrate embodiments, and unless otherwise requested, This disclosure is not intended to limit its scope. Nothing in this specification should be construed as indicating that any unclaimed element is essential to the implementation of this disclosure.

[0161] Disjunctive phrases such as “at least one of X, Y, or Z” are intended to be understood, unless otherwise specified, in contexts where they are commonly used to indicate that an item, term, etc., may be X, Y, Z, or any combination thereof (e.g., X, Y, and / or Z). Therefore, such disjunctive phrases are not intended, and should not, imply that a particular embodiment requires the presence of at least one X, at least one Y, or at least one Z, respectively.

[0162] Preferred embodiments of the Disclosure, including the best known form for carrying out the Disclosure, are described herein. Modifications of these preferred embodiments will be apparent to those skilled in the art by reading the foregoing description. Those skilled in the art should be able to adopt such modifications as appropriate, and the Disclosure may be practiced in ways other than those specifically described herein. Accordingly, the Disclosure includes all modifications and equivalents of the subject matter described in the claims, as permitted by applicable law. Furthermore, any combination of the above elements in all possible variations thereof is encompassed by the Disclosure unless otherwise indicated herein.

[0163] All references herein, including publications, patent applications, and patents, are referred to in this specification to the same extent as they are referred to individually and specifically, and as if they were included in their entirety.

[0164] While the above specification describes aspects of the disclosure with reference to specific embodiments, those skilled in the art will recognize that the disclosure is not limited thereto. The various features and aspects of the above disclosure may be used individually or together. Furthermore, embodiments may be used in any number of environments and applications beyond those described herein, without departing from the broader spirit and scope of this specification. Accordingly, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method implemented by a computer, The computing service monitors one or more performance metrics of a set of worker nodes in a computing cluster, The computing service detects when the performance metric falls below the performance threshold, In response to detecting that the performance metric falls below the performance threshold, the computing service performs a first adjustment to the number of worker nodes in the set of worker nodes of the computing cluster. The computing service acquires training data for a machine learning model, at least in part, based on performing the first adjustment. The computing service trains the machine learning model using the training data and supervised machine learning algorithms. The computing service includes obtaining an output indicating a predicted performance change in the computing cluster, the output being obtained at least in part on the basis of providing one or more subsequent performance metrics of the computing cluster as input to the machine learning model, and the method further includes A computer-implemented method comprising the computing service performing a second adjustment to the set of worker nodes based at least in part on the output indicating the predicted performance change in the computing cluster.

2. The method implemented by a computer according to claim 1, wherein the coordination of the set of worker nodes further includes the computing service generating a scaling task, the scaling task being performed by a computing process, and the computing process updating metadata associated with the computing cluster upon completion of the scaling task.

3. The computer-based method according to claim 1, wherein the output indicating the predicted performance change indicates how many worker nodes will be used to compute the task at a later time, and the later time occurs within a predetermined period in the future.

4. A computer-based method according to claim 1, wherein performing the first adjustment or the second adjustment includes increasing the number of worker nodes or decreasing the number of worker nodes.

5. The method, as described in claim 1, which involves performing the first adjustment, including provisioning a certain number of additional worker nodes to the set of worker nodes in the computing cluster.

6. The computer-implemented method according to claim 5, further comprising determining that provisioning a certain number of additional worker nodes resulted in a subsequent performance metric exceeding the performance threshold, wherein the training data is generated in response to the determination that the certain number of additional worker nodes resulted in the subsequent performance metric.

7. The computer-based method according to claim 6, wherein the training data includes one or more performance metrics, the subsequent performance metrics, and a certain number of the additional worker nodes provisioned during the first period.

8. The method implemented by the computer according to claim 1, wherein the one or more performance metrics include at least one of the number of pending queries, the number of pending tasks, a latency measurement, processing utilization, or memory utilization.

9. A computing device, One or more processing devices that are communicatively coupled to a computer-readable medium, The system comprises a computer-readable medium that stores non-temporary computer executable program instructions, and when the non-temporary computer executable program instructions are executed by one or more processing devices, they cause the computing device to perform an operation, and the operation is Monitoring performance metrics for a set of worker nodes in a computing cluster, Detecting when performance metrics fall below performance thresholds, In response to detecting that the performance metric falls below the performance threshold, a first adjustment is made to the number of worker nodes in the set of worker nodes of the computing cluster. Obtaining training data for a machine learning model based at least in part on the first adjustment, Training the machine learning model using the aforementioned training data and supervised machine learning algorithm, This includes obtaining an output showing the expected performance changes in the computing cluster, the output being obtained at least in part on the basis of providing subsequent performance metrics of the computing cluster as input to the machine learning model, and the operation further includes, A computing device comprising performing a second adjustment to a set of worker nodes based at least in part on the output indicating the predicted performance change in the computing cluster.

10. The computing device according to claim 9, wherein the coordinating of the set of worker nodes further includes the computing service generating a scaling task, the scaling task being performed by a computing process, and the computing process updating metadata associated with the computing cluster upon completion of the scaling task.

11. The computing device according to claim 10, wherein the output indicating the predicted performance change indicates how many worker nodes will be used to compute the task at a later time, and the later time occurs within a predetermined period in the future.

12. The computing device according to claim 9, wherein performing the first adjustment or the second adjustment includes increasing the number of worker nodes or decreasing the number of worker nodes.

13. The computing device according to claim 9, wherein performing the first adjustment includes provisioning a certain number of additional worker nodes to the set of worker nodes in the computing cluster, the computing device performs an additional operation, the additional operation includes determining that provisioning the certain number of additional worker nodes resulted in a subsequent performance metric exceeding the performance threshold, and the training data is generated in response to the determination that the certain number of additional worker nodes resulted in the subsequent performance metric.

14. The computing device according to claim 13, wherein the training data includes one or more performance metrics, the subsequent performance metrics, and a certain number of the additional worker nodes provisioned during the first period.

15. A non-temporary computer-readable storage medium for storing computer executable program instructions, wherein, when the computer executable program instructions are executed by a processing device of a computing device, the computing device causes the computing device to perform an action, and the action is Monitoring performance metrics for a set of worker nodes in a computing cluster, Detecting when performance metrics fall below performance thresholds, In response to detecting that the performance metric falls below the performance threshold, a first adjustment is made to the number of worker nodes in the set of worker nodes of the computing cluster. Obtaining training data for a machine learning model based at least in part on the first adjustment, Training the machine learning model using the aforementioned training data and supervised machine learning algorithm, This includes obtaining an output showing the expected performance changes in the computing cluster, the output being obtained at least in part on the basis of providing subsequent performance metrics of the computing cluster as input to the machine learning model, and the operation further includes, A non-transient computer-readable storage medium, comprising performing a second adjustment on a set of worker nodes based at least in part on the output indicating the predicted performance change in the computing cluster.

16. The non-temporary computer-readable storage medium according to claim 15, wherein coordinating the set of worker nodes further includes the computing service generating a scaling task, the scaling task being performed by a computing process, and the computing process updating metadata associated with the computing cluster upon completion of the scaling task.

17. The non-temporary computer-readable storage medium according to claim 16, wherein the output indicating the predicted performance change indicates how many worker nodes will be used to compute the task at a later time, and the later time occurs within a predetermined period in the future.

18. The non-temporary computer-readable storage medium according to claim 15, wherein performing the first adjustment or the second adjustment includes increasing the number of the set of worker nodes or decreasing the number of the set of worker nodes.

19. The non-temporary computer-readable storage medium according to claim 15, wherein performing the first adjustment comprises provisioning a certain number of additional worker nodes to the set of worker nodes in the computing cluster, the computing device performs an additional operation which comprises determining that provisioning the certain number of additional worker nodes resulted in a subsequent performance metric exceeding the performance threshold, and the training data is generated in response to the determination that the certain number of additional worker nodes resulted in the subsequent performance metric.

20. The computing device according to claim 19, wherein the training data includes one or more performance metrics, the subsequent performance metrics, and a certain number of the additional worker nodes provisioned during the first period.

Images