Workload system including failover capacity reservation based on request threshold reliability and cluster reliability

Abstract

A method (600) includes determining a cluster reliability (162) of a computing cluster (142) that includes a maximum computing capacity (144). The method includes receiving a provisioning request for the computing cluster (162), the provisioning request including a threshold reliability of the computing cluster (15). In response to the provisioning request, the method includes determining a reserved computing capacity for the computing cluster based on the threshold reliability using the cluster reliability (172). The reserved calculation capacity is smaller than the maximum calculation capacity. Based on the reserved computing capacity and the maximum computing capacity, the method includes determining an unreserved computing capacity for the computing cluster (174). The method includes provisioning the computing cluster for execution of a user workload (20). The user workload is executed on the unreserved computing capacity. The reserved computing capacity is not initially available to execute the user workload.

Description

Technical Field

[0001] The present disclosure relates to an adaptable workload system. Background Art

[0002] A service level objective (SLO) is a constituent element of a service level agreement (SLA) that defines the means for measuring performance. For example, an SLO can be used to define how to measure the performance of a service provider that provides hardware and / or software to a customer to perform a workload. These SLOs are typically structured for services associated with a platform (i.e., hardware and software) and do not have specific application for any workload running on top of the service. Therefore, the workload must be adapted to fit the guarantee regarding platform uptime. Uptime refers to the time that the platform is in operation. Uptime typically remains static relative to the workload running on the SLO. Summary of the invention

[0003] One aspect of the present disclosure provides a computer-implemented method for an adaptable workload system, which, when executed by data processing hardware, causes the data processing hardware to perform an operation. The operation includes determining a cluster reliability of a computing cluster including a maximum computing capacity. The cluster reliability represents the reliability of the computing cluster when all of the maximum computing capacity is utilized. The operation also includes receiving a configuration request from a user requesting configuration of a computer cluster. The configuration request includes a threshold reliability of the computing cluster. In response to receiving the configuration request, the reserved computing capacity of the computing cluster is determined using the cluster reliability of the computing cluster and based on the threshold reliability of the computing cluster. The reserved computing capacity is less than the maximum computing capacity. The operation also includes determining the unreserved computing capacity of the computing cluster based on the reserved computing capacity of the computing cluster. The computing cluster is configured for execution of a user workload associated with a user, and the execution of the user workload is performed on the unreserved computing capacity of the computing cluster. The operation also includes reserving the reserved computing capacity of the computing cluster. The reserved computing capacity of the computing cluster is initially unavailable for execution of the user workload.

[0004] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, the threshold reliability includes a percentage of uptime of the computing cluster. Optionally, the operation includes detecting a failure of the computing cluster. The failure affects unreserved computing capacity. In these examples and in response to detecting the failure, the operation includes designating at least a portion of the reserved computing capacity as available for execution of user workloads.

[0005] In some examples, the computing cluster includes multiple components, and determining the cluster reliability of the computing cluster includes: for each respective component in the multiple components, determining the respective component reliability and aggregating each respective component reliability. In some implementations, determining the reserved computing capacity of the computing cluster includes: receiving a threshold reliability update request including a second threshold reliability from a user. Optionally, the threshold reliability can be adjusted based on the second threshold reliability.

[0006] In some examples, after configuring the computing cluster, the operation includes monitoring failures that affect unreserved computing capacity. In some examples, the computing cluster includes multiple nodes, and reserving the reserved computing capacity of the computing cluster includes: marking one or more nodes of the multiple nodes. Each corresponding node of the multiple nodes may include a maximum node computing capacity, and reserving the reserved computing capacity of the computing cluster may include: for each corresponding node of the multiple nodes, establishing a computing capacity limit that is less than the maximum node computing capacity of the corresponding node. In some implementations, the operation also includes detecting a failure of a corresponding node of the multiple nodes, and in response to detecting the failure, removing the computing capacity limit from at least one corresponding node of the multiple nodes. Optionally, the operation of receiving a configuration request includes: receiving a user input indication in a graphical user interface, the graphical user interface being executed on the data processing hardware for display on a screen that communicates with the data processing hardware, the user input indication indicating selection of a graphical element to set a threshold reliability.

[0007] Another aspect of the present disclosure provides a system for adaptable workloads. The system includes data processing hardware and memory hardware that communicates with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include determining the cluster reliability of a computing cluster including a maximum computing capacity. The cluster reliability represents the reliability of the computing cluster when all of the maximum computing capacity is utilized. The operations also include receiving a configuration request from a user requesting configuration of a computer cluster. The configuration request includes a threshold reliability of the computing cluster. In response to receiving the configuration request, the reserved computing capacity of the computing cluster is determined using the cluster reliability of the computing cluster and based on the threshold reliability of the computing cluster. The reserved computing capacity is less than the maximum computing capacity. The operations also include determining the unreserved computing capacity of the computing cluster based on the reserved computing capacity of the computing cluster. The computing cluster is configured for the execution of a user workload associated with a user, and the execution of the user workload is performed on the unreserved computing capacity of the computing cluster. The operations also include reserving the reserved computing capacity of the computing cluster. The reserved computing capacity of the computing cluster is initially unavailable for the execution of the user workload.

[0008] This aspect may include one or more of the following optional features. In some implementations, the threshold reliability includes an uptime percentage of the computing cluster. Optionally, the operation includes detecting a failure of the computing cluster. The failure affects unreserved computing capacity. In these examples and in response to detecting the failure, the operation includes designating at least a portion of the reserved computing capacity as available for execution of the user workload.

[0009] In some examples, the computing cluster includes multiple components, and determining the reliability of the computing cluster includes: for each respective component in the multiple components, determining the respective component reliability and aggregating each respective component reliability. In some implementations, determining the reserved computing capacity of the computing cluster includes: receiving a threshold reliability update request including a second threshold reliability from a user. Optionally, the threshold reliability can be adjusted based on the second threshold reliability.

[0010] In some examples, after configuring the computing cluster, the operation includes monitoring failures that affect unreserved computing capacity. In some examples, the computing cluster includes multiple nodes, and reserving the reserved computing capacity of the computing cluster includes: marking one or more nodes of the multiple nodes. Each corresponding node of the multiple nodes may include a maximum node computing capacity, and reserving the reserved computing capacity of the computing cluster may include: for each corresponding node of the multiple nodes, establishing a computing capacity limit that is less than the maximum node computing capacity of the corresponding node. In some implementations, the operation also includes detecting a failure of a corresponding node of the multiple nodes, and in response to detecting the failure, removing the computing capacity limit from at least one corresponding node of the multiple nodes. Optionally, the operation of receiving a configuration request includes: receiving a user input indication in a graphical user interface, the graphical user interface being executed on the data processing hardware for display on a screen that communicates with the data processing hardware, the user input indication indicating selection of a graphical element to set a threshold reliability.

[0011] The details of one or more implementations of the present disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a schematic diagram of an example system for an adaptable workload.

[0013] Figure 2A and Figure 2B is a diagram of a compute cluster with unreserved compute capacity and reserved compute capacity.

[0014] Figure 3 is a schematic diagram of components for monitoring at least one of the computing clusters for a failure.

[0015] Figure 4 is a schematic diagram of a node in a computing cluster experiencing a failure.

[0016] Figure 5A and Figure 5B is a schematic diagram of an example user device having a graphical user interface for adjusting computing capacity of a computing cluster.

[0017] Figure 6 is a flow diagram of an example operational arrangement of a method for providing an adaptable workload system.

[0018] Figure 7 is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein.

[0019] Like reference numerals in the various drawings indicate like elements. DETAILED DESCRIPTION

[0020] A service level objective (SLO) is a constituent element of a service level agreement (SLA) that defines a means for measuring performance. For example, a service level objective (SLO) may govern the measurement of the performance of a container orchestration platform for one or more computing clusters in a computing cluster. An SLO may govern the monitoring and configuration of a computing cluster to meet a specific uptime, where uptime is the time when a computing cluster is in operation (i.e., available to a customer). However, conventional SLOs are typically applied to services, not to user workloads that can be run on top of services. Therefore, SLOs become one-size-fits-all services, which results in adapting user workloads to fit the uptime parameters set forth in the SLO. For example, uptime is typically static relative to workloads running on the SLO. Adapting user workloads to SLOs may limit the customizability of SLOs, and while some user workloads may easily adapt to service limitations, some user workloads may therefore lag behind or otherwise operate inefficiently.

[0021] The implementation of this article relates to an adaptable workload system that automatically adapts to achieve a dynamic SLO (e.g., adjusted by a user) for executing a user workload. In other words, the system allows, for example, a user to adjust the SLO based on the user workload rather than the service. The user can select the SLO that best suits the user workload via the adaptable workload system, and in response, the system adjusts the capacity of the service. For example, when resources are limited, the system can adapt to selectively prioritize user workloads. Specifically, a hardware failure can cause the system to adapt the service to meet the capacity requirements of the user workload. In other words, the SLO is a property that can be modulated by the user based on the current workload requirements. This configuration of the SLO can result in dynamic changes that are adapted to assist, for example, in configuring the uptime guarantee set by the user. For example, a user can use a service governed by an SLO for development, which requires a lower uptime guarantee than when the service is used for production workloads.

[0022] When the user adjusts the SLO, the service is configured as part of the adaptable workload system because the adaptable workload system identifies differences (e.g., differences in uptime guarantees) based on the SLO adjustment. For example, when the user adjusts the SLO based on the current workload requirements, the system can automatically adapt the service to effectively update and adjust the uptime guarantee. This allows the user to effectively select the SLO that best suits the current workload requirements. Services can be configured via both the control plane and the data plane.

[0023] In some examples, the adaptable workload system dynamically adjusts service aspects of both the control plane and the data plane. In other examples, the adaptable workload system dynamically adjusts only a portion of the system (e.g., the data plane). For example, the data plane of the service refers to the computing capacity of the computing cluster that the user uses to execute the workload. In this example, the data plane is executed with a sliding scale of uptime based on the node capacity on the computing cluster. In other words, the system determines the appropriate computing capacity based on the SLO adjusted or selected by the user, and adjusts the available capacity of the computer cluster appropriately.

[0024] refer to Figure 1In some implementations, the example adaptable workload system 100 includes a remote system 140 that communicates with one or more user devices 10 via a network 112. The remote system 140 can be a single computer, multiple computers, or a distributed system (e.g., a cloud environment) with scalable / elastic resources 114, which include computing resources 116 (e.g., data processing hardware) and / or storage resources 118 (e.g., memory hardware). The data storage 120 (i.e., remote storage device) can be overlaid on the storage resource 118 to allow one or more of the clients (e.g., user devices 10) or the computing resource 116 to scalably use the storage resource 118.

[0025] The remote system 140 is configured to receive a configuration request 14 from a user device 10 associated with a corresponding user 12 via, for example, a network 112, 112a. The user device 10 may correspond to any computing device, such as a desktop workstation, a laptop workstation, or a mobile device (i.e., a smart phone). The user device 10 includes computing resources 16 (e.g., data processing hardware) and / or storage resources 18 (e.g., memory hardware).

[0026] The user 12 executes one or more user workloads 20 on the computing cluster 142 as part of the service 110 provided by the adaptable workload system 100. The computing cluster 142 includes one or more nodes 200, 200a-n, each of which includes computing resources 146 (e.g., data processing hardware) and / or storage resources 148 (e.g., memory hardware). The node data repository 149 can be overlaid on the storage resource 148. In some examples, the computing cluster 142 includes local hardware associated with the user 12. In this example, the user device 10, the remote system 140, and the computing cluster 142 communicate via one or more networks 112 (such as a public network 112a and a private or local network 112, 112b). In other examples, the computing cluster 142 is part of the remote system 140 (i.e., cloud computing). The control plane of the service 110 can be executed on the remote system 140, and the data plane of the service 110 can be executed on the computing cluster.

[0027] The computing cluster 142 has a maximum computing capacity 144, and each node 200 of the computing cluster 142 has a maximum node computing capacity 210. Each node 200 may have a different maximum node computing capacity 210, and in general, the maximum computing capacity 144 of the computing cluster 142 is the sum of the maximum node computing capacities 210 of each node 200. The user workload 20 uses a portion of the maximum computing capacity 144 and a portion of one or more maximum node computing capacities 210 when executing, depending on the number of nodes 200 across which the user workload 20 is distributed.

[0028] The remote system 140 (or, in some examples, the computing cluster 142) executes an adaptable workload controller 150. The adaptable workload controller 150 includes a reliability module 160 that determines a cluster reliability 162 of the computing cluster 142 and / or the node 200. For example, the reliability module 160 receives cluster information 50 that includes information about the computing cluster 142 and / or the nodes 200 of the computing cluster 142, such as the type, manufacturer, model, and number of various components of the nodes 200 and / or the computing cluster 142. Using the cluster information 50, the reliability module 160 determines a cluster reliability 162 (e.g., mean time to failure (MTTF)) of the computing cluster. In some implementations, the reliability module 160 determines a node reliability 202, 202a-n for each node 200, and the cluster reliability 162 is an aggregation of each node reliability 202. For simplicity, the adaptable workload controller 150 and the adaptable workload system 100 are described more generally with respect to the computing cluster 142. However, it is contemplated that the operations described herein may also be applied to nodes 200 of computing cluster 142 .

[0029] The cluster reliability 162 of the computing cluster 142 may represent the cluster reliability 162 of the computing cluster 142 when the entirety of the maximum computing capacity 144 of the computing cluster 142 is needed (i.e., any event that reduces the maximum computing capacity 144 is considered a failure). For example, when the computing cluster 142 includes multiple nodes 200, the failure of any single node 200 will result in a reduction in the maximum computing capacity 144. It is generally contemplated that the cluster reliability 162 of the computing cluster 142 may be related to the MTTF of the computing cluster 142. The adaptable workload controller 150 automatically adjusts the availability of a portion of the maximum computing capacity 144 of the computing cluster 142 based on the determined cluster reliability 162, as further described below.

[0030] like Figure 1 As shown, the adaptable workload controller 150 receives a provisioning request 14 from a user 12 via a network 112 requesting provisioning of a computing cluster 142. In some examples, the provisioning request 14 includes a threshold reliability 15 for the computing cluster 142. The threshold reliability 15 may include an uptime percentage of the computing cluster 142.

[0031] In addition to the reliability module 160, the adaptable workload controller 150 of the adaptable workload system 100 includes a compute capacity module 170 and a provisioner 180. The compute capacity module 170 receives the provisioning request 14 from the user 12 and the cluster reliability 162 from the reliability module 160, and determines a reserved compute capacity 172 for the compute cluster 142. As described in more detail below, the reserved compute capacity 172 represents the amount of compute capacity that must be reserved (based on the cluster reliability 162) in order to meet the threshold reliability 15.

[0032] The reserved computing capacity 172 is less than the maximum computing capacity 144, and the computing capacity module 170 determines the reserved computing capacity 172 based on the threshold reliability 15 and the cluster reliability 162 of the computing cluster 142. In other words, in response to receiving the provisioning request 14, the computing capacity module 170 determines the reserved computing capacity 172 of the computing cluster 142 using the cluster reliability 162 and the threshold reliability 15 of the computing cluster 142. The computing capacity module 170 also determines the unreserved computing capacity 174 based on the maximum computing capacity 144 and the reserved computing capacity 172. For example, the reserved computing capacity 172 is subtracted from the maximum computing capacity 144 to determine the unreserved computing capacity 174 of the computing cluster 142. It should be understood that the computing capacity module 170 may alternatively first determine the unreserved computing capacity 174 and derive the reserved computing capacity 172 based on the unreserved computing capacity 174.

[0033] The configurator 180 configures the computing cluster 142 to execute the user workload 20 associated with the user 12. The user workload 20 is initially executed only on the unreserved computing capacity 174 of the computing cluster 142. That is, the configurator 180 configures the computing cluster 142 so that only an amount of computing capacity of the computing cluster 142 equal to the unreserved computing capacity 174 is available for executing the user workload 20. The reserved computing capacity 172 of the computing cluster 142 is reserved for potential later use (e.g., in the event of a failure). That is, the reserved computing capacity 172 of the computing cluster 142 is initially unavailable for execution of the user workload 20 (i.e., in order to meet the threshold reliability 15), but may become available at a later time. The configurator 180 issues one or more configuration commands 182 to configure the computing cluster 142.

[0034] Reference now Figure 2A In this example, the computing cluster 142 includes six nodes 200a-f, but in practice, the cluster 142 may include any number of nodes 200. The reliability module 160 ( Figure 1) can determine a respective node reliability 202, 202a-f for each respective node 200a-f in the one or more nodes 200. When determining the cluster reliability 162 of the computing cluster 142, the respective node reliability 202 of each respective node in the respective nodes 200a-f can be aggregated. In some examples, the reliability module 160 determines the node reliability 202 of each node 200 based on one or more component reliabilities of the components (e.g., CPU, RAM, non-volatile storage, network interface, etc.) that make up the node 200. Subsequently, the computing capacity module 170 ( Figure 1 ) may determine reserved computing capacity 172 and unreserved computing capacity 174 using corresponding node reliabilities 202 (and / or component reliabilities).

[0035] In some implementations, reserving computing capacity 172 refers to reserving at least one of the plurality of nodes 200a-f from executing user workload 20. Reserving a node 200 may be accomplished by “tagging” the node 200. In some examples, one or more of the nodes 200a-f are tagged, resulting in reserving computing capacity 172. For example, Figure 2A Two nodes 200e, 200f are shown reserved by the adaptable workload controller 150 to represent the reserved computing capacity 172, while the four nodes 200a-d of the computing cluster 142 represent the unreserved computing capacity 174 that can execute the user workload 20. That is, in this example, the reserved computing capacity 172 is created by reserving one or more full nodes 200 from executing the user workload 20. It is contemplated that the reserved computing capacity 172 may include more or less than two nodes 200, and the unreserved computing capacity 174 may include more or less than four nodes 200.

[0036] Additionally or alternatively, such as Figure 2BAs shown, the reserved computing capacity 172 and the unreserved computing capacity 174 can be implemented or applied separately to each node 200 of the computing cluster 142. In these implementations, a portion or percentage of the maximum node computing capacity 210 of each node 200 of the computing cluster 142 is designated as the reserved computing capacity 172, and the remaining portion is designated as the unreserved computing capacity 174. That is, in these examples, each node 200 reserves a corresponding portion of its maximum node computing capacity 210. For example, when 20% of the maximum computing capacity 144 of the computing cluster is reserved, each node 200 reserves 20% of its corresponding maximum node computing capacity 210. In FIG. 2C, the unreserved computing capacity 174 of the computing cluster 142 is distinguished from the reserved computing capacity 172 by a dotted line represented on each node 200 of the computing cluster 142. For example, the reserved computing capacity 172 is shown as the upper area of ​​the node 200, and the unreserved computing capacity 174 is shown as the lower area of ​​the node 200. Although not shown here, in addition to each node reserving a portion of the maximum node computing capacity 210, one or more nodes 200 may be marked or otherwise completely reserved from operation.

[0037] Reference now Figure 3 In some implementations, the adaptable workload controller 150 includes a monitor 310 and a resource controller 312. The monitor 310 monitors for failures in the unreserved computing capacity 174 of the computing cluster 142 (e.g., during the execution of the user workload 20). For example, the monitor 310 receives operational data 314 related to the operation of the corresponding node 200 from the computing cluster 142. The failure may include a hardware and / or software failure of the node 200 (e.g., any component that makes up the node 200), or a hardware and / or software failure of a network that maintains communication with the node 200.

[0038] When the monitor 310 detects a failure that affects the unreserved computing capacity 174 (i.e., the computing cluster 142 provides the user 12 with the ability to access all of the unreserved computing capacity 174), the resource controller 312 can designate at least a portion of the reserved computing capacity 172 as available for execution of the user workload 20. For example, the resource controller 312 sends a capacity adjustment command 316 to the computing cluster 142 to modify or adjust the designation of the reserved computing capacity 172 so that at least a portion of the reserved computing capacity 172 of the computing cluster 142 is available for execution of the user workload 20. In some examples, the failure that affects the unreserved computing capacity 174 can cause the resource controller 312 to send a reconfiguration request to the adaptable workload controller 150. The reconfiguration request can reconfigure the reserved computing capacity 172 and the unreserved computing capacity 174 of the corresponding node 200.

[0039] Reference now Figure 4 , in from Figure 2A Continuing with this example, the monitor 310 detects a failure of the node 200a. In response to the monitor 310 detecting the failure, the resource controller 312 removes the computing capacity limit from the node 200e to compensate for the failed node 200a, thereby "replenishing" the unreserved computing capacity 174 back to its original capacity. That is, the resource controller 312 modifies the designation of one of the nodes 200 (e.g., node 200e) that is currently designated as a portion of the reserved computing capacity 172 (e.g., removes a mark). Here, the node 200e is identified with a dashed perimeter to indicate the change in designation from the reserved computing capacity 172 to the unreserved computing capacity 174. In this example, after the failure, five of the nodes 200a-e are designated as unreserved computing capacity 174 (where the failed node 200a does not contribute its corresponding maximum node computing capacity 210), and only one of the nodes 200f remains designated as the reserved computing capacity 172. It is also contemplated that the failure of one node 200a may cause the resource controller 312 to individually adjust the reserved computing capacity 172 of each of the nodes 200b-f to be greater than the initially specified unreserved computing capacity 174 ( Figure 2B ) has a greater percentage of unreserved compute capacity 174.

[0040] Reference now Figure 5A, schematic diagram 500a includes a user device 10 executing a graphical user interface (GUI) 510 for display on a screen 501 of the user device 10. The GUI 510 enables a user 12 to interact with an adaptable workload controller 150 via the user device 10. The GUI 510 includes a graphical representation 512 of a threshold reliability 15 of a computing cluster 142, the graphical representation depicting the threshold reliability 15 along a graphical axis 514. The user 12 may provide a user input indication (e.g., via a touch screen, voice command, mouse, keyboard, etc.) to the GUI 510 indicating a selection of a graphical element 522 to set a value for the threshold reliability 15, and cause the user device 10 to send the value of the threshold reliability 15 to the adaptable workload controller 150. For example, the example shows that the user provides a user input indication to set a value for the threshold reliability 15 by manipulating a graphical slider 522, 522a along the graphical axis 514 to a first position 524a. That is, movement of the graphical slider 522a along the graphical axis 514 in a first direction (relative to the schematic diagram 500a) may increase the value of the threshold reliability 15, while movement of the graphical slider 522a along the graphical axis 514 in an opposite second direction may decrease the value of the threshold reliability. In other examples, the user 12 may provide other types of user input indications indicating input / selection of other types of graphical elements 522 in order to set the value of the threshold reliability 15, such as via a text box, button, dial, switch, etc.

[0041] For example, the GUI 510 includes a table 520 (in addition to or in lieu of the axis 514) from which the user 12 selects the desired reserved computing capacity 172 and / or threshold reliability 15 for operation of the computing cluster 142 by selecting the number of unreserved nodes 200. Here, the GUI 510 receives another user input indication indicating a selection of another graphical element (e.g., button) 522, 522b within the table 520, which causes the GUI 510 to translate the graphical slider 522a along the graphical axis 514 of the graphical representation 512 from a first position 524a to a different position corresponding to the adjusted value of the threshold reliability 15. For example, the table 520 displayed in the GUI 510 depicts that selecting four nodes 200a-d as the unreserved computing capacity 174 results in an expected computing capacity of 99.9923%, while selecting five nodes 200a-e as the unreserved computing capacity 174 results in an expected computing capacity of 99.8958%. That is, although less computational capacity is reserved, updates to the estimate are reduced. It is also contemplated that the user 12 may manipulate the graphical slider 522a along the graphical axis 514 of the graphical representation 512 of the threshold reliability 15 to incrementally adjust the threshold reliability 15 by setting the second threshold reliability 518 .

[0042] Reference now Figure 5B, schematic diagram 500b continues Figure 5A Here, the user can provide additional user input indications that manipulate the graphical slider 522a and / or indicate selection of another graphical element 522b to adjust the value of the threshold reliability 15. For example, the example shows that the user provides additional user input indications to set an updated value of the threshold reliability 15 by manipulating the graphical slider 522a along the graphical axis 514 to the second position 524b. That is, the user 12 manipulates the graphical slider 522a and or selects the graphical element 522b within the table 520 to adjust the position of the threshold reliability 15 from the first position 524a ( Figure 5A ) to a second position 524b. The user device 10 transmits a threshold reliability update request 516 including a second threshold reliability 518 to the adaptable workload controller 150. The computing capacity module 170 adjusts the threshold reliability 15 based on the selected second threshold reliability 518. In other words, when determining the unreserved computing capacity 174 of the computing cluster 142, the computing capacity module 170 may receive a threshold reliability update request 516 including a second threshold reliability 518 from the user 12. Here, the user 12 may adjust the graphical representation 512 to translate from the first position 524a to the second position 524b representing the second threshold reliability 518. Specifically, the user 12 manipulates a portion of the graphical representation 512 along the graphical axis 514 so that the graphical axis 514 and the graphical representation 512 of the threshold reliability 15 operate as a sliding scale. Here, the user 12 adjusts the threshold reliability from 99.99% uptime ( Figure 5A )Adjusted to 99.89% uptime ( Figure 5B ).

[0043] Figure 66 is a flow diagram of an exemplary operational arrangement of a method 600 for an adaptable workload system. The computer-implemented method 600, when executed by the data processing hardware 116, causes the data processing hardware 116 to perform operations. At operation 602, the method 600 includes determining a cluster reliability 162 of a computing cluster 142 including a maximum computing capacity 144. The cluster reliability 162 represents the cluster reliability 162 of the computing cluster 142 when all of the maximum computing capacity 144 is utilized. At operation 604, the method includes receiving a configuration request 14 from a user 12. The configuration request 14 requests configuration of the computing cluster 142 and includes a threshold reliability 15 for the computing cluster 142. At operation 606, in response to receiving the configuration request 14, the method determines a reserved computing capacity 172 of the computing cluster 142 based on the threshold reliability 15 using the cluster reliability 162 of the computing cluster 142. The reserved computing capacity 172 is less than the maximum computing capacity 144. At operation 608, the method determines an unreserved computing capacity 174 of the computing cluster based on the reserved computing capacity 172 and the maximum computing capacity 144 of the computing cluster 142. At operation 610, the method includes configuring the computing cluster 142 for execution of the user workload 20 associated with the user 12. The execution of the user workload 20 is performed on the unreserved computing capacity 174 of the computing cluster 142. At operation 612, the method includes reserving the reserved computing capacity 172 of the computing cluster 142. The reserved computing capacity 172 of the computing cluster 142 is initially unavailable for execution of the user workload 20.

[0044] Figure 7 is a schematic diagram of an example computing device 700 that can be used to implement the systems and methods described in this document. Computing device 700 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown here, their connections and relationships, and their functions are intended to be exemplary only and are not intended to limit implementations of the inventions described and / or claimed in this document.

[0045] The computing device 700 includes a processor 710, a memory 720, a storage device 730, a high-speed interface / controller 740 connected to the memory 720 and a high-speed expansion port 750, and a low-speed interface / controller 760 connected to a low-speed bus 770 and the storage device 730. Each of the components 710, 720, 730, 740, 750, and 760 is interconnected using various buses and can be installed on a common motherboard or installed in other ways as appropriate. The processor 710 can process instructions for execution within the computing device 700, including instructions stored in the memory 720 or on the storage device 730, to display graphical information of a graphical user interface (GUI) on an external input / output device (such as a display 780 coupled to the high-speed interface 740). In other implementations, multiple processors and / or multiple buses and multiple memories and multiple types of memories can be used as appropriate. In addition, multiple computing devices 700 can be connected (for example, as a server library, a blade server group, or a multi-processor system), where each device provides part of the necessary operations.

[0046] The memory 720 stores information non-temporarily within the computing device 700. The memory 720 may be a computer-readable medium, a volatile memory unit, or a non-volatile memory unit. The non-temporary memory 720 may be a physical device for storing programs (e.g., instruction sequences) or data (e.g., program state information) on a temporary or permanent basis for use by the computing device 700. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0047] The storage device 730 is capable of providing mass storage for the computing device 700. In some implementations, the storage device 730 is a computer-readable medium. In various implementations, the storage device 730 may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or an array of devices (including devices in a storage area network or other configuration). In other implementations, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer-readable medium or a machine-readable medium, such as the memory 720, the storage device 730, or a memory on the processor 710.

[0048] The high-speed controller 740 manages bandwidth-intensive operations of the computing device 700, while the low-speed controller 760 manages less bandwidth-intensive operations. This division of responsibilities is exemplary only. In some implementations, the high-speed controller 740 is coupled to the memory 720, the display 780 (e.g., through a graphics processor or accelerator), and the high-speed expansion port 750 that can accept various expansion cards (not shown). In some implementations, the low-speed controller 760 is coupled to the storage device 730 and the low-speed expansion port 790. The low-speed expansion port 790, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner; or to a networking device, such as a switch or a router, for example, through a network adapter.

[0049] As shown, computing device 700 can be implemented in a variety of different forms. For example, the computing device can be implemented as a standard server 700a or multiple implementations in a group of such servers 700a, implemented as a laptop computer 700b, or implemented as part of a rack server system 700c.

[0050] Various implementations of the systems and techniques described herein may be implemented in digital electronic and / or optical circuit systems, integrated circuit systems, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that may be executed and / or interpreted on a programmable system that includes at least one programmable processor that may be either special-purpose or general-purpose and that may be coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0051] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0052] The processes and logic flows described in this specification can be performed by one or more programmable processors (also referred to as data processing hardware), which execute one or more computer programs to perform functions by operating on input data and generating outputs. The processes and logic flows can also be performed by a dedicated logic circuit system, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as a disk, a magneto-optical disk, or an optical disk, or be operably coupled to receive data from one or more mass storage devices or transfer data to one or more mass storage devices or both. However, a computer does not have to have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0053] To provide interaction with a user, one or more aspects of the present disclosure may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor) or a touch screen for displaying information to the user, and optionally a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including sound, voice, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0054] A variety of implementations have been described. However, it should be understood that various modifications may be made without departing from the spirit and scope of the present disclosure. Accordingly, other implementations are within the scope of the appended claims.

Claims

1. A computer-implemented method (600) which, when executed by data processing hardware (116), causes the data processing hardware (116) to perform operations, characterized in that: The operations include: receiving a cluster reliability (162) of a computing cluster (142) including a maximum computing capacity (144), the cluster reliability (162) representing the reliability of the computing cluster (142) when all of the maximum computing capacity (144) is utilized; Receiving a provisioning request (14) from a user (12) requesting provisioning of the computing cluster (142), the provisioning request (14) including a threshold reliability (15) of the computing cluster (142); In response to receiving the provisioning request (14), using the cluster reliability (162) of the computing cluster (142), determine a reserved computing capacity (172) of the computing cluster (142) based on the threshold reliability (15) of the computing cluster (142), the reserved computing capacity (172) being less than the maximum computing capacity (144); Determining an unreserved computing capacity (174) of the computing cluster (142) based on the reserved computing capacity (172) and the maximum computing capacity (144) of the computing cluster (142); configuring the computing cluster (142) for execution of a user workload (20) associated with the user (12), the execution of the user workload (20) being performed on the unreserved computing capacity (174) of the computing cluster (142); and The reserved computing capacity (172) of the computing cluster (142) is reserved, the reserved computing capacity (172) of the computing cluster (142) being initially unavailable for execution of the user workload (20).

2. The method (600) according to claim 1, characterized in that: The threshold reliability (15) includes an uptime percentage of the computing cluster (142).

3. The method (600) according to claim 1 or claim 2, characterized in that: The operations also include: detecting a failure of the computing cluster (142), the failure affecting the unreserved computing capacity (174) of the computing cluster (142); and In response to detecting the failure, at least a portion of the reserved computing capacity (172) is designated as available for execution of the user workload (20).

4. The method (600) according to any one of claims 1 to 3, characterized in that: The computing cluster (142) includes a plurality of components; and Receiving the cluster reliability (162) of the computing cluster (142) includes: For each respective component of the plurality of components, determining respective component reliability; and Aggregate the reliability of each corresponding component.

5. The method (600) according to any one of claims 1 to 4, characterized in that: Determining the reserved computing capacity (172) of the computing cluster (142) includes: receiving a threshold reliability update request (516) including a second threshold reliability (15) from the user (12); and The threshold reliability (15) is adjusted based on the second threshold reliability (15).

6. The method (600) according to any one of claims 1 to 5, characterized in that: The operations also include, after provisioning the computing cluster (142), monitoring for failures affecting the unreserved computing capacity (174).

7. The method (600) according to any one of claims 1 to 6, characterized in that: The computing cluster (142) includes a plurality of nodes (200); as well as Reserving the reserved computing capacity (172) of the computing cluster (142) includes marking one or more nodes (200) of the plurality of nodes (200).

8. The method (600) according to any one of claims 1 to 7, characterized in that: The computing cluster (142) includes a plurality of nodes (200), each respective node (200) of the plurality of nodes (200) including a maximum node computing capacity (210); as well as Reserving the reserved computing capacity (172) of the computing cluster (142) includes establishing, for each respective node (200) of the plurality of nodes (200), a computing capacity limit that is less than the maximum node computing capacity (210) of the respective node (200).

9. The method (600) according to claim 8, characterized in that: The operations also include: detecting a failure of a corresponding one of the plurality of nodes (200); and In response to detecting the failure, the computational capacity limitation is removed from at least one corresponding node (200) of the plurality of nodes (200).

10. The method (600) according to any one of claims 1 to 9, characterized in that: Receiving the configuration request (14) includes: receiving a user input indication in a graphical user interface (510), the graphical user interface being executed on the data processing hardware (116) for display on a screen (501) in communication with the data processing hardware (116), the user input indication indicating selection of a graphical element (522) to set the threshold reliability (15).

11. A system (100), characterized in that: include: Data processing hardware (116); as well as Memory hardware (118) in communication with the data processing hardware (116), the memory hardware (118) storing instructions that, when executed on the data processing hardware (116), cause the data processing hardware (116) to perform operations, the operations comprising: receiving a cluster reliability (162) of a computing cluster (142) including a maximum computing capacity (144), the cluster reliability (162) representing the reliability of the computing cluster (142) when all of the maximum computing capacity (144) is utilized; Receiving a provisioning request (14) from a user (12) requesting provisioning of the computing cluster (142), the provisioning request (14) including a threshold reliability (15) of the computing cluster (142); In response to receiving the provisioning request (14), using the cluster reliability (162) of the computing cluster (142), determine a reserved computing capacity (172) of the computing cluster (142) based on the threshold reliability (15) of the computing cluster (142), the reserved computing capacity (172) being less than the maximum computing capacity (144); Determining an unreserved computing capacity (174) of the computing cluster (142) based on the reserved computing capacity (172) and the maximum computing capacity (144) of the computing cluster (142); configuring the computing cluster (142) for execution of a user workload (20) associated with the user (12), the execution of the user workload (20) being performed on the unreserved computing capacity (174) of the computing cluster (142); and The reserved computing capacity (172) of the computing cluster (142) is reserved, the reserved computing capacity (172) of the computing cluster (142) being initially unavailable for execution of the user workload (20).

12. The system (100) according to claim 11, characterized in that The threshold reliability (15) includes an uptime percentage of the computing cluster (142).

13. The system (100) according to claim 11 or claim 12, characterized in that The operations also include: detecting a failure of the computing cluster (142), the failure affecting the unreserved computing capacity (174) of the computing cluster (142); and In response to detecting the failure, at least a portion of the reserved computing capacity (172) is designated as available for execution of the user workload (20).

14. The system (100) according to any one of claims 11 to 13, characterized in that: The computing cluster (142) includes a plurality of components; and Receiving the cluster reliability (162) of the computing cluster (142) includes: For each respective component of the plurality of components, determining respective component reliability; and Aggregate the reliability of each corresponding component.

15. The system (100) according to any one of claims 11 to 14, characterized in that Determining the reserved computing capacity (172) of the computing cluster (142) includes: receiving a threshold reliability update request (516) including a second threshold reliability (15) from the user (12); and The threshold reliability (15) is adjusted based on the second threshold reliability (15).

16. The system (100) according to any one of claims 11 to 15, characterized in that The operations also include, after provisioning the computing cluster (142), monitoring for failures affecting the unreserved computing capacity (174).

17. The system (100) according to any one of claims 11 to 16, characterized in that: The computing cluster (142) includes a plurality of nodes (200); as well as Reserving the reserved computing capacity (172) of the computing cluster (142) includes marking one or more nodes (200) of the plurality of nodes (200).

18. The system (100) according to any one of claims 11 to 17, characterized in that: The computing cluster (142) includes a plurality of nodes (200), each respective node (200) of the plurality of nodes (200) including a maximum node computing capacity (210); as well as Reserving the reserved computing capacity (172) of the computing cluster (142) includes establishing, for each respective node (200) of the plurality of nodes (200), a computing capacity limit that is less than the maximum node computing capacity (210) of the respective node (200).

19. The system (100) according to claim 18, characterized in that The operations also include: detecting a failure of a corresponding one of the plurality of nodes (200); and In response to detecting the failure, the computational capacity limitation is removed from at least one corresponding node (200) of the plurality of nodes (200).

20. The system (100) according to any one of claims 11 to 19, characterized in that Receiving the configuration request (14) includes: receiving a user input indication in a graphical user interface (510), the graphical user interface being executed on the data processing hardware (116) for display on a screen (501) in communication with the data processing hardware (116), the user input indication indicating selection of a graphical element (522) to set the threshold reliability (15).

Images