Resource conservation in containerized systems

Abstract

A method (700) for conserving resources in a distributed system (114) includes receiving an event criteria list (124) from a resource controller (122). The event criteria list includes one or more events (125) observed by the resource controller, and the resource controller controls at least one target resource (126) and is configured to respond to events from the event criteria list that occur. The method also includes determining whether the resource controller is idle. When the resource controller is idle, the method includes terminating the resource controller, determining whether any event from the event criteria list occurs after terminating the resource controller, and recreating the resource controller when at least one event from the event criteria list occurs after terminating the resource controller.

Description

Technical Field

[0001] The present disclosure relates to conserving system resources in a containerized system. Background Art

[0002] In general, it is well known that personal computers reduce resource usage of idle programs. For example, the Unix family of operating systems (OS) exhibit this when a process is blocked reading a file (perhaps a network socket or a pipe). If there is no input to read, the blocked thread will not be scheduled onto the processor (because this allows there to be more threads than available CPU cores). The OS may also perform memory paging to over-subscribe memory, and idle processes will often be paged out. Some cloud-based services (via distributed systems) provide containerized orchestration systems, which often have idle containers or processes. For example, ( ) is built around the idea of ​​a controller, whose role is to reconcile the desired state of the world, as expressed in resources, with the actual state of the world, in some other system or some other resource. For example, a container running a controller process often spends most of its time waiting for changes to objects in an application programming interface (API), and if those objects change slowly, the controller may be essentially idle. But even if the controller's local node OS is able to reduce the process's resource requirements, the orchestration system's scheduler cannot process that information. Summary of the invention

[0003] One aspect of the present disclosure provides a method for saving resources of a containerized system. The method includes receiving an event criteria list from a resource controller at data processing hardware. The event criteria list includes one or more events observed by the resource controller, and the resource controller controls at least one container and is configured to respond to events from the event criteria list that occur. At least one container encapsulates an application process. The method also includes determining by the data processing hardware whether the resource controller is idle. When the resource controller is idle, the method includes terminating the resource controller by the data processing hardware, and determining by the data processing hardware whether any event from the event criteria list occurs after terminating the resource controller. The method also includes recreating the resource controller by the data processing hardware when at least one event from the event criteria list occurs after terminating the resource controller.

[0004] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, determining whether a resource controller is idle includes polling the resource controller. Additionally or alternatively, determining whether a resource controller is idle may include analyzing events that have occurred during a threshold time period, and determining that a resource controller is idle when no events on the event criteria list have occurred within the threshold time period.

[0005] In some examples, terminating the resource controller includes scaling the deployment size of the resource controller to zero, and recreating the resource controller includes scaling the deployment size of the resource controller to one. Optionally, the event criteria list may include a timer event. In some implementations, determining whether any event on the event criteria list occurs includes polling data.

[0006] In some examples, the method includes determining by data processing hardware whether a termination time period since a resource controller was terminated satisfies a threshold time period. In response to determining that the termination time period does satisfy the threshold time period, the method includes recreating the resource controller by data processing hardware. In some implementations, the method includes determining by data processing hardware whether the resource controller needs to be maintained or reconfigured, and in response, recreating the resource controller by data processing hardware. Receiving an event criteria list from a resource controller may include receiving a plurality of event criteria lists from a plurality of resource controllers executed simultaneously, wherein each event criteria list is associated with a corresponding resource controller. The number of resource controllers executed simultaneously may be based on system resources. The method may also include determining by data processing hardware whether any event on a plurality of event criteria lists occurs simultaneously. In response to determining that events on a plurality of event criteria lists occur simultaneously, the method may include recreating a group of resource controllers in sequence by data processing hardware, each resource controller in the group of resource controllers being associated with a corresponding event criteria list of a plurality of event criteria lists. In some examples, recreating the group of resource controllers in sequence includes recreating the group of resource controllers in sequence based on the priority of each resource controller.

[0007] The method may also include recreating the resource controller based on the system metric by the data processing hardware. In some implementations, the system metric is a measurement of the system resource. Optionally, the event criteria list includes a dynamic event criteria list. The method may also include, before terminating the resource controller when the resource controller is idle, determining by the data processing hardware whether the measurement of the system resource meets the system resource threshold. When the measurement of the system resource meets the threshold and the resource controller is idle, the method includes terminating the resource controller by the data processing hardware. The measurement of the system resource may include the measurement of system processing / computing, network access, storage and / or memory resources. In some examples, when receiving the event criteria list from the resource controller, the method includes receiving the event criteria list from a hypertext transfer protocol (HTTP) endpoint. The resource controller may be associated with a system resource reservation, and terminating the resource controller then removes the system resource reservation.

[0008] Another aspect of the present disclosure provides a system, including data processing hardware and memory hardware that communicates with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include receiving an event criteria list from a resource controller. The event criteria list includes one or more events observed by the resource controller, and the resource controller controls at least one container and is configured to respond to events from the event criteria list that occur. At least one container encapsulates an application process. The operations also include determining whether the resource controller is idle. When the resource controller is idle, the operations include terminating the resource controller and determining whether any event from the event criteria list occurs after terminating the resource controller. The operations also include recreating the resource controller when at least one event from the event criteria list occurs after terminating the resource controller.

[0009] Implementations of this aspect of the disclosure may include one or more of the following optional features. In some implementations, determining whether a resource controller is idle includes polling the resource controller. Determining whether a resource controller is idle may additionally or alternatively include: analyzing events that have occurred during a threshold time period; and determining that the resource controller is idle when no events on the event criteria list occur within the threshold time period.

[0010] In some examples, terminating the resource controller includes scaling the deployment size of the resource controller to zero, and recreating the resource controller includes scaling the deployment size of the resource controller to one. Optionally, the event criteria list may include a timer event. In some implementations, determining whether any event on the event criteria list occurs includes polling data.

[0011] In some examples, the operation includes determining whether the termination time period since the resource controller was terminated satisfies the threshold time period. In these examples, in response to determining that the termination time period does meet the threshold time period, the operation includes recreating the resource controller. In some implementations, the operation includes determining whether the resource controller needs to be maintained or reconfigured, and in response to determining that the resource controller needs to be maintained or reconfigured, recreating the resource controller. Receiving an event criteria list from a resource controller may include receiving multiple event criteria lists from multiple resource controllers executed simultaneously, wherein each event criteria list is associated with a corresponding resource controller. The number of resource controllers executed simultaneously may be based on system resources. The operation may then also include determining whether any event on the multiple event criteria lists occurs simultaneously. In response to determining that the events on the multiple event criteria lists occur simultaneously, the operation may also include sequentially recreating a group of resource controllers, each resource controller in the group of resource controllers being associated with a corresponding event criteria list of the multiple event criteria lists. In some examples, sequentially recreating a group of resource controllers includes sequentially recreating a group of resource controllers based on the priority of each resource controller.

[0012] The operation may also include recreating the resource controller based on the system metric. In some implementations, the system metric is a measurement of system resources. Optionally, the event criteria list includes a dynamic event criteria list. The operation may also include determining whether the measurement of system resources meets the system resource threshold before terminating the resource controller when the resource controller is idle. When the measurement of system resources meets the threshold and the resource controller is idle, the operation may include terminating the resource controller. The measurement of system resources may include the measurement of system processing / computing, network access, storage and / or memory resources. In some examples, when receiving the event criteria list from the resource controller, the operation includes receiving the event criteria list from the HTTP endpoint. The resource controller may be associated with a system resource reservation, and terminating the resource controller then removes the system resource reservation.

[0013] The details of one or more implementations of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a schematic diagram of an exemplary system for conserving resources in a containerized system.

[0015] Figure 2 yes Figure 1 Schematic diagram of exemplary components of an idle determiner of a system.

[0016] Figure 3 yes Figure 1A schematic diagram of exemplary components of a process terminator for a system.

[0017] Figure 4 yes Figure 1 A schematic diagram of exemplary components of an event observer for a system.

[0018] Figure 5 yes Figure 1 A schematic diagram of exemplary components of a process creator for a system.

[0019] Figure 6 is a graph of resource availability over time for an exemplary system for conserving resources in a containerized system.

[0020] Figure 7 is a flow diagram of an exemplary operational arrangement of a method for conserving resources in a containerized system.

[0021] Figure 8 is a schematic diagram of an exemplary computing device that can be used to implement the systems and methods described herein.

[0022] Like reference numbers in the various drawings indicate like elements. DETAILED DESCRIPTION

[0023] Due at least in part to advances in remote and distributed computing, containerized applications and systems for orchestrating containerized applications (e.g., ) are becoming increasingly popular. Containerized applications (i.e., virtualization) allow for independent user or application space instances to exist. Each instance (i.e., container) appears to the application as its own individual computer with access to all resources required for execution (e.g., storage, network access, etc.). However, the application in the container will only be able to see and access the resources assigned to its corresponding container. This helps with security, mobility, scaling, and upgrades of applications in distributed or cloud environments.

[0024] Containers will typically be confined to a single application or process. Some container orchestration systems deploy pods as the smallest available computing unit. A pod is a group of one or more containers, each of which shares an isolation boundary (e.g., an IP address). Controllers control resources in a pod. The controller is responsible for monitoring the health of the pod, containers, and resources (and recreating the pod / container when necessary), replicating and scaling the pod, and monitoring external events (of the pod). Currently, pod controllers typically run continuously, and each controller has resource overhead (e.g., a controller may be a go process that consumes processor and memory resources). This has led to bundling controllers together into a single process, which saves resources but runs counter to the design goal of a containerized system (in which each container or group of containers is independent of each other). In addition, this approach is cumbersome because the source code for each controller must be combined into a single binary, which makes it very difficult to update a single controller. In addition, controllers using different library versions or different languages ​​or runtimes may not be combined into a single binary.

[0025] When a process is idle, the operating system (OS) of a personal computer usually basically suspends the process. For example, a process blocked while waiting to read a file will not be scheduled onto the processor until the file is available. The operating system can also perform memory paging to oversubscribe memory, and idle processes are usually paged out. Similarly, containerized applications are often idle. For example, a containerized orchestration system controller often spends most of its time waiting for changes in objects in an application programming interface (API), which may basically make the controller idle between changes. However, even if the local node OS of the controller may be able to reduce the resource requirements of the process through scheduling or memory paging, the orchestration system scheduler cannot process this information, and the typical orchestration system configuration commands the OS not to page out the process (e.g., disable swapping) and try to avoid resource over-commitment to obtain more consistent response times. Therefore, for a typical containerized system, oversubscription of system resources may be difficult, if not impossible. The implementation of the present disclosure solves these technical problems by creating a master controller that allows resource conservation in a containerized system without requiring significant modifications to the containerized system.

[0026] Implementations herein are directed to systems and methods for conserving resources in a containerized system without requiring modifications to the controller or containers. Figure 1 In some implementations, the exemplary system 100 includes a remote system 114. The remote system 114 can be a single computer, multiple computers, or a computer with scalable / elastic computing resources 118 (e.g., data processing hardware 810 ( Figure 8)) and / or storage resources 116 (e.g., memory hardware 820 ( Figure 8 )) of a distributed system (e.g., a cloud environment). The remote system 114 executes one or more deployments 120, 120a-n, and each deployment 120 includes a resource controller 122, 122a-n and a set of one or more target resources 126, 126a-n controlled by the resource controller 122 of the corresponding deployment 120. For example, the target resource 126 can be a container group that includes one or more containers 128, 128a-n, where each container 128 executes an application or process. In other examples, the target resource is a load balancer, a domain name system (DNS) record, or other cloud resource. In the example shown, each resource controller 122 is associated with at least one event criteria list 124, 124a-n, and each target resource 126 (e.g., a container group) executes a container 128 (i.e., an application or process). Each target resource 126 within a deployment 120 can be co-located and can share some resources (e.g., network access) with other target resources 126 within the same deployment 120, while prohibiting sharing other resources (e.g., memory). In other words, a given target resource 126 within a corresponding deployment 120 may not be allowed to share resources with other target resources 126 located within the same deployment 120 .

[0027] In some implementations, each controller 122 within a respective deployment 120 can control each container group 126 (and the containers 128 associated therewith) of the respective deployment 120 by scaling (increasing or decreasing) the deployment size or scale 322 of the respective deployment 120 and / or container group 126. The deployment size can be a replica field. For example, if the deployment size 322 increases (e.g., changes from one to two), the number of instances (e.g., container groups 126) within the respective deployment 120 increases (e.g., the number of container groups 126 increases from one to two). Similarly, if the deployment size 322 decreases, the number of instances (e.g., container groups 126) within the respective deployment 120 decreases. In some examples, setting the deployment size 322 of a deployment 120 to zero terminates or suspends execution of the respective deployment 120 and one or more container groups 126 and containers 128 within the respective deployment 120.

[0028] The event criteria list 124 associated with each controller 122 includes specifications for events 125 associated with a container group 126 or container 128 within the corresponding deployment 120. Events 125 can take any number of forms (e.g., API object changes, http requests, messages in queues, emails, time, system load, other system metrics, etc.). For example, a DNS application that responds to Domain Name System (DNS) changes requires notification when a DNS API object change occurs, and also requires notification when a target DNS record changes, or a timer can be used to poll for target changes. The controller 122 observes or monitors the occurrence of each event 125 on the event criteria list 124. These events 125 may occur infrequently. For example, continuing the DNS example, changes to DNS are typically infrequent, and therefore the DNS application and its corresponding controller 122 will typically be idle until a DNS change occurs. Despite being idle, the deployment 120 maintains the same system resource reservation settings for active states in order to fulfill reasonable response times. That is, deployment 120 requires a typical system to reserve a set amount of system resources (eg, memory / storage / compute / network access) regardless of the state of deployment 120 (ie, idle, active, etc.).

[0029] Continue to refer Figure 1 , the exemplary system 100 includes a master controller 150 executing on a remote system 114 and communicating with each resource controller 122 of a deployment 120. The master controller 150 may monitor the execution of each controller 122 deployed on the remote system 114 and terminate / suspend the execution of controllers 122 that become idle in order to recover unused resources reserved for idle target resources 126. For example, an idle determiner 200 of the master controller 150 may determine when a controller of a deployment is idle, and a process terminator 300 of the master controller 150 may terminate or otherwise suspend the execution of the deployment 120, thereby recovering the resources reserved for the terminated deployment 120. In some implementations, the process terminator 300 terminates the deployment 120 by setting the deployment size (e.g., scale) 322 of the deployment 120 to zero.

[0030] In order to respond to changes that may subsequently occur, the master controller 150 may further implement an event observer 400. In the example shown, the event observer 400 receives the event criteria list 124 from the controller 122 of the corresponding deployment 120 before the process terminator 300 terminates the corresponding deployment 120. Therefore, the event observer 400 allows the master controller 150 to observe the events 125 in the event criteria list 124 that are associated with the deployment 120 that has been terminated / suspended as a result. In other words, since the termination / suspended execution of the deployment 120 thus terminates / suspends the ability of the corresponding controller 122 to observe the events 125 in the event criteria list 124, the master controller 150 assumes the responsibility of the controller 122 by receiving the event criteria list 124 to observe the occurrence of the events 125 in the event criteria list 124 when the execution of the deployment 120 (and the corresponding controller 122) is suspended / terminated. When the event observer 400 detects or determines that an event 125 in the event criteria list 124 has occurred, the process creator 500 of the main controller 150 can redeploy the corresponding deployment 120 (eg, by setting the deployment size (eg, scale) 322 to 1).

[0031] Reference Figure 2In some implementations, the idle determiner 200 includes a process poller 212. The process poller 212 periodically (e.g., once per second, once per minute, etc.) polls 214 the controller 122 to determine whether the corresponding resource controller 122 is idle. For example, the process poller 212 can poll 214 the controller 122 to grab details 216 from the controller 122 to determine whether the controller 122 is idle. The details can include the corresponding event criteria list 124, the processing state of the controller 122, version information of the observed event, etc. Additionally or alternatively, the controller 122 can actively expose the details 216 by exposing the poller 212 to directly read to determine a measure of idleness (e.g., time since last activity). In addition to the process poller 212, or as an alternative, the idle determiner 200 can implement an event monitor 220 configured to receive events 125 that may be on the event criteria list 124 of the controller 122. When the event monitor 220 receives an event 125, the event monitor 220 determines whether the event 125 is one of the events 125 on the event criteria list 124 that includes the controller 122 or an event on another event criteria list 124 from a different controller 122. If the event monitor 220 does not receive any event 125 on the event criteria list 124 within a threshold time period (determined by the timer 222), the event monitor 220 can conclude that the controller 122 is idle. If the process poller 212 and / or the event monitor 220 determine that the controller 122 is idle, the idle determiner 200 can send an idle determination 230 indicating that the controller 122 (and the corresponding deployment 120) is idle to the process terminator 300. In some implementations, determining whether the controller 122 is idle includes analyzing further events caused by the controller 122 when the controller 122 has completed responding to the event. For example, when the controller 122 completes processing changes, the controller 122 can update the object with the "observedGeneration" field.

[0032] Figure 3A process terminator 300 is shown that, in response to receiving an idle determination 230 from the idle determiner 200 indicating an idle resource controller 122, terminates (or otherwise suspends) an idle resource controller 122. For example, the process terminator 300 may include a process scaler 320 that sets a scale (e.g., deployment size) 322 of the controller 122 to zero. The scale 322 may determine the number of instances (e.g., container groups 126) of the deployment 120, and thus a scale 322 of zero will remove (e.g., suspend / terminate execution of) all instances (e.g., container groups 126) of the deployment 120 (and corresponding controllers 122 and containers 128). In addition to the process scaler 320, the process terminator 300 may implement a resource monitor 330 that receives resource metrics 332 from a system 332. When the resource metrics 332 indicate that system resources are abundant, the master controller 150 (via the process terminator 300) may determine that it is more desirable to maintain the controller 122 when in an idle state rather than terminate the idle controller 122, because it is less important to restore the resources reserved for the idle controller 122 when system resources are abundant, while avoiding the cost incurred by redeploying the previously terminated / suspended controller 122. Similarly, for controllers 122 that are known to frequently change from an idle state to an active state, the cost incurred by terminating the execution of these controllers 122 that become idle to redeploy the controller 122 generally outweighs the benefit of restoring unused resources when the controller 122 is idle. Therefore, implementations herein may include the process terminator 300 preventing the termination / suspending of the execution of controllers 122 that are known to frequently switch between an idle state and an active state. The frequency with which the controller switches between an active state and an idle state may be predicted based on the type of container 128 executed in the corresponding container group 126 or based on historical data indicating the frequency with which the controller 122 is redeployed and / or the frequency with which the event 125 observed by the controller 122 occurs.

[0033] In some examples, the process terminator weighs the resource metric 332 and the frequency with which a given controller 122 is known or predicted to switch between active and idle states. In the example shown, the resource monitor 330 receives the system resource metric 332 to determine whether the amount of available system resources meets the threshold amount of resources. For example, the threshold amount of resources may be met when the amount of available system resources exceeds a value associated with the threshold amount of resources. Here, when the idle determination 230 indicates that the controller 122 is idle and the amount of available system resources (e.g., provided in the resource metric 332) cannot meet the threshold amount of resources, the process scaler 320 may determine to scale the controller 122 to zero (e.g., by setting the deployment size 322 to zero) to terminate the controller 122. Otherwise, when the amount of system resources meets the threshold amount of resources, the process scaler 320 may maintain the current scaling of the controller 122 to prevent termination of the controller 122 despite the idle determination 230 indicating that the controller 122 is idle. In some implementations, the master controller 150 saves the state of the controller 122 before termination, and when the controller 122 is redeployed, it returns to the saved state (ie, the controller 122 is suspended rather than terminated).

[0034] As described above, terminating the controller 122 makes the controller 122 unavailable to respond to the occurrence of the event 125 on the event criteria list 124. To compensate for the terminated and therefore unavailable controller 122, the event observer 400 of the master controller 150 assumes the responsibility of the terminated controller 122 by observing the occurrence of the event 125 on the event criteria list 124. Figure 4 , the event observer 400 of the main controller 150 includes an event monitor 420, which receives the event criteria list 124 from the controller 122 before the controller 122 terminates. In some examples, the event criteria list 124 is implemented through a library, and thus the controller 122 does not require any code modification to expose the corresponding event criteria list 124 to the event observer 400. The event monitor 420 also receives events 125 occurring within the system 100 and determines whether any of these events 125 include or correspond to an event 125 on the event criteria list 124. The event criteria list 124 can be dynamic, because the controller 122 can update its event criteria list 124 at any time, such as in response to other events it may receive. The controller 122 can dynamically publish changes to the event criteria list 124, or alternatively, the event observer 400 can obtain changes from the controller 122. In some examples, event observer 400 automatically updates event criteria list 124 of a terminated controller 122 (ie, creates a resource-based event) based on resource metrics 332 or other input.

[0035] In the example shown, the event observer 400 also includes a resource monitor 430. The resource monitor 430 of the event observer 400 and the resource monitor 330 of the process terminator 300 can be the same component or a different component. In some examples, the resource monitor 430 receives the system resource measurement 332 to determine whether there are enough system resources available to create a new deployment 120 (and a subsequent new controller 122) at present. For example, if the available system resource amount cannot meet the redeployment threshold resource amount, the resource monitor 430 can veto the redeployment of the deployment 120. For example, when the available system resource amount is less than the value associated with the redeployment threshold resource amount, the redeployment threshold resource amount may not meet the threshold resource amount. When the event monitor 420 determines that the event 125 on the event criteria list 124 occurs and the resource monitor 430 determines that enough system resources are currently available (that is, by meeting the redeployment threshold resource amount), the event observer 400 sends a create or redeploy command 440 to the process creator 500 to redeploy the controller 122 that was previously terminated / suspended. As described herein, redeploying a controller 122 results in redeploying a corresponding deployment 120 and corresponding target resources (e.g., a container group 126 and a corresponding container 128) to execute on a remote system 114, thereby resulting in allocation of system resources reserved by the corresponding deployment 120 and the corresponding container group 126 and required to execute the corresponding deployment 120 and the corresponding container group 126. In the event that the event monitor 420 determines that an event 125 on the event criteria list occurs but the resource monitor 430 determines that the amount of system resources is insufficient (e.g., the redeployment threshold system resource amount is not satisfied), the system 100 can forgo immediate redeployment of the controller 122 and take a number of different actions. For example, the system 100 can queue the redeployment of the controller 122, adjust the value of the redeployment threshold system resource amount, or modify the timer, such as shortening the idle timer to more aggressively terminate the idle controller 122.

[0036] Figure 5The process creator 500 is shown receiving a create / redeploy command 440 from the event observer 400. In some implementations, the process creator 500 implements a process expander 520 that receives the command 440. The process expander 520 and the process expander 320 of the process terminator 300 can be the same component or a different component. In the example shown, the process expander 520 of the process creator 500 increases the scale 322 of the deployment 120 or controller 122 (e.g., increases the deployment size 322). For example, the scale 322 is increased by setting the deployment size 322 from 0 to 1 or some other integer greater than 1. This has the effect of recreating or redeploying a previously terminated controller 122, whereby the value of the scale 322 indicates the number of container groups 126 (e.g., instances) to be redeployed to execute within the deployment 120 to be controlled by the corresponding controller 122. The incremented value of the scale 322 of the redeployed controller 122 set by the process creator 500 may correspond to the number of container groups 126 associated with the deployment 120 before the deployment 120 or execution of the controller 122 is terminated. When a controller 122 is created (such as when the scale 322 is changed from 0 to 1), the controller 122 will perform a reconciliation loop (or other self-healing mechanism) to determine the desired state of the deployment 120 and the actual state of the deployment 120, and reconcile the differences between the two. Using this reconciliation, the controller 122 will discover and / or learn the events 125 that triggered the redeployment of the deployment 120 and the controller 122. Therefore, the master controller 150 may not need to update or notify the controller 122 of past events.

[0037] In the example shown, the process creator 500 also implements a timer 530. The timer 530 allows the process creator 500 to periodically redeploy the controller 122 without triggering the event 125. That is, the process creator 500 can determine whether the termination time period since the resource controller 122 has been terminated meets the threshold time period, and in response, recreate the container group 126. For example, even if the event observer 400 does not detect an event 125 on the event criteria list 124, it may be desirable to redeploy the controller 122 once every ten minutes to allow for reconciliation loops. This may help achieve "drift fixes" or allow periodic operations, such as backups. Each controller 122 can indicate whether such assistance is necessary, and the master controller 150 can react accordingly. That is, each controller 122 can indicate to the master controller 150 whether the controller 122 needs to operate periodically, and then the master controller 150 responds to the indication accordingly. In some implementations, the controller 122 can indicate a time-based periodic event criteria, or an event criteria that matches at a specific time. For example, the controller 122 may set a periodic event to schedule a reconciliation every 10 minutes, or in another example, schedule an event to take effect at 2 a.m. for daily backups. Periodic schedules or time-based policies of arbitrary complexity may be expressed as the next scheduled event. When evaluating whether to terminate a container group 126, the master controller 150 may consider the remaining time until the next time-based event.

[0038] Continue to refer Figure 5 In some implementations, the process creator 500 also includes a health checker 540. The health checker 540 receives the system events 125 and can determine based on the events 125 that the controller 122 should be redeployed, even if the events 125 are not on the event criteria list 124. For example, when a power failure event or a network failure event is received, the health checker 540 can determine that the controller should be redeployed for maintenance or reconfiguration. As another example, the master controller 150 can determine that a newer software version or configuration of the controller 122 should be installed. After the deployment 120 is redeployed, the idle determiner 200 again determines when the deployment 120 is restored back to idle.

[0039] In other implementations, each controller 122 is configured to run in a "one shot" mode. That is, the container group 126 or container 128 executes until each application in the deployment 120 (executing within the container 128) reaches a steady state (i.e., an idle state), and the controller 122 then automatically exits. The controller may additionally notify the master controller 150 that the process is complete. The master controller 150 may then take responsibility for watching for events 125 associated with the terminated controller 122, and redeploying the deployment 120 when a triggering event 125 occurs.

[0040] In some examples, the master controller 150 acts as a scheduler for deployment 120 or target resources 126 (e.g., container groups). That is, the master controller 150 can coordinate the deployment of controllers 122 to limit the total resource (e.g., memory) usage. For example, if an event 125 occurs that triggers the deployment of multiple controllers 122 at the same time, the master controller 150 can appropriately schedule the deployment of each controller 122 to adapt to the available resources.

[0041] Reference Figure 6 , a graph 600 depicts resource availability over time for an exemplary system 100 executing a plurality of container groups 126. The horizontal x-axis depicts time, and the vertical y-axis depicts resource consumption. The graph 600 may be referenced Figure 1 100 is described with reference to the system 100 of FIG. The system 100 can create / instantiate multiple container groups 126, each of which includes a group of one or more containers 128 with isolated resources (e.g., computing, memory, storage, network access, etc.) and a corresponding resource controller 122 that controls one or more container groups 126. Each container 128 within the container group 126 can encapsulate a corresponding application to provide a secure execution environment for the corresponding application. When configuring the container group 126, the system 100 generally considers the total available system resources, and then schedules the creation of the container group 126 so that the amount of system resources reserved for the use of the created container group 126 does not exceed the total available system resources. The horizontal dashed line indicates the total available system resources. In particular, in time period T0, container groups P0, P1 are executing, while P2, P3, P4 are being queued and scheduled for execution. Because executing container groups P2-P4 and container groups P0, P1 during time period T0 would exceed the total available system resource amount, the execution of each of container groups P2-P4 is delayed, for example, by designating container groups P2-P4 in a queue so that the execution of container groups P2-P4 occurs at a time after time period T0 when a sufficient amount of system resources are available.

[0042] In time period T1, the execution of container group P1 is completed, and the system 100 terminates container group P1, thereby releasing enough system resources to be used for executing container group P2 without exceeding the total available system resources. Therefore, the system 100 starts the execution of container group P2, and container group P0 continues to execute during time T1. In time period T2, the execution of container group P0 is completed, and the system 100 terminates container group P1, thereby releasing enough system resources to be used for executing container group P3, without exceeding the total available system resources. Finally, in time period T3, the execution of container groups P2 and P3 is completed, and the system terminates container groups P2 and P3, thereby releasing enough system resources to be used for executing container group P4, without exceeding the total system resources. In this way, the system 100 can appropriately schedule the allocation of limited system resources for use by multiple container groups 126, each container group 126 including a corresponding group of one or more containers 128 and a corresponding resource controller 122 for controlling the one or more container groups 126.

[0043] The deployment or target resources 126 (e.g., container groups) can be scheduled in a variety of ways. For example, the system 100 can use round-robin, weighted round-robin, first-in-first-out, etc. The controller 122 can have an associated priority, and the controller 122 associated with a higher priority that has been terminated (e.g., due to being idle) is redeployed before the controller 122 associated with a lower priority. The priority can be set (e.g., by a system administrator) or automatically determined by the system 100. In some examples, the priority is assigned based on the time sensitivity of the container 128 (i.e., the application). That is, the less sensitive the application is to time, the lower the priority assigned to it. In some implementations, the main controller 150 can utilize the functionality of the scheduler, such as targeting the controller 122 to operate within a resource-constrained quota or low-priority scheduling class, or marking the controller 122 as a good candidate for eviction. The main controller 150 can also operate as an extension of the scheduler to implement behaviors that cannot be easily implemented in a general scheduler.

[0044] Figure 7 700 is a flow chart of an exemplary method 700 for conserving resources in a distributed system. The flow chart begins at operation 702 by processing data on the data processing hardware 810 ( Figure 8) begins by receiving an event criteria list 124 from a controller 122. The event criteria list 124 describes one or more events 125 observed by the resource controller 122, and the resource controller 122 is configured to respond to an event 125 that occurs and matches the event criteria list 124. The event criteria list 124 may include timer events. The event criteria list 124 may also include a dynamic event criteria list. In some implementations, the method 700 includes receiving multiple event criteria lists 124 from multiple resource controllers 122 executing simultaneously, each event criteria list 124 being associated with a corresponding resource controller 122. Based on the total available system resources, the number of resource controllers 122 executing simultaneously may be limited. In some examples, receiving the event criteria list 124 from the resource controller 122 includes retrieving the event criteria list 124 from an HTTP endpoint.

[0045] At step 704, the method 700 includes determining, by the data processing hardware 810, whether the resource controller 122 is idle. In some examples, the method 700 includes polling the resource controller 122 to determine whether the resource controller 122 is idle. In other examples, the method 700 includes analyzing the events 125 that have occurred during the threshold time period, and determining that the resource controller 122 is idle when no events 125 on the event criteria list 124 have occurred within the threshold time period.

[0046] When the resource controller 122 is idle, the method 700 includes, at step 706, terminating, by the data processing hardware 810, the resource controller 122. In some implementations, the master controller 150 determines when the resource controller 122 is idle and terminates the resource controller by setting (e.g., scaling) the deployment size 322 of the resource controller 122 to zero. Prior to terminating the resource controller 122 when the resource controller 122 is idle, the method 700 optionally includes determining, by the data processing hardware 810, whether a measurement 332 of system resources meets a system resource threshold. For example, the system resource threshold may be met when the measurement 332 of system resources currently used or reserved for use by the system 100 exceeds a value corresponding to the system resource threshold. When the measurement 332 of system resources meets the threshold and the resource controller 122 is both idle, the method 700 includes terminating the resource controller 122. The measurement 332 of system resources may include measurements of system processing / computing, network access, storage, and / or memory resources.

[0047] At step 708, method 700 includes executing, by data processing hardware 810 ( Figure 8) determines whether any event 125 from the event criteria list 124 occurred after terminating the resource controller 122. The method 700 may include polling the data to determine whether any event 125 has occurred. When at least one event 125 from the event criteria list 124 occurs after terminating the resource controller 122, the method 700 includes, at step 710, recreating the resource controller 122 by the data processing hardware 810. For example, recreating the resource controller 122 may include re-deploying (e.g., via the master controller 150 executing on the data processing hardware 810) the resource controller 122 by setting (e.g., scaling) the deployment size 322 of the controller 122 from zero to one (or some other integer greater than 1 depending on the number of instances / container groups 126 to be re-deployed and controlled by the controller 122). The resource controller 122 may be associated with a system resource reservation, and terminating the resource controller 122 may remove the system resource reservation. The system resource reservation may indicate the amount and / or type of system resources required by the resource controller 122 and the associated container groups 126 when in use.

[0048] In some implementations, the method 700 includes determining, by the data processing hardware 810, whether a termination time period since the resource controller 122 was terminated satisfies a threshold time period, and in response to determining that the termination time period does satisfy the threshold time period, recreating, by the data processing hardware 810, the resource controller 122. In some examples, the method 700 includes determining, by the data processing hardware 810, whether the resource controller 122 requires maintenance or reconfiguration. In these examples, in response to determining that the resource controller 122 does require maintenance or reconfiguration, the method 700 then includes recreating, by the data processing hardware 810, the resource controller 122.

[0049] Optionally, the method 700 may include determining, by the data processing hardware 810, whether any of the events 125 on the plurality of event criteria lists 124 occur simultaneously. In response to determining that the events 125 on the plurality of event criteria lists 124 occur simultaneously, the method 700 then includes sequentially recreating, by the data processing hardware 810, a group of resource controllers 122, each resource controller 122 in the group of resource controllers 122 being associated with a respective event criteria list 124 of the plurality of event criteria lists 124. Sequentially recreating the group of resource controllers 122 may include sequentially recreating the group of resource controllers 122 based on a priority of each resource controller 122. In other examples, the method 700 includes recreating, by the data processing hardware 118, the resource controllers 122 based on the system metrics 332. The system metrics 332 may include measurements of system resources.

[0050] A software application (i.e., software resource) may refer to computer software that enables a computing device to perform tasks. In some examples, a software application may be referred to as an "application," "app," or "program." Exemplary applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0051] Figure 8 800 that can be used to implement the systems and methods described in this document. Computing device 800 is intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions are intended to be exemplary only, and are not intended to limit implementations of the inventions described and / or claimed in this document.

[0052] The computing device 800 includes a processor 810 (e.g., data processing hardware), a memory 820 (e.g., memory hardware), a storage device 830, a high-speed interface / controller 840 connected to the memory 820 and a high-speed expansion port 850, and a low-speed interface / controller 860 connected to a low-speed bus 870 and the storage device 830. Each of the components 810, 820, 830, 840, 850, and 860 are interconnected using various buses and can be mounted on a common motherboard or in other suitable ways. The processor 810 can process instructions for execution within the computing device 800, including instructions stored in the memory 820 or on the storage device 830, to display graphical information for a graphical user interface (GUI) on an external input / output device, such as a display 880 coupled to the high-speed interface 840. In other implementations, multiple processors and / or multiple buses can be used as appropriate along with multiple memories and multiple types of memories. Furthermore, multiple computing devices 800 may be connected, with each device providing portions of the necessary operations (eg, as a server array, a group of blade servers, or a multi-processor system).

[0053] The memory 820 stores information non-temporarily within the computing device 800. The memory 820 may be a computer-readable medium, a volatile memory unit, or a non-volatile memory unit. The non-temporary memory 820 may be a physical device used to temporarily or permanently store programs (e.g., sequences of instructions) or data (e.g., program state information) for use by the computing device 800. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0054] The storage device 830 can provide mass storage for the computing device 800. In some implementations, the storage device 830 is a computer-readable medium. In various implementations, the storage device 830 can be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or a device array, including devices in a storage area network or other configurations. In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods such as described above. The information carrier is a computer-readable medium or a machine-readable medium, such as a memory 820, a storage device 830, or a memory on a processor 810.

[0055] The high-speed controller 840 manages bandwidth-intensive operations for the computing device 800, while the low-speed controller 860 manages less bandwidth-intensive operations. Such allocation of responsibilities is exemplary only. In some implementations, the high-speed controller 840 is coupled to the memory 820, the display 880 (e.g., through a graphics processor or accelerator), and to a high-speed expansion port 850 that can accept various expansion cards (not shown). In some implementations, the low-speed controller 860 is coupled to the storage device 830 and the low-speed expansion port 890. The low-speed expansion port 890, which can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device, such as a switch or a router, for example, through a network adapter.

[0056] The computing device 800 can be implemented in many different forms, as shown in the figure. For example, it can be implemented as a standard server 800a, or multiple times in a group of such servers 800a, as a laptop computer 800b, or as part of a rack server system 800c.

[0057] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system that includes at least one programmable processor that may be special or general purpose and coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0058] These computer programs (also referred to as programs, software, software applications or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0059] The processes and logic flows described in this specification can be performed by one or more programmable processors (also referred to as data processing hardware), which execute one or more computer programs to perform functions by operating on input data and generating outputs. Processes and logic flows can also be performed by special logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits). Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., disks, magneto-optical disks, or optical disks) for storing data, or be coupled in an operational manner to receive data from one or more mass storage devices or to transfer data to one or more mass storage devices or both. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including, for example: semiconductor memory devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CDROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits.

[0060] To provide interaction with a user, one or more aspects of the present disclosure may be implemented on a computer having a display device, such as a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen, for displaying information to the user, and optionally having a keyboard and a pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0061] Many implementations have been described. However, it will be appreciated that various modifications can be made without departing from the spirit and scope of the present disclosure. Therefore, other implementations are within the scope of the following claims.

Claims

1. A method (700) for saving resources in a containerized system, comprising: receiving, at data processing hardware (810), from a resource controller (122), an event criteria list (124), the event criteria list (124) comprising one or more events (125) observed by the resource controller (122), the resource controller (122) controlling at least one target resource (126) and configured to respond to the occurrence of one or more events (125) from the event criteria list (124); determining, by the data processing hardware (810), whether the resource controller (122) is idle; and When the resource controller (122) is idle: terminating, by the data processing hardware (810), the resource controller (122) by scaling the deployment size (322) of the resource controller (122) to zero; determining, by the data processing hardware (810), whether any event (125) from the event criteria list (124) occurs after terminating the resource controller (122); as well as When at least one event (125) of the one or more events from the event criteria list (124) occurs after terminating the resource controller (122), The resource controller (122) is recreated by the data processing hardware (810).

2. The method (700) of claim 1, wherein: Determining whether the resource controller (122) is idle includes polling the resource controller (122).

3. The method (700) of claim 1, wherein: Determining whether the resource controller (122) is idle includes: analyzing events that have occurred during a threshold time period (125); and The resource controller (122) is determined to be idle when none of the one or more events (125) on the event criteria list (124) occurs within the threshold time period.

4. The method (700) of claim 1, wherein: Recreating the resource controller (122) includes scaling the deployment size (322) of the resource controller (122) to one.

5. The method (700) of claim 1, wherein: The event criteria list (124) includes a timer event (125).

6. The method (700) of claim 1, wherein: Determining whether any event (125) on the event criteria list (124) occurs includes polling data to determine when an event (125) on the event criteria list (124) occurs.

7. The method (700) of claim 1, further comprising, after terminating the resource controller (122): determining, by the data processing hardware (810), whether a termination time period since termination of the resource controller (122) satisfies a threshold time period; and In response to determining that the expiration time period does satisfy the threshold time period, the resource controller (122) is recreated by the data processing hardware (810).

8. The method (700) of claim 1, further comprising, after terminating the resource controller (122): determining, by the data processing hardware (810), whether the resource controller (122) needs to be maintained or reconfigured; and In response to determining that the resource controller (122) needs to be maintained or reconfigured, the resource controller (122) is recreated by the data processing hardware (810).

9. The method (700) of claim 1, wherein: Receiving the event criteria list (124) from the resource controller (122) includes receiving a plurality of event criteria lists (124) from a plurality of resource controllers (122) executing simultaneously, each event criteria list (124) being associated with a respective resource controller (122).

10. The method (700) of claim 9, wherein: The number of resource controllers (122) executing simultaneously is based on the total available system resources (332).

11. The method (700) of claim 9, further comprising: Determining, by the data processing hardware (810), whether any events (125) on a plurality of event criteria lists (124) occur simultaneously; as well as In response to determining that events (125) on multiple event criterion lists (124) occur simultaneously, the data processing hardware (810) sequentially recreates a group of resource controllers (122), each resource controller (122) in the group of resource controllers (122) being associated with a corresponding event criterion list (124) of the multiple event criterion lists (124).

12. The method (700) of claim 11, wherein: Recreating the group of resource controllers (122) in sequence includes recreating the group of resource controllers (122) in sequence based on a priority of each resource controller (122).

13. The method (700) of claim 1, further comprising recreating, by the data processing hardware (810), the resource controller (122) based on system metrics (332).

14. The method (700) of claim 13, wherein: The system metrics (332) include measurements (332) of system resources.

15. The method (700) of claim 1, wherein: The event criteria list (124) includes a dynamic event criteria list (124).

16. The method (700) of claim 1, further comprising, before terminating the resource controller (122) when the resource controller (122) is idle: determining, by the data processing hardware (810), whether the measurement (332) of a system resource satisfies a system resource threshold; and The resource controller (122) is terminated by the data processing hardware (810) when both the measurement (332) of the system resource meets the system resource threshold and the resource controller (122) is idle.

17. The method (700) of claim 16, wherein: The measurements (332) of system resources include measurements of system processing / computing, network access, storage and / or memory resources.

18. The method (700) of claim 1, wherein: Receiving the event criteria list (124) from the resource controller (122) includes receiving the event criteria list (124) from an HTTP endpoint.

19. The method (700) according to any one of claims 1 to 18, wherein: The resource controller (122) is associated with a system resource reservation, and terminating the resource controller (122) removes the system resource reservation.

20. A system (100) for conserving resources in a containerized system, comprising: Data processing hardware (810); as well as Memory hardware (820) in communication with the data processing hardware (810), the memory hardware (820) storing instructions that, when executed on the data processing hardware (810), cause the data processing hardware (810) to perform operations, the operations comprising: receiving an event criteria list (124) from a resource controller (122), the event criteria list (124) comprising one or more events (125) observed by the resource controller (122), the resource controller (122) controlling at least one target resource (126) and configured to respond to the occurrence of the one or more events (125) from the event criteria list (124); determining whether the resource controller (122) is idle; and When the resource controller (122) is idle: terminating the resource controller (122) by scaling the deployment size (322) of the resource controller (122) to zero; determining whether any event (125) from the event criteria list (124) occurs after terminating the resource controller (122); as well as When at least one event (125) of the one or more events from the event criteria list (124) occurs after terminating the resource controller (122), the resource controller (122) is recreated.

21. The system (100) of claim 20, wherein: Determining whether the resource controller (122) is idle includes polling the resource controller (122).

22. The system (100) of claim 20, wherein: Determining whether the resource controller (122) is idle includes: analyzing events that have occurred during a threshold time period (125); and The resource controller (122) is determined to be idle when no events (125) on the event criteria list (124) have occurred within the threshold time period.

23. The system (100) of claim 20, wherein: Recreating the resource controller (122) includes scaling the deployment size (322) of the resource controller (122) to one.

24. The system (100) of claim 20, wherein: The event criteria list (124) includes a timer event (125).

25. The system (100) of claim 20, wherein: Determining whether any event (125) on the event criteria list (124) occurs includes polling data to determine when an event (125) on the event criteria list (124) occurs.

26. The system (100) of claim 20, wherein: The operations further include, after terminating the resource controller (122): determining whether a termination time period since termination of the resource controller (122) satisfies a threshold time period; and In response to determining that the expiration time period does satisfy the threshold time period, the resource controller is recreated (122).

27. The system (100) of claim 20, wherein: The operations further include, after terminating the resource controller (122): determining whether the resource controller (122) needs to be maintained or reconfigured; and In response to determining that the resource controller (122) needs to be maintained or reconfigured, the resource controller (122) is recreated.

28. The system (100) of claim 20, wherein: Receiving the event criteria list (124) from the resource controller (122) includes receiving a plurality of event criteria lists (124) from a plurality of resource controllers (122) executing simultaneously, each event criteria list (124) being associated with a respective resource controller (122).

29. The system (100) of claim 28, wherein: The number of resource controllers (122) that execute simultaneously is based on the total amount of available system resources (332).

30. The system (100) of claim 28, wherein: The operations also include: determining whether any events (125) on a plurality of event criteria lists (124) occur simultaneously; and In response to determining that events (125) on multiple event criteria lists (124) occur simultaneously, a group of resource controllers (122) is recreated in sequence, and each resource controller (122) in the group of resource controllers (122) is associated with a corresponding event criteria list (124) of the multiple event criteria lists (124).

31. The system (100) of claim 30, wherein: Recreating the group of resource controllers (122) in sequence includes recreating the group of resource controllers (122) in sequence based on a priority of each resource controller (122).

32. The system (100) of claim 20, wherein: The operations also include recreating the resource controller (122) based on the system metrics (332).

33. The system (100) of claim 32, wherein: The system metrics (332) include measurements (332) of system resources.

34. The system (100) of claim 20, wherein: The event criteria list (124) includes a dynamic event criteria list (124).

35. The system (100) of claim 20, wherein: The operations further include, before terminating the resource controller (122) when the resource controller (122) is idle: determining whether the measurement of system resources (332) satisfies a system resource threshold; and When the measurement (332) of the system resource meets the system resource threshold and the resource controller (122) is both idle, the resource controller (122) is terminated.

36. The system (100) of claim 35, wherein: The measurements (332) of system resources include measurements of system processing / computing, network access, storage and / or memory resources.

37. The system (100) of claim 20, wherein: Receiving the event criteria list from the resource controller includes receiving the event criteria list from an HTTP endpoint.

38. The system (100) according to any one of claims 20-37, wherein: The resource controller is associated with a system resource reservation, and terminating the resource controller removes the system resource reservation.

Images