Customized cross-venue resource selection for containerized applications
By dynamically selecting cloud provider network resources through the control plane server of Container Management Service (CMS), the scheduling strategy problem between cloud and on-premises resources for containerized applications is solved, improving performance stability and resource utilization.
Patent Information
- Application Number
- CN202380028684.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-22
- Filing Date
- 2023-03-06
- Publication Date
- 2024-11-08
AI Technical Summary
Existing containerized applications struggle to efficiently utilize dynamic scheduling strategies between cloud and on-premises resources, leading to performance instability and resource waste.
By using the control plane server of Container Management Service (CMS) to identify external resource sets and dynamically select resources from cloud provider networks based on performance and availability metrics, cross-site resource selection for containers can be achieved.
It improves the performance stability and resource utilization of containerized applications, ensures timely switching to cloud resources under high load, and reduces resource waste.
Smart Images

Figure CN118922820A_ABST
Abstract
Description
Background Art
[0001] A software container represents a logical wrapper of a software application that abstracts the application from the computing environment in which it is executed. For example, a containerized version of a software application may include the software code and any dependencies used by the code, so that the application can be run on a server hosting a suitable container engine (e.g., or Containers are virtualized at the operating system level, and therefore generally represent a more lightweight packaging for running applications on a host computing system. Existing software applications can be "containerized" by packaging the software application in an appropriate manner and generating other artifacts (e.g., container images, container files, or other configurations) to enable the application to run in a container engine. Some modern applications and services are organized as a collection of tasks, each of which is run using a set of containers. BRIEF DESCRIPTION OF THE DRAWINGS
[0002] Figure 1 An example system environment is shown in which clients of a provider network's container management service can specify cross-premises resource selection policies for their containerized applications, according to at least some embodiments.
[0003] Figure 2 Example scenarios are illustrated in accordance with at least some embodiments in which, during periods of increased workload, a containerized application may be run using a combination of resources at a provider network and at locations external to the provider network.
[0004] Figure 3 Example scenarios are shown in which a container management service of a provider network may monitor the availability of resources at locations external to the provider network to implement resource selection policies, according to at least some embodiments.
[0005] Figure 4 Example types of resources that may be used by a container management service to execute a container are shown, according to at least some embodiments.
[0006] Figure 5 Example elements of a resource selection policy for a containerized application are shown according to at least some embodiments.
[0007] Figure 6 Example program interactions related to policy-based resource selection for containerized applications are shown according to at least some embodiments.
[0008] Figure 7Example scenarios are shown in which a secure communication channel can be established between a control plane of a container management service and an agent deployed at a container execution resource, according to at least some embodiments.
[0009] Figure 8 is a flow diagram illustrating aspects of operations that may be performed to govern cross-premises resource selection for containerized applications according to at least some embodiments.
[0010] Fig. 9 Example scenarios are shown in which containerized application components can be executed at various availability zone domains and other architectural units according to at least some embodiments.
[0011] Fig.10 is a block diagram illustrating an example computing device that may be used in at least some embodiments.
[0012] Although embodiments are described herein by exemplifying several embodiments and illustrative drawings, it will be recognized by those skilled in the art that embodiments are not limited to the described embodiments or drawings. It should be understood that the drawings and their detailed description are not intended to limit the embodiments to the specific forms disclosed, but on the contrary, the present invention will cover all modifications, equivalents and alternatives that fall within the spirit and scope defined by the appended claims. The titles used herein are only for organizational purposes and are not intended to be used to limit the scope of the specification or claims. As used throughout this application, the word "may" is used in a permissive sense (i.e., meaning possible) rather than a mandatory sense (i.e., meaning must). Similarly, the word "include / including / includes" means including but not limited to. When used in the claims, the term "or" is used as an inclusive "or" rather than an exclusive "or". For example, the phrase "at least one of x, y or z" means any one of x, y and z and any combination thereof. Unless otherwise expressly stated, throughout this application, articles such as "a / an" are generally understood to include one or more described items. Thus, phrases such as "a means configured to..." are intended to include one or more of said means. Such one or more of said means may also be collectively configured to perform the stated statements. For example, "a processor configured to perform statements A, B, and C" may include a first processor configured to perform statement A working in conjunction with a second processor configured to perform statements B and C. DETAILED DESCRIPTION
[0013] The present disclosure relates to methods and apparatus for enabling clients of a cloud-based container management service (who may prefer to use resources at customer-owned premises for containerized applications managed by the service) to specify conditions (such as periods of high workload levels) under which some of the application's containers should instead be run at cloud-based computing resources. A client of a container management service (CMS), whose management component or control plane is implemented at a data center of a cloud provider network, may register resources (such as a set of physical or virtual machines) at locations outside the cloud provider network as candidates for running tasks of the client's containerized application. The client may also provide specifications for various containerized applications to be orchestrated by the CMS, such as machine images of containers to be used for tasks of the application, a minimum number of tasks to be run in parallel for managed service components of the application, and the like. The CMS client may also indicate one or more sets of container execution resources to be used for the application, and rules to be used to dynamically select execution resources for scheduling additional containers from a registered resource pool outside the cloud provider network or from a resource pool available at a data center of the cloud provider network.
[0014] Consider a scenario where a CMS client has invested in a sizeable set of resources RS1 at a client datacenter outside of a cloud provider network. Such a client may indicate to the CMS that RS1 at the client datacenter should be used by the CMS to launch containers for App1 tasks as long as the workload level of the containerized application App1 remains below a specified threshold T. The client may also indicate that any additional containers for App1 tasks should be launched by the CMS at a specified class of cloud provider network computing resources if the workload level exceeds T for at least some period of time. A variety of different classes of cloud provider network computing resources are available to the CMS, including CMS-managed computing instances of a virtualized computing service (VCS), client-acquired and client-managed computing instances of the VCS, interruptible CMS-managed computing instances of the VCS (computing resources with dynamically changing costs to the client based on demand, which may have to be abandoned if / when demand increases), etc. Additional App1 tasks may be launched, for example, in response to specific task requests from the client and / or based on scaling rules specified by the CMS client for the containerized application. The CMS may analyze various metrics (e.g., performance metrics and / or availability metrics) collected from containerized applications to determine whether and when cloud-based resources should be used instead of external-site resources. By specifying rules for selecting different types of resources, CMS clients may be able to ensure that their external-site computing resources are utilized to the desired extent, while also ensuring that cloud-based resources are available for their containerized applications on demand and when needed. CMS clients may also specify rules for moving containerized application workloads from cloud-based resources back to external resources (e.g., after a workload surge that caused the use of cloud resources has subsided). CMS may also be referred to as a container orchestration service, container service, cloud container service, container engine, or container cloud service.
[0015] Those skilled in the art will recognize in light of this disclosure that certain embodiments may be able to achieve various advantages, including some or all of the following: (a) providing desired performance levels (such as application request response times) for containerized applications that selectively run primarily at locations external to a cloud-provided network's data center, but that may sometimes reach workload levels that cannot be handled comfortably at external locations; and / or (b) ensuring that utilization levels of various types of container execution resources selected by clients of a container management service remain within desired ranges.
[0016] According to some embodiments, the system may include one or more control plane servers (CPS) of a container management service (CMS) of a cloud provider network, and one or more computing resource pools including resources located at one or more data centers of the provider network. One or more CPSs may be configured to identify an external resource set to be used to run at least some tasks of a containerized application on behalf of a CMS client. The resources of the external resource set (such as physical and / or virtual machines) may be located at a location outside the provider network. Individual tasks of the containerized application may be run using one or more software containers, wherein in at least some embodiments, various details about the mapping of tasks to containers, machine images of containers, etc. are provided to the CMS by the client via a programming interface.
[0017] In various embodiments, the CPS may obtain burst mode resource selection rules for containerized applications via such programming interfaces. The resource selection rules may indicate one or more conditions or criteria under which the software containers for the application's tasks will run at resources at a data center of the provider network rather than using an external set of resources. In various embodiments, one or more of the CPSs may analyze a set of metrics for the application, which is collected at least in part from the external set of resources during a time interval in which the application's containers run at the external set of resources. In some embodiments, in response to detecting that a condition indicated in the resource selection rule has been met based at least in part on an analysis of the set of metrics, one or more CPSs may cause a specific software container for a specific task of the application to run at a specific resource in a computing resource pool of the cloud provider network.
[0018] In at least some embodiments, containerized applications managed by the CMS on behalf of CMS clients may be configured to respond to application requests from various sources, such as clients of the containerized applications distributed across various networks, such as the public Internet or the cloud provider network itself. In one such embodiment, a load balancer may be configured or established, for example, at an external location with the aid of a CMS control plane server, for distributing such application requests among the various resources deployed for the containerized application. In scenarios where portions of an application are implemented using resources within a provider network data center, while the remainder is implemented at an external location, the CMS control plane server may provide the load balancer with connection information about the different resources currently being used for the application at different points in time. In various embodiments, this may enable the load balancer to forward application requests to application containers / resources deployed within the provider network as well as containers / resources at an external location (or more generally, anywhere its resources are being used).
[0019] In various embodiments, any number of different types of cloud-based container execution resources may be used by the CMS for portions of a containerized application. In one embodiment, a CMS client may pre-launch a set of computing instances or virtual machines at a virtualized computing service (VCS) and programmatically notify the CMS (e.g., as part of a resource selection rule) that such pre-launched client-managed computing instances should be used for applications under specified conditions. In another embodiment, a client may indicate to the CMS via a programming interface that at least a portion of a containerized application will be implemented as a managed service by the CMS without requiring the CMS client to acquire computing resources for this portion of the application. In this scenario, the CMS may dynamically acquire (or select from a pre-allocated pool) resources for some tasks of the application on behalf of the CMS client; in some cases, the CMS client may not necessarily be provided with information identifying the acquired resources. In one embodiment, the VCS may support the acquisition of interruptible computing instances (also known as "spot" instances), which are acquired at low cost from currently unused VCS data center computing capacity, but may have to be abandoned when demand from other sources increases, resulting in increased costs. In some embodiments, such interruptible computing resources may be specified by a CMS client as an option for a containerized application and used by the CMS based on conditions indicated by the client.
[0020] To determine whether the conditions indicated in the resource selection rules specified by the client have been met, in various embodiments, the CMS (e.g., one or more of the CPSs of the CMS) may analyze metrics collected from resources associated with the corresponding containerized application, for example, including metrics obtained from an external resource set. In different embodiments, such metrics may include performance metrics, such as resource utilization levels (CPU utilization, memory utilization, storage device utilization, network link utilization, network device utilization, etc.), latency of various application requests or lower-level operations performed for the application, application request arrival rates, current available resource capacity of the external resource set (which may change over time), etc. In some embodiments, instead of or in addition to performance metrics, availability / failure related metrics (such as counts of failed application requests, containers that exited unexpectedly, etc.) may also be used to check the conditions indicated in the resource selection rules. In some embodiments, the CMS may be able to track the available resource capacity available to the client application at the external location because the client may register the new server with the CMS control plane when the new server comes online for the containerized application at the external location, and because an agent of the CMS that can track the local resource utilization levels of the server is installed at each of the servers.
[0021] According to some embodiments, the CMS may provide various kinds of information about the mix of resources used for containerized applications. For example, in response to a program request, an indication of any combination of the following may be provided to a CMS client: (a) one or more software containers of an application running at an external set of resources during a specific time interval; (b) one or more software containers of an application running using provider network resources at a provider network data center during a specific time interval; and / or (c) details of specific conditions that led to the use of provider network resources for an application during a specific time interval (e.g., what specific data led to the deployment of provider network resources, when and where specific data was collected, etc.).
[0022] In one embodiment, the client may deploy additional container execution resources at the external location over time (e.g., in response to determining that the existing set of resources is insufficient to handle increasing workload levels or for other reasons). In such an embodiment, the CMS may obtain (or be provided with) an indication of the deployment of such additional external resources, and may select the additional resources for launching containers when new containers must be launched for the application. In one embodiment, the resource selection rules specified by the client may indicate that at least some tasks are to be actively transferred or migrated back to the external location - for example, as and when new capacity becomes available at the external location, one or more of the containers currently running for the application at the provider network resources may be terminated, and replacement containers may be started at the newly provisioned external resources.
[0023] In various embodiments, a corresponding CMS agent (e.g., a software program that establishes a secure network connection with the CMS control plane) may be started at each external resource at which a CMS managed container is to be run. The CMS agent may communicate with the CMS control plane server, receive commands to start containers, provide health status information related to the containers to the CMS control plane, provide performance metrics and / or available capacity metrics for the external resources, etc. In at least some embodiments, a corresponding system management agent may also be run at each such external resource, which may be used by the CMS client to make configuration changes to or view configuration information for various containers (on its behalf) running at the external resource, if necessary. In one embodiment, such CMS agents and / or system management agents may also be run at the provider network resources used to run the client's containerized applications.
[0024] In at least some embodiments, as indicated above, the CMS and / or VCS may be implemented as one of a set of services in a provider network or cloud computing environment. A cloud provider network (sometimes referred to simply as a "cloud") refers to a pool of network-accessible computing resources (such as computing, storage and network resources, applications, and services), which may be virtualized or bare metal. A cloud may provide convenient, on-demand network access to a shared pool of configurable computing resources that may be programmatically provisioned and released in response to customer commands. These resources may be dynamically configured and reconfigured to adjust for variable loads. Cloud computing may therefore be viewed as both applications delivered as services over a publicly accessible network (e.g., the Internet, a cellular communications network) and the hardware and software in a cloud provider data center that provides these services.
[0025] The cloud provider network can be formed into multiple regions, where a region is a separate geographic area in which the cloud provider clusters data centers. Such regions may also be referred to as regions defined by the provider network, because their boundaries do not necessarily coincide with the boundaries of countries, states, etc. Each region may include two or more availability zones connected to each other via a private high-speed network (e.g., a fiber-optic communication connection). An availability zone (also referred to as an availability domain or simply a "zone") refers to an isolated fault domain including one or more data center facilities, which have power, network, and cooling separated from data center facilities in another availability zone. A data center refers to a physical building or housing that houses the servers of the cloud provider network and provides power and cooling for the servers of the cloud provider network. Preferably, the availability zones within a region are located far enough apart from each other so that the same natural disaster does not take more than one availability zone offline at the same time. Customers can connect to the availability zones of the cloud provider network via a publicly accessible network (e.g., the Internet, a cellular communication network) through a transit center (TC). TCs can be considered as the main backbone locations that link customers to the cloud provider network and can be co-located at other network provider facilities (e.g., Internet service providers, telecommunications providers) and securely connected to availability zones (e.g., via virtual private networks (VPNs) or direct connections). Two or more TCs can be operated per region for redundancy. Each region is connected to a global network that connects each region to at least one other region. The cloud provider network can deliver content from points of presence located outside of these regions but networked to these regions through edge locations and regional edge cache servers (points of presence or POPs). This division and geographic distribution of computing hardware enables the cloud provider network to provide customers with low-latency access to resources with a high degree of fault tolerance and stability on a global scale.
[0026] In some embodiments, the container of the application of the CMS client can be started in the VCS region, at the edge location of the VCS, or at the VCS extension location. The edge location (or "edge zone") as referred to herein can be structured in several ways. In some implementations, the edge location can be an extension of the underlying network of the cloud provider, including a limited amount of capacity provided outside the availability zone (for example, in a small data center or other facility of the cloud provider located near the customer workload and possibly far away from any availability zone). Such edge locations can be called local areas (due to being more local or closer to a group of users than traditional availability zones). The local area can be connected to a publicly accessible network such as the Internet in various ways (for example, directly, via another network or via a private connection with the area). Although the local area usually has a more limited capacity than the region, in some cases, the local area can have a considerable capacity, such as thousands or more racks. Some local areas can use an architecture similar to a typical cloud provider data center. The extended location of the VCS can include a portion where one or more data plane servers of the VCS computing instance of the place owned by the client can be started at it. In various embodiments, a special highly secure channel using various types of tunneling technologies can be established to transmit commands (e.g., commands to launch compute instances and / or containers) from the control plane servers of the CMS VCS (which are maintained at the provider network data center) to the extended location data plane servers.
[0027] The cloud provider network may implement a variety of computing resources or services, which, in addition to the CMS and VCS, may include data processing services (e.g., map-reduce, data streaming, and / or other large-scale data processing techniques), data storage services (e.g., object storage services, block-based storage services, or data warehouse storage services), packet processing services, and / or any other type of network-based services (which may include various other types of storage, processing, analysis, communication, event handling, visualization, and security services). The resources required to support the operation of such services (e.g., computing and storage resources) may be provisioned in an account associated with the cloud provider, in contrast to resources requested by users of the cloud provider network, which may be provisioned in a user account.
[0028] In various embodiments, various network-accessible services may be implemented at one or more data centers, edge locations, and / or extended locations of a provider network. The network-accessible computing services may include elastic computing cloud services (referred to in various implementations as elastic computing services, virtual machine services, computing cloud services, computing engines, or cloud computing services). Such services may provide computing instances (also referred to as guest virtual machines, or simply "instances") with different computing and / or memory resources, which are managed by a computing virtualization service (referred to in various implementations as elastic computing services, virtual machine services, computing cloud services, computing engines, or cloud computing services). In one embodiment, each of the virtual computing instances may correspond to one of several instance types or families. An instance type may be characterized by its hardware type, computing resources (e.g., the number, type, and configuration of virtualized central processing units (VCPUs or CPU cores)), memory resources (e.g., the capacity, type, and configuration of local memory), storage resources (e.g., the capacity, type, and configuration of locally accessible storage), network resources (e.g., the characteristics of its network interfaces and / or network capabilities), hardware accelerator resources, and / or other suitable descriptive characteristics (e.g., an "overclockable" instance type with a baseline performance guarantee and the ability to periodically overclock above the baseline, or a non-overclockable or dedicated instance type that is allocated and guaranteed a fixed amount of resources). Each instance type may have a specific ratio of processing, local storage, memory, and network resources, and different instance families may also have different types of these resources. Multiple sizes of these resource configurations may be used within a given instance type. Using instance type selection functionality, an instance type may be selected for a customer, for example, based (at least in part) on input from the customer. For example, a customer may select an instance type from a set of predefined instance types. As another example, a customer may specify the desired resources for an instance type and / or the requirements of a workload that the instance will run, and the instance type selection functionality may select an instance type based on such specifications. An appropriate host for the requested instance type may be selected based at least in part on factors such as collected network performance metrics, resource utilization levels at different available hosts, and the like. In some embodiments, instances of several different instance types may be launched at an extended location in response to a programmatic request from a client. In some embodiments, other types of network accessible services may also be implemented at the cloud provider network, such as packet processing services, database services, wide area network (WAN) services, and the like.
[0029] In various embodiments, traffic and operations of a cloud provider network (or individual services of a cloud provider network) can be broadly subdivided into two categories: control plane operations carried via a logical control plane and data plane operations carried via a logical data plane. The data plane represents the movement of user data through a distributed computing system, while the control plane represents the movement of control signals through a distributed computing system. The control plane typically includes one or more control plane components distributed across and implemented by one or more control servers. Control plane traffic typically includes management operations, such as system configuration and management (e.g., resource placement, hardware capacity management, diagnostic monitoring, or system status information). The data plane includes customer resources (e.g., computing instances, containers, block storage volumes, databases, or file storage devices) implemented on the cloud provider network. Data plane traffic typically includes non-management operations, such as transmitting customer data to and from customer resources. Certain control plane components (e.g., first-tier control plane components, such as a control plane for virtualized compute services) are typically implemented on a separate set of servers from the data plane servers, while other control plane components (e.g., second-tier control plane components, such as analytics services) may share virtualized servers with the data plane, and control plane traffic and data plane traffic may be sent over separate / different networks.
[0030] Figure 1 An example system environment according to at least some embodiments is shown, in which a client of a provider network's container management service can specify a cross-site resource selection policy for its containerized application. As shown, the system 100 includes resources and artifacts of a software container management service (CMS) 125 of a cloud provider network 102A, including a container registry 174, an external resource set database 176, containerized application task / service specifications 177, client-specified resource selection policies / rules 178 (including workload surge management rules), and a control plane server 150. In the depicted embodiment, the cloud provider network 102A may also include a plurality of other services, such as a virtualized computing service (VCS) 110, various storage services (not shown), and the like.
[0031] The CMS may, in the depicted embodiment, implement a set of programming interfaces 121, such as one or more web-based consoles, command line tools, graphical user interfaces, a set of application programming interfaces (APIs), etc. Such interfaces may be utilized by CMS clients to submit messages and requests from various client devices 180 (such as desktop computers, laptop computers, mobile computing devices, etc.), and receive corresponding responses from the CMS.
[0032] In the depicted embodiment, the CMS can help its clients orchestrate and execute various types of containerized applications. A given containerized application may include a collection of one or more tasks, each of which is executed using one or more software containers (e.g., whose machine images and other properties can be specified by the client via programming interface 121). The CMS can reduce the management burden on the containerized application owner by, for example, delivering container images to the desired execution resources, managing container lifecycle transfers (e.g., starting / terminating containers), managing software updates, monitoring applications and taking actions based on collected data, etc. Some containerized applications (or parts thereof) can be implemented as managed services by the CMS at the request of the CMS client; for such managed services, the CMS may be responsible for, for example, ensuring that at least a specified number of tasks (or task replicas) are run at resources acquired and managed by the CMS, without the CMS client acquiring or specifying resources to be used for the tasks.
[0033] In various embodiments, any of a variety of resource types may be utilized by the CMS to run containers for CMS client applications. Figure 1 In the described embodiments, some of the CMS clients may wish to utilize computing resources at locations outside of a provider network datacenter for at least a portion of the client's containerized applications; other CMS clients may wish to use only provider network resources.
[0034] In the depicted embodiment, a CMS client that prefers to use external resources for at least a subset of its application tasks / containers can use programming interface 121 to provide information about external resources to the CMS control plane. For example, CMS client C1 can register one or more container execution resources 115 at an external location 112 (e.g., a data center owned / controlled / leased by C1) as a set of external resources to be used for one or more of C1's containerized applications. Similarly, another CMS client C2 can register container execution resources 117 as an external resource set via programming interface 121. In the depicted embodiment, some clients can register external resources at several different locations - for example, client C1 can also register container execution resources 119 at different cloud provider networks 102B (i.e., cloud provider networks owned and operated by organizations different from cloud provider network 102A). The external resources of some CMS clients may be completely located in other cloud provider networks - for example, the container execution resources 119 of CMS client C3 may be located in a data center of cloud provider network 102B, rather than at a location owned / controlled by C3. In the depicted embodiment, information provided regarding external resources (eg, network addresses, permissions required to access the resources when needed, etc.) may be stored in an external resource set database 176 of the CMS 125 .
[0035] In the depicted embodiment, some CMS clients may also acquire and manage resources within VCS 110 for at least a portion of their containerized applications (e.g., using autoscaling groups or other scalability management structures supported by VCS). For example, in the depicted embodiment, client-managed container execution resource pool 161 of CMS client C1 may include the set of VCS compute instances of autoscaling group ASG1. Similarly, CMS client C3 may have set up client-managed container execution resource pool 162. For some containerized applications, the CMS client may indicate to the CMS that the provider network-based container execution resources (if and when they are used) will be fully managed by the CMS without the client acquiring and managing the resources. In the depicted embodiment, CMS-managed container execution resource pool 163 may be used by the CMS for such applications. In at least some embodiments, at least a portion of the CMS-managed container execution pool 163 may be dynamically populated with actual resources (e.g., as and when a given new container requires resources), rather than pre-populating compute instances or other types of servers. Similarly, in some embodiments, the population level of the client-managed container execution resource pool 161 may also change dynamically, for example, based on scaling rules selected by the client and implemented using an auto-scaling group.
[0036] CMS clients may utilize programming interface 121 to inform the CMS control plane of the various containers to be used for different applications, such as identifiers / locations of machine images for containers, hardware / software requirements for containers, etc. In some embodiments, such information may be stored in a container registry 174. In the described embodiments, clients may utilize programming interface 121 to provide a definition or specification 177 of their containerized application, indicating, for example, the mix of tasks that make up the application, the connectivity requirements between tasks, whether part or all of the application is to be implemented by the CMS as a managed service and, if so, the task count requirements for the managed service, and / or other properties of the application. In some embodiments, containerized applications may be specified as logical groupings of tasks and / or services called "clusters," which may be mapped to corresponding groups of containers for implementation.
[0037] exist Figure 1In the depicted embodiment, a CMS client may use programming interface 121 to specify resource selection policies / rules 178 for its containerized applications. For example, such rules may indicate one or more conditions under which a particular kind of resource will be selected for the next task / container launched for an application. For example, such rules may indicate that, while under normal operating conditions (as defined using one or more metric thresholds), an external set of resources should be used to run containers for a given application, under specified circumstances (e.g., when the arrival rate of application requests for a containerized application remains above a threshold for a sustained time interval), one or more containers implementing the application should be launched at a provider network resource (such as resources of pool 161, 162, or 163).
[0038] In some embodiments, the control plane server 150 of the CMS may implement such resource selection rules as follows: the control plane server may continuously analyze the metrics indicated in the conditions of the rule and use the analyzed metrics to make decisions about which type of resources will be used for one or more containers for the next task (or next set of tasks) to be launched for the application. For example, if the default or preferred mode of operation (indicated in the resource selection rule specified by the client C1) is to launch containers of the application App1 for the client C1 at the external location 112 of C1, the control plane server 150 may first launch a certain number of tasks at the container execution resource 115, for example, for an initial time period of T seconds in the life of the application. As new metrics are collected from the container execution resource 115 and analyzed at the CMS control plane, resources for additional tasks (which may be launched based on the scaling requirements of the application) may be selected based on the results of the analysis (e.g., from the container execution resource 115, from the client's resource pool 161 at the cloud provider network 102A, and / or from C1's resources 119 at the cloud provider network 102B).
[0039] In the depicted embodiment, if the rules specified by the client indicate that tasks are to be moved from cloud provider network resources back to a specified set of external resources under specified conditions, such actions may also be initiated by the CMS control plane server as and when the conditions are detected. For example, if the burst of application requests for C1's containerized application subsides (during which resources of pool 161 or pool 163 were used for the application), then subsequent tasks (if any) launched for this application may again utilize resources at site 112 according to the rules specified by C1. In one embodiment, a more proactive approach may be taken to shift application workloads back to the target resource set or location - for example, rather than waiting until a decision must be made about launching new tasks or containers, existing containers may be terminated and replacement containers may be launched at the target resource set or location. In various embodiments, in general, using the kinds of rules described above, CMS clients may be given as much flexibility as they desire regarding selecting resources of different pools / kinds / locations to use for their containerized applications.
[0040] In at least some embodiments, in addition to the containers themselves, a load balancer may be established by the CMS at an external location for distributing application requests among a desired set of containers at the external location. For example, a corresponding load balancer may be set up at locations 112, 114 and / or within cloud provider network 102B. Note that, if desired, a load balancer may also be set up for containerized applications that use resources of provider network 102A—for example, a corresponding load balancer may be configured for pool 161 or pool 162, or for a set of CMS-managed resources deployed for a given client application in pool 163. In the depicted embodiment, if and when cloud provider network resources are deployed for containerized applications, requests for the containerized applications are distributed using an external location load balancer, the CMS control plane may provide information to the external location load balancer so that a subset of requests can be forwarded to the cloud provider network resources.
[0041] In various embodiments, a representation of the current mix of resources used for a given application and / or metrics collected and analyzed to make resource selection decisions may be provided to the CMS client upon request via the programming interface 121. Thus, if desired, the CMS client may be provided with an explanation for the decision to use provider network resources rather than client premises resources, or more generally, the reasons for using any particular kind of resources for the client's containerized application.
[0042] Figure 2 Example scenarios are illustrated in accordance with at least some embodiments in which, during periods of increased workload, a containerized application may be run using a combination of resources at a provider network and at locations external to the provider network. Figure 2 Along the Y axis, the Figure 1 The workload levels 210 of the containerized applications implemented by the CMS 125 are shown as increasing along the timeline 200 . Figure 2 In the depicted embodiment, it is assumed that the number or amount of external site resources available to the application is fixed.
[0043] The CMS client on whose behalf the containerized application is running has specified (e.g., using a resource selection rule of the kind described above) a burst threshold 205 for the workload. The rule indicates that resources from an external resource pool at an external location are to be used for the application during a period of time when the workload (e.g., as measured using a metric indicated by the client, such as the total number of application requests per second for the containerized application) remains below the threshold 205. The rule may also indicate that during a period of time when the workload exceeds the burst threshold, resources based on the provider network may be used for the application, e.g., in addition to resources from the external resource pool.
[0044] exist Figure 2 In the depicted embodiment, the following types of resource selection decisions may be made by the CMS based on the rules specified by the client. Between time T0 and time T1, resources at the external location are available only to containers for the application. Between T1 and T2, during the time period when the workload meets the burst threshold, provider network resources may be used to launch at least some additional containers for the application. At T2, the workload drops below the threshold again, so only in the described example scenario can external location resources be used to launch any additional containers launched for the application. Between T3 and T4, the burst criteria may be met again, so provider network resources may be used for additional containers for the application. After T4, external location resources may be used again for any additional containers because the workload has dropped below the burst threshold indicated in the rule.
[0045] Please note that in order to illustrate the concepts involved and the overall intention of the client, Figure 2 A somewhat simplified view of events is shown; in reality, the decision to begin using provider network resources may not be instantaneous, may require some time to verify that the increase beyond the burst threshold is sustained rather than transient, etc. In addition, some details about the containers launched at the provider network resources have not been specified. For example, whether containers launched at the provider network during the burst intervals T1-T3 and T3-T4 are allowed to continue running, or whether they are terminated after the workload drops below the burst threshold while replacement containers are launched at the external location. In some embodiments, the resource selection rules may include guidance from the client on how to handle such containers.
[0046] As indicated above, in relation to Figure 2 In the discussed scenario, it is assumed that the total amount of resources available at the external site remains fixed. In practice, a CMS client may sometimes (e.g., in response to increasing application workload trends) decide to add computing power to its external resource set over time. Figure 3 An example scenario is shown in which a container management service of a provider network can monitor the availability of resources at locations external to the provider network to implement a resource selection policy, according to at least some embodiments. Figure 3 In the lower part of FIG. 3 , the timeline 300 is shown with respect to Figure 2 The workload variation curve is similar to that shown in Figure 2 The indicated burst threshold 205 is similar to the initial or original burst threshold 305A. However, for Figure 3 In the scenario depicted, the assumption that the capacity of the external resource set remains fixed is not valid. Figure 3 As indicated in the upper portion of , the resources available at the external location 312 increase: the client (e.g., in response to a sudden increase in workload between T1 and T2) adds capacity at a time between T2 and T3 (T2+δ). In addition, the resource selection rules specified by the client may indicate to the CMS that the amount of available capacity at the external location should also be considered when making resource selection decisions for the application. In some embodiments, the amount of capacity may be tracked, for example, with the aid of a CMS agent installed at some or all of the servers or virtual machines in the external resource set. It may be necessary for each resource in the external resource set to include such a CMS agent, or a single CMS agent may be provided with connections to several resources of the external resource set, and each CMS agent may provide information about the utilization level of the resource to the CMS control plane. In one embodiment, as an alternative or in addition to using a CMS agent to obtain available capacity metrics from the external resource set, the CMS control plane may utilize a monitoring service of a provider network; for example, a CMS client may enable the monitoring service to access external resources to be used for CMS-managed containers, and at least some information about available capacity may be transmitted to the CMS control plane by the monitoring service.
[0047] exist Figure 3 In the embodiment described, according to the client's Figure 3 Based on the rules specified in the scenario, the CMS can make the following types of resource selection decisions. Between time T0 and time T1, such as Figure 2As shown, resources at the external location are available only for containers for the application. Between T1 and T2, during the time period when the workload meets the burst threshold 305A and the available resource capacity at the external location has not increased, the provider network resources can be used to launch at least some additional containers for the application. At T2, the workload drops below the threshold again, so only in the described example scenario can the external location resources be used to launch any additional containers launched for the application. Now, at T2+δ, more capacity is added at the external location. In the depicted embodiment, the added capacity can be sufficient so that the CMS can continue to use the external location resources even if the workload rises above the burst threshold 305A at time T3. Figure 3 In the example shown in , according to the policy specified by the client, after T2+δ, there is no need to use the provider network resources for the application. Figure 3 In the scenario shown, the client may specify a new, higher burst threshold 305B after adding capacity to the external site, so that if the workload rises above the new burst threshold, the provider network resources may be used again. In some embodiments, rather than the client specifying a threshold to be used to decide whether to use provider network resources, the CMS may automatically set the threshold (e.g., using a measure of available resource capacity obtained from the external site).
[0048] Figure 4 Example categories of resources available for container management services to execute containers are shown according to at least some embodiments. One such category may include client-managed provider network resources 405, such as VCS compute instances and / or autoscaling groups set up by a CMS client at one or more data centers of a provider network at which the CMS is implemented. An autoscaling group may refer to a collection of compute instances that are managed together for the purpose of scaling one or more applications. In various embodiments, a VCS client may specify parameters for such an autoscaling group, such as a minimum number of compute instances of one or more categories, a maximum number of compute instances, policies to be used to add compute instances or remove compute instances from a collection, and the like.
[0049] A second category of resources that can be used to execute containers may include CMS-managed non-interruptible provider network resources 410, such as VCS compute instances and / or autoscaling groups acquired and released by the CMS control plane. This category of computing power can be described as non-interruptible because after a particular resource of the category is allocated to an application's container, the resource can continue to be used for the container as long as the application needs it, without having to be abandoned based on increasing demand for computing resources (e.g., from other applications). In contrast, CMS-managed interruptible provider network-based resources 415 may include resources (such as VCS's "bid" compute instances) that are typically acquired at very low cost during time periods when the VCS has a large amount of spare or unused resources, but may have to be abandoned when demand from other applications increases. It should be noted that in various embodiments, it is not necessary to provide details of the resource (such as a compute instance identifier) to a CMS client that acquires a resource of category 410 or 415 on its behalf by the CMS. Instead, a "serverless" experience can be provided to the CMS client, in which the CMS client does not have to be concerned with acquiring or managing the resources at which the containers of its containerized application run.
[0050] In various embodiments, client-managed client-site resources 420 available for containers managed by the CMS may include virtual machines and / or physical servers located at a site outside of a provider network's data center. The CMS client may be responsible for acquiring / provisioning and at least partially configuring such resources. Another type of container execution resource used by the CMS on behalf of the client may include VCS-managed client-site resources 425, such as "sentinel" compute instances. Sentinels include a pool of provider network compute and storage capacity set up at a client site at the request of a client (e.g., including server racks transferred from the provider network to the site). A secure connection may be established between a sentinel management agent at the external site and the VCS control plane, and commands sent by the client to the VCS control plane to launch a compute instance at the external site may be relayed from the VCS control plane to the sentinel management agent for implementation.
[0051] In some embodiments, a CMS client may wish to utilize at least some resources of a different provider network (a provider network operated by an organization other than the operator of the provider network on which the CMS is implemented). Such provider networks may be referred to as "external" provider networks. Client-managed external provider network resources 430 may represent another category of resources that may be used by the CMS to run containerized applications. The CMS client may be responsible for acquiring and configuring external provider network resources and providing various types of information about such resources to the CMS control plane (e.g., network addresses, computing power, required credentials, etc.), which may be needed to run software containers on behalf of the client. In various embodiments, the CMS client is used to select when making a decision to launch a new container. Figure 4 The rules for any of the types of resources shown and / or the rules for "migrating" containers between different types of resources can be specified by the CMS client and implemented by the CMS control plane. Figure 4 Other categories of resources not shown may also or alternatively be used by containerized applications of the CMS client.
[0052] Figure 5 Example elements of a resource selection policy for a containerized application are shown in accordance with at least some embodiments. In at least some embodiments, a representation of a containerized application resource selection policy 510 (either for a specified application or for multiple applications) may be transmitted by a CMS client to a CMS control plane via a programming interface. In some implementations, such policies may be referred to as capacity provider policies or simply capacity providers. In the depicted embodiment, a given policy may indicate, for example, a preferred or default resource category 515 to be used for an application, e.g. Figure 4 One of the indicated resource types. For example, the CMS client may indicate that client-managed resources at an external location should be used by default for at least some types of tasks for an application. In at least some embodiments, a set of allowed non-default resource types 520 may be specified in a policy.
[0053] In the depicted embodiment, rules / conditions for selecting non-default resources 525 for a given task or container may be included in the policy. Such rules may be based on, for example, performance or capacity metrics (e.g., metrics of currently used resources and / or metrics of available resource pools that have been identified for the application), failure metrics, and / or any other kind of signal that the client wishes to utilize (e.g., changes in dynamic pricing of VCS resources). In some embodiments, rules / conditions for reverting to default resources 530 may be specified in the policy.
[0054] In some embodiments where a given containerized application includes several different types of tasks (which may be specified in an application definition or cluster definition sent by a CMS client to the CMS control plane), different policies may be used to select resources for different task types. Consider a scenario where an application includes two different types of tasks (Type A and Type B) that differ from one another in properties such as the kind of computing resources required (e.g., CPU vs. GPU vs. dedicated accelerators), the amount of computing resources required, the extent to which the tasks need to utilize data stored at the client's location, the sensitivity of the data being processed, and the like. A client may wish to use CMS-managed non-interruptible resources for type A tasks of an application as a non-default, and to use CMS-managed interruptible resources for type B tasks of an application. In such an embodiment, a resource selection policy may include a policy applicability setting 535 that specifies the specific task type to which the policy is to apply. In some embodiments, a given resource selection policy may indicate a resource selection policy for selecting a resource for a containerized application. Figure 4 Rules / conditions for displaying resources of any kind (and actually passing tasks between said resources).
[0055] Figure 6 Example program interactions related to policy-based resource selection for containerized applications are shown according to at least some embodiments. In the depicted embodiments, the features and functions of Figure 1 CMS 612, similar to CMS 125, may implement a set of programming interfaces 677. The programming interfaces may include, for example, one or more web-based consoles, command line tools, APIs, and / or graphical user interfaces. In various embodiments, the programming interfaces may be used by CMS clients 610 (such as owners / managers of containerized applications) to submit requests and messages related to using the above-described types of resource selection strategies, and receive corresponding responses.
[0056] In the depicted embodiment, client 610 may submit one or more RegisterContainers requests 614 via programming interface 677, providing information about various kinds of software containers to be used for one or more applications. The information may include, for example, an identifier of a machine image to be used for the container, a location where the machine image may be found (e.g., at a storage service of a provider network where the CMS is implemented), performance requirements required for the container (e.g., the number of virtual or physical CPUs with a specified computing power, the amount of memory, etc.), etc. In some embodiments, the information may be stored at a metadata repository (such as a container registry of the CMS), and a ContainersRegistered message 615 may be sent to the client.
[0057] In some embodiments, the CMS client may provide the specification of the containerized application via an ApplicationTasksAndServicesDefinition message 617. In this message, an indication of the tasks that make up the application, the CMS managed services that make up the application, the containers to be used for each task or service, and other information required to execute the containerized application may be provided. In some embodiments, the definitions of the tasks / services and the mapping between tasks and containers (or services and containers) may be stored at the CMS, and a DefsStored message 621 may be sent to the client.
[0058] In at least some embodiments, information about a set of resources at an external location to be used for one or more of the client's applications may be provided via a RegisterExternalResources message 623. Such a message may include, for example, the names and network addresses of physical and / or virtual machines to be used for the client's applications at the external location, tags indicating subcategories to which individual resources have been allocated by the client, etc. In one embodiment, the provided information may be stored at the CMS, and an ERsRegistered message 625 may be sent to the client.
[0059] In the depicted embodiment, the CMS client 610 may submit a representation of a resource selection policy of the above kind using a CreateResourceSelectionPolicy request 628. The policy may include various rules that may be used by the CMS control plane to select one or more kinds of resources for new or additional containers based on specified conditions, e.g., to replace default kinds of resources. In some embodiments, the policy may be stored in a repository of the CMS, and an RSPStored message 633 may be sent to the client. In some embodiments, the resource selection policy may be specified as part of an application definition (e.g., in an ApplicationTasksAndServicesDefinition message 617), rather than in a separate CreateResourceSelectionPolicy request.
[0060] In some embodiments, a StartAppWithResourceSelectionPolicy request 635 may be submitted by a CMS client, indicating that a specified containerized application should be started using a specified resource selection policy. Note that in some embodiments, a given resource selection policy may apply to multiple applications. In the depicted embodiment, an application may be started by the CMS, and an AppStarted message 637 may be sent to the client.
[0061] In one embodiment, a CMS client may change one or more of the parameters of a resource selection policy (e.g., after the policy has been stored and / or implementation of the policy has begun at the CMS). In such cases, a ModifyResourceSelectionPolicy request 641 may be sent by the client, indicating the changes to be made to the policy. For example, the client may add new conditions for selecting non-default resources, remove conditions for reverting to default resources, etc. The CMS may make the requested changes and send an RSP Modified response message 643 to the client.
[0062] In some embodiments, a ShowCurrentResourceMix request 645 may be submitted by the client to view the different kinds of (and computations of) resources currently being used for a specified containerized application. An indication of the different kinds of resources currently being used for the application may be provided by the CMS in a ResourceMix response message 647. In some embodiments, a visualization of the resource combination may be presented, e.g., showing how many containers / tasks are running at the client's premises, how many containers / tasks are running at the provider network, etc. In some embodiments, the reason for the decision to use a particular kind of resource may be provided in response to the ShowCurrentResourceMix request - e.g., the CMS may indicate that, as a result of analyzing performance metrics PM from the client premises, K containers were launched at provider network resources at time T.
[0063] The CMS client may submit a GetMetrics request 649 to view collected metrics for containerized applications run by the CMS on behalf of the client. In response, the CMS control plane may send one or more MetricsSets messages 651 indicating, for example, utilization levels of various resources (CPU, memory, storage, etc.) used by individual containers being run, network bandwidth utilization levels, message latency, application request response times, failure counts (e.g., failures of application requests), etc. It should be noted that in at least some embodiments, in addition to the Figure 6 In addition to those interactions shown, programmatic interactions related to the use of resource selection policies may also be supported by the VCS.
[0064] Figure 7An example scenario according to at least some embodiments is shown, in which a secure communication channel can be established between the control plane of a container management service and an agent deployed at a container execution resource. In the depicted embodiment, the CMS client wishes to run a corresponding CMS agent at each of the resources (servers or virtual machines) at which the tasks of the containerized application are run. For example, at an external location 750A, a CMS agent 744A runs at a virtual machine or server 752A, and a CMS agent 744B runs at a virtual machine or server 752B. Similarly, at another external location 750B, a CMS agent 744C runs at a virtual machine or server 752C, and a CMS agent 744D runs at a virtual machine or server 752D. In the depicted embodiment, each of the CMS agents can establish a secure connection (e.g., using a protocol similar to transport layer security or TLS) with a CMS control plane server 710 at a provider network data center 705 as a prerequisite for running a container at a corresponding virtual machine or server (such as a container 746A, 746B, 746C, or 746D). Arrows 771A, 771B, 771C and 771D represent network connections between CMS agents and CMS control planes. A CMS agent at a given server or virtual machine may send information about currently running containers 746 at this resource, resource utilization levels of servers or virtual machines, failed or unresponsive containers, etc. to the CMS control plane. The CMS agent may stop and start containers 746 at the request of the CMS control plane server. In at least one implementation, the CMS agent itself may run as a container. In at least some embodiments, the information provided by the CMS agent may be used to select resources for additional containers of the CMS client according to policies / rules specified by the client. In one embodiment, a monitoring service of a provider network may be used to collect various metrics and / or log messages from a virtual machine or server, for example, in addition to or in lieu of using a CMS agent to collect information, and the data gathered by the monitoring service may also be used to select resources for additional containers of applications of the CMS client.
[0065] In some embodiments, as part of preparing external resources for use by the CMS, a corresponding agent of the system management service (SMS) 715 may also be started at the resource. For example, SMS agents 745A, 745B, 745C, and 745D may be started at virtual machines or servers 752A, 752B, 752C, and 752D, respectively, and each SMS agent may establish a secure connection with the control plane server of the SMS at the provider network data center, as indicated by arrows 772A, 772B, 772C, and 772D. In some implementations, the SMS agent may run as a daemon or privileged process. The SMS agent enables the SMS service to update, manage, and configure the various subcomponents of the virtual machine or server. In some embodiments, the SSM agent may enable the client of the CMS to establish a communication session that may be used to interact with a program running in a container 746 at a given virtual machine or server. Note that in at least some embodiments, the CMS agent and the SMS agent may also run at a virtual machine or server for a containerized application within the provider network. In some embodiments, a given CMS agent and / or a given SMS agent may be used to manage containers at (and / or provide metrics from) several different virtual machines or servers.
[0066] Figure 8 801 is a flowchart illustrating aspects of operations that may be performed to manage cross-site resource selection for containerized applications according to at least some embodiments. As shown in element 801, an external resource set (ERS) may be identified at a control plane server of a CMS, the ERS comprising a provider network external to which a resource selection similar to a provider network is implemented. Figure 1 The ERS may be specified, for example, in a resource registration request submitted by a CMS client via a programming interface, or in response to other programmatic input from a client, indicating computing resources to be used for at least a portion of a containerized application App1. The CMS may (e.g., in response to a specific task request from a CMS client and / or according to a managed service specification of App1) begin running one or more containers for one or more tasks of App1 at the ERS.
[0067] In the depicted embodiment, various resource types (such as resource types) that indicate the selection of a container for running an application may be determined at the control plane of the CMS. Figure 4In some embodiments, the rules or policies may also be obtained from the CMS client via a programming interface. Such resource selection rules may indicate, for example, that ERS resources should be used for applications by default, but under specified workload surge conditions, resources of the provider network (e.g., resources at a provider network data center) may be used instead of or in addition to ERS resources.
[0068] In the depicted embodiment, a set of metrics for the application may be analyzed at the CMS control plane, including performance metrics, availability / failure metrics, and / or indications of changes to computing capabilities at the external site (element 807). The metrics may be obtained via any of a variety of sources, such as CMS agents installed at resources at the external site and / or at provider network resources, monitoring services of the provider network that have been permitted to capture metrics from ERS resources, and the like.
[0069] In various embodiments, in response to detecting (e.g., using analyzed metrics) that a condition indicated in a rule or policy has been satisfied, the CMS control plane may select a provider network resource (rather than an ERS resource) for launching a container for a task or service for an application (element 811). In the depicted embodiment, any of a variety of triggering events may result in a determination to launch a new (or replacement) container—e.g., a CMS client may submit a request to run an additional task for an application, a service definition for an application may require launching additional tasks based on an arrival rate of application requests, a failure of a previously launched container may require replacing the container, etc. Note that in at least some embodiments, a determination to launch a new container may be orthogonal to a decision as to what type of resource to use to run the new container.
[0070] In various embodiments, after a resource type has been selected based on a rule / policy, a specific resource of this type may be identified (e.g., from a dynamically allocated pool of such resources or a pre-allocated pool of such resources) and a container may be started at this resource (element 814). It should be noted that in various embodiments, Figure 8 Some of the operations shown may be implemented in an order different from that shown in the figure, or may be performed in parallel rather than sequentially. In addition, in one or more implementations, it may not be necessary to Figure 8 Some of the operations shown.
[0071] Fig. 9 Example scenarios are shown in which containerized application components may be executed at various availability zones and other architectural units, according to at least some embodiments. Fig. 9In the illustrated embodiment, a particular region 910 of a cloud provider network may include a pair of availability zones, AZ 921A and AZ 921B. Some software containers 961A of containerized applications managed by CMS may run within AZ 921A, while other software containers may run within AZ 921B. For some containerized applications such as cellular communications applications (including, for example, public or private 5G network applications), CMS-managed containers 961C may run at a third-party data center 930, such as a data center maintained by a telecommunications company. Other CMS-managed containers 961D may run in the local region 940. At a CMS client's data center 950, CMS-managed containers 961E may run at a provider network extension 952 (outpost), and / or CMS-managed containers 961F may run using a client-managed architecture 954. In various embodiments, a given containerized application may use Fig. 9 Resources at any combination of one or more of the different kinds of architectural units shown. In the depicted embodiment, a specific location and resource type to be used for a container for an application may be dynamically selected by the CMS using resource selection rules specified by a client on whose behalf the containerized application is running.
[0072] In at least some embodiments, a server implementing the types of techniques described herein (e.g., including CMS control plane functionality and other container management, analysis, and detection tasks) may include a general-purpose computer system that includes or is configured to access one or more computer-accessible media. Fig.10 Such a general purpose computing device 9000 is shown. In the illustrated embodiment, the computing device 9000 includes one or more processors 9010 coupled to a system memory 9020 (which may include both non-volatile memory modules and volatile memory modules) via an input / output (I / O) interface 9030. The computing device 9000 also includes a network interface 9040 coupled to the I / O interface 9030.
[0073] In various embodiments, the computing device 9000 may be a single processor system including one processor 9010 or a multiprocessor system including several processors 9010 (e.g., two, four, eight, or another suitable number). The processor 9010 may be any suitable processor capable of executing instructions. For example, in various embodiments, the processor 9010 may be a general-purpose or embedded processor that implements any of a variety of instruction set architectures (ISAs) (such as, x86, PowerPC, SPARC, ARM, or MIPS ISAs, or any other suitable ISAs). In a multiprocessor system, each of the processors 9010 may generally, but not necessarily, implement the same ISA. In some implementations, a graphics processing unit (GPU) and / or a field programmable gate array (FPGA) may be used as an alternative or in addition to a conventional processor.
[0074] The system memory 9020 may be configured to store program instructions and data accessible to the processor(s) 9010. In at least some embodiments, the system memory 9020 may include both a volatile portion and a non-volatile portion; in other embodiments, only volatile memory may be used. In various embodiments, the volatile portion of the system memory 9020 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM, or any other type of memory. For the non-volatile portion of the system memory (which may include, for example, one or more NVDIMMs), in some embodiments, flash-based memory devices may be used, including NAND flash devices. In at least some embodiments, the non-volatile portion of the system memory may include a power source, such as a supercapacitor or other power storage device (e.g., a battery). In various embodiments, memristor-based resistive random access memory (ReRAM), three-dimensional NAND technology, ferroelectric RAM, magnetoresistive RAM (MRAM), or any of various types of phase change memory (PCM) may be used at least for the non-volatile portion of the system memory. In the illustrated embodiment, program instructions and data implementing one or more desired functions, such as those methods, techniques, and data described above, are shown as stored within system memory 9020 as code 9025 and data 9026 .
[0075] In one embodiment, the I / O interface 9030 may be configured to coordinate I / O traffic between the processor 9010, the system memory 9020, and any peripheral devices in the device (including the network interface 9040 or other peripheral interfaces such as various types of persistent and / or volatile storage devices). In some embodiments, the I / O interface 9030 may perform any necessary protocol, timing, or other data transformations to convert data signals from one component (e.g., the system memory 9020) into a format suitable for use by another component (e.g., the processor 9010). In some embodiments, the I / O interface 9030 may include support for devices attached by: various types of peripheral buses (including various kinds of hardware accelerators), for example, such as variants of the peripheral component interconnect (PCI) bus standard or the universal serial bus (USB) standard. In some embodiments, the functionality of the I / O interface 9030 may be split into two or more separate components, such as, for example, a north bridge and a south bridge. Moreover, in some embodiments, some or all of the functionality of I / O interface 9030 (such as an interface to system memory 9020 ) may be incorporated directly into processor 9010 .
[0076] The network interface 9040 may be configured to allow the computing device 9000 to communicate with other devices 9060 (e.g., such as Figures 1 to 9 In various embodiments, the network interface 9040 may support communication via any suitable wired or wireless general-purpose data network (e.g., such as various types of Ethernet networks). In addition, the network interface 9040 may support communication via a telecommunications / telephone network (such as an analog voice network or a digital fiber-optic communication network), via a storage area network (such as a Fiber Channel SAN), or via any other suitable type of network and / or protocol.
[0077] In some embodiments, system memory 9020 may represent memory configured to store memory used to implement Figures 1 to 99000 . However, in other embodiments, program instructions and / or data may be received, sent, or stored on different types of computer-accessible media. In general, computer-accessible media may include non-transitory storage media or memory media, such as magnetic media or optical media, such as a disk or DVD / CD coupled to the computer device 9000 via an I / O interface 9030. Non-transitory computer-accessible storage media may also include any volatile or non-volatile media, such as RAM (e.g., SDRAM, DDR SDRAM, RDRAM, SRAM, etc.), ROM, etc., which may be included in some embodiments of the computing device 9000 as system memory 9020 or another type of memory. In some embodiments, multiple non-transitory computer-readable storage media may collectively store program instructions that, when executed on or across one or more processors, implement at least a subset of the methods and techniques described above. Computer accessible media may also include transmission media or signals such as electrical signals, electromagnetic signals, or digital signals communicated via a communication medium such as a network and / or wireless link, such as may be implemented via network interface 9040. In various embodiments, multiple computing devices such as Fig.10 Some or all of those shown may be used to implement the described functionality; for example, software components running on a variety of different devices and servers may cooperate to provide functionality. In some embodiments, portions of the described functionality may be implemented using storage devices, network devices, or special-purpose computer systems in addition to or instead of being implemented using a general-purpose computer system. As used herein, the term "computing device" refers to at least all of these types of devices, and is not limited to these types of devices.
[0078] Embodiments of the present disclosure may be described in terms of the following:
[0079] Clause 1. A system, comprising:
[0080] The control plane servers for the provider network’s container management service; and
[0081] A pool of computing resources in a provider network’s data centers;
[0082] The control plane server is configured as:
[0083] identifying a set of external resources to be used to run at least some tasks of the containerized application, wherein resources of the set of external resources are located at a location external to a provider network, and wherein individual ones of the tasks are performed using one or more software containers;
[0084] obtaining, via a programming interface, a burst mode resource selection rule for a containerized application, wherein the burst mode resource selection rule indicates a condition that a software container of a task of the containerized application is to be run at a resource at a data center of a provider network rather than using an external set of resources;
[0085] analyzing the set of metrics, wherein the set of metrics is collected at least in part from the set of external resources during a time interval in which a software container of a task of the containerized application is executed at the set of external resources; and
[0086] In response to detecting that the condition has been met based at least in part on the analysis of the set of metrics, a specific software container of a specific task of the containerized application is caused to run at a specific resource of the pool of computing resources.
[0087] Clause 2. The system of clause 1, wherein the control plane server is further configured to:
[0088] An indication is obtained that the containerized application is to be implemented at least in part as a managed service without obtaining container execution resources at a provider network on behalf of a client on which the containerized application is running, wherein specific resources are obtained by a container management service to execute a specific software container.
[0089] Clause 3. A system according to any of clauses 1 to 2, wherein the set of metrics indicates one or more of: (a) resource utilization level, (b) latency, (c) application request arrival rate, or (d) available resource capacity of an external set of resources.
[0090] Clause 4. The system of any one of clauses 1 to 3, wherein the control plane server is further configured to:
[0091] In response to a programmatic request, provide an indication of one or more of: (a) one or more software containers of a containerized application running at an external set of resources during a specific time interval, (b) one or more software containers of a containerized application running at a data center of a provider network during a specific time interval, or (c) results in a measurement of use of provider network resources by the containerized application during the specific time interval.
[0092] Clause 5. The system of any one of clauses 1 to 4, wherein the control plane server is further configured to:
[0093] Providing connection information to a load balancer associated with the application so that the load balancer can forward application requests to specific resources, wherein the load balancer operates at a location external to the provider network.
[0094] Clause 6. A computer-implemented method comprising:
[0095] identifying, at a container management service of the provider network, a set of external resources to be used to run at least some software containers of the application, wherein resources of the set of external resources are located at a location external to the provider network;
[0096] Enable the application's software container to run on resources in an external resource set through container management services;
[0097] determining, by the container management service, conditions under which a software container of the application is to be run at a resource at a data center of the provider network; and
[0098] In response to detecting that the condition has been met, a specific software container for a specific task of the application is caused to run at a specific resource at a data center of the provider network by the container management and control service.
[0099] Clause 7. The computer-implemented method of clause 6, wherein the application is executed on behalf of a client of the provider network, and wherein the particular resource comprises a computing instance launched at a virtualized computing service of the provider network at the request of the client.
[0100] Clause 8. The computer-implemented method of clause 6, further comprising:
[0101] An indication is obtained from the client at the container management service that the application is to be implemented at least in part as a managed service without requiring the client to obtain container execution resources at the provider network, wherein specific resources are selected by the container management service to execute a specific software container.
[0102] Clause 9. The computer-implemented method of any one of clauses 6 to 7, further comprising:
[0103] A set of performance metrics of the application is obtained at the container management service, including at least one performance metric collected at the set of external resources, wherein detecting that the condition has been met is based at least in part on the set of performance metrics.
[0104] Clause 10. A computer-implemented method as described in Clause 9, wherein the performance metrics collected at the external resource set indicate one or more of: (a) resource utilization level, (b) latency, (c) application request arrival rate, or (d) available resource capacity of the external resource set.
[0105] Clause 11. A computer-implemented method according to any one of clauses 6 to 7 or 9, the method further comprising:
[0106] A set of availability metrics for the application is obtained at the container management service, including at least one availability metric collected at the set of external resources, wherein detecting that the condition has been met is based at least in part on the set of availability metrics.
[0107] Clause 12. A computer-implemented method according to any one of clauses 6 to 7, 9 or 11, the method further comprising:
[0108] An indication is provided by a container management service in response to a programmatic request of one or more of: (a) one or more software containers of an application running at an external set of resources during a specific time interval, (b) one or more software containers of an application running at a data center of a provider network during a specific time interval, or (c) conditions that result in use of provider network resources by an application during a specific time interval.
[0109] Clause 13. A computer-implemented method according to any one of clauses 6 to 7, 9 or 11 to 12, the method further comprising:
[0110] After the particular software container has been launched at the particular resource at the data center of the provider network, obtaining at the container management service an indication of one or more resources that have been added to the set of external resources at a location external to the provider network; and
[0111] A resource from among the one or more resources that have been added to the external resource set is selected by the container management service to run another software container of the application.
[0112] Clause 14. The computer-implemented method of any of clauses 6-7, 9, or 11-13, wherein identifying the set of external resources is based at least in part on input received via a programming interface of the container management service.
[0113] Clause 15. The computer-implemented method of any of clauses 6-7, 9, or 11-14, wherein determining the condition is based at least in part on input received via a programming interface of a container management service.
[0114] Clause 16. A non-transitory computer-accessible storage medium storing program instructions that, when executed on a processor:
[0115] causing, by a container management service of the provider network, at least some software containers of the application to run at a set of external resources identified for the application, wherein resources of the set of external resources are located at a location external to the provider network;
[0116] determining, through the container management service, conditions under which the software container of the application is to be run at resources at a data center of the provider network; and
[0117] In response to detecting that the condition has been met, a specific software container for a specific task of the application is caused to run at a specific resource at a data center of the provider network by the container management and control service.
[0118] Clause 17. The non-transitory computer-accessible storage medium of clause 16, wherein the application is executed on behalf of a client of a provider network, and wherein the particular resource comprises a computing instance launched at a virtualized computing service of the provider network at the request of the client.
[0119] Clause 18. The non-transitory computer-accessible storage medium of Clause 16 storing further program instructions that, when executed on a processor:
[0120] An indication is obtained from the client at the container management service that the application is to be implemented at least in part as a managed service without requiring the client to obtain container execution resources at the provider network, wherein specific resources are selected by the container management service to execute a specific software container.
[0121] Clause 19. A non-transitory computer accessible storage medium according to any one of clauses 16 to 17 storing further program instructions which, when executed on a processor:
[0122] A set of performance metrics of the application is obtained at the container management service, including at least one performance metric collected at the set of external resources, wherein detecting that the condition has been met is based at least in part on the set of performance metrics.
[0123] Clause 20. A non-transitory computer accessible storage medium according to any one of clauses 16 to 17 or 19 storing further program instructions which, when executed on a processor:
[0124] Caused by a container management service in response to a programmatic request, an indication of one or more of: (a) one or more software containers of an application running at an external set of resources during a specific time interval, (b) one or more software containers of an application running at a data center of a provider network during a specific time interval, or (c) resulting in a measurement of use of provider network resources by the application during the specific time interval.
[0125] in conclusion
[0126] Various embodiments may also include receiving, sending, or storing instructions and / or data implemented according to the foregoing description on a computer-accessible medium. In general, a computer-accessible medium may include: storage media or memory media such as magnetic media or optical media, such as disks or DVD / CD-ROMs; volatile or non-volatile media such as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc.; and transmission media or signals (such as electrical signals, electromagnetic signals, or digital signals) that are communicated via communication media such as networks and / or wireless links.
[0127] The various methods shown in the figures and described herein represent exemplary embodiments of the methods. The methods may be implemented in software, hardware, or a combination thereof. The order of the methods may be changed, and various elements may be added, rearranged, combined, omitted, modified, etc.
[0128] It will be apparent to those skilled in the art having the benefit of this disclosure that various modifications and changes may be made. It is intended to cover all such modifications and changes, and the above description is therefore to be regarded in an illustrative rather than a limiting sense.
Claims
1. A system, comprising: The control plane server for the provider network's container management service; and A computing resource pool of a data center of the provider network; The control plane server is configured as follows: identifying a set of external resources to be used to run at least some tasks of the containerized application, wherein resources of the set of external resources are located at a location external to the provider network, and wherein individual ones of the tasks are performed using one or more software containers; obtaining, via a programming interface, a burst mode resource selection rule for the containerized application, wherein the burst mode resource selection rule indicates a condition that a software container of a task of the containerized application is to be run at resources at a data center of the provider network rather than using the set of external resources; analyzing a set of metrics, wherein the set of metrics is collected at least in part from the set of external resources during a time interval in which a software container of a task of the containerized application is executed at the set of external resources; and In response to detecting that the condition has been satisfied based at least in part on the analysis of the set of metrics, a specific software container for a specific task of the containerized application is caused to run at a specific resource of the pool of computing resources.
2. The system of claim 1, wherein the control plane server is further configured to: Obtaining an indication that the containerized application is to be implemented at least in part as a managed service without obtaining container execution resources at the provider network on behalf of a client on which the containerized application is running, wherein the specific resources are obtained by the container management service to execute the specific software container.
3. A system according to any one of claims 1 to 2, wherein the set of metrics indicates one or more of the following: (A) resource utilization level, (b) latency, (c) application request arrival rate, or (d) available resource capacity of the set of external resources.
4. The system according to any one of claims 1 to 3, wherein the control plane server is further configured to: In response to a programmatic request, provide an indication of one or more of: (a) one or more software containers of the containerized application running at the set of external resources during a specific time interval, (b) one or more software containers of the containerized application running at the data center of the provider network during a specific time interval, or (c) a measurement of use of provider network resources by the containerized application during a specific time interval.
5. The system according to any one of claims 1 to 4, wherein the control plane server is further configured to: The connection information is provided to a load balancer associated with the application so that the load balancer can forward the application request to the specific resource, wherein the load balancer operates at the location external to the provider network.
6. A computer-implemented method comprising: identifying, at a container management service of a provider network, a set of external resources to be used to run at least some software containers of an application program, wherein resources of the set of external resources are located at a location external to the provider network; Enable the software container of the application to run at the resources of the external resource set through the container management and control service; determining, by the container management service, conditions under which a software container of the application is to be run at a resource at a data center of the provider network; as well as In response to detecting that the condition has been met, the container management service causes a specific software container of a specific task of the application to run on a specific resource in a data center of the provider network.
7. The computer-implemented method of claim 6, wherein the application is executed on behalf of a client of the provider network, and wherein the specific resource comprises a computing instance launched at a virtualized computing service of the provider network at the request of the client.
8. The computer-implemented method of any one of claims 6 to 7, further comprising: A set of performance metrics of the application is obtained at the container management service, including at least one performance metric collected at the set of external resources, wherein detecting that the condition has been met is based at least in part on the set of performance metrics.
9. The computer-implemented method of any one of claims 6 to 8, further comprising: Obtaining, at the container management service, a set of availability metrics for the application, including at least one availability metric collected at the set of external resources, wherein detecting that the condition has been met is based at least in part on the set of availability metrics.
10. The computer-implemented method of any one of claims 6 to 9, further comprising: The container management service provides, in response to a programmatic request, an indication of one or more of: (a) one or more software containers of the application running at the set of external resources during a specific time interval, (b) one or more software containers of the application running at the data center of the provider network during the specific time interval, or (c) conditions that result in use of provider network resources by the application during the specific time interval.
11. The computer-implemented method of any one of claims 6 to 10, further comprising: After the particular software container has been launched at the particular resource at the data center of the provider network, obtaining, at the container management service, an indication of one or more resources that have been added to the set of external resources at the location outside the provider network; as well as The container management service selects a resource from the one or more resources that have been added to the external resource set to run another software container of the application.
12. The computer-implemented method of any one of claims 6 to 11, wherein identifying the set of external resources is based at least in part on input received via a programming interface of the container management service.
13. The computer-implemented method of any one of claims 6 to 12, wherein determining the condition is based at least in part on input received via a programming interface of the container management service.
14. A non-transitory computer accessible storage medium storing program instructions which, when executed on a processor: causing, by a container management service of a provider network, at least some software containers of an application to run at a set of external resources identified for the application, wherein resources of the set of external resources are located at a location external to the provider network; determining, by the container management service, conditions under which the software container of the application is to be run on resources at a data center of the provider network; and In response to detecting that the condition has been met, the container management service causes a specific software container of a specific task of the application to run on a specific resource in a data center of the provider network.
15. The non-transitory computer accessible storage medium of claim 14 storing further program instructions which, when executed on the processor: The container management service causes, in response to a programmatic request, an indication of one or more of: (a) one or more software containers of the application running at the set of external resources during a specific time interval, (b) one or more software containers of the application running at the data center of the provider network during a specific time interval, or (c) a measurement of the application's use of provider network resources during the specific time interval.
