Providing tenant isolation in a multi-tenant API gateway with microservice containers

Abstract

The present disclosure relates to providing tenant isolation in a multi-tenant API gateway with microservice containers. A system can host APIs for a number of different tenants and receive requests from many different client devices. When a request is received, the associated tenant can be identified and a router can determine whether a container instance is available to service the request. The container instance can be a blank container instance that includes an internal endpoint, a web server, and a runtime environment. The blank container instance can be agnostic to a particular tenant. To associate the container instance with a tenant, a data store, such as a key-value data store, can retrieve configuration files that transform the agnostic container instance into a container instance associated with a particular tenant and include configuration code to execute the requisite API functions. A pool of blank containers and filled containers can be efficiently managed.

Description

[0001] This application is a divisional application of the invention patent application filed on September 28, 2018, with application number 201880068977.3 and entitled "Providing Tenant Isolation in a Multi-Tenant API Gateway Using Microservice Containers". Technical Field

[0002] This disclosure relates to managing API gateways, and more specifically to providing tenant isolation in multi-tenant API gateways using microservice containers. Background Technology

[0003] An Application Programming Interface (API) gateway is a fully managed service that allows developers to create, publish, maintain, monitor, and secure APIs for public use. Tenants can define APIs and publish them to these public API gateways, such as cloud platforms, and then make them available to client devices. In large-scale API gateways, the environment can include a multi-tenant environment, where multiple different tenants host APIs that are available through the same hosting service. Summary of the Invention

[0004] In some embodiments, a method of using containers to isolate tenants to serve requests in a multi-tenant environment may include: receiving a first request for a first service provided by a first tenant; selecting an empty container in the multi-tenant environment; loading a first configuration implementing the first service into the container; serving the first request from the container; receiving a second request for a second service provided by a second tenant; flushing the first configuration from the container; and serving the second request from the container.

[0005] In some embodiments, a non-transitory computer-readable medium includes instructions that, when executed by one or more processors, cause one or more processors to perform operations, including: receiving a first request for a first service provided by a first tenant; selecting an empty container in a multi-tenant environment; loading a first configuration implementing the first service into the container; serving the first request from the container; receiving a second request for a second service provided by a second tenant; flushing the first configuration from the container; and serving the second request from the container.

[0006] In some embodiments, a system may include one or more processors and one or more memory devices, the memory devices including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including: receiving a first request for a first service provided by a first tenant; selecting an empty container in a multi-tenant environment; loading a first configuration implementing the first service into the container; serving the first request from the container; receiving a second request for a second service provided by a second tenant; flushing the first configuration from the container; and serving the second request from the container.

[0007] In any embodiment, any one or all of the following features may be included in any combination and are not limited thereto. The container may be one of multiple containers in a multi-tenant environment instantiated to serve requests from client devices. The first configuration may include the size of the heap in memory that the first service can use. After flushing the first configuration from the container, the container may include a runtime process with an embedded server and an internal endpoint. The internal endpoint may be invoked by a router in the multi-tenant environment to serve a second request. The first configuration may include multiple actions chained together to serve the request. The multi-tenant environment may prevent the container from simultaneously serving requests associated with different tenants. The multi-tenant environment may allow the container to simultaneously serve requests associated with a single tenant. The method / operation may also include: receiving a third request for a second service provided by a second tenant; and serving the third request from the container without flushing the second configuration from the container. The first service may include a public API made available through the multi-tenant environment. Attached Figure Description

[0008] A further understanding of the nature and advantages of the invention can be achieved by referring to the remainder of this specification and the accompanying drawings, in which similar reference numerals are used throughout the drawings to refer to similar parts. In some cases, sub-labels are associated with reference numerals to indicate one of a plurality of similar parts. When reference is made to reference numerals without specifying existing sub-labels, it is intended to refer to all such plurality of similar parts.

[0009] Figure 1 A simplified block diagram of a system for processing requests in a multi-tenant environment, according to some embodiments, is shown.

[0010] Figure 2 The illustration shows a schematic diagram of a container instance 108 according to some embodiments, which is prepared to serve tenant requests.

[0011] Figure 3 The diagram illustrates a block diagram showing how an empty container instance is populated with files from a data repository, according to some embodiments.

[0012] Figure 4 The illustration shows how a filled container can serve requests according to some embodiments.

[0013] Figure 5 The diagram illustrates a simplified block diagram of a container being returned to a pool of available containers according to some embodiments.

[0014] Figure 6A The diagram illustrates a state diagram of the lifecycle of a container in a multi-tenant environment according to some embodiments.

[0015] Figure 6BThe diagram illustrates an alternative state diagram of the container lifecycle in a multi-tenant environment according to some embodiments.

[0016] Figure 7 The diagram illustrates a method for using containers to isolate tenants in a multi-tenant environment to serve requests.

[0017] Figure 8 The diagram illustrates a flowchart of a method for managing the runtime state and service configuration of containers in a multi-tenant environment, according to some embodiments.

[0018] Figure 9 The diagram illustrates a flowchart of a method for efficiently allocating container pools to serve requests in a multi-tenant environment, according to some embodiments.

[0019] Figure 10 A simplified block diagram is shown for implementing some embodiments of a distributed system.

[0020] Figure 11 The diagram illustrates a simplified block diagram of the components of a system environment through which services provided by the components of the embodiment system can be offered as cloud services.

[0021] Figure 12 An exemplary computer system in which various embodiments can be implemented is illustrated. Detailed Implementation

[0022] This document describes an implementation of an API gateway for management. A cloud system can host APIs for multiple different tenants and receive requests from many different client devices. When a request is received, the associated tenant can be identified, and the router can determine whether a container instance is available to serve the request. A container instance can be an empty container instance that includes internal endpoints, a web server, and a runtime environment. An empty container instance may not be associated with a specific tenant. To associate a container instance with a tenant, a data repository (such as a key-value data repository) can retrieve configuration files that transform an agnostic container instance into a container instance associated with a specific tenant and include configuration code for executing the necessary API functions. When the API in the container completes a service request (or multiple requests from a single tenant), runtime state information can be saved back to the data repository, and the contents of the container instance can be flushed. This ensures isolation between tenant data in a multi-tenant environment while still allowing for container reuse and efficient management of pooled resources. The data repository can maintain a service registry so that routers in the system can allocate new container instances when needed and deallocate them when no longer in use. The data repository can also store runtime state information, configuration, and applications for tenant APIs that can be distributed on demand to any container.

[0023] Figure 1A simplified block diagram of a system for processing requests in a multi-tenant environment, according to some embodiments, is illustrated. Examples of API gateways can be used throughout this disclosure. However, the embodiments described herein are not limited thereto. Alternatively, the functionality described for processing requests to an API gateway can be implemented in any system having an interface for receiving requests.

[0024] Multiple client devices 102 (e.g., smartphones, laptops, tablets, workstations, servers, etc.) can send requests to a public interface 104, which may include a Load Balancer as a Service (LBaaS) interface. These requests can be associated with a specific tenant in a multi-tenant environment and can reference specific services provided by that tenant, such as API functions. For example, a request received from one of the client devices 102 may include information such as “Tenant:ACME, API:ProcessInvoice”. The public interface 104 can centralize requests to one or more routers 106.

[0025] Router 106 can communicate periodically and in real-time with data repository 112 to identify one or more containers assigned to tenants and capable of handling specific requests. Figure 1 In the example, a request for tenant 2 will have three different container instances that router 106 can choose from. Among the set of container instances assigned to tenant 2, router 106 can select one or more container instances loaded with the configuration and / or application for the service request. For example, router 106 can select one of the three container instances assigned to "Tenant:ACME" loaded with the configuration for handling "API:ProcessInvoice". If no instance is already loaded with that configuration, router 106 can select an existing empty container or instantiate a new empty container. Router 106 can then cause data repository 112 to transfer the "API:ProcessInvoice"-specific configuration to the existing / new empty container to create a containerized service that can handle the specific request for "Tenant:ACME".

[0026] Once the containerized service has processed the request and (if necessary) returned a response to the client device, it can either be reused to serve additional requests for that particular tenant / service combination, or its contents can be flushed from the container to make it available to other tenants. When a container is flushed and reassigned to a different tenant, the runtime information that needs to be stored in the container can be sent back to the data repository 112. This ensures that the next time a particular application or API is instantiated in an empty container, the runtime information can be delivered and used by the new container to continue execution from where it left off.

[0027] Registration service 114 can be used to register new tenants to a multi-tenant environment. New tenants can use the interface of registration service 114 to generate new tenant entries in data repository 112 and define parameters for their use of the multi-tenant environment. These parameters can include the maximum, minimum, average, and expected number of container instances available to that particular tenant. Furthermore, new tenants can select the number and type of routers used for service requests, different configuration services available, different configurations of runtime container pools, and other aspects of the registration process.

[0028] Configuration service 110 can provide a web-based user interface that allows tenants to define configurations and select applications that can be made available through a public interface. For example, some embodiments may provide a series of web forms and dropdowns that allow tenants to select from multiple different microservices, applications, functions, etc., to build an API. Some embodiments may also allow tenants to select applications that can run, such as applications that limit the number of times the API can be called within a predetermined timeframe. After selecting / defining configurations and applications through configuration service 110, they can be stored in data center 112 for placement in empty containers within the system.

[0029] Note that a given gateway can serve more than one tenant simultaneously. In some embodiments, multiple gateways may exist that are completely isolated and run in different availability domains or data centers. For example, there might be a gateway for Chicago and a gateway for Phoenix. However, these two gateways may still serve the same tenant. End users can still deploy APIs to one or any combination of multiple gateways using a unique gateway ID. In some embodiments, each gateway may have its own unique key-value data repository for storing configurations, applications, etc.

[0030] The following diagrams illustrate an example of how a traversal system can handle requests. These examples are not intended to be limiting. Figure 2The illustration depicts a container instance 108 according to some embodiments, which is preparing to serve a tenant's request. At this stage, router 106 has received the request to be served. First, router 106 may work in conjunction with data repository 112 to determine if any available container instances are operational and available to serve the request. Data repository 112 may include service registry 214, which catalogs each available instance in the system. Service registry 214 may also track which APIs and tenants are assigned to each instantiated container. Data repository 112 may update service registry 214 in real time when a new container is instantiated or an existing container is removed. Router 106 may maintain a local copy of at least a portion of service registry 214. Data repository 112 may periodically update router 106 with a list of changes to service registry 214. In some embodiments, data repository 112 may update router 106 with a list of available container instances that can receive the request. In some embodiments, because the total list of instances in the system may be very large, only changes to service registry 214 or a portion thereof are transmitted to router 106.

[0031] like Figure 1 As shown, some embodiments may include multiple routers 106. One of the routers 106-2 may be designated as the master router. The master router 106-2 may be designated as the only router among the routers 106 that is allowed to instantiate new container instances when needed. Therefore, if Figure 2 Router 106 is designated as the master router, so it is free to instantiate new container instances when it needs to serve a request for a specific tenant. Master router 106-2 can analyze the settings provided by the tenant during registration to ensure that the number of container instances allocated to the tenant in the pool is within the minimum / maximum number for that specific tenant. If one of routers 106 is processing a request to instantiate a new container, router 106-1 can forward the request to master router 106-2, and master router 106-2 can determine whether to instantiate the new container. By allowing only master router 106-2 to instantiate new containers, this helps ensure efficient management of the container pool size. For example, if six different routers receive simultaneous requests for a specific service from a specific tenant, each of these routers might independently decide to instantiate a new container, even if a single instantiation of a container might be sufficient. Centralizing each of these requests to master router 106-2 ensures that only the minimum number of new containers are instantiated.

[0032] Conversely, if router 106 and / or data repository 112 determine that a tenant has been allocated a large number of unused container instances, or that a large number of unallocated containers are available in the container pool, then it can be determined that one or more of these unused containers can be eliminated from the system, thereby freeing up memory and processing power to enable existing containers to operate. This provides a dynamic pool of container instances that can serve requests. This pool can dynamically grow / shrink based on how much request traffic is received at any given time. As described below, flushing container contents allows containers to be reused across different tenants while still ensuring that tenants are isolated from each other.

[0033] One technical advantage achieved through these embodiments is the efficient management of container pool size. A runtime pool can be a collection of all runtime instances sharing common attributes. Runtime instances in a container pool can be sequentially numbered starting from index 0, such that a given runtime instance is uniquely identified by a pair of values: {pool name, instance index}. Pool management can depend on the specific container environment used by each embodiment. For example, pool functionality can be implemented using a Java class via the Docker daemon REST API to create / start / stop runtime containers as needed. This class can allow the environment to set the pool name, port(s), debug port(s), timeout values ​​for container state transitions (described in detail below), and so on. Some embodiments may also allow the class to specify a “minSize” value, which represents the number of running container instances not bound to any tenant that can be used for service requests. Furthermore, a “maxSize” value can specify the maximum number of runtime container instances in the container pool.

[0034] Once a service is assigned to a container in the gateway, it can perform periodic "heartbeats" as an indication to other services that it is still alive and functioning correctly. For example, when a service is loaded into a container, it can perform a heartbeat to let one or more routers know that it is available for service requests. Performing a heartbeat may include updating the corresponding entry in the service registry of data repository 112. These heartbeat transmissions can be used to determine the lifecycle of the container in the gateway. For example, some embodiments may use a time-to-live (TTL) interval, after which the service can be considered inactive. Alternatively, some embodiments may specify the number of heartbeats that can be skipped / missed before the service is considered inactive. Heartbeats can occur periodically, such as every 10 seconds, 20 seconds, 30 seconds, 60 seconds, etc.

[0035] exist Figure 2In the example, router 106 can identify empty container 108-3 that is not currently assigned to a specific tenant, or is assigned to a requesting tenant but is not populated with the configuration to run the specific API requested. Note that containers 108-1 and 108-2 are populated with configuration and assigned to specific tenants. Because they operate in separate containers from container 108-3, the data and operations of these three containers can be strictly isolated from each other.

[0036] Figure 3 The diagram illustrates a block diagram showing how an empty container instance is populated with files from data repository 112, according to some embodiments. (Continue) Figure 2 For example, an empty container 108-3 can be allocated to handle request 304 for a specific tenant. First, container 108-3 can be assigned to that tenant so that requests from other tenants cannot be served through container 108-3. Next, the tenant and API information from request 304 can be used to look up configuration, application, and runtime information in data repository 112. In some embodiments, data repository 112 can be a key-value data repository. Some embodiments may also allow data repository 112 to be distributed across different systems in a multi-tenant environment or across different platforms. For example, the tenant and / or API from request 304 can be used as a key to look up values ​​in data repository 112 that return configuration 202, application 204, and / or any runtime state information.

[0037] In some embodiments, the key-value data repository 112 can be used to persist tenant-specific configurations to disk. Furthermore, the key-value data repository can provide a central service registry 214, allowing all running microservices to "register" themselves so that other services can locate and invoke them. For example, such as The distributed key-value data repository can be used to store state information such as: configurations obtained from the management service (e.g., policies, APIs, applications, schedule metadata, etc.); runtime container status in the service registry 214; container pool configurations (e.g., minimum, maximum, policies, timeout intervals, etc.); tenant registration status (e.g., tenants, tenant pool bindings, etc.); rate limiting configurations, real-time status, etc.

[0038] Configuration 202 can include existing configurations. Figure 1The configuration 202 is a pipeline of actions 206 defined in the configuration service 110. These actions 206 can be chained together to form an API or other service for processing data. Each action 206 may include things such as: receiving a request, parsing the payload in the request, processing the data in the payload, modifying the data in the payload, calling another service to obtain information, writing the information to a file or database, and / or generating a response. In some embodiments, the configuration 202 can handle multiple requests simultaneously. Therefore, a single container populated with the configuration 202 can handle multiple requests for that particular API and that particular tenant. In general, the configuration is stateless (e.g., RESTful) and highly efficient in processing information and generating responses quickly.

[0039] In addition to returning configuration 202, data repository 112 can also return one or more applications 204, which can also run in a container along with API configuration 202. For example, application 204 could be a bandwidth-limited application that restricts the number of times a specific API can be called within a predetermined time interval (e.g., only 100 requests can be served per hour). Compared to configuration 202, application 204 may need to save runtime state information between applications' executions. Figure 3 In this context, application 204 may not have been run before, so there's a possibility that runtime state information for application 204 isn't saved in data repository 112. Container 108-3 can now represent a microservice assigned to a single tenant to handle specific API requests. Once container 108-3 is bound to a tenant, the instance can load all new configuration information, such as API definitions, applications, plans, and subscriptions, from data repository 112 on demand.

[0040] An empty container can be a software container, such as... Containers, and multi-tenant environments can include orchestrated container platforms, such as Instantiating a new empty container can include generating an empty container from a container image and populating it with a minimum number of software processes, which will be common to any configuration used in the system. For example, some embodiments may specify an empty container as... A container that includes a runtime environment, such as the Java Runtime. (JRE), web server 210 (such as an HTTP server), and internal endpoint 208. Internal endpoint 208 can be exposed to router 106 and can be used by router 106 to send requests to container 108-3. In some embodiments, other than Other container environments besides containers. For example, some implementations use UNIX processes to start / stop runtime containers.

[0041] When an empty container is populated, data repository 112 can transfer configuration 202 and application 204 to container 108-3. If runtime state data 302 is available in data repository 112, it will also be transferred to container 108-3 in this case. Runtime state data 302 is generated by the application during application runtime and stored in container 108-3. For example, application 204 can record the number of requests received within a given time interval. This information can be stored in runtime state 302 and transferred back to data repository 112 when container 108-3 is flushed. Generally, transferring configuration 202, application 204, and / or runtime state 302 to an empty but instantiated container 108-3 is a relatively lightweight process that can be completed very quickly and efficiently to process requests without noticeable delays.

[0042] Figure 4 The illustration shows how a populated container 108-3 can serve requests according to some embodiments. Container 108-3 can serve any request for that tenant to the API defined by configuration 202. In some cases, this may include processing only a single request that caused configuration 202 to be sent to container 108-3. In other cases, this may include processing multiple similar requests sent to router 106 for the same tenant. After all requests have been processed and responses (if any) have been sent back to the requesting client device, container 108-3 may become idle or passive. Although container 108-3 is still assigned or bound to a specific tenant, it is not currently being used to process any requests. After a predetermined time interval, container 108-3 may be deassigned to that specific tenant and returned to the pool of available containers 108-3, awaiting assignment to a new tenant with a new configuration.

[0043] Before flushing container 108-3 and reassigning it to a different tenant, any runtime state information 302 generated or updated by application 204 running on container 108-3 can be stored in data repository 112. Then, when configuration 202 and / or application 204 are reassigned to a new container to serve future requests, the runtime state information 302 can be transferred to the different container.

[0044] Figure 5 A simplified block diagram illustrating container 108-3 being returned to the pool of available containers according to some embodiments is shown. After runtime state information 302 is transmitted back to the data repository 112, the configuration 202, application 204, and / or runtime state 302 of container 108-3 can be flushed. Router 106 can then reassign the empty container 108-3 to different tenants to serve different API calls. While in Figure 5Only container 108-3 is shown as empty, but actual deployments typically include multiple empty containers. Router 106 can use different strategies to assign requests to one of the multiple available empty containers, such as a round-robin strategy. As mentioned above, if container 108-3 is not assigned to a new tenant / API within a predetermined time interval, then container 108-3 can be removed from the multi-tenant environment to save memory and / or compute resources.

[0045] As described above, data repository 112 facilitates these operations by assigning configuration, application, and runtime state to various containers running in a multi-tenant environment. Data repository 112 can also receive configuration information from tenants during initial registration and even at runtime. Data repository 112 also maintains service registry 214, which continuously monitors the status of the container pool. Data repository 112 uses service registry 214 to communicate with router 106 to determine when the available container pool should grow and / or shrink.

[0046] Figure 6A The diagram illustrates a state diagram of the container lifecycle in a multi-tenant environment according to some embodiments. In the initial state 602, the container does not exist. In state 604, the container has been instantiated with the aforementioned processing set (e.g., web server, endpoint, etc.), but the container is not bound to or assigned to a specific tenant and is empty. Upon service request, the container can enter state 606, in which it is bound to or assigned to a specific tenant and populated with configuration, application, and / or runtime state information. State 606 is referred to as the active state because the container may be actively serving requests received from router 106. In state 608, the container may still be bound to or assigned to a specific tenant, but it is passive because it is not actively processing any requests using its internally stored configuration. After a predetermined idle time interval, the container can be unassigned or unbound in state 610. When the container is no longer bound to a specific tenant, its internal configuration, application, and / or runtime state can be flushed. In some embodiments, an unbound container does not need to flush its internal contents until it is reassigned to a new tenant. If the container is not assigned to a new tenant, then the container can be removed in state 612.

[0047] Figure 6B The illustrations depict some embodiments. Figure 6AAnother view of the state diagram illustrates the lifecycle of a container in a multi-tenant environment. In the initial state 602, the container either does not exist or has been removed from the container environment. After a container instance is created, it can be unbound in state 604. If a container is idle for a predetermined time interval (called the "unbound timeout"), it can be removed from the environment, returning to state 602. Alternatively, in state 606, the container can be assigned to a tenant and its configuration, application, state information, etc., can be loaded. From the bound state and active state 606, if the router shrinks the container pool, the container can be removed from the environment, thus transitioning back to state 602. Containers can also be released from a tenant, flushed of their tenant-specific content, and returned to the unbound container pool in state 604. Furthermore, if a bound active container in state 606 is idle without serving any client requests for a predetermined time interval known as the "idle tenant timeout," it can become passive in state 608. Starting from the bound and passive state 608, the "passive timeout" interval may expire, causing the container to transition from the bound and passive container state 608 to the unbound container pool in state 604. The container's lifecycle can transition between states as long as the container exists.

[0048] Figure 7 A flowchart illustrating a method for isolating tenants to serve requests using containers in a multi-tenant environment is illustrated. The method may include receiving a first request for a first service provided by a first tenant (702). The method may also include selecting an empty container in the multi-tenant environment (704). The method may additionally include loading a first configuration implementing the first service into the container (706). The method may further include serving the first request from the container (708). The method may further include receiving a second request for a second service provided by a second tenant (710). The method may additionally include flushing the first configuration from the container (712). The method may further include serving the second request from the container (714). Some embodiments may include a system including one or more processors and one or more memories for performing these method steps. Other embodiments may include a non-transitory computer-readable medium storing instructions that cause one or more processors to perform these method steps.

[0049] In any embodiment, one or more of the following features may be included in any combination and are not limited. The container may be one of multiple containers in a multi-tenant environment instantiated to serve requests from client devices. After flushing a first configuration from the container, the container may include a runtime process with an embedded server and an internal endpoint. The internal endpoint may be invoked by a router in the multi-tenant environment to serve a second request. The first configuration may include multiple actions chained together to serve the request. The multi-tenant environment may prevent the container from simultaneously serving requests associated with different tenants. The multi-tenant environment may allow the container to simultaneously serve requests associated with a single tenant. The method may further include: receiving a third request for a second service provided by a second tenant; and serving the third request from the container without flushing the second configuration from the container. The first service may include a public API made available through the multi-tenant environment.

[0050] It should be recognized that, Figure 7 The specific steps shown provide specific methods for using containers to isolate tenants to serve requests in a multi-tenant environment, according to various embodiments of the invention. According to alternative embodiments, other sequences of steps may also be performed. For example, alternative embodiments of the invention may perform the steps outlined above in a different order. Furthermore, Figure 7 The steps shown may include multiple sub-steps, which can be performed in various orders suitable for the individual steps. Furthermore, depending on the specific application, additional steps may be added or removed. Many variations, modifications, and substitutions will be recognized by those skilled in the art.

[0051] Figure 8 A flowchart illustrating a method for managing the runtime state and service configuration of containers in a multi-tenant environment according to some embodiments is shown. The method may include receiving an indication that a request for a service provided in the multi-tenant environment has been received (802). The method may additionally include identifying a configuration implementing the service, wherein the configuration is stored in a data repository (804). The method may also include sending the configuration to a container in the multi-tenant environment to serve the request (806). The method may further include receiving runtime state from the container (808). The method may further include storing the runtime state in a data repository, wherein the configuration is flushed from the container (810). Some embodiments may include a system including one or more processors and one or more memories for performing these method steps. Other embodiments may include a non-transitory computer-readable medium storing instructions that cause one or more processors to perform these method steps.

[0052] In any embodiment, one or more of the following features may be included in any combination and are not limited thereto. Configuration may be provided by tenants in the multi-tenant environment prior to runtime. The data repository may include a key-value data repository. The key-value data repository may include a distributed key-value data repository. The identity of the tenant associated with the service may be a key in the key-value data repository, and the configuration and runtime state may be values ​​in the key-value data repository. The data repository may also be stored in a registry of containers available in the multi-tenant environment. When a new container based on the container registry becomes available in the multi-tenant environment, the data repository may update one or more routers in the multi-tenant environment.

[0053] It should be recognized that, Figure 8 The specific steps illustrated provide specific methods for managing the runtime state and service configuration of containers in a multi-tenant environment, according to various embodiments. According to alternative embodiments, other sequences of steps may also be performed. For example, alternative embodiments of the invention may perform the steps outlined above in a different order. Furthermore, Figure 8 The steps shown may include multiple sub-steps, which can be performed in various orders suitable for the individual steps. Furthermore, depending on the specific application, additional steps may be added or removed. Many variations, modifications, and substitutions will be recognized by those skilled in the art.

[0054] Figure 9 A flowchart illustrating a method for efficiently allocating a container pool to serve requests in a multi-tenant environment, according to some embodiments, is shown. The method may include allocating multiple containers to a first tenant in the multi-tenant environment (902). The method may also include identifying one or more containers among the multiple containers that were allocated to the first tenant but are not currently being used by the first tenant (904). The method may additionally include flushing the contents of one or more containers (906). The method may also include reassigning one or more containers to a second tenant in the multi-tenant environment (908). Some embodiments may include a system including one or more processors and one or more memories for performing these method steps. Other embodiments may include a non-transitory computer-readable medium storing instructions that cause one or more processors to perform these method steps.

[0055] In any embodiment, one or more of the following features may be included in any combination and are not limited thereto. After flushing the contents of one or more containers, the one or more containers need not be assigned to any tenant during the first time interval before being reassigned to a second tenant. The method may further include: identifying a second or more containers among a plurality of containers assigned to a first tenant but not currently being used by the first tenant; determining that no other tenant needs the second or more containers; and removing the second or more containers from the multitenant environment. The method may additionally include: determining that the first tenant is receiving more requests than the plurality of containers can serve. The method may further include instantiating a new plurality of containers; and assigning the new plurality of containers to the first tenant. The method may further include assigning a container previously assigned to another tenant to the first tenant. A gateway in a multitenant environment may reassign these one or more containers to a second tenant within the multitenant environment.

[0056] It should be recognized that, Figure 9 The specific steps illustrated provide specific methods for efficiently allocating container pools to serve requests in a multi-tenant environment, according to various embodiments. According to alternative embodiments, other sequences of steps may also be performed. For example, alternative embodiments of the invention may perform the steps outlined above in a different order. Furthermore, Figure 9 The steps shown may include multiple sub-steps, which can be performed in various orders suitable for the individual steps. Furthermore, depending on the specific application, additional steps may be added or removed. Many variations, modifications, and substitutions will be recognized by those skilled in the art.

[0057] Each method described herein can be implemented by a computer system. Each step of these methods can be executed automatically by the computer system and / or can provide input / output involving a user. For example, a user can provide input for each step in the method, and each of these inputs can provide a specific output in response to a request for such input, wherein the output is generated by the computer system. Each input can be received in response to a corresponding requested output. Furthermore, input can be received from a user, as a data stream from another computer system, retrieved from a memory location, retrieved via a network, requested from a web service, etc. Similarly, output can be provided to a user, provided as a data stream to another computer system, stored in a memory location, sent via a network, provided to a web service, etc. In short, each step of the methods described herein can be executed by a computer system and can involve any number of inputs, outputs, and / or requests from and to the computer system, which may or may not involve a user. It can be said that those steps not involving a user are executed automatically by the computer system without human intervention. Therefore, it will be understood from this disclosure that each step of each method described herein can be modified to include inputs and outputs to and from a user, or can be performed automatically by the computer system without human intervention, wherein any determination is generated by a processor. Furthermore, some embodiments of each method described herein can be implemented as a set of instructions stored on a tangible, non-transient storage medium to form a tangible software product.

[0058] Figure 10 A simplified diagram is depicted for implementing a distributed system 1000 according to one embodiment. In the illustrated embodiment, the distributed system 1000 includes one or more client computing devices 1002, 1004, 1006, and 1008, each of which can be configured to execute and operate client applications, such as web browsers, proprietary clients (e.g., Oracle Forms), via one or more networks 1010. A server 1012 can be communicatively coupled to remote client computing devices 1002, 1004, 1006, and 1008 via network 1010.

[0059] In various embodiments, server 1012 may be adapted to run one or more services or software applications provided by one or more components of the system. In some embodiments, these services may be provided as web-based services or cloud services, or provided to users of client computing devices 1002, 1004, 1006, and / or 1008 under a Software as a Service (SaaS) model. Users operating client computing devices 1002, 1004, 1006, and / or 1008 may then use one or more client applications to interact with server 1012 to utilize the services provided by these components.

[0060] In the configuration illustrated in the figures, software components 1018, 1020, and 1022 of system 1000 are shown as being implemented on server 1012. In other embodiments, one or more components of system 1000 and / or the services provided by these components may also be implemented by one or more of client computing devices 1002, 1004, 1006, and / or 1008. A user operating the client computing device can then utilize one or more client applications to use the services provided by these components. These components can be implemented in hardware, firmware, software, or a combination thereof. It should be recognized that various different system configurations are possible and may differ from the distributed system 1000. The embodiments shown in the figures are therefore an example of a distributed system for implementing the system of the embodiments and are not intended to be limiting.

[0061] Client computing devices 1002, 1004, 1006, and / or 1008 can be portable handheld devices (e.g., Cellular phone Computing tablets, personal digital assistants (PDAs), or wearable devices (e.g., Google) Head-mounted displays (or similar devices) that run Microsoft Windows And / or software for various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, etc.), and with internet access, email, and SMS services enabled. Or other communication protocols. The client computing device can be a general-purpose personal computer, for example, including those running various versions of Microsoft... Apple Personal computers and / or laptops running Linux operating systems. Client computing devices can be any commercially available operating system. Workstation computers running UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, such as Google Chrome OS). Alternatively or additionally, client computing devices 1002, 1004, 1006, and 1008 may be any other electronic device capable of communicating via one or more networks 1010, such as thin client computers, internet-enabled gaming systems (e.g., with or without...). The gesture input device is the Microsoft Xbox game console and / or a personal messaging device.

[0062] Although the exemplary distributed system 1000 is shown as having four client computing devices, any number of client computing devices can be supported. Other devices (such as devices with sensors) can interact with the server 1012.

[0063] The one or more networks 1010 in the distributed system 1000 can be any type of network familiar to those skilled in the art, capable of supporting data communication using any variety of commercially available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internet Message Switching), AppleTalk, etc. By way of example only, the one or more networks 1010 can be a local area network (LAN), such as a LAN based on Ethernet, Token Ring, etc. The one or more networks 1010 can be a wide area network (WAN) and the Internet. It can include virtual networks, including but not limited to Virtual Private Networks (VPNs), intranets, extranets, Public Switched Telephone Networks (PSTN), infrared networks, wireless networks (e.g., according to the IEEE 802.11 protocol suite), (and / or any other wireless protocol operating network); and / or any combination of these and / or other networks.

[0064] Server 1012 can consist of one or more general-purpose computers, dedicated server computers (as an example, including PC (personal computer) servers), Servers can be configured as servers, mid-range servers, mainframe computers, rack-mounted servers, server farms, server clusters, or any other suitable arrangement and / or combination. In various embodiments, server 1012 may be adapted to run one or more services or software applications described in the foregoing disclosure. For example, server 1012 may correspond to a server used to perform the processes described above according to embodiments of this disclosure.

[0065] Server 1012 can run any of the operating systems discussed above, as well as any commercially available server operating system. Server 1012 can also run various additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, etc. Servers, database servers, etc. Exemplary database servers include, but are not limited to, those commercially available database servers from Oracle, Microsoft, Sybase, IBM, etc.

[0066] In some implementations, server 1012 may include one or more applications to analyze and integrate data feeds and / or event updates received from users of client computing devices 1002, 1004, 1006, and 1008. As an example, data feeds and / or event updates may include, but are not limited to, feed, The server 1012 may update or receive real-time updates and continuous data streams from one or more third-party information sources, which may include real-time events related to sensor data applications, financial quotation machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and the like. The server 1012 may also include one or more applications to display the data feeds and / or real-time events via one or more display devices of client computing devices 1002, 1004, 1006, and 1008.

[0067] The distributed system 1000 may also include one or more databases 1014 and 1016. Databases 1014 and 1016 may reside in various locations. As an example, one or more of databases 1014 and 1016 may reside on non-transient storage media local to server 1012 (and / or residing within server 1012). Alternatively, databases 1014 and 1016 may be located remotely from server 1012 and communicate with server 1012 via a network-based connection or a dedicated connection. In one set of embodiments, databases 1014 and 1016 may reside in a storage area network (SAN) familiar to those skilled in the art. Similarly, any necessary files for performing the functions of server 1012 may be appropriately stored locally on server 1012 and / or remotely. In one set of embodiments, databases 1014 and 1016 may include relational databases, such as those provided by Oracle, adapted to store, update, and retrieve data in response to commands in SQL format.

[0068] Figure 11 This is a simplified block diagram of one or more components of a system environment 1100 according to an embodiment of the present disclosure, through which services provided by one or more components of the embodiment system can be provided as cloud services. In the illustrated embodiment, system environment 1100 includes one or more client computing devices 1104, 1106, and 1108 that can be used by a user to interact with a cloud infrastructure system 1102 providing cloud services. The client computing devices can be configured to operate client applications, such as web browsers, proprietary client applications (e.g., Oracle Forms), or some other application, which can be used by the user of the client computing devices to interact with the cloud infrastructure system 1102 to use the services provided by the cloud infrastructure system 1102.

[0069] It should be recognized that the cloud infrastructure system 1102 depicted in the figures may have other components besides those depicted. Furthermore, the embodiment shown in the figures is merely one example of a cloud infrastructure system that can be incorporated into embodiments of the present invention. In some other embodiments, the cloud infrastructure system 1102 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations or arrangements.

[0070] Client computing devices 1104, 1106, and 1108 may be devices similar to those described above for 1002, 1004, 1006, and 1008.

[0071] While the exemplary system environment 1100 is shown with three client computing devices, any number of client computing devices can be supported. Other devices, such as those with sensors, can interact with the cloud infrastructure system 1102.

[0072] One or more networks 1110 can facilitate data communication and exchange between clients 1104, 1106, and 1108 and cloud infrastructure system 1102. Each network can be any type of network familiar to those skilled in the art that supports data communication using any of a variety of commercially available protocols, including those described above for one or more networks 1010.

[0073] The cloud infrastructure system 1102 may include one or more computers and / or servers, which may include those computers and / or servers described above for server 1012.

[0074] In some embodiments, services provided by a cloud infrastructure system may include a variety of services available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc. Services provided by a cloud infrastructure system can be dynamically scaled to meet the needs of users of the cloud infrastructure system. A specific instantiation of a service provided by a cloud infrastructure system is referred to herein as a "service instance." Generally, any service available to users from a cloud service provider's system via a communication network (such as the Internet) is referred to as a "cloud service." Typically, in a public cloud environment, the servers and systems that constitute the cloud service provider's system are different from the customer's own local servers and systems. For example, a cloud service provider's system may host applications, and users can subscribe to and use applications on demand via a communication network such as the Internet.

[0075] In some examples, services within a computer network cloud infrastructure may include protected computer network access to storage devices, hosted databases, hosted web servers, software applications, or other services provided to users by the cloud provider, or as otherwise known in the art. For example, services may include password-protected access to remote storage devices in the cloud via the Internet. As another example, services may include web-based hosted relational databases and scripting language middleware engines for private use by networked developers. As yet another example, services may include access to email software applications hosted on a cloud provider's website.

[0076] In some embodiments, cloud infrastructure system 1102 may include a suite of application, middleware, and database service products delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle public cloud provided by this assignee.

[0077] In various embodiments, cloud infrastructure system 1102 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by cloud infrastructure system 1102. Cloud infrastructure system 1102 may provide cloud services via different deployment models. For example, services may be provided under a public cloud model, where cloud infrastructure system 1102 is owned by an organization selling cloud services (e.g., owned by Oracle), and the services are available to the general public or businesses in different industries. As another example, services may be provided under a private cloud model, where cloud infrastructure system 1102 operates only for a single organization and can provide services to one or more entities within that organization. Cloud services may also be provided under a community cloud model, where cloud infrastructure system 1102 and the services provided by cloud infrastructure system 1102 are shared by several organizations in the relevant community. Cloud services may also be provided under a hybrid cloud model, which is a combination of two or more different models.

[0078] In some embodiments, the services provided by the cloud infrastructure system 1102 may include one or more services offered under the Software as a Service (SaaS) category, the Platform as a Service (PaaS) category, the Infrastructure as a Service (IaaS) category, or other service categories that include hybrid services. A customer may subscribe to one or more services provided by the cloud infrastructure system 1102 via a subscription order. The cloud infrastructure system 1102 then performs processing to deliver the services in the customer's subscription order.

[0079] In some embodiments, the services provided by the cloud infrastructure system 1102 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall into the SaaS category. For example, the SaaS platform may provide the ability to build and deliver on-demand application suites on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure used to provide SaaS services. By utilizing the services provided by the SaaS platform, customers can leverage applications running on the cloud infrastructure system. Customers can obtain application services without having to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services providing solutions for sales performance management, enterprise integration, and business agility for large organizations.

[0080] In some embodiments, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services that fall into the PaaS category. Examples of platform services may include, but are not limited to, services that enable organizations (such as Oracle) to integrate existing applications on a shared, public architecture and to leverage the shared services provided by the platform to build new applications. The PaaS platform can manage and control the underlying software and infrastructure used to provide PaaS services. Customers can access PaaS services provided by the cloud infrastructure system without having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), etc.

[0081] By leveraging services provided by a PaaS platform, customers can employ programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware Service), and Java cloud services. In one embodiment, the database cloud service may support a shared services deployment model that enables organizations to aggregate database resources and provide database-as-a-service to customers in the form of a database cloud. Within the cloud infrastructure system, the middleware cloud service provides customers with a platform for developing and deploying various business applications, and the Java cloud service provides customers with a platform for deploying Java applications.

[0082] Various infrastructure services can be provided by IaaS platforms within cloud infrastructure systems. Infrastructure services facilitate the management and control of underlying computing resources (such as storage devices, networks, and other basic computing resources) for customers to utilize services provided by SaaS and PaaS platforms.

[0083] In some embodiments, the cloud infrastructure system 1102 may further include infrastructure resources 1130 for providing resources to customers of the cloud infrastructure system for delivering various services. In one embodiment, infrastructure resources 1130 may include a combination of pre-integrated and optimized hardware (such as servers, storage devices, and networking resources) to perform services provided by PaaS platforms and SaaS platforms.

[0084] In some embodiments, resources in cloud infrastructure system 1102 can be shared by multiple users and dynamically reallocated as needed. Furthermore, resources can be allocated to users in different time zones. For example, cloud infrastructure system 1130 can enable a first group of users in a first time zone to utilize the resources of the cloud infrastructure system for a specified number of hours, and then enable the same resources to be reallocated to another group of users located in a different time zone, thereby maximizing resource utilization.

[0085] In some embodiments, multiple internal shared services 1132 may be provided, shared by different components or modules of the cloud infrastructure system 1102 and by services provided by the cloud infrastructure system 1102. These internal shared services may include, but are not limited to: security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, cloud-enabled services, email services, notification services, file transfer services, etc.

[0086] In some embodiments, cloud infrastructure system 1102 can provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) within the cloud infrastructure system. In one embodiment, cloud management functionality may include the ability to provision, manage, and track customer subscriptions received by cloud infrastructure system 1102.

[0087] In one embodiment, as illustrated in the figure, cloud management functionality may be provided by one or more modules, such as an order management module 1120, an order orchestration module 1122, an order supply module 1124, an order management and monitoring module 1126, and an identity management module 1128. These modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0088] In exemplary operation 1134, a customer using a client device (such as client device 1104, 1106, or 1108) can interact with cloud infrastructure system 1102 by requesting one or more services provided by cloud infrastructure system 1102 and placing an order to subscribe to one or more services provided by cloud infrastructure system 1102. In some embodiments, the customer can access cloud user interfaces (UIs) (cloud UI 1112, cloud UI 1114, and / or cloud UI 1116) and place subscription orders via these UIs. Order information received by cloud infrastructure system 1102 in response to a customer placing an order may include information identifying the customer and the one or more services provided by cloud infrastructure system 1102 that the customer wishes to subscribe to.

[0089] After a customer places an order, the order information is received via cloud UI 1112, 1114 and / or 1116.

[0090] At operation 1136, the order is stored in order database 1118. Order database 1118 may be one of several databases operated by cloud infrastructure system 1118 and operating together with other system components.

[0091] At operation 1138, the order information is forwarded to the order management module 1120. In some cases, the order management module 1120 can be configured to perform order-related billing and accounting functions, such as verifying the order and reserving the order after verification.

[0092] At operation 1140, order information is transmitted to order orchestration module 1122. Order orchestration module 1122 can use the order information to orchestrate services and supply resources for customer orders. In some cases, order orchestration module 1122 can use the services of order supply module 1124 to orchestrate resource supply to support subscribed services.

[0093] In some embodiments, the order orchestration module 1122 enables the management of business processes associated with each order and applies business logic to determine whether an order should be made available for provisioning. At operation 1142, upon receiving a new subscription order, the order orchestration module 1122 sends a request to the order provisioning module 1124 to allocate resources and configure those resources required to fulfill the subscription order. The order provisioning module 1124 enables the allocation of resources for the services ordered by the customer. The order provisioning module 1124 provides an abstraction layer between the cloud services provided by the cloud infrastructure system 1100 and the physical implementation layer for providing the resources used to provide the requested services. Therefore, the order orchestration module 1122 can be isolated from implementation details such as whether services and resources are actually provided on demand or pre-provided and allocated / assigned only upon request.

[0094] At operation 1144, once services and resources are supplied, notifications of the supplied services can be sent to customers on client devices 1104, 1106, and / or 1108 via the order provisioning module 1124 of the cloud infrastructure system 1102.

[0095] At operation 1146, the order management and monitoring module 1126 can manage and track customer subscription orders. In some cases, the order management and monitoring module 1126 can be configured to collect service usage statistics in subscription orders, such as storage usage, data transfer volume, number of users, system uptime, and system downtime.

[0096] In some embodiments, the cloud infrastructure system 1100 may include an identity management module 1128. The identity management module 1128 may be configured to provide identity services, such as access management and authorization services within the cloud infrastructure system 1100. In some embodiments, the identity management module 1128 may control information about customers who wish to utilize services provided by the cloud infrastructure system 1102. Such information may include information authenticating the identities of these customers and information describing what actions these customers are authorized to perform relative to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). The identity management module 1128 may also include management of descriptive information about each customer and how and by whom this descriptive information can be accessed and modified.

[0097] Figure 12 An exemplary computer system 1200 in which various embodiments of the present invention can be implemented is shown. System 1200 can be used to implement any of the computer systems described above. As shown, computer system 1200 includes a processing unit 1204 that communicates with a plurality of peripheral subsystems via a bus subsystem 1202. These peripheral subsystems may include a processing acceleration unit 1206, an I / O subsystem 1208, a storage subsystem 1218, and a communication subsystem 1224. Storage subsystem 1218 includes a tangible computer-readable storage medium 1222 and system memory 1210.

[0098] Bus subsystem 1202 provides a mechanism for allowing various components and subsystems of computer system 1200 to communicate with each other as intended. While bus subsystem 1202 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1202 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of the various bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0099] A processing unit 1204, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 1200. One or more processors may be included in the processing unit 1204. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 1204 may be implemented as one or more independent processing units 1232 and / or 1234, each including a single-core or multi-core processor. In other embodiments, the processing unit 1204 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0100] In various embodiments, processing unit 1204 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processor(s) 1204 and / or storage subsystem 1218. With appropriate programming, processor(s) 1204 can provide the various functions described above. Computer system 1200 may additionally include processing acceleration unit 1206, which may include digital signal processor (DSP), dedicated processor, etc.

[0101] I / O subsystem 1208 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keyboards, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as those from Microsoft… Motion sensors enable users to control devices such as Microsoft products via a natural user interface using gestures and voice commands. The 360 ​​game controller's input device interacts with it. The user interface input device may also include eye gesture recognition devices, such as detecting eye movements from the user (e.g., "blinking" when taking a photo and / or making a menu selection) and translating the eye gestures into the input device (e.g., Google). Google input in ) Blink detector. Additionally, the user interface input device may include enabling the user to interact with a voice recognition system (e.g., ...) via voice commands. Voice recognition sensing devices for interaction with navigators.

[0102] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointing sticks, game panels and drawing tablets, as well as audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0103] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1200 to a user or other computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0104] Computer system 1200 may include a storage subsystem 1218 containing software elements, shown as currently located in system memory 1210. System memory 1210 may store program instructions that can be loaded and executed on processing unit 1204, as well as data generated during the execution of these programs.

[0105] Depending on the configuration and type of the computer system 1200, the system memory 1210 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that can be immediately accessed by the processing unit 1204 and / or are currently being operated and executed by the processing unit 1204. In some implementations, the system memory 1210 may include various different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), which contains basic routines that facilitate the transfer of information between the elements of the computer system 1200 during startup, may typically be stored in ROM. As an example, but not a limitation, the system memory 1210 also includes application programs 1212, program data 1214, and an operating system 1216, which may include client applications, web browsers, middleware applications, relational database management systems (RDBMS), etc. As an example, the operating system 1216 may include various versions of Microsoft... Apple and / or Linux operating system, and various commercially available... Or a UNIX-like operating system (including but not limited to various GNU / Linux operating systems, Google...) OS operating systems, etc.) and / or such as iOS, Phone OS 10OS and A mobile operating system based on the OS operating system.

[0106] The storage subsystem 1218 may also provide a tangible computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the above-described functionality when executed by a processor may be stored in the storage subsystem 1218. These software modules or instructions may be executed by the processing unit 1204. The storage subsystem 1218 may also provide a repository for storing data used according to the present invention.

[0107] The storage subsystem 1200 may also include a computer-readable storage medium reader 1220 that can be further connected to the computer-readable storage medium 1222. Together with and optionally in conjunction with the system memory 1210, the computer-readable storage medium 1222 can comprehensively represent a remote, local, fixed, and / or removable storage device plus storage medium for temporarily and / or more persistently containing, storing, transmitting, and retrieving computer-readable information.

[0108] The computer-readable storage medium 1222 containing code or portions thereof may also include any suitable medium known or used in the art, including storage and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassette, magnetic tape, disk storage or other magnetic storage devices, or other tangible computer-readable media. This may also include non-tangible computer-readable media such as data signals, data transmissions, or any other medium that can be used to transmit desired information and can be accessed by the computing system 1200.

[0109] As an example, computer-readable storage medium 1222 may include a hard disk drive that reads or writes to a non-removable non-volatile magnetic medium, a disk drive that reads or writes to a removable non-volatile magnetic disk, and a removable non-volatile optical disc drive (such as CD ROM, DVD, and...). An optical disc drive that reads from or writes to a disk or other optical medium. Computer-readable storage medium 1222 may include, but is not limited to, Disk drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVDs, digital audio tapes, and so on. Computer-readable storage media 1222 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computer system 1200.

[0110] The communication subsystem 1224 provides an interface to other computer systems and networks. The communication subsystem 1224 serves as an interface for receiving data from other systems and sending data from computer system 1200 to other systems. For example, the communication subsystem 1224 enables computer system 1200 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1224 may include radio frequency (RF) transceiver components (e.g., advanced data network technologies using cellular telephone technologies, such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), GPS receiver components, and / or other components for accessing wireless voice and / or data networks. In some embodiments, as an addition to or alternative to the wireless interface, the communication subsystem 1224 may provide a wired network connection (e.g., Ethernet).

[0111] In some embodiments, the communication subsystem 1224 may also represent one or more users who can use the computer system 1200 to receive input communications in the form of structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc.

[0112] As an example, the communication subsystem 1224 can be configured to receive data feeds 1226 from users of social networks and / or other communication services in real time, such as... feed, Updates, web feeds such as rich site summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0113] Furthermore, the communication subsystem 1224 can also be configured to receive data in the form of a continuous data stream, which may include event streams 1228 and / or event updates 1230 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quotation machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and so on.

[0114] The communication subsystem 1224 can also be configured to output structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1200.

[0115] Computer system 1200 can be one of various types, including handheld portable devices (e.g., Cellular phone Computing tablets, PDAs), and wearable devices (e.g., Google). Head-mounted displays, PCs, workstations, mainframes, information stations, server racks, or any other data processing systems.

[0116] Due to the ever-evolving nature of computers and networks, the description of the computer system 2000 depicted in the figures is merely a concrete example. Many other configurations with more or fewer components than the system depicted in the figures are possible. For example, custom hardware may be used and / or specific elements may be implemented using hardware, firmware, software (including applets), or a combination thereof. Additionally, connections to other computing devices, such as network input / output devices, may also be employed. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and / or methods for implementing the various embodiments.

[0117] In the foregoing description, numerous specific details have been set forth for purposes of explanation in order to provide a thorough understanding of various embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without some of these specific details. In other instances, well-known structures and devices are illustrated in block diagram form.

[0118] The foregoing description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the foregoing description of the exemplary embodiments will provide those skilled in the art with enabling descriptions for implementing the exemplary embodiments. It should be understood that various changes can be made to the function and arrangement of the elements without departing from the spirit and scope of the invention as set forth in the appended claims.

[0119] Specific details have been set forth in the foregoing description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that these embodiments can be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may have been shown as components in block diagram form to avoid obscuring the embodiments with unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may have been shown without unnecessary detail to avoid obscuring the embodiments.

[0120] Furthermore, it should be noted that the various embodiments may be described as processes, depicted as flowcharts, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations can be executed in parallel or simultaneously. Additionally, the order of operations can be rearranged. A process terminates when its operations are completed, but there may be other steps not included in the diagram. Processes can correspond to methods, functions, procedures, subroutines, subroutines, etc. When a process corresponds to a function, its termination can correspond to the function returning to the calling function or the main function.

[0121] The term "computer-readable medium" includes, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and various other media capable of storing, containing, or carrying one or more instructions and / or data. A code segment or machine-executable instruction can represent any combination of procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. A code segment can be coupled to another code segment or hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or transmitted via any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0122] Furthermore, embodiments can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments for performing the necessary tasks can be stored in a machine-readable medium. The processor can then perform the necessary tasks.

[0123] In the foregoing specification, various aspects of the invention have been described with reference to specific embodiments thereof; however, those skilled in the art will recognize that the invention is not limited thereto. The various features and aspects disclosed above may be used individually or in combination. Furthermore, embodiments may be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of this specification. Accordingly, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

[0124] Furthermore, for illustrative purposes, the methods have been described in a specific order. It should be understood that, in alternative embodiments, the methods may be performed in a different order than described. It should also be understood that the methods described above can be executed by hardware components or can be implemented as a sequence of machine-executable instructions that can be used to cause a machine (such as a general-purpose or special-purpose processor or logic circuit programmed with instructions) to execute the methods. These machine-executable instructions can be stored on one or more machine-readable media, such as CD-ROMs or other types of optical discs, floppy disks, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, flash memory, or other types of machine-readable media suitable for storing electronic instructions. Alternatively, the methods can be executed by a combination of hardware and software.

[0125] Cross-references to related applications

[0126] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 566,422, filed September 30, 2017; U.S. Patent Application No. 62 / 566,431, filed September 30, 2017; and U.S. Patent Application No. 62 / 566,435, filed September 30, 2017. The entire disclosure of U.S. Patent Applications 62 / 566,422, 62 / 566,431, and 62 / 566,435 is incorporated herein by reference, including the appendices.

Claims

1. A method for managing the runtime state and service configuration of containers in a multi-tenant environment, the method comprising: Receive an indication that a request for the services provided in the multi-tenant environment has been received; Identify the configuration that implements the service, wherein the configuration is stored in a data repository; The configuration is sent to the container in the multi-tenant environment to serve the request; Receive runtime state from the container; as well as The runtime state is stored in the data repository, wherein the configuration is flushed from the container.

2. The method of claim 1, wherein the configuration is provided by the tenant of the multi-tenant environment prior to runtime.

3. The method of claim 1, wherein the data repository comprises a key-value data repository.

4. The method of claim 3, wherein the key-value data repository comprises a distributed key-value data repository.

5. The method of claim 3, wherein the identity of the tenant associated with the service includes keys in the key-value data repository, and wherein the configuration and runtime state include values ​​in the key-value data repository.

6. The method of claim 1, wherein the data repository is further stored in a registry of a container available in the multi-tenant environment.

7. The method of claim 6, wherein when a new container becomes available in the multi-tenant environment, the data repository updates one or more routers in the multi-tenant environment based on the container's registry.

8. The method according to claim 1, further comprising: After the configuration is rinsed from the container, an indication is received that a second request for the service has been received; The configuration is sent to a second container in the multi-tenant environment to serve the second request; Send the runtime state to the second container; Receive updated runtime state from the second container; as well as The updated runtime state is stored in the data repository.

9. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations including: Receive an indication that a request for a service provided in a multi-tenant environment has been received; Identify the configuration that implements the service, wherein the configuration is stored in a data repository; The configuration is sent to the container in the multi-tenant environment to serve the request; Receive runtime state from the container; as well as The runtime state is stored in the data repository, wherein the configuration is flushed from the container.

10. The non-transitory computer-readable medium according to claim 9, wherein, The operation also includes: Assign multiple containers to the first tenant in the multi-tenant environment; Identify one or more containers among the plurality of containers that have been assigned to the first tenant but are not currently being used by the first tenant; Rinse the contents of the one or more containers; and Reassign the one or more containers to a second tenant in the multi-tenant environment.

11. The non-transitory computer-readable medium of claim 10, wherein after the contents of the one or more containers are flushed, the one or more containers are not assigned to any tenant during a first time interval before being reassigned to the second tenant.

12. The non-transitory computer-readable medium according to claim 10, wherein, The operation also includes: Identify one or more second containers among the plurality of containers that have been assigned to the first tenant but are not currently being used by the first tenant; It is determined that no other tenants require the second or more containers; and Remove the second or more containers from the multi-tenant environment.

13. The non-transitory computer-readable medium according to claim 10, wherein, The operation also includes: It is determined that the first tenant is receiving more requests than the multiple containers can handle.

14. The non-transitory computer-readable medium according to claim 13, wherein, The operation also includes: Instantiate multiple new containers; and The new multiple containers are assigned to the first tenant.

15. The non-transitory computer-readable medium according to claim 13, wherein, The operation also includes: The container that was previously assigned to another tenant is assigned to the first tenant.

16. The non-transitory computer-readable medium of claim 10, wherein the gateway of the multi-tenant environment reassigns the one or more containers to the second tenant in the multi-tenant environment.

17. A system comprising: One or more processors; as well as One or more memory devices, including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations including: Receive an indication that a request for a service provided in a multi-tenant environment has been received; Identify the configuration that implements the service, wherein the configuration is stored in a data repository; The configuration is sent to the container in the multi-tenant environment to serve the request; Receive runtime state from the container; and The runtime state is stored in the data repository, wherein the configuration is flushed from the container.

18. The system of claim 17, wherein the operation further comprises: Receive the first request for the first service provided by the first tenant; Select an empty container in the multi-tenant environment; The first configuration for implementing the first service is loaded into the container; The first request is processed from the container service; Receive a second request for a second service provided by the second tenant; The first configuration is flushed from the container, wherein, after flushing the first configuration from the container, the container includes a runtime process having an embedded server and internal endpoints; as well as The second request is served from the container.

19. The system of claim 18, wherein the container is one of a plurality of containers in the multi-tenant environment, the plurality of containers being instantiated to serve requests from client devices.

20. The system of claim 18, wherein the first configuration includes the size of the heap in the memory that the first service can use.

Images