Method for improving performance of multi-tenant application based on docker container platform and storage medium

Abstract

The application discloses a method for improving performance of multi-tenant applications based on a Docker container platform and a storage medium, which adopts the operation steps of a tenant-specific Docker component process, LC container CPU binding and soft interrupt binding, and dynamic adjustment within a tenant. First, the application ingeniously achieves isolation at the tenant level by using isolation technology at the container level, and does not involve modification of container bottom-layer code, so that the application has good universality. Compared with a traditional resource adjustment method, the application only involves adjustment of resources within a tenant, does not break the isolation between tenants, and has higher fairness. Finally, the application can instantly perceive performance change of an application, quickly recover when performance decline occurs, and thus can improve throughput of BE applications under the premise of ensuring performance of LC applications, and improve resource utilization of a tenant.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and more specifically to a method for improving the performance of multi-tenant applications based on the Docker container platform. Background Technology

[0002] Cloud computing data centers are widely popular because they provide application developers with elastic services, pay-as-you-go billing, and efficient and flexible development environments. Tenants first purchase the necessary resources from cloud service providers (such as Alibaba Cloud), and then use the tools provided by the cloud service providers to containerize and deploy applications. For example, Docker, the most popular container technology in cloud computing, can package tenant applications into images and upload them to the cloud repository. These images can then be pulled from the cloud repository anytime, anywhere to create Docker containers for deployment and execution. Because multiple tenants share the cloud environment maintenance costs, the cost of building and maintaining applications is lower than that of traditional physical server deployment models.

[0003] To improve data center cluster resource utilization, cloud service providers typically deploy and run applications from multiple tenants on the same host. Therefore, resource isolation between tenants is crucial for ensuring fairness and application performance. However, the isolation technology used by Docker containers, based on Linux's Cgroups (Control Groups), is limited to controlling and isolating resource consumption of processes within the container, without considering the resource consumption of processes outside the container (e.g., background processes on the host). Malicious tenants can exploit such processes to escape the container and continuously consume host resources, thereby interfering with and attacking other tenants' applications, causing performance degradation.

[0004] Tenant applications are typically categorized into latency-critical (LC) applications, represented by high-priority tasks, and best-effort (BE) applications, represented by low-priority tasks. To ensure the performance of LC applications, container management platforms usually evict or kill BE applications when resource contention arises. However, the platform does not consider that in a multi-tenant model, it may lead to the killing of other tenants' BE applications to ensure the current tenant's LC application runs. Summary of the Invention

[0005] The present invention proposes a method for improving the performance of multi-tenant applications based on the Docker container platform, which can at least solve one of the above-mentioned technical problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for improving the performance of multi-tenant applications based on a Docker container platform includes:

[0008] Step 1: Tenant-specific Docker component process

[0009] Sharing Docker component processes on the same host among multiple tenants can cause competition between applications from different tenants, leading to performance interference. By utilizing multiple Docker backends and host resource pools, strong isolation can be achieved between tenants. By default, Docker does not distinguish between containers from different tenants, which may cause damage to other tenants' application containers during subsequent resource adjustments, resulting in a decrease in tenant benefits. Labels are used to mark containers according to their respective tenants, preventing the breakdown of tenant isolation.

[0010] Step 2: LC container CPU binding and soft interrupt binding

[0011] The applications within a tenant can be divided into LC applications and BE applications. Since LC and BE applications have different performance goals, the performance of LC applications is prioritized during initialization. This is achieved by allocating dedicated CPUs to LC containers based on priority, while BE and Docker background component processes share the remaining CPUs within the tenant, thus preventing interference with LC containers and performance degradation.

[0012] When containers send or receive data packets, they trigger soft interrupts, which need to be processed by the kernel process Ksoftirqd. By default, the CPU handling soft interrupts is randomized, which may break the isolation between tenants. Therefore, by utilizing the network card's XPS and RFS technologies, the same CPU is used for sending and receiving data packets to enhance the isolation between tenants.

[0013] Step 3: Dynamic Adjustment Within the Tenant Unit

[0014] Based on the settings in the first two steps, strong isolation between tenants can be guaranteed. However, resource utilization within a tenant is at a low level. Therefore, we dynamically adjust the internal performance of each tenant's LC container by monitoring its performance. Specifically, we adjust the sharing level of LC containers. When the LC container's performance targets are met and sufficient resources are available, a BE container is selected to share it; conversely, when the LC container's performance targets are compromised, the BE container sharing it is evicted. By continuously executing this process, we aim to significantly improve the resource utilization within a tenant while ensuring the overall performance of the LC containers.

[0015] As can be seen from the above technical solution, the present invention proposes a performance guarantee strategy for multi-tenant applications based on the Docker container platform to overcome the above-mentioned shortcomings of the prior art. While ensuring that applications among multiple tenants do not interfere with each other, dynamic resource adjustments are made within the tenant to reduce the number of times LC applications experience performance degradation, improve the resource utilization within the tenant, and thus improve the overall host throughput.

[0016] This invention is based on a multi-tenant application performance guarantee strategy for Docker container platforms, employing steps such as tenant-specific Docker component processes, LC container CPU binding and soft interrupt binding, and dynamic adjustment within the tenant. Firstly, because this invention cleverly utilizes container-level isolation technology to achieve tenant-level isolation without modifying the underlying container code, it has good versatility. Furthermore, compared to traditional resource adjustment methods, this method only involves internal resource adjustments within the tenant and does not break the isolation between tenants, thus offering higher fairness. Finally, this method can instantly detect changes in application performance and quickly recover even if performance degradation occurs. Therefore, it can improve the throughput of the BE application while ensuring the performance of the LC application, thereby increasing the resource utilization of the tenant. Attached Figure Description

[0017] Figure 1 A schematic diagram of the multi-tenant model of the Docker container platform;

[0018] Figure 2 A schematic diagram of the improved multi-tenant model with multiple Docker backends;

[0019] Figure 3 This is a diagram illustrating the method for allocating container resources within a tenant during initialization.

[0020] Figure 4 A schematic diagram illustrating the architecture implemented on the Docker platform;

[0021] Figure 5 This is a schematic diagram illustrating the implementation process of the performance guarantee strategy for multi-tenant applications on the Docker container platform according to the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0023] like Figure 1 As shown in this embodiment, the method for improving the performance of multi-tenant applications based on the Docker container platform includes:

[0024] Step 1: Tenant-specific Docker component process

[0025] Docker uses a client-server architecture. Tenants send and execute commands such as create and run to the Docker daemon via the Docker client (dockercli). The Docker daemon, in turn, generates a set of Docker component processes to assist in the normal operation of Docker containers. Multiple tenants share the same set of Docker component processes, and the resource control of these processes is not subject to Docker container cgroup control, which can easily lead to a breakdown of resource isolation between tenants. Therefore, we design a dedicated Docker component process for each new tenant, allowing containers within a tenant to interact only with their own Docker daemon. Then, based on the tenant's purchase quantity N... i A new resource range S is allocated from the host's resource pool. i (i is the tenant number), isolate the Docker component process in S i And a label is set for each Docker container to indicate which tenant it belongs to.

[0026] Step 2: LC container CPU binding and soft interrupt binding

[0027] Based on the resource quantity N obtained in the first step i and resource scope S i Then, within this scope, resources are allocated specifically to each container or component process. We pre-reserve at least M CPUs (M is a configurable parameter) for all BE containers and component processes to share. By recording the runtime information of the LC container, the pre-allocated resource amount R of the LC container can be obtained. j and priority P j (j represents the j-th LC container of the current tenant), we use the cpuset technology in Cgroup to set the runnable CPU range for each LC container according to its priority. If there are still resources remaining after all LC containers are bound to CPUs (i.e., N... i -∑R j >M) Then the size of all BE containers and component processes can be expanded from M to N. i -∑R j Conversely, if the conditions are not met, the lower-priority LC container will share M containers with the BE container and component processes.

[0028] By default, when a container sends or receives network packets, it triggers a soft interrupt, which is handled by the kernel process ksofirqd (one ksofirqd process per CPU core). However, the CPU processing the packets may not be the same as the CPU running the container, causing the container's actual CPU consumption to exceed the pre-allocated amount. By utilizing network interface card (NIC) XPS (Transmit Packet Steering) and RFS (Receive Flow Steering) technologies, it can be ensured that the same CPU is used for processing network packets when the container sends and receives packets.

[0029] Step 3: Dynamic Adjustment Within the Tenant Unit

[0030] We determine if performance degradation occurs by monitoring the performance metrics (e.g., request latency) of each LC application. If the LC application performance remains normal for a period of time (α, a configurable parameter), we consider this LC container to be in a "shareable" state. Then, we monitor the CPU utilization of each core; if it falls below a certain threshold (β, a configurable parameter), we consider this core to be a "shareable core." We update the state every cycle T and, based on container priority, select a container from the lowest priority LC container (if any) or BE container to share the CPU core of the "shareable core" of the LC container in the "shareable" state, gradually increasing the sharing level using cpushare. If an LC container is in a "non-shareable" state, we evict the container sharing with it.

[0031] The following are examples:

[0032] The following is a schematic diagram illustrating the implementation process of the performance guarantee strategy for multi-tenant applications based on the Docker container platform according to specific embodiments, with reference to the accompanying drawings.

[0033] Example 1:

[0034] This embodiment is based on the Docker container platform's multi-tenant application performance guarantee strategy. It automatically allocates and adjusts resources according to the amount of resources purchased by the tenant and the required number of LC containers. The specific implementation method is as follows:

[0035] Figure 1This describes the default multi-tenant model of the Docker platform. All tenants on the same host use the client entry point (docker-client) to execute commands such as creating, modifying, and deleting containers in the Docker daemon. The Docker daemon uses containerd to specifically control the container lifecycle. By default, Docker containers from different tenants are not distinguished, and therefore they share all resources of the same host. To prevent containers from exhausting all resources on the host, the Cgroup mechanism can be used to limit and isolate the resources consumed by processes inside containers. However, the Docker platform does not control the resources directly or indirectly consumed by processes outside the containers (such as ksofirqd and docker-daemon). This can lead to the actual resource consumption among tenants not reaching the pre-purchased amount, and may also cause containers to compete with these processes, resulting in degraded application performance. Therefore, it is necessary to deepen resource isolation between tenants at the multi-tenant level to ensure application performance.

[0036] This embodiment is based on the Docker container platform's multi-tenant application performance guarantee strategy, specifically including the following steps:

[0037] Step 1: Tenant-specific Docker component process

[0038] Figure 2 This diagram illustrates the allocation of dedicated Docker component processes to two different tenants. First, different docker-daemon instances are created for tenant A and tenant B respectively. Then, the host's resources (e.g., CPU, memory, network) are allocated based on the amount of resources they purchase. Since the Docker component processes are forked from the Docker daemon process, the resource usage scope of application containers for different tenants can be specified by setting the docker-daemon's Cgroup.

[0039] Since native Docker does not distinguish which tenant a container belongs to, resource adjustments may result in the killing or eviction of containers belonging to other tenants. Using the Label attribute to mark application containers from different tenants ensures that subsequent resource adjustments will not affect application containers belonging to other tenants.

[0040] Step 2: LC container CPU binding and soft interrupt binding

[0041] Figure 3A diagram illustrating the resource allocation method for containers within a tenant during initialization is provided. Based on the preliminary definition of the resource range available to the tenant obtained in the first step, the current tenant's application containers should be allocated within this range. The diagram shows that the tenant purchased a total of 6 CPUs, comprising two LC application containers and three BE application containers. To ensure the quality of service for LC applications and prevent interference between containers, dedicated CPUs are allocated to LC applications according to their priority (LC1 exclusively uses CPU1, LC2 exclusively uses CPU2, and CPU3). Since the service quality of BE applications is lower than that of LC applications, they can tolerate greater interference and runtime latency. Therefore, the three BE applications and the Docker component processes share the remaining CPUs.

[0042] Since LC applications are typically online applications that interact with users, they frequently send network data packets in large quantities. The CPUs that process the sending and receiving of these data packets may not be the same, or they may even be processed on CPUs that have been allocated to other tenants.

[0043] By configuring the network interface card (NIC) XPS (Transmit Packet Steering) and RFS (Receive Flow Steering) technologies, the receive queue (rx-queue) and send queue (tx-queue) are mapped one-to-one with a specified CPU, thereby ensuring that the same CPU processes network packets when the container sends and receives them.

[0044] Step 3: Dynamic Adjustment Within the Tenant Unit

[0045] Based on the initial allocation state obtained in step two, since the resource consumption of the LC application is positively correlated with the number of user requests (mostly low request volume, with occasional request peaks), the resource consumption of the LC container fluctuates and its resource utilization is low. In contrast, the resource consumption of the BE application is typically higher. According to the resource allocation in step two, it is possible that resources exclusively allocated to the LC container will be wasted, while the BE application will experience resource shortages. Therefore, it is necessary to increase the sharing level of the LC container at appropriate times and evict the BE container sharing it when the LC application container is damaged. Figure 4As shown, the monitor collects data on each LC container of each tenant every period T, gathering the latency of the LC container and the CPU utilization of its dedicated core. Based on the information collected by the monitor, the resource manager finds that the latency of tenant A's LC2 container meets the Quality of Service (QoS) target and that the utilization rates P1 and P2 of CPU2 and CPU3 are both less than the sharing threshold β. Therefore, the resource manager performs a sharing operation on tenant A's LC2 container, selecting the highest-priority container BE1 from the BE containers and expanding its available CPU set from CPU4-6 to CPU2-6. Simultaneously, the resource manager finds that tenant B's LC1 container's QoS is compromised due to sharing. Even though the utilization rates P3 and P4 of CPU7 and CPU8 are both less than the sharing threshold β, the resource manager still performs an eviction operation on tenant B's LC1 container, reducing the available CPU set for all shared BE containers on the LC1 container's dedicated core from CPU7-10 to CPU9-10.

[0046] In this embodiment, by employing a multi-tenant application performance guarantee strategy based on the Docker container platform, on the one hand, by allocating resource pools to tenants and statically partitioning host resource pools, the invention achieves isolation at the tenant level, ensuring fairness among tenants. On the other hand, within each tenant, the invention utilizes a container resource pre-allocation strategy and a runtime dynamic adjustment mechanism to significantly improve the throughput of the BE application while ensuring the performance of the LC container, thereby increasing resource utilization within the tenant. This invention does not involve the underlying container design; through reasonable upper-layer management and allocation, it is compatible with most existing container management platforms, and since the main overhead is in periodic monitoring and sampling, the additional overhead is low. Therefore, this invention can guarantee the application performance of multi-tenant users on the Docker platform and improve the CPU resource utilization of the cluster.

[0047] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of any of the methods described above.

[0048] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any of the methods described above.

[0049] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of any of the methods described in the above embodiments.

[0050] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0051] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0052] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for improving the performance of multi-tenant applications based on a Docker container platform, characterized in that, Includes the following steps, Step 1: Tenant-specific Docker component process By utilizing multiple Docker backends and host resource pools, strong isolation between tenants is achieved; by using labels to mark containers as belonging to their tenants, the isolation between tenants is prevented from being broken. Step 2: LC container CPU binding and soft interrupt binding The applications within a tenant are divided into LC applications and BE applications. LC containers are allocated dedicated CPUs based on priority, while BE and Docker background component processes share the remaining CPUs of the tenant to prevent interference with LC containers. When a container sends or receives data packets, it triggers a soft interrupt, which needs to be handled by the kernel process Ksoftirqd. By utilizing the network card's XPS and RFS technologies, the CPU for sending and receiving data packets is made the same, thereby enhancing the isolation between tenants. Step 3: Dynamic Adjustment Within the Tenant Unit By monitoring the performance of each tenant's LC container, dynamic adjustments are made within the tenant; specifically, the sharing level of the LC container is adjusted, and under the condition that the performance target of the LC container is met and the set resources are available, a BE container is selected to share with it. If the performance goals of an LC container are compromised, the BE container that it shares with will be evicted.

2. The method for improving the performance of multi-tenant applications based on the Docker container platform according to claim 1, characterized in that: The first step, the tenant-specific Docker component process, specifically includes: First, different docker-daemon are generated for tenant A and tenant B respectively, and then the resources of the host are allocated according to the amount of resources they purchased. Since Docker component processes are forked from the Docker daemon process, the resource usage scope of application containers for different tenants can be specified by setting the docker-daemon's Cgroups. Use the Label attribute to mark the application containers of different tenants so that subsequent resource adjustments will not affect the application containers of other tenants.

3. The method for improving the performance of multi-tenant applications based on the Docker container platform according to claim 2, characterized in that: The second step: LC container CPU binding and soft interrupt binding, specifically including: Based on the preliminary definition of the resource range that tenants can use obtained in the first step, the current tenant's application containers are allocated within this range; Assume the tenant purchased a total of 6 CPUs, with two LC application containers and three BE application containers; the LC applications are allocated dedicated CPUs according to their priority, i.e., LC1 has exclusive use of CPU1, LC2 has exclusive use of CPU2, and CPU3, while the three BE applications and Docker component processes share the remaining CPUs. LC applications are online applications that interact with users and frequently send network data packets in a dense manner. The CPUs that process the sending and receiving of data packets may not be the same, or even the CPUs that have been allocated to other tenants. By configuring the network interface card (XPS) and network sync (RFS) technologies, the receive queue and send queue are mapped one-to-one with a specified CPU, thus ensuring that the same CPU processes network packets when the container sends and receives them.

4. The method for improving the performance of multi-tenant applications based on the Docker container platform according to claim 3, characterized in that: The third step involves dynamic adjustments within the tenant's premises; specifically, it includes: Based on the initial allocation state obtained in the second step; increase the sharing degree of LC containers during the setting, and evict BE containers that share with LC application containers when LC application containers are damaged; Assume that the monitor collects data on each LC container of each tenant every period T, collecting the latency of the LC container and the CPU utilization of its dedicated core; the resource manager finds that the latency of tenant A's LC2 container meets the service quality target and that the utilization P1 and P2 of the occupied CPU2 and CPU3 are both less than the sharing threshold β based on the information collected by the monitor. Then the resource manager performs a sharing operation on tenant A's LC2 container, that is, selects the container BE1 with the highest weighted priority from the BE containers and expands the set of CPUs that it can run on from CPU4-6 to CPU2-6. Meanwhile, the resource manager discovered that tenant B's LC1 container's QoS was compromised due to sharing. Even though the utilization rates of CPU7 and CPU8, P3 and P4, were both less than the sharing threshold β, the resource manager still evicted tenant B's LC1 container, reducing the set of CPUs that could run on the LC1 container's exclusive core from CPU7-10 to CPU9-10.

5. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 4.

Images