Container orchestration method and system applied to kubernetes

Abstract

A container orchestration method and system applied to Kubernetes, the method comprising: a Webhook component intercepts and scans a Pod creation request to determine whether a first environment variable is included in a target container, the first environment variable being used to indicate that the target container is a sidecar container; in the case where the target container includes the first environment variable, the target container is Downward API formalized; after the Downward API formalization is completed, the API server writes the Pod configuration information of the target Pod to a distributed database, and the sidecar container is Downward API formalized, so as to facilitate subsequent modification of the startup command of the container through the Downward API, implement the exit mechanism of the sidecar container, and realize the universality of the container exit scheme.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a container orchestration method and system for Kubernetes. Background Technology

[0002] Kubernetes is an open-source system for automatically deploying, scaling, and managing containerized target applications. Currently, Kubernetes has become a universal standard in the global container orchestration field, and increasingly, short tasks such as artificial intelligence (AI) computing are adopting Kubernetes for job orchestration.

[0003] However, when users use Kubernetes short task orchestration in production environments, they find that Pods, as the smallest orchestration unit in Kubernetes, become unusable when faced with a mix of long and short task containers. This is because when the main container, acting as a short task, finishes its computation and exits, the sidecar container, acting as a long task, continues to run. This not only causes the resources occupied by the Pod to remain idle, resulting in a significant waste of computational, storage, and network resources, but also causes the Kubernetes short task controller to consider the task incomplete, leading to a misjudgment of the task status as a timeout.

[0004] To address this issue, existing solutions can only be tailored to specific scenarios and cannot adapt to diverse container orchestration environments. Therefore, a more flexible container exit control scheme is urgently needed to improve the universality of the container exit mechanism. Summary of the Invention

[0005] This application provides a container orchestration method and system for Kubernetes, which addresses the problem of low universality of container exit mechanisms in container orchestration within the K8S system.

[0006] Firstly, a container orchestration method for Kubernetes is provided, comprising: an API server obtaining a Pod creation request for a target Pod sent by a short task controller; a Webhook component intercepting the Pod creation request and scanning the container configuration information of the target container in the Pod creation request to determine whether the target container includes a predefined first environment variable, the first environment variable being used to indicate that the target container is a sidecar container, and the target container being any container included in the target Pod; if the target container includes the first environment variable, the Webhook component performing a Downward API formalization transformation on the target container, the Downward API formalization transformation including: copying the content of the startup command of the target container into the Pod annotations field, and mounting the Pod annotations field to a second environment variable in a Downward API manner; and the API server writing the Pod configuration information of the target Pod to a distributed database after the Downward API formalization transformation is completed.

[0007] In one possible implementation, the method further includes: the sidecar controller determining that a container exit mechanism needs to be executed on the target container; if the target Pod is located on a real node, the sidecar controller stops the target container by invoking the Container In-Process Interface (CRI); or, if the target Pod is located on a virtual node, the sidecar controller executes a Kubernetes in-place upgrade strategy on the target container, wherein executing the Kubernetes in-place upgrade strategy includes: stopping the target container by modifying the image field and the Podannotations field of the target container.

[0008] In one possible implementation, the method further includes: the sidecar controller determining that a container exit mechanism needs to be executed on the target container, including: an event listener monitoring the working status of containers in the target Pod; the event listener determining whether all main containers in the target Pod have exited if the container restart policy of the target Pod is never, wherein never means that the containers are not restarted under any circumstances; and the event listener triggering the sidecar controller to execute the container exit mechanism on the target container if all main containers in the target Pod have exited.

[0009] In one possible implementation, the method further includes: the sidecar controller determining that a container exit mechanism needs to be executed on the target container, including: an event listener monitoring the working status of containers in the target Pod; the event listener, if the container restart policy of the target Pod is onfailure, determining whether all main containers in the target Pod have successfully executed their tasks and exited, wherein onfailure refers to restarting abnormal containers when containers in the Pod exit abnormally; the event listener, if all main containers in the target Pod have successfully executed their tasks and exited, triggering the sidecar controller to execute the container exit mechanism on the target container.

[0010] Secondly, this application provides a container orchestration system for Kubernetes, comprising: an API server for receiving a Pod creation request for a target Pod sent by a short task controller; a Webhook component for intercepting the Pod creation request and scanning the container configuration information of the target container in the Pod creation request to determine whether the target container includes a predefined first environment variable, the first environment variable being used to indicate that the target container is a sidecar container, and the target container being any container included in the target Pod; the Webhook component is further configured to perform a Downward API formalization transformation on the target container when the target container includes the first environment variable, the Downward API formalization transformation including: copying the content of the startup command of the target container into the Podannotations field, and mounting the Pod annotations field to a second environment variable in a Downward API manner; the API server is further configured to, after the Downward API formalization transformation is completed, write the Pod configuration information to the target Pod.

[0011] Thirdly, a container orchestration method for Kubernetes is provided, comprising: a sidecar controller determining that a container exit mechanism needs to be executed on a target container in a target Pod, wherein the sidecar container is set on the master node in the Kubernetes cluster; when the target Pod is located on a real node, the sidecar controller sends an instruction message to an interaction component to instruct the target container to be stopped; the interaction component is set on a node in the Kubernetes cluster and is used to interact with the container through the Container In-Process Interface (CRI); the interaction component calls the CRI to stop the target container.

[0012] Fourthly, a container orchestration system for Kubernetes includes: a sidecar controller, located in the master node of a Kubernetes cluster; and an interaction component, located in a node of the Kubernetes cluster, used to interact with containers via the Container In-Process Interface (CRI). The sidecar controller is used to determine whether a container exit mechanism needs to be executed on a target container in a target Pod; and, if the target Pod is located on a real node, to instruct the interaction component to stop the target container. The interaction component is used to invoke the CRI to stop the target container.

[0013] Fifthly, an electronic device is provided, including a processor configured to call a computer program from memory, wherein when the computer program is executed, the processor is configured to execute a method performed by an API server in the first aspect or any possible implementation thereof, or to execute a method performed by a Webhook component in the first aspect or any possible implementation thereof, or to execute a method performed by a sidecar controller in the first aspect or any possible implementation thereof.

[0014] In a sixth aspect, a computer-readable storage medium is provided for storing a computer program comprising code for executing a method executed by an API server in the first aspect or any possible implementation thereof, or code for executing a method executed by a Webhook component in the first aspect or any possible implementation thereof, or code for executing a method executed by a sidecar controller in the first aspect or any possible implementation thereof.

[0015] In a seventh aspect, a computer program product is provided, comprising a computer program including code for executing a method performed by an API server in the first aspect or any possible implementation thereof, or code for executing a method performed by a Webhook component in the first aspect or any possible implementation thereof, or code for executing a method performed by a sidecar controller in the first aspect or any possible implementation thereof.

[0016] In an eighth aspect, an electronic device is provided, including a processor configured to call a computer program from a memory, wherein when the computer program is executed, the processor is configured to perform the method executed by the sidecar controller in the third aspect above, or to perform the method executed by the interactive component in the third aspect above.

[0017] A ninth aspect provides a computer-readable storage medium for storing a computer program including code for performing the methods executed by the sidecar controller in the third aspect above, or code for performing the methods executed by the interactive component in the third aspect above.

[0018] In a tenth aspect, a computer program product is provided, comprising a computer program including code for performing the methods executed by the sidecar controller in the third aspect above, or code for performing the methods executed by the interactive component in the third aspect above.

[0019] In this embodiment of the application, during the Pod creation process, the Kubernetes Webhook mechanism can identify the sidecar container based on the first environment variable and perform a Downward API formalization on the sidecar container, so that the container's startup command can be modified through the Downward API to realize the exit mechanism of the sidecar container. This solution can bypass compatibility limitations and realize the universality of the container exit solution. Users do not need to adapt for each scenario. One solution can solve a variety of short task container orchestration scenarios. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of the K8S architecture according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram illustrating an application scenario of the K8S architecture according to another embodiment of this application;

[0023] Figure 3 This is a flowchart illustrating a container arrangement method according to an embodiment of this application;

[0024] Figure 4 This is a flowchart illustrating a container arrangement method according to another embodiment of this application;

[0025] Figure 5 This is a schematic diagram illustrating the specific process of a container orchestration method applied to Kubernetes according to an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the structure of a device 600 according to an embodiment of this application. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0028] First, the terms used in this application will be explained.

[0029] Kubernetes (K8S) is an open-source system for automatically deploying, scaling, and managing containerized applications. Alternatively, it's a container orchestration system used for container management and load balancing between containers.

[0030] A container can be understood as a special type of process that can divide resources, files, state, and configuration into an independent space. This independent space can be moved to any host machine without affecting the operation of the software within it.

[0031] Pod: The smallest unit of container orchestration in Kubernetes. A Pod can contain multiple containers. Typically, a server cluster can contain multiple nodes, and each node can contain multiple Pods.

[0032] Short task container orchestrators, also known as short task controllers, are container orchestration methods specifically designed for temporary / short-term computation tasks. A computation task within a Pod is considered complete only after all containers in the Pod have exited. Typical tasks requiring short task orchestration include AI computation and search recommendation / advertising computation tasks. In Kubernetes (K8S), short task container orchestrators can include Kubernetes job controllers (or job controllers as referred to below). Alternatively, K8S can also support third-party short task controllers.

[0033] Short task containers: Containers that run for a period of time or exit voluntarily immediately after computation. In short task container orchestration scenarios, the main container for all user task computations is a short task container.

[0034] Long-running containers are containers that run continuously and generally do not terminate voluntarily. For example, long-running containers are commonly used in web applications, log collection programs, and system administration programs. Long-running containers generally refer to sidecar containers.

[0035] Main container: The container used to accomplish the main purpose, generally referring to the container that handles task calculations.

[0036] Sidecar container: A container that assists the main container in completing computations. For example, a Pod contains a main container and multiple sidecar containers. This application is applicable to scenarios where the main container is a short-task container and the sidecar is a long-task container.

[0037] Command / args: The container's startup command.

[0038] ETCD is a distributed key-value database used for configuration sharing and service discovery. It provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. Applications can read and write data from ETCD. Key-value storage refers to storing data in a hierarchically organized directory, such as in a standard file system.

[0039] Webhook component: A callback component reserved for users in Kubernetes, which is a custom extensible mechanism.

[0040] Container restart policy: This is the default policy for restarting containers within a Pod in Kubernetes. This policy applies to all containers within the Pod. The restart policy only affects container restart actions of the kubelet on the same node. When a container in a Pod exits, the kubelet calculates the restart delay using an exponential rollback mechanism, such as 10s, 20s, or 40s. If the container has been running for a period of time without encountering problems, the kubelet resets the restart rollback timer for that container.

[0041] Specifically, the Pod configuration parameters include a `restart policy` field, whose possible values ​​are `always`, `onfailure`, and `never`. As an example, the different values ​​for the container restart policy correspond to the following scenarios:

[0042] always: In any situation, if the containers in the Pod are not running, these containers will always be automatically restarted.

[0043] onfailure: Restarts the container in the Pod only when it exits abnormally. The exit code is 0 when the container exits normally, and a non-zero exit code when it exits abnormally.

[0044] never: Never restart the container under any circumstances.

[0045] It should be noted that in short task container orchestration scenarios, there are only two possible values: never and onfailure; there is no possible value: always.

[0046] Container runtime interface (CRI): A set of interfaces defined by Kubernetes for interacting with the container runtime.

[0047] OpenKruise is an open-source cloud-native application automation management suite from Alibaba, which is based on the Kubernetes architecture and extends the standard.

[0048] Figure 1 This is a schematic diagram illustrating an application scenario of the K8S architecture according to an embodiment of this application. For example... Figure 1 As shown, a K8S cluster consists of a master node and nodes.

[0049] The Master node is the control node of the cluster, responsible for managing and controlling the entire cluster and executing control commands in Kubernetes. The Master node mainly includes the following components.

[0050] The Application Programming Interface Server (API server) serves as the unified entry point for the cluster and the coordination center for all components within it. Cluster administrators and users access Kubernetes (K8S) through the API server. All operations related to adding, deleting, modifying, searching, and monitoring object resources are processed by the API server before being submitted to ETCD storage. Other components need to query or modify data through the API server; only the API server has direct access to ETCD.

[0051] The term "administrator" can refer to backend staff who run and maintain the Kubernetes (K8S) architecture, or cloud product providers, such as those offering log services or monitoring services. The term "user" can refer to personnel who utilize the resources provided by the K8S architecture to run containers for application development.

[0052] The controller manager is the automated control center for resource objects in Kubernetes, used to maintain and manage the cluster's state, such as fault detection, automatic scaling, and updates. The controller manager also includes job controllers, which are used to orchestrate containers designed for temporary / short-term computational tasks. As an example, the job controller can also be replaced by other third-party short-task container orchestration controllers.

[0053] Scheduler: Responsible for resource scheduling in Kubernetes, such as scheduling Pods to appropriate machines according to a predefined scheduling policy.

[0054] ETCD: A distributed key-value database used to store the state data of the entire cluster. For example, information about Pods, services, and other objects.

[0055] Node: Used to run workloads assigned by the Master node. A Node mainly includes the following components.

[0056] Kubelet is the agent of the Master node on the Node nodes. It is used to manage the lifecycle of containers running on the local node, such as container creation, startup, and shutdown. It also works with the Master node to implement basic cluster management functions.

[0057] Kube-proxy: Responsible for implementing network proxy for Pods, used to maintain network rules and load balancing.

[0058] In Kubernetes, native short-task container orchestration controllers (e.g., Job controllers) determine task completion and success based on the Pod as the basic unit. However, a Pod often contains not only the main container for the task but also sidecar containers responsible for log collection and routine maintenance tasks. The short-task controller needs all containers in a Pod to exit before determining that the Pod's computation task is complete, and all containers must return success before the Pod is considered to have succeeded. However, these different containers are often maintained by different departments, or even different platforms or companies. For example, in some short-task computation scenarios, the main container is provided by the user, while the sidecar containers are provided by the carrier. The main container is a short-task container, while the sidecar containers are long-task containers. Because the sidecar containers cannot detect when the main container exits, and the user is unaware of which sidecar containers correspond to the main container, the sidecar containers continue running even after the main container has completed its task computation. When the short-task controller detects that a container in the Pod has not completed, it will wait indefinitely until it determines that the task has failed due to a timeout. This wastes computing and storage resources and can cause misjudgments by the short-task controller, making short-task container orchestration impossible in this scenario.

[0059] There is no universal or productized solution to this problem in the industry; solutions are usually customized for specific scenarios. Therefore, a more widely applicable proactive container exit mechanism is needed. This mechanism should proactively stop long-task sidecar containers when the main container is detected to be complete, thereby saving resources and improving the management efficiency of short-task container orchestration in the Kubernetes architecture.

[0060] To address the aforementioned issues, this application adopts a container orchestration scheme for Kubernetes. The main idea is to implement a container exit mechanism by annotating sidecar containers with environment variables. In this scheme, administrators or users can identify sidecar containers that need to be processed using predefined environment variable identifiers in the Pod template. Subsequently, when the short task controller starts job orchestration and begins creating Pods, the Webhook component and Jobcontroller select the corresponding sidecar containers based on these identifiers for modification and processing, and, in conjunction with the state of the main container, choose an appropriate time to exit these sidecar containers.

[0061] To achieve the above solution, this application's embodiment modifies the functionality of the Webhook component in the K8S architecture and adds a sidecar controller to the Master node. The following will combine... Figure 2 This application introduces the modified K8S architecture based on an embodiment.

[0062] Figure 2 This is a schematic diagram illustrating an application scenario of the K8S architecture according to another embodiment of this application. For example... Figure 2 As shown, this embodiment adds a sidecar controller, or sidecar terminator controller, to the Master node. The sidecar controller implements the container exit mechanism for sidecar containers within the Pod. The sidecar controller also includes an event listener. The event listener monitors the state of containers within the Pod and initiates a judgment logic process when the container state changes to determine whether to trigger the sidecar controller to execute the container exit mechanism.

[0063] Furthermore, this application also defines the functionality of the Webhook component. The Webhook component can be used to intercept and modify control signals sent from the API server to ETCD, such as Pod creation requests and container startup commands (Command / args). Optionally, the Webhook component can be set in any module of the Master node, such as the API server or the Controller manager. In some examples, the Webhook component can also be set in a node.

[0064] Optionally, such as Figure 2 As shown, the embodiments of this application are applicable to Kubernetes architectures with interactive components installed. An interactive component refers to a component deployed on a node in the cluster that can interact with containers or container runtime via CRI. As an example, an interactive component can be a component running a daemon process. As an example and not a limitation, an interactive component can include the kruise-daemon component in OpenKruise.

[0065] The following section describes a scheme that utilizes the sidecar-daemon component to execute container stop policies, thereby implementing a proactive exit mechanism for sidecar containers. Here, the container stop policy refers to a strategy employed by the sidecar controller to coordinate the daemon process and stop the container via a CRI (Container Restart Request). For example, container stop policies include, but are not limited to, OpenKruise's container restart request (CRR) policy.

[0066] It should be understood that Figure 1 or Figure 2 The application scenarios described above are merely examples and not limitations. In practice, appropriate modifications and additions or subtractions can be made to the above scenarios, and the solutions of the embodiments of this application will still be applicable.

[0067] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0068] Figure 3 This is a flowchart illustrating a container arrangement method according to an embodiment of this application. The steps in this method can be derived from... Figure 2 Each component within it executes. For example... Figure 3 As shown, the method includes the following.

[0069] S301. The API server obtains the Pod creation request for the target Pod sent by the short task controller.

[0070] Optionally, the aforementioned short task controller can be a Job controller in Kubernetes, or any other third-party short task container orchestration controller. The aforementioned API server may include... Figure 3 API server in the context of [the API server].

[0071] Optionally, the Pod creation request includes Pod configuration information, which may include container configuration information and information at the Pod level itself. The container configuration information may include environment variables and other information about the container running within the Pod. The Pod-level information may include the Pod's name, identity (identifier, ID), security policies, mounted storage disks, and other information.

[0072] S302 The Webhook component intercepts the Pod creation request and scans the container configuration information of the target container in the Pod creation request to determine whether the target container includes a predefined first environment variable. The first environment variable is used to indicate that the target container is a sidecar container, and the target container is any container included in the target Pod.

[0073] Optionally, when creating a sidecar container, the administrator, user, or system can define the state of the first environment variable within the sidecar container to facilitate identification of the first environment variable during subsequent processing. For example, assuming the first environment variable is represented as SIDECAR_TERMINATION, then defining SIDECAR_TERMINATION = "true" indicates that the container is a sidecar container.

[0074] S303. When the target container includes the first environment variable, the Webhook component performs a Downward API formalization on the target container. The Downward API formalization includes copying the contents of the target container's startup command into the Pod annotations field and mounting the Pod annotations field to the second environment variable in the manner of Downward API.

[0075] The Pod annotations field is an annotation field in a Pod. The Downward API is an API interface in Kubernetes. The Downward API can expose information to running containers through environment variables, which allows containers to use their own or the cluster's information without having to obtain it through a Kubernetes client or API server.

[0076] After the formal transformation of the Downward API described above, since the container's startup command (command / args) can be executed by driving the Downward API through environment variables, the container's startup command (command / args) can be modified by changing the contents of the environment variables in the subsequent process, thereby bypassing the limitations of the Kubernetes architecture.

[0077] As an example, the second environment variable mentioned above can be represented as START_COMMAND. Mounting the Pod annotations field to the second environment variable using the Downward API can be represented as: Annotations[START_COMMAND] = "xxxx".

[0078] S304. After the Downward API formalization is completed, the API server writes the Pod configuration information of the target Pod to the distributed database.

[0079] Optionally, the aforementioned distributed database includes ETCD, or it could be other types of distributed databases.

[0080] Optionally, after the Downward API formalization is completed, the Webhook component returns a success status code to the API server. The API server then writes the Pod configuration information from the modified Pod creation request into ETCD to await subsequent scheduling and container creation execution operations.

[0081] In this embodiment of the application, during the Pod creation process, the Kubernetes Webhook mechanism can identify the sidecar container based on the first environment variable and perform a Downward API formalization on the sidecar container, so that the container's startup command can be modified through the Downward API to realize the exit mechanism of the sidecar container. This solution can bypass compatibility limitations and realize the universality of the container exit solution. Users do not need to adapt for each scenario. One solution can solve a variety of short task container orchestration scenarios.

[0082] Since the commands and arguments of containers within a Kubernetes Pod cannot be directly modified, this application utilizes Kubernetes Webhooks to formally modify the sidecar container using a Downward API. Upon Pod creation, the sidecar container's commands and arguments are written to environment variables and mounted to Pod Annotations via the Downward API. Because Pod Annotations are modifiable resources, when container exit is required, the sidecar container's commands and arguments can be indirectly modified by modifying the Pod Annotations. This bypasses compatibility limitations, allowing any container to reuse the same image that can be quickly and proactively exited, thus achieving high versatility.

[0083] After a Pod is scheduled and a container is successfully created, the container begins to perform task computation. During the computation process, an event listener monitors the status of the Pod. When a container in the Pod changes, the event listener determines whether a container exit mechanism needs to be executed for the sidecar container and triggers the sidecar controller to execute the container exit mechanism for the target container.

[0084] In some examples, the sidecar controller determines that a container exit mechanism needs to be implemented for the target container, including: an event listener monitoring the working status of containers in the target Pod; if the container restart policy of the target Pod is "never", the event listener determines whether all the main containers in the target Pod have exited, where "never" means that the containers will not be restarted under any circumstances; if all the main containers in the target Pod have exited, the event listener triggers the sidecar controller to implement the container exit mechanism for the target container.

[0085] In some examples, the sidecar controller determines that a container exit mechanism needs to be implemented for the target container, including: an event listener monitoring the working status of containers in the target Pod; if the container restart policy of the target Pod is onfailure, the event listener determines whether all the main containers in the target Pod have successfully completed their tasks and exited, where onfailure refers to restarting abnormal containers when containers in the Pod exit abnormally; if all the main containers in the target Pod have successfully completed their tasks and exited, the event listener triggers the sidecar controller to implement the container exit mechanism for the target container.

[0086] After the sidecar controller determines that a container exit mechanism needs to be executed, it can determine whether the node where the Pod resides is a real node or a virtual node, and execute different container exit strategies based on the node type. The aforementioned real node can refer to a regular node, indicating that the Pod is running on a physical host. The aforementioned virtual node can refer to Virtual-kubelet; a virtual node is simply a program that is not backed by a real physical host and cannot be interacted with via CRI calls.

[0087] In some examples, the sidecar controller determines that a container exit mechanism needs to be implemented for the target container; if the target Pod is located on a live node, the sidecar controller stops the target container by invoking the CRI.

[0088] As an example, the sidecar controller stops the target container by invoking a CRI, including: the sidecar controller sending an instruction message to the interactive component to instruct the target container to be stopped, and the interactive component invoking a CRI to stop the target container.

[0089] Interactive components refer to components deployed on nodes in the cluster that can interact with containers or container runtime via CRI. As an example, an interactive component could be a component running a daemon process. As an example, and not a limitation, interactive components can include the kruise-daemon component in OpenKruise.

[0090] For example, the sidecar controller stops the container via CRI by coordinating the daemon process. Alternatively, it can use a container stopping strategy to stop the target container. Here, the container stopping strategy refers to a method employed by the sidecar controller to stop the container via CRI by coordinating the daemon process. As an example, container stopping strategies include, but are not limited to, OpenKruise's CRR strategy.

[0091] In this embodiment of the application, an interactive component is set in the node. This interactive component can interact with the container or the container in progress through the container CRI. Therefore, the sidecar controller can implement the container exit mechanism by calling the CRI. The operation is simple, highly versatile, and takes less time, which can improve the efficiency of container exit.

[0092] In some other examples, where the target Pod is located on a virtual node, the sidecar controller executes an in-place Kubernetes upgrade policy on the target container. Executing the in-place Kubernetes upgrade policy includes stopping the target container by modifying the target container's image field and Pod annotations field. The purpose of modifying the Pod annotations field is to modify the target container's startup command.

[0093] For Pods running on Virtual-Kubelet virtual nodes, since the CRI interface cannot be called, an in-situ upgrade strategy will be executed.

[0094] The principle behind the in-situ upgrade strategy is as follows: a Pod can stop its container, modify the image fields within the Pod, and then rebuild a new container image. However, in the original approach, Kubernetes only supports modifying the Pod's image fields and does not allow modification of Command / args. This requires the replacement image to be strongly compatible with the original image. Since different users run different types of containers, this approach lacks universality.

[0095] Therefore, this embodiment of the application modifies the Command / args using the Downward API via the Webhook component before the Pod is created, bypassing Kubernetes restrictions. Thus, the Command / args can be indirectly modified by altering Pod Annotations within the in-place upgrade strategy. Specifically, the Command / args can be modified to stop execution via the Downward API. After the Pod stops the container and the Command / args are modified, the newly created container image can be directly stopped. This approach utilizes an in-place upgrade strategy but does not require customization for each image to ensure compatibility with different image startup commands (Command / args). Therefore, it avoids strong compatibility issues between images and does not require consideration of the container type, thus improving the versatility of the solution.

[0096] In this embodiment, modification of the container is avoided. The following techniques are mainly used to solve the problem: for ordinary nodes, they can be stopped by calling CRI; for virtual nodes, the Kubernetes Pod in-situ upgrade strategy is used to replace the original sidecar container image with an image that can be quickly and actively exited, thereby achieving the purpose of stopping it.

[0097] This application proposes a general container proactive exit scheme, which enables the main container to proactively exit other running containers after completing computation, thereby saving resources and guiding the short task controller in Kubernetes to correctly determine the task completion status.

[0098] This application's embodiments are not only applicable to the native Kubernetes Job controller, but also to any third-party extended short-task container orchestration scenario. It can monitor the completion status of the user's main container's computation; when the main container completes its computation and exits, the sidecar container is stopped, guiding the Job controller and other short-task controllers to complete the correct task status determination, saving resources. This solution can handle various Kubernetes short-task container orchestration scenarios without requiring modifications to the main container or sidecar container, exhibiting high versatility.

[0099] Figure 4 This is a flowchart illustrating a container arrangement method according to an embodiment of this application. The steps in this method can be derived from... Figure 2 Each component within it executes. For example... Figure 4 As shown, the method includes the following.

[0100] S401, the sidecar controller determines that a container exit mechanism needs to be executed on the target container in the target Pod, and the sidecar container is set in the master node in the Kubernetes cluster.

[0101] S402. When the target Pod is located on a real node, the sidecar controller sends an instruction to the interaction component to instruct the target container to stop. The interaction component is set in a node in the Kubernetes cluster and is used to interact with the container through the Container In-Process Interface (CRI).

[0102] Interactive components refer to components deployed on nodes in the cluster that can interact with containers or container runtime via CRI. As an example, an interactive component could be a component running a daemon process. As an example, and not a limitation, interactive components can include the kruise-daemon component in OpenKruise.

[0103] For example, the sidecar controller stops the container via CRI by coordinating the daemon process. Alternatively, it can use a container stopping strategy to stop the target container. Here, the container stopping strategy refers to a method employed by the sidecar controller to stop the container via CRI by coordinating the daemon process. As an example, container stopping strategies include, but are not limited to, OpenKruise's CRR strategy.

[0104] S403. The interactive component calls CRI to stop the target container.

[0105] In this embodiment of the application, an interactive component is set in the node. This interactive component can interact with the container or the container in progress through the container CRI. Therefore, the sidecar controller can implement the container exit mechanism by calling the CRI. The operation is simple, highly versatile, and takes less time, which can improve the efficiency of container exit.

[0106] Optionally, in S401, the specific method by which the sidecar controller determines whether a container exit mechanism needs to be executed on the target container in the target Pod can be found in [reference needed]. Figure 3 For the sake of brevity, the relevant descriptions in the document will not be repeated here.

[0107] Figure 5 This is a schematic flowchart illustrating a container orchestration method applied to Kubernetes according to an embodiment of this application. Optionally, Figure 5 The Job controller in this application can be replaced by other third-party short task container orchestration controllers, and this application does not limit this. Figure 5 As shown, the method includes the following.

[0108] S501. During the Pod creation process in the Job controller, a Pod creation request is sent to the API server, and the API server receives the Pod creation request accordingly.

[0109] Optionally, the Pod creation request includes Pod configuration information, which may include container configuration information and information at the Pod level itself. The container configuration information may include environment variables and other information about the container running within the Pod. The Pod-level information may include the Pod's name, identity (identifier, ID), security policies, mounted storage disks, and other information.

[0110] S502. Before the API server writes the Pod configuration information in the Pod creation request to ETCD, the Webhook component intercepts the Pod creation request.

[0111] S503: The Webhook scans the container configuration information in the Pod creation request to see if any container includes a predefined first environment variable. The first environment variable indicates that the container is a sidecar container. If a container contains the first environment variable, it means that the container is a sidecar container, and then S504 is executed, which is to perform the Downward API formalization transformation.

[0112] Optionally, if no container in the Pod contains the first environment variable, no action is required, and the API server can write the Pod configuration information to ETCD.

[0113] As an example, when the Job controller creates a Pod, the Pod creation request is intercepted by the Webhook component. Before the API Server writes the Pod configuration information to ETCD and completes object storage, the Webhook component scans the containers in the Pod to see if they contain the predefined first environment variable. If they do, it means that the container is a sidecar container that needs to be processed, and it is then transformed into a Downward API formalization.

[0114] S504. Perform a Downward API formalization transformation on the sidecar container.

[0115] The formal modifications to the Downward API include: copying the contents of the target container's startup command into the Podannotations field, and mounting the Pod annotations field to a second environment variable in the manner of the Downward API.

[0116] After the Downward API is formalized, the container's startup command (command / args) can be executed by driving the Downward API through environment variables. Therefore, in subsequent processes, the container's startup command (command / args) can be modified by changing the contents of environment variables, thereby bypassing the limitations of the Kubernetes architecture.

[0117] S505. After the formal transformation of the Downward API is completed, the Webhook component returns a success status code to the API server. The API server writes the Pod configuration information in the modified Pod creation request into ETCD to await subsequent scheduling and container creation and execution operations.

[0118] S506. After a Pod is scheduled and a container is successfully created, the container begins to perform task computation. During the computation process, the event listener monitors the status of the Pod. When the container in the Pod changes, the event listener determines whether the container exit mechanism needs to be executed for the sidecar container.

[0119] The event listener's decision-making logic can be found in [link to relevant documentation]. Figure 5 The above judgment logic process includes the following:

[0120] A1. Determine if any container carries a predefined first environment variable; if it carries the first environment variable, determine the Pod's container restart policy; if it does not carry the first environment variable, end the determination process.

[0121] A2. If the Pod's container restart policy is never, then execute A4.

[0122] A3. If the Pod's container restart policy is onfailure, then execute A5.

[0123] A4. Determine if all main containers in the Pod have exited. If all have exited, determine that the container exit mechanism needs to be executed and trigger the sidecar controller to execute the container exit mechanism; if not all have exited, end the judgment logic process.

[0124] A5. Determine if all main containers in the Pod have successfully completed their tasks and exited. If yes, determine that the container exit mechanism needs to be executed and trigger the sidecar controller to execute the container exit mechanism; otherwise, end the judgment logic process.

[0125] The exit code is 0 when the container exits normally, and a non-zero exit code when it exits abnormally.

[0126] S507. If the event listener determines that the sidecar container needs to exit, it triggers the sidecar controller to execute the container exit mechanism.

[0127] See also Figure 5 The sidecar controller implements the container exit mechanism, which includes the following.

[0128] B1. Determine whether the node containing the Pod is a real node or a virtual node; if it is a real node, execute B2; if it is a virtual node, execute B3.

[0129] B2. For Pods running on real nodes, execute the container stop policy.

[0130] For example, the sidecar controller can inform the interactive component via a protocol that the target container needs to be stopped, and instruct it to call CRI to exit the target container.

[0131] B3. For Pods running on virtual nodes, stop the sidecar container by executing Kubernetes' in-place upgrade policy.

[0132] For Pods running on Virtual-Kubelet virtual nodes, since CRI cannot be invoked, an in-situ upgrade strategy will be executed.

[0133] Specifically, a new container image can be created by modifying the image field of the target container, and the Command / args can be changed to stop running via the Downward API. Thus, after the Pod stops the container and the Command / args is modified, the newly created container image can stop running directly.

[0134] Optionally, once the controller stops the sidecar container in a timely manner, the Job controller can listen for changes in the Pod's state and detect that the Pod has reached the terminated state. At this point, the job controller can correctly determine the completion status of the task computation based on the success or failure status of the Pod.

[0135] In this embodiment, since the commands / args of containers within a Kubernetes Pod cannot be directly modified, this application utilizes Kubernetes Webhook to formally modify the sidecar container using a Downward API. Upon Pod creation, the commands / args of the sidecar container are written to environment variables and mounted to PodAnnotations via the Downward API. Because PodAnnotations are modifiable resources, during in-place upgrades, the commands / args of the sidecar container can be indirectly modified by modifying PodAnnotations, thereby bypassing compatibility limitations. This allows any container to reuse the same image that can be quickly and proactively exited, achieving a high degree of versatility.

[0136] In this embodiment, CRI or Kubernetes in-situ upgrade mechanism is used to correctly guide Kubernetes to perform short-task container orchestration without modifying the container image, while saving computing, storage, network and other resources.

[0137] Figure 6 This is a schematic diagram of the structure of a device 600 according to an embodiment of this application. The device 600 is used to execute the methods performed by various components in Kubernetes in the embodiments of this application.

[0138] The device 600 includes a processor 610, which is configured to execute computer programs or instructions stored in the memory 620, or to read data stored in the memory 620, to perform the methods described in the above method embodiments. Optionally, there may be one or more processors 610.

[0139] Optionally, such as Figure 6As shown, the device 600 also includes a memory 620 for storing computer programs or instructions and / or data. The memory 620 may be integrated with the processor 610 or may be disposed separately. Optionally, there may be one or more memories 620.

[0140] Optionally, such as Figure 6 As shown, the device 600 also includes a communication interface 630, which is used for receiving and / or transmitting signals. For example, the processor 610 is used to control the communication interface 630 to receive and / or transmit signals.

[0141] Optionally, the device 600 is used to implement the operations performed by the components in K8S in the various method embodiments described above.

[0142] For example, processor 610 is used to execute computer programs or instructions stored in memory 620 to implement the related operations performed by the API server, or the related operations performed by the Webhook component, or the related operations performed by the sidecar controller, or the related operations performed by the interactive component, or the related operations performed by the event listener.

[0143] It should be pointed out that, Figure 6 The device 600 can be any of the components in the foregoing embodiments, or a component (such as a chip) or functional module of each component, and is not limited here.

[0144] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU (which can be understood as a type of microprocessor), or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0145] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0146] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0147] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the method executed by the API server, or the steps in the method executed by the Webhook component, or the steps in the method executed by the sidecar controller, or the steps in the method executed by the interactive component, or the steps in the method executed by the event listener.

[0148] Accordingly, this application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, causes the processor to implement the steps in the method executed by the API server, or the steps in the method executed by the Webhook component, or the steps in the method executed by the sidecar controller, or the steps in the method executed by the interactive component, or the steps in the method executed by the event listener.

[0149] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0151] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0152] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0153] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0154] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0155] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0156] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0157] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A container orchestration method applied to Kubernetes, characterized in that, The method comprises the following steps: an API server acquires a Pod creation request of a target Pod sent by a short task controller; a Webhook component intercepts the creation request of the Pod, and scans container configuration information of a target container in the Pod creation request to determine whether the target container includes a predefined first environment variable, the first environment variable being used to indicate that the target container is a sidecar container, the target container being any container included in the target Pod; the Webhook component performs Downward API formalization modification on the target container in the case that the target container includes the first environment variable, the Downward API formalization modification comprising: copying content in a startup command of the target container into a Pod annotations field, and mounting the Pod annotations field into a second environment variable in the form of Downward API; the API server writes Pod configuration information of the target Pod into a distributed database after the Downward API formalization modification is completed; the method further comprises the following steps: the sidecar controller determines that a container exit mechanism needs to be performed on the target container; in the case that the target Pod is located in a virtual node, the sidecar controller performs an in-place upgrade strategy of Kubernetes on the target container, wherein the performing the in-place upgrade strategy of Kubernetes comprises: stopping the target container by modifying an image field and the Pod annotations field of the target container.

2. The method of claim 1, wherein, the method further comprises the following steps: in the case that the target Pod is located in a real node, the sidecar controller stops the target container by calling a container runtime interface CRI.

3. The method of claim 2, wherein, the sidecar controller determining that a container exit mechanism needs to be performed on the target container comprises: an event listener listens to a working state of a container in the target Pod; the event listener determines whether all primary containers in the target Pod have exited in the case that a container restart policy of the target Pod is never, wherein the never means that a container is not restarted under any circumstances; the event listener triggers the sidecar controller to perform a container exit mechanism on the target container in the case that all primary containers in the target Pod have exited.

4. The method of claim 2, wherein, the sidecar controller determining that a container exit mechanism needs to be performed on the target container comprises: an event listener listens to a working state of a container in the target Pod; the event listener determines whether all primary containers in the target Pod have successfully performed a task and exited in the case that a container restart policy of the target Pod is onfailure, wherein the onfailure means that an abnormal container is restarted when a container in a Pod abnormally exits. The event listener triggers the sidecar controller to execute the container exit mechanism on the target container in a case that all the main containers in the target Pod execute the task successfully and exit.

5. A container orchestration system applied to Kubernetes, characterized in that, Comprise: An API server configured to receive a Pod creation request of a target Pod sent by a short task controller; A webhook component configured to intercept the creation request of the Pod, and scan container configuration information of a target container in the Pod creation request to determine whether the target container includes a predefined first environment variable, the first environment variable being used to indicate that the target container is a sidecar container, the target container being any container included in the target Pod; The webhook component is further configured to perform Downward API formalization on the target container in a case that the target container includes the first environment variable, the Downward API formalization comprising: copying content in a startup command of the target container to a Pod annotations field, and mounting the Pod annotations field to a second environment variable in a Downward API manner; The API server is further configured to write Pod configuration information of the target Pod to a distributed database after the Downward API formalization is completed; The system further comprises a sidecar controller configured to: determine that the container exit mechanism needs to be executed on the target container; and execute an in-place upgrade strategy of Kubernetes on the target container in a case that the target Pod is located in a virtual node, wherein the execution of the in-place upgrade strategy of Kubernetes comprises: stopping the target container by modifying an image field and the Pod annotations field of the target container.

6. The system of claim 5, wherein, The sidecar controller is further configured to: stop the target container by invoking a container runtime interface (CRI) in a case that the target Pod is located in a real node.

7. The system of claim 6, wherein, The sidecar controller further comprises an event listener configured to: listen to a working state of a container in the target Pod; determine whether all the main containers in the target Pod have exited in a case that a restart policy of the target Pod is never, wherein the never means that the container is not restarted under any circumstances; and trigger the sidecar controller to execute the container exit mechanism on the target container in a case that all the main containers in the target Pod have exited.

8. The system of claim 6, wherein, The sidecar controller further comprises an event listener, and the event listener is configured to: monitor the working state of the containers in the target Pod; in a case where the restart policy of the containers in the target Pod is onfailure, determine whether all the main containers in the target Pod execute the task successfully and exit, wherein the onfailure means that the abnormal container is restarted when the container in the Pod abnormally exits; and in a case where all the main containers in the target Pod execute the task successfully and exit, trigger the sidecar controller to execute the container exit mechanism on the target container.

9. An electronic device, comprising: The electronic device comprises a processor configured to invoke a computer program from a memory, and when the computer program is executed, the processor is configured to perform the method performed by the API server in any one of claims 1 to 4, or perform the method performed by the Webhook component in any one of claims 1 to 4, or perform the method performed by the sidecar controller in any one of claims 2 to 4.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by the processor, the computer execution instructions are configured to implement the method performed by the API server in any one of claims 1 to 4, or implement the method performed by the Webhook component in any one of claims 1 to 4, or implement the method performed by the sidecar controller in any one of claims 2 to 4.

11. A computer program product, characterised in that, The computer program product stores computer execution instructions, and when the computer execution instructions are executed by the processor, the computer execution instructions are configured to implement the method performed by the API server in any one of claims 1 to 4, or implement the method performed by the Webhook component in any one of claims 1 to 4, or implement the method performed by the sidecar controller in any one of claims 2 to 4.

Images