A container orchestration method and apparatus

Abstract

A container orchestration method and device are used to improve resource utilization. The container orchestration method comprises: a management entity obtaining first configuration information, the first configuration information being used to configure a first container node set, the first container node set comprising at least one container node, any container node being used to run a plurality of container deployment units, and any container deployment unit being used to run at least one container; the management entity creating at least one transition resource according to the first configuration information; the at least one transition resource corresponding to the at least one container node in a one-to-one manner, and any transition resource being used to create a corresponding container node; and the management entity sending resource configuration information of the at least one transition resource to a first service entity.

Description

Technical Field

[0001] This application relates to the field of mobile communication technology, and in particular to a container orchestration method and apparatus. Background Technology

[0002] Various virtual machine technologies ushered in the era of cloud computing, and containers, as the next generation of virtualization technology, are gradually emerging. A container is a virtual environment container, which can be viewed as a lightweight virtual machine. Applications running in different containers are isolated to a certain extent. The main difference between containers and virtual machines is that virtual machines rely on a hypervisor, using it to run multiple different slave operating systems on top of the main operating system. Each slave operating system can be considered a virtual machine, while containers are managed by a container engine. Virtual machines take several minutes to start, while containers can start in milliseconds.

[0003] In a container cluster management system (Kubernetes, K8S), the smallest deployable unit can be called a container deployment unit (this article uses pod as an example). Containers run within pods. Currently, in the field of container orchestration, the common container orchestration method is to divide each pod into independent container nodes. However, this orchestration method cannot maximize the utilization of the physical machine (or virtual machine) resources occupied by the container node, resulting in significant overhead and potential waste of resources.

[0004] How to improve resource utilization is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a container arrangement method and apparatus to improve resource utilization.

[0006] Firstly, this application provides a container orchestration method, which can be applied to a management entity, or components (such as processors, chips, chip systems, circuits, functional modules, or others) within the management entity, or software modules. The method may include: the management entity obtaining first configuration information, the first configuration information being used to configure a first set of container nodes, the first set of container nodes containing at least one container node, any container node being used to run multiple container deployment units, and any container deployment unit being used to run at least one container; the management entity creating at least one transitional resource based on the first configuration information; the at least one transitional resource corresponding one-to-one with at least one container node, any transitional resource being used to create the corresponding container node; and the management entity sending resource configuration information of the at least one transitional resource to a first service entity.

[0007] Using this method, the management entity can send resource configuration information of at least one transitional resource to the first service entity, thereby enabling the first service entity to create a first set of container nodes (e.g., a first container node). Since any container node in the first set of container nodes is used to run multiple container deployment units, the waste of resources allocated to that container node can be reduced, and resource utilization can be improved.

[0008] In one possible design, before the management entity sends resource configuration information of at least one transitional resource to the first service entity, the management entity may also select the first service entity from among multiple service entities as the service entity for deploying the first set of container nodes based on at least one transitional resource.

[0009] With this design, the management entity can select the deployment location of the first container node set from among multiple service entities, thereby improving the accuracy of the deployment of the first container node set.

[0010] In one possible design, the resource configuration information of each of the aforementioned at least one transitional resource includes the identification information of the first service entity.

[0011] With this design, the resource configuration information of the first transition resource can include the identification information of the first service entity. This allows the service entity that receives the configuration information of the first transition resource to match the deployment location of the first transition resource with its own identification information, thereby enabling self-checking of the deployment location and improving the accuracy of the deployment of the first container node set.

[0012] In one possible design, at least one transition resource includes a first transition resource, which is a resource of the first virtual deployment unit; at least one container node includes a first container node, and the first virtual deployment unit corresponds to the first container node; the management entity can also determine that the state of the first virtual deployment unit is running, and the management entity can delete the first transition resource.

[0013] With this design, the management entity can delete the first transitional resource when it determines that the first virtual deployment unit is in the running state, thereby reducing the occupation of storage resources and improving the utilization rate of storage resources.

[0014] In one possible design, the process by which the management entity determines that the state of the first virtual deployment unit is running may include: the management entity receiving first indication information from the first service node, the first indication information being used to indicate that the state of the first virtual deployment unit be updated to running.

[0015] With this design, the management entity can receive first instruction information from the first service entity, and thus determine the status of the first virtual deployment unit based on the first instruction information.

[0016] In one possible design, at least one container node includes a first container node, and the management entity can also obtain second configuration information, which is used to instruct at least two first container deployment units to run in the first container node; the management entity can also create at least two business resources according to the second configuration information; the at least two business resources correspond to the resources of at least two container deployment units respectively; the management entity can also send the resource configuration information of at least two business resources to the first service entity, which is used to run at least two first container deployment units in the first container node.

[0017] With this design, the management entity can determine at least two container deployment units to be run through the second configuration information; and instruct the first service entity to run at least two container deployment units on the first container node through at least two business resources. In other words, the first service entity can run business based on the created first container node, and each container node can run at least two container deployment units, which can improve the utilization rate of resources at the first container node.

[0018] In one possible design, before the management entity sends resource configuration information for at least two business resources to the first service entity, the management entity may also receive a response message from the first service node, which indicates that the first container node has been successfully created.

[0019] With this design, the management entity can determine that the first service entity has successfully created the first container node by responding to the response message before sending at least two business resources to the first service entity, thus reducing the waste of signaling resources.

[0020] In one possible design, the process by which the management entity obtains the first configuration information may include: the management entity receiving a first request, the first request being used to request configuration of a first set of container nodes; the first request including the first configuration information.

[0021] With this design, the management entity can receive the first request, thereby determining the first set of container nodes that need to be configured and determining the first configuration information.

[0022] In one possible design, the process by which the management entity obtains the first configuration information may include: the management entity receiving a second request, the second request being used to request the migration of the first set of container nodes from the second service entity to the first service entity, the second request including the first configuration information.

[0023] Accordingly, the process of the management entity sending resource configuration information of at least one transitional resource to the first service entity may include: the management entity sending a third request to the first service entity, the third request being used to request the migration of the first set of container nodes from the second service entity to the first service entity, the third request including resource configuration information of at least one first transitional resource.

[0024] With this design, the management entity can receive a second request to determine the first set of container nodes that need to be migrated and to determine the first configuration information. During the migration process, the management entity can also send a third request to the first service entity, so that the first service entity can clearly understand the task of migrating the first container nodes.

[0025] In one possible design, the management entity may also send a fourth request to request the migration of the second set of container nodes from the first service entity to the third service entity; the second set of container nodes includes at least one container node, any container node in the second set of container nodes is used to run multiple container deployment units, and any container deployment unit is used to run at least one container.

[0026] With this design, the management entity can also send a fourth request to the first service entity, thereby enabling the migration of the container node set (e.g., the second container node set) within the first service entity.

[0027] In one possible design, at least one container node includes a first container node; the first configuration information includes at least one of the following: the name of the first container node; the number of CPUs corresponding to the first container node; the memory capacity information corresponding to the first container node; the tag information of the first container node; the name and version of the guest operating system image corresponding to the first container node; the operating system kernel version corresponding to the first container node; the number of isolated CPU cores corresponding to the first container node; the large page type and number corresponding to the first container node; the temporary disk capacity corresponding to the first container node; the persistent volume capacity corresponding to the first container node; and the network plane accessed by the first container node.

[0028] With this design, the first configuration information can indicate the resource information corresponding to at least one container node, thereby enabling the management entity to clearly understand the resource requirements of the container node to be created, and thus accurately find the appropriate service entity.

[0029] In one possible design, at least one transitional resource includes a first transitional resource, which corresponds to a first container node. The first transitional resource is a resource of a first virtual deployment unit. The resources of the first virtual deployment unit include at least one of the following: the name of the first virtual deployment unit; the tag information of the first virtual deployment unit; the number of CPU isolated cores of the first virtual deployment unit; the large page type and number of the first virtual deployment unit; the temporary disk capacity of the first virtual deployment unit; the persistent volume capacity of the first virtual deployment unit; the network plane accessed by the first virtual deployment unit; the corresponding operating system kernel version of the first virtual deployment unit; the name of each container in the first virtual deployment unit; the image name of the simplified guest operating system used by each container in the first virtual deployment unit; the expected value of the memory capacity corresponding to the first virtual deployment unit; the expected value of the number of CPUs corresponding to the first virtual deployment unit; the limit value of the memory capacity corresponding to the first virtual deployment unit; the limit value of the number of CPUs corresponding to the first virtual deployment unit; the type of container node running the first virtual deployment unit; and the deployment location information of the first container node corresponding to the first virtual deployment unit.

[0030] With this design, the resources of the first virtual deployment unit include the resource requirements corresponding to the first virtual deployment unit, thereby enabling the first service entity to clearly define the resource configuration requirements of the first container node and accurately complete the configuration of the first container node.

[0031] Secondly, this application provides a container orchestration method, which can be applied to a first service entity, or components (such as processors, chips, chip systems, circuits, functional modules, or others) within the first service entity, or software modules. The method may include: the first service entity receiving resource configuration information from a first transitional resource from a management entity; the first service entity creating a first container node based on the resource configuration information of the first transitional resource; the first container node being used to run multiple container deployment units, each container deployment unit being used to run at least one container.

[0032] In one possible design, before the first service entity creates the first container node, the first service entity may also determine that the resource configuration information of the first transition resource includes the identification information of the first service entity.

[0033] In one possible design, the first transitional resource is the resource of the first virtual deployment unit; after the first service entity creates the first container node, the first service entity can also send a first instruction message to the management entity, which is used to instruct the state of the first virtual deployment unit to be updated to runtime.

[0034] In one possible design, the first service entity can also receive resource configuration information of at least two business resources from the management entity; the first service entity can also create at least two first container deployment units based on the resource configuration information of the at least two business resources; the at least two business resources correspond one-to-one with the at least two first container deployment units; the first service entity can also run at least two first container deployment units on the first container node.

[0035] In one possible design, before the first service entity creates at least two first container deployment units, the first service entity may also create container volumes corresponding to at least two first container deployment units and virtual network interfaces corresponding to at least two first container deployment units based on the resource configuration information of at least two business resources; the first service entity may also hot-insert container volumes and virtual network interfaces into the first container node.

[0036] With this design, the first service entity can hot-plug container volumes and virtual network interfaces into the first container node before creating at least two first container deployment units, thereby enabling dynamic adjustment of the resources of the first container node.

[0037] In one possible design, after the first service entity creates the first container node, the first service entity can also send a response message to the management entity, indicating that the first container node has been successfully created.

[0038] In one possible design, the process by which the first service entity receives resource configuration information of the first transition resource from the management entity may include: the first service entity receiving a third request from the management entity, the third request being used to request the migration of the first container node from the second service entity to the first service entity; the third request including resource configuration information of the first transition resource.

[0039] Accordingly, the process by which the first service entity creates the first container node based on the resource configuration information of the first transition resource may include: the first service entity establishing a first connection with the second service entity based on a third request; the first service entity obtaining the resource configuration information of the first transition resource through the first connection; and the first service entity creating the first container node based on the resource configuration information of the first transition resource.

[0040] In one possible design, the first service entity may also receive a fourth request from the management entity, which requests the migration of the second container node set from the first service entity to the third service entity; the second container node set includes at least one container node, any container node is used to run multiple container deployment units, and any container deployment unit is used to run at least one container; the first service entity may also establish a second connection with the third service entity; the first service entity may also send at least one transition resource corresponding to the second container node set to the third service entity through the second connection; the at least one transition resource corresponding to the second container node set is used by the third service entity to create the second container node set.

[0041] In one possible design, the first service entity can also periodically query and update the migration progress of the second set of container nodes.

[0042] With this design, the first service entity, as the migration device of the second set of container nodes, can detect the migration progress in real time.

[0043] In one possible design, the first service entity can also delete the second container node set when the migration progress indicator of the second container node set has been successfully migrated.

[0044] With this design, when the migration progress indicator of the second container node set has been successfully migrated, the first service entity can delete the second container node set, thereby releasing available resources and improving resource utilization.

[0045] Thirdly, embodiments of this application provide a container orchestration apparatus. The apparatus can implement any possible implementation of any of the first to second aspects described above.

[0046] In one optional implementation, the apparatus may include modules, units, or means corresponding one-to-one to the methods / operations / steps / actions that perform any possible implementation of any of the first to second aspects. These modules, units, or means may be hardware circuits, software, or a combination of hardware circuits and software. In another optional implementation, the apparatus includes a processing module (sometimes also called a processing unit) and a communication module (sometimes also called a transceiver module, communication unit, etc.). The communication module is capable of both sending and receiving functions. When the communication module performs the sending function, it may be called a sending unit (sometimes also called a sending module); when the communication module performs the receiving function, it may be called a receiving unit (sometimes also called a receiving module). The sending unit and the receiving unit may be the same functional module, referred to as the communication module, which performs both sending and receiving functions; or, the sending unit and the receiving unit may be different functional modules, with "communication module" being a collective term for these functional modules.

[0047] For example, when the apparatus is used to perform the method described in any one of the first to second aspects, the apparatus may include a processing module and a communication module.

[0048] Fourthly, embodiments of this application also provide a container orchestration apparatus, including a processor for executing a computer program (or computer-executable instructions) stored in a memory, such that when the computer program (or computer-executable instructions) is executed, the apparatus performs a method as in any possible implementation of any of the first to second aspects.

[0049] In one possible implementation, the processor and memory are integrated together.

[0050] In another possible implementation, the memory is located outside the container orchestration device.

[0051] The container orchestration apparatus also includes a communication interface for communicating with other devices, such as sending or receiving data and / or signals. Exemplarily, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.

[0052] Fifthly, a computer-readable storage medium is provided for storing a computer program or instructions that, when executed, enable the implementation of a method in any possible implementation of the first or second aspect, and the method shown in any possible implementation thereof.

[0053] In a sixth aspect, a computer program product containing instructions is provided, which, when run on a computer, enables the implementation of any possible implementation of the first or second aspect.

[0054] In a seventh aspect, embodiments of this application also provide a container orchestration apparatus for performing a method of any possible implementation of any of the first to second aspects described above.

[0055] Eighthly, a chip or chip system is provided, comprising logic circuitry (or, as understood, a processor, which may include logic circuitry, etc.), and further comprising input / output interfaces. The input / output interfaces can be used to input messages or to output messages. The input / output interfaces can be the same interface, i.e., the same interface can implement both sending and receiving functions; or, the input / output interface includes an input interface and an output interface, the input interface being used to implement the receiving function, i.e., to receive messages; and the output interface being used to implement the sending function, i.e., to send messages. The logic circuitry can be used to perform operations other than the sending and receiving functions in any possible implementation of any of the first to second aspects described above; the logic circuitry can also be used to transmit messages to the input / output interfaces or to receive messages from other container orchestration devices from the input / output interfaces. The chip system can be used to implement any possible implementation of any of the first to second aspects described above. The chip system can be composed of chips or can include chips and other discrete devices.

[0056] Optionally, the chip system may also include a memory, which can be used to store instructions, and the logic circuits can call the instructions stored in the memory to implement the corresponding functions.

[0057] A ninth aspect provides a communication system that may include a management entity and a first service entity. The management entity may be used to implement the methods shown in the first aspect and any possible implementation thereof, and the first service entity may be used to implement the methods shown in the second aspect and any possible implementation thereof.

[0058] Optionally, the communication system may also include other service entities besides the first service entity; this application does not impose any limitations on this.

[0059] The technical effects brought about by the second to ninth aspects above can be found in the description of the beneficial effects of the corresponding solutions in the first aspect above, and will not be repeated here. Attached Figure Description

[0060] Figure 1 A network function virtualization architecture diagram provided in this application embodiment;

[0061] Figure 2 A container orchestration architecture diagram provided in this application embodiment;

[0062] Figure 3 A structural example diagram of a service entity provided in an embodiment of this application;

[0063] Figure 4 A flowchart illustrating a container orchestration method provided in an embodiment of this application;

[0064] Figure 5 A flowchart illustrating another container orchestration method provided in an embodiment of this application;

[0065] Figure 6 A flowchart illustrating another container orchestration method provided in an embodiment of this application;

[0066] Figure 7 A flowchart illustrating another container orchestration method provided in an embodiment of this application;

[0067] Figure 8 This is a schematic diagram of the structure of a container arrangement device provided in an embodiment of this application;

[0068] Figure 9 This is a schematic diagram of another container arrangement device provided in an embodiment of this application. Detailed Implementation

[0069] To facilitate understanding of the embodiments of this application, the application scenarios of this application will be introduced below. The business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0070] In the fields of virtualization technology and cloud computing platforms, network functions virtualization (NFV) architecture can migrate traditional telecommunications network functions from dedicated hardware devices to virtualization software running on standard-based servers, storage, and switches, thereby improving network flexibility and efficiency.

[0071] Figure 1 This application provides an NFV architecture diagram. For example... Figure 1As shown in the diagram, the NFV architecture includes: Operational Support System (OSS) / Business Support System (BSS), Element Management System (EMS), Virtual Network Function (VNF), Hypervisor module (also known as the hypervisor virtualization layer), Compute / Network / Storage modules, NFV Orchestrator (NFVO), Virtual Network Function Manager (VNFM), Container Infrastructure Service Cluster Management (CCM), Container Infrastructure Service Management (CISM), Virtual Infrastructure Manager (VIM), and Physical Infrastructure Manager (PIM). (Reference) Figure 1 Each VNF can fulfill the service requirements of an application (APP) through container resources; each VNF corresponds to a container infrastructure service (CIS), which is used to orchestrate and manage VNFs.

[0072] In some examples, the container resource can be a virtual machine (VM) running on a hypervisor module. The hypervisor module is responsible for managing physical resources and creating and managing VMs; in this scenario, the container operating system can be the virtual machine operating system (guest OS). In other examples, the container resource can be a bare metal (BM) server; in this scenario, the container operating system can be the host operating system (host OS).

[0073] In some examples, CIS and CISM can be deployed in a containerized manner through a container cluster management system (Kubernetes, K8S) (not shown in the figure); a pod is the basic deployment unit of K8S, and a pod consists of a group of containers working on the same node.

[0074] This application does not limit the number of each component in the NFV architecture diagram; for example, the NFV architecture diagram may include multiple EMS (two in the example in the diagram), and the NFV architecture diagram may include multiple VNF (three in the example in the diagram).

[0075] The following section describes the functions of each component in the NFV architecture.

[0076] NFVO is the top-level controller of the entire NFV architecture, responsible for overall service orchestration and lifecycle management, and coordinating the work of components such as VNF, VNFM, and VIM.

[0077] VNFM is responsible for managing the lifecycle of individual VNFs, including creation, configuration, monitoring, and termination. In some examples, VNFM is controlled by NFVO, and VNFM can interact with VNFs.

[0078] CCM is responsible for managing and orchestrating the lifecycle management of multiple container clusters, and for issuing the container cluster management plane (e.g., CISM) and the container cluster data plane (e.g., CIS).

[0079] CISM is responsible for ensuring the efficient operation of containerized applications, including resource discovery, resource allocation, container management (including container lifecycle management), and container scheduling. In some examples, CISM can manage the underlying resources provided by CIS (such as compute resources, storage resources, etc.).

[0080] VIM is responsible for managing virtualization resources (including VMs, containers, etc.) and handling faults.

[0081] PIM is responsible for managing physical resources such as servers, storage, and network devices. In some examples, PIM can collaborate with VIM to optimize resource allocation.

[0082] OSS / BSS provides functions such as business process management and customer management, and supports the configuration, activation and fault handling of network services.

[0083] EMS is responsible for managing and monitoring physical or virtual network elements to ensure their proper functioning.

[0084] The compute / network / storage module provides the compute, network, and storage resources required for the virtualization environment.

[0085] The following describes the functions of each interface in the NFV architecture.

[0086] or-vnfm is the interface between NFVO and VNFM.

[0087] or-ccm is the interface between NFVO and CCM.

[0088] or-vi is the interface between NFVO and NFVI.

[0089] ve-vnfm is the interface between VNFM and VNF.

[0090] nf-k8s is the interface between K8S and NFV.

[0091] nf-vi is the interface between the hypervisor module and VIM.

[0092] k8s-vnfm is the interface between Kubernetes (K8S) and VNFM.

[0093] k8s-ccm is the interface between Kubernetes and CCM.

[0094] vi-k8S is the interface between VIM and K8S.

[0095] vi-vnfm is the interface between VIM and VNFM.

[0096] vi-ccm is the interface between VIM and CCM.

[0097] In addition, declarative interface resources can be added between Kubernetes and VNFM. Figure 1 The identifier is IF1.

[0098] In the field of container orchestration, to ensure the isolation and security between different pods, a common orchestration approach is to divide each pod into independent container nodes, and each container node usually occupies a uniform standard of resources (VM or BM). However, the resources required to run a single pod are relatively small, and this orchestration approach cannot maximize the utilization of the resources occupied by each container node, resulting in a large overhead and potentially wasting resources.

[0099] To improve resource utilization, this application provides a container orchestration method to support running multiple pods on a single container node, thereby improving resource utilization. Figure 1 Based on the NFV architecture shown, in order to more accurately introduce the container orchestration method provided in this application, this application also provides... Figure 2 The container orchestration architecture diagram shown below; Figure 2 This can be understood as... Figure 1 Middle components ( Figure 1 The architecture obtained through abstract processing (such as CIS and CISM, etc.) is displayed in bold in the middle.

[0100] like Figure 2 As shown, the container orchestration architecture diagram includes management entities and service entities.

[0101] Management entities (e.g.) Figure 1 The CISM framework can include a database, general controllers, specialized controllers, and a scheduler. The database can be a distributed key-value store (e.g., ETCD), a small database management system using Structured Query Language (MySQL) (or a relational database), etc. The database, general controllers, specialized controllers, and scheduler can interact with each other through an application programming interface server (API server). Furthermore, the northbound interface of the API server can include resource interfaces for container node sets and resource interfaces for container node hot migration tasks.

[0102] Assuming the service entity (corresponding to) Figure 1 The VNFs and / or CIS in the container orchestration architecture utilize the resources of the BM server to achieve virtualization, thereby meeting the service requirements of the application. Service entities can include the host OS, container node manager, high-level runtime (HLR), low-level runtime (LLR) (two are shown in the diagram), and container nodes (two are shown in the diagram). Each container node occupies the resources of a microvirtual machine (microVM), and any container node can run one or more pods. Each microVM is implemented through the virtualization of the BM server's resources. Furthermore, the container orchestration architecture diagram can also include clients; the management entity can interact with clients based on newly added interfaces.

[0103] In some examples, the high-level runtime can be the container daemon (containerd), and the low-level runtime can be kata.

[0104] In this context, the container node manager can act as a proxy deployed on a single service entity in Kubernetes, assuming the functions of a kubelet (a small-scale Kubernetes manager). The container node manager can interact with the high-level runtime through the container runtime interface (CRI). Optionally, the high-level runtime can extend the functionality of the container engine; the low-level runtime can extend the functionality of the container platform proxy. Under the orchestration and scheduling of the high-level runtime, each low-level runtime can manage container nodes through the container platform proxy.

[0105] In one possible design, management is performed separately for each container node serving a different number of pods, thereby improving the accuracy of container node management. In this embodiment, container nodes that allow only one pod to run are classified as first-class container nodes (which may be referred to as secure containers (SC) in this application), and container nodes that allow multiple (two or more) pods to run are classified as second-class container nodes (which may be referred to as secure container nodes (SCN) in this application). Figure 2 Based on the container orchestration architecture diagram shown, this application also provides Figure 3 The diagram shows an example of the structure of a service entity. Figure 3 As shown, the service entity includes a container platform agent module (playing the role of low-level runtime), a container engine module (playing the role of high-level runtime), first-type container nodes, second-type container nodes, physical hardware, and a host OS. The physical hardware is virtualized to obtain multiple container resources, and each container node can occupy one container resource to run a container operating system. The container platform agent module includes a first-type container node agent and a second-type container node agent. The container engine module includes a first-type container node runtime and a second-type container node runtime. The first-type container node agent and the first-type container node runtime are used to manage the first-type container nodes, and the second-type container node agent and the second-type container node runtime are used to manage the second-type container nodes.

[0106] The following is combined with Figure 4 The technical solution of this application will be described in detail with specific method embodiments. Figure 4 As shown, the container orchestration method provided in this application may include the following S401 to S404.

[0107] S401: The management entity obtains first configuration information, which is used to configure a first set of container nodes. The first set of container nodes contains at least one container node, and any container node is used to run multiple container deployment units, and any container deployment unit is used to run at least one container. In other words, all container nodes in the first set of container nodes belong to the second type of container nodes.

[0108] Optionally, at least one container node includes a first container node; correspondingly, the first configuration information includes at least one of the following: the name of the first container node; the number of central processing units (CPUs) corresponding to the first container node (including expected and limit values); the memory capacity information corresponding to the first container node; the tag information of the first container node; the name and version of the image of the streamlined guest operating system corresponding to the first container node; the operating system kernel version corresponding to the first container node; the number of isolated CPU cores corresponding to the first container node; the type and number of large pages corresponding to the first container node; the temporary disk capacity corresponding to the first container node; the persistent volume capacity corresponding to the first container node; and the network plane accessed by the first container node.

[0109] In some examples, the first configuration information includes multiple fields, each used to store different information. The CPU specification (CPU Spec) field stores the number of isolated CPU cores corresponding to the first container node; the memory specification (Mem Spec) field stores the type and number of large pages corresponding to the first container node; the storage specification (Strorage Spec) field stores the temporary disk capacity and the persistent volume capacity corresponding to the first container node; and the network specification (Net Spec) field stores the network plane accessed by the first container node.

[0110] In one possible design, the process by which the management entity obtains the first configuration information can differ depending on the scenario. The following examples illustrate the processes by which the management entity obtains the first configuration information in Scenario 1 and Scenario 2.

[0111] Scenario 1: The management entity receives a first request from the client. The first request is used to request the configuration of the first set of container nodes; the first request includes the first configuration information.

[0112] Thus, in the scenario of the initial creation of the first container node (i.e., scenario one), the client can send a first request to the management entity so that the management entity can further initiate the process of creating the first set of container nodes.

[0113] Scenario 2: The process of the management entity obtaining the first configuration information may include: the management entity receiving a second request from the client, the second request being used to request the migration of the first set of container nodes from the second service entity to the first service entity, the second request including the first configuration information.

[0114] Thus, in the scenario of migrating the first container node (i.e., scenario two), the client can send a second request to the management entity so that the management entity can further initiate the process of migrating the first set of container nodes from the second service entity to the first service entity.

[0115] S402: The management entity creates at least one transitional resource based on the first configuration information; each transitional resource corresponds one-to-one with at least one container node, and any one transitional resource is used to create the corresponding container node. That is, the number of container nodes in the first container node set is the same as the number of transitional resources.

[0116] Wherein, at least one transition resource includes a first transition resource, at least one container node includes a first container node, and the first transition resource corresponds to the first container node.

[0117] Optionally, the first transitional resource is the resource of the first virtual deployment unit; the first virtual deployment unit corresponds to the first container node.

[0118] It should be understood that in the embodiments of this application, the virtual deployment unit is not a deployment unit running on a container node, but a deployment unit used to create a container node; that is, the virtual deployment unit is different from the container deployment unit.

[0119] In some examples, both virtual deployment units and container deployment units can be defined as pods, but they are not the same in nature. For example, when the runtime class name field in a deployment unit is used to indicate the creation of a container node, the deployment unit is essentially a virtual deployment unit, not a container deployment unit.

[0120] Accordingly, the resources of the first virtual deployment unit include at least one of the following: the name of the first virtual deployment unit (corresponding to the name of the first container node); the tag information of the first virtual deployment unit (corresponding to the tag information of the first container node); the number of CPU isolated cores of the first virtual deployment unit (corresponding to the number of CPU isolated cores of the first container node); the type and number of large pages of the first virtual deployment unit (corresponding to the type and number of large pages of the first container node); the temporary disk capacity of the first virtual deployment unit (corresponding to the temporary disk capacity of the first container node); the persistent volume capacity of the first virtual deployment unit (corresponding to the persistent volume capacity of the first container node); the network plane accessed by the first virtual deployment unit (corresponding to the network plane accessed by the first container node); the corresponding operating system kernel version of the first virtual deployment unit (corresponding to the operating system kernel version of the first container node); the name of each container in the first virtual deployment unit (corresponding to); the first virtual The image name of the simplified guest operating system used by each container in the deployment unit (corresponding to the image name and version of the simplified guest operating system corresponding to the first container node); the expected memory capacity corresponding to the first virtual deployment unit (corresponding to the expected value in the memory capacity information corresponding to the first container node); the expected number of CPUs corresponding to the first virtual deployment unit (corresponding to the expected value in the CPU number information corresponding to the first container node); the memory capacity limit corresponding to the first virtual deployment unit (corresponding to the limit value in the memory capacity information corresponding to the first container node); the CPU number limit corresponding to the first virtual deployment unit (corresponding to the limit value in the CPU number information corresponding to the first container node); the type of container node running the first virtual deployment unit (e.g., first-type container node or second-type container node); and the deployment location information of the first container node corresponding to the first virtual deployment unit (identification information of the service entity to which the first container node belongs).

[0121] Optionally, the resource configuration information of each transition resource includes the identification information of the first service entity. In some examples, the management entity may also store the deployment location information of the container node corresponding to the transition resource in the resource configuration information of any transition resource. For example, the first transition resource corresponds to the first container node, and the deployment location of the first container node is the first service entity; accordingly, the management entity may store the identification information of the first service entity in the resource configuration information of the first transition resource to indicate the deployment location of the first container node.

[0122] In one possible example, S402 may specifically include a management entity storing the resources of a first set of container nodes in a database based on first configuration information; the management entity perceives the resources related to the container nodes already stored in the database through a list-watch mechanism, thereby creating at least one transitional resource. The list-watch mechanism is a resource monitoring method provided by CISM. The list operation is used to retrieve all data for a specified resource, while the watch operation is used to monitor resource change events in real time. The controller in CISM can continuously execute list operations to obtain the latest state of the resources, while simultaneously monitoring dynamic changes in the resources through watch operations.

[0123] S403: The management entity sends resource configuration information for at least one transitional resource to the first service entity. Correspondingly, the first service entity may receive resource configuration information for at least one transitional resource (including the first transitional resource) from the management entity.

[0124] Optionally, in Scenario 2, the management entity obtains the first configuration information through a second request. Correspondingly, the process of the management entity sending resource configuration information of at least one transitional resource to the first service entity may include: the management entity sending a third request to the first service entity, the third request requesting the migration of the first container node set (including the first container node) from the second service entity to the first service entity, the third request including resource configuration information of at least one transitional resource (including the first transitional resource) corresponding to the first container node set. In some examples, the management entity may also send the aforementioned third request to the second service entity, thereby notifying the second service entity to migrate the first container node set out.

[0125] In one possible design, before the management entity sends configuration information of at least one transitional resource to the first service entity, the management entity may also select the first service entity from among multiple service entities as the service entity for deploying the first set of container nodes based on the at least one transitional resource.

[0126] The following section uses the first transition resource and its corresponding first container node as examples to illustrate the creation process of the first container node. It should be understood that the creation process of other container nodes in the first container node set can refer to the creation process of the first container node.

[0127] S404: The first service entity creates a first container node based on the resource configuration information of the first transitional resource. The first container node is used to run multiple container deployment units, and each container deployment unit is used to run at least one container.

[0128] Optionally, before the first service entity creates the first container node, the first service entity may also determine that the resource configuration information of the first transitional resource includes the identification information of the first service entity. In this way, the first service entity can determine that the deployment location of the first container node is itself (i.e., the first service entity), avoiding container orchestration errors.

[0129] Optionally, in Scenario 2, the first service entity confirms through a third request that the source service entity to be migrated is the second service entity. Correspondingly, the process of creating the first container node (i.e., S404) may include: the first service entity establishing a first connection between itself and the second service entity based on the third request; the first service entity obtaining resource configuration information of the first transition resource through the first connection; and the first service entity creating the first container node based on the resource configuration information of the first transition resource. In this way, the first service entity can migrate from other service entities (such as the second service entity) into the first container node set, improving the flexibility of the NFV architecture and enhancing overall efficiency.

[0130] In the embodiments of this application, the first connection can be a hot migration connection or a cold migration connection, and this application does not impose any restrictions.

[0131] Optionally, during the migration of the first container node, the second service entity can periodically query and update the migration progress of the first container node. Optionally, when the migration progress of the first container node indicates that the migration has been successful, the second service entity can also delete the first container node. In this way, after the migration of the first container node is completed, the second service entity can delete the first container node, thereby saving resources.

[0132] Optionally, after the first service entity successfully creates the first container node (i.e., S404), the first service entity may also send a response message to the management entity; correspondingly, the management entity receives the response message from the first service entity; wherein the response message is used to indicate that the first container node has been successfully created.

[0133] Optionally, after receiving the response message, the management entity can also store the mapping relationship between the first container node and the first service entity. This way, when the management entity needs to run services on the first container node, it can query the deployment location of the first container node through the aforementioned mapping relationship, thereby improving service scheduling efficiency.

[0134] In one possible design, the management entity can also determine that the first virtual deployment unit is in running state, and can also delete the first transitional resource. Thus, when the first virtual deployment unit's state is updated to running (i.e., the first container node was successfully created), the management entity can delete the first transitional resource, thereby saving resources.

[0135] Optionally, the process by which the management entity determines that the state of the first virtual deployment unit is running may include: the first service entity may also send a first instruction message to the management entity; correspondingly, the management entity receives the first instruction message from the first service node; wherein the first instruction message is used to instruct the state of the first virtual deployment unit to be updated to running.

[0136] It should be understood that the first service entity may send the first instruction information to the management entity once it has determined that the first container node has been created.

[0137] Using the aforementioned S401 to S404, the first service entity creates a corresponding container node based on the resource configuration information of each transition resource, thereby successfully creating a first container node set (including the first container node). Any container node in the first container node set is used to run multiple container deployment units, reducing the waste of resources allocated to that container node.

[0138] The deployment process of the first container node is described in detail below; the container orchestration method provided in this application also includes S405 to S409.

[0139] S405: The management entity obtains the second configuration information; the second configuration information is used to instruct at least two first container deployment units to run on the first container node.

[0140] In some examples, the process by which the management entity obtains the second configuration information in S405 includes: the client sending a container node deployment request to the management entity; and the management entity receiving the container node deployment request; wherein the container node deployment request is used to request the deployment of at least two first container deployment units in the first container node of the first service entity; and the container node deployment request includes the aforementioned second configuration information.

[0141] S406: The management entity creates at least two business resources based on the second configuration information; the at least two business resources correspond to the resources of at least two container deployment units respectively.

[0142] S407: The management entity sends resource configuration information for at least two business resources to the first service entity. The resource configuration information for at least two business resources is used to run at least two container deployment units on the first container node. Correspondingly, the first service entity receives the resource configuration information for at least two business resources from the management entity.

[0143] Optionally, the management entity may determine before executing S407 that the first service entity has successfully created the first container node. For example, the management entity may receive a response message from the first service node before S407, indicating that the first container node has been successfully created.

[0144] S408: The first service entity creates at least two first container deployment units based on the resource configuration information of at least two business resources; wherein, at least two business resources and at least two first container deployment units correspond one-to-one.

[0145] In one possible design, before executing S406, the first service entity can also create at least two container volumes corresponding to the first container deployment unit and at least two virtual network interfaces corresponding to the first container deployment unit based on the resource configuration information of at least two business resources; the first service entity can also hot-insert container volumes and virtual network interfaces into the first container node.

[0146] S409: The first service entity runs at least two of the aforementioned first container deployment units on the first container node.

[0147] By using the aforementioned S405 to S409, the first service entity can deploy at least two first container deployment units in the first container node to support at least two container deployment units provided by the client to share the resources (such as computing resources and storage resources) of the same container node; since the security relationship between the at least two first container deployment units defined by the client is conflict-free, the resource utilization of a single container node can be improved while ensuring security.

[0148] In one possible design, container nodes already created in the first service entity can also be migrated to other service entities. For example, if the first service entity lacks resources, it can migrate some of its created container nodes to other service entities. Alternatively, the client may instruct the migration of some container nodes to other container nodes. The migration process for container nodes already created in the first service entity (e.g., the second set of container nodes) is described in detail below; the container orchestration method provided in this application also includes steps S410 to S413.

[0149] S410: The management entity may also send a fourth request to the third service entity; correspondingly, the third service entity receives the fourth request from the management entity; wherein the fourth request is used to request the migration of the second container node set from the first service entity to the third service entity; the fourth request includes third configuration information, which is used to configure the second container node set, the second container node set including at least one container node, any container node being used to run multiple container deployment units, and any container deployment unit being used to run at least one container. In some examples, the management entity may also send the aforementioned fourth request to the third service entity, thereby notifying the third service entity to implement the migration of the second container node set.

[0150] In some examples, the second set of container nodes may intersect with the first set of container nodes; for example, the second set of container nodes may be contained within the first set of container nodes.

[0151] Prior to S410, the client could send a fifth request to the management entity, which was used to request the migration of the second set of container nodes from the first service entity to the third service entity. The fifth request could include the aforementioned third configuration information.

[0152] S411: The third service entity establishes a second connection between itself and the first service entity based on the fourth request. In this embodiment, the second connection can be a hot migration connection or a cold migration connection; this application does not impose any restrictions.

[0153] S412: The second service entity sends at least one transitional resource (including the second transitional resource) corresponding to the second container node set to the third service entity via the second connection. Correspondingly, the third service entity receives at least one transitional resource corresponding to the second container node set from the third service entity via the second connection. The second transitional resource is a resource of the second virtual deployment unit; the content of the resources of the second virtual deployment unit can be referred to the content of the resources of the first virtual deployment unit mentioned above, and will not be repeated here.

[0154] S413: The third service entity creates a second container node set (including the second container node) based on at least one transition resource (including the second transition resource) corresponding to the second container node set in S412 above. It should be understood that the process of the third service entity creating the second container node set can also be understood as the first service entity migrating the second container node set to the third service entity.

[0155] Optionally, during the migration process of the aforementioned second set of container nodes, the first service entity may also periodically query and update the migration progress of the second set of container nodes.

[0156] Optionally, when the migration progress indicator of the second container node set has shown successful migration, the first service entity can also delete the second container node set. This way, after the migration of the second container node set is complete, the first service entity can delete the second container node set, thereby saving resources.

[0157] Using the methods shown in S410 to S413, the management entity can instruct the first service entity to migrate some container nodes to other service entities (such as the third service entity), which can improve the flexibility of the NFV architecture and enhance overall efficiency.

[0158] In conjunction with the container orchestration methods provided in S401 to S413 above, this application also provides three embodiments, each describing the process of creating a set of container nodes (see reference). Figure 5 The deployment process of container nodes (see reference) Figure 6 ) and the migration process of container nodes (refer to Figure 7 In this embodiment, Examples 1 and 2 can refer to the container orchestration method corresponding to Scenario 1 above; Example 3 can refer to the container orchestration method corresponding to Scenario 2 above. In the following embodiments, the management entity is CISM, and the service entity is CIS; the first type of container node can be identified as SC, and the second type of container node can be identified as SCN.

[0159] Example 1 (Creation process of container node set):

[0160] Assume that each container node in the set of container nodes to be created (including container node A) belongs to the second type of container node, meaning that any container node (including container node A) is used to run at least two pods. Based on this, the aforementioned set of container nodes to be created can be identified as an SCN set. Figure 5 As shown, the container orchestration method may include S501 to S513.

[0161] S501: The client uploads the image of the container operating system (e.g., the virtual machine operating system of a micro virtual machine) corresponding to container node A to CISM through VNFM (e.g., the VFNM interface or interface). The information uploaded by the client may include the name and kernel version of the container operating system.

[0162] In some examples, S501 may not be executed, meaning the client may not need to upload the image of the container operating system corresponding to container node A. In such examples, the container operating system of container node A can be the default container operating system.

[0163] It should be understood that the client can use a method similar to S501 to upload the container operating system images of other container nodes in the SCN set besides container node A.

[0164] S502: The client sends a request message to CISM via VFNM to create resources for an SCN set. The parameters carried in this request message (refer to the first configuration information in S401 to S413) may include at least one of the following: the name of each container node in the SCN set, the number of SCNs in the SCN set, the number of CPUs corresponding to each SCN, the memory capacity corresponding to each SCN, and the tag information corresponding to each SCN (this tag is used for scheduling and selecting container nodes when creating a Pod application).

[0165] Optionally, the parameters carried by the request message may also include CPU Spec, Mem Spec, Strength Spec, and Net Spec.

[0166] It should be understood that S501 and S502 can be executed together or separately (without regard to the order), and this application does not impose any restrictions. When S501 and S502 are executed together, the parameters carried in the request message may also include the name and kernel version of the container operating system corresponding to container node A.

[0167] S503: CISM stores the resources of the SCN set in its database. In some examples, CISM creates the resources of the SCN set based on the parameters carried in the request message and saves them to the CISM database.

[0168] S504: CISM reports the creation result of the SCN Set resources to VNFM (resources of the SCN Set have been created).

[0169] S505: CISM creates at least one temporary pod resource based on the resources in the SCN Set. In some examples, CISM (e.g.) Figure 2 or Figure 3 The dedicated controller in CISM uses a List-Watch mechanism to detect the resources of newly added container node sets (SCN Sets) and creates a corresponding number of temporary pod resources based on the number of SCNs in the container node set (SCN Set). For example, if the container node set includes 8 SCNs, CISM can create 8 temporary pod resources.

[0170] In some examples, CISM can map the various fields of a temporary pod resource to the resources in the SCN Set, as shown in the following example:

[0171] API version: Version 1 (v1);

[0172] Type (kind): pod;

[0173] Metadata - Name: String (derived from the parameter: name in the SCN Set resource);

[0174] Labels - Scheduler: string (derived from the parameter in the SCN Set resource: label information);

[0175] Annotations - Kernel Version: string (derived from the parameter in the SCN Set resource: OS kernel version);

[0176] Specification (spec) - runtime class name (used to distinguish different types of container nodes): SC or SCN; for example, when the container nodes in the container node set are type 2 container nodes, the content of the runtime class name field in each temporary pod resource is set to SCN (i.e., the identifier of type 2 container nodes).

[0177] Containers - Name: string (derived from the parameter: name in the SCN Set resource);

[0178] Image: string (derived from the parameter in the SCN Set resource: container OS image name);

[0179] Resources - Limits - Memory: Number (num) (e.g., 200 mebibytes (Mi)) (derived from the memory capacity information in the SCN Set resource).

[0180] Resource - Limit Value - Number of CPUs: Value (e.g., 700 millicores (m)) (derived from the parameter in the SCN Set resource: number of CPUs information);

[0181] Resource - Expected Values ​​(requests) - Memory Capacity: Value (e.g., 200Mi) (derived from the parameter in the SCN Set resource: memory capacity information);

[0182] Resource - Expected Value - Number of CPUs: Value (e.g., 700m) (derived from the parameter in the SCN Set resource: CPU number information).

[0183] S506: CISM selects the deployment location of the container node corresponding to each ephemeral pod resource; the deployment location can be an existing physical node (used to deploy container nodes). In some examples, CISM (e.g.) Figure 2 or Figure 3The scheduler in CISM finds a matching deployment location based on the temporary pod resources (CPU, memory, etc.). The resource configuration information of the temporary pod resources includes a node name field, which stores the deployment location of the container node corresponding to the temporary pod resource. Correspondingly, CISM can also fill the deployment location information into the node name field of the temporary pod resource and store the resource configuration information of the temporary pod resource in the CISM database.

[0184] In some examples, S505 and S506 can be executed together, and this application does not impose any restrictions.

[0185] S507: The process of CIS starting to create temporary pod resources.

[0186] It should be understood that since the content of the runtime class name field in each temporary pod resource is set to SCN (i.e., the identifier of the second type of container node), the "starting the process of creating temporary pod resources" on the CIS side can be understood as "CIS confirming the existence of container nodes to be created".

[0187] In some examples, CIS detects newly added temporary pod resources through a List-Watch mechanism, thereby triggering the process of creating temporary pod resources on the CIS side.

[0188] S508: CIS determines the specifications of the Micro VM corresponding to the container node to be created. In some examples, CIS (e.g.) Figure 2 or Figure 3 The container node manager and low-level runtime in CIS can set Micro VM resources based on temporary Pod resources; CIS can also set Micro VM label information based on the content of the label field corresponding to the temporary Pod resources.

[0189] S509: CIS (Request) retrieves the image of the container node to be created.

[0190] In some examples, the CIS checks locally whether the image corresponding to the image field of the temporary Pod resource is stored. If the corresponding image is stored locally, the CIS can download the image locally; accordingly, the S510 action is skipped. If the corresponding image is not stored locally, the CIS can send a request message to the CISM to obtain the image; accordingly, S510 continues.

[0191] S510: CIS receives the image path from CISM and downloads the image based on that image path.

[0192] S511: After downloading the image from CIS, start the process of creating a container node (SCN).

[0193] S512: CIS updates the status of temporary pods.

[0194] In some examples, CIS starts a timer and periodically queries the creation progress of the container node (SCN) based on the timer.

[0195] In some examples, CIS can also update the status of ephemeral pods based on the creation progress of container nodes (SCNs); for example, when the corresponding container node (SCN) is created, the status of the ephemeral pod is set to running.

[0196] S513: CISM can store the deployment location of each container node locally.

[0197] Optionally, CISM can also delete temporary pod resources.

[0198] In some examples, when CISM (e.g.) Figure 2 or Figure 3 When the dedicated controller in CISM detects a change in the status of a temporary Pod to "running" via a List-Watch mechanism, it determines that the container node has been successfully created. After CISM confirms that the container node has been created, it can also store the deployment location information corresponding to each container node (SCN). After CISM confirms that the container node has been created, it can also delete the temporary Pod resource.

[0199] In some examples, the deployment location of the container node is the existing physical node (the physical node corresponding to the CIS), and the storage location for each container node is the physical node corresponding to that container node; an example of a physical node is shown below:

[0200] API version: v1;

[0201] kind: node (or physical node);

[0202] metadata-name: string (derived from the parameter: name in the SCN Set resource);

[0203] metadata-labels: string (e.g., custom labels used to identify node attributes, such as role, region, etc.);

[0204] metadata-scheduler; string (parameters from SCN Set resources: tag information);

[0205] spec-capacity-cpu: num (e.g., 4) (derived from the parameter in the SCN Set resource: number of CPUs);

[0206] spec-capacity-memory: num (e.g., 16 gigabytes (Gi)) (derived from the parameter in the SCN Set resource: amount of memory);

[0207] Allocable-cpu: num (e.g., 3) (initial value comes from the parameter in the SCN Set resource: number of CPUs); this value can be adjusted in real time; allocable-cpu is used to indicate the CPU resources that are currently available for allocation on the physical node;

[0208] allocatable-memory: num (e.g., 12 Gi) (initial value comes from the parameter in the SCN Set resource: memory quantity); this value can be adjusted in real time; allocatable-memory is used to indicate the memory capacity resources that can be allocated to the current physical node;

[0209] Information (info) (used to indicate other information about the physical node) - kernel-version: string (derived from a parameter in the SCN Set resource: OS kernel version);

[0210] info – container os-image: string (derived from the parameter in the SCN Set resource: the image name of the container OS);

[0211] info -ontainer-runtime-version: SCN (This can be a fixed value in this embodiment);

[0212] info -kubelet-version: string (determined based on the SCN Manager version and can be adjusted in real time).

[0213] Example 2 (Deployment process of container nodes):

[0214] It should be understood that Example 2 should be executed after the container node creation is completed in Example 1. Example 2 is illustrated using the initial deployment of a pod as an example. The process for modifying and deleting a pod can be referred to Example 2, and will not be repeated in this application. Figure 6 As shown, the container orchestration method may include S601 to S613.

[0215] S601: The client sends a container node deployment request to CISM via VFNM. The parameters carried in this container node deployment request (refer to the second configuration information in S401 to S413) may include resource information such as CPU, memory, volume, and network, and the label field carries a label information matching the container node (SCN). In some examples, this container node deployment request may be a request to create a deployment configuration resource; in other examples, it may be a request to create statefulsets resources.

[0216] S602: CISM creates container node deployment resources based on the container node deployment request; the parameters of the container node deployment resources correspond to the parameters in the container node deployment request. The container node deployment resource can be a Deployment resource or a StatefulSets resource; accordingly, CISM creates a Deployment resource (or a StatefulSets resource) based on the parameters carried in the container node deployment request. CISM can also save the created container node deployment resources to its database.

[0217] S603: CISM identifies multiple pod resources to be created corresponding to the deployment resources of a container node. In some examples, CISM (e.g.) Figure 2 or Figure 3 The dedicated controller in CISM detects newly added container node deployment resources through List-Watch. CISM creates a corresponding number of Pod resources based on the parameters (quantity field) in the container node deployment resources, and determines the name, resources (CPU, memory, etc.), image, tag information, annotations, and other fields corresponding to multiple Pod resources based on the parameters in the container node deployment resources.

[0218] S604: CISM stores the previously created pod resources in the database.

[0219] S605: CISM reports the creation results of container node deployment resources to VNFM (multiple pod resources have been created).

[0220] S606: CISM identifies multiple pod resources to be scheduled. In some examples, CISM (e.g.) Figure 2 or Figure 3 The dedicated controller in the system detects newly added Pod resources through the List-Watch mechanism; CISM (e.g., Figure 2 or Figure 3 The scheduler in the program can also allocate Pod resources.

[0221] S607: CISM determines the deployment location of container nodes corresponding to multiple pod resources. In some examples, CISM (e.g.) Figure 2 or Figure 3 The scheduler in CISM determines the deployment location (physical node) of the container node (SCN) corresponding to the pod resource based on the parameters carried by the pod resource. The resource configuration information of each pod includes the node name. The node name field is used to store the deployment location of the container node corresponding to the pod resource. Correspondingly, CISM can also fill the deployment location information into the node name of the pod resource and store the resource configuration information of the pod resource in the CISM database.

[0222] S608: CIS initiates the process of creating the aforementioned pod resource. In some examples, CIS detects the newly added pod resource through a List-Watch mechanism, thereby triggering the process of creating the pod resource on the CIS side.

[0223] S609: CIS creates the container volume and virtual network interface corresponding to the pod to be created. In some examples, CIS (e.g.) Figure 2 or Figure 3 The container node manager creates container volumes and container network devices based on the container volume attribute field and container network device attribute field of the Pod resource. In this embodiment, the container volume can be a virtual disk (vDISK) device, and the container network device can be a virtual network interface card (vNIC) device.

[0224] S610: CIS hot-plugs container volumes and virtual network interfaces into container nodes. In some examples, CIS (e.g.) Figure 2 or Figure 3 The container node manager obtains the deployment location of the container node (SCN) corresponding to the pod resource based on the node name field of the pod resource, and hot-inserts the previously created container volume and container network device into the container node (SCN).

[0225] S611: CIS (request) retrieves container image information for the pod to be created.

[0226] In some examples, the CIS checks locally whether the container image for a newly added Pod resource is stored. If the container image is already stored locally, the CIS can download it locally, skipping the S612 action. If the corresponding image is not stored locally, the CIS can send a request message to the CISM to retrieve the container image.

[0227] S612: CIS receives the container image path from CISM and downloads the container image based on that image path.

[0228] S613: After downloading the container image in the CIS, run the image of multiple pods on the Container Nodes (SCNs) corresponding to the aforementioned multiple pod resources. In some examples, the CIS (e.g., Figure 2 or Figure 3 The container node manager (SCN) creates Pods on the Container Nodes (SCNs) corresponding to multiple Pod resources and runs the container images corresponding to the multiple Pod resources, thereby running the container application corresponding to each Pod; CIS (e.g., Figure 2 or Figure 3 The container node manager can also configure the container network.

[0229] Example 3 (Container Node Migration Process):

[0230] Container node A has been created into a secure container using the method in Example 1 (in some examples, it can also be replaced by a set of container nodes); the migration process of container nodes is described below using the example of migrating a container node from a source CIS to a target CIS. Figure 7 As shown, the container orchestration method may include S701 to S716.

[0231] S701: The client sends a migration request to CISM to create a container node to be migrated through VFNM (e.g., the VFNM interface or interface). The parameters carried in the migration request (some of which can be referred to in the first configuration information in S401 to S413) may include the name of the SCN to be migrated, the migration type (cold or hot), the name of the source node (taking the source CIS as an example in this embodiment), the name of the target node (taking the target CIS as an example in this embodiment), and the label of the container node (SCN) to be migrated.

[0232] S702: CISM creates a container node migration resource (SCN migration) based on the aforementioned migration request. In some examples, CISM creates the container node migration resource based on the parameters in the aforementioned migration request and saves it to the CISM database.

[0233] S703: CISM reports the creation result of container node migration resources to VNFM (container node migration resources have been created).

[0234] S704: CISM migrates resources based on already created container nodes, creating temporary pod resources. In some examples, CISM (e.g., ...) Figure 2 or Figure 3The dedicated controller in CISM detects newly added container node migration resources through the List-Watch mechanism, finds the corresponding container node set (SCN Set) resources based on the parameters in the container node migration resources (such as SCN name), and creates the aforementioned temporary Pod resources based on the resources in the container node set; CISM can also set the content of the runtime class name field in the temporary pod resources to SCN (i.e., the identifier of the second type of container node).

[0235] In some examples, CISM can map parameters from the resources of a container node set to various fields of a temporary Pod resource, as shown in the following examples:

[0236] apiVersion: v1;

[0237] kind: Pod;

[0238] metadata-name: string (derived from the parameter: name in the SCN Set resource);

[0239] labels-scheduler: string (parameters from the SCN Set resource: label information);

[0240] annotations-cpuSpec: string (derived from the parameter: CPU Spec in the SCN Set resource);

[0241] annotations -memSpec: string (derived from the parameter: Mem Spec in the SCN Set resource);

[0242] annotations -strorageSpec: string (derived from the parameter:StrorageSpec in the SCN Set resource);

[0243] annotations -netSpec: string (derived from the parameter: Net Spec in the SCN Set resource);

[0244] annotations -kernelVersion: string (parameter from SCN Set resource: OS kernel version);

[0245] spec-runtimeClassName:SCN (used to distinguish different types of container nodes; for example, SCN is the identifier for the second type of container node).

[0246] containers-name: string (derived from the parameter: name in the SCN Set resource);

[0247] image: string (derived from the parameter in the SCN Set resource: container OS image name);

[0248] resources-limits-memory: num (e.g., 200Mi) (derived from the parameter in the SCN Set resource: memory capacity);

[0249] resources-limits-cpu: num (e.g., 700m) (derived from the parameter in the SCN Set resource: number of CPUs);

[0250] resources-requests-memory: num (e.g., 200Mi) (derived from the parameter in the SCN Set resource: memory capacity);

[0251] resources-requests-cpu: num (e.g., 700m) (derived from the parameter in the SCN Set resource: number of CPUs).

[0252] S705: CISM identifies temporary pod resources to be migrated. In some examples, CISM (e.g.) Figure 2 or Figure 3 The dedicated controller in the system detects newly added temporary Pod resources through the List-Watch mechanism; CISM (e.g.) Figure 2 or Figure 3 The scheduler in the program can also perform temporary Pod resource allocation.

[0253] S706: CISM determines the deployment location of the container node corresponding to the aforementioned temporary pod resource. In some examples, CISM (e.g.) Figure 2 or Figure 3 The scheduler in CISM determines the deployment location (physical node) of the container node (SCN) corresponding to the temporary pod resource based on the parameters carried by the temporary pod resource. The resource configuration information of the temporary pod includes the node name. The node name field is used to store the deployment location of the container node corresponding to the temporary pod resource. Correspondingly, CISM can also fill the deployment location information into the node name of the temporary pod resource and store the resource configuration information of the temporary pod resource in the CISM database.

[0254] S707: The target CIS initiates the process of creating the aforementioned temporary pod resource. In some examples, the target CIS detects the newly added temporary pod resource through a List-Watch mechanism, thereby triggering the process of creating the temporary pod resource on the target CIS side.

[0255] S708: The target CIS determines the specifications of the Micro VM corresponding to the container node to be migrated corresponding to the aforementioned temporary pod resources. In some examples, the target CIS (e.g.) Figure 2 or Figure 3 The container node manager and low-level runtime in the target CIS can set Micro VM resources based on temporary Pod resources; the target CIS can also set the label information of the Micro VM based on the field content of the label information (label) field corresponding to the temporary Pod resources.

[0256] S709: The target CIS (request) retrieves the image of the container node to be created.

[0257] In some examples, the target CIS checks locally whether the image corresponding to the image field of the temporary Pod resource is stored. If the corresponding image is stored locally, the target CIS can download the image locally; accordingly, the S710 action is skipped. If the corresponding image is not stored locally, the target CIS can send a request message to CISM to obtain the image; accordingly, S710 continues.

[0258] S710: The target CIS receives the image path from the CISM and downloads the image based on that image path.

[0259] S711: After the target CIS downloads the image, the process of creating a container node (SCN) is initiated.

[0260] S712: The target CIS updates the status of the temporary pod.

[0261] In some examples, the target CIS starts a timer and periodically queries the creation progress of the container node (SCN) based on the timer.

[0262] In some examples, the target CIS can also update the status of the ephemeral pod based on the creation progress of the container node (SCN); for example, when the corresponding container node (SCN) is created, the status of the ephemeral pod is set to running.

[0263] S713: CISM can store the deployment location of each container node locally.

[0264] Optionally, CISM can also delete temporary pod resources.

[0265] In some examples, when CISM (e.g.) Figure 2 or Figure 3 When the dedicated controller in CISM detects a change in the status of a temporary Pod to "running" via a List-Watch mechanism, it determines that the container node has been successfully created. After CISM confirms that the container node has been created, it can also store the deployment location information corresponding to each container node (SCN). After CISM confirms that the container node has been created, it can also delete the temporary Pod resource.

[0266] In some examples, the deployment location of the container node is the existing physical node (the physical node corresponding to the target CIS), and the storage location of each container node is the physical node corresponding to that container node; an example of a physical node is shown below:

[0267] API version: v1;

[0268] kind: node (or physical node);

[0269] metadata-name: string (derived from the parameter: name in the SCN Set resource);

[0270] metadata-labels: string (e.g., custom labels used to identify node attributes, such as role, region, etc.);

[0271] metadata-scheduler; string (parameters from SCN Set resources: tag information);

[0272] spec-capacity-cpu: num (e.g., 4) (derived from the parameter in the SCN Set resource: number of CPUs);

[0273] spec-capacity-memory: num (e.g., 16 gigabytes (Gi)) (derived from the parameter in the SCN Set resource: amount of memory);

[0274] Allocable-cpu: num (e.g., 3) (initial value comes from the parameter in the SCN Set resource: number of CPUs); this value can be adjusted in real time; allocable-cpu is used to indicate the CPU resources that are currently available for allocation on the physical node;

[0275] allocatable-memory: num (e.g., 12 Gi) (initial value comes from the parameter in the SCN Set resource: memory quantity); this value can be adjusted in real time; allocatable-memory is used to indicate the memory capacity resources that can be allocated to the current physical node;

[0276] Information (info) (used to indicate other information about the physical node) - kernel-version: string (derived from a parameter in the SCN Set resource: OS kernel version);

[0277] info – container os-image: string (derived from the parameter in the SCN Set resource: the image name of the container OS);

[0278] info -ontainer-runtime-version: SCN (This can be a fixed value in this embodiment);

[0279] info -kubelet-version: string (determined based on the SCN Manager version and can be adjusted in real time).

[0280] S714: A point-to-point hot migration connection is established between the target CIS and the source CIS corresponding to the container node (SCN), and the container node (SCN) is migrated (e.g., iterative state replication) through the hot migration connection.

[0281] S715: The source CIS updates the status of container node migration resources.

[0282] In some examples, the source CIS can also start a timer until the hot migration of the container node (SCN) is complete or the timer expires. In some examples, the CIS starts a timer to periodically query the hot migration progress of the container node (SCN) based on the timer.

[0283] In some examples, CIS can also update the status of container node migration resources based on the hot migration progress of container nodes; for example, when the hot migration of the corresponding container node (SCN) is completed, the status of the container node migration resources is set to success or failure.

[0284] It should be understood that "hot migration" in S714 and S715 can be replaced with "cold migration" without affecting the implementation of this embodiment.

[0285] S716: CISM can also delete the state of container node migration resources. In some examples, when CISM (e.g.) Figure 2 or Figure 3 When the dedicated controller in the container node detects that the status of the container node migration resource has changed to success or failure through the List-Watch mechanism, it deletes the corresponding container node migration resource.

[0286] It should be understood that CISM can restart the migration of container nodes even when the migration of resources fails.

[0287] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of different embodiments are consistent and can be referenced by each other. The technical features of different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0288] It should also be noted that each step in the above embodiments can be executed by the corresponding device, or by components such as chips, processors, or chip systems within that device. The embodiments of this application do not limit their execution. The above embodiments are merely illustrative examples of execution by the corresponding device. Furthermore, the specific implementation methods or examples in the above embodiments do not limit the solutions provided by the embodiments of this application.

[0289] Based on the same technical concept, this application provides a container orchestration device. The device includes modules, units, or means that perform the method steps in the above method embodiments. The functions, units, or means can be implemented by software, hardware, or hardware executing corresponding software.

[0290] For example, see Figure 8 The container orchestration device 800 may include a processing module 801 and a communication module 802.

[0291] Optionally, the communication module 802 may include a sending module and / or a receiving module. The sending module is used to perform the sending operation in the above method embodiments. The receiving module is used to perform the receiving operation in the above method embodiments. It should be noted that the container orchestration device 800 may only include the sending module and not the receiving module. Alternatively, the container orchestration device 800 may only include the receiving module and not the sending module. Specifically, it depends on whether the above scheme executed by the container orchestration device 800 includes both sending and receiving actions.

[0292] The processing module 801 is used for data processing. The communication module 802 can implement the corresponding communication functions.

[0293] Optionally, the container orchestration device 800 may further include a storage module, which can be used to store instructions and / or data. The processing module 801 can read the instructions and / or data in the storage module so that the container orchestration device 800 can implement the aforementioned method embodiments.

[0294] It should be understood that all relevant content of each step involved in the above method embodiments can be referenced to the functional description of the corresponding functional module.

[0295] The processing module 801 in the above embodiments can be implemented by at least one processor or processor-related circuitry. The communication module 802 can be implemented by a transceiver or transceiver-related circuitry. The communication module 802 can also be referred to as a communication module or a communication interface.

[0296] For example, the container orchestration device 800 can be a management entity or a component configured within the management entity. The communication module 802 is used to acquire first configuration information, which is used to configure a first set of container nodes. The first set of container nodes includes at least one container node, any container node is used to run multiple container deployment units, and any container deployment unit is used to run at least one container. The processing module 801 is used to create at least one transition resource based on the first configuration information. Each transition resource corresponds one-to-one with at least one container node, and any transition resource is used to create a corresponding container node. The communication module 802 is also used to send resource configuration information of at least one transition resource to the first service entity.

[0297] For example, the container orchestration device 800 may be a first service entity or a component configured within the first service entity. The communication module 802 is used to receive resource configuration information of a first transitional resource from a management entity; the processing module 801 is used to create a first container node based on the resource configuration information of the first transitional resource; the processing module 801 is also used to run multiple container deployment units, each container deployment unit being used to run at least one container.

[0298] The following is another structural schematic diagram of the container orchestration device according to an embodiment of this application. For example... Figure 9 As shown in the figure, this application embodiment also provides a container orchestration device 900, including:

[0299] At least one processor 901; and a communication interface 903 communicatively connected to the at least one processor 901; the at least one processor 901 causes the device to perform the method steps in the above method embodiments through the communication interface 903 by executing instructions stored in the memory 902.

[0300] The memory 902 may be located outside the container orchestration device 900. Alternatively, the memory 902 may be located inside the container orchestration device 900. Optionally, the container orchestration device 900 includes the memory 902, which is connected to the at least one processor 901, and stores instructions executable by the at least one processor 901. (Appendix) Figure 9 The dashed line indicates that memory 902 is optional for container orchestration device 900.

[0301] The processor 901 and the memory 902 can be coupled through an interface circuit or integrated together; no restriction is imposed here.

[0302] This application embodiment does not limit the specific connection medium between the processor 901, memory 902, and communication interface 903. This application embodiment... Figure 9 The processor 901, memory 902, and communication interface 903 are connected via a bus 904. Figure 9 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0303] Taking a management entity as an example, when the container orchestration device 900 is a management entity, the management entity may include a processor, a memory, and a transceiver. The memory may store computer program code, and the transceiver includes a transmitter and a receiver.

[0304] The processor is primarily used for processing communication protocols and data; controlling terminal devices; executing software programs; and processing data from those programs. The memory is mainly used for storing software programs and data. The transmitter sends signals to other container orchestration devices or equipment, and the receiver receives signals from other container orchestration devices or equipment.

[0305] When the container orchestration device 900 is a chip in a terminal device, the chip may include a processor, memory, and a transceiver. The transceiver may be an input / output circuit or a communication interface. The processor may be a processing module integrated on the chip, a microprocessor, or an integrated circuit. In the above method embodiments, the sending operation of the terminal device can be understood as the output of the chip, and the receiving operation of the terminal device in the above method embodiments can be understood as the input of the chip.

[0306] It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0307] For example, the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0308] It should be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0309] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0310] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0311] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium, including a program or instructions, which, when run on a computer, cause the methods in the above method embodiments to be executed.

[0312] Based on the same technical concept, embodiments of this application also provide a computer program product, including instructions that, when run on a computer, cause the methods in the above method embodiments to be executed.

[0313] Based on the same technical concept, embodiments of this application also provide a communication system, which may include a management entity and a first service entity. For example, this communication system can be used to implement... Figure 4-7 The method flow is described in the text. Optionally, the communication system may also include other container orchestration devices.

[0314] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0315] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0316] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0317] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0318] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of different embodiments are consistent and can be referenced by each other. The technical features of different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0319] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The order of the process numbers described above does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

Claims

1. A container orchestration method, characterized by, Applied to managing entities, the method includes: Obtain first configuration information, the first configuration information is used to configure a first container node set, the first container node set contains at least one container node, any container node is used to run multiple container deployment units, and any container deployment unit is used to run at least one container. Based on the first configuration information, at least one transitional resource is created as a virtual deployment unit; the at least one transitional resource corresponds one-to-one with the at least one container node, and any one transitional resource is used to create the corresponding container node, the at least one container node including the first container node; Send the resource configuration information of the at least one transitional resource to the first service entity; Obtain second configuration information, which is used to indicate that at least two first container deployment units are running in the first container node; Based on the second configuration information, create at least two business resources; The resource configuration information of the at least two business resources is sent to the first service entity. The resource configuration information of the at least two business resources is used to run the at least two first container deployment units on the first container node.

2. The method of claim 1, wherein, Before sending the resource configuration information of the at least one transitional resource to the first service entity, the method further includes: Based on the at least one transition resource, the first service entity is selected from among multiple service entities as the service entity for deploying the first set of container nodes.

3. The method of claim 1 or 2, wherein, The resource configuration information of each transitional resource includes the identification information of the first service entity.

4. The method of any one of claims 1-3, wherein, The first virtual deployment unit corresponds to the first container node, and the resources of the first virtual deployment unit are used to create the first container node; the method further includes: The status of the first virtual deployment unit is determined to be running; Delete the resources of the first virtual deployment unit.

5. The method of claim 4, wherein, Determining that the first virtual deployment unit is in operation includes: Receive a first indication message from the first service node, the first indication message being used to indicate that the status of the first virtual deployment unit be updated to running.

6. The method of any one of claims 1-5, wherein, Before sending the resource configuration information of the at least two service resources to the first service entity, the method further includes: Receive a response message from the first service node, the response message indicating that the first container node has been successfully created.

7. The method of any one of claims 1-6, wherein, The process of obtaining the first configuration information includes: A first request is received, the first request being used to request configuration of the first set of container nodes; the first request includes the first configuration information.

8. The method of any one of claims 1-7, wherein, The process of obtaining the first configuration information includes: Receive a second request, the second request being used to request the migration of the first container node set from the second service entity to the first service entity, the second request including the first configuration information; Sending the resource configuration information of the at least one transitional resource to the first service entity includes: A third request is sent to the first service entity, the third request being used to request the migration of the first set of container nodes from the second service entity to the first service entity, the third request including resource configuration information of the at least one transition resource.

9. The method of any one of claims 1-8, wherein, The method further includes: A fourth request is sent, the fourth request being used to request the migration of the second set of container nodes from the first service entity to the third service entity; the second set of container nodes includes at least one container node, any container node in the second set of container nodes is used to run multiple container deployment units, and any container deployment unit is used to run at least one container.

10. The method of any one of claims 1-9, wherein, The at least one container node includes a first container node; the first configuration information includes at least one of the following: The name of the first container node; CPU count information corresponding to the first container node; The memory capacity information corresponding to the first container node; The tag information of the first container node; The name and version of the client operating system image corresponding to the first container node; The operating system kernel version corresponding to the first container node; The number of CPU isolated cores corresponding to the first container node; The type and number of large pages corresponding to the first container node; The capacity of the temporary disk corresponding to the first container node; The capacity of the persistent volume corresponding to the first container node; The network plane accessed by the first container node.

11. The method of claim 10, wherein, The at least one transition resource includes a first transition resource, which corresponds to the first container node and is a resource of the first virtual deployment unit; the resources of the first virtual deployment unit include at least one of the following: The name of the first virtual deployment unit; The tag information of the first virtual deployment unit; The number of CPU isolated cores in the first virtual deployment unit; The type and number of large pages in the first virtual deployment unit; The temporary disk capacity of the first virtual deployment unit; The persistent volume capacity of the first virtual deployment unit; The network plane accessed by the first virtual deployment unit; The corresponding operating system kernel version of the first virtual deployment unit; The name of each container in the first virtual deployment unit; The image name of the simplified guest operating system used by each container in the first virtual deployment unit; The expected value of the memory capacity corresponding to the first virtual deployment unit; The expected number of CPUs corresponding to the first virtual deployment unit; The memory capacity limit for the first virtual deployment unit; The limit on the number of CPUs corresponding to the first virtual deployment unit; The type of container node running the first virtual deployment unit; The deployment location information of the first container node corresponding to the first virtual deployment unit.

12. A container orchestration method, characterized by, Applied to the first service entity, the method includes: Receive resource configuration information from the management entity for the first transitional resource; Based on the resource configuration information of the first transition resource, a first container node is created; the first container node is used to run multiple container deployment units, and any container deployment unit is used to run at least one container. Receive resource configuration information from at least two business resources of the management entity; Based on the resource configuration information of the at least two business resources, at least two first container deployment units are created; the at least two business resources correspond one-to-one with the at least two first container deployment units. The at least two first container deployment units are run on the first container node.

13. The method of claim 12, wherein, Before creating the first container node, the process also includes: It is determined that the resource configuration information of the first transitional resource includes the identification information of the first service entity.

14. The method of claim 12 or 13, wherein, The first transitional resource is the resource of the first virtual deployment unit; after creating the first container node, the method further includes: Send a first instruction message to the management entity, the first instruction message being used to instruct the status of the first virtual deployment unit to be updated to running.

15. The method of any one of claims 12-14, wherein, Before creating at least two first container deployment units, the method further includes: Based on the resource configuration information of the at least two business resources, create container volumes corresponding to the at least two first container deployment units and virtual network interfaces corresponding to the at least two first container deployment units; The container volume and the virtual network interface are hot-inserted into the first container node.

16. The method of any one of claims 12-15, wherein, After creating the first container node, the process also includes: A response message is sent to the management entity, indicating that the first container node has been successfully created.

17. The method of any one of claims 12-16, wherein, The receipt of resource configuration information from the first transitional resource of the management entity includes: A third request is received from the management entity, the third request being used to request the migration of the first container node from the second service entity to the first service entity; the third request includes resource configuration information of the first transition resource; The step of creating a first container node based on the resource configuration information of the first transition resource includes: Based on the third request, establish a first connection with the second service entity; The resource configuration information of the first transition resource is obtained through the first connection; Based on the resource configuration information of the first transition resource, the first container node is created.

18. The method of any one of claims 12-17, wherein, The method further includes: A fourth request is received from the management entity, the fourth request being used to request the migration of the second set of container nodes from the first service entity to the third service entity; the second set of container nodes includes at least one container node, any container node being used to run multiple container deployment units, and any container deployment unit being used to run at least one container; Establish a second connection with the third service entity; At least one transitional resource corresponding to the second container node set is sent to the third service entity through the second connection; the at least one transitional resource corresponding to the second container node set is used by the third service entity to create the second container node set.

19. The method of claim 18, wherein, The method further includes: Periodically query and update the migration progress of the second set of container nodes.

20. The method of claim 19, wherein, The method further includes: When the migration progress indicator of the second container node set has been successfully migrated, delete the second container node set.

21. A container orchestration apparatus, characterized by comprising: Includes units or modules for performing the method as described in any one of claims 1-20.

22. A container orchestration apparatus, characterized by comprising: It includes at least one processor for executing computer programs or instructions to implement the method as described in any one of claims 1-20.

23. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, and when the computer program or instructions are executed by the container orchestration device, the method as described in any one of claims 1-20 is implemented.

24. A computer program product, characterized in that, When the computer program product is executed by a computer, the computer performs the method as described in any one of claims 1-20.

Images