A computer system, a container management method and device

Abstract

A computer system, a container management method and device. The method is applied to an offloading card. The offloading card is inserted into a computing node, and a communication channel is established between the offloading card and the computing node. The offloading card is connected to a management node through a network. In this application, the management node sends a container creation request to the offloading card. After receiving the container creation request, the offloading card obtains a container image according to the container creation request and saves the container image locally. Then, the offloading card notifies the computing node through the communication channel to create a container in the computing node according to the container image. The computing node no longer needs to manage the container. Instead, the offloading card inserted in the computing node creates and manages the container, and the computing node no longer needs to consume resources to support the container management function, improving the resource utilization rate of the computing node.

Description

[0001] Cross - reference to related applications

[0002] This application claims the priority of a Chinese patent application with an application number of 202010932403.5 and an invention title of "Container - based Server System and Offloading Card" filed with the State Intellectual Property Office of the People's Republic of China on September 8, 2020. The entire content of which is incorporated herein by reference. Technical Field

[0003] This application relates to the field of communication technologies, and particularly to a computer system, a container management method, and an apparatus. Background Art

[0004] With the continuous development of cloud - based technologies, due to the advantages of containers such as being lightweight and high - speed, the instances deployed in data centers are gradually transitioning from virtual machines to containers. Currently, most container management architectures are built based on the existing infrastructure as a service (IAAS) layer. In addition to deploying containers on computing nodes, service proxy functions related to the IAAS layer and container management components also need to be deployed to implement the management of containers on computing nodes.

[0005] The container management components deployed on computing nodes will occupy the resources on the computing nodes, resulting in resource consumption of the computing nodes. Summary of the Invention

[0006] This application provides a computer system, a container management method, and an apparatus to reduce the resource consumption for implementing container management on computing nodes.

[0007] In a first aspect, an embodiment of this application provides a container management method. The method is applied to an offloading card and can be executed by the offloading card. The offloading card is inserted into a computing node, and a communication channel is established between the offloading card and the computing node. The offloading card is connected to a container cluster management node (which can also be simply referred to as a management node) through a network. In this method, the management node can send a container creation request to the offloading card. After receiving the container creation request sent by the management node, the offloading card can obtain a container image according to the container creation request. For example, the offloading card can obtain the container image from a container image repository and save the container image on a storage resource that the offloading card can access. The storage resource can be a local memory of the offloading card or a memory connected to the offloading card. After that, the offloading card can notify the computing node to create a container in the computing node according to the container image through the communication channel.

[0008] Through the above method, the computing node no longer needs to directly interact with the management node. That is to say, the computing node no longer needs to manage the containers. Instead, the offloading card inserted on the computing node creates and manages the containers. The computing node no longer needs to consume resources to support the container management function, improving the resource utilization rate of the computing node.

[0009] In a possible implementation, when the offloading card notifies the computing node to create a container in the computing node according to the container image, it can create a virtual device. For the sake of distinction, the virtual device here is called the first virtual device. After the offloading card creates the first virtual device, it can associate the container image with the first virtual device and notify the computing node to create the container running environment of the container through the communication channel and mount the first virtual device to the root directory of the container.

[0010] Through the above method, the offloading card provides the container image to the computing node in the form of a virtual device, ensuring that the computing node can use the container image to create a container. The way of creating a container is relatively simple and convenient.

[0011] In a possible implementation, the offloading card can also be connected to the storage service node where the storage service is deployed through the network, and the offloading card can provide storage resources for the containers on the computing node. Specifically, the offloading card first applies for storage resources from the storage service node; then, it sets the virtual device according to the storage resources. For the sake of distinction, the virtual device here is called the second virtual device; after the setting of the second virtual device is completed, the offloading card can mount the second virtual device to the directory of the container through the communication channel.

[0012] Through the above method, providing the storage resources of the container to the computing node in the form of the second virtual device can enable the containers on the computing node to access the storage resources through the second virtual device and store data on the storage resources, making it possible for the offloading card to provide storage resources for the containers on the computing node and further reducing the resource consumption on the computing node.

[0013] In a possible implementation, when the offloading card sets the second virtual device according to the storage resources, it can first create the second virtual device. After creating the second virtual device, it can associate the storage resources with the second virtual device.

[0014] Through the above method, the offloading card can create a virtual device locally and provide it to the containers on the computing node so that the containers on the computing node can obtain storage resources.

[0015] In a possible implementation, the storage resource can be an object storage resource or a block storage resource. When the storage resource is a file storage resource, the offloading card can directly provide the file storage resource to the computing node in the form of a network file system, and notify the computing node to mount the network file system to the directory of the container, that is, the file storage resource can be not associated with the second virtual device.

[0016] Through the above method, the offloading card can provide different types of storage resources to the containers in the computing node, which is applicable to object storage, file storage, and block storage scenarios, effectively expanding the applicable range.

[0017] In a possible implementation, when the offloading card mounts the second virtual device to the directory of the container through the communication channel, different mounting methods can be adopted for different types of containers. If the container is a normal container, the offloading card can directly mount the second virtual device to the directory (such as the storage directory) of the container through the communication channel. If the container is a secure container, the offloading card will directly pass the second virtual device to the secure container virtual machine used to deploy the container through the communication channel, and the secure container virtual machine will mount the second virtual device to the directory of the container.

[0018] Through the above method, different mounting methods are adopted for different types of containers to ensure that the container can obtain the storage resource, which is convenient for the subsequent container to store data on the corresponding storage resource.

[0019] In a possible implementation, the offloading card can be connected to the network service node through the network. In addition to providing storage resources to the containers of the computing node, the offloading card can also provide network resources to the containers of the computing node. The offloading card can first apply for network resources from the network service node; after obtaining the network resources, the offloading card can set up a virtual device according to the network resources. Here, for the convenience of distinction, the virtual device is called the third virtual device. After the third virtual device is set up, the offloading card can set the third virtual device in the container through the communication channel.

[0020] Through the above method, providing the network resources of the container to the computing node in the form of the third virtual device can enable the containers on the computing node to obtain network resources through the third virtual device, enabling the containers to have network capabilities, making it possible for the offloading card to provide network resources for the containers on the computing node, ensuring that the offloading card can implement the container management function, and further reducing the resource consumption on the computing node.

[0021] In a possible implementation, when the offloading card sets up the third virtual device according to the network resources, it can first create the third virtual device; after creating the third virtual device, the network resources can be associated with the third virtual device.

[0022] Through the above method, the offloading card can create a virtual device locally and provide it to the containers on the computing node, so that the containers on the computing node can obtain network resources and the containers can have network capabilities.

[0023] In a possible implementation, the offloading card sets network processing rules for the third virtual device. The network processing rules include some or all of the following: load balancing policy, security group policy, quality of service, routing rules, and address mapping rules. Among them, the security group policy may include access control lists (ACLs), and the address mapping rules include network address translation (NAT) and full network address translation (FULL NAT). Among them, NAT includes, but is not limited to, destination network address translation (DNAT), source network address translation (SNAT), and port network address translation (PNAT).

[0024] Through the above method, by setting network processing rules for the virtual network card device, the container can have service discovery capabilities and network policy capabilities, etc., so that the container has strong network capabilities.

[0025] In a possible implementation, when the offloading card sets the third virtual device in the container through the communication channel, different setting methods can be adopted for different types of containers. If the container is a normal container, the offloading card can add the third virtual device to the container's namespace through the communication channel. If the container is a secure container, the offloading card can directly pass the third virtual device to the secure container virtual machine used to deploy the container through the communication channel.

[0026] Through the above method, different setting methods are adopted for different types of containers to ensure that the containers can obtain the network resources, which is convenient for the subsequent containers to have network capabilities.

[0027] In a possible implementation, there are many types of communication channels. The embodiments of the present application do not limit the specific type of the communication channel. For example, the communication channel can be a PCIe channel.

[0028] Through the above method, the offloading card and the computing node can perform relatively efficient information interaction through the PCIe channel, further ensuring that the offloading card can manage the containers on the computing node.

[0029] In a second aspect, an embodiment of the present application further provides a container management device, which is located in the offloading card and has the function of implementing the behavior of the offloading card in the method example of the first aspect. The beneficial effects can be seen in the description of the first aspect and will not be elaborated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions. In a possible design, the structure of the device includes a transmission unit, an acquisition unit, and a notification unit. Optionally, it further includes a first setting unit and a second setting unit. These units can execute the corresponding functions in the method example of the first aspect. For specific details, refer to the detailed description in the method example and will not be elaborated here.

[0030] In a third aspect, an embodiment of the present application further provides a device, which can be an offloading card and has the function of implementing the behavior of the offloading card in the method example of the first aspect. The beneficial effects can be seen in the description of the first aspect and will not be elaborated here. The structure of the device includes a processor and a memory. The processor is configured to support the offloading card to execute the corresponding functions in the method of the first aspect. The memory is coupled to the processor and stores the necessary program instructions and data of the communication device. The structure of the communication device further includes a communication interface for communicating with other devices.

[0031] In a fourth aspect, an embodiment of the present application further provides a computer system. The beneficial effects can be seen in the relevant description of the first aspect and will not be elaborated here. The computing system includes an offloading card and a computing node. The offloading card is inserted into the computing node, and a communication channel is established between the offloading card and the computing node. The offloading card is also connected to the container cluster management node through a network.

[0032] The offloading card is used to receive a container creation request sent by the container cluster management node and obtain a container image according to the container creation request;

[0033] The computing node is used to obtain the container image through the communication channel and create a container according to the container image.

[0034] In a possible implementation manner, after obtaining the container image, the offloading card can create a first virtual device, associate the container image with the first virtual device, and can also provide the first virtual device to the computing node through the communication channel. When the computing node obtains the container image, it can obtain the first virtual device through the communication channel. After obtaining the first virtual device, it can create a container running environment for the container and mount the first virtual device to the root directory of the container.

[0035] In a possible implementation, the offloading card can also be connected to the storage service node through the network. The offloading card and the computing node can cooperate to configure storage resources for the containers on the computing node. The offloading card can first apply to the storage service node for storage resources; after obtaining the storage resources, the offloading card can set up a second virtual device according to the storage resources and provide the second virtual device to the computing node through the communication channel. After obtaining the second virtual device through the communication channel, the computing node can mount the second virtual device under the directory of the container.

[0036] In a possible implementation, when the offloading card sets up the second virtual device according to the storage resources, it can first create the second virtual device; after creating the second virtual device, it can associate the storage resources with the second virtual device.

[0037] In a possible implementation, the storage resources can be object storage resources or block storage resources. When the storage resources are file storage resources, the offloading card can directly provide the file storage resources to the computing node in the form of a network file system, and notify the computing node to mount the network file system under the directory of the container, that is, the file storage resources can be not associated with the second virtual device. The computing node can mount the network file system under the directory of the container under the notification of the offloading card.

[0038] In a possible implementation, when the computing node mounts the second virtual device under the directory of the container, different mounting methods can be adopted for different types of containers. For ordinary containers, which are containers different from secure containers, the computing node can directly mount the second virtual device under the directory of the container; for secure containers, the computing node can directly pass through the second virtual device to the secure container virtual machine used to deploy the container, and the secure container virtual machine mounts the second virtual device under the directory of the container.

[0039] In a possible implementation, the offloading card is also connected to the network service node through the network. The offloading card and the computing node can cooperate to configure network resources for the containers on the computing node. The offloading card can first apply to the network service node for network resources; after obtaining the network resources, it can set up a third virtual device according to the network resources and provide the third virtual device to the computing node through the communication channel. The computing node can obtain the third virtual device through the communication channel and set the third virtual device in the container.

[0040] In a possible implementation, when the offloading card sets up the third virtual device according to the network resources, it can create the third virtual device; associate the network resources with the third virtual device.

[0041] In a possible implementation, when the offloading card sets the third virtual device according to network resources, it can set network processing rules for the third virtual device. The network processing rules include some or all of the following: load balancing policy, security group policy, routing rules, address mapping rules, and quality of service (QoS).

[0042] In a possible implementation, when the computing node sets the third virtual device in a container, different mounting methods can be used for different types of containers. For ordinary containers, the computing node can add the third virtual device to the container's namespace. For secure containers, the computing node directly passes the third virtual device to the secure container virtual machine used to deploy the container.

[0043] In a possible implementation, the communication channel is a Peripheral Component Interconnect Express (PCIe) channel for high-speed external devices.

[0044] In a fifth aspect, an embodiment of the present application further provides a container management method, which is cooperatively executed by the offloading card and the computing node. The beneficial effects can be referred to the relevant descriptions in the first aspect and will not be elaborated here. The offloading card is inserted into the computing node, a communication channel is established between the offloading card and the computing node, and the offloading card is also connected to the container cluster management node through the network.

[0045] The offloading card receives a container creation request sent by the container cluster management node and obtains a container image according to the container creation request;

[0046] The computing node obtains the container image through the communication channel and creates a container according to the container image.

[0047] In a possible implementation, after obtaining the container image, the offloading card can create a first virtual device; it can also associate the container image with the first virtual device and provide the first virtual device to the computing node through the communication channel. When obtaining the container image, the computing node can obtain the first virtual device through the communication channel, create a container runtime environment for the container, and mount the first virtual device to the root directory of the container.

[0048] In a possible implementation, the offloading card is also connected to a storage service node through the network. The offloading card and the computing node can cooperate to configure storage resources for the container. The offloading card can first apply for storage resources from the storage service node; then, set the second virtual device according to the storage resources. After setting the second virtual device, the second virtual device can be provided to the computing node through the communication channel. The computing node can obtain the second virtual device through the communication channel and mount the second virtual device to the directory of the container after obtaining the second virtual device.

[0049] In a possible implementation, when the offloading card sets the second virtual device according to the storage resources, it may first create the second virtual device; after creating the second virtual device, it associates the storage resources with the second virtual device.

[0050] In a possible implementation, the storage resources can be object storage resources or block storage resources. When the storage resources are file storage resources, the offloading card can directly provide the file storage resources to the computing node in the form of a network file system, notify the computing node to mount the network file system to the directory of the container, and the computing node can mount the network file system to the directory of the container under the notification of the offloading card. In this case, the file storage resources may not be associated with the second virtual device.

[0051] In a possible implementation, when the computing node mounts the second virtual device to the directory of the container, for a normal container, that is, a container different from a secure container, the computing node can directly mount the second virtual device to the directory of the container. For a secure container, the computing node can pass through the second virtual device to the secure container virtual machine used to deploy the container, and the secure container virtual machine mounts the second virtual device to the directory of the container.

[0052] In a possible implementation, the offloading card is also connected to the network service node through the network. The offloading card can also cooperate with the computing node to configure network resources for the container so that the container has network capabilities. The offloading card can first apply for network resources from the network service node; then, set the third virtual device according to the network resources; and provide the third virtual device to the computing node through the communication channel. The computing node can obtain the third virtual device through the communication channel and set the third virtual device in the container.

[0053] In a possible implementation, when the offloading card sets the third virtual device according to the network resources, it may first create the third virtual device, and after creating the third virtual device, it can associate the network resources with the third virtual device.

[0054] In a possible implementation, when the offloading card sets the third virtual device according to the network resources, it can also set network processing rules for the third virtual device. The network processing rules include some or all of the following: load balancing policy, security group policy, routing rule, address mapping rule, quality of service.

[0055] In a possible implementation, when the computing node sets the third virtual device in the container, for a normal container, it can add the third virtual device to the namespace of the container. For a secure container, the computing node passes through the third virtual device to the secure container virtual machine used to deploy the container.

[0056] In a possible implementation, the communication channel is a Peripheral Component Interconnect Express (PCIe) channel for high-speed external devices.

[0057] In a sixth aspect, the present application further provides a computer-readable storage medium storing instructions, which, when running on a computer, cause the computer to execute the methods described in the first aspect and each possible implementation of the first aspect, or execute the methods described in the fifth aspect and each possible implementation of the fifth aspect.

[0058] In a seventh aspect, the present application further provides a computer program product containing instructions, which, when running on a computer, cause the computer to execute the methods described in the first aspect and each possible implementation of the first aspect, or execute the methods described in the fifth aspect and each possible implementation of the fifth aspect.

[0059] In an eighth aspect, the present application further provides a computer chip connected to a memory. The chip is configured to read and execute a software program stored in the memory, and execute the methods described in the first aspect and each possible implementation of the first aspect, or execute the methods described in the fifth aspect and each possible implementation of the fifth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 FIG. is a schematic diagram of the architecture of a system provided by the present application;

[0061] Figure 2 FIG. is a schematic diagram of the architecture of another system provided by the present application;

[0062] Figure 3 FIG. is a schematic diagram of a method for creating a container provided by the present application;

[0063] Figure 4 FIG. is a flowchart of a method for creating a container provided by the present application;

[0064] Figure 5 FIG. is a schematic diagram of a method for deleting a container provided by the present application;

[0065] Figure 6 FIG. is a schematic diagram of a method for configuring container storage resources provided by the present application;

[0066] Figure 7 FIG. is a schematic diagram of a method for configuring container network resources provided by the present application;

[0067] Figure 8 FIG. is a schematic diagram of the structure of a container management device provided by the present application;

[0068] Figure 9 FIG. is a schematic diagram of the structure of a device provided by the present application. Detailed implementation manners

[0069] As Figure 1 shown, it is a schematic diagram of a system architecture provided by an embodiment of the present application. The system includes a container management cluster 100 and a computing node cluster 200.

[0070] The container management cluster 100 is located between the user and the computing node cluster 200 and can interact with the user and the computing node cluster 200. The user manages the containers on the computing nodes 210 rented or owned by the user by interacting with the container management cluster 100. Here, the management includes, but is not limited to: creating containers, deleting containers, querying containers, etc.

[0071] The container management cluster 100 may include one or more management nodes 110, and each management node 110 can manage the containers on one or more computing nodes 210 in the computing cluster.

[0072] The embodiments of the present application do not limit the location where the management node 110 is deployed and the specific form of the management node 110. For example, the management node 110 may be a computing node 210 deployed in a cloud computing device system or an edge computing device system, or may be a terminal computing device close to the user side. Different management nodes 110 may be deployed in the same system or in different systems. For example, the multiple management nodes 110 may all be deployed in a cloud computing device system or an edge computing system, or the multiple management nodes 110 may be distributedly deployed in a cloud computing device system, an edge computing system, and a terminal computing device.

[0073] The computing node cluster 200 includes one or more computing nodes 210, and an offloading card 220 may also be inserted on each computing node 210. The embodiments of the present application do not limit the architecture type of the computing node 210. The computing node 210 may be a computing node 210 with an X86 architecture or a computing node 210 with an ARM architecture.

[0074] The offloading card 220 inserted on the computing node 210 is a hardware device with certain data processing capabilities. The offloading card 220 may include components such as a processor, a memory, a hardware acceleration device, a network card, etc., or the offloading card 220 may be connected to a network card. In the embodiments of the present application, the offloading card 220 can interact with the management node 110, and notify the computing node 210 where it is located to create a container according to the instruction issued by the management node 110, or can also manage the containers on the computing node 210 where it is located according to the instruction issued by the management node 110. Here, the management includes, but is not limited to: creating containers, deleting containers, querying containers.

[0075] Taking the creation of a container as an example, the interaction process among the user, the container management cluster 100, and the compute node cluster 200 will be described as follows:

[0076] The user can send a container creation request to the management node 110 in the container management cluster 100 through the client. The container creation request may carry the resource configuration information of the container, and the resource configuration information of the container can indicate the resources required to be occupied by the container. After receiving the container creation request, the management node 110 can record the resource configuration information of the container locally, and then select a target compute node for the container according to the resource status of one or more compute nodes 210 it manages, and schedule the container to the target compute node. The offloading card 220 inserted on the target compute node will monitor the scheduling operation of the management node 110. When it detects that the management node 110 schedules the container to the target compute node, the offloading card 220 will prepare corresponding resources for the container and notify the compute node 210 where it is located to create the container using the resources.

[0077] The target compute node creates the container upon the notification of the offloading card 220. After the offloading card 220 detects that the container creation is completed, it can report the status information of the container (the status information includes but is not limited to the running status of the container, the running status of the services on the container, the resource usage of the container, etc.) to the management node 110, and the management node 110 can display the status information of the container to the user through the client. The user can also query the status information of the container through the client.

[0078] The deletion of the container and the container query are similar to the above process. The difference lies in the different information interacted among the user, the container management cluster 100, and the compute node cluster 200. For specific details, refer to the foregoing description and will not be elaborated here.

[0079] As can be seen from the above description, the function of container management is offloaded to the offloading card 220, and the offloading card 220 realizes the container management of the compute node 210. The compute node 210 only needs to run the container, and the compute node 210 no longer has the container management function, thus reducing the resources required to implement the container management function on the compute node 210 and enabling the effective utilization of the resources on the compute node 210.

[0080] Next, specifically for a compute node 210 and the offloading card 220 inserted on the compute node 210, the structures of the compute node 210 and the offloading card 220 inserted on the compute node 210 will be described. Refer to Figure 2 , the offloading card 220 can interact with one or more management nodes 110 in the container management cluster 100. The offloading card 220 can also interact with the compute node 210 where it is located.

[0081] In addition, in order to configure or update network resources for the containers on the computing node 210, the offloading card 220 can also interact with the virtual network service node 300 (which can also be simply referred to as the network service node), connect to the virtual network service node 300 through the network. A virtual network service is deployed on the virtual network service node 300, and the virtual network service node 300 can provide virtual network services for the computing node 210 and the containers on the computing node 210. The virtual network server is an external service on which the containers depend, and can provide network resources for the containers, enabling the containers on different computing nodes 210 to achieve network interconnection and endowing the containers with network capabilities.

[0082] In order to configure or update storage resources for the containers on the computing node 210, the offloading card 220 can also interact with the storage service node 400 and connect to the storage service node 400 through the network. The storage service node 400 can deploy storage services such as block storage services, file storage services, or object storage services. Block storage services, file storage services, and object storage services all belong to distributed storage services. Distributed storage services refer to the ability to deploy storage resources distributively on different storage nodes. The storage service node 400 can provide storage resources for the computing node 210 and the containers on the computing node 210, so that the data in the computing node 210 or the containers on the computing node can be stored on the storage nodes.

[0083] A network proxy module 221 and a storage proxy module 222 can be deployed in the offloading card 220. The network proxy module 221 is used to interact with the virtual network service node 300 to apply for network resources for the containers on the computing node 210 from the virtual network service node 300. The storage proxy module 222 is used to interact with the storage service node 400 to apply for storage resources for the containers on the computing node 210 from the storage service node 400.

[0084] The offloading card 220 can manage the containers on the computing node 210 under the instruction of the management node 110. Specifically, the offloading card 220 includes a management proxy module 223, a container runtime module 224, a container storage module 225, and a container network module 226.

[0085] Among them, the management agent module 223 is the overall commander for container management implemented on the offloading card 220. This management agent module 223 can trigger the container runtime module 224 to prepare a container image and a running environment for the container, can trigger the container storage module 225 to prepare storage resources for the container, and trigger the container network module 226 to prepare network resources for the container. The management agent module 223 can also interact with the management node 110 to report the status information of the containers on the computing node 210 and the resource status on the computing node 210 to the management node 110. After the management agent module 223 is started on the offloading card 220, it can communicate with the computing node 210 (such as the front-end agent module 211 on the computing node 210), obtain the resource status on the computing node 210 through the front-end agent module 211, and after obtaining the resource status on the computing node 210, report the resource status on the computing node 210 to the management node 110. When the management node 110 needs to schedule containers, it can use the resource status on the computing node 210 as a reference to determine the target computing node to which the container needs to be scheduled.

[0086] The container runtime module 224 can create a container image for the container and set up a running environment. The embodiments of the present application do not limit the specific form of the container image. For example, the container runtime module 224 obtains the container image from a container image repository and loads the container image onto a storage resource accessible by the offloading card 220 (the container runtime module 224). For example, the storage resource can be a local storage resource of the offloading card 220 or a storage resource connected to the offloading card 220, such as a disk or other storage devices. Then, the container runtime module 224 presents the container image to the computing node 210 in the form of a virtual (virtual function, VF) device (such as the first VF device in the embodiments of the present application). This virtual device can be presented to the computing node 210 in the form of a single root i / o virtualization (SRIOV) device. Here, the protocol supported by the SRIOV device is not limited, and the protocol can be virtio-blk, virtio-scsi, virtio-fs, or virtio-9p, etc.

[0087] The container in the embodiments of the present application can be an ordinary container, that is, the container does not require high security and does not need to be isolated from other containers. The container runtime module 224 can set up a lightweight isolation environment required for the container to run. The setup of this lightweight isolation environment requires configuring namespaces, control groups (cgroups), etc.

[0088] Among them, the namespace is used to isolate the resources required by the container, such as inter-process communication (IPC), network resources, file system, etc. Through the namespace, the resources required by this container can be isolated from the resources required by other containers, achieving the effect of exclusive resource usage. Cgroup is used to limit the resources isolated by the namespace. For example, weights (representing priorities) can be set for these resources, and the usage amounts of the resources can be configured, etc.

[0089] The container can also be a secure container. Compared with ordinary containers, the secure container has higher requirements for security and needs to be isolated from other containers. The container runtime module 224 can first build a secure container virtual machine exclusive to the secure container through the front-end proxy module 211, and then notify the front-end proxy module 211 to create a container in the secure container virtual machine to obtain a container with high security isolation, that is, a secure container. Among them, the secure container virtual machine is a virtual machine specifically deployed for secure containers.

[0090] The container network module 226 is used to prepare network resources for the containers on the computing node 210. Through this network resource, the containers on the computing node 210 can achieve network intercommunication with other containers, enabling the containers to have network capabilities. The network capabilities of the containers can include container-to-container network intercommunication capabilities, service discovery capabilities, and network policy control capabilities.

[0091] When creating a container, the container network module 226 applies for network resources from the virtual network service node 300 through the network proxy module 221. The network resources can include network port resources and can also include other network resources. Then, the applied network resources are associated with a virtual device (such as the fourth VF device in the embodiment of the present application), and the network resources are presented to the computing node 210 in the form of a virtual device for the container to use. The containers on the computing node 210 can interact with other containers through this virtual device.

[0092] The container storage module 225 can prepare storage resources for the containers on the computing node 210, such as block storage resources, file storage resources, and object storage resources. The container storage module 225 can interact with the storage service node 400 through the storage proxy module 222 in the offloading card 220 to apply for storage resources, such as block storage resources and object storage resources, mount the storage resources to the offloading card 220, and then the container storage module 225 presents the storage resources to the containers in the computing node 210 through virtual devices (such as the second VF device and the third VF device in the embodiment of the present application). For the description of the virtual device, reference can be specifically made to the foregoing description, which will not be elaborated here.

[0093] In the embodiments of the present application, the offloading card 220 and the computing node 210 can communicate through the Internet protocol, and the Internet protocol can be the peripheral component interconnect express (PCIe) protocol, that is, the communication channel between the offloading card 220 and the computing node 210 is a PCIe channel. The embodiments of the present application do not limit the specific form of communication between the offloading card 220 and the computing node 210 through the PCIe channel. For example, the offloading card 220 can be connected to the computing node 210 in the form of a network card based on the PCIe protocol and supporting the network protocol stack to communicate with the computing node 210, or can be connected to the computing node 210 in the form of a virtio-vsock device based on the PCIe protocol and the virtio architecture to communicate with the computing node 210. In the embodiments of the present application, only the communication channel between the offloading card 220 and the computing node 210 being a PCIe channel is taken as an example for illustration, and the embodiments of the present application do not limit the specific type of the communication channel between the offloading card 220 and the computing node 210. Any communication channel that can enable communication between the offloading card 220 and the computing node 210 is applicable to the embodiments of the present application.

[0094] The following will respectively describe the container management, the configuration of storage resources, and the configuration method of network resources with reference to the accompanying drawings.

[0095] (1). Container management.

[0096] Container management includes container creation, container deletion, etc., covering the entire life cycle of the container. The processes of container creation and container deletion will be described separately here:

[0097] I. Container creation.

[0098] As Figure 3 shown, it is a schematic diagram of a container creation method provided by the embodiments of the present application. The method includes:

[0099] Step 301: The user sends a container creation request to the management node 110 in the container management cluster 100. The container creation request carries the resource configuration information of the container, and the resource configuration information of the container can indicate the resources required to be occupied by the container. The resources include but are not limited to: processors, memory space, storage resources, network resources (such as the network resources can be the host network, or an independent network, etc.).

[0100] The resource configuration information of a container can describe information such as the type and size of the resources required by the container. For example, the container resource configuration information can indicate the number of processors and the size of the memory space; the container resource configuration information can also indicate that the type of the storage resource is a block storage resource, a file storage resource, or an object storage resource, and the size of the storage resource. The container resource configuration information can also indicate that the network resource is the host network (i.e., the network of the computing node 210 needs to be reused) or an independent network (i.e., a network configured separately for the container and independent of the computing node 210), and can also indicate the service discovery ability (service) and network control policy (network policy) ability that the network resource needs to support, as well as the number of network ports.

[0101] The embodiments of the present application do not limit the way in which the user interacts with the management node 110. For example, the user can select or input the resource configuration information of the container to be created through a client deployed on the user side. After the client detects the resource configuration information of the container selected or input by the user, upon the trigger of the user (such as the user clicks the "Create" option on the interface provided by the client), a container creation request is sent to the management node 110.

[0102] For another example, the user can directly interact with the management node 110. The management node 110 can provide a container creation interface for the user. On this interface, the user can select or input the configuration information of the container to be created, and the user triggers (such as the user clicks the "Create" option on the interface provided by the management node 110) the container creation request.

[0103] Step 302: After receiving the container creation request, the management node 110 schedules the container according to the container configuration information and the resource status of each computing node 210 it manages, and sends a container creation request to the target computing node.

[0104] The resource status of each computing node 210 managed can be pre-collected by the management node 110. For any computing node 210 managed by the management node 110, the management node 110 can obtain the resource status on the computing node 210 through the management agent module 223 on the offloading card 220 inserted on the computing node 210. The resource status on the computing node 210 can indicate the idle resources of the computing node 210. The resources here include but are not limited to: memory space, processors, storage resources, etc.

[0105] The management node 110 can actively send a resource status acquisition request to the management agent module 223 on the offloading card 220 to request the management agent module 223 to report the resource status of the computing node 210. Furthermore, the resource status on the computing node 210 is obtained from the management agent module 223.

[0106] The management agent module 223 on the offloading card 220 can also actively report the resource status of the computing node 210 to the management node 110. For example, after the offloading card 220 is started, it periodically reports the resource status of the computing node 210 to the management node 110.

[0107] The management node 110 scheduling the container means determining the target computing node where the container needs to be deployed and sending a container creation request to the target computing node. There are many ways for the management node 110 to send the container creation request. Two of them are listed below:

[0108] First, the management node 110 schedules the container through the container resource database. The container resource database is a database jointly maintained by each management node 110 in the container management cluster 100. This container resource database records the relevant information of the containers on each computing node 210 in the computing node cluster 200 (such as resource configuration information, container identification information, container status information) and the computing node 210 where the container is located. That is to say, this container resource database includes the correspondence between the container and the computing node 210.

[0109] The management node 110 can select a computing node 210 whose idle resources can support the container from each of these computing nodes 210 as the target computing node. After the management node 110 determines the target computing node, it updates the scheduling result to the container resource database, and this scheduling result indicates the target computing node where the container needs to be deployed.

[0110] When a new container needs to be created, after the management node 110 determines the target computing node, it can update the correspondence between the container and the target computing node to the container resource database.

[0111] It should be noted that the container resource database recording the container and the computing node 210 means recording the resource configuration information of the container, the identification information of the container, and the identification information of the computing node 210. Here, the specific type of the container identification information is not limited. For example, it can be the identification configured for the container when the container is created, or it can be the container name. Any way that can uniquely identify the container is applicable to the embodiments of the present application. Here, the specific type of the computing node 210 identification information is not limited. For example, it can be the identification of the computing node 210 in the computing node cluster 200, or it can be the name of the computing node 210. Any way that can uniquely identify the computing node 210 is applicable to the embodiments of the present application.

[0112] Second, after the management node 110 selects a target computing node capable of deploying the container from each of the computing nodes 210, it can directly send a container creation request to the target computing node (such as the container management module in the target computing node).

[0113] Optionally, the management node 110 can also save the scheduling result in the container resource database.

[0114] After detecting that the management node 110 schedules the container to the target computing node, the management agent module 223 can start creating the container. Creating the container mainly includes two aspects of operations. One is to configure the container image (see step 303), and the other is to install the container running environment (see step 304).

[0115] Step 303: The management agent module 223 uninstalled in the card 220 triggers the container runtime module 224 to create a container image for the container and provides the container image to the target computing node through the first virtual device. The container image is a collection of configuration files and tool libraries required for container operation, such as required library files, system configuration files, system tools, etc.

[0116] For the two ways in which the management node 110 schedules the container to the target computing node listed in step 302, the management agent module 223 can detect the scheduling operation of the management node 110 in the following two ways.

[0117] For the first way, the management agent module 223 can monitor the container resource database in real time, determine the container to be deployed on the target computing node by monitoring the container resource data, and when it monitors that the container resource database is updated, determine whether there is a new container to be deployed to the target computing node according to the added information in the container resource database.

[0118] For the second way, when the management agent module 223 receives the container creation request, it determines that the management node 110 schedules the container to the target computing node.

[0119] When the container runtime module 224 configures the container image for the container, it can obtain the container image from a container image repository deployed remotely, load the container image onto a storage resource accessible by the uninstalled card 220. Then, create a first VF device and bind (also known as associate) the container image to the first VF device. The container runtime module 224 provides the container image to the target computing node through the first VF device. The first VF device can be an SRIOV device.

[0120] It should be noted that when the container runtime module 224 obtains the container image from the container image repository, it can also obtain the container image on demand. That is, only part of the data of the container image is obtained, and this part of the data is associated with the first VF device and provided to the target computing node. During the subsequent startup or running of the container, the container runtime module 224 obtains the other required data of the container image from the container image repository and provides this other data to the target computing node through the first VF device.

[0121] Step 304: The container runtime module 224 in the card 220 is unloaded, and the front-end proxy module 211 in the target computing node builds a running environment for the container in the target computing node.

[0122] For different types of containers, different running environments are built.

[0123] If the container is a normal container, the container runtime module 224 creates a normal container running environment for the container on the target computing node through the front-end proxy module 211, which includes configuring namespaces and cgroups.

[0124] If the container is a secure container, the container runtime module 224 creates a secure container virtual machine for the container on the target computing node through the front-end proxy module 211. After the secure container virtual machine is started, the container runtime module 224 creates a corresponding running environment for the container inside the secure container virtual machine.

[0125] Step 305: The front-end proxy module 211 mounts the first VF device to the root directory (rootfs) of the container under the instruction of the management proxy module 223.

[0126] When the first VF device is an SRIOV device, the front-end proxy module 211 can access the first VF device based on protocols such as virtio-scsi and virtio-blk. After the front-end proxy module 211 detects the first VF device, for a normal container, it can directly mount the first VF device to the root directory of the container. For a secure container, the front-end proxy module 211 can pass through the first VF device to the secure container virtual machine, and the secure container virtual machine mounts the first virtual machine device to the root directory of the container.

[0127] Step 306: After the container is successfully created, the management proxy module 223 synchronizes the status information of the container to the container cluster resource database.

[0128] See Figure 4 the process schematic diagram when creating a container, Figure 4Among them, the management node 110 can schedule a container to the computing node 210. When the offloading card 220 detects that a container is scheduled to the computing node 210, the management agent module 223 can trigger the container runtime module 224 to retrieve the container image accurately for the container, and provide the container image to the target computing node through the first VF device. The management agent module 223 in the target computing node can call the container runtime module 224 to create a running environment for the container.

[0129] II. Container deletion.

[0130] As Figure 5 shown, it is a schematic diagram of a container deletion method provided by an embodiment of the present application. The method includes:

[0131] Step 501: The user sends a container deletion request to the management node 110 in the container management cluster 100. The container deletion request includes the identification information of the container.

[0132] The way for the user to send a container deletion request to the management node 110 in the container management cluster 100 is similar to the way for the user to send a container creation instruction to the management node 110 in the container management cluster 100. For details, please refer to the foregoing description and will not be elaborated here.

[0133] Step 502: After receiving the container creation request, the management node 110 instructs the management agent module 223 in the target computing node to delete the container.

[0134] There are two ways for the management node 110 to instruct the management agent module 223 in the target computing node to delete the container:

[0135] The first way is that the management node 110 marks the container status as deleted in the resource database. The management agent module 223 in the target computing node determines that the container needs to be deleted by monitoring the resource database.

[0136] The second way is that the management node 110 sends a container deletion instruction to the management agent module 223 in the target computing node, instructing the management agent module 223 to delete the container.

[0137] Step 503: The management agent module 223 instructs the container runtime module 224 to delete the container.

[0138] The container runtime module 224 can release the running environment of the container by calling the front-end agent module 211 in the target computing node. The ways to release the running environment of different types of containers are also different.

[0139] If the container is an ordinary container, the front-end proxy module 211 can send an end signal to the process of the container. After the process of the container ends, the namespace and cgroup occupied by the container are released, and the first VF device bound to the container image can also be unmounted.

[0140] If the container is a secure container, the front-end proxy module 211 can first send an end signal to the container process through the secure container virtual machine. After the process of the container ends, the resources occupied by the container in the secure container virtual machine are cleared, and the first VF device bound to the container image is unmounted. After the resources occupied by the container in the virtual machine are cleared, the secure container virtual machine process can be ended.

[0141] (2) Storage resource configuration.

[0142] After creating a container, storage resources can also be configured for the container. For example Figure 6 As shown in the flowchart of configuring storage resources for a container, when the management agent module 223 detects that the management node 110 schedules the container to the target computing node, such as detecting an update of the resource database or receiving a container creation request, it can determine the storage resources that need to be configured for the container according to the updated resource database or the container creation request. The management agent module 223 triggers the container storage module 225 to configure storage resources for the container. The container storage module 225 can apply for storage resources from different types of storage service nodes 400 (such as block storage service nodes, object storage service nodes, and file storage service nodes) through the storage proxy module 222. After that, the container storage module 225 establishes an association relationship between the virtual device and the storage resources through the storage service proxy module, and provides the virtual device to the computing node 210. The front-end proxy module 211 in the computing node 210 can mount the virtual device to the storage directory of the container.

[0143] The storage resources include, but are not limited to, block storage resources, object storage resources, file storage resources, or local storage resources, etc.

[0144] Next, for different types of storage resources, the methods of configuring storage resources for containers will be described.

[0145] 1) Block storage resources, which can be presented in the form of block devices.

[0146] The block storage resource can be pre - applied by the storage proxy module 222 in the offloading card 220, or when the management proxy module 223 determines that a container is scheduled to the target computing node, it triggers the storage proxy module 222 to apply to the storage service node 400. That is, whether it is pre - application or real - time application, the block storage resource is applied by the storage proxy module 222. After the storage proxy module 222 applies for the block storage resource, it can mount the block storage resource into the offloading card 220, that is, present the block storage resource to the offloading card 220 in the form of a device for the offloading card 220 to use.

[0147] The container storage module 225 can create a second VF device and associate the block storage resource with the second VF device. That is to say, establish an association relationship between the block storage resource and the second VF device. During the process that the management proxy module 223 triggers the container runtime module 224 to create a container, the container storage module 225 can notify the front - end proxy module 211 in the target computing node to mount the second VF device to the storage directory of the container. The second VF device can be a virtual device that supports the virito - blk or virtio - scsi protocol.

[0148] Among them, for ordinary containers, the front - end proxy module 211 on the target computing node can directly mount the second VF device to the storage directory of the container (the management proxy module 223 can instruct the front - end proxy module 211 to mount the second VF device to the storage directory of the container). For secure containers, the front - end proxy module 211 on the computing node 210 can pass the second VF device directly to the secure container virtual machine, and the secure container virtual machine mounts the second VF device to the storage directory of the container.

[0149] 2), Object storage resource, which can be presented in the form of a bucket.

[0150] The object storage resource can be pre - applied by the storage proxy module 222 in the offloading card 220, or when the management proxy module 223 determines that a container is scheduled to the target computing node, it triggers the storage proxy module 222 to apply to the storage service node 400. That is, whether it is pre - application or real - time application, the object storage resource is applied by the storage proxy module 222. After the storage proxy module 222 applies for the object storage resource, it can mount the object storage resource into the offloading card 220, that is, present the object storage resource to the offloading card 220 in the form of a device for the offloading card 220 to use.

[0151] The container storage module 225 may create a third VF device and associate the object storage resource with the third VF device, that is, establish an association relationship between the object storage resource and the third VF device. During the process that the management agent module 223 triggers the container runtime module 224 to create a container, the container storage module 225 may notify the computing node 210 to mount the third VF device to the storage directory of the container. The third VF device may be a virtual device that supports the virtio-fs or virtio-9p protocol.

[0152] Among them, for ordinary containers, the proxy module on the computing node 210 may directly mount the third VF device to the storage directory of the container. For secure containers, the proxy module on the computing node 210 may pass through the third VF device to the secure container virtual machine, and the secure container virtual machine mounts the third VF device to the storage directory of the container.

[0153] When the container needs to read data from or store data in the bucket, it can access the bucket through the portable operating system interface (POSIX).

[0154] 3) Local storage resources refer to the storage resources in the computing node 210.

[0155] The container storage module 225 allocates local storage resources for the container through the front-end proxy module 211 in the target computing node. The local storage resources may be a subdirectory of a storage partition in the computing node 210, or an independent storage partition, or an independent storage partition.

[0156] After allocating the local storage resources, during the process that the management agent module 223 triggers the container runtime module 224 to create a container, the management agent module 223 may instruct the front-end proxy module 211 to mount the local storage resources to the storage directory of the container.

[0157] Among them, for ordinary containers, the front-end proxy module 211 may directly mount the local storage resources to the storage directory of the container. For secure containers, the front-end proxy module 211 may use a file sharing protocol (such as the virtio-9p or virtio-fs protocol) to share the local storage resources with the secure container virtual machine. Inside the secure container virtual machine, the secure container virtual machine mounts the local storage resources to the storage directory of the container.

[0158] 4) File storage resources

[0159] The file storage resource can be pre-applied by the storage proxy module 222 in the offloading card 220, or can be triggered by the management proxy module 223 to apply to the storage service node 400 by the storage proxy module 222 when it is determined that the container is scheduled to the target computing node. That is, whether it is pre-applied or applied in real time, the file storage resource is applied by the storage proxy module 222.

[0160] The container storage module 225 mounts the file storage resource in the form of a network file system on the target computing node or the secure container virtual machine through the front-end proxy module 211 for the container to use.

[0161] Among them, for ordinary containers, the front-end proxy module 211 on the computing node 210 can directly mount the network file system to the storage directory of the container. For secure containers, the front-end proxy module 211 on the computing node 210 can mount the network file system to the storage directory of the container in the secure container virtual machine.

[0162] After configuring the storage resource for the container, the data generated during the running of the container can be stored in the storage resource. For the local storage resource, the container can directly store the generated container in the local storage resource.

[0163] For block storage resources and object storage resources, the container can send the generated data to the storage proxy module 222 in the offloading card 220 (specifically, to the storage back-end driver in the storage proxy module 222) by means of the VF devices (such as the second VF device and the third VF device) associated with the storage resource. The storage proxy module 222 sends the generated data to the storage service node 400, and the storage service node 400 stores the generated data in the storage resource allocated for the container.

[0164] For file storage resources, since the file storage service is a storage service built based on network attached storage (NAS) and is network-dependent. Therefore, in the process of storing the generated data in the corresponding file storage resource, the network resources configured for the container need to be used. Specifically, the container can send the generated data to the network proxy module 221 in the offloading card 220, and the network proxy module 221 sends the generated data to the storage node through the network resources (such as ports) configured for the container, and the storage node stores the generated data in the storage resource allocated for the container.

[0165] In the embodiment of the present application, after storage resources are configured for the container, it is also allowed to reclaim the storage resources. The process of reclaiming storage resources is opposite to the process of configuring storage resources. The management agent module 223 can uninstall the VF device associated with the storage resource (such as block storage resource or object storage resource) through the front agent module in the target computing node. After uninstalling the VF device, the management agent module 223 can instruct the container storage module 225 to cancel the association between the storage resource and the VF device. After the association is canceled, the container storage module 225 can instruct the storage agent module 222 to uninstall the storage resource from the unloading card 220.

[0166] For file storage resources, when reclaiming storage resources, the management agent module 223 can uninstall the file storage resources through the front agent module in the target computing node.

[0167] (3) Container network configuration.

[0168] After creating a container, you can also configure network resources for the container. Based on the network resources, data can be exchanged between containers to achieve the network capabilities that the container needs.

[0169] The network capabilities that containers need to have include: network interoperability between containers, service discovery, and network policy control. The following describes these three aspects separately:

[0170] 1) Network interoperability between containers.

[0171] The network interoperability between containers is the most basic network capability that containers must have. This network interoperability between containers requires that data can be exchanged between containers.

[0172] In order to enable the container to have the ability to communicate with other containers through the network, it is necessary to first configure network resources for the container, such as a network port, so that the container can exchange data with other containers through the network port.

[0173] When the management agent module 223 on the target computing node detects that a container is scheduled to the target computing node, if an update of the resource database is detected or a container creation request is received, the network resources that need to be configured for the container can be determined based on the updated resource database or the container creation request, and the management agent module 223 can trigger the container network module 226 to prepare network resources for the container. The container network module 226 applies for network resources from the virtual network service node 300 through the network agent module 221, such as applying for a network port, and obtains information about the network port (such as a port identifier, a port number) and an Internet protocol (IP) address and other information.

[0174] The container network module 226 creates a fourth VF device and establishes an association relationship between the fourth VF device and network resources. The fourth VF device can be a virtual device abstracted from the network card inherent in the offloading card 220, or a virtual device that supports the virtio-net protocol.

[0175] After obtaining network resources, during the process in which the management agent module 223 triggers the container runtime module 224 to create a container, the container network module 226 provides the fourth VF device to the target computing node. The container network module 226 can notify the front-end proxy module 211 in the target computing node to allocate the fourth VF device to the container. Upon receiving this notification, the front-end proxy module 211 in the target computing node allocates the fourth VF device to the container. Among them, for an ordinary container, the front-end proxy module 211 adds the fourth VF device to the namespace of the container. For a secure container, the front-end proxy module 211 can directly pass the fourth VF device to the secure container so that the fourth VF device can be used by the container.

[0176] 2) Container service discovery capability.

[0177] Functionally differentiated, containers can be divided into a back-end server and a front-end application. The front-end application is usually user-facing. Users can operate on the front-end application to meet their own needs, such as clicking "query", "run", etc. in the front-end application. The back-end server can provide operations and data for the front-end application to cooperate with the front-end application to display the final result to the user.

[0178] For any front-end application, it can be connected to multiple different back-end servers, that is, the information from one front-end application can be received by one of the multiple different back-end servers. To further control the information interaction between the back-end server and the front-end application, a service discovery instance can be added. The service discovery instance can be connected to the multiple back-end servers. The service discovery instance can also be distributedly deployed on the computing nodes 210 where the multiple back-end servers are located. The service discovery instance can also be distributedly deployed and inserted in the offloading cards 220 of the computing nodes 210 where the multiple back-end servers are located, and the offloading card 220 cooperates to execute the function of the service discovery instance. As a possible implementation manner, the service discovery instances distributed on each computing node 210 or offloading card 220 can also be configured on the fourth VF device associated with network resources, that is, a load balancing policy is configured on the fourth VF device.

[0179] The service discovery instance can receive information from the front-end application, and the destination address of the information is the address of the service discovery instance. After receiving the information, it transmits the information to one of the multiple back-end servers based on the load balancing policy. Among them, the load balancing policy indicates the rules to be followed for selecting one back-end server from the multiple back-end servers. For example, the load balancing policy can indicate selecting a back-end server in the idle state or selecting the back-end server with the strongest data processing ability. Another example is that the load balancing policy can indicate the proportion of information that the multiple back-end servers can receive.

[0180] The service discovery instance updates the destination address in the information to the address of one of the back-end servers, and sends the information with the updated destination address to one of the back-end servers, so that the back-end server processes data according to the information. The service discovery instance can also feedback the processing result of the back-end server to the front-end application.

[0181] From the above description, it can be seen that the service discovery instance is used to implement load balancing and distribute the information from the front-end application to the back-end servers, and the load balancing policy and the corresponding relationship between the service discovery instance and the back-end servers are configured by users. For example, the management node 110 can configure the discovery service under the operation of the user. The configuration operations performed by the management node 110 include configuring the address of the service discovery instance that supports the discovery service, the corresponding relationship between the service discovery instance and the back-end servers, and the load balancing policy. The container network module 226 in the offloading card 220 inserted on the computing node 210 can monitor the configuration operations of the management node 110 and create a service discovery instance.

[0182] The process of the container network module 226 in the offloading card 220 creating the service discovery instance is mainly a process of configuring service access rules, and the service access rules include the load balancing policy and the corresponding relationship between the address of the service discovery instance and the addresses of the back-end servers. That is to say, the service discovery instance includes the load balancing policy and the corresponding relationship between the service access address and the address of the corresponding container.

[0183] When the container network module 226 determines the address of the container corresponding to the service discovery instance on each computing node 210 through interaction with the management node 110, the address of the container can include the Internet Protocol (IP) address and network port of the container, and configure the load balancing policy and the corresponding relationship between the service access address and the address of the container.

[0184] The embodiments of the present application do not limit the specific type and deployment location of the service discovery instance. For example, the service discovery instance can be centrally deployed on a computing node 210. For another example, the service discovery instance can be distributedly deployed on multiple computing nodes 210. Specifically, the service discovery instance can be distributedly deployed in the offloading cards 220 inserted in each computing node 210. Any instance that can achieve load balancing is applicable to the embodiments of the present application.

[0185] When the container network module 226 determines that a relevant container has changed by interacting with the management node 110, the container network module 226 can also update the service discovery instance according to the address of the container after the change. Specifically, the container network module 226 can monitor the changes of the containers on the computing node 210. The changes of the containers include but are not limited to new creation (a new container is created on the computing node 210), deletion (a container existing on the computing node 210 is deleted), migration (the service of a container on the computing node 210 is migrated to another container), etc. When the container network module 226 determines the address of the container after the change, it updates the correspondence between the service access address and the address of the container after the change.

[0186] When the container after the change is a newly created container, the container network module 226 can add the correspondence between the service access address and the address of the newly created container.

[0187] When the container after the change is a deleted container, the container network module 226 can delete the correspondence between the service access address and the address of the deleted container.

[0188] When the container after the change is a migrated container, the container network module 226 can update the correspondence between the service access address and the address of the container before migration to the correspondence between the service access address and the address of the container after migration.

[0189] After creating or updating the service discovery instance, when the service discovery instance receives information whose destination address is the address of the service discovery instance, it can convert the destination address in the information according to the load balancing policy and the correspondence between the service access address and the address of the container, and forward it to the backend server.

[0190] 3) Network policy control ability.

[0191] Network resources provide the possibility for network interconnection between containers, while the network policy control ability further restricts the way of network interconnection between containers. The network policy control ability is implemented based on the security group policy. The security group policy specifies the containers allowed for interconnection and also the containers not allowed for interconnection. The security group policy includes Access Control Lists (ACL), and the ACL can indicate which container information can be accepted and which container information can be rejected.

[0192] To implement this network policy control ability, a policy control instance can be added. This policy control instance can be connected to multiple containers, centrally deployed on a device, and connected to the computing nodes where the multiple containers are located. This policy control instance can also be distributedly deployed on the computing nodes 210 where the multiple containers are located. This policy control instance can also be distributedly inserted into the offloading cards of the computing nodes 210 where the multiple containers are located, and the offloading card 220 cooperates to execute the functions of the policy control instance. As a possible implementation, the policy control instances distributed on each computing node 210 or offloading card 220 can also be configured on the fourth VF device associated with the network resources, that is, the security group policy is configured on the fourth VF device.

[0193] This policy control instance can receive information from different containers and forward the information. Taking the example that the policy control instance receives information from container 1 and the destination address of the information is the address of container 2, after receiving the information, the policy control instance determines whether the information can be sent to container 2 based on the security group policy. If it is determined that the information can be sent to container 2, the policy control instance forwards the information 2 to container 2. If it is determined that the information cannot be sent to container 2, the forwarding of the information is refused.

[0194] And this security group policy is configured by the user. For example, the user configures the containers allowed for interconnection and the containers not allowed for interconnection on the client. After detecting the user's configuration, the client can send the user's configuration to the management node 110, that is, send the identification information of the containers allowed for interconnection and the identification information of the containers not allowed for interconnection to the management node 110. After receiving the user's configuration, the management node 110 can send an instruction to the container network module 226 through the management proxy module 223. This instruction is used to indicate the identification information of the containers allowed for interconnection and the identification information of the containers not allowed for interconnection. The management node 110 can configure the security group policy under the user's operation. The configuration operations performed by the management node 110 include configuring the corresponding relationships of the containers allowed for interconnection and the corresponding relationships of the containers not allowed for interconnection. The container network module 226 in the offloading card 220 inserted in each computing node 210 can monitor the configuration operations of the management node 110 and create a policy control instance.

[0195] In the process of the container network module 226 in the offloading card 220 creating the policy control instance, it mainly configures the security group policy, which indicates the corresponding relationships between the addresses of the containers allowed to communicate with each other and the corresponding relationships between the addresses of the containers not allowed to communicate with each other.

[0196] When the container network module 226 obtains the addresses of relevant containers by interacting with the management node 110, the addresses of the containers may include the internet protocol (IP) addresses and network ports of the containers, and the security group policy is set according to the configuration operations of the management node 110.

[0197] The embodiments of the present application do not limit the specific type and deployment location of the policy control instance. For example, the policy control instance can be centrally deployed on a computing node 210. For another example, the policy control instance can be distributedly deployed on multiple computing nodes 210. Specifically, the policy control instance can be distributedly deployed in the offloading cards 220 inserted in each computing node 210. Any instance that can achieve load balancing is applicable to the embodiments of the present application.

[0198] When the container network module 226 determines that a relevant container has changed by interacting with the management node 110, the container network module 226 can also update the policy control instance according to the addresses of the changed containers. Specifically, the container network module 226 can monitor the changes of the containers on the computing node 210. The changes of the containers include but are not limited to new creation (a new container is created on the computing node 210), deletion (the existing container on the computing node 210 is deleted), migration (the service of a container on the computing node 210 is migrated to another container), etc. When the container network module 226 determines the addresses of the changed containers, it updates the corresponding relationships between the addresses of the containers allowed to communicate with each other and the corresponding relationships between the addresses of the containers not allowed to communicate with each other.

[0199] When the changed container is a newly created container, the container network module 226 can add the address of the newly created container to the addresses of the containers allowed to communicate with each other, and the corresponding relationship between the address of the newly added container and the addresses of other containers.

[0200] When the changed container is a deleted container, the container network module 226 can delete the corresponding relationship between the address of the deleted container and the addresses of other containers in the security group policy.

[0201] When the changed container is a migrated container, the container network module 226 can update the corresponding relationship between the address of the container before migration and the addresses of other containers in the security group policy to the corresponding relationship between the address of the container after migration and the addresses of other containers.

[0202] After creating or updating a policy control instance, when the policy control instance receives information with a destination address being the address of the policy control instance, it can determine whether it can forward the information according to the security group policy. After determining that the information can be transformed, it transforms the information; otherwise, it rejects forwarding the information.

[0203] Such as Figure 7 is a schematic diagram of the process for configuring a container network. Figure 7 In it, the management node 110 can trigger the container network module 226 to configure network resources for a container by uninstalling the management agent module 223 in the offloading card 220. The container storage module 225 can apply for network resources from a network service node through the network service proxy module. After that, the container network module 226 establishes an association relationship between a virtual device and the network resources through the network service proxy module and provides the virtual device to the computing node 210. The front-end proxy module 211 in the computing node 210 can allocate the virtual device to a container. The management node 110 can trigger the container network module 226 to configure service access rules (such as a load balancing policy) and security group policies for a container by uninstalling the management agent module 223 in the offloading card 220.

[0204] In addition to the network interconnection ability, service discovery ability, and container network policy control ability between containers, it is also possible to configure quality of service (QoS), routing rules, and address mapping rules for a container. Among them, the quality of service is used to standardize the quality of service for information sent by a container, such as standardizing the delay, blocking, monitoring, speed limiting, etc. of the information. The routing rule is used to indicate the gateway to which the information sent by a container needs to be sent, that is, based on this routing rule, the information sent by a container can be routed to the gateway. The address mapping rule is used to implement the conversion between a local area network address and a public network address. The address mapping rule includes NAT and FULL NAT. Among them, NAT includes some or all of the following: SNAT, DNAT, PNAT.

[0205] The configuration of the quality of service, routing rules, and address mapping rules is similar to the configuration method of a service discovery instance. A user can configure some or all of the rules of the quality of service, routing rules, and address mapping rules through a client. After the management node detects the user's configuration, the management node can trigger the container network module 226 to create an instance that can implement some or all of the above rules (this instance can be deployed distributively, that is, centrally deployed on a device. For example, the container network module 226 can configure some or all of the above rules to the fourth VF device. For specific details, please refer to the foregoing content and will not be elaborated here.

[0206] Based on the same inventive concept as the method embodiments, the embodiments of the present application further provide a container management device for executing the method performed by the offloading card in any of the above method embodiments. For related features, reference can be made to the above method embodiments and will not be elaborated herein. As Figure 8 shown, a container management device provided by an embodiment of the present application is shown. The container management device 800 may be located on the offloading card. The offloading card is inserted into the computing node, and a communication channel is established between the container management device 800 and the computing node. The container management device 800 is further connected to the container cluster management node through a network; the container management device 800 is used to manage the containers on the computing node. The container management device 800 includes a transmission unit 801, an acquisition unit 802, and a notification unit 803. Optionally, it further includes a first setting unit 804 and a second setting unit 805.

[0207] The transmission unit 801 is configured to receive a container creation request sent by the container cluster management node. The transmission unit 801 may be used to implement the method of the management agent module 223 receiving the container creation request in the above method embodiments.

[0208] The acquisition unit 802 is configured to obtain a container image according to the container creation request. The acquisition unit 802 may be used to implement the method of the container runtime module 224 obtaining the container image in the above method embodiments.

[0209] The notification unit 803 is configured to notify the computing node to create a container in the computing node according to the container image through the communication channel. The notification unit 803 may be used to implement the method of the container runtime module 224 notifying the computing node to create a container in the above method embodiments.

[0210] As a possible implementation manner, when notifying the computing node to create a container in the computing node according to the container image, the notification unit 803 may further create a first virtual device; and associate the container image with the first virtual device; then, notify the computing node to create a container runtime environment for the container and mount the first virtual device to the root directory of the container.

[0211] As a possible implementation manner, the container management device 800 is further connected to a storage service node through a network. The transmission unit 801 may apply for storage resources from the storage service node; that is, the transmission unit 801 executes the method performed by the storage agent module 222 in the above method embodiments.

[0212] The first setting unit 804 may set a second virtual device according to the storage resources; the first setting unit 804 may execute the method of the container storage module 225 configuring the virtual device in the above method embodiments.

[0213] A notification unit 803 is configured to mount a second virtual device to a directory of a container through a communication channel. The notification unit 803 may execute the method for mounting a virtual device by the container storage module 225 in the above method embodiments.

[0214] As a possible implementation manner, when the first setting unit 804 sets the second virtual device according to the storage resources, it may create the second virtual device; and then associate the storage resources with the second virtual device.

[0215] As a possible implementation manner, the storage resources may be object storage resources or block storage resources. When the storage resources are file storage resources, the notification unit 803 may provide the file storage resources to the containers on the computing nodes in the form of a network file system, and notify the computing nodes to mount the network file system to the directories of the containers.

[0216] As a possible implementation manner, when the notification unit 803 mounts the second virtual device to the directory of the container through the communication channel, when the container is a normal container, the notification unit 803 may mount the second virtual device to the storage directory of the container through the communication channel. When the container is a secure container, the notification unit 803 may directly pass the second virtual device to the secure container virtual machine for deploying the container through the communication channel, and the secure container virtual machine mounts the second virtual device to the storage directory of the container.

[0217] As a possible implementation manner, the container management device is also connected to a network service node through a network. The transmission unit 801 may apply for network resources from the network service node; that is, the transmission unit 801 executes the method performed by the network proxy module 221 in the above method embodiments.

[0218] The second setting unit 805 may set a third virtual device according to the network resources; the second setting unit 805 may execute the method for configuring a virtual device by the container network module 226 in the above method embodiments.

[0219] A notification unit 803 is configured to set a third virtual device in a container through a communication channel. The notification unit 803 may execute the method for mounting a virtual device by the container network module 226 in the above method embodiments.

[0220] As a possible implementation manner, when the second setting unit 805 sets the third virtual device according to the network resources, it may create the third virtual device; and then, associate the network resources with the third virtual device.

[0221] As a possible implementation manner, when the second setting unit 805 sets the third virtual device according to network resources, network processing rules may be set for the third virtual device. The network processing rules include some or all of the following: load balancing policy, security group policy, quality of service, routing rule, address mapping rule.

[0222] As a possible implementation manner, when the notification unit 803 sets the third virtual device in a container through a communication channel, when the container is a normal container, the notification unit 803 adds the third virtual device to the namespace of the container through the communication channel. When the container is a secure container, the notification unit 803 directly passes the third virtual device to the secure container virtual machine for deploying the container through the communication channel.

[0223] As a possible implementation manner, the communication channel is a Peripheral Component Interconnect Express (PCIe) channel.

[0224] It should be noted that the division of units in the embodiments of the present application is illustrative. It is only a logical function division, and there may be other division methods in actual implementation. In the embodiments of the present application, each functional unit may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated units may be implemented in the form of hardware or in the form of software functional units.

[0225] The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more collections of available media. The available media may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium may be a solid state drive (SSD).

[0226] In a simple embodiment, those skilled in the art can conceive that the offloading card or the container management device in the above embodiments can adopt Figure 9 the form shown.

[0227] Such as Figure 9 the device 900 shown, which includes at least one processor 901, a memory 902. Optionally, a communication interface 903 may also be included.

[0228] The memory 902 may be a volatile memory, such as a random access memory; the memory may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 902 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 902 may be a combination of the above memories.

[0229] In the embodiments of the present application, the specific connection medium between the above-mentioned processor 901 and the memory 902 is not limited.

[0230] The processor 901 may be a central processing unit (CPU), and the processor 901 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, artificial intelligence chips, chips on chips, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. When the processor 1201 communicates with other devices, data transmission may be performed through the communication interface 903, such as receiving a container creation request, applying for storage resources, and applying for network resources.

[0231] When the container management device adopts Figure 9 the form shown, Figure 9 the processor 901 in

[0232] Specifically, Figure 8The functions / implementation processes of the transmission unit 801, the acquisition unit 802, the notification unit 803, the first setting unit 804, and the second setting unit 805 can all be implemented by Figure 9 the processor 901 in Figure 8 calling the computer-executable instructions stored in the memory 902. Or, Figure 9 the functions / implementation processes of the acquisition unit 802, the notification unit 803, the first setting unit 804, and the second setting unit 805 in Figure 8 can be implemented by Figure 9 the processor 901 in

[0233] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0234] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one or more flows and / or Figure 1 blocks or multiple blocks.

[0235] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one or more flows and / or Figure 1 blocks or multiple blocks.

[0236] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the process Figure 1 in one process or a plurality of processes and / or boxes Figure 1 steps for the functions specified in one box or a plurality of boxes.

[0237] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to cover these modifications and variations.

Claims

1. A computer system, characterized in that, it includes an offloading card and a computing node. The offloading card is inserted into the computing node, and a communication channel is established between the offloading card and the computing node. The offloading card is also connected to a container cluster management node through a network, and the offloading card is also connected to a storage service node through a network. Wherein, the offloading card is used to receive a container creation request sent by the container cluster management node, obtain a container image according to the container creation request; create a first virtual device; associate the container image with the first virtual device; notify the computing node to create a container running environment for the container and mount the first virtual device to the root directory of the container; the computing node is used to create a container running environment for the container and mount the first virtual device to the root directory of the container; the offloading card is further used to: apply for storage resources from the storage service node; set a second virtual device according to the storage resources; the computing node is further used to: obtain the second virtual device through the communication channel and mount the second virtual device under the directory of the container.

2. The system according to claim 1, characterized in that, the offloading card is also connected to a network service node through a network. The offloading card is further used to: apply for network resources from the network service node; set a third virtual device according to the network resources; the computing node is further used to obtain the third virtual device through the communication channel and set the third virtual device in the container.

3. The system according to any one of claims 1 to 2, characterized in that, the communication channel is a Peripheral Component Interconnect Express (PCIe) channel.

4. A container management method, characterized in that, the method is applied to an offloading card. The offloading card is inserted into a computing node, and a communication channel is established between the offloading card and the computing node. The offloading card is also connected to a container cluster management node through a network, and the offloading card is also connected to a storage service node through a network. The method includes: receiving a container creation request sent by the container cluster management node; obtaining a container image according to the container creation request; creating a first virtual device; associating the container image with the first virtual device; notifying the computing node to create a container running environment for the container and mount the first virtual device to the root directory of the container; applying for storage resources from the storage service node; setting a second virtual device according to the storage resources; mounting the second virtual device to the directory of the container through the communication channel.

5. The method according to claim 4, characterized in that, the setting the second virtual device according to the storage resources includes: creating the second virtual device; associating the storage resources and the second virtual device.

6. The method according to claim 4 or 5, characterized in that, the storage resources include some or all of the following: object storage resources, block storage resources.

7. The method according to claim 4 or 5, characterized in that, Mounting the second virtual device under the directory of the container through the communication channel includes: If the container is a secure container, directly passing the second virtual device to the secure container virtual machine for deploying the container, and mounting the second virtual device into the directory of the container by the secure container virtual machine.

8. The method according to claim 4 or 5, characterized in that, The unload card is further connected to a network service node through a network, and the method further includes: Applying for network resources from the network service node; Setting a third virtual device according to the network resources; Setting the third virtual device in the container through the communication channel.

9. The method according to claim 8, characterized in that, Setting the third virtual device according to the network resources includes: Creating the third virtual device; Associating the network resources with the third virtual device.

10. The method according to claim 8, characterized in that, Setting the third virtual device according to the network resources further includes: Setting network processing rules for the third virtual device, and the network processing rules include some or all of the following: load balancing policy, security group policy, routing rules, address mapping rules, quality of service.

11. The method according to claim 8, characterized in that, Setting the third virtual device in the container through the communication channel includes: Adding the third virtual device to the namespace of the container.

12. The method according to claim 8, characterized in that, Setting the third virtual device in the container through the communication channel includes: If the container is a secure container, directly passing the third virtual device to the secure container virtual machine for deploying the container.

13. The method according to any one of claims 4, 5, 9, 10, 11, and 12, characterized in that, The communication channel is a Peripheral Component Interconnect Express (PCIe) channel.

14. A container management device, characterized in that, The device is applied to an unload card, the unload card is inserted into a computing node, a communication channel is established between the device and the computing node, the device is further connected to a container cluster management node through a network, and the device is further connected to a storage service node through a network. The device includes a transmission unit, an acquisition unit, a notification unit, and a first setting unit: The transmission unit is configured to receive a container creation request sent by the container cluster management node; The acquisition unit is configured to acquire a container image according to the container creation request; The notification unit is configured to create a first virtual device; Associating the container image with the first virtual device; notifying the computing node to create a container runtime environment of the container and mounting the first virtual device to the root directory of the container; The transmission unit is further configured to apply for storage resources from the storage service node; The first setting unit is configured to set a second virtual device according to the storage resources; The notification unit is further configured to mount the second virtual device under the directory of the container through the communication channel.

15. The device according to claim 14, wherein, when the first setting unit sets the second virtual device according to the storage resource, it is further configured to: create the second virtual device; associate the storage resource and the second virtual device.

16. The device according to claim 14 or 15, wherein, the storage resource includes part or all of the following: object storage resource, block storage resource.

17. The device according to claim 14 or 15, wherein, when the notification unit mounts the second virtual device to the directory of the container through the communication channel, it is specifically configured to: if the container is a secure container, directly pass the second virtual device to the secure container virtual machine for deploying the container, and the secure container virtual machine mounts the second virtual device to the directory of the container.

18. The device according to claim 14 or 15, wherein, the device is further connected to a network service node through a network, and the device further includes a second setting unit; the transmission unit is configured to apply for network resources from the network service node; the second setting unit is configured to set a third virtual device according to the network resources; the notification unit is configured to set the third virtual device in the container through the communication channel.

19. The device according to claim 18, wherein, when the second setting unit sets the third virtual device according to the network resources, it is specifically configured to: create the third virtual device; associate the network resource and the third virtual device.

20. The device according to claim 18, wherein, when the second setting unit sets the third virtual device according to the network resources, it is further configured to: set network processing rules for the third virtual device, and the network processing rules include part or all of the following: load balancing policy, security group policy, routing rule, address mapping rule, quality of service.

21. The device according to claim 18, wherein, when the notification unit sets the third virtual device in the container through the communication channel, it is specifically configured to: add the third virtual device to the namespace of the container.

22. The device according to claim 18, wherein, when the notification unit sets the third virtual device in the container through the communication channel, it is specifically configured to: if the container is a secure container, directly pass the third virtual device to the secure container virtual machine for deploying the container.

23. The device according to any one of claims 14, 15, 19, 20, 21, 22, wherein, the communication channel is a Peripheral Component Interconnect Express (PCIe) channel for high-speed external devices.

24. A device, wherein, it includes a memory and a processor; the memory stores program instructions, and the processor runs the program instructions to execute the method according to any one of claims 4 to 13.

25. A computer-readable storage medium, wherein, Instructions are stored in a computer-readable storage medium, which, when run on a computer, cause the computer to execute the method according to any one of claims 4 to 13.

Images