Container Network Management System

Abstract

This application relates to a container network management system. The system includes: a network management module and at least one container cluster. The container cluster includes multiple computing nodes, and a container network interface plugin and resource objects are deployed on each computing node. Among them, the network management module divides multiple IP address pools according to the IP addresses of each computing node; each computing node calls the container network interface plugin to determine the target IP address corresponding to the resource object from multiple IP address pools according to the container configuration file of the resource object. In this system, the network architecture of the container cluster is deployed through the container network interface plugin, unifying the network mode within each container cluster. In addition, by pre-dividing the address pool through the network management module, isolation of IP resources and isolation of network areas are achieved under the tenant application. At the same time, when each resource object is restarted or newly created, the target IP address can be quickly and accurately obtained through the pre-divided address pool.

Description

Technical Field

[0001] This application relates to the technical fields of computer networks and cloud computing, and particularly to a container network management system. Background Art

[0002] Kubernetes (abbreviated as k8s) is a distributed architecture solution based on container technology. Generally, k8s itself does not provide network functions, but network plugins are used to provide corresponding network solutions. For example, the network plugin can be a Container Network Interface (CNI) plugin, and different network solutions can be integrated into k8s through CNI.

[0003] Taking the solution to the cross-host communication problem as an example, in related technologies, when CNI solves the cross-host communication problem, according to the dependency relationship with the host network, the implementation modes of the network solution include but are not limited to: overlay mode, routing mode, and underlay mode. Among them, the underlay mode, as the network mode with the strongest dependence on the underlying layer, has advantages in network performance. In practical applications, the network implementation solution of underlay can usually adopt the macvlan network mode.

[0004] However, in related technologies, when building containers in the macvlan network mode, the IP addresses of the containers cannot be effectively managed. Summary of the Invention

[0005] Based on this, to solve the above technical problems, it is necessary to provide a container network management system that can effectively manage the IP addresses of each container in a container cluster. The system includes: a network management module and at least one container cluster. The container cluster includes multiple computing nodes, and each computing node is deployed with a container network interface plugin and a resource object;

[0006] The network management module divides multiple IP address pools according to the IP addresses of each computing node; the multiple IP address pools are isolated from each other;

[0007] Each computing node determines the target IP address corresponding to the resource object from the multiple IP address pools by calling the container network interface plugin according to the container configuration file of the resource object.

[0008] In one embodiment, the container cluster further includes a master node, and the master node includes an IP address management unit; each computing node includes a node management unit, and the container network interface plugin includes an IP address management plugin;

[0009] Each computing node determines the target IP address corresponding to the resource object from the multiple IP address pools by calling the container network interface plugin according to the container configuration file of the resource object, including:

[0010] The node management unit calls the IP address management plugin and sends an IP address request to the IP address management unit; the container configuration file of the resource object is carried in the IP address request.

[0011] The IP address management unit determines the target IP address from multiple IP address pools according to the container configuration file of the resource object, and returns the target IP address to the node management unit.

[0012] In one embodiment, the container cluster further includes a resource status database, and multiple IP address pools are located in the resource status database;

[0013] The IP address management unit determines the target IP address from multiple IP address pools according to the container configuration file of the resource object, including:

[0014] The IP address management unit determines the IP address planning information of the resource object according to the container configuration file of the resource object.

[0015] The IP address management unit determines the target address pool from the resource status database according to the IP address planning information, and obtains the target IP address from the target address pool.

[0016] In one embodiment, obtaining the target IP address from the target address pool includes:

[0017] If there is a specified IP address in the container configuration file, the IP address management unit determines the IP address in the target address pool that is the same as the specified IP address as the target IP address;

[0018] If there is no specified IP address in the container configuration information, the IP address management unit determines the target IP address from the target address pool according to the IP address lookup policy.

[0019] In one embodiment, the IP address lookup policy includes: sorting according to the release time of the IP address. If the current state of the resource object is the restart state, the IP address with the latest release time in the target address pool is determined as the target IP address; if the current state of the resource object is the new state, the IP address with the earliest release time is determined as the target IP address.

[0020] In one embodiment, the system further includes:

[0021] Each computing node binds at least two physical network cards in its own node to obtain a logical network card; the IP address space of the logical network card is stored in the resource status database and is used to create multiple IP address pools.

[0022] Each computing node creates multiple virtual local area network (VLAN) interfaces on a logical network card; the multiple VLAN interfaces are used to isolate multiple resource objects on each computing node.

[0023] In one embodiment, each computing node creates multiple VLAN interfaces on a logical network card, including:

[0024] Each computing node creates multiple VLAN interfaces on a logical network card by using physical network card virtualization.

[0025] In one embodiment, the system further includes:

[0026] Each computing node assigns the IP addresses of the multiple VLAN interfaces to multiple resource objects on the computing node; among them, the resource objects connected to the same VLAN interface communicate through the VLAN interface.

[0027] In one embodiment, the network management module divides multiple IP address pools according to the IP addresses of each computing node, including:

[0028] The network management module reads the IP address space of the logical network card of each computing node from the resource status database, and uses the IP address spaces of the logical network cards of multiple computing nodes in the container cluster as the total address pool;

[0029] The network management module divides multiple IP address pools from the total address pool according to a preset address pool division strategy;

[0030] Among them, the total address pool is used for the expansion of multiple IP address pools.

[0031] In one embodiment, the address pool division strategy includes application deployment requirements and container deployment requirements, and the multiple IP address pools include an application pool, a network space pool, and a default pool;

[0032] The network management module divides multiple IP address pools from the total address pool according to a preset address pool division strategy, including:

[0033] The network management module divides an application address pool from the total address pool according to application deployment requirements;

[0034] The network management module divides a network space pool and a default pool from the total address pool according to container deployment requirements;

[0035] Among them, the application pool, the network space pool, and the default pool are isolated from each other; the IP addresses in the application address pool are reserved addresses for the target application, the IP addresses in the network space pool are candidate addresses for the target container, and the IP addresses in the default pool are candidate addresses for non-target containers.

[0036] In one embodiment, the system further includes:

[0037] If the container cluster is a dual-stack cluster, the container network interface plugins deployed on each computing node support both the first Internet protocol and the second Internet protocol;

[0038] wherein, the first Internet protocol and the second Internet protocol are different protocols, and the computing node stores the corresponding relationship between the first Internet protocol and the second Internet protocol.

[0039] The container network management system provided in this application includes a network management module and at least one container cluster. The container cluster includes multiple computing nodes, and each computing node deploys a container network interface plugin and a resource object. Among them, the network management module divides multiple IP address pools according to the IP addresses of each computing node; each computing node calls the container network interface plugin to determine the target IP address corresponding to the resource object from multiple IP address pools according to the container configuration file of the resource object. In this system, the network architecture of the container cluster is deployed through the container network interface plugin, unifying the network mode within each container cluster. In addition, the network management module pre-divides multiple IP address pools, and the multiple IP address pools are isolated from each other. In this way, by pre-dividing the address pools by the network management module, the isolation of IP resources and the isolation of network areas are realized under the tenant application. At the same time, when each resource object is restarted or newly created, the target IP address can be quickly and accurately obtained through the pre-divided address pool, achieving the technical effect of effectively managing the IP addresses in the container cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic application diagram of the CNI model in one embodiment;

[0041] Figure 2 It is a schematic structural diagram of the container network management system in one embodiment;

[0042] Figure 3 It is a schematic structural diagram of the container network management system in another embodiment;

[0043] Figure 4 It is a schematic diagram of the acquisition process of the target IP address in one embodiment;

[0044] Figure 5 It is a schematic structural diagram of the container network management system in another embodiment;

[0045] Figure 6 It is a schematic diagram of the acquisition process of the target IP address in another embodiment;

[0046] Figure 7 It is a schematic diagram of the IP address division method in one embodiment;

[0047] Figure 8 Schematic diagram of IP address pool division in an embodiment

[0048] Figure 9 Schematic diagram of the process of virtualizing the network card of a computing node in an embodiment

[0049] Figure 10 Schematic diagram of the MACVlan networking solution in an embodiment Detailed implementation manners

[0050] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0051] Cloud computing resources usually adopt an operation mode in which a management platform (for example, the open-source cloud computing management platform OpenStack) uniformly manages multiple resources and provides them to multiple tenants at the same time. The open-source project Kubernetes (abbreviated as K8s) based on OpenStack is one of the most widely used container cluster management systems at present. It enables enterprises to use the cloud computing resources of the cluster like using a single computer, improves the utilization efficiency of computer resources, and solves many problems such as the automated deployment, elastic scaling, and life cycle management of applications.

[0052] Since K8s itself does not provide network functions, its network solutions are provided by network plugins, and these network plugins complete network communication in the K8s container cluster by configuring the interface specifications of the Container Network Interface (CNI).

[0053] Figure 1 It is an application schematic diagram of a CNI model provided by the present application. As Figure 1 shown, the container running environments on each computing node in the container cluster are connected to various network plugins (Plugin) through CNI. The network plugins include: Loopback plugin, Bridge plugin, Precision Time Protocol (PTP) plugin, IP Virtual local area network (IPVlan) plugin, Media Access Control Virtual local area network (MACVlan) plugin, and other third-party plugins.

[0054] Based on Figure 1In the CNI model shown, a container in the container runtime environment can bind multiple network plugins through CNI and thus join multiple networks. CNI only focuses on allocating network resources when creating a container and deleting network resources when destroying a container, which makes the CNI specification very lightweight and easy to implement, and has been widely used.

[0055] Among them, only two concepts are involved in the CNI model: Container and Network. A container is an environment with an independent Linux network namespace. A container needs to have its own Linux network namespace, which is a necessary condition for joining a network. A network represents a group of entities that can be interconnected, and these entities have their own independent and unique IP addresses. These entities can be containers, physical machines, or other network devices (such as routers), etc.

[0056] The settings and operations of the CNI for container networks are specifically implemented through plugins. CNI plugins include two types: CNI plugins and IP Address Management (IPAM) plugins. CNI plugins are responsible for configuring network resources for containers, and IPAM plugins are responsible for allocating and managing the IP addresses of containers. Among them, the IPAM plugin, as part of the CNI plugin, works in coordination with the CNI plugin.

[0057] In practical applications, when CNI completes network communication in the k8s container cluster, two problems need to be solved: one is how to establish the network stack of the base container (i.e., the infra container); the other is how to solve cross-host communication.

[0058] For the first problem, building a container network stack requires an IP address and corresponding gateway routing information. Therefore, it can be achieved through the IPAM plugin in the CNI plugin. Specifically, when the CNI plugin is running, it calls the IPAM plugin to obtain the corresponding IP address and passes it to the network namespace (Network Namespace, netns) belonging to the container.

[0059] For the second problem, CNI has multiple implementation models for solving cross-host communication problems. According to the dependency relationship with the host (i.e., the computing node where the container is deployed) network, it is divided into three models: the overlay network model (i.e., the Overlay network model), the routing network model, and the underlying physical network model (i.e., the Underlay network model).

[0060] (1) Overlay network model

[0061] The most important feature of the Overlay network model is to create tunnels between host machines and achieve cross-host network communication through tunnel forwarding. The essence of tunnel forwarding is to encapsulate the communication packets of both containers into packets between their respective host machines and complete data exchange through the network tunnels of the host machines. The basic requirement for this virtual network is that each host machine only needs to support the tunnel protocol, and there are no special requirements for the underlying network. In this virtual network, the container cluster has a high degree of autonomy over IP addresses. The IP segments used by containers are independent of the host and do not preempt the host's IP resources. Once network traffic leaves the host machine where the container is located, it will be encapsulated into packets between host machines and does not depend on the underlying network. Typical representatives of the Overlay model include: the VXLAN network mode of Flannal, the IPIP mode of calico, Weave, etc.

[0062] Among them, Flannel is a container network example in K8s for achieving better network communication between containers and between hosts. Flannel organizes all Pods in a virtual large second-layer network (the data link layer of the Open Systems Interconnections (OSI) network model, and the second-layer network processes frame transmission between two adjacent nodes on the network) of the same subnet. The backend forwarding methods supported by Flannel include (Virtual eXtensible Local Area Network, VXLAN) and host-gw. The above-mentioned Pod is the smallest / simplest basic unit created or deployed by K8s, and a Pod represents a process running on the cluster. A Pod can encapsulate one or more application containers, storage resources, an independent network IP, and policy options for managing and controlling the running mode of the container.

[0063] Calico is a pure three-layer network plugin, and Calico includes two network modes: IPIP and BGP. Weave is a network option for K8s CNI. Weave creates a mesh overlay network between each node in the container cluster, and flexible routing can be performed between participants.

[0064] (2) Routing network model

[0065] The routing network model mainly achieves cross-host communication through routing. The container and the host machine belong to different network segments. The main difference between the routing mode and the Overlay mode is that there is no need to establish a tunnel for communication. However, it requires the underlying network to have the ability to reach the second layer and has a certain dependence on the underlying network. Typical representatives include the host-gw network mode of Flannel, the BGP network mode of calico, etc.

[0066] (3)Underlay Network Model

[0067] In the Underlay network model, containers and the host machine are on the same layer of the network, sharing IP resources with the host machine. Communication between containers strongly depends on the underlying network. Its typical representatives include the SR-IOV mode and the MACVlan mode, etc.

[0068] Among them, SR-IOV stands for Single Root I / O Virtualization, that is, single-layer I / O virtualization. The SR-IOV technology is a hardware-based virtualization solution that can improve performance and scalability. The SR-IOV standard allows for the efficient sharing of Peripheral Component Interconnect Express (PCIe) devices between virtual machines, and it is implemented in hardware, enabling I / O performance comparable to that of the native machine.

[0069] The MACVlan technology is essentially a network card virtualization technology. It does not require the creation of a Linux bridge. Instead, it creates virtual sub-interfaces on the physical Ethernet interface. Each sub-interface has its own MAC address, and logically these virtual sub-interfaces are equivalent to the physical network card. The effect of using the MACVlan technology is that multiple IP addresses can be bound to a single physical network card, and each IP address has its own MAC address.

[0070] However, the function implementation of MACVlan in the open-source community is weak. It only supports all pods sharing a single parent interface, without any isolation, does not support service communication, does not support IPv6, and there is no corresponding IP management plugin, etc.

[0071] Among them, a service defines such an abstraction: a logical grouping of pods and a policy for accessing them. That is to say, this group of pods can be accessed by the service, and the service can be regarded as the external interface of a group of pods providing the same service.

[0072] For traditional architectures, when migrating from a virtualization platform to a container platform, some old habits need to be maintained, such as making security policies based on IP, and the IP addresses occupied by services after containerization cannot change. Then, how to maximize compatibility with the migration from virtualization to containers and maintain extreme network performance and unified network resource management is the core issue that the container platform needs to solve. In this application, the MACVlan mode is selected to address the problems in the above solutions, solve the problem of no isolation in the network area in the MACVlan mode in related technologies, and implement the corresponding IP address management function. At the same time, the control plane is added to manage the IP address pools under different CNI plugins, maximizing compatibility with the transformation of traditional applications to containerization.

[0073] In one embodiment, as Figure 2 shown, a container network management system is provided. The system 100 includes a network management module 110 and at least one container cluster 120. The container cluster 120 includes multiple computing nodes 121, and a container network interface plugin 1211 and a resource object 1212 are deployed on each computing node 121. The network management module 110 divides multiple IP address pools according to the IP addresses of each computing node 120. The multiple IP address pools are isolated from each other. Each computing node 120 determines the target IP address corresponding to the resource object 1212 from the multiple IP address pools by calling the container network interface plugin 1211 according to the container configuration file of the resource object 1212.

[0074] Among them, the container network can be a network system based on K8s, and the resource object can be a Pod and / or a container.

[0075] As Figure 2 shown, the container network management system includes a container network management module and at least one container cluster. The network management module can manage the IP addresses of each container cluster separately and divide the address pools. It can also manage the IP addresses of at least one container cluster uniformly. The embodiments of this application do not limit this.

[0076] As an example, the container network management module can be deployed in a switch, a router, or a terminal device. The multiple computing nodes in at least one container cluster are deployed on multiple servers, and each computing node corresponds to one server. A container running environment is provided inside each computing node, and multiple pods can be deployed. One or more containers can be created in each pod. It should be noted that this application does not limit the number of computing nodes, pods, and containers in the container cluster, and they can be deployed or created according to actual needs.

[0077] In a possible implementation, when the network management module divides multiple address pools, it can put different IP addresses into the same IP address pool for applications under a tenant. That is, all applications of a tenant are deployed in one IP address pool to achieve isolation of IP resources under different tenants. It can also put different IP addresses into different IP address pools for applications under the same tenant. That is, different applications of a tenant are deployed in different IP address pools to achieve isolation between network regions.

[0078] Furthermore, this application can also combine the IP address pool with the underlying Vlan, and allocate IP addresses under different Vlans to different IP address pools, which can achieve isolation of the second-layer networks of different Vlans.

[0079] In addition, containers in the computing node need to obtain corresponding IP addresses when restarting and creating new ones. Therefore, the computing node calls the CNI plugin to obtain the target IP address according to the container configuration file of the resource object to which the container belongs.

[0080] Among them, the container configuration file includes but is not limited to: computing node name, pod name and IP address, number of containers, and deployed applications. The CNI plugin includes but is not limited to the MACVlan plugin and the IPAM plugin.

[0081] In the embodiment of this application, the container network management system includes a network management module and at least one container cluster. The container cluster includes multiple computing nodes, and a container network interface plugin and a resource object are deployed on each computing node. Among them, the network management module divides multiple IP address pools according to the IP addresses of each computing node; each computing node calls the container network interface plugin to determine the target IP address corresponding to the resource object from multiple IP address pools according to the container configuration file of the resource object. In this system, the network architecture of the container cluster is deployed through the container network interface plugin, unifying the network mode within each container cluster. In addition, the network management module has pre-divided multiple IP address pools, and the multiple IP address pools are isolated from each other. In this way, through the pre-divided address pools by the network management module, isolation of IP resources and isolation of network regions are achieved under tenant applications. At the same time, when each resource object restarts or is newly created, the target IP address can be quickly and accurately obtained through the pre-divided address pool, achieving the technical effect of effectively managing the IP addresses in the container cluster.

[0082] Based on the above Figure 2 shown container network management system, in one embodiment, as Figure 3As shown in the figure, in the system 100, the container cluster 120 further includes a master node 122, and the master node 122 includes an IP address management unit 1221; each computing node 121 includes a node management unit 1213, and the CNI plugin 1211 includes an IP address management plugin.

[0083] Among them, the master node is used to manage the application deployment of multiple computing nodes in the container cluster to deploy Pods to appropriate computing nodes. The number of master nodes can be one, deployed on one server, or multiple, respectively deployed on multiple servers; in addition, the master node is also used to provide the application programming interface (API) service of K8s, as the unified entry of system management instructions, and operations such as adding, deleting, modifying, and querying resource objects are processed by the API server and then stored.

[0084] In actual implementation, the node management unit in the computing node can be implemented through the pre-deployed Kubelet software, and Kubelet is used to maintain and manage all containers on the computing node to make the running state of the Pod consistent with the expected state. The container runtime environments currently supported by K8s include Docker and Rocket, etc.

[0085] In addition, in this application, in order to effectively manage IP addresses, the deployed CNI plugin is an IP address management plugin, that is, the above-mentioned IPAM plugin, and the target IP address is obtained through the IPAM plugin in specific applications.

[0086] In one embodiment, as Figure 4 shown, the implementation process in which each computing node determines the target IP address corresponding to the resource object by calling the container network interface plugin from multiple IP address pools according to the container configuration file of the resource object includes the following steps:

[0087] Step 410: The node management unit calls the IP address management plugin and sends an IP address request to the IP address management unit; the IP address request carries the container configuration file of the resource object.

[0088] In a possible implementation manner, when the node management unit listens to the application distribution and pod creation events from the master node, it calls the CNI interface, and then calls the decompressed MACVlan and IPAM binary files in the local / opt / cni / bin directory to enable the IPAM plugin. Then, the node management unit reads the container configuration file of the resource object and sends an IP address request to the IP address management unit.

[0089] Step 420: The IP address management unit determines a target IP address from multiple IP address pools according to the container configuration file of the resource object, and returns the target IP address to the node management unit.

[0090] In a possible implementation, the IP address management unit reads IP indication information from the container configuration file of the resource object, determines the only IP address pool to which the target IP address belongs from multiple IP address pools according to the IP indication information, and then obtains the target IP address from the uniquely determined IP address pool. Moreover, after obtaining the target IP address, the IP address management unit returns the target IP address to the node management unit, so that the node management unit can assign the target IP address to the resource object.

[0091] In this embodiment, the node management unit manages the deployment of each resource object in each computing node. When creating or restarting a container, the node management unit calls the IP address management plugin according to the container configuration file of each resource object, sends an IP address request to the IP address management unit, and then the IP address management unit determines the target IP address of the resource object from multiple address pools. In this way, through the IP address management unit, the target IP address can be accurately and effectively determined for each resource object.

[0092] Based on the above Figure 3 shown container network management system, in one embodiment, as Figure 5 shown, in this system 100, the container cluster 120 further includes a resource status database 123, and multiple IP address pools are located in the resource status database 123.

[0093] Among them, the resource status database is used to store the available IP resources of multiple computing nodes. The network management module divides multiple IP address pools in the resource status database according to the IP addresses of each computing node, and the multiple IP address pools are isolated from each other. When each computing node starts or creates a new resource object, the target IP address corresponding to the resource object is obtained from the resource status database through the IP address management unit.

[0094] After introducing the resource status database, in one embodiment, as Figure 6 shown, the implementation process in which the IP address management unit determines the target IP address from multiple IP address pools according to the container configuration file of the resource object in the above step 420 includes the following steps:

[0095] Step 610: The IP address management unit determines the IP address planning information of the resource object according to the container configuration file of the resource object.

[0096] Among them, the IP address planning information may include IP address specification information and / or IP address pool specification information.

[0097] As an example, when the IP address planning information is the IP address pool specification information, the relevant information of the specified IP address pool can be written into the annotation of the metadata of the resource object. Metadata is data used to describe resource objects and contains a set of attributes defined by different names, such as labels, annotations, namespaces, etc. An annotation is non-identifying metadata used to attach any non-identifying metadata to a resource object.

[0098] Step 620: The IP address management unit determines the target address pool from the resource status database according to the IP address planning information, and obtains the target IP address from the target address pool.

[0099] Among them, the resource status database includes multiple address pools, and the target address pool is any address pool in the resource status database.

[0100] As an example, the resource status database can be an etcd database. Etcd is a highly available distributed key-value database.

[0101] In this embodiment, the container network management system includes a resource status database. By storing the IP address pools divided by multiple network management modules in the resource status database, the security and unified allocation of IP addresses are ensured. In this way, after receiving the container configuration file of the resource object, the IP address management unit can determine the target address pool from the resource status database according to the IP address planning information of the resource object, so as to obtain the target IP address from the target address pool.

[0102] In one embodiment, the present application also provides an IP address pool division method, which is applied to any of the above container network management systems, and its execution entity can be a network management module. As Figure 7 shown, the network management module divides multiple IP address pools according to the IP addresses of each computing node, including the following steps:

[0103] Step 710: The network management module reads the IP address spaces of the logical network cards of each computing node from the resource status database, and uses the IP address spaces of the logical network cards of multiple computing nodes in the container cluster as the total address pool.

[0104] Among them, the network management module manages the IP addresses of multiple container clusters and docks with the ipam server side of the IPAM plugin downward. The server side corresponds to the resource status database and serves as the entry for the IP management of the entire container cluster.

[0105] In this step, the network management module uses the IP address space of the logical network cards of multiple computing nodes in the container cluster as the total address pool, and divides multiple IP address pools from the total address pool. Therefore, when the IP address management unit obtains the target IP address of a resource object, it can first determine the target address pool from multiple IP address pools, and then obtain the target IP address of the resource object from the target address pool. In this way, the IP address management unit does not need to query the total address pool in the resource status database to obtain the target IP address, and can quickly and accurately obtain the target IP address from the target address pool, reducing the query time and improving the efficiency of obtaining the target IP address.

[0106] Step 720: The network management module divides multiple IP address pools from the total address pool according to the preset address pool division policy.

[0107] Among them, the total address pool is used for the expansion of multiple IP address pools and is a pool without any allocation / reservation operations. Under no circumstances will an IP address be obtained from the total address pool. This total address pool can dynamically provide elastic scaling IP addresses for multiple IP address pools as the container horizontally scales (Horizontal Pod Autoscaling, HPA).

[0108] It should be noted that this application introduces the concept of network space (netspace) in the network management module. The network space can be flexibly divided according to application needs, and different network spaces are isolated from each other. Based on this concept, this application divides multiple IP address pools from the total address pool. An IP address pool is a network space, and the IP address pools are isolated from each other.

[0109] In a possible implementation manner, the address pool division policy includes application deployment requirements and container deployment requirements, and the multiple IP address pools include an application pool, a network space pool, and a default pool.

[0110] Furthermore, the implementation process of the network management module dividing multiple IP address pools from the total address pool according to the preset address pool division policy can be: the network management module divides an application address pool from the total address pool according to the application deployment requirements; the network management module divides a network space pool and a default pool from the total address pool according to the container deployment requirements. The application pool, the network space pool, and the default pool are isolated from each other; the IP addresses in the application address pool are reserved addresses for the target application, the IP addresses in the network space pool are candidate addresses for the target container, and the IP addresses in the default pool are candidate addresses for non-target containers.

[0111] It should be noted that applications can be divided into stateful applications and stateless applications. The difference lies in whether the state information is saved by the requestor or the responder. Since the requestor is responsible for saving, it is stateless, and if the responder saves it, it is stateful. Stateless applications do not care about who the responder is and whether it is necessary to synchronize information among various responders. The response service can be deleted at any time without affecting others, with high fault tolerance. If the load balancing of the distributed service fails, data will not be lost, there is no memory consumption, and it can be directly deployed and put into use. Stateful applications need to synchronize data in a timely manner, and there may be incomplete data synchronization resulting in data loss and memory resource consumption for saving data, etc.

[0112] Therefore, in the embodiments of the present application, according to the application deployment requirements, for stateful applications, an application pool is provided for them to obtain their corresponding target IP addresses from the application pool; for stateless applications, the target address pool needs to be further determined according to the deployment requirements of the container running the application.

[0113] As an example, as Figure 8 shown, the network management module divides the IP address pool as follows: The total address pool is used for the expansion of multiple IP address pools. The multiple IP address pools divided from the total address pool include an application pool, a network space pool, and a default pool. Among them, it is specified that the application pool is either in the total address pool or in a certain network space pool, not in the default pool, and there is no situation involving cross-IP address pools.

[0114] In this embodiment, the network management module divides multiple IP address pools from the total address pool according to the preset address pool division strategy. Each IP address pool is isolated from each other, realizing the isolation of IP addresses in the network area. In addition, for stateful applications that want a fixed IP, the network management module can reserve the IP address for the application. When the application starts, it will obtain the reserved IP address from the application pool, and still obtain the original IP address after restarting. In this way, the network management module can effectively manage the IP addresses in the container cluster and also improve the acquisition efficiency of the target IP address.

[0115] Furthermore, based on the above-mentioned multiple divided address pools, in one embodiment, the implementation process of obtaining the target IP address from the target address pool in step 620 can be: If there is a specified IP address in the container configuration file, the IP address management unit determines the IP address in the target address pool that is the same as the specified IP address as the target IP address; if there is no specified IP address in the container configuration information, the IP address management unit determines the target IP address from the target address pool according to the IP address search strategy.

[0116] Among them, the IP address lookup policy includes: sorting according to the release time of the IP address. If the current state of the resource object is the restart state, the IP address with the latest release time in the target address pool is determined as the target IP address; if the current state of the resource object is the new state, the IP address with the earliest release time is determined as the target IP address.

[0117] As an example, the resource status database includes three address pools: the application pool, the network space pool, and the default pool. When the IP address management unit determines the target IP address in multiple IP address pools, it mainly includes two steps: screening and binding.

[0118] (1) The IP address screening needs to meet the following conditions:

[0119] The first condition: If the application reserves an IP address, there is a specified IP address in the container configuration file. Therefore, according to the specified IP address, the specified IP address can be selected in the application pool as the target IP address corresponding to the resource object.

[0120] The second condition: If the ipam.com / netspace field is specified under spec.template.metadata.annotations of the container configuration file Yaml file, the target IP address is determined from the network space pool; if the ipam.com / netspace field is not specified under spec.template.metadata.annotations of the Yaml file, the target IP address is determined from the default address pool.

[0121] Among them, Spec describes the required state of the resource object; metadata is used to describe the data of the resource object, and annotation is used to attach any non-identifying metadata to the object. Annotation can contain one or more groups of key / value.

[0122] (2) Binding the IP address needs to meet IP affinity:

[0123] For stateful applications (statefulset), the original IP address is obtained based on the occupancy information of the IP address; for stateless applications, screening is performed in the network space pool and the default pool according to the release time of the IP address to determine the target IP address.

[0124] In this embodiment, based on a plurality of pre-divided address pools and container configuration files, the IP address management unit can determine a target address pool from the plurality of IP address pools through a screening operation; further, the IP address management unit determines a target IP address from the target address pool through a binding operation and binds it to a resource object. In this way, by dividing the address pool, the efficiency of determining the IP address is greatly improved.

[0125] Based on any of the above container network management systems 100, in one embodiment, as Figure 9 shown, the computing nodes in the container network management system 100 further perform the following steps:

[0126] Step 910: Each computing node binds at least two physical network cards in its own node to obtain a logical network card; the IP address space of the logical network card is stored in the resource status database and is used to create a plurality of IP address pools.

[0127] Among them, multiple physical network cards can be deployed on one computing node, and the actual number of physical network cards deployed on the computing node in the embodiments of the present application is not limited.

[0128] In a possible implementation manner, the implementation process of step 910 can be: select two interfaces of the physical network card for primary and standby binding (bond), and after binding, obtain a total logical network card. Instead of using the physical network card, create a virtual local area network interface through the logical network card to deploy the network framework.

[0129] In addition, it should be noted that if the container cluster is a dual-stack cluster, the container network interface plugins deployed on each computing node support both the first Internet protocol and the second Internet protocol at the same time; the first Internet protocol and the second Internet protocol are different protocols, and the corresponding relationship between the first Internet protocol and the second Internet protocol is stored in the computing node.

[0130] As an example, the first Internet protocol can be Internet Protocol Version 4 (IPv4), and the second Internet protocol can be Internet Protocol Version 6 (IPv6). The corresponding relationship between IPv4 and IPv6 is pre-stored in the physical network card of the computing node.

[0131] In this way, the MACVlan plugin in the container network interface plugin can support both IPv4 and IPv6 in the IPAM plugin. After the MACVlan plugin obtains the IPv4 address and IPv6 address of the pod in the IPAM plugin, it will enable the IPv6 support mode in the netns. The pod will finally bind the IPv4 and IPv6 addresses to the eth0 network card, enabling the pod to be accessed via both IPv4 and IPv6 and supporting communication using both network protocols.

[0132] Step 920: Each computing node creates multiple virtual local area network interfaces on the logical network card; the multiple virtual local area network interfaces are used for network isolation of multiple resource objects on each computing node.

[0133] In a possible implementation, each computing node creates multiple virtual local area network interfaces on the logical network card by means of physical network card virtualization. Further, each computing node assigns the IP addresses of the multiple virtual local area network interfaces to the multiple resource objects on the computing node; among them, the resource objects connected to the same virtual local area network interface communicate through the virtual local area network interface.

[0134] Among them, MACVlan is a virtualization technology on the physical network card. In step 920, by creating multiple MAC layer vlan interfaces on the logical network card, two-layer network isolation is achieved, and at the same time, it can effectively limit the network storm problem caused by creating a large two-layer due to the excessive cluster scale.

[0135] That is to say, the parent interface of the eth0 virtual network card of each resource object is the vlan sub-interface of the logical network card on each computing node. For resource objects in the same vlan, their two-layer networks are interconnected and can communicate directly without passing through the host. Resource objects in different vlans need to enable network policies to communicate with each other.

[0136] As an example, Figure 10 is a schematic diagram of a MACVlan networking solution provided by this application. Refer to Figure 10 , the resource object is a pod. Each of computing node 1 and computing node 2 has 2 physical network cards, and the interfaces are respectively ens1f0 and ens1f1. The 2 physical network cards are bound to form a bond virtual network card. Two virtual local area network interfaces bond0.910 and bond0.920 are created on the bond virtual network card using MACVlan. The virtual local area network interface serves as the parent interface and can be connected to the virtual network card eth0 of different pods.

[0137] Among them, MACVlan selects the bridge mode. In this mode, the MACVlan sub-interface cannot communicate directly with the host, but the sub-interfaces can communicate directly with each other. Utilizing this feature, create a MACVlan sub-interface on the logical network card of the host, called VMAC. At the same time, assign the IP address on the Vlan sub-interface of the virtual network card eth0 of the pod to VMAC.

[0138] Optionally, MACVlan also includes an IPv4 website and the corresponding IPv6 website, as well as routing information.

[0139] During actual communication, pods in the same local area network can communicate directly without passing through the host (i.e., the computing node where the pod is located). For example, Figure 10 for pods pod1, pod2, and pod4 in, their connections are all virtual local area network interfaces bond0.910, belonging to MACVlan1. Therefore, the communication between pod1, pod2, and pod4 can be forwarded through MACVlan2 acting as a gateway. Similarly, the connections of pod3 and pod5 are both virtual local area network interfaces bond0.920, belonging to MACVlan2. Therefore, the communication between pod3 and pod5 can be forwarded through MACVlan2 acting as a gateway. The communication between pods that do not belong to the same MACVlan must be forwarded through the physical network card of the host.

[0140] As an example, set the destination address to service_cidr and the gateway to the IP address corresponding to VMAC in the pod. Then, when accessing the service in the pod, traffic is forwarded through VMAC, and the traffic in the pod is forwarded to the host. Then, through the IPtables rule / IPset rule, a connection is established with the endpoints. The endpoints provide the connection between the service and the pod, and access the pod through the endpoint.

[0141] In this embodiment, for the layer-2 MAC network in the container network interface plugin, by introducing the Vlan technology, the MACVlan container network is securely isolated. At the same time, a one-to-one mapping relationship between IPv4 and IPv6 is made for the IP resources, realizing the support for the IPv4 and IPv6 dual-stack network, and maximizing the compatibility with the IPv4 single-stack container network.

[0142] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0143] Those of ordinary skill in the art can understand that to implement all or part of the processes in the above system, it can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above embodiments. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0144] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0145] The above-described embodiments only represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several deformations and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of the patent of this application should be subject to the appended claims.

Claims

1. A container network management system, characterized in that, The system includes: a network management module and at least one container cluster. The container cluster includes multiple computing nodes, and a container network interface plugin and resource objects are deployed on each of the computing nodes; The network management module divides multiple IP address pools according to the IP addresses of the computing nodes; the multiple IP address pools are isolated from each other; The container cluster further includes a master node, and the master node includes an IP address management unit; each computing node includes a node management unit, and the container network interface plugin includes an IP address management plugin; each computing node determines a target IP address corresponding to the resource object from the multiple IP address pools by calling the container network interface plugin according to the container configuration file of the resource object, including: The node management unit calls the IP address management plugin and sends an IP address request to the IP address management unit; the container configuration file of the resource object is carried in the IP address request; The IP address management unit determines the target IP address from the multiple IP address pools according to the container configuration file of the resource object and returns the target IP address to the node management unit.

2. The system according to claim 1, wherein The container cluster further includes a resource status database, and the multiple IP address pools are located in the resource status database; The IP address management unit determines the target IP address from the multiple IP address pools according to the container configuration file of the resource object, including: The IP address management unit determines the IP address planning information of the resource object according to the container configuration file of the resource object; The IP address management unit determines a target address pool from the resource status database according to the IP address planning information and obtains the target IP address from the target address pool.

3. The system according to claim 2, characterized in that, The obtaining the target IP address from the target address pool includes: If a specified IP address exists in the container configuration file, the IP address management unit determines the IP address in the target address pool that is the same as the specified IP address as the target IP address; If a specified IP address does not exist in the container configuration file, the IP address management unit determines the target IP address from the target address pool according to an IP address search strategy.

4. The system according to claim 3, characterized in that, The IP address search strategy includes: sorting according to the release time of the IP address. If the current state of the resource object is a restart state, the IP address with the latest release time in the target address pool is determined as the target IP address; if the current state of the resource object is a new state, the IP address with the earliest release time is determined as the target IP address.

5. The system according to claim 2, wherein The system further includes: Each computing node binds at least two physical network cards in its own node to obtain a logical network card; the IP address space of the logical network card is stored in the resource status database and is used to create the multiple IP address pools; Each computing node creates multiple virtual local area network interfaces on the logical network card; the multiple virtual local area network interfaces are used to isolate the multiple resource objects on each computing node.

6. The system according to claim 5, wherein Each of the computing nodes creates a plurality of virtual local area network interfaces on the logical network card, including: Each of the computing nodes creates the plurality of virtual local area network interfaces on the logical network card by means of physical network card virtualization.

7. The system according to claim 5, characterized in that, The system further includes: Each of the computing nodes assigns the IP addresses of the plurality of virtual local area network interfaces to a plurality of the resource objects on the computing node; wherein, the resource objects connected to the same virtual local area network interface communicate through the virtual local area network interface.

8. The system according to claim 2, wherein The network management module divides a plurality of IP address pools according to the IP addresses of each of the computing nodes, including: The network management module reads the IP address space of the logical network card of each of the computing nodes from the resource status database, and uses the IP address spaces of the logical network cards of the plurality of computing nodes in the container cluster as the total address pool; The network management module divides a plurality of IP address pools from the total address pool according to a preset address pool division strategy; wherein, the total address pool is used for expanding the plurality of IP address pools.

9. The system according to claim 8, characterized in that, The address pool division strategy includes application deployment requirements and container deployment requirements, and the plurality of IP address pools include an application pool, a network space pool, and a default pool; The network management module divides a plurality of IP address pools from the total address pool according to a preset address pool division strategy, including: The network management module divides an application address pool from the total address pool according to the application deployment requirements; The network management module divides the network space pool and the default pool from the total address pool according to the container deployment requirements; wherein, the application pool, the network space pool, and the default pool are isolated from each other; the IP addresses in the application address pool are reserved addresses for the target application, the IP addresses in the network space pool are candidate addresses for the target container, and the IP addresses in the default pool are candidate addresses for non-target containers.

10. The system according to any one of claims 1-4, characterized in that, The system further includes: If the container cluster is a dual-stack cluster, the container network interface plug-ins deployed on each of the computing nodes support both a first Internet protocol and a second Internet protocol at the same time; wherein, the first Internet protocol and the second Internet protocol are different protocols, and the computing node stores the corresponding relationship between the first Internet protocol and the second Internet protocol.

Images