DPU-based cloud native multi-CNI unified IPAM management method

Abstract

The invention relates to the technical field of multi-CNI management, in particular to a cloud native multi-CNI unified IPAM management method based on a DPU, the method centrally manages IP distribution of all CNIs through an IPAM module of OVN-K8s, and the method comprises the following steps: S1, creating corresponding subnet information for each CNI in the OVN-K8s; s2, the IPAM module of each CNI is configured to be an IPAM module using an OVN-K8s module; s3, Annotation is configured for the Pod, so that default network information and second network information of the Pod point to the corresponding CNI; and S4, inquiring the IP address information of the Pod internal network interface. The method has the advantages that the unified IPAM is used for managing the IP information in the environment needing multiple CNI networks, cluster resources can be optimized, the cluster operation and maintenance cost can be reduced, and the Pod utilization rate of a user is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-CNI management, and specifically to a unified IPAM management method for cloud-native multi-CNIs based on DPU. Background Art

[0002] DPU is the abbreviation of "Data Processing Unit", that is, a data processing unit or a dedicated data processor. It is a new generation of computing chip centered on data and I / O-intensive, and is one of the important computing power chips in the data center scenario. It adopts a software-defined technical route to support the virtualization of the infrastructure resource layer. The DPU has the functions of improving the efficiency of the computing system, reducing the total cost of ownership of the overall system, and improving the data processing efficiency and reducing the performance loss of other computing chips, and can provide a corresponding computing engine for high-bandwidth, low-latency, and data-intensive computing scenarios.

[0003] Container cloud is a cloud service product implemented by deploying container services on cluster servers through Docker technology. It belongs to the cloud delivery mode of the PaaS layer. Container cloud can be deployed in two ways: one is to deploy containers on virtual machines (in many traditional enterprises, containers are usually deployed on virtual machines); the other is to directly deploy containers on bare-metal servers. The reason for deploying containers to virtual machines is that they require the isolation and security features provided by virtual machines. However, in fact, to achieve the same effect, it is not necessarily necessary to use virtual machines, or with the development of technology, these problems may be solved in containers.

[0004] Kubernetes is a portable, extensible, open-source system for automatically deploying, scaling, and managing containerized applications, used to manage containerized workloads and services, and to promote declarative configuration and automation. Kubernetes has a large and rapidly growing ecosystem, and the scope of use of its services, support, and tools is quite extensive. The name Kubernetes comes from the Greek language, meaning "helmsman" or "pilot". Kubernetes is also known as K8s, which is an abbreviation of Kubernetes. This abbreviation is because there are eight characters between k and s. Google open-sourced the Kubernetes project in 2014. Kubernetes is built on Google's experience of running production workloads at scale for more than a decade, and combines the best ideas and practices in the community. It integrates the containers that make up the application into logical units, which is convenient for management and service discovery.

[0005] A Pod is the smallest basic unit for deployment and management in a Kubernetes cluster, enabling collaborative addressing and scheduling. A Pod actually groups one or more containers together to form an abstract collection of one or a group of services (processes). Inside a Pod, the network and storage can be shared. However, it is only logically similar to a virtual machine, but not a real virtual machine.

[0006] In an existing Kubernetes cluster, when a Pod requires multiple CNI networks, each CNI has its own IP manager (IPAM), which makes the Pod network configuration complex, difficult to maintain, and wastes the resources of the Kubernetes cluster.

[0007] Linux containers provide a lightweight virtualization method that can run multiple virtual environments, namely containers, on a single host simultaneously. Different from technologies such as Xen or KVM, in these technologies, the processor simulates the entire hardware environment and the hypervisor controls the virtual machines. While containers achieve virtualization at the operating system level, at this level, the kernel is responsible for controlling the isolated containers.

[0008] CNI, namely Container Network Interface, is a project under the CNCF, consisting of a series of specifications, libraries, and some plugins for configuring Linux container network interfaces. CNI focuses on the network allocation work when a container is created and the corresponding network resource release work when the container is deleted.

[0009] OVN (Open Virtual Network) is an open-source project that provides software-defined network (SDN) capabilities, especially for use in conjunction with Open vSwitch (OVS). OVN has powerful virtual network functions, including logical switching, logical routing, firewall rules, and load balancing. The distributed gateway is a key concept in OVN, aiming to provide efficient and reliable network connections, especially suitable for large-scale distributed environments.

[0010] The basic concepts related to the distributed gateway are as follows: Logical router: In OVN, the logical router is responsible for forwarding traffic between different logical switches. It is like a virtual router that can route data packets between different virtual networks.

[0011] Distributed Gateway: A distributed gateway disperses gateway functions across multiple computing nodes instead of concentrating them on a single node. This architecture helps improve network scalability and reliability while reducing network bottlenecks and the probability of single-point failures.

[0012] OVN - K8s is an integration solution of Open Virtual Network (OVN) and Kubernetes (K8S), aiming to provide a high-performance, flexible, and powerful network solution for Kubernetes. It builds a stable and scalable Kubernetes cluster network by leveraging the distributed virtual network capabilities of OVN.

[0013] In current multi-CNI network solutions, each set of CNI requires its own IPAM manager to allocate and manage IP information for itself. This results in waste of cluster resources and complex and cumbersome operation and maintenance configurations, increasing the operation and maintenance and resource costs for customers.

[0014] Therefore, there is an urgent need to design a unified IPAM management method for cloud-native multi-CNI based on DPU to implement the IP management function (IPAM) for the remaining CNI (such as Calico) through OVN - K8s, reduce the operation and maintenance of customers and the Kubernetes cluster resource costs, enable the Kubernetes cluster resources to be more fully utilized on customer business Pods, reduce waste of cluster resources and the complexity of operation and maintenance configurations, and reduce customer operation and maintenance and resource costs. Summary of the Invention

[0015] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a unified IPAM management method for cloud-native multi-CNI based on DPU to implement the IP management function (IPAM) for the remaining CNI (such as Calico) through OVN - K8s, reduce the operation and maintenance of customers and the Kubernetes cluster resource costs, enable the Kubernetes cluster resources to be more fully utilized on customer business Pods, reduce waste of cluster resources and the complexity of operation and maintenance configurations, and reduce customer operation and maintenance and resource costs.

[0016] To achieve the above object, a unified IPAM management method for cloud-native multi-CNI based on DPU is designed. The method centrally manages the IP allocation of all CNIs through the IPAM module of OVN-K8s. The method is as follows: S1. Create corresponding subnet information for each CNI in OVN-K8s; S2. Configure the IPAM module of each CNI to use the IPAM module of OVN-K8s; S3. Configure Annotations for the Pod so that its default network and second network information point to the corresponding CNI; S4. Query the IP address information of the internal network interface of the Pod.

[0017] Preferably, the present invention further includes: Step S2: Combine the IPAM module of Calico in CNI with the IPAM module of OVN-K8s so that OVN-K8s can manage and allocate the subnet information of each CNI.

[0018] Preferably, the present invention further includes: Step S3: The IPAM module of OVN-K8s allocates IP addresses according to the subnet information corresponding to each CNI and writes them into the Annotations of the Pod. When the Pod constructs the network, it will read the configuration information of the corresponding CNI according to the Annotations information. Preferably, the present invention further includes: The method is specifically as follows: S1. Environment preparation, configure the Kubernetes cluster; install supported CNI plugins, including OVN-Kubernetes and Calico; install Multus CNI to manage multi-network configurations; S2. Configure multiple CNI plugins, create Network Attachment Definition for different CNI plugins, and these definitions will specify different networks, including: S21. Configure the Calico network, use the IPAM module of OVN-K8s to centrally manage and allocate IP addresses; S22. Configure the OVN-K8s network, use OVN-K8s to create corresponding subnet resources for the Calico CNI network; S3. Create Pods, and the Pods use different CNIs, including: S31. Use Calico as the default CNI and declare that the Pod uses Calico as the default network; S32. Use OVN-K8s as the default CNI and Calico as the second CNI.

[0019] Compared with the prior art, the advantages of the present invention are as follows: Based on K8S with DPU as a node, in an environment that requires multi-CNI networks, the present invention uses unified IPAM to manage IP information, which can optimize cluster resources, reduce cluster operation and maintenance costs, and improve the utilization rate of user Pods. Brief Description of the Drawings

[0020] Figure 1 is a schematic diagram of the existing multi-CNI and multi-IPAM management process; Figure 2 is a schematic diagram of the unified IPAM management process for multiple CNIs of the present invention. Specific implementation manners

[0021] To make the purpose, principle and structure of the present invention clearer, the following further elaborates with reference to the accompanying drawings and specific embodiments.

[0022] OVN-K8s has the following main features and working mechanisms: Logical network management: Supports virtual routers, virtual switches, load balancers, etc. Distributed network functions: Include NAT, ACL, firewalls, etc. High-performance data plane: Rely on OVS to achieve high-performance packet forwarding.

[0023] The architecture of OVN is divided into two layers: Southbound database: This layer stores the status information related to the underlying network devices.

[0024] Northbound database: This layer defines the desired status of the user's logical network.

[0025] See Figure 1, when deploying a multi-CNI environment in the Kubernetes cluster of the current DPU, each CNI has its own IPAM management. In the figure, OVN Subnet represents the subnet resources provided by OVN-K8s for allocating IP addresses to Pods. The OVN-K8s ipamcontroller is a manager responsible for managing the IP address allocation of OVN-K8s. Pod Annotations represent Pod annotations, which are fields used to store Pod configuration information in Kubernetes. Write Network Config means writing network configuration, and the OVN-K8s ipam controller writes network configuration information into Pod Annotations. OVN-K8s CNI represents the OVN-Kubernetes CNI plugin, which is responsible for creating and managing OVN network interfaces for Pods. Read OVN Config represents the operation of reading OVN configuration. The OVN-K8s CNI reads the OVN network configuration from Pod Annotations to create network interfaces and configure the network for Pods. The host-local ipam controller is an IPAM manager responsible for managing the IP address allocation of the local host. Calico CNI is the CNI plugin of Calico, which is responsible for creating and managing Calico network interfaces for Pods. Read IP / route Config represents the operation of reading IP / route configuration. The Calico CNI reads the Calico network configuration from Pod Annotations, including IP addresses and routing information, to configure the Calico network for Pods. A Pod is the basic deployment and management unit in Kubernetes, which contains containers and related configuration information. In the figure, the Pod creates two network interfaces, rth0 and net1, through the two CNI plugins, OVN-K8s and Calico, respectively, thus realizing multi-network configuration. The Pod can access the OVN-Kubernetes and Calico networks through these two network interfaces respectively to achieve multi-network communication.

[0026] The present invention provides a unified IPAM management method for cloud-native multi-CNIs based on DPU, including: 1) First, create subnet Subnet information of the network segment corresponding to the CNI (such as Calico) in OVN-K8s.

[0027] 2) Set the IPAM module of the corresponding CNI (such as Calico CNI) to the corresponding OVN-K8s program.

[0028] 3) Set the default network and the second network information of the Pod to the corresponding CNI, such as Calico CNI.

[0029] 4) Check the IP information of the network cards inside the Pod.

[0030] Specifically, refer to Figure 2 .

[0031] Step S1. Create the corresponding subnet information for each CNI in OVN-K8s, and configure the subnet segment information of the Subnet of a CNI (such as Calico CNI) in OVN-K8s. Specifically, Step S1 includes: environment preparation, configuring the Kubernetes cluster; installing the supported CNI plugins, including OVN-Kubernetes and Calico; installing Multus CNI to manage the multi-network configuration.

[0032] Step S2. Configure the IPAM module of each CNI to use the IPAM module of OVN-K8s, and set the k8s.v1.cni.cncf.io / networks information when creating a Pod. Specifically, Step S2 includes: configuring multiple CNI plugins, creating network connection definitions Network Attachment Definition for different CNI plugins, and these definitions will specify different networks, including: S21. Configure the Calico network to uniformly manage and allocate IP addresses using the IPAM module of OVN-K8s. S22. Configure the OVN-K8S network to create the corresponding subnet resources for the calico CNI network using OVN-K8s.

[0033] Step S3. Configure the Annotations for the Pod so that its default network and the second network information point to the corresponding CNI. The OVN-K8s ipam manager will allocate IP information according to the Subnet network of the corresponding CNI and write it into the Annotations. Specifically, Step S3 includes: creating a Pod, and the Pod uses different CNIs, including: S31. Use Calico as the default CNI and declare that the Pod uses Calico as the default network. S32. Use OVN-K8s as the default CNI and Calico as the second CNI.

[0034] Step S4. Query the IP address information of the internal network interface of the Pod. When the Pod constructs the network, it will read the configuration information of the corresponding CNI according to the Annotations information.

[0035] Example 1: 1. For the environment preparation in Step S1, specifically as follows: Configure the Kubernetes cluster: Ensure that the Kubernetes cluster, the underlying platform for deploying and managing containerized applications, is running properly.

[0036] Install network plugins: Install CNI (Container Network Interface) plugins that support multiple network configurations, such as OVN-K8s, Calico, etc. These plugins are responsible for providing network connections for containers.

[0037] Install Multus CNI: As the core component in Kubernetes for managing multiple network configurations, it is used to manage multiple network configurations.

[0038] 2. Install Multus CNI in step S1 as follows: Multus is the core component for multi-network support and is used to configure multiple CNIs.

[0039] Specific method for installing Multus: Run the kubectl apply –f installation command to load and install the configuration files of Multus CNI. These files define the behavior and configuration of Multus to enable it to work properly in the cluster. Verify whether Multus is installed successfully: kubectl get pods -n kube-system | grep multus.

[0040] 3. Configure multiple CNI plugins in step S2 as follows: Create network connection definitions, Network Attachment Definition, for different CNI plugins, and these definitions will specify different networks.

[0041] 1) Configure the Calico network in step S21 as follows.

[0042] cat<<EOF | kubectl apply -f – apiVersion: "k8s.cni.cncf.io / v1" kind: NetworkAttachmentDefinition metadata: name: calico namespace: default spec: config: '{ "cniVersion": "0.3.0", "type": "calico", "master": "enp1s0", "mode": "bridge", "ipam": { "type": "ovn-k8s-cni-ipam", # This sentence means to set the ovn-kubernetes IPAM manager here to uniformly manage and allocate IPs for the calico CNI "provider": "calico.default" # Set the same as the spec.provider corresponding to the subnet } }' EOF.

[0043] 2) The configuration of the OVN-K8S network in step S22 is as follows: Use ovn-kubernetes to create corresponding subnet resources for the calico CNI network.

[0044] cat<<EOF | kubectl apply -f – apiVersion: ovn.yusur.tech / v1 kind: Subnet metadata: name: calico spec: cidrBlock: 192.168.122.0 / 24 default: false enableDHCP: false enableEcmp: false enableIPv6RA: false excludeIps: - 192.168.122.1 - 192.168.122.10..192.168.122.150 gateway: 192.168.122.1 natOutgoing: false protocol: IPv4 provider: calico.default # Use the same provider as the one corresponding to the CNI setting vpc: default # Default value is default EOF.

[0045] 4. In the said step S3, the Pod uses different CNIs, specifically as follows.

[0046] 1) Calico as the default CNI: cat<<EOF | kubectl apply -f – apiVersion: v1 kind: Pod metadata: annotations: v1.multus-cni.io / default-network: default / calico # This sentence indicates setting Calico as the default CNI labels: app: nginx5-ovn name: nginx5-ovn namespace: default spec: containers: - command: - / bin / sh - -c - nginx -g "daemon off;" image: harbor.yusur.tech / leid / ubuntu:22.04 imagePullPolicy: IfNotPresent name: nginx5-ovn EOF.

[0047] 2) Calico as the second CNI cat<<EOF | kubectl apply -f – apiVersion: v1 kind: Pod metadata: annotations: k8s.v1.cni.cncf.io / networks: default / calico # This sentence indicates setting Calico CNI as the secondary CNI v1.multus-cni.io / default-network: kube-system / ovn-network # This sentence indicates setting the OVN-K8s CNI network as the default CNI labels: app: nginx7-ovn name: nginx7-ovn namespace: default spec: containers: - command: - / bin / sh - -c - nginx -g "daemon off;" image: harbor.yusur.tech / leid / ubuntu:22.04 imagePullPolicy: IfNotPresent name: nginx7-ovn EOF.

[0048] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and innovative concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. A unified IPAM management method for cloud native multi-CNI based on DPU, characterized in that: The method centrally manages the IP allocation of all CNIs through the IPAM module of OVN-K8s. The method is as follows: S1. Create corresponding subnet information for each CNI in OVN-K8s; S2. Configure each CNI’s IPAM module to use OVN-K8s’ IPAM module. S3. Configure Annotations for the Pod so that its default network and second network information point to the corresponding CNI; S4. Query the IP address information of the Pod's internal network interface.

2. A unified IPAM management method for cloud native multi-CNI based on DPU as claimed in claim 1, characterized in that: Step S2: Combine the IPAM module of Calico in the CNI with the IPAM module of OVN-K8s, so that OVN-K8s can manage and allocate the network segment Subnet information of each CNI.

3. A unified IPAM management method for cloud native multi-CNI based on DPU as claimed in claim 1, characterized in that: Step S3: The IPAM module of OVN-K8s allocates IP addresses according to the subnet information corresponding to each CNI, and writes it into the Annotations of the Pod. When the Pod sets up the network, it will read the configuration information of the corresponding CNI based on the Annotations information.

4. A unified IPAM management method for cloud native multi-CNI based on DPU as claimed in claim 1, characterized in that: The method is specifically as follows: S1. Prepare the environment and configure the Kubernetes cluster; install supported CNI plugins, including OVN-Kubernetes and Calico; install Multus CNI to manage multiple network configurations; S2. Configure multiple CNI plugins and create Network Attachment Definitions for different CNI plugins. These definitions will specify different networks, including: S21. Configure Calico network and use OVN-K8s IPAM module to uniformly manage and allocate IP addresses; S22. Configure the OVN-K8s network and use OVN-K8s to create corresponding subnet resources for the Calico CNI network; S3. Create Pods that use different CNIs, including: S31. Set Calico as the default CNI and declare that the Pod uses Calico as the default network; S32. Set OVN-K8s as the default CNI and Calico as the second CNI.

Images