isolated data interface of a container deployed to a compute node

Abstract

This application relates to isolated data interfaces for containers deployed to compute nodes. Generally, techniques are described for deploying a logically related group of one or more containers ("jars") that support the Data Plane Development Kit (DPDK) to support fast path packet communication over a data channel between a virtual router and a jar. In one example, a computing device includes a virtual router that includes processing circuitry and is configured to implement a virtual network in a computing infrastructure that includes the computing device to enable communication between virtual network endpoints connected via the virtual network. The computing device includes a jar that contains a containerized application, where the virtual router and the jar are configured to create a Unix domain socket using a file system resource that is accessible by the jar and by the virtual router and is not accessible by any other jar deployed to the computing device.

Description

[0001] This application claims priority to U.S. Patent Application No. 17 / 652,250, filed February 23, 2022, U.S. Provisional Patent Application No. 63 / 242,434, filed September 9, 2021, U.S. Provisional Patent Application No. 202141008464, filed March 1, 2021, and Indian Provisional Patent Application No. 202141008548, filed March 1, 2021; the entire contents of each application are incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to virtualized computing infrastructures, and more specifically, to deploying containers to virtualized computing infrastructures. BACKGROUND

[0003] In a typical cloud data center environment, there are a large number of interconnected servers that provide compute and / or storage capacity to run various applications. For example, a data center can include a facility that hosts applications and services for subscribing customers (i.e., customers of the data center). For example, a data center can host all of the infrastructure equipment, such as networking and storage systems, redundant power supplies, and environmental controls. In a typical data center, clusters of storage systems and application servers are interconnected through a high-speed switching fabric provided by one or more layers of physical network switches and routers. More complex data centers provide infrastructure that spans the globe, as well as subscribing customer support equipment located in various physical hosting facilities.

[0004] Virtualized data centers are becoming the core foundation of modern information technology (IT) infrastructures. In particular, modern data centers have widely utilized virtualized environments in which virtual hosts, also referred to herein as virtual execution elements, such as virtual machines or containers, are deployed and executed on an underlying computing platform of physical computing devices.

[0005] Virtualization of data centers can provide several advantages. One advantage is that virtualization can significantly improve efficiency. With the advent of multi-core microprocessor architectures in which each physical CPU has a large number of cores, the underlying physical computing devices (i.e., servers) are becoming more and more powerful, making virtualization easier and more efficient. A second advantage is that virtualization provides important controls over the computing infrastructure. With physical computing resources becoming replaceable resources, such as in cloud-based computing environments, the provisioning and management of computing infrastructure becomes easier. Thus, in addition to the efficiency and increased return on investment (ROI) that virtualization provides, enterprise IT personnel generally prefer virtualized computing clusters in data centers because of the management advantages they have.

[0006] Containerization is a virtualization scheme based on operating system-level virtualization. A container is a lightweight, portable execution element suitable for applications that are isolated from each other and from the host. Since containers are not tightly coupled with the host hardware computing environment, applications can be bound to container images and executed as a single lightweight package on any host or virtual host that supports the underlying container architecture. Thus, containers solve the problem of how to make software work in different computing environments. Containers provide a promise of consistent operation from one computing environment to another, virtual or physical.

[0007] Due to the lightweight nature of containers, a single host can typically support many more container instances than traditional virtual machines (VMs). Containers are typically short-lived, can be created and moved more efficiently than VMs, and they can also be managed as groups of logically related elements (sometimes referred to as "pods" for certain orchestration platforms, such as Kubernetes). These container characteristics impact the requirements for container networking solutions: the network should be agile and scalable. VMs, containers, and bare-metal servers can need to coexist in the same computing environment and enable communication between different application deployments. Container networking should also be indeterminate for the variety of orchestration platforms used to deploy containerized applications.

[0008] Computing infrastructure that manages deployments and the infrastructure used for application execution can involve two main roles: (1) orchestration - for automatically deploying, scaling, and operating applications across a cluster of hosts and providing the computing infrastructure, which can include container-centric computing infrastructure; and (2) network management - for creating virtual networks in the network infrastructure to enable packet communication between applications running on virtual execution environments (e.g., containers or VMs) and between applications running in legacy (e.g., physical) environments. Software-defined networking facilitates network management. SUMMARY

[0009] Generally described, techniques are described for deploying a logically related group of one or more containers ("pods") that support a data plane-centric software framework, e.g., containers implemented with native applications using a Data Plane Development Kit (DPDK) (referred to herein as "DPDK containers" or "DPDK applications") to enable packet communication on a data channel between virtual routers and pods. In some examples, the virtual routers are DPDK-based virtual routers such that packets originating from or destined for the containers can be exchanged between the containers and the virtual routers in user space, bypassing the kernel of the computing device hosting the pods and the virtual routers.

[0010] A container network interface module or plugin (CNI) is an application container networking solution that is a runtime executable program used by a container orchestration system to configure a network interface into a container network namespace and configure a computing device ("host") that hosts a container, which can be a member of a pod. The computing device can also be referred to as a "compute node" or "server." The CNI also assigns a network address (e.g., IP address) to the network interface and can also add routes related to the interface, e.g., routes for a default gateway and one or more name servers.

[0011] As described herein, the control flow for deploying pods and configuring pods and virtual routers inside a compute node can include creating high-speed data paths between the pods and the virtual routers in a manner that isolates the pods from each other. For example, a DPDK container can use a vhost / virtio library to create an Ethernet interface for communicating with the host or more specifically the virtual router. If the host provides a common file system resource for communications of multiple pods that communicate with the virtual router, then any pod can see and access data exchanged between other pods and the virtual router. This data can include packets, configuration files, or other sensitive information. To facilitate isolation, the host provides each pod access to a separate dedicated file system resource for communications between the virtual router and the pod. That is, each pod can use its assigned file system resource to communicate with the virtual router but does not have visibility or other access to similar file system resources assigned to other pods for this purpose. Using its assigned file system resource, each pod can exchange control protocol messages with the virtual router to establish a high-speed data path between the virtual router and the pod for high-throughput packet forwarding and processing. In some examples, the assigned file system resource is a socket file or directory that is a Unix domain socket for control channel operations used to create and manage the high-speed data path. The high-speed data path can be an accelerated data path using DPDK.

[0012] The techniques can provide one or more technical advantages. For example, the techniques described herein can facilitate isolation between pods executing on the same compute node. Existing schemes for creating respective control channels between multiple pods and virtual routers of a compute node include providing the same socket directory path (e.g., / var / run / vrouter) to all pods as a folder to store socket files, and then each container installs the same socket directory path to access the socket files for binding interfaces in user space to create their control channels with the virtual router for creating high-speed data paths. As a result, each pod can have visibility into the control channels of each other pod. In contrast, the described techniques result in each pod having different file system resources to support the control channels and prevent the pods from accessing the control channels of other pods, which would otherwise allow the pods to access configuration files or other control or management information of other pods. As another example, in the example where the CNI obtains the pod identifier from the virtual router, the techniques can reduce backlog on the orchestration system by avoiding some communications from the CNI to the orchestration system, which can be caused by a large number of workloads being concurrently deployed to multiple different compute nodes.

[0013] In one example, a computing device includes a virtual router including processing circuitry and configured to implement a virtual network in a computing infrastructure including the computing device to enable communication between virtual network endpoints connected via the virtual network. The computing device includes a pod including a containerized application, wherein the virtual router and the pod are configured to create a Unix domain socket using a file system resource that is accessible by the pod and the virtual router and is not accessible by any other pod deployed to the computing device.

[0014] In one example, a method includes obtaining, by an orchestrator, a pod specification for a pod, wherein the pod specification includes data for generating socket description data that indicates a file system resource that the pod uses to create a Unix domain socket when the pod is deployed to a computing device, the file system resource being accessible by the pod and a virtual router and not accessible by any other pod deployed to the computing device; and outputting, by the orchestrator, the pod specification to an orchestration agent executing on the computing device, wherein the orchestration agent is an agent of the orchestrator.

[0015] In one example, a method includes: creating, by a virtual router and a pod executing on a computing device, a Unix domain socket using a socket directory, the socket directory being accessible by the pod and the virtual router and not accessible by any other pod deployed to the computing device; implementing, by the virtual router, a virtual network in a computing infrastructure including the computing device to enable communication between virtual network endpoints connected via the virtual network; and exchanging, by the virtual router and the pod, control communications for a data plane development kit (DPDK) interface between the virtual router and the pod using the Unix domain socket.

[0016] The details of one or more examples of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a block diagram of an example computing infrastructure that can implement the example techniques described herein;

[0018] Figure 2 is a block diagram of an example computing device that facilitates data channel isolation between pods in accordance with the techniques described in this disclosure;

[0019] Figure 3 is a block diagram of an example computing device that is an example operation of a controller and orchestrator of a virtualized computing infrastructure in accordance with the techniques described in this disclosure;

[0020] Figure 4 is a flow diagram illustrating one example workflow for creating a data interface between a pod and a virtual router in accordance with some aspects described in this disclosure;

[0021] Figure 5 is a block diagram of an example computing device with a DPDK-based virtual router and containerized DPDK workloads with interfaces established in accordance with one or more aspects of the disclosure;

[0022] Figure 6 is a flow diagram of example operational modes of computing device components in accordance with the techniques described herein;

[0023] Figure 7 is a flow diagram illustrating example operational modes of an orchestrator in accordance with the techniques described herein.

[0024] Throughout the specification and drawings like reference numerals refer to like elements. DETAILED DESCRIPTION

[0025] Figure 1is a block diagram illustrating an example computing infrastructure 8 in which examples of the technology described herein can be implemented. Generally, a data center 10 provides an operating environment for applications and services for customer sites 11 (shown as "Customers 11") that are coupled to the data center through a service provider network 7. The data center 10 can, for example, host infrastructure equipment such as networking and storage systems, redundant power, and environmental controls. The service provider network 7 is coupled to a public network 15, which can represent one or more networks managed by other providers and thus can form part of a large-scale public network infrastructure such as the Internet. The public network 15 can represent, for example, a local area network (LAN), a wide area network (WAN), the Internet, a virtual LAN (VLAN), an enterprise LAN, a Layer 3 virtual private network (VPN), an Internet Protocol (IP) intranet operated by the service provider operating the service provider network 7, an enterprise IP network, or some combination thereof.

[0026] While the customer sites 11 and the public network 15 are primarily shown and described as edge networks of the service provider network 7, in some examples one or more of the customer sites 11 and the public network 15 can be a tenant network within the data center 10 or another data center. For example, the data center 10 can host multiple tenants (customers), each associated with one or more virtual private networks (VPNs) that can implement one of the customer sites 11.

[0027] The service provider network 7 provides packet-based connectivity to additional customer sites 11, data centers 10, and public networks 15. The service provider network 7 can represent a network owned and operated by a service provider to interconnect multiple networks. The service provider network 7 can implement multi-protocol label switching (MPLS) forwarding and in this case can be referred to as an MPLS network or MPLS backbone. In some cases, the service provider network 7 represents a plurality of interconnected autonomous systems, such as the Internet, that provides services from one or more service providers.

[0028] In some examples, the data center 10 can represent one of many geographically distributed network data centers. As Figure 1As shown, data center 10 can be a facility that provides network services to customers. Customers of a service provider can be collective entities such as businesses, governments, etc., or can be individuals. For example, a network data center can host web services for multiple businesses and end users. Other example services can include data storage, virtual private networks, traffic engineering, file services, data mining, scientific or supercomputing, etc. Although shown as separate edge networks of service provider network 7, elements of data center 10 such as one or more physical network functions (PNFs) or virtualized network functions (VNFs) can be included within the core of service provider network 7.

[0029] In this example, data center 10 includes storage and / or computing servers (or “nodes”) interconnected via a switching fabric 14 provided by one or more layers of physical network switches and routers, with servers 12A-12X (herein “servers 12”) depicted as coupled to top-of-rack (TOR) switches 16A-16N. Servers 12 are computing devices and can also be referred to herein as “hosts” or “host devices.” Although only servers 12A coupled to TOR switch 16A are shown in detail in Figure 1 In this example, data center 10 includes storage and / or computing servers (or “nodes”) interconnected via a switching fabric 14 provided by one or more layers of physical network switches and routers, with servers 12A-12X (herein “servers 12”) depicted as coupled to top-of-rack (TOR) switches 16A-16N. Servers 12 are computing devices and can also be referred to herein as “hosts” or “host devices.” Although only servers 12A coupled to TOR switch 16A are shown in detail in

[0030] Switching fabric 14 in the illustrated example includes interconnected top-of-rack (TOR) (or other “leaf”) switches 16A-16N (collectively “TOR switches 16”) coupled to a distribution layer of chassis (or “spine” or “core”) switches 18A-18M (collectively “chassis switches 18”). Although not shown, data center 10 can also include, for example, one or more non-edge switches, routers, hubs, gateways, security devices such as firewalls, intrusion detection and / or intrusion prevention devices, servers, computer terminals, laptops, printers, databases, wireless mobile devices (such as cellular telephones or personal digital assistants), wireless access points, bridges, cable modems, application accelerators, or other network devices. Data center 10 can also include one or more physical network functions (PNFs) such as physical firewalls, load balancers, routers, route reflectors, broadband network gateways (BNGs), mobile core network elements, and other PNFs.

[0031] In this example, TOR switches 16 and chassis switches 18 provide servers 12 with redundant (multihomed) connectivity to IP fabric 20 and service provider network 7. Chassis switches 18 aggregate communication flows and provide connectivity between TOR switches 16. Switches 16 can be network devices that provide Layer 2 (MAC) and / or Layer 3 (e.g., IP) routing and / or switching functionality. TOR switches 16 and chassis switches 18 can each include one or more processors and memory and can execute one or more software processes. Chassis switches 18 are coupled to IP fabric 20, which can perform Layer 3 routing to route network traffic between data center 10 and customer sites 11 over service provider network 7. The switching architecture of data center 10 is merely an example. For example, other switching architectures can have more or fewer switching layers. IP fabric 20 can include one or more gateway routers.

[0032] The term "packet flow," "communication flow," or simply "flow" refers to a set of data packets originating from a particular source device or endpoint and destined for a particular destination device or endpoint. For example, a single packet flow can be identified by a 5-tuple: <source network address, destination network address, source port, destination port, protocol>. This five-tuple generally identifies a packet flow to which a received data packet corresponds. An n-tuple refers to any n- item drawn from a 5-tuple. For example, a two-tuple of data packets can refer to a combination of <source network address, destination network address> or <source network address, source port>.

[0033] Servers 12 can each represent a compute server or a storage server. For example, each of servers 12 can represent a compute device, such as an x86 processor-based server, configured to operate according to the techniques described herein. Servers 12 can provide a network function virtualization infrastructure (NFVI) for an NFV architecture.

[0034] Any of servers 12 can be configured with virtual execution elements by virtualizing resources of the server to provide isolation between one or more processes (applications) executing on the server. "Hypervisor-based" or "hardware-level" or "platform" virtualization refers to the creation of virtual machines, each of which includes a guest operating system for executing one or more processes. Generally, a virtual machine provides a virtualized / guest operating system for executing an application program in an isolated virtual environment. Since the virtual machine is virtualized from the physical hardware of the host server, the executing application is isolated from the hardware of the host and other virtual machines. Each virtual machine can be configured with one or more virtual network interfaces for communicating on a corresponding virtual network.

[0035] A virtual network is a logical construct implemented on top of a physical network. Virtual networks can be used to replace VLAN-based isolation and provide multi-tenancy in virtualized data centers (such as Data Center 10). Each tenant or application can have one or more virtual networks. Each virtual network can be isolated from all other virtual networks unless explicitly permitted by security policies.

[0036] Virtual networks can use data center 10 gateway routers ( Figure 1 (Not shown) Connects to and spans Physical Multiprotocol Label Switching (MPLS) Layer 3 Virtual Private Network (L3VPN) and Ethernet Virtual Private Network (EVPN) networks. Virtual networks can also be used to implement Network Functions Virtualization (NFV) and service chaining.

[0037] Virtual networks can be implemented using various mechanisms. For example, each virtual network can be implemented as a Virtual Local Area Network (VLAN), a Virtual Private Network (VPN), etc. Virtual networks can also be implemented using two networks—a physical underlying network consisting of IP structure 20 and switching structure 14, and a virtual overlay network. The role of the physical underlying network is to provide an "IP structure" that provides unicast IP connections from any physical device (server, storage device, router, or switch) to any other physical device. The underlying network can provide a unified, low-latency, non-blocking, high-bandwidth connection from any point in the network to any other point in the network.

[0038] As further described below regarding virtual router 21A, the virtual routers running on server 12 create a virtual overlay network over the physical underlying network using a dynamic network of "tunnels" between them. For example, these overlay tunnels could be MPLS over GRE / UDP tunnels, VXLAN tunnels, or NVGRE tunnels. The underlying physical routers and switches may not store any tenant state for virtual machines or other virtual execution elements, such as any media access control (MAC) addresses, IP addresses, or policies. The forwarding tables of the underlying physical routers and switches may, for example, contain only the IP prefix or MAC address of physical server 12. (An exception is the gateway router or switch that connects the virtual network to the physical network, which may contain tenant MAC or IP addresses.)

[0039] The virtual routers 21 of the servers 12 typically contain the state of each tenant. For example, they can contain a separate forwarding table (routing instance) for each virtual network. This forwarding table contains the IP prefixes (in case of layer 3 overlay) or MAC addresses (in case of layer 2 overlay) of the virtual machines or other virtual execution elements (e.g. the pods of containers). No single virtual router 21 needs to contain all IP prefixes or all MAC addresses of all virtual machines in the entire data center. A given virtual router 21 only needs to contain the routing instance of those that are locally present on the server 12 (i.e. at least one virtual execution element is present on the server 12).

[0040] The control plane protocol between the network controllers 24 or control plane nodes of the physical gateway routers (or switches) can be BGP (Netconf can also be used for management). This is the same control plane protocol that is also used for MPLS L3VPN and MPLS EVPN. The protocol between the network controllers 24 and the virtual routers 21 can be based on XMPP, for example.

[0041] “Container-based” or “operating system” virtualization refers to the virtualization of operating systems to run multiple isolated systems on a single machine (virtual or physical). Such isolated systems represent containers, such as the containers provided by the open source DOCKER container application or CoreOS Rkt (“Rocket”). Like virtual machines, each container is virtualized and can remain isolated from the host and other containers. However, unlike virtual machines, each container can omit a separate operating system and instead provide an application suite and application-specific libraries. Typically, containers are executed by a host as isolated user space instances and can share the operating system and common libraries with other containers executing on the host. As a result, containers can require less processing power, storage, and network resources than virtual machines. A group of one or more containers can be configured to share one or more virtual network interfaces to communicate on a corresponding virtual network.

[0042] In some examples, containers are managed by their host kernel to allow for the restriction and prioritization of resources (CPU, memory, block I / O, network, etc.) without the need to launch any virtual machines, in some cases using namespace isolation features that allow for a fully isolated view of the operating environment of an application (e.g., a given container), including process trees, networks, user identifiers, and mounted file systems. In some examples, containers can be deployed in accordance with Linux Containers (LXC), which is an operating system-level virtualization method for running multiple isolated Linux systems (containers) on a control host using a single Linux kernel.

[0043] The servers 12 host virtual network endpoints for one or more virtual networks that run on the physical network represented herein by the IP fabric 20 and the switching fabric 14. While a data center-based switching network is primarily described, other physical networks, such as the service provider network 7, can form the basis for one or more virtual networks.

[0044] Each of the servers 12 can host one or more virtual execution elements, each having at least one virtual network endpoint for one or more virtual networks configured in the physical network. A virtual network endpoint for a virtual network can represent one or more virtual execution elements that share a virtual network interface for the virtual network. For example, a virtual network endpoint can be a virtual machine, a pod of one or more containers (e.g., j avelins), or another other virtual execution element, such as a Layer 3 endpoint for the virtual network. The term “virtual execution element” includes virtual machines, containers, and other virtualized computing resources that provide an at least partially independent execution environment for an application. The term “virtual execution element” can also encompass a pod of one or more containers. As shown, the server 12A hosts one virtual network endpoint in the form of a pod 22A having one or more containers. However, a server 12 can execute as many virtual execution elements as the hardware resource limitations of a given server 12 allow. Each virtual network endpoint can use one or more virtual network interfaces to perform packet I / O or otherwise process packets. For example, a virtual network endpoint can use one virtual hardware component (e.g., an SR-IOV virtual function) enabled by the NIC 13A to perform packet I / O and receive / transmit packets on one or more communication links with the TOR switch 16A. Other examples of virtual network interfaces are described below. Figure 1

[0045] ​Each server 12 includes at least one network interface card (NIC) 13, each of which includes at least one interface to exchange packets with TOR switches 16 over a communication link. For example, server 12A includes NIC 13A. Any of the NICs 13 can provide one or more virtual hardware components 21 for virtualizing input / output (I / O). Virtual hardware components for I / O can be virtualizations of physical NICs (“physical functions”). For example, in Single Root I / O Virtualization (SR-IOV) as described in the Peripheral Component Interface Special Interest Group SR-IOV Specification, PCIe physical functions of a network interface card (or “network adapter”) are virtualized to present one or more virtual network interfaces as “virtual functions” for use by individual endpoints executing on server 12. In this way, virtual network endpoints can share the same PCIe physical hardware resources and virtual functions are an example of virtual hardware components 21. As another example, one or more servers 12 can implement Virtio, a readily available paravirtualization framework, e.g., for Linux operating systems, that provides emulated NIC functions as a kind of virtual hardware component to provide virtual network interfaces to virtual network endpoints. As another example, one or more servers 12 can implement Open vSwitch to perform distributed virtual multilayer switching among one or more virtual NICs (vNICs) for hosting virtual machines, where such vNICs can also represent a kind of virtual hardware component to provide virtual network interfaces to virtual network endpoints. In some cases, virtual hardware components are virtual I / O (e.g., NIC) components. In some cases, virtual hardware components are SR-IOV virtual functions. In some examples, any of servers 12 can implement a Linux bridge that emulates a hardware bridge and forwards packets between virtual network interfaces of the server or between virtual network interfaces of the server and physical network interfaces of the server. For Docker implementations of containers hosted by servers, Linux bridges or other operating system bridges, exchanging packets between containers executing on a server can be referred to as a “Docker bridge.” As used herein, the term “virtual router” can include a Contrail or Tungsten fabric virtual router, an Open vSwitch (OVS), an OVS bridge, a Linux bridge, a Docker bridge, or other device and / or software located on a host device and performing switching, bridging or routing of packets between virtual network endpoints of one or more virtual networks, where the virtual network endpoints are hosted by one or more servers 12.

[0046] Any of the NICs 13 can include an internal device switch to switch data between virtual hardware components associated with the NIC. For example, for an SR-IOV-capable NIC, the internal device switch can be a virtual Ethernet bridge (VEB) to switch between SR-IOV virtual functions and, correspondingly, between endpoints configured to use the SR-IOV virtual functions, where each endpoint can include one guest operating system. The internal device switch can also be referred to as a NIC switch, or for SR-IOV implementations, an SR-IOV NIC switch. The virtual hardware components associated with the NIC 13A can be associated with a Layer 2 destination address, which can be assigned by the NIC 13A or a software process responsible for configuring the NIC 13A. The physical hardware components (or “physical functions” for SR-IOV implementations) are also associated with a Layer 2 destination address.

[0047] The one or more servers 12 can each include a virtual router 21 that performs one or more routing instances for a corresponding virtual network within the data center 10 to provide a virtual network interface and route packets between virtual network endpoints. Each routing instance can be associated with a network forwarding table. Each routing instance can represent a virtual routing and forwarding instance (VRF) for an Internet Protocol - Virtual Private Network (IP-VPN). A data packet received by the virtual router 21A of the server 12A (shown as “DPDK vROUTER 21A”), for example, from an underlying physical network fabric of the data center 10 (i.e., the IP fabric 20 and the switch fabric 14) can include an outer header to allow the physical network fabric to tunnel a payload or “inner packet” to a physical network address of a network interface card 13A of the server 12A for performing the virtual router. The outer header can include not only the physical network address of the network interface card 13A of the server, but also a virtual network identifier, such as a VxLAN tag or a Multiprotocol Label Switching (MPLS) label that identifies one of the virtual networks and a corresponding routing instance performed by the virtual router 21A. The inner packet includes an inner header with a destination network address that conforms to a virtual network addressing space of the virtual network identified by the virtual network identifier.

[0048] The virtual router 21 terminates the virtual network overlay tunnels and determines the virtual network of a received data packet based on the tunnel encapsulation header of the data packet and forwards the data packet to the appropriate destination virtual network endpoint of the data packet. For example, for the server 12A, for each data packet that is outbound from a virtual network endpoint (e.g., pod 22A) hosted by the server 12A, the virtual router 21A appends a tunnel encapsulation header indicating the virtual network of the data packet to generate an encapsulated or “tunneled” data packet and the virtual router 21A outputs the encapsulated data packet to a physical destination computing device, e.g., another one of the servers 12, via an overlay tunnel for the virtual network. As used herein, the virtual router 21 can perform the operations of a tunnel endpoint to encapsulate internal data packets from a virtual network endpoint to generate tunneled data packets and decapsulate tunneled data packets to obtain internal data packets for routing to other virtual network endpoints.

[0049] The virtual router 21A is a data plane development kit (DPDK) enabled virtual router. That is, the virtual router 21A uses DPDK as the data plane. In this mode, the virtual router 21A runs as a user space application that links to DPDK libraries (not shown). This is a performance version of the virtual router, typically used by telecom companies, where the VNF is typically a DPDK based application. The performance of the virtual router 21A as a DPDK virtual router can achieve ten times higher throughput than a virtual router operating as a kernel based virtual router. The poll mode drivers (PMDs) of DPDK use the physical interface, rather than the interrupt based drivers of the Linux kernel.

[0050] A user I / O (UIO) kernel module (e.g., vfio or uio_pci_generic) can be used to expose the registers of the physical network interface in user space so that the DPDK PMD has access. When the NIC 13A is bound to the UIO driver, it moves from the Linux kernel space to the user space and is therefore no longer managed by the Linux operating system and is not visible. Thus, the DPDK application (i.e., the virtual router 21A in the present example) has full management of the NIC 13. This includes packet polling, packet processing, and packet forwarding. The user packet processing steps can be performed by the virtual router 21A DPDK data plane with limited or no involvement of the kernel (not shown in the kernel). Figure 1 The nature of this “poll mode” as compared to interrupt mode makes the virtual router 21A DPDK data plane packet processing / forwarding more efficient, especially when the packet rate is high. During packet I / O, there are limited or no interrupts and context switches.

[0051] Additional details of the DPDK vRouter example are found in “DAY ONE: CONTRAIL DPDK vRouter,” by Kiran KN et al. of Juniper Networks, Inc., 2021, which is incorporated by reference herein in its entirety.

[0052] The compute infrastructure 8 implements an automation platform for automating the deployment, scaling, and operation of virtual execution elements across the servers 12 to provide a virtualized infrastructure for executing application workloads and services. In some examples, the platform can be a container orchestration platform that provides a container-centric infrastructure for automating the deployment, scaling, and operation of containers to provide a container-centric infrastructure. In the context of virtualized compute infrastructure, “orchestration” generally refers to the provisioning, scheduling, and management of host servers available to the orchestration platform for virtual execution elements and / or applications and services executing on such virtual execution elements. In particular, container orchestration allows for the coordination of containers and refers to, for example, the deployment, management, scaling, and configuration of containers to host servers by a container orchestration platform. Examples of orchestration platforms include Kubernetes, Docker Swarm, Mesos / Marathon, OpenShift, OpenStack, VMware, and Amazon ECS.

[0053] The elements of the automation platform of the compute infrastructure 8 include at least the servers 12, the orchestrator 23, and the network controller 24. Containers can be deployed to the virtualized environment using a cluster-based framework in which a cluster’s cluster master node manages the deployment and operation of containers to one or more cluster slave nodes of the cluster. The terms “master node” and “slave node” as used herein encompass different orchestration platform terminology for similar devices, with the primary distinction being between a management element of the cluster and a container host device of the cluster. For example, the Kubernetes platform uses the terms “cluster master node” and “slave node,” while the Docker Swarm platform refers to a swarm manager and swarm nodes.

[0054] The orchestrator 23 and the network controller 24 can be executed on separate computing devices or can be executed on the same computing device. Each of the orchestrator 23 and the network controller 24 can be a distributed application executed on one or more computing devices. The orchestrator 23 and the network controller 24 can implement respective master nodes for one or more clusters, each cluster having one or more slave nodes (also referred to as “compute nodes”) implemented by respective servers 12.

[0055] Generally, the network controller 24 controls network configuration of the data center 10 fabric, e.g., establishes one or more virtual networks for packet communication between virtual network endpoints. The network controller 24 provides a logically and in some cases physically centralized controller to facilitate operation of one or more virtual networks within the data center 10. In some examples, the network controller 24 can operate in response to configuration input received from the orchestrator 23 and / or an administrator / operator. Additional information regarding the network controller 24 in connection with other apparatus of the data center 10 or other software-defined network operations see International Application No. PCT / US2013 / 044378 entitled "PHYSICAL PATH DETERMINATION FOR VIRTUAL NETWORK PACKET FLOWS" filed June 5, 2013, see U.S. Patent Application No. 14 / 226,509 entitled "Tunneled Packet Aggregation for Virtual Networks" filed March 26, 2014, each of which is incorporated by reference herein as if fully set forth herein.

[0056] Generally, the orchestrator 23 controls deployment, scaling, and operation of containers in the cluster of servers 12, and provides a computing infrastructure, which can include a container-centric computing infrastructure. The orchestrator 23, and in some cases the network controller 24, can implement a respective cluster master for one or more Kubernetes clusters. As one example, Kubernetes is a container management platform that provides portability across public and private clouds, which can both provide virtualized infrastructure for the container management platform. Reference is made below to Figure 3 Example components of a Kubernetes orchestration system are described.

[0057] In one example, the pod 22A is a Kubernetes pod and is an example of a virtual network endpoint. A pod is a group of one or more logically related containers (e.g., containers 20A, 20B, 20C) that share a common network namespace and are scheduled on the same host machine. In some examples, the pod 22A is a virtual network endpoint that is configured to communicate with other virtual network endpoints via one or more virtual networks. Figure 1The pod 22A is an instantiation of a set of containers (not shown), shared storage of the containers, and options on how to run the containers. In the case of instantiation for execution, the pod can also be referred to as a "pod replica." Each container of the pod 22A is an example of a virtual execution element. The containers of a pod are always co-scheduled and run in a shared context on a single server. The shared context of a pod can be a set of Linux namespaces, cgroups, and other isolation aspects. Within the context of a pod, a single application can apply further sub-isolation. Typically, containers in a pod have a common IP address and port space, and are able to detect each other through localhost. Since they have a shared context, containers within a pod also communicate with each other using inter-process communication (IPC). Examples of IPC include SystemV semaphores or POSIX shared memory. Typically, containers belonging to different pods have different IP addresses and are unable to communicate through IPC without a configuration that enables this functionality. Containers that are members of different pods typically communicate with each other through the pod IP address.

[0058] The server 12A includes a container platform 19A for running containerized applications, such as those of the pod 22A. The container platform 19A receives requests from the orchestrator 23 to obtain and host containers in the server 12A. The container platform 19A fetches and executes the containers.

[0059] The container platform 19A includes a container network interface (CNI) 17A that configures virtual network interfaces for virtual network endpoints. The orchestrator 23 and the container platform 19A use the CNI 17A to manage networks for pods, including the pod 22A. For example, the CNI 17A creates a virtual network interface to connect a pod to the virtual router 21A and enables a container of such pod to communicate with other virtual network endpoints over a virtual network via the virtual network interface. For example, the CNI 17A can insert a virtual network interface of a virtual network into a network namespace of a container in the pod 22A and configure (or request configuration of) a virtual network interface of the virtual network in the virtual router 21A such that the virtual router 21A is configured to send packets received from the virtual network via the virtual network interface to the container of the pod 22A and send packets received from the container of the pod 22A on the virtual network via the virtual network interface. The CNI 17A can assign a network address (e.g., a virtual IP address of the virtual network) and can establish a route for the virtual network interface. In Kubernetes, by default, all pods can communicate with all other pods without using network address translation (NAT). In some cases, the orchestrator 23 and the network controller 24 create a service virtual network and a pod virtual network that are shared by all namespaces, and assign service and pod network addresses from these networks, respectively. In some cases, all pods in all namespaces that are spawned in a Kubernetes cluster are able to communicate with each other, and network addresses for all pods can be assigned from a pod subnet specified by the orchestrator 23. When a user creates an isolated namespace for a pod, the orchestrator 23 and the network controller 24 can create a new pod virtual network and a new shared service virtual network for the new isolated namespace. Pods in the isolated namespace that are spawned in the Kubernetes cluster draw network addresses from the new pod virtual network, and corresponding services for such pods draw network addresses from the new service virtual network.

[0060] The CNI 17A can represent a library, a plugin, a module, a runtime, or other executable code of the server 12A. The CNI 17A can comply at least in part with the Container Network Interface (CNI) specification or the rkt network proposal. The CNI 17A can represent Contrail, OpenContrail, Multus, Calico, cRPD, or other CNIs. The CNI 17A can also be referred to as a network plugin or a CNI plugin or a CNI example. Separate CNIs can be invoked by, for example, the Multus CNI to establish different virtual network interfaces for the pod 202A.

[0061] CNI 17A is invoked by orchestrator 23. For the purposes of the CNI specification, a container can be considered synonymous with a Linux network namespace. What this corresponds to depends on the particular container runtime implementation: for example, in implementations of the application container specification (e.g., rkt), each pod runs in a unique network namespace. In Docker, however, a network namespace typically exists for each individual Docker container. For the purposes of the CNI specification, a network refers to a set of entities that are uniquely addressable and can communicate with each other. This can be individual containers, machines / servers (real or virtual), or some other network device (e.g., a router). Containers can be conceptually added to or removed from one or more networks. The CNI specification specifies some considerations for a consistent plugin ("CNI plugin").

[0062] Pod 22A includes a containerized DPDK workload that is designed to use DPDK to accelerate packet processing, e.g., by using DPDK libraries to exchange data with other components.

[0063] Pod 22A is configured with a management interface 26 for management or control, and is configured with a data interface 27 for high-throughput packet processing, and more specifically for sending and receiving packets through virtual router 21A to implement a high-throughput application. Management interface 26 can be the default interface of pod 22A. Pod 22A can implement management interface 26 as an Ethernet interface (e.g., named "eth0"), while virtual router 21A can implement management interface 26 as a tap interface, a virtio user interface, or other type of interface.

[0064] Pod 22A and virtual router 21A exchange packets using data interface 27. Data interface 27 can be a DPDK interface. Pod 22A and virtual router 21A can establish data interface 27 using vhost. Pod 22A can operate according to an aggregation model. Pod 22A can use a virtual device, e.g., a virtio device with a vhost user adapter, for user space container inter-process communication for data interface 27. Figure 5An example is shown in which virtio in this example can be a shared memory based solution for exchanging packets with virtual router 21 A. The shared memory can be a huge page that is initialized in DPDK and managed by the virtio device. Thus, workload of pod 22A is built on DPDK and uses DPDK operation data interface 27. Pod 22A and virtual router 21 A can use vhost to pull up the DPDK interface. In some examples, pod 22A can operate as a vhost server and virtual router 21 A as a vhost client for setting up the DPDK interface. In some examples, virtual router 21 A can operate as a vhost server and pod 22 as a vhost client for setting up the DPDK interface.

[0065] CNI 17A can incorporate Figure 1 One or more other components shown, management interface 26 and data interface 27 are configured for pod 22A. As noted above, these can be different types of interfaces. Each of management interface 27 and data interface 27 can be referred to herein as a virtual network interface. Other example types of virtual network interfaces are described below. Any container of pod 22A can utilize (i.e., share) any virtual network interface of pod 22A.

[0066] A virtual network interface can represent a virtual Ethernet (“veth”) pair, where each end of the pair is a separate device (e.g., a Linux / Unix device), one end of the pair is assigned to pod 22A, and one end of the pair is assigned to virtual router 21 A. A veth pair or end of a veth pair is sometimes referred to as a “port.” A virtual network interface can represent a macvlan network, which has a media access control (MAC) address assigned to pod 22A and virtual router 21 A for communication between containers of pod 22A and virtual router 21 A. A virtual network interface can alternatively be referred to as a virtual machine interface (VMI), a pod interface, a container network interface, a tap interface, a veth interface, or simply a network interface (in a particular context).

[0067] In Figure 1 In example server 12A, pod 22A is a virtual network endpoint in one or more virtual networks. Orchestration 23 can store or otherwise manage configuration data for application deployment that specifies the virtual network and specifies pod 22A (or one or more containers therein) as a virtual network endpoint of the virtual network. For example, orchestration 23 can receive the configuration data from a user, an operator / administrator, or other machine system.

[0068] As part of the process of creating pod 22A, orchestrator 23 requests network controller 24 to create corresponding virtual network interfaces for the virtual networks (indicated in the configuration data) for the virtual networks. Pod 22A can have different virtual network interfaces for each virtual network to which it belongs, as well as management interface 26. For example, data interface 27 can be a virtual network interface for a particular virtual network. Additional data interfaces can be configured for other virtual networks. Network controller 24 processes the request to generate interface configuration data for the virtual network interfaces of pod 22A. The interface configuration data can include a container or pod unique identifier and a list or other data structure that specifies network configuration data for configuring the virtual network interfaces for each virtual network interface. The network configuration data for a virtual network interface can include a network name, an assigned virtual network address, a MAC address, and / or a domain name server value. An example of interface configuration data in JavaScript Object Notation (JSON) format is as follows. Management interface 26 and data interface 27 can have separate interface configuration data.

[0069] Network controller 24 sends the interface configuration data to server 12A, and more specifically, in some cases, to virtual router 21A. To configure the virtual network interfaces of pod 22A, orchestrator 23 can invoke CNI 17A. CNI 17A obtains the interface configuration data from virtual router 21A and processes it. CNI 17A creates each virtual network interface specified in the interface configuration data. For example, CNI 17A can attach one end of a veth pair that implements management interface 26 to virtual router 21A, and can attach the other end of the same veth pair to pod 22A, which can implement the virtual user using a virtual user.

[0070] The following is an example interface configuration data for pod 22A as a virtual network interface for management interface 26.

[0071]

[0072]

[0073] A conventional CNI plugin is invoked by a container platform / runtime to receive an add command from the container platform to add a container to a single virtual network, and subsequently can invoke the plugin to receive a Del(ete) command from the container / runtime and remove the container from the virtual network. The term “invoke” can refer to instantiation of a software component or module in memory as executable code for execution by processing circuitry.

[0074] According to the techniques of this disclosure, the control flow to deploy the pod 22A and configure the pod 22A can include creating a high-speed data path between the pod 22A and the virtual router 21A by isolating the pod 22A from being able to access the respective data interfaces between other pods and the virtual router 21A deployed on the server 12A Figure 1 (not shown in the middle). For example, the pod 22A can use vhost / virtio libraries to create an Ethernet interface for communicating with the server 12A or more specifically with the virtual router 21A. To facilitate the isolation between the pods, the server 12A provides the pod 22A with access to unique file system resources for communicating between the virtual router 21A and the pod 22A. In the illustrated example, as part of deploying the pod 22A and creating the data interface 27, the CNI 17A passes file system resource description data 29 to the virtual router 21A. The file system resource description data 29 can include a file name, a directory with a directory path, a name, a socket directory with a directory path, a socket name, an identifier, or a combination thereof. The file system resource description data 29 describes resources within the file system of the server 12A, e.g., file names and directories, that will be known to the pod 22A but not to other pods deployed on the server 12A. In some examples, to ensure the uniqueness of the file system resources, the file system resource description data 29 is based on at least a portion of data describing the pod 22A, e.g., a substring of a pod identifier (ID), a unique identifier (UID), or a universally unique identifier (UUID). Such information can be included in a pod manifest (e.g., pod.yaml) and passed from the orchestrator 23 to the CNI 17A.

[0075] Based on the file system resource description data 29, the virtual router 21A creates an inter-process communication (IPC) channel using the file system resources. The IPC channel can be, for example, a Unix domain socket, or another type of interface through which the pod 22A and the virtual router 21A can exchange control messages. Domain sockets use the file system as their address space. Processes reference domain sockets as inodes, and multiple processes can use the same socket for communication. The server of the communication binds the UNIX socket to a path in the file system, so the client can use the path to connect with it.

[0076] Pod 22A can communicate with the virtual router using its allocated file system resources, but does not have visibility or other access to similar file system resources allocated to other pods for this purpose. Pod 22A can use its allocated file system resources to exchange control protocol messages with the virtual router 21A to configure data interface 27. Data interface 27 may include a control channel and a DPDK channel. The control channel uses file system resources allocated for control messages, and the DPDK channel is a high-speed data path between the virtual router 21A and pod 22A for high-throughput packet forwarding and processing. In some examples, the control channel using the allocated file system resources is a separate virtual network interface, such as management interface 26. In some examples, the allocated file system resources are socket files or directories of Unix domain sockets, which operate as control channels for creating and managing high-speed data paths. The high-speed data path may be an accelerated data path using DPDK.

[0077] Figure 2 This is a block diagram of an example computing device that facilitates data channel isolation between pods according to the techniques described in this disclosure. Figure 2 The computing device 200 can represent a real or virtual server, and can represent Figure 1 This is an example instance of any server 12. In this example, computing device 200 includes a bus 242 that couples hardware components of the computing device 200 hardware environment. Bus 242 couples a network interface card (NIC) 230, a storage disk 246, and one or more hardware-based processors 210 (hereinafter referred to as "processor 210"). NIC 230 may be SR-IOV compliant. In some cases, the front-side bus may couple processor 210 and storage device 244. In some examples, bus 242 may couple storage device 244, processor 210, and NIC 230. Bus 242 may represent a Peripheral Component Interface (PCI) Fast (PCIe) bus. In some examples, a Direct Memory Access (DMA) controller may control DMA transfers between components coupled to bus 242. In some examples, components coupled to bus 242 control DMA transfers between components coupled to bus 242.

[0078] Processor 210 may include one or more processors, each processor including an independent execution unit to execute instructions conforming to an instruction set architecture, the instructions being stored in a storage medium. The execution unit may be implemented as a separate integrated circuit (IC) or may be combined within one or more multi-core processors (or “many-core” processors), each multi-core processor being implemented using a single IC (i.e., a chip multiprocessor).

[0079] Disk 246 represents a computer-readable storage medium, including volatile and / or non-volatile, removable and / or non-removable media, implemented in any method or technology for storage of information, such as processor-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), EEPROM, flash memory, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the processor 210.

[0080] Main memory 244 includes one or more computer-readable storage media, which can include random access memory (RAM) such as various types of dynamic RAM (DRAM), including DDR2 / DDR3 SDRAM, or static RAM (SRAM), a flash memory, or any other form of fixed or removable storage media, which can be used for storing data and code-instructions that can be accessed by the computer. Main memory 244 provides a physical address space consisting of addressable memory locations.

[0081] Network interface card (NIC) 230 includes one or more physical interfaces 232 configured to exchange packets using links of an underlying physical network. Interfaces 232 can include port interface cards having one or more network ports. NIC 230 can also include on-card memory, e.g., to store packet data. Direct memory access transfers between NIC 230 and other devices coupled to bus 242 can read / write from / to the NIC memory.

[0082] Memory 244, NIC 230, storage disk 246, and processor 210 can provide an operating environment for a software stack including an operating system kernel 214 executing in kernel space. Kernel 214 can represent, for example, a Linux, a Berkeley Software Distribution (BSD), another Unix variant kernel, or a Windows Server operating system kernel, available from Microsoft Corporation. In some examples, the operating system can execute a hypervisor and one or more virtual machines managed by the hypervisor. Example hypervisors include Kernel-based Virtual Machine (KVM) for Linux kernels, Xen, ESXi available from VMware, Windows Hyper-V available from Microsoft, and other open source and proprietary hypervisors. The term hypervisor can encompass a virtual machine manager (VMM). The operating system including kernel 214 provides an execution environment for one or more processes in user space 245.

[0083] Kernel 214 includes a physical driver 225 that uses network interface card 230. Network interface card 230 may also implement SR-IOV to enable the sharing of physical network functions (I / O) between one or more virtual execution elements, such as containers 229A and 229B or one or more virtual machines. Figure 2 (Not shown in the image). Shared virtual devices (e.g., virtual functions) can provide dedicated resources, allowing each virtual execution element to access the dedicated resources of NIC 230. Therefore, NIC 230 appears as a dedicated NIC for each virtual execution element. A virtual function can represent a lightweight PCIe function that shares physical resources with physical functions used by physical driver 225 and other virtual functions. For an SR-IOV-enabled NIC 230, the NIC 230 can have thousands of available virtual functions according to the SR-IOV standard; however, for I / O-intensive applications, the number of configured virtual functions is typically much smaller.

[0084] The computing device 200 can connect to a physical network switching infrastructure, which includes an overlay network that extends the switching infrastructure from physical switches to software or "virtual" routers coupled to physical servers connected to the switching infrastructure, including a DPDK-based virtual router 220. The virtual router can be a physical server (e.g., Figure 1 A server 12) executes a process or thread or component thereof that dynamically creates and manages one or more virtual networks that can be used for communication between virtual network endpoints. In one example, a virtual router uses an overlay network to implement each virtual network, which provides the ability to separate the virtual address of an endpoint from the physical address (e.g., IP address) of the server on which the endpoint is executed. Each virtual network can use its own addressing and security scheme and can be considered orthogonal to the physical network and its addressing scheme. Various techniques can be used to transmit packets within and between virtual networks on a physical network. As used herein, the term "virtual router" can include Open vSwitch (OVS), an OVS bridge, a Linux bridge, a Docker bridge, or other means and / or software residing on a host device and performing the exchange, bridging, or routing of packets between virtual network endpoints of one or more virtual networks hosted by one or more servers 12.

[0085] Virtual router 220 can be executed as a kernel module or a user-space DPDK process (virtual router 220 is displayed in user space 245). Virtual router agent 216 can also be executed in user space. Figure 2In the example computing device 200, the virtual router 220 executes within user space as a DPDK-based virtual router, but in various implementations the virtual router 220 can execute within a hypervisor, host operating system, host application, or virtual machine. The virtual router agent 216 connects to the network controller 24 using a channel that is used to download configuration and forwarding information. The virtual router agent 216 programs this forwarding state into a virtual router data (or “forwarding”) plane represented by the virtual router 220. The virtual router 220 and the virtual router agent 216 can be processes.

[0086] The virtual router 220 can replace and incorporate the virtual routing / bridging functionality of the Linux bridge / OVS module that is typically used for Kubernetes deployed pods 202. The virtual router 220 can perform bridging (e.g., E-VPN) and routing (e.g., L3VPN, IP-VPN) for virtual networks. The virtual router 220 can perform network services such as applying security policies, NAT, multicast, mirroring, and load balancing.

[0087] The virtual router 220 can be multi-threaded and execute on one or more processor cores. The virtual router 220 can include multiple queues. The virtual router 220 can implement a packet processing pipeline. The virtual router agent 216 can stitch the pipeline from the simplest to the most complex depending on the operations to be applied to the packet. The virtual router 220 can maintain multiple instances of the forwarding base. The virtual router 220 can use RCU (read-copy-update) locks to access and update tables.

[0088] To send packets to other compute nodes or switches, the virtual router 220 (data plane) uses one or more physical interfaces 232. Generally, the virtual router 220 exchanges overlay packets with workloads such as VMs or pods 202 (in the middle). Figure 2 The virtual router 220 is configured with multiple virtual router network interfaces (e.g., vifs). These interfaces can include interfaces to the underlying NIC 230 (vif0 / 0) - which in some examples is a Linux bound interface; a kernel interface vhost0 (or vif0 / 2) for exchanging packets with the host operating system; and an interface to the virtual router agent 216 pkt0 (or vif0 / 2) for obtaining forwarding state from the network controller and sending exception packets. Additional virtual router network interfaces are used to communicate with virtual execution elements such as DPDK pods 202A-202B (collectively “pods 202”). In some contexts described in this disclosure, the virtual router network interfaces are alternatively referred to as virtual network interfaces because the virtual network interfaces of the virtual execution elements including the pods or other workloads are implemented in part using the virtual router network interfaces.

[0089] The virtual network interfaces 212A-212B (collectively referred to as "virtual network interface 212") and 213A-213B (collectively referred to as "virtual network interface 213") of the virtual router 220 are as follows: Figure 2 As shown. Each of the virtual network interfaces 212 and 213 can be a tap interface, a DPDK interface, a Unix domain socket, a virtio interface, a veth interface, a macvlan, another type of interface, or a combination of interface types.

[0090] In a kernel-based deployment of virtual router 220 (not shown), virtual router 220 is installed as a kernel module within the operating system. Virtual router 220 logs itself to the TCP / IP stack to receive packets from any desired operating system interface. The interface can be a binding, physical, tap (for VMs), veth (for containers), etc. In this kernel-mode, virtual router 220 relies on the operating system to send and receive packets from different interfaces. For example, the operating system can expose a tap interface supported by the vhost-net driver to communicate with workloads. Once virtual router 220 has logged packets from that tap interface, the TCP / IP stack sends all packets to it. Virtual router 220 sends packets via the operating system interface. Additionally, NIC queues (physical or virtual) are handled by the operating system. Packet processing may run in interrupt mode, which will generate interrupts and may result in frequent context switching. When packet rates are high, the overhead of frequent interrupts and context switching can overwhelm the operating system, leading to performance degradation.

[0091] In the DPDK-based deployment of virtual router 220 (such as...) Figure 2The virtual router 220 is installed as a user space 245 application linked to the DPDK libraries (shown). This can result in faster performance than a kernel-based deployment, especially in the presence of high packet rates. The DPDK Poll Mode Driver (PMD) uses the physical interface 232 instead of the kernel's interrupt-based driver. The registers of the physical interface 232 can be exposed in the user space 245 so that the PMD can access them; the physical interface 232 bound in this way is no longer managed by or visible to the host operating system, and the DPDK-based virtual router 220 manages the physical interface 232. This includes packet polling, packet processing, and packet forwarding. In other words, the user packet processing steps are performed by the virtual router 220 DPDK data plane. This "poll mode" nature makes the virtual router 220 DPDK data plane packet processing / forwarding more efficient than interrupt mode when packet rates are high. There are relatively fewer interrupts and context switches during packet I / O compared to the kernel mode virtual router 220, and in some cases, interrupts and context switches during packet I / O can be avoided entirely.

[0092] The pods 202 are DPDK-based pods. In general, each pod 202A-202B can be assigned one or more virtual network addresses for use in a corresponding virtual network, where each virtual network can be associated with a different virtual subnet provided by the virtual router 220. The pod 202B can be assigned its own virtual Layer 3 (L3) IP address, e.g., for sending and receiving communications, but can not be aware of the host IP address of the computing device 200 in which the pod 202B resides. The virtual network addresses can thus be different from the logical addresses of the underlying physical computer system, e.g., the computing device 200.

[0093] The computing device 200 includes a virtual router agent 216 that controls the virtual network overlay of the computing device 200 and orchestrates the routing of packets within the computing device 200. In general, the virtual router agent 216 communicates with a network controller 24 Figure 1 ) for a virtualization infrastructure, which generates commands to create virtual networks and configure network virtualization endpoints, e.g., the computing device 200, and more specifically, the virtual router 220 and any of the virtual network interfaces 212, 213. By configuring the virtual router 220 based on information received from the network controller 24, the virtual router agent 216 can support configuring network isolation, policy-based security, gateways, source network address translation (SNAT), load balancers, and service chaining capabilities for orchestration.

[0094] In one example, network packets (e.g., Layer 3 (L3) IP packets or Layer 2 (L2) Ethernet packets) generated or consumed by containers 229A-B within a virtual network domain can be encapsulated in another packet (e.g., another IP or Ethernet packet) transported by a physical network. Packets transported in the virtual network can be referred to herein as "inner packets," while the physical network packets can be referred to herein as "outer packets" or "tunnel packets." Router 220 can perform encapsulation and / or decapsulation of virtual network packets within physical network packets. This functionality is referred to herein as tunneling, and can be used to create one or more overlay networks. In addition to IPinIP, other example tunneling protocols that can be used include IP over Generic Routing Encapsulation (GRE), VxLAN, Multiprotocol Label Switching (MPLS) over GRE, MPLS over User Datagram Protocol (UDP), etc. Virtual router 220 performs tunneling encapsulation / decapsulation for packets originating / destined to any containers of pod 202, and virtual router 220 exchanges packets with pod 202 via a bridge of bus 242 and / or NIC 230.

[0095] As described above, network controller 24 can provide a logically centralized controller to facilitate operation of one or more virtual networks. Network controller 24 may, for example, maintain a routing information base, e.g., one or more routing tables storing routing information for the physical network as well as one or more overlay networks. Virtual router 220 implements one or more Virtual Routing and Forwarding instances (VRFs) 222 for the respective virtual network for which virtual router 220 operates as a respective tunnel endpoint. In general, each VRF 222 stores forwarding information for a corresponding virtual network and identifies where a packet is to be forwarded and whether the packet is to be encapsulated in a tunnel protocol, e.g., with a tunnel header that can include one or more headers of different layers of the virtual network protocol stack. Each VRF 222 can include a network forwarding table storing routing and forwarding information for the virtual network.

[0096] NIC 230 can receive a tunnel packet. Virtual router 220 processes the tunnel packet to determine the virtual network of the source and destination endpoints of the inner packet from the tunnel encapsulation header. Virtual router 220 can strip the Layer 2 header and tunnel encapsulation header to forward the inner packet internally only. The tunnel encapsulation header can include a virtual network identifier, e.g., a VxLAN tag or MPLS label, that indicates the virtual network, e.g., a virtual network corresponding to one of the VRFs 222. The VRF can include forwarding information for the inner packet. For example, the VRF can map a destination Layer 3 (L3) address of the inner packet to a virtual network interface 212. In response, the VRF forwards the inner packet to pod 202A via virtual network interface 212.

[0097] Containers 229A-229B can also use internal packets as source virtual network endpoints. For example, container 229A can generate Layer 3 internal packets destined for a destination virtual network endpoint executed by another computing device (i.e., not computing device 200) or another of containers 229A-229B. Container 229A sends the Layer 3 internal packets to virtual router 220 via virtual network interface 212.

[0098] Virtual router 220 receives internal packets and Layer 2 headers and determines the virtual network of the internal packets. Virtual router 220 can use any of the aforementioned virtual network interface implementation technologies (e.g., macvlan, veth, etc.) to determine the virtual network. Virtual router 220 uses VRF 222A corresponding to the virtual network of the internal packets to generate an outer header for the internal packets, which includes an outer IP header overlaying the tunnel and a tunnel encapsulation header identifying the virtual network. Virtual router 220 encapsulates the internal packets with the outer header. Virtual router 220 can encapsulate the tunnel packets with a new Layer 2 header having a destination Layer 2 (L2) address associated with a device outside computing device 200 (e.g., TOR switch 16 or a server 12). If outside computing device 200, virtual router 220 uses physical function 221 to output tunnel packets with the new Layer 2 header to NIC 230. NIC 230 outputs packets on its outbound interface. If the destination is another virtual network endpoint executed on computing device 200, then virtual router 220 will route the packet to the appropriate one of virtual network interfaces 212, 213.

[0099] In some examples, the controller of computing device 200 (e.g., Figure 1 The network controller 24) configures a default route in each pod 202 so that the pod uses the virtual router 220 as the initial next hop for outbound packets. In some examples, the NIC 230 is configured with one or more forwarding rules so that all packets received from the pod are switched to the virtual router 220.

[0100] More specifically, pods 202A-202B can represent Figure 1 Example instance of pod 22A. Pod 202A includes one or more containers 229A, and pod 202B includes one or more containers 229B.

[0101] More specifically, a 204 error on a container platform can represent... Figure 1 An example instance of container platform 19A. Container platform 204 includes container engine (or "runtime") 208, orchestration agent 209, and service agent 211. CNI 206 can represent Figure 1 Example instance of CNI17A.

[0102] Container engine 208 includes code executable by processor 210. Container runtime 208 can be one or more computer processes. Container engine 208 runs containerized applications in the form of containers 229A-229B. Container engine 208 can represent Dockert, rkt, or other container engines used to manage containers. Generally, container engine 208 receives, requests, and manages objects such as images, containers, networks, and volumes. An image is a template with instructions to create a container. A container is an executable instance of an image. Based on instructions from orchestration agent 209, container engine 208 can obtain images and instantiate them into executable containers 229A-229B within pods 202A-202B.

[0103] Service agent 211 includes code executable by processor 210. Service agent 211 can be one or more computer processes. Service agent 211 monitors the addition and removal of services and endpoint objects, and maintains the network configuration of computing device 200, for example, to ensure communication between pods and containers using services. Service agent 211 can also manage iptables to capture traffic to the virtual IP address and port of the service and redirect traffic to the proxy port of the proxy-supported pod. Service agent 211 can represent the kube agent of a subordinate node of a Kubernetes cluster. In some examples, container platform 204 does not include service agent 211, or disables service agent 211 to facilitate CNI 206 configuring virtual router 220 and pod 202.

[0104] Orchestration agent 209 includes code executable by processor 210. Orchestration agent 209 can be one or more computer processes. Orchestration agent 209 can represent a kubelet of a subordinate node of a Kubernetes cluster. Orchestration agent 209 is an orchestrator (e.g., Figure 1 The orchestration agent 209 acts as an agent for the orchestration engine 208, receiving container specification data for the container and ensuring that the container is executed by the computing device 200. The container specification data may be a manifest file sent from the orchestration engine 23 to the orchestration agent 209, or received indirectly via a command-line interface, an HTTP endpoint, or an HTTP server. The container specification data may be a pod specification (e.g., a PodSpec—a YAML (yet another markup language) or JSON object describing the pod) for a pod 202 of the container 229. Based on the container specification data, the orchestration agent 209 instructs the container engine 208 to obtain and instantiate the container image of the container 229 for the computing device 200 to execute the container 229.

[0105] The orchestration agent 209 instantiates CNI 206 and configures one or more virtual network interfaces for each pod 202. CNI 206 can represent...Figure 1 An example instance of the CNI 17A. For example, the orchestration agent 209 receives the container specification data of the pod 202A and instructs the container engine 208 to create the pod 202A with the container 229A based on the container specification data of the pod 202A. The orchestration agent 209 also invokes the CNI 206 to configure the virtual network interface 212A of the pod 202A with a virtual network corresponding to one VRF 222. Any of the virtual network interfaces 213a-213b can represent Figure 1 An example instance of the management interface 26 described in the Figure 2 An example instance of the data interface 27 described in the Management virtual network interfaces 213a-213b can be connected to different VRFs (or respective different VRFs) than the VRF of any of the data virtual network interfaces 212a-212b.

[0106] The CNI 206 can obtain interface configuration data for configuring the virtual network interfaces of the pod 202. The virtual router agent 216 operates as a virtual network control plane module for enabling the network controller 24 to configure the virtual router 220. Unlike the orchestration control plane (including the container platform 204 for the affiliate nodes and the master node, e.g., the orchestrator 23) that manages provisioning, scheduling, and managing virtual execution elements, the virtual network control plane (including the network controller 24 and the virtual router agent 216 for the affiliate nodes) manages the configuration of the virtual network implemented in the data plane by the virtual router 220 of the affiliate nodes. The virtual router agent 216 communicates the interface configuration data of the virtual network interfaces to the CNI 206 to enable the orchestration control plane element (i.e., the CNI 206) to configure the virtual network interfaces according to the configuration state determined by the network controller 24, thereby bridging the gap between the orchestration control plane and the virtual network control plane. Further, this can enable the CNI 206 to obtain interface configuration data for multiple virtual network interfaces of a pod and configure the multiple virtual network interfaces.

[0107] According to the techniques described in this disclosure, the CNI 206 provides socket description data 217 to the virtual router agent 216, facilitating file system isolation between the pods 202. For example, to configure the data interface 212A between the virtual router 220 and the pod 202A, the CNI 206 obtains socket description data 217 specifying a directory and a socket name for a Unix domain socket. Based on the socket description data 217, the virtual router 220 and the DPDK pod 202A create a Unix domain socket 215A, which can operate as a control channel for the data interface 212A. The Unix domain socket 215A is created with a socket directory based on the directory specified by the socket description data 217, and the Unix domain socket 215A is named and referenced using the socket name specified by the socket description data 217. Using the socket 215A, the virtual router 220 and the pod 202A exchange control messages to negotiate a DPDK ring for the DPDK-based data interface 212A over shared memory. In other words, a user space vhost protocol occurs once the communication channel for the socket 215A is established. During this protocol message exchange, the pod 202A and the virtual router 220 exchange information about the pod's memory regions, virtual addresses, and supported features. At the end of the message exchange, the virtio ring is enabled, and data communication can occur between the pod 202A and the virtual router 220.

[0108] The CNI 206 can support IPv6 IPv4, non-DPDK interfaces, DPDK interfaces, configure virtual network interfaces for DPDK-based virtual routers, and / or configure virtual network interfaces for kernel-based virtual routers. The CNI 206 can validate user input, e.g., socket description data found in a pod manifest. If the indicated vhost mode of the DPDK application is incompatible with the vhost mode of the virtual router 220, the CNI 206 can return an error.

[0109] Example techniques for generating a socket directory path are described elsewhere in this disclosure. Similar operations can be performed by the CNI 206, the virtual router agent 216, the virtual router 220, and the pod 202B for creating the socket 215B and the data interface 212B.

[0110] The socket directory specified by the socket description data 217 is for the pod 202B or any other pod deployed to the computing device 200 to use for creating a Unix domain socket for a data interface. The socket directory is not used by the virtual router 220 to create a Unix domain socket for a data interface. The socket directory is not used by the virtual router agent 216 to create a Unix domain socket for a data interface. Figure 3The pod 202A (except for the pod that includes the virtual router 220, if any) can be unknown and / or inaccessible. The socket directory specified by the socket description data 217 can be different from the socket directory specified by the corresponding socket description data used to create the data interface 212B. Because the socket 215A and the socket 215B use different socket directories, which have been created using different specified socket directories in the respective socket description data, the pods 202 cannot access and / or know of each other's data interfaces 212A and 212B.

[0111] An example call flow of the CNI 206 and the virtual router agent 216 operations is as follows.

[0112] The network controller manager (e.g., the network controller manager 325 of Figure 3 The network controller manager listens to the pod create / delete / update events of the orchestrator 23. Upon detecting a new pod create event, the network controller manager parses the pod information and creates the corresponding interface configuration data for each virtual network interface to be configured for the pod 202A. The network controller manager optionally passes this interface configuration data to the virtual router agent 216 via the network controller 24, which has an interface with the virtual router agent 216 to manage the virtual routing in the computing device 200.

[0113] In parallel with the above passing of the interface configuration data to the virtual router agent 216, the orchestration agent 209 obtains the "set pod" event generated by the orchestrator 23 for deploying the pod 202A to the computing device 200. As part of deploying the pod 202A, the orchestration agent 209 invokes the CNI 206 to configure the network for the pod 202A. The CNI 206 in turn obtains the list of virtual network interfaces for the pod 202A and the interface configuration data from the virtual router agent 216. In some examples, the CNI 206 invokes the GET POD_CONFIG method of the virtual router agent 216 interface. The following is an example of the interface configuration data for the virtual network interface 212A of the pod 202A:

[0114]

[0115]

[0116] The CNI 206 processes the interface configuration data and requests the virtual router agent 216A to configure the virtual network interfaces of the virtual router 220. For example, the CNI 206 can invoke the port add message (e.g., POST VMI) of the virtual router agent 216 API. The following are example parameters of the request message to configure the virtual network interfaces from the CNI 206 to the virtual router agent 216A:

[0117]

[0118]

[0119] The request message includes additional parameters for the socket description data 217. That is, the socket description data 217 in this example includes vhostsocket-dir and vhostsocket-filename parameters (shown in bold). The vhostsocket-dir is the socket directory for the socket 215A. In this example, the value for vhostsocket-dir is based on a path of / var / run / vrouter (the base folder for vrouter) and a unique value for that folder in / var / run / VRouter. In this case, the unique value is based on a unique identifier for the pod 202A, which is included in the interface configuration data for the virtual network interface 212A of the pod 202A. In some examples, the container identifier can be the container identifier. As a result, the pods 202B will not be able to access or see this directory, and the data interfaces 212 of the pods 202 are isolated from each other. In this case, the vhostsocket-filename parameter value is also based on the identifier for the pod 202A (vm-id) and the name of the interface (“netl”). The vhostsocket-filename can be used as the socket 215A socket name. This parameterization of these values, generated by the CNI 206 and provided to the virtual router agent 216, is different from existing schemes in which the socket directory is fixed and the socket name is automatically derived by the virtual router agent 216 based on the virtual network interface name or identifier.

[0120] The virtual router agent 216 processes the request message, and the virtual router 220 and virtual router agent 216 use this information to configure the data interface 212A for the pod 202A. If in vhost server mode, the virtual router 220 listens on a socket directory for the socket 215A. If in client mode, the virtual router 220 initiates communication on the socket directory for the socket 215A. The virtual router 220 and pod 202A exchange control messages in compliance with the vhost protocol to negotiate a DPDK ring for the DPDK-based data interface 212A over shared memory. In this way, the virtual router 220 is configured with a virtual router end of the data interface 212A, which is a DPDK data path between the virtual router 220 and the pod 202A for high-speed packet processing and forwarding. The virtual router agent 216 also configures the data interface 212A with network information, e.g., MAC and IP addresses, any required VLANs, DNS and gateway information, MTU, and in some cases other interface data, e.g., interface name / id. The virtual router agent 216 can obtain at least some of the network information for configuring the data interface 212A from the network controller 24. Whether the virtual router 220 is in client mode or server mode for the control protocol on the socket 215A can be specified in the request message from the CNI 206 to the virtual router agent 216A. In the example above, this parameter is vhostuser-mode (shown in bold).

[0121] The CNI 206, in turn, obtains network information for the data interface 212A from the virtual router agent 216. For example, the CNI 206 can call the GET VM IP method of the virtual router agent 216 API to request the network information. Below is an example of network information for the data interface 212A:

[0122] CNI 206 can report at least some network information to orchestrator 23 via orchestration agent 209. CNI 206 provides or configures at least some network information to pod 202A so that containers 229A of pod 202A can use data interface 212A as a virtual network interface. Such network information can include MAC and IP addresses and an interface name. CNI 206 also provides socket description data for socket 215A to pod 202A, e.g., socket directory and socket name. These values can be the same as the values that CNI 206 provides to virtual router agent 216A for socket description data. CNI 206 can also provide interface name and / or network configuration data for data interface 212A to pod 202A, e.g., IP address, MAC address, DNS data, MTU, etc. This information and socket description data can be made available to pod 202A through a configuration file in a volume mounted via a hostPath directory or via an annotation installed through a down API.

[0123] Alternatively, pod 202A can derive some or all of the socket description data instead of being provided by CNI 206. For example, pod 202A can derive a socket name for socket 215A based on a container identifier for one of containers 229A, an interface name or interface identifier for data interface 212A, a pod identifier or unique identifier, a namespace, an annotation, a label, or any combination of two or more of the above. An example script for deriving a socket name as the first 12 characters of the container identifier and an interface name is:

[0124] IFNAME = netl

[0125] CONTAINERID = $(sed -ne ' / hostname / p' ' / proc / 1 / task / 1 / mountinfo' | awk -F'' '{print $6}')

[0126] SOCKNAME = ${CONTAINERID:0:12}-$IFNAME

[0127] A containerized application running within a pod can query pod-related network information and configuration using the app-netutils API.

[0128] In some examples, the computing device 200 and orchestrator 23 can use a mutating webhook to provide socket description data for creating the socket 215A. For example, a component can implement an http callback with an API provided by kubernetes. If the component intends to modify a resource object, the component can use a mutating admission webhook. For this type of webhook, the action taken can be modified or denied. Alternatively, if the component wants to enforce custom policies, the component can use a validating admission webhook. For this type of webhook, the action taken can be accepted or denied, and no modification is allowed. The webhook records a rule with the API server 320, e.g., the rule: on pod creation. In one event, if there is a match with the recorded rule, the API server 320 will initiate a call to the defined webhook for the network hook, and pass the resource object to the defined handler function. The action taken on the kubernetes resource object will be as used for the type of webhook.

[0129] In the case of a mutating admission webhook and a validating admission webhook, these webhooks can be registered for the same rule. The mutating admission webhook is called first, and then on success, the API server 320 can call the validating admission webhook to make the final decision to accept or deny the request.

[0130] An advantage of using webhooks is that all workloads deployed under the DPDK category can be categorized by defining a tag, thus simplifying the definitions that users need to make. This can be a nearby feature for users who want to define a tag so that the volume and volume mount can be injected by the network hook in the pod definition (i.e., pod spec). Another advantage is that it reduces errors, ensuring that the volume mount in the container associated with the volume defined using hostpath “ / var / run / vrouter” has a subpath with POD_UID, and the environment variable used in subPathExpr is available. The configuration chart can control the functionality of the webhooks, e.g., enabling / disabling validation or mutation, or specifying user-defined default parameters. If the pod is rescheduled to a different node or deleted, the socket file and unique folder will remain in the host. Therefore, it is important to listen to the pod deletion event to take appropriate action to clean up the directory and socket file. CRUD operations on annotations can be defined here. If a particular feature is needed, new annotations can be added, modified, or deleted, and custom warning messages can be added to indicate the reason for denial.

[0131] Pod 202A (or more specifically, a DPDK application or library executing as part of pod 202A in some cases) establishes a socket based on the socket description data. If in vhost server mode, pod 202A listens for socket directory of socket 215A. If in client mode, pod 202A initiates communication on the socket directory of socket 215A. Whether pod 202A is in vhost server mode or client mode can be specified by CNI 206, as with the vhostuser-mode parameter in the example above (shown in bold). Virtual router 220 and pod 202A exchange control messages in compliance with the vhost protocol to negotiate a DPDK ring for DPDK-based data interface 212A over shared memory. In this way, pod 202A is configured with a workload / pod side of data interface 212A, which is a DPDK data path between virtual router 220 and pod 202A for high-speed packet processing and forwarding. Having established data interface 212A, virtual router 220 can process and forward network packets from or to virtual networks of pod 202A.

[0132] Similar call flows can be performed by CNI 206, virtual router agent 216, virtual router 220, and pod 202B to create data interface 212B using socket 215B. As a result, pod 202 has separate DPDK interfaces established using separate socket-based control channels that are not accessible to each other. Additional example details, implementations, and alternative implementations of the above-described functionality of configuring data interfaces using socket description data are described below.

[0133] In some examples, CNI 206 obtains socket description data from orchestrator 23, rather than through orchestration agent 209. CNI 206 can parse a pod manifest (described elsewhere in this disclosure) that indicates a socket directory and / or socket name for socket 215A. In some examples, virtual router agent 216 can obtain interface configuration data from a routing protocol process executing on the computing device. Example details of virtual router agent 216 communicating with a routing protocol process (e.g., a containerized routing protocol daemon) are described in U.S. Patent Application No. 17 / 649,632, filed February 1, 2022, entitled “Containerized Router with Virtual Networking,” which is hereby incorporated by reference in its entirety. In such examples, CNI 206 can be tailored for the routing protocol process. For example, CNI 206 can communicate socket description data 217 to the virtual router agent via a REST interface. The routing protocol process can add a VRF, update a virtual network interface, a routing target, and a next hop. Similar to described elsewhere, virtual router agent 216 adds data interface 212A to virtual router 220 and includes the socket directory and socket name for socket 215A, establishes a vhost control channel, to negotiate and establish a vhost datapath (or DPDK datapath).

[0134] A pod 202 can be created and deployed using an improved workflow that produces a DPDK-capable container. Below is an example script to create a DPDK pod image. The script is a Dockerfile, but other tools can be used as well:

[0135]

[0136]

[0137] According to the techniques of this disclosure, socket description data 217 can be based on data provided by orchestrator 23. For example, a pod manifest for pod 202A can indicate a socket directory for socket 215A using a string that can include a variable for a unique string that distinguishes the socket directory for socket 215A from other socket directories configured on computing device 200, for example. An example pod manifest (“pod.yaml”) for pod 202A is as follows:

[0138]

[0139]

[0140]

[0141] The above boldface parameters can be used to specify socket description data 217. " / var / run / vrouter / pod <num>The mountPath of the volume installation of the shared directory ("shared-dir") of the "vhost" socket 215A can be used as the vhost socket path, or to share data between CNI 206 and pod 202A. Another example mountPath can be " / var / run / vrouter / <poduid>" / var / run / vrouter / <poduuid>". variable <num> 、 <poduid> 、 <poduuid>Before being included in socket description data 217 to virtual router agent 216, the pod identifier, unique identifier, or universally unique identifier of pod 202A can be replaced by CNI 206. The hostPath of the shared-dir of the volume is also specified in a similar manner. A volume is a directory accessible by containers in the pod, specified by the volume structure. Orchestration agent 209 locally mounts each volume at the specified mountPath and maintains a list of host paths. In the case of a data interface 212, the hostPath volume mounts a file or directory from the file system of computing device 200 into the pod to provide access to data outside the pod for the socket directory of socket 215 ("shared-dir"). The hostPath path (here, " / var / run / vrouter / pod <num>") for specifying a directory location on the computing device 200. Once the container is started, the container creates a path in the container and binds the path in the container to the corresponding host path. Likewise, a pod <num>Can be done by CNI 206, which receives the incomplete hostPath path from orchestrator 23, e.g., via communication or in the pod manifest.

[0142] CNI 206 can use hostPath shared-dir installation and subPathExpr, make the socket directory of socket 215A different, e.g., unique, using different pod identifiers (or unique identifiers or uuids). Below is another example of a pod manifest using subPathExpr:

[0143]

[0144]

[0145] To create and use a unique socket directory for socket 215A, based on the above specification, the socket directory will be created as “ / var / run / vrouter / <POD_UID> / ” (hostPath path + $POD_UID[subPathExpr]). CNI 206 adds data interface 212A with the port add message to virtual router agent 216, which has been extended to include socket description data, here the socket directory and socket name. Virtual router agent 216 configures virtual router 220 with the new virtual network interface and includes the socket description data, so that virtual router 220 can listen or create socket 215A based on the socket description data. Because CNI 206 does not receive the UUID in the pod manifest, CNI 206 must be able to assume that the socket directory derivation logic is consistent, e.g., in the above example, / var / run / vrouter / <POD_UID>. If the user uses an option different from POD_UID, then socket 215A setup and data interface 212A setup can fail. The specification should be documented so that the user is conveyed the specification requirements.

[0146] In the above example, compute device 200 needs to have the “var / run / vrouter” directory in order to deploy the pod. Orchestration agent 209 creates the directory named “var / run / vrouter / <pod_UID> / dpdk0” at the time of pod deployment, and then installs the path to “ / vhu” for the workload. The workload can continue to use “ / vhu”, but on compute device 200, the content will be available on the extended unique path “var / run / vrouter / <pod_UID> / dpdk0”.

[0147] The pod manifest can be further annotated to specify the new interface type that the DPDK-based virtual router 220 communicates with the pod 202. The new interface type to be specified can include virtio-user / tap or vhost-user interface. Below is an example schema of a pod manifest for a pod 202:

[0148]

[0149]

[0150] The interfacetype parameter can specify "vhost" or other string to indicate a DPDK pod, or can specify virtio-user / veth to indicate a non-DPDK pod. In this way, the computing device 200 can support and concurrently execute a mix of DPDK and non-DPDK pods. The appvhost-mode parameter can specify whether the DPDK application of the pod is running vhost in client mode or server mode. The CNI 206 can return an error if the virtual router 220 is not compatible with the specified mode. The virtual router 220 can support veth interfaces, DPDK interfaces, virtio-user PMD to veth communication, vhost PMD to virtio-user (DPDK), virtio-user, and / or upstream vhost.

[0151] The configuration data for specifying the operation of the CNI 206 can include additional parameters that support the techniques herein. Below is an example of a CNI configuration:

[0152]

[0153]

[0154] The vrouter-mode parameter can be used to handle default cases. For example, if the pod annotation "interfacetype" (as described above) is missing from the pod manifest, the default value for the interface type can be derived by the CNI 206 from the vrouter-mode, i.e., virtio-user if DPDK is supported, e.g., vhost-user.

[0155] Figure 3 is a block diagram of an example computing device operating as an instance of a controller and orchestrator of virtualized computing infrastructure in accordance with the techniques described in this disclosure. Figure 1 The computing device 300 of FIG. 30 can represent a centralized or distributed system of one or more real or virtual servers configured to perform the operations of at least one of the network controller 24 and the orchestrator 23. In this way, the computing device 300 can implement one or more master nodes for the respective cluster in some cases.

[0156] Although illustrated and described as being executed by a single computing device 300, the scheduler 322, API server 320, controller manager 326, network controller 324, network controller manager 325, and configuration store 328 can be distributed among multiple computing devices 300 that make up a computing system or hardware / server cluster. In other words, each of the multiple computing devices 300 can provide a hardware operating environment for one or more instances of any one or more of the scheduler 322, API server 320, controller manager 326, network controller 324, network controller manager 325, or configuration store 328. The network controller 324 can represent an example instance of the network controller 24 of Figure 2

[0157] In this example, the computing device 300 includes a bus 342 that couples the hardware components of the computing device 300 hardware environment. The bus 342 couples a network interface card (NIC) 330, a storage disk 346, and one or more microprocessors 310 (hereinafter “processors 310”). In some cases, a frontside bus can couple the processors 310 and the storage 344. In some examples, the bus 342 can couple the memory 344, the processors 310, and the NIC 330. The bus 342 can represent a peripheral component interface (PCI) express (PCIe) bus. In some examples, a direct memory access (DMA) controller can control DMA transfers between components coupled to the bus 242. In some examples, components coupled to the bus 342 control DMA transfers between components coupled to the bus 342.

[0158] The processors 310 can include one or more processors each including independent execution units to execute instructions in conformance with an instruction set architecture, the instructions stored in a storage medium. The execution units can be implemented as separate integrated circuits (ICs) or can be combined in one or more multi-core processors (or “many core” processors), each using a single IC (i.e., a chip multiprocessor).

[0159] ​Disk 346 represents a computer-readable storage medium, including volatile and / or non-volatile, removable and / or non-removable media, implemented in any method or technology for storage of information, such as processor-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), EEPROM, flash memory, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the processor 310.

[0160] Main memory 344 includes one or more computer-readable storage media, which can include random access memory (RAM) such as various types of dynamic RAM (DRAM), including DDR2 / DDR3 SDRAM, or static RAM (SRAM), a flash memory, or any other form of fixed or removable storage media, which can be used for storing data and code-in the form of instructions or data structures that can be accessed by the computer. Main memory 344 provides a physical address space that is accessed via an address space of the processor 310.

[0161] Network interface card (NIC) 330 includes one or more interfaces 332 configured to exchange packets using links of an underlying physical network. Interfaces 332 can include port interface cards having one or more network ports. NIC 330 can also include on-card memory, e.g., to store packet data. Direct memory access transfers between NIC 330 and other devices coupled to bus 342 can read / write from / to the NIC memory.

[0162] Memory 344, NIC 330, storage disk 346, and processor 310 can provide an operating environment for a software stack that includes an operating system kernel 314 executing in kernel space. Kernel 314 can represent, for example, a Linux, a Berkeley Software Distribution (BSD), another Unix variant kernel, or a Windows Server operating system kernel, available from Microsoft Corporation. In some examples, the operating system can execute a hypervisor and one or more virtual machines managed by the hypervisor. Example hypervisors include Kernel-based Virtual Machine (KVM) for Linux kernels, Xen, ESXi available from VMware, Windows Hyper-V available from Microsoft, and other open source and proprietary hypervisors. The term hypervisor can encompass a virtual machine manager (VMM). The operating system including kernel 314 provides an execution environment for one or more processes in user space 345. Kernel 314 can include a physical driver 327 that uses network interface card 230.

[0163] The computing device 300 can be coupled to a physical network switch fabric that includes an overlay network extending the fabric from a physical switch to a software or "virtual" router coupled to a physical server of the fabric, e.g., the virtual router 220 of Figure 1 The computing device 300 can configure the cluster's affiliate nodes using one or more dedicated virtual networks.

[0164] The API server 320, the scheduler 322, the controller manager 326, and the configuration store can implement a master node of a cluster, which can also be referred to as a "master component." The cluster can be a Kubernetes cluster, and the master node can be a Kubernetes master node, in which case the master component is a Kubernetes master component.

[0165] The API server 320 includes code executable by the processor 310. The API server 320 can be one or more computer processes. The API server 320 authenticates and configures data for objects, e.g., virtual execution elements (e.g., container pods), services, and replication controllers. A service can be an abstraction that defines a logical unit of work and a policy for accessing pods. A set of pods implementing a service is selected based on the service definition. A service can be implemented in part as a load balancer, or otherwise include a load balancer. The API server 320 can implement a Representational State Transfer (REST) interface to handle REST operations and provide a front end to a shared state of a respective cluster stored in the configuration store 328. The API server 320 can authenticate and authorize requests. The API server 320 communicates with other components to instantiate virtual execution elements in the computing infrastructure 8. The API server 320 can represent a Kubernetes API server.

[0166] The configuration store 328 is a backing store for all cluster data. The cluster data can include cluster state and configuration data. The configuration data can also provide a backend for service discovery and / or provide a locking service. The configuration store 328 can be implemented as a key-value store. The configuration store 328 can be a central database or a distributed database. The configuration store 328 can represent an etcd store. The configuration store 328 can represent a Kubernetes configuration store.

[0167] The scheduler 322 includes code executable by the processor 310. The scheduler 322 can be one or more computer processes. The scheduler 322 monitors newly created or requested virtual execution elements (e.g., container pods) and selects an affiliate node on which the virtual execution element is to run. The scheduler 322 can select an affiliate node based on resource requirements, hardware constraints, software constraints, policy constraints, location, etc. The scheduler 322 can represent a Kubernetes scheduler.

[0168] Generally, the API server 320 can call the dispatcher 322 to schedule a pod, which can select an affiliate node and return an identifier of the selected affiliate node to the API server 320, which can write the identifier to the configuration store 328 associated with the virtual execution element. The API server 320 can call the orchestration agent 209 of the selected affiliate node, which can cause the container engine 208 of the selected affiliate node to obtain the pod from the storage server and create the pod on the affiliate node. The orchestration agent 209 of the selected affiliate node can update the state of the virtual execution element to the API server 320, which saves the new state to the configuration store 328. In this way, the computing device 300 instantiates, e.g., to the server 12, a new container in the computing infrastructure 8.

[0169] The controller manager 326 includes code executable by the processor 310. The controller manager 326 can be one or more computer processes. The controller manager 326 can be embedded in a core control loop that monitors the shared state of the cluster by obtaining notifications from the API server 320. The controller manager 326 can attempt to move the state of the cluster toward a desired state. Example controllers (not shown) managed by the controller manager 326 can include replication controllers, endpoint controllers, namespace controllers, and service account controllers. The controller manager 326 can perform lifecycle functions, e.g., namespace creation and lifecycle, event garbage collection, terminated pod garbage collection, cascading deletion garbage collection, node garbage collection, etc. The controller manager 326 can represent a Kubernetes controller manager of a Kubernetes cluster.

[0170] The network controller 324 includes code executable by the processor 310. The network controller 324 can include one or more computer processes. The network controller 324 can be implemented using virtual machines, containers, or as local processes. The network controller 324 can be a logically centralized but physically distributed software-defined network (SDN) controller that is responsible for providing management, control, and analytics functions for virtualized networks. In particular, the network controller 324 can be a logically centralized control and management plane for the computing infrastructure 8 and orchestrates virtual routers for one or more affiliate nodes.

[0171] The network controller 324 can provide cloud networks for computing architectures operating on network infrastructures. The cloud networks can include private clouds for enterprises or service providers, infrastructure as a service (IaaS), and virtual private clouds (VPCs) for cloud service providers (CSPs). The private cloud, VPC, and IaaS use cases can involve multi-tenant virtualized data centers, such as Figure 1 to Figure 2 In this case, multiple tenants of the data center share the same physical resources (physical servers, physical storage, physical networks). Each tenant is allocated its own logical resources (virtual machines, containers, or other forms of virtual execution elements; virtual storage; virtual networks). These logical resources are isolated from each other unless the security policy specifically allows otherwise. Virtual networks in the data center can also be interconnected to physical IP VPNs or L2 VPNs.

[0172] The network controller 324 can provide network function virtualization (NFV) for the network, e.g., business edge networks, broadband subscriber management edge networks, and mobile edge networks. NFV involves orchestration and management of network functions, e.g., firewalls, intrusion detection or prevention systems (IDS / IPS), deep packet inspection (DPI), caching, wide area network (WAN) optimization, etc., in virtual machines, containers, or other virtual execution elements, rather than on physical hardware devices. The primary drivers for network service virtualization in this market are time to market and cost optimization.

[0173] The network controller 324 programs the network infrastructure elements to create virtual networks and can create interface configurations for virtual network interfaces of the virtual networks.

[0174] Further information regarding the operation of the network controller 24 in conjunction with other devices of the data center 10 or other software-defined networks is found in International Application No. PCT / US2013 / 044378 and U.S. Patent Application No. 14 / 226,509, which are hereby incorporated by reference above.

[0175] The network controller manager 325 includes code executable by the processor 310. The network controller manager 325 can be one or more computer processes. The network controller manager 325 acts as an interface between the orchestration-facing elements (e.g., the scheduler 322, the API server 320, the controller manager 326, and the configuration store 328) and the network controller 324. Generally, the network controller manager 325 monitors for new objects (e.g., pods and services) of the cluster. The network controller manager 325 can isolate pods in a virtual network and connect the pods with services.

[0176] The network controller manager 325 can execute as a container of the cluster master node. In some cases, using the network controller manager 325 enables disabling of the service proxy (e.g., the Kubernetes kube-proxy) of the minion nodes, thereby implementing all pod connections using virtual routers, as described herein.

[0177] Network Controller Manager 325 can use the orchestration platform's controller framework to listen for (or monitor) changes to objects defined in the API and add annotations to some of these objects. Annotations can be labels or other identifiers specifying object attributes (e.g., "virtual network green"). Network Controller Manager 325 can use the interface to Network Controller 324 to create network solutions for applications, defining network objects such as virtual networks, virtual network interfaces, and access control policies. Network Controller 324 can implement network solutions in the computing infrastructure, for example, by configuring one or more virtual networks and virtual network interfaces in a virtual router.

[0178] The following example deployment configuration for this application consists of pods and their virtual network information:

[0179]

[0180]

[0181]

[0182] This metadata information is copied to each pod replica created by controller manager 326. When these pods are notified, network controller manager 325 can create the virtual networks listed in the comments ("Red Network", "Blue Network", and "Default / External Network" in the example above) and create a virtual network interface per pod replica for each virtual network (e.g., pod 202A) with a unique private virtual network address from the cluster-wide address block (e.g., 10.0 / 16) of the virtual network.

[0183] According to the technology disclosed herein, network controller manager 325 can obtain new network annotations from the pod inventory to specify the interface type (e.g., "interface type") and / or specify the vhost mode (e.g., "appvhost-mode"). Network controller manager 325 can pass this network annotation data to network controller 324, which then provides the network annotation data to virtual router agent 216. The network annotation data can be included in interface configuration data used to configure the virtual network interface of the pod, as referenced above. Figure 1 to Figure 3 As described above. If the interface type annotation indicates the creation of a DPDK type interface, the virtual router 220 can use the indicated vhost mode to connect to the pod's DPDK app to negotiate the socket for the DPDK interface between the pod and the virtual router 220.

[0184] Figure 4 The various components, functional units, and / or modules illustrated in and / or described herein or elsewhere in this disclosure can perform operations described using software executing in and / or on one or more computing devices. For example, a computing device can execute one or more such modules with multiple processors or multiple devices. A computing device can execute one or more such modules as virtual machines executing on underlying hardware. One or more such modules can be executed as one or more services of an operating system or computing platform. One or more such modules can be executed as one or more executable programs at an application layer of a computing platform. In other examples, functionality provided by a module can be implemented by a dedicated hardware device. Although certain modules, data stores, components, programs, executable programs, data items, functional units, and / or other items contained in the storage devices can be illustrated separately, one or more of these items can be combined and operate as a single module, component, program, executable program, data item, or functional unit. For example, one or more modules or data stores can be combined or partially combined such that they operate or provide functionality as a single module. Also, one or more modules can operate in conjunction with each other such that, for example, one module acts as a service or extension of another module. Further, each module, data store, component, program, executable program, data item, functional unit, or other item illustrated in the storage devices can include multiple components, subcomponents, modules, submodules, data stores, and / or other components or modules or data stores not illustrated. Also, each module, data store, component, program, executable program, data item, functional unit, or other item illustrated in the storage devices can be implemented in various ways. For example, each module, data store, component, program, executable program, data item, functional unit, or other item illustrated in the storage devices can be implemented as part of an operating system executing on a computing device.

[0185] Figure 2 to Figure 3 is a flow diagram illustrating one example workflow for creating a data interface between a pod and a virtual router in accordance with some aspects described in this disclosure. For purposes of example, reference is made to Figure 5 The operation is described in terms of components of the computing devices 200 and 300. The API server 320 receives a request to instantiate the pod 202A and modifies the configuration store 328 by generating and storing configuration information for creating the pod 202A (402). The pod specification (pod.yaml) includes data for generating socket description data. The scheduler 322 can select the computing device 200 as a host affiliate node for the pod 202A. The API server 320 can annotate the pod 202A with a list of one or more virtual networks and an identifier for the pod (e.g., a universally unique identifier (pod uuid) or a simple unique identifier (pod uid)). Other forms of identifiers for the pod can be used. The annotations can be tags of the pod configuration that indicate the virtual networks, e.g., "virtual network A" and "virtual network B." The annotations can also indicate that the interfaces for the virtual networks should be DPDK interfaces (e.g., using "interface type").

[0186] The network controller manager 325 listens for new objects from the API server 320, determines that the pod 202A is to be instantiated on the computing device 200, and determines from the annotations that the pod 202A needs to have virtual network interfaces for the virtual networks indicated in the annotations. The listening can be responsive to subscribing to API server 320 notifications on a RESTful interface, for example.

[0187] The network controller manager 325 instructs the network controller 324 to create the virtual networks and create virtual network interfaces for the pod 202A of the virtual networks (404). The network controller manager 325 can annotate the pod with the respective uuids of the one or more virtual network interfaces (e.g., vni uuids) to be created by the network controller 324 and the respective unique private virtual network addresses (and in some cases MAC addresses) that are assigned. Other forms of identifiers for the virtual network interfaces can be used.

[0188] The network controller 324 can associate the virtual network interfaces with the pod in the interface configuration data for the pod 202A. For example, the network controller 324 can create a list of virtual network interfaces for the virtual networks and can associate the vni uuids with the pod uuid in the interface configuration data for the pod 202A. The vni-uuids can be another identifier for the virtual network interfaces, e.g., a virtual machine interface identifier. The network controller 324 can send the interface configuration data 25 to the virtual router agent 216 of the virtual router 220 of the computing device 200 and configure the respective virtual network interfaces 212 in the computing device 200 (406). The virtual router agent 216 can store the association of each vni uuid with the respective configured virtual network interface.

[0189] To set up the pod 202A, the orchestration agent 209 fetches the container specification data for the pod 202A and ensures that the container is executed by the compute device 200 (408). The container specification data can include a pod uuid for the pod 202A. The orchestration agent 209 calls the CNI 206 to configure virtual network interfaces for the pod 202A (410). The CNI 206 requests (412) and obtains interface configuration data (414) from the virtual router agent 216. The CNI 206 can obtain the interface configuration data from the virtual router agent 216 by requesting the interface configuration data for the pod corresponding to the pod uuid included in the container specification data for the pod 202A.

[0190] To create each virtual network interface 212 indicated in the interface configuration data 25 (416), the CNI 206 can create the interface in the pod 202A by passing in the configuration data (418) and can make any necessary changes on the compute device 200. The configuration data in step 418 can include, for example, IP address (IPv4 / IPv6), MAC, DNS configuration, MTU, interface name, and vhost socket dir path and socket name to the vhost control channel (socket 215A) to the DPDK application pod 202A.

[0191] The CNI 206 informs the virtual router agent 216 of the now running (by configuration of the pod 202A) virtual network interfaces 212 through a respective port add message (420). The CNI 206 can also obtain virtual network addresses from the virtual router agent 216 (422), or by calling the appropriate IPAM plugin. The CNI 206 can configure the virtual network addresses within the pod 202A network namespace and can establish routing by calling the virtual router agent 216. Alternatively, the CNI 206 can configure the virtual network addresses within the pod 202A network namespace and can establish routing consistent with the IP address management section by calling the appropriate IPAM plugin. The CNI can update the orchestration control plane by notifying the orchestration agent 209 (424).

[0192] Figure 1 is a block diagram illustrating an example computing device 500 with a DPDK- based virtual router and containerized DPDK workloads, with interfaces established in accordance with one or more aspects of the present disclosure. The virtual router agent 504 can represent an example instance of the virtual router agent 216. The vRouter DPDK data plane 502 (hereinafter "virtual router 502") can represent an example instance of the virtual router 220 or any DPDK vRouter 21. The virtual router 502 performs the "forwarding plane" or packet forwarding functions of a virtual router for the computing device 500, while the virtual router agent 504 performs the "control plane" functions of a virtual router.

[0193] The virtual router agent 504 includes configuration data 534, virtual routing and forwarding instance configurations 536 ("VRFs 536"), and policies 538. The virtual router agent 504 exchanges control information with one or more virtual network controllers (e.g., the network controller 24 of Figure 6 The virtual router agent 504 can report analytics state and install forwarding state to the forwarding information 526 of the virtual router 502. The virtual router agent 504 can apply slow path packet processing to the first (initial) packet of each new flow that passes through the virtual router 502 and install a corresponding flow entry into the flow table for the new flow so that the virtual router 502 performs fast path processing for subsequent packets of the flow. The forwarding information 526 can include separate forwarding information 528A-528N for corresponding VRFs 536.

[0194] The virtual router 502 can include multiple logical forwarding cores 510, which can be pthreads. The cores 510 can run in tight loops, also known as poll mode. DPDK queues can be used to exchange packets between them. Each core 510 has a receive queue that can be used by other cores 510 to queue packets that need to be processed by that core. The cores also poll different virtual router 502 interface queues, e.g., physical, VM, and tap. The virtual router 502 can thus be a multi-threaded user space application. The forwarding cores 510 poll the physical and virtual interfaces. In addition, the forwarding cores 510 can perform packet processing for the virtual router 502 according to a packet processing pipeline.

[0195] A particular forwarding core of the core 510 ("vhost core") handles messages between the DPDK pod 506 and the virtual router 502 on a vhost user control channel 562 in either client or server mode. Communication with the DPDK pod 506 is through a UNIX socket 560. Once the communication channel is established, the userspace vhost protocol occurs. During this protocol message exchange, the pod 506 and the vRouter exchange information about the pod memory region, virtio ring address, and supported features. At the end of the message exchange, the virtio ring 522 on shared memory 520 is enabled, and data communication between the pod 506 and the virtual router 502 can occur. In this way, the ring 522 and the control channel 562 constitute a virtual network interface for the pod 506 as a DPDK interface. As noted above, the virtual network interface is configured with network configuration data (e.g., overlay / virtual IP addresses) for forwarding packets originating from or destined for the pod 506.

[0196] The pod 506 is a DPDK-enabled pod, and the DPDK native application 508 is a DPDK-enabled containerized application that sets up a DPDK ring mapped on a virtio ring using a virtio PMD 516 and virtio device 517 and exchanges overlay packets with the virtual router 502.

[0197] Another core 510 is responsible for sending and receiving packets between the virtual router agent 504 and the virtual router 502. Once the virtual router agent is established to the virtual network link channel 552 (e.g., through a socket), an initialization sequence of this communication is triggered. The virtual router agent 504 uses the network link channel to add a "pktO" or "vif 0 / 2" interface to the virtual router 502 process. The virtual router 502 then creates a UNIX socket in the path " / var / run / vrouter / dpdk_pktO". The virtual router agent 504 will use this socket to send packets to the virtual router 502. In addition, another UNIX socket is connected to which the vRouter agent has created. The path of the virtual router agent 504 UNIX socket is " / var / run / vrouter / agent_pktO". The virtual router 502 will use this socket to send packets to the virtual router agent 504. The socket protocol is of the PACKET type, which means that the channel only transfers packets.

[0198] The interface vif 0 / 1 is connected to the interface vhostO in the Linux kernel. A separate management interface 544 connects the ethO 540 of the pod 506 and the virtio / tap interface 542 of the virtual router 502.

[0199] Trajectory or tungsten fabric is an example network control architecture. The CNI 206 can be developed for a trajectory or tungsten fabric. The trajectory / tungsten fabric controller can be one example of the network controller 24.

[0200] Figure 7 A flowchart for an example mode of operation of a computing device group in accordance with the techniques described herein. The virtual router 220 and the DPDK-enabled pod 202A create a Unix domain socket 215A using a socket directory that is accessible by the pod 202A but not accessible by any other pod (except the pod including the virtual router 220) deployed to the computing device 200 (e.g., the pod 202B) (602). The virtual router 220 implements a virtual network in a computing infrastructure including the computing device 200 to enable communication between virtual network endpoints connected via the virtual network (604). The virtual network endpoints can include the pod 202A. Using the Unix domain socket 215A, the virtual router 220 and the pod 202A exchange control communications for a DPDK-based data interface 212A between the virtual router 220 and the pod 202A (606).

[0201] ​ A flowchart for an example mode of operation of an orchestrator in accordance with the techniques described herein is shown. Although described with respect to the orchestrator 23, the operations 700 can be performed by any of the orchestrators or orchestration systems described herein. The orchestrator 23 obtains a pod specification for a pod (702). The pod specification includes data for generating socket description data that indicates a file system resource that the pod uses to create a Unix domain socket when the pod is deployed to a computing device, the file system resource being accessible by the pod and not accessible by any other pod deployed to the computing device. The orchestrator 23 outputs the pod specification to an orchestration agent executing on the computing device, where the orchestration agent is an agent of the orchestrator (704).

[0202] The techniques described herein can be implemented in hardware, software, firmware, or any combination thereof. Various features described as modules, units or components can be implemented together in an integrated logic device or separately as discrete but interoperable logic devices or other hardware devices. In some cases, various features of electronic circuitry can be implemented as one or more integrated circuit devices, such as integrated circuit chips or chip sets.

[0203] If implemented in hardware, the present disclosure can involve the use of an apparatus such as a processor or an integrated circuit device, such as an integrated circuit chip or chipset. Alternatively or additionally, if implemented in software or firmware, the techniques can at least partially be implemented by computer-readable data storage mediums including instructions that, when executed by a processor, cause the processor to perform one or more of the methods described above. For example, computer-readable data storage media can store such instructions for execution by a processor.

[0204] A computer-readable medium can form part of a computer program product, which can include packaging materials. A computer-readable medium can include a computer data storage medium, e.g., random access memory (RAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read only memory (EEPROM), FLASH memory, magnetic or optical data storage media, and the like. In some examples, an article of manufacture can include one or more computer-readable storage media.

[0205] In some examples, a computer-readable storage medium can include a non-transitory medium. The term "non-transitory" can indicate that the storage medium is not embodied in a carrier wave or a propagated signal. In certain examples, a non-transitory storage medium can store data that can, over time, change (e.g., in RAM or cache).

[0206] Code or instructions can be software and / or firmware executed by processing circuitry including one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor" as used herein can refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein can be provided within software modules or hardware modules within the apparatus.< / num> < / num> < / poduuid> < / poduid> < / num> < / poduuid> < / poduid> < / num>

Claims

1. A computing device, comprising: A virtual router, including processing circuitry and configured to implement a virtual network in a computing infrastructure including the computing device, so as to enable communication between virtual network endpoints connected via the virtual network; as well as Pods, including containerized applications, The virtual router and the pod are configured to create Unix domain sockets using the file system resources of the computing device's file system. The file system resources are accessible by the pod and the virtual router but not by any other pod deployed to the computing device. The file system resources include the socket directory of the Unix domain socket, the socket directory having a directory path that includes at least a portion of the pod identifier of the pod.

2. The computing device according to claim 1, wherein, The virtual router and the pod are configured to use the Unix domain socket as a control channel to exchange control communication of the data interface between the virtual router and the pod.

3. The computing device according to claim 2, wherein, The data interface includes the Data Plane Development Kit (DPDK) interface.

4. The computing device according to claim 2, wherein, The data interface is used for the virtual network interface configured in the virtual router so that the pod can communicate with other virtual network endpoints of the virtual network.

5. The computing device according to any one of claims 1 to 4, further comprising: The Container Network Interface (CNI) module includes processing circuitry. as well as A virtual router proxy for the virtual router, the virtual router proxy including processing circuitry. The CNI is configured to send socket description data indicating the file system resources to the virtual router proxy, and The virtual router agent provides the socket description data to the virtual router to create the Unix domain socket.

6. The computing device according to claim 5, wherein, The CNI is configured to generate socket description data indicating the file system resource based on the pod identifier of the pod.

7. The computing device according to claim 6, further comprising: An orchestration agent, including processing circuitry, wherein the orchestration agent is an agent for an orchestrator of the computing infrastructure including the computing device. The CNI is configured to receive the pod identifier from the orchestrator via the orchestration agent.

8. The computing device according to claim 5, wherein, The virtual router agent is configured to create a virtual network interface in the virtual router, which uses a data interface between the virtual router and the pod created using Unix domain sockets.

9. The computing device according to any one of claims 1 to 4, further comprising: The Container Network Interface (CNI) module includes processing circuitry. as well as An orchestration agent, including processing circuitry, wherein the orchestration agent is an agent for an orchestrator of the computing infrastructure including the computing device, and The CNI is configured to receive the pod identifier of the pod from the orchestrator via the orchestration agent, and generate socket description data based on the pod identifier that indicates the socket directory and socket name of the Unix domain socket.

10. The computing device according to any one of claims 1 to 4, further comprising: A virtual router agent for the virtual router, the virtual router agent including processing circuitry and configured to: Receive interface configuration data from the network controller for the computing infrastructure, wherein the interface configuration data is for the virtual network interface of the pod, and wherein the interface configuration data indicates that the interface type of the virtual network interface is a Data Plane Development Kit (DPDK) interface; and In response to determining that the virtual network interface is a DPDK interface, the virtual network interface in the virtual router is configured as a DPDK interface.

11. The computing device according to any one of claims 1 to 4, further comprising: A virtual router agent for the virtual router, the virtual router agent including processing circuitry and configured to: Receive interface configuration data from the network controller for the computing infrastructure, wherein the interface configuration data is for the virtual network interface of the pod, and wherein the interface configuration data indicates the vhost mode of the pod for control communication via the Unix domain socket exchange; and In response to determining that the vhost mode of the pod is incompatible with the vhost mode of the virtual router, an error is output.

12. The computing device according to any one of claims 1 to 4, further comprising: The Container Network Interface (CNI) module includes processing circuitry. A virtual router proxy for the virtual router, the virtual router proxy including processing circuitry. The CNI module is configured to use a network hook to obtain socket description data indicating the file system resources from the orchestrator. The CNI module is configured to send the socket description data to the virtual router proxy, and The virtual router agent provides the socket description data to the virtual router to create the Unix domain socket.

13. A computer network method, comprising: The orchestrator obtains the pod specification, wherein the pod specification includes data for generating socket description data, the socket description data indicating file system resources used by the pod to create Unix domain sockets for the file system of the computing device when the pod is deployed to the computing device. The file system resources are accessible by the pod and the virtual router but not by any other pod deployed to the computing device; and The file system resources include the socket directory of the Unix domain socket, the socket directory having a directory path that includes at least a portion of the pod identifier of the pod; and The orchestrator outputs the pod specification to an orchestration agent executing on the computing device, wherein the orchestration agent is an agent of the orchestrator.

14. The method of claim 13, further comprising: The orchestration agent deploys the pod to the computing device; and The orchestration agent mounts the file system resources into the pod.

15. The method according to any one of claims 13 to 14, wherein, The socket description data includes the mountPath of the shared directory to be shared by the pod and the virtual router of the computing device, and the subPathExpr, which is the pod identifier of the pod.

16. A computer network method, comprising: A Unix domain socket is created by a virtual router and a pod running on the computing device using a socket directory of the computing device's file system. The socket directory is accessible by the pod and the virtual router but not by any other pod deployed to the computing device, wherein the socket directory has a directory path that includes at least a portion of the pod identifier of the pod. A virtual network is implemented by the virtual router within a computing infrastructure including the computing device, enabling communication between virtual network endpoints connected via the virtual network; and The virtual router and the pod use the Unix domain socket to exchange control communication between the virtual router and the pod via the Data Plane Development Kit (DPDK) interface.

17. The method according to claim 16, wherein, The DPDK interface is used as a virtual network interface configured in the virtual router, for the pod to communicate with other virtual network endpoints of the virtual network.

Images