Computer-implemented method, computer system and computer program product

Abstract

Embodiments are directed to container storage systems in remote Pods. A worker node virtual machine determines that a volume is available for attachment to the worker node virtual machine. Middle-ware of the worker node virtual machine causes a Pod container storage interface to attach the volume to a Pod virtual machine. In response to attaching the volume to the Pod virtual machine, the middle-ware of the worker node virtual machine causes the Pod container storage interface to mount the volume to the Pod virtual machine, such that the volume is available for use by the Pod virtual machine.

Description

Background Technology

[0001] This invention relates generally to computer systems, and more specifically to computer implementations of methods, computer systems, and computer program products configured and arranged for use in remote Pods in Kubernetes.

[0002] Kubernetes, often referred to as K8s, is an open-source container orchestration system for automating the deployment, scaling, and management of computer applications. Specifically, it aims to provide a platform for automating the deployment, scaling, and operation of application containers across host clusters. Kubernetes works with a suite of container tools and runs containers within a cluster, typically with images built using Docker. Docker is a collection of platforms that is a PaaS (Platform as a Service) offering that uses OS-level virtualization to deliver software in packages called containers. Containers are isolated from each other and bundle their own software, libraries, and configuration files; containers can communicate with each other through well-defined channels. All containers can share the services of a single operating system kernel. The basic unit of scheduling in Kubernetes is the Pod. A Pod is a set of containerized components. A Pod contains one or more containers, ensuring that these containers reside on the same node. Many cloud services offer Kubernetes-based platforms or infrastructures on which Kubernetes can serve as a platform for deploying services. The scheduler is a pluggable component that selects which node an unscheduled Pod (i.e., the basic entity managed by the scheduler) runs on based on resource availability. The scheduler tracks resource usage on each node to ensure that workloads are not scheduled beyond available resources. While existing technologies for Pods with containers configured to launch and run software applications on nodes in the cloud are suitable for their intended purpose, what is needed is a system with certain features of embodiments of the present invention. Summary of the Invention

[0003] Embodiments of the present invention relate to a computer-implemented method for a new container storage system in a remote Pod of Kubernetes. A non-limiting example of the computer-implemented method includes determining, by a worker node virtual machine, that a volume is available to be attached to the worker node virtual machine. The computer-implemented method includes using intermediate software of the worker node virtual machine to cause the Pod container storage interface to attach the volume to the Pod virtual machine. Furthermore, the computer-implemented method includes, in response to attaching the volume to the Pod virtual machine, using the intermediate software of the worker node virtual machine to cause the Pod container storage interface to mount the volume to the Pod virtual machine, making the volume available to the Pod virtual machine.

[0004] This provides an improvement over known methods by attaching and mounting volumes to and from a remote Pod that is running in a virtual machine separate from the worker node, thereby overcoming the problem that volumes attached to the worker node cannot be mounted to the remote Pod when the remote Pod is running in a virtual machine separate from the worker node.

[0005] In addition to one or more of the features described above or below, or as an alternative, in a further embodiment of the invention, the volume is a persistent volume. Therefore, this advantageously provides a technique for mounting a persistent volume to a remote Pod running in a virtual machine detached from the worker node.

[0006] In addition to one or more features described above or below, or as an alternative, in a further embodiment of the invention, the worker node virtual machine is configured to receive a request to attach the volume, and the request triggers the intermediate software to attach the volume to the Pod virtual machine, rather than attaching the volume to the worker node virtual machine. Therefore, this advantageously provides a technique for attaching volumes to remote Pods running in virtual machines detached from the worker node.

[0007] In addition to one or more features described above or below, or as an alternative, in a further embodiment of the invention, the worker node virtual machine is configured to receive a request to mount a volume, which triggers intermediate software to mount the volume to the Pod virtual machine, rather than mounting the volume to the worker node virtual machine. Therefore, this advantageously provides a technique for mounting volumes to remote Pods running in virtual machines detached from the worker node.

[0008] Other embodiments of the present invention feature the implementation of the above-described method in computer systems and computer program products.

[0009] Additional technical features and benefits are achieved through the technology of this invention. Embodiments and aspects of the invention are described in detail herein, and these embodiments and aspects are considered part of the claimed subject matter. For a better understanding, reference is made to the specific embodiments and accompanying drawings. Attached Figure Description

[0010] The details of the exclusive rights described herein are specifically pointed out and clearly claimed in the claims at the end of this specification. The foregoing and other features and advantages of embodiments of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0011] Figure 1 A block diagram of an example computer system used in conjunction with one or more embodiments of the present invention is depicted;

[0012] Figure 2 A block diagram of a system for providing and using remote containers in Kubernetes is depicted according to one or more embodiments of the present invention;

[0013] Figure 3 A block diagram depicts a node and Pod hierarchy in a system for providing and using remote Pods as virtual machines in Kubernetes, according to one or more embodiments of the present invention;

[0014] Figure 4 A block diagram depicts an example architecture of a system for providing and using remote containers as virtual machines in Kubernetes, according to one or more embodiments of the present invention;

[0015] Figure 5 A flowchart depicts a process for providing and using a remote Pod as a virtual machine in Kubernetes, according to one or more embodiments;

[0016] Figure 6 An example network architecture for providing and using remote Pods as virtual machines in Kubernetes is described according to one or more embodiments of the present invention;

[0017] Figure 7 A flowchart illustrating a computer implementation process for providing and using remote Pods as virtual machines in Kubernetes, according to one or more embodiments of the present invention, is described.

[0018] Figure 8 A block diagram depicts an example container storage interface architecture in a system for a new container storage system in a remote Pod in Kubernetes, according to one or more embodiments of the present invention.

[0019] Figure 9 A block diagram is depicted, illustrating further details of an architecture for installing a new container storage system in a remote Pod in Kubernetes using one or more new components in the container storage interface driver, according to one or more embodiments of the present invention.

[0020] Figure 10 This is a block diagram of a process for installing a new container storage system in a remote Pod in Kubernetes using a new container storage interface in the system, according to one or more embodiments of the present invention.

[0021] Figure 11Block diagrams depict different functions according to one or more embodiments of the present invention, which can be invoked and / or used to install a new container storage system in a remote Pod in Kubernetes using one or more new components in the container storage interface driver;

[0022] Figure 12 A block diagram of an extended container storage interface object for installing a new container storage system in a remote Pod of Kubernetes is depicted according to one or more embodiments of the present invention;

[0023] Figure 13 This is a flowchart of a computer-implemented method according to one or more embodiments of the present invention, the method being used to attach and mount a persistent volume to a remote Pod virtual machine instead of its worker node virtual machine;

[0024] Figure 14 A cloud computing environment according to one or more embodiments of the present invention is described; and

[0025] Figure 15 An abstract model layer is described according to one or more embodiments of the present invention. Detailed Implementation

[0026] One or more embodiments of the present invention provide methods, computer systems, and computer program products for arranging and configuring computer implementations for providing new container storage systems in remote Pods of Kubernetes. One or more embodiments of the present invention introduce a new worker and Pod hierarchy, wherein Pod virtual machines logically belong to worker node virtual machines, but physically run in virtual machines remote from worker virtual machines. Therefore, Pod virtual machines, operating within their own virtual machines, offer both better isolation and better performance. Furthermore, one or more embodiments provide a new container storage system by adding a Container Storage Interface (CSI) Pod plugin that can be used with CSI controller plugins and CSI node plugins. According to one or more embodiments, persistent volumes such as file storage and / or block storage can be attached to remote Pod virtual machines and mounted into containers within the remote Pod using the new CSI Pod plugin.

[0027] In a typical Kubernetes cluster, workloads (e.g., software applications) run as containers within Pods, which are processes on worker nodes. Kata Containers (which is open-source and Open Container Initiative (OCI) compliant) introduces virtual machines as Pods, allowing workloads to run within guest virtual machines on worker nodes used for cloud-native applications. However, Kata Pods running as guest virtual machines on worker node virtual machines introduce performance issues. Specifically, second-level or nested virtual machines typically experience a 5%-10% penalty in CPU and memory usage / efficiency, and even greater penalties in I / O, such as I / O operations per second (IOPS). In the worst-case scenario, the I / O degradation is approximately 30%.

[0028] One or more embodiments provide a novel system that allows Pod virtual machines to run (physically) as peer virtual machines while still logically belonging to worker node virtual machines. This aligns the system with Kubernetes functionality, avoiding the performance penalties / issues associated with nested virtual machines in Kata. According to one or more embodiments, the technical benefits and advantages include better isolation between worker node virtual machines and remote Pod virtual machines, as worker nodes cannot directly access the remote Pod virtual machines they manage. These benefits and advantages include better performance for remote Pod virtual machines in terms of CPU, memory, and disk usage / performance. Furthermore, one or more embodiments demonstrate better scalability because remote Pod virtual machines are not limited by the resources of worker node virtual machines and can be deployed on any host.

[0029] Furthermore, CSI drivers typically do not operate within remote Pod systems because CSI first attaches the persistent volume to the worker node and then mounts the volume into the Pod contained within the worker node. However, according to one or more embodiments, the remote Pod virtual machine runs in a separate virtual machine from the worker node virtual machine, and is therefore not contained within the worker node virtual machine. Therefore, a novel container storage system is provided that uses a new CSI Pod plugin component within the remote Pod virtual machine to attach and mount persistent volumes when the remote Pod virtual machine is not contained within the worker node virtual machine. One or more embodiments disclose an extended VolumeAttachment object to describe volume attachments and techniques to facilitate attaching persistent volumes to remote Pod virtual machines.

[0030] Turn now Figure 1The present invention generally illustrates a computer system 100 according to one or more embodiments of the present invention. As described herein, the computer system 100 may be an electronic computer framework including and / or employing any number and combination of computing devices and networks utilizing different communication technologies. The computer system 100 may be easily expandable, scalable, and modular, with the ability to be changed to different services or reconfigured independently of other features. The computer system 100 may be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, the computer system 100 may be a cloud computing node. The computer system 100 may be described in the general context of computer system executable instructions such as program modules executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. The computer system 100 may be practiced in a distributed cloud computing environment in which tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules may reside in local and remote computer system storage media including memory storage devices.

[0031] like Figure 1 As shown, computer system 100 has one or more central processing units (CPUs) 101a, 101b, 101c, etc. (collectively or generally referred to as processor 101). Processor 101 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Processor 101 (also referred to as processing circuitry) is coupled to system memory 103 and various other components via system bus 102. System memory 103 may include read-only memory (ROM) 104 and random access memory (RAM) 105. ROM 104 is coupled to system bus 102 and may include a basic input / output system (BIOS) or a successor such as a unified extensible firmware interface (UEFI), which controls certain basic functions of computer system 100. RAM is read-write memory coupled to system bus 102 for use by processor 101. System memory 103 provides temporary memory space for the operation of the instructions during operation. System memory 103 may include random access memory (RAM), read-only memory, flash memory, or any other suitable memory system.

[0032] Computer system 100 includes an input / output (I / O) adapter 106 and a communication adapter 107 coupled to a system bus 102. I / O adapter 106 may be a Small Computer System Interface (SCSI) adapter that communicates with a hard disk 108 and / or any other similar component. I / O adapter 106 and hard disk 108 are collectively referred to herein as mass storage 110.

[0033] Software 111 executing on computer system 100 may be stored in mass storage device 110. Mass storage device 110 is an example of a tangible storage medium readable by processor 101, wherein software 111 is stored as instructions for execution by processor 101 to operate computer system 100, as described below with respect to the various figures. Examples of computer program products and the execution of such instructions are discussed in more detail herein. Communication adapter 107 interconnects system bus 102 with network 112, which may be an external network, enabling computer system 100 to communicate with other such systems. In one embodiment, a portion of system memory 103 and mass storage 110 jointly store an operating system, which may be used for coordination Figure 1 The functions of the different components shown can be applied to any suitable operating system.

[0034] Additional input / output devices are shown connected to the system bus 102 via display adapter 115 and interface adapter 116. In one embodiment, adapters 106, 107, 115, and 116 may be connected to one or more I / O buses connected to the system bus 102 via an intermediate bus bridge (not shown). A display 119 (e.g., a screen or display monitor) is connected to the system bus 102 via display adapter 115, which may include a graphics controller to improve performance for graphics-intensive applications; the display adapter 115 may include a video controller. Keyboard 121, mouse 122, speakers 123, etc., may be interconnected to the system bus 102 via interface adapter 116, which may include, for example, a super I / O chip integrating multiple device adapters into a single integrated circuit. Suitable I / O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols such as Peripheral Component Interconnect (PCI) and High-Speed ​​Peripheral Component Interconnect (PCI). Therefore, as Figure 1 The computer system 100 configured therein includes processing capabilities in the form of a processor 101, storage capabilities including system memory 103 and mass storage 110, input devices such as a keyboard 121 and a mouse 122, and output capabilities including a speaker 123 and a display 119.

[0035] In some embodiments, the communication adapter 107 may use any suitable interface or protocol (such as an Internet Minicomputer System Interface) to transmit data. The network 112 may be a cellular network, radio network, wide area network (WAN), local area network (LAN), or the Internet. External computing devices may be connected to the computer system 100 via the network 112. In some examples, the external computing device may be an external network server or a cloud computing node.

[0036] It should be understood that, Figure 1 The block diagram is not intended to indicate that computer system 100 will include Figure 1 All components shown. Conversely, computer system 100 may include... Figure 1 Any suitable fewer or additional components not shown herein (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Furthermore, the embodiments described herein with respect to computer system 100 may be implemented with any suitable logic, wherein the logic mentioned herein may include any suitable hardware (e.g., processor, embedded controller, or application-specific integrated circuit, etc.), software (e.g., applications, etc.), firmware, or any suitable combination of hardware, software, and firmware.

[0037] Figure 2 This is a block diagram of a system 200 for providing and using remote Pods in Kubernetes according to one or more embodiments of the present invention. System 200 includes a computer system 202 coupled to computer systems 251, 252, 253, and 254. Computer system 202 may be referred to as a control node, master node, Kubernetes deployment controller, etc. Computer system 202 may be part of a control plane. Computer systems 251, 252, 253, and 254 may each be a server in a cluster. For illustrative purposes, computer systems 251, 252, 253, and 254 may each host one or more worker nodes in a cluster and may generally be referred to as computer systems 251-254. Computer systems 251-254 may be for nodes and for running virtual machines, as further discussed herein. Computer systems 202 and 251-254 may include Figure 1 Any elements and functions of the computer system 100 discussed herein are to be performed according to one or more embodiments.

[0038] Kubernetes has a control plane / panel at the top, also known as the master node. After the control plane (e.g., computer system 202) receives the workload (which is an application), the API server 204 saves the application as an API object to "etcd". In Kubernetes, the controller manager is responsible for orchestration throughout the control cycle. This control cycle is used to perform orchestration, which helps to spawn the Pods needed for these applications. Once a Pod appears, the scheduler watches for changes to the new Pod. If the scheduler finds a new Pod, it helps run the scheduling algorithm and writes the results (such as the node name on the NodeName field of the Pod object, i.e., the so-called binding operation). The scheduler then writes the binding results back to etcd; this is the scheduler's working process. Thus, a Pod is bound to a node, which is called scheduling. For example, a Pod is bound to a worker node.

[0039] kubelet (e.g., Figure 3 The kubelet (312) in the pod observes / monitors changes to all pod objects, and when it finds a pod bound to a node, the kubelet executes all subsequent tasks. After obtaining this information, the kubelet invokes the containerization process running on each machine and runs each container within that pod. At this point, the containers (e.g., ...) Figure 3 The CRI314 in the runtime environment can invoke other runtime environments (e.g., runc, shim, etc.). Therefore, the runtime can help establish these namespaces and cgroups, and help build the containers required by the application.

[0040] Figure 3 This is a block diagram of the node and Pod hierarchy in a system 200 for providing and using remote Pods in Kubernetes, according to one or more embodiments of the present invention. Figure 3Worker node virtual machine 310, Pod virtual machine 301, Pod virtual machine 302, and Pod virtual machine 303 are depicted. In any combination of any options, worker node virtual machine 310 may run on the same and / or different hosts as Pod virtual machines 301, 302, and 303, where Pod virtual machines 301, 302, and 303 may reside on the same and / or different hosts. For example, worker node virtual machine 310 and Pod virtual machine 301 may run on the same host (e.g., computer system 251). Moreover, worker node virtual machine 310 and Pod virtual machine 301 may run on two different hosts (e.g., computer systems 251 and 252), respectively. Similarly, worker node virtual machine 310 and Pod virtual machine 302 may run on the same host (e.g., on computer system 251) and / or different hosts (e.g., on one of computer systems 251 and 253, respectively). Similarly, worker node virtual machine 310 and pod virtual machine 303 may run on the same host (e.g., on computer system 251) and / or different hosts (e.g., one on computer systems 251 and 254 respectively). Although each pod virtual machine 301, 302, 303 may be bound to worker node virtual machine 310, it should be understood that each of pod virtual machines 301, 302, 303 may run on its own host, which may be different from the host running worker node virtual machine 310. Although three pods are discussed for illustrative purposes, it should be understood that the embodiments are not intended to be limited to three pods. Furthermore, each host (such as computer systems 251-254) has its own host operating system running its own hypervisor. When pod virtual machines 301, 302, 303 run on different hosts and / or different computer systems than worker node virtual machine 301, the hypervisors for each different host and / or computer system must be in the same logical network.

[0041] The control plane has one or more Application Programming Interface (API) servers 204 configured to communicate with the kubelet 312. API servers 204 (also known as API-servers or API servers) are lightweight web applications that allow Kubernetes to create and expose data APIs without requiring custom development. The kubelet is the master "node agent" running on each worker node. The kubelet can register a node with the API server using a hostname, flags that override the hostname, and / or any one or more of the cloud provider's specific logic. The kubelet uses and works with Pod descriptors, such as PodSpecs. A PodSpec is a YAML or JSON object that describes a Pod. Kubernetes resources are created declaratively, thus utilizing, for example, YAML files. As understood by those skilled in the art, Kubernetes resources such as Pods, services, and deployments can be created using YAML files. The kubelet takes a set of PodSpecs provided through different mechanisms (primarily via the API server) and ensures that the containers described in those PodSpecs are running and healthy.

[0042] exist Figure 3 In this context, worker node virtual machine 310 has a kubelet 312, which is configured to use the container runtime interface (CR1) 314 to create one or more shims 320_1, 320_2, and 320_3. Shims 320_1, 320_2, and 320_3 are generally referred to as shims 320. Each shim 320_1, 320_2, and 320_3 has a one-to-one relationship with its own Pod virtual machines 301, 302, and 303. Each individual shim 320_1, 320_2, and 320_3 is configured to create its own Pod virtual machines 301, 302, and 303, respectively, such that Pod virtual machines 301, 302, and 303 are not nested within and / or within guest virtual machines of worker node virtual machine 310. The Container Network Interface (CNI) 316 is configured to allow the IP network system in Pod virtual machines 301, 302, and 303 to be managed by the IP network system in worker node virtual machine 310. CNI 314, CNI 316, and pads 320_1, 320_2, and 320_3 are each software having computer-executable instructions that can be executed on a processor such as processor 101.

[0043] CRI 314 is sometimes referred to as CRI containerization, containerization, or containerized CRI. CRI 314 is a high-level runtime and daemon that can be thought of as an API panel for other runtimes in the stack. While lower-level runtimes like runc handle the actual running of containers, higher-level runtimes such as containerization handle container lifecycles, image management, and abstractions to the lower runtime. While lower-level runtimes provide mechanisms for building containers, containerization has mechanisms for building container platforms and APIs for remote applications to provide monitoring and delegation. CRI includes APIs, specifications / requirements, and libraries for container runtimes to integrate with Kubelets on nodes. CNI 316 is a runtime and / or specification for managing network resources on a cluster. CNI consists of specifications and libraries for writing plugins to configure network interfaces in containers, as well as several supported plugins. CNI itself focuses on container network connectivity and removing allocated resources when a container is deleted. Because of this, CNI has broad support and a simple specification implementation. When a Kubernetes component cannot communicate with another component, the pad 320 is software configured to translate between a component and its associated Kubernetes interface. For example, the pad 320 retrieves CR I commands and translates them into an agent 420 (in...). Figure 4 The container runtime shims are software pieces that reside between the container manager (containerization, cri-o, podman) and the container runtime (runc, crun), resolving the integration issues of these counterparts.

[0044] To provide further details, Figure 4 This is a block diagram of an example architecture in a system 200 for providing and using remote Pods as virtual machines in Kubernetes, according to one or more embodiments of the present invention. For simplicity, Figure 4 The computer system 202, which omits the control plane, should be understood as existing. Figure 4 An instance of using a pad 320_1 in worker node virtual machine 310 to provide, create, and / or build a Pod virtual machine 301 remote from worker node virtual machine 310 is described. Pod virtual machine 301 may be hosted on a separate host, remote from the host executing worker node virtual machine 310. Pod virtual machine 301 may be hosted on the same host as worker node virtual machine 310, while still being a separate or remote virtual machine. While the example discusses using a pad 320_1 in worker node virtual machine 310 to build Pod virtual machine 301, it should be understood that the description similarly applies to pad 320_2 for building Pod virtual machine 302 and pad 320_3 for building Pod virtual machine 303.

[0045] Figure 5 This is a flowchart of a process 500 for providing and using remote containers as computer implementations for Kubernetes, according to one or more embodiments of the present invention. Figure 5 The computer-implemented process 500 can be implemented using system 200. Therefore, the computer-implemented process 500 will now be described with reference to system 200.

[0046] At box 502, worker node virtual machine 310 is configured to receive Pod descriptions or Pod descriptors for creating new Pods in the cluster on the host. For example, kubelet 312 is configured to receive Pod descriptions from computer system 202. Pod descriptions may include Pod specifications (e.g., PodSpec) and are present in YAML or JSON files. Each Kubernetes deployment uses a so-called Pod template. As understood by those skilled in the art, this Pod template provides specifications that determine how a Pod should look, what applications run inside the Pod's containers, etc. With the help of a scheduler, the controller manager of computer system 202 can coordinate or assign Pods to worker node virtual machines 310.

[0047] At box 504, worker node virtual machine 310 is configured to use kubelet 312 to monitor Pods and invoke sandbox functions (as an environment) in runtime software (with computer-executable instructions). In one or more embodiments, after pad 320_1 has been started, pad 320_1 can invoke sandbox functions as a compute environment, as discussed further below. Sandbox 460 or Pod sandbox is an isolated compute environment on worker node virtual machine. For example, kubelet 312 is configured to invoke sandbox functions in CRI 314 (e.g., run Pod sandbox) to start running Pods. A sandbox is a security mechanism used to isolate running programs, typically attempting to mitigate the spread of system failures and / or software vulnerabilities. Sandboxes are typically used to execute programs or code without risking harm to the host or operating system. Kubelet 312 and / or CRI 314 can invoke CNI 316.

[0048] At box 506, worker node virtual machine 310 is configured to launch / create a new pad 320_1 to prepare a new Pod (if it has not already been created above). For example, CRI 314 may invoke pad 320_1 to create a remote virtual machine for a new Pod. Pad 320_1 is configured to create / launch a shell Pod 464 in a Pod sandbox (environment) such that pad 320_1 and / or the shell Pod appear to worker node virtual machine 310 as an actual Pod virtual machine. The shell Pod 464 may be an incomplete Pod used as a location holder for the Pod virtual machine to be created. More specifically, worker node virtual machine 310 interacts with pad 320_1 that manages / runs a Pod sandbox 460 with shell Pod 464, such that pad 320_1 represents the Pod. Due to shell Pod 464, worker node virtual machine 310 treats and / or recognizes pad 320_1 that manages / runs a Pod sandbox 460 with shell Pod 464 as and / or as an actual Pod virtual machine. Shell Pod 464 is started just like any other Pod, but is paused before setup is complete.

[0049] At box 508, worker node virtual machine 310 is configured to use pad 320_1 to invoke (Pod) virtual machine service 432 and cause virtual machine service 432 to create / instantiate Pod virtual machine 301. Pad 320_1 includes computer-executable instructions for communicating with virtual machine service 432 and requesting the creation of Pod virtual machine 301 on a host. Virtual machine service 432 is a service of the cluster and provides access to an endpoint. Pad 320_1 of worker node virtual machine 310 invokes virtual machine service 432 because it needs a service endpoint, and virtual machine service 432 then selects a host (e.g., computer system) based on its policies (such as host resource usage). For example, worker node virtual machine 310 may be hosted on computer system 251, and Pod virtual machine 301 may be hosted separately on the same computer system 251 (i.e., not nested within worker node virtual machine 310) and / or hosted on a different computer system 252. Unlike existing technology systems that create pods within the worker node virtual machine itself, according to one or more embodiments of the present invention, the pad 320_1 has computer-executable instructions that cause the virtual machine service 432 to obtain virtual resources (including CPU, memory, I / O, etc.) outside the worker node virtual machine 310 for creating the Pod virtual machine 301.

[0050] At box 510, worker node virtual machine 310 is configured to use pad 320_1 to perform Pod virtual machine initialization, create / invoke / build agent 420, create / invoke / build internal CRI 414 (e.g., internal containerized CRI) in Pod virtual machine 301, and establish network and tunnel between worker node virtual machine 310 and Pod virtual machine 301. To establish the network, pad 320_1 is configured to communicate with and / or instruct virtual machine service 432 to cause network processor 434 to create a logical network connection between worker node virtual machine 310 and Pod virtual machine 301. This logical network connection is an overlay network over the service-hosted network. Using the overlay network, worker node virtual machine 310 is configured to provide an Internet Protocol (IP) address to Pod virtual machine 301. As described herein, the logical network connection creates a tunnel for communication between worker node virtual machine 310 and Pod virtual machine 301. Worker node virtual machine 310 has a worker node identifier, and Pod virtual machine 301 has a Pod identifier, both of which are used to create the logical network connection and are stored in [the relevant database]. Figure 6 In the mapping table 650. Worker node virtual machine 310 and Pod virtual machine 301 are on the same network via a logical connection from a virtual Ethernet (e.g., eth0) in worker node virtual machine 310 to a virtual Ethernet (e.g., eth0) in Pod virtual machine 301. Network processor 434 can work in conjunction with a hypervisor to provide logical network connectivity. Network processor 434 runs on a host (e.g., computer system 251) and is responsible for creating a service-hosted network to connect remote Pod virtual machines and worker node virtual machines. When Pod virtual machines and worker node virtual machines are running on two separate hosts, there can be two separate host operating systems and two network processors working together on the network.

[0051] After establishing a logical connection, pad 320_1 creates agent 420, and agent 420 in Pod virtual machine 301 is configured to communicate with pad 320_1 in worker node virtual machine 310. In one or more embodiments, pad 320_1 may invoke agent 420 to be created in Pod virtual machine 301. In one or more embodiments, pad 320_1 may create agent 420 in sandbox 460 and forward agent 420 to Pod virtual machine 301. Agent 420 includes computer-executable instructions operating as discussed herein. After establishing a logical connection, pad 320_1 creates internal CRI 414, and internal CRI 414 is configured to create containers in Pod virtual machine 301. Agent 420 is configured to help forward requests between pad 320 and internal CRI 414. In one or more embodiments, pad 320_1 may invoke internal CRI 414 to create containers in Pod virtual machine 301 using agent 420. In one or more embodiments, pad 320_1 may create CRI 414 in sandbox 460 and forward CRI 414 to Pod virtual machine 301 via agent 420.

[0052] At box 512, shim 320_1 is configured to invoke / cause internal CRI 414 to create sample containers 440 and 441. For example, shim 320_1 is configured to forward APIs and / or API objects intended for use with shell Pod 464 to Pod virtual machine 301 via proxy 420, specifically to internal CRI 414, so that internal CRI 414 can create containers 440 and 441. Shim 320_1 is configured to send a request to proxy 420, which then redirects the request to internal CRI 414 within Pod virtual machine 301. CRI 414 is instructed to invoke containerization to create containers 440 and 441. Containers 440 and 441 are each opened / run to start / instantiate a separate software application on Pod virtual machine 301, causing the software application to execute on the host running Pod virtual machine 301. Conversely, containers 440 and 441 are created within the shell Pod 464 of sandbox 460, and containers 440 and 441 are created within Pod virtual machine 301. Furthermore, a paused container 462 can be started within sandbox 460. The paused container 462 is an incomplete and / or shelled container.

[0053] At box 514, a custom cAdvisor 410 and / or kubelet 312 (the custom cAdvisor 410 can be within kubelet 312) are configured to invoke the resource-aware service 430 to report Pod virtual machine 301 for resource monitoring. Reporting Pod virtual machine 301 includes reporting the Pod identity of Pod virtual machine 301 to the resource-aware service 430 and API server 204. The resource-aware service 430 is configured to maintain resources for Pod virtual machine 301.

[0054] Figure 6 This is a block diagram of an example network architecture in a system 200 for providing and using remote containers as virtual machines in Kubernetes, according to one or more embodiments of the present invention. Figure 6 Some components are omitted from the system 200 to avoid obscuring the figures, but it should be understood that system 200 includes the components and their functions as discussed herein. For simplicity, Pod virtual machines 301 and 302 are shown, but this description applies to Pod virtual machine 303 and any additional Pod virtual machines.

[0055] The host network processor 434 is responsible for providing access to the service-bearing network to establish an overlay network for worker node virtual machines 310 and Pod virtual machines 301, 302. The service-bearing network is the physical infrastructure on which the overlay network is built. The service-bearing network is the underlying network responsible for delivering packets across the network. The overlay network can be a Virtual Local Area Network (VLAN) and / or any applicable network configured by CNI 316. As initialized by shim 320_1 during Pod virtual machine creation, each of the Pod virtual machines 301, 302 has a unique Pod namespace. For example, Pod virtual machine 301 has Pod namespace 601, and Pod virtual machine 302 has Pod namespace 602. Similarly, shim 320_1 is configured to create proxy network namespaces 611 and 612 in worker node virtual machine 310. Each Pod namespace 601, 602 is an object in Pod virtual machines 301, 302, and each proxy network namespace 611, 612 is an object in worker node virtual machine 310. Using the Pod identifier of Pod virtual machine 301 stored in mapping table 650, pad 320_1 specifies proxy network namespace 611 as the proxy or proxy network of Pod virtual machine 301. Using the Pod identifier of Pod virtual machine 302 stored in mapping table 650, pad 320_1 specifies proxy network namespace 612 as the proxy or proxy network of Pod virtual machine 302. This mapping is stored in mapping table 650.

[0056] Each Pod virtual machine 301, 302 has virtual Ethernet connections for entering and leaving the Pod namespace containing these containers. For example, Pod virtual machine 301 has a virtual Ethernet connection (VETH1) in Pod namespace 601, which is connected to another virtual Ethernet connection (VETH0) outside Pod namespace 601 within Pod virtual machine 301, where these virtual Ethernet connections (VETH1 and VETH0) are a pair. Pod virtual machine 302 has a virtual Ethernet connection (VETH1) in Pod namespace 602, which is connected to another virtual Ethernet connection (VETH0) outside Pod namespace 602 within Pod virtual machine 302, where the virtual Ethernet connections (VETH1 and VETH0) are a pair on Pod virtual machine 302. In worker node virtual machine 310, proxy network namespace 611 has a virtual Ethernet connection (VETH1) in proxy network namespace 611, which is connected to another virtual Ethernet connection (VETH0) outside proxy network namespace 611, where the virtual Ethernet connections (VETH1 and VETH0) are a pair. In worker node virtual machine 310, proxy network namespace 612 has a virtual Ethernet connection (VETH3) connected to another virtual Ethernet connection (VETH2) outside of proxy network namespace 612, where virtual Ethernet connections (VETH3 and VETH2) are another pair. Proxy network namespaces 611 and 612 are each connected to the bridge (with their own IP address) via their respective virtual Ethernet connections.

[0057] CNI 316 and / or shim 320_1 are configured to create a tunnel, such as a VPN (e.g., tunnel 0), between the agent network namespace 611 in worker node virtual machine 310 and the Pod namespace 601 in Pod virtual machine 301, such that traffic (e.g., data) is mirrored / copied and communicates back and forth between worker node virtual machine 310 and Pod namespace 601. Similarly, CNI 316 and / or shim 320_2 are configured to create another tunnel, such as a VPN (e.g., tunnel 1), between the agent network namespace 612 in worker node virtual machine 310 and the Pod namespace 602 in Pod virtual machine 301, such that traffic (e.g., data) is mirrored / copied and communicates back and forth between worker node virtual machine 310 and Pod namespace 602. In other words, tunnels and virtual Ethernet connections mirror traffic back and forth. CNI 316 and / or pad 320_1 are configured to assign unique IP addresses to each Pod namespace 601, 602 in Pod virtual machines 301, 302 respectively, all of which are stored in mapping table 650.

[0058] As discussed in this paper, Pod virtual machines 301, 302, and 303 can be deployed to run on any host, which may be different from the host running worker node virtual machine 310, and / or one or more Pod virtual machines 301, 302, and 303 may be on the same host as worker node virtual machine 310, while other Pod virtual machines are on different hosts. From the worker node's perspective, it is unaware that the actual container footprint exists on other virtual machines different from worker node virtual machine 310. This means that there is no CNI code on the Pod virtual machines. Furthermore, worker node virtual machine 310 interfaces with pads 320_1, 320_2, and 320_3, for which proxy network namespaces have already been created; therefore, using the proxy network namespaces, worker node virtual machine 310 sends and receives data from the Pod namespaces as if Pod virtual machines 301, 302, and 303 existed on the same host.

[0059] Figure 7 This is a flowchart of a computer-implemented method 700 for instantiating / starting and using a remote Pod virtual machine that is far from its worker node virtual machine, according to one or more embodiments of the present invention. Figure 7 The computer-implemented method 700 can be implemented using system 200. Where appropriate, it can be referred to... Figure 1-6 .

[0060] At block 702 of the computer-implemented method 700, worker node virtual machine 310 is configured to instantiate / launch / invoke intermediate software (e.g., pad 320) within worker node virtual machine 310. For example, CRI 316 (e.g., containerized) may instantiate / launch / invoke pad 320 within worker node virtual machine 310.

[0061] At box 704, worker node virtual machine 310 is configured to use intermediate software (e.g., pad 320_1) to cause the creation of Pod virtual machines (e.g., Pod virtual machine 301), which are located remotely from worker node virtual machine 310. For example, Pod virtual machine 301 is not nested and / or contained within worker node virtual machine 310. Pod virtual machine 301 may be hosted on the same host as worker node virtual machine 310, for example, both may be hosted on computer system 251. Pod virtual machine 301 and worker node virtual machine 310 may be hosted on different hosts, for example, one may be hosted on computer system 251 and the other on computer system 252.

[0062] At box 706, worker node virtual machine 310 is configured to establish an overlay network between intermediate software (e.g., pad 320_1) in worker node virtual machine 310 and Pod space (e.g., Pod namespace 601) in Pod virtual machine (e.g., Pod virtual machine 301).

[0063] At box 708, worker node virtual machine 310 is configured to use an overlay network to enable containers (e.g., containers 440, 441) to be created in Pod virtual machines (e.g., Pod virtual machine 301), wherein worker node virtual machine 310 is configured to use an overlay network to manage communication with Pod virtual machines (e.g., Pod virtual machine 301).

[0064] Intermediate software (e.g., pad 320_1) is configured to generate an isolated computing environment on worker node virtual machine 310. For example, pad 320_1 is configured to launch / generate sandbox 460 as an isolated computing environment on worker node virtual machine 310.

[0065] Intermediate software (e.g., pad 320_1) is configured to create a proxy network space in an isolated computing environment on a worker node virtual machine. For example, pad 320_1 is configured to initiate / create a proxy network namespace 611 in a sandbox 460 as an isolated computing environment on a worker node virtual machine 310.

[0066] The middleware (e.g., pad 320_1) is configured to enable a logical network connection between the agent network space (e.g., agent network namespace 611) on the worker node virtual machine 310 and the Pod space (e.g., Pod namespace 601) on the Pod virtual machine (e.g., Pod virtual machine 301). The logical network (via an overlay network) can be a tunnel, such as a virtual private network using a virtual LAN, virtual Ethernet, etc., for communication between the agent network namespace 611 on the worker node virtual machine 310 and the container namespace 601 (with a running software application formed using container 440).

[0067] Internet Protocol (IP) addresses are assigned to the proxy network space (e.g., proxy network namespace 611) of worker node virtual machine 310, and intermediate software is configured to reallocate / move IP addresses to Pod spaces (e.g., Pod namespace 601). For example, pad 320_1 is configured to pull an image intended for creating a container, and pad 320_1 is configured to create / start container 462 in sandbox 460, where container 462 is subsequently paused. Container 462 is paused and is a shell container. Once pad 320_1 notifies CNI 316 that container 462 has been created / started on worker node virtual machine 310, CNI 316 (e.g., using Classless Inter-Domain Routing (CIDR)) assigns TCP / IP addresses to the paused container 462, and pad 320_1 is configured to move / assign TCP / IP addresses from proxy network namespace 611 (which may be in sandbox 460) in worker node virtual machine 310 to Pod virtual machine 301, thereby establishing a network. Therefore, the Pod namespace 601 is assigned a TCP / IP address.

[0068] In response to receiving a container intended for use in an isolated computing environment on the worker node virtual machine, the worker node virtual machine 310 is configured to transfer the container to the Pod virtual machine for association with the Pod space. For example, pad 320_1 is configured to move containers 440, 441 to container namespace 601. Intermediate software (e.g., pad 320_1) is configured to instantiate the software application in containers 440, 441 on the Pod virtual machine (e.g., Pod virtual machine 301). Therefore, the software application is configured to execute on Pod virtual machine 301, which is remote from worker node virtual machine 310.

[0069] Figure 8 This is a block diagram of an exemplary container storage interface architecture in a system 200 for a new container storage system in a remote Pod of Kubernetes according to one or more embodiments of the present invention. Figure 8Some components are omitted from the system 200 to avoid obscuring the figures, but it should be understood that system 200 includes the components and their functions as discussed herein. For simplicity, Pod virtual machine 301 is shown, but this description applies to Pod virtual machines 302, 303 and any additional Pod virtual machines.

[0070] Figure 8 An example is shown of how a persistent volume 820 in storage device 810 is attached and mounted to a remote Pod (e.g., remote Pod virtual machine 301) when the remote Pod virtual machine 301 is running in a virtual machine separate from the worker node virtual machine 310. According to one or more embodiments, unlike ephemeral storage and cloud object storage, the persistent volume 820 can be file storage, block storage, etc. Files on disk within a container are ephemeral, which can cause problems for applications running in containers. One problem is file loss when the container crashes; the kubelet restarts the container, but in a clean state. Generally, it's important to note that any volume attached to a worker node virtual machine cannot be mounted to a remote Pod virtual machine because they run in separate virtual machines. However, one or more embodiments are configured to attach and mount the persistent volume 820 to the Pod virtual machine 301 using Container Storage Interface (CSI) controller plugin 802, CSI node plugin 804, pad 320, and CSI Pod plugin 806.

[0071] The Container Storage Interface (CSI) is a specification for establishing an industry-standard interface that container orchestration systems (COs) can use to expose arbitrary storage systems to their containerized workloads. "In-tree" refers to code residing within the core Kubernetes storage repository. "Out-of-tree" refers to code residing somewhere outside the core Kubernetes storage repository. A CSI volume plugin refers to code residing somewhere outside the core Kubernetes storage repository. A CSI volume driver is an out-of-tree CSI-compatible implementation of a volume plugin that can be used in Kubernetes via the Kubernetes CSI volume plugin.

[0072] A persistent volume (PV) is a segment of storage in a cluster that has been provisioned by an administrator or dynamically provisioned using a storage class. A persistent volume is a resource in the cluster, just as a node is a cluster resource. Persistent volumes are volume plugins, but have a lifecycle independent of any individual Pod that uses them. This API object captures the details of the storage implementation, whether it's NFS, iSCSI, or a storage system from a specific cloud provider. File storage, block storage, and object storage are storage formats that maintain, organize, and present data in different ways, each with its own capabilities and limitations. File storage organizes data and represents it as a hierarchical structure of files in folders; block storage divides data into arbitrarily organized, uniformly sized volumes; object storage manages data and links it to associated metadata.

[0073] For example, CSI controller plugin 802 in the control plane of computer system 202 is configured to create a persistent volume 820 in (hardware) storage device 810 and prepare to attach the persistent volume 820 to worker node virtual machine 310. Instead of attaching the persistent volume 820 to worker node virtual machine 310, CSI controller plugin 802 is configured to invoke CSI node plugin 804 in worker node virtual machine 310. CSI node plugin 804 is configured to invoke pad 320_1, causing pad 320_1 to trigger and / or instruct CSI Pod plugin 806 to attach the persistent volume 820 to container virtual machine 301. In one or more embodiments, CSI controller plugin 802 and / or CSI node plugin 804 may attempt / request to attach the persistent volume 820 to Pod shell 464 in Pod sandbox 460 of worker node virtual machine 310, which triggers pad 320_1 to invoke CSI Pod plugin 806. Therefore, pad 320_1 causes and / or instructs CSI Pod plugin 806 to attach persistent volume 820 to Pod virtual machine 301. After attaching the persistent volume to Pod virtual machine 301, pad 320_1 is configured to cause and / or instruct CSI Pod plugin 806 to mount persistent volume 820 to Pod virtual machine 301, specifically to containers 440, 441 for use as persistent storage.

[0074] VolumeAttachment captures the intent to attach a specified volume to a specified node and / or detach a specified volume from a specified node. VolumeAttachment objects are namespace-free. The in-tree CSI volume plugin implements the following internal Kubernetes volume interfaces: 1) mounting / unmounting a volume to a specific path; and 2) attaching / unattaching a volume to a given node. For mounting and unmounting, the in-tree volume plugin's SetUp and TearDewn methods trigger NodePublishvolume and NodeUnpublishvolume CSI calls via a Unix domain socket (UDS). Therefore, Kubernetes generates a unique target_path (unique per Pod per volume) for the CSI plugin to mount the volume via NodePublishvolume. Later, upon successful completion of the NodeUnpublishvolume call (once volumeunmount has been verified), Kubernetes deletes the directory.

[0075] Figure 9 This is a block diagram illustrating further details of an architecture for installing a new container storage system in a remote Pod in Kubernetes using one or more new components in a container storage interface driver, according to one or more embodiments of the present invention. Figure 10 This is a block diagram illustrating the process of installing a new container storage system in a remote container in Kubernetes using a new container storage interface in system 200, according to one or more embodiments of the present invention. Although not explicitly stated in the original text... Figure 10 As shown, but as those skilled in the art will understand, there are different functions performed by the control plane in computer system 202 to attach and mount volumes to containers. One or more embodiments further provide a process for attaching a persistent volume 820 (e.g., file storage and / or block storage) to a remote Pod virtual machine 301 and mounting the persistent volume 820 to one or more containers 440, 441.

[0076] exist Figure 9 In order to deploy containerized (third-party) CSI volume drives, one can utilize... Figure 9 To create a CSI volume driver container that implements volume plug-in behavior and exposes a gRPC interface through Unix domain sockets (including controllers, nodes, and identity services). Figure 9This can be used to bundle CSI volume driver containers with optional helper containers (such as external attachers, external providers, node driver registrars, cluster driver registrars, external resizing devices, external snapshotters, liveness probes, etc.) that assist the CSI volume driver containers in interacting with the Kubernetes system. More specifically, Figure 9 The following Kubernetes objects can be created to facilitate communication with Kubernetes controllers, StatefulSets, or deployments with CSI volume drivers: containers provided by Kubernetes; cluster-driver-registrants; external providers (required for provisioning / deleting operations); external attachers (required for attaching / detaching operations); external resizing devices (required for resizing operations); external snapshotters (required for volume-level snapshot operations); and liveness probes. Figure 9 This includes the emptyDir volume mounted by all containers, including the CSI volume driver. The CSI volume driver container creates its Unix domain socket in this directory to enable communication with the Kubernetes helper containers. Figure 9 It can include a DaemonSet to facilitate communication with each instance of the kubelet, each instance having a CSI volume driver container, a node driver registrar responsible for registering Unix domain sockets with the kubelet, and a liveness probe.

[0077] See Figure 9 Using the control plane in computer system 202, users, who may be instances (such as applications) and / or objects, are configured to create persistent volume claim (PVC) objects on API server 204 and / or request API server 204 to create persistent volume claims. A persistent volume claim (PVC) is a user's request for storage. A persistent volume claim is similar to a Pod. Just as a Pod consumes node resources, a persistent volume claim consumes persistent volume resources. Just as a Pod can request specific levels of resources (e.g., CPU and memory), a persistent volume claim can request specific sizes and access modes (e.g., access modes can be mounted as ReadWriteOnce, ReadOnlyNow, ReadWriteNow, etc.).

[0078] While API server 204 is creating a PVC object, an external provider is configured to monitor / monitor the PVC object. Watching involves observing changes to the described resource and returning them as a stream of add, update, and remove notifications. Once the external provider finds the PVC object on API server 204, it is configured to transmit a volume create request to CSI controller plugin 802, for example, via Google® Remote Procedure Call (gRPC) through Unix Domain Sockets (UDS). CSI controller plugin 802 is configured to create the volume and notify the external provider that the volume has been created. The external provider is configured to instruct API server 204 to create a persistent volume (PV) object. The controller manager (e.g., CSI controller plugin 802) is configured to create a volume attachment object and send the created volume attachment object to API server 204; the external provider is configured to update the PVC object and bind the PVC object to the volume attachment object, making the PVC object available. An external attacher is configured to monitor / monitor volume attachment objects, and once found, the external attacher is configured to issue a controller publish volume to CSI controller plugin 802 (e.g., via gRPC through UDS). Therefore, CSI controller plugin 802 is configured to attach the volume and then notify the external attacher that the volume is attached to worker node virtual machine 310. The external attacher is configured to instruct API server 204 to update the volume attachment object. Attach / detach operations are also handled by the external component (“attacher”). The attacher, on behalf of the external CSI volume driver, monitors the Kubernetes API for new volume attachment objects and triggers appropriate calls to the CSI volume driver to attach the volume. Even if the underlying CSI driver does not support the ControllerPublishVolume call, the attacher monitors the VolumeAttachment object and marks it as attached, as Kubernetes is unaware of this. While example illustrations of actions taken at the control plane of computer system 202 have been discussed, it should be understood that, as those skilled in the art will recognize, more actions, fewer actions, and different actions in some cases may occur.

[0079] See Figure 10This will show further details regarding worker node virtual machine 310 and remote Pod virtual machine 301. In action 1002, the attach volume performed by Kubelet 312 in worker node virtual machine 310 is triggered by CSI controller plugin 802, and kubelet 312 is configured to monitor / monitor volume attach objects in API server 204 in the control plane. In action 1004, kubelet 312 is configured to notify CSI node plugin 804 to perform node-stage volume (e.g., via gRPC using UDS), and CSI node plugin 804 is configured to send a create volume Pod attachment to the control plane in action 1006. Instead of CSI node plugin 804 mounting the volume to worker node virtual machine 310, CSI node plugin 804 is configured to transfer the Pod-stage volume to pad 320_1 in action 1008, and then pad 320_1 passes a request for the Pod-stage volume to CSI Pod plugin 806 in remote Pod virtual machine 301 in action 1010. Therefore, CSI Pod plugin 806 is configured to attach persistent volume 820 to Pod virtual machine 301 upon receiving a volume request in the Pod stage, and this is a volume Pod attachment. At action 1012, CSI Pod plugin 806 is configured to send the obtained volume Pod attachment to the API server 204 in the control plane via pad 320_1 of worker node virtual machine 310. Thus, in action 1012, CSI Pod plugin 806 retrieves the volume Pod attachment object from the control plane created in action 1006. "Get" is the method for reading the status of the specified volume attachment.

[0080] exist Figure 10In action 1014, kubelet 312 is configured to instruct CSI node plugin 804 to perform node volume publishing (e.g., via gRPC using UDS), instead of CSI node plugin 804 mounting the volume to worker node virtual machine 310. In action 1016, CSI node plugin 804 is configured to transfer the created CSI Pod volume to API server 204 in the control plane on computer system 202. Specifically, in action 1016, a CSIPodvolume object is created in the control plane, which describes the volume information and will be used in action 1022. In action 1018, kubelet 312 on worker node virtual machine 310 is configured to instruct pad 320_1 to run a Pod sandbox (e.g., sandbox 460), and pad 320_1 can use the Pod sandbox (e.g., sandbox 460) as a template and / or computing environment, which may manifest as an agent and / or (temporary) location (e.g., file path) for mounting volumes on worker node virtual machine 310; instead of mounting to Pod sandbox 460, Pod shell 464, and / or paused container 462, at action 1018, pad 320_1 on worker node virtual machine 310 is configured to publish the Pod publication volume to CSI Pod plugin 806 on Pod virtual machine 301, which causes CSI Pod plugin 806 to mount persistent volume 820 to containers 440, 441 at action 1020. CSI Pod plugin 806 can mount persistent volume 820 as a CSIPod volume. In action 1022, CSI Pod plugin 806 sends the obtained CSI container volume to kubelet 312 via pad 320_1. In action 1022, CSI Pod plugin 806 retrieves the CSI Pod volume (CSIPodvolume) object created in action 1016.

[0081] Figure 11 A block diagram depicts various functions, according to one or more embodiments of the present invention, that are invoked and / or utilized to install a new container storage system in a remote container in Kubernetes using one or more new components in a container storage interface driver. CSI controller plugin 802, CSI node plugin 804, and CSI Pod plugin 806 can execute and / or invoke functions to attach and mount persistent volume 820 to containers 440, 441 in Pod virtual machines 301, 302, 303, instead of to worker node virtual machine 310. CSI controller plugin 802, CSI node plugin 804, and CSI Pod plugin 806 can be considered as CSI drivers.

[0082] Figure 12A block diagram of an Extended Container Storage Interface object, according to one or more embodiments of the present invention, is depicted for installing a new container storage system in a remote container of Kubernetes. The attachor indicates the name of the volume drive to which the request is to be handled. This is the name returned by GetPluginName(). NodeName is the node to which the volume should be attached. The source is VolumeAttachmentSource, which represents the volume to be attached; VolumeAttachmentSource can be attached via an external attachor. According to one or more embodiments, the Extended Container Storage Interface object may be used and / or implemented by CSI Controller Plugin 802, CSI Node Plugin 804, and CSI Pod Plugin 806.

[0083] Figure 13 This is a flowchart of a method 1300 implemented by a computer for attaching and mounting a persistent volume 820 to a remote Pod virtual machine rather than its worker node virtual machine, according to one or more embodiments of the present invention. Figure 13 The computer implementation method 1300 can be implemented using system 200.

[0084] At box 1302, worker node virtual machine 310 is configured such that a deterministic volume (e.g., persistent volume 820) can be attached to worker node virtual machine 310. For example, kubelet 312 of worker node virtual machine 310 is configured to monitor / monitor and discover volume attachment objects (e.g., ...). Figure 10 Action 1002 in the middle acts as a request to attach persistent volume 820 to worker node virtual machine 310.

[0085] At box 1304, worker node virtual machine 310 is configured to use middleware (e.g., pad 320) of worker node virtual machine 310 to cause Pod container storage interface (e.g., CSI Pod plugin 806) to attach a volume to Pod virtual machine 301. For example, pad 320 of worker node virtual machine 310 is configured to send a request to attach a volume (e.g., a request for a Pod stage volume (e.g., action 1010)) to CSI Pod plugin 806, which causes CSI Pod plugin 806 to attach persistent volume 820 to Pod virtual machine 301.

[0086] At box 1306, worker node virtual machine 310 is configured to, in response to attaching a volume to pod virtual machine 301, use middleware (e.g., pad 320) of the worker node virtual machine to enable the pod container storage interface (e.g., CSIPod plugin 806) to mount a volume (e.g., persistent volume 820) onto pod virtual machine 301, thus making the volume available to pod virtual machine 301. For example, pad 320 is configured to feed a request / instruction, such as publishing a volume using a pod (e.g., action 1020), and CSIPod plugin 806 to mount persistent volume 820 onto pod virtual machine 301. Further, CSI Pod plugin 806 can mount persistent volume 820 into containers 440 and 441. After using pod sandbox 460 as a proxy for a mount request from kubelet 312, pad 320 can send a request / instruction to mount persistent volume 820.

[0087] The volume is persistent volume 820. Persistent volume 820 includes file storage. Persistent volume 820 includes block storage. Worker node virtual machine 310 is configured to receive and / or generate a request to mount the volume (e.g., at action 1004), and instead of attaching the volume to worker node virtual machine 310, the request triggers intermediate software (e.g., pad 320) to attach the volume to Pod virtual machine 301 (e.g., actions 1008, 1010).

[0088] According to one or more embodiments, worker node virtual machine 310 is configured to receive and / or generate a request to mount a volume (e.g., at action 1014), and instead of mounting the volume to worker node virtual machine 310, the request triggers intermediate software (e.g., pad 320) to mount the volume to Pod virtual machine 301 (e.g., actions 1018, 1020). The intermediate software (e.g., pad 320) is configured to use an isolated computing environment (e.g., Pod sandbox 460) on worker node virtual machine 310 as a proxy for mounting the volume on Pod virtual machine 301.

[0089] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0090] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing power, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0091] The features are as follows:

[0092] On-demand self-service: Cloud consumers can automatically and unilaterally configure computing power, such as server time and network storage, as needed, without human interaction with service providers.

[0093] Extensive network access: Capabilities are available through networks and accessed via standard mechanisms that facilitate the use of heterogeneous thin client platforms or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0094] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is a sense of location independence because consumers typically do not have control or knowledge of the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0095] Rapid elasticity: Capacity can be provided quickly and flexibly (in some cases, automatically) to shrink rapidly and expand rapidly. For consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time.

[0096] Measurable services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0097] The service model is as follows:

[0098] Software as a Service (SaaS): This provides consumers with the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from different client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0099] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created by the consumer or acquired using programming languages ​​and tools supported by the provider onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environment.

[0100] Infrastructure as a Service (IaaS): This provides consumers with the capability to deliver processing, storage, networking, and other basic computing resources that enable them to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather have control over the operating system, storage, deployed applications, and potentially limited control over selected networking components (e.g., host firewalls).

[0101] The deployment model is as follows:

[0102] Private cloud: Cloud infrastructure intended for organization operations only. It can be managed by the organization or a third party and can exist on-site or off-site.

[0103] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0104] Public cloud: Makes cloud infrastructure available to the general public or large industry groups and is owned by an organization that sells cloud services.

[0105] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported (e.g., cloud bursting for load balancing between clouds).

[0106] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure comprising a network of interconnected nodes.

[0107] See now Figure 14 The diagram illustrates an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 to which local computing devices used by cloud consumers can communicate. These local computing devices include, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped in one or more networks (not shown), such as private clouds, community clouds, public clouds, or hybrid clouds (as described above), or combinations thereof. This allows the cloud computing environment 50 to provide infrastructure, platforms, and / or software as services that cloud consumers do not need to maintain on their local computing devices. It should be understood that... Figure 14The types of computing devices 54A-N shown are intended to be illustrative only, and computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0108] See now Figure 15 This demonstrates the 50 (cloud computing environment) Figure 14 This provides a set of functional abstractions. It should be understood beforehand that... Figure 15 The components, layers, and functions shown are merely illustrative, and embodiments of the invention are not limited thereto. As described, the following layers and corresponding functions are provided:

[0109] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a mainframe 61; a RISC (Reduced Instruction Set Computer) based server 62; a server 63; a blade server 64; a storage device 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0110] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0111] In one example, management layer 80 may provide the following functionalities: Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 84 provides cloud resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 85 provides pre-scheduling and procurement of cloud resources based on anticipated future needs according to the SLA.

[0112] Workload layer 90 provides examples of functions that can be utilized in a cloud computing environment. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis and processing 94; transaction processing 95; and software applications implemented in workloads and functions 96 (e.g., software applications in worker node virtual machines 310, software applications in Pod virtual machines 301, 302, 303, etc.).

[0113] This document describes various embodiments of the invention with reference to the accompanying drawings. Alternative embodiments of the invention may be devised without departing from its scope. Various connections and positional relationships (e.g., above, below, adjacent, etc.) between elements are illustrated in the following description and drawings. Unless otherwise specified, these connections and / or positional relationships may be direct or indirect, and the aspects of the invention herein are limiting. Therefore, the connection of entities may refer to direct or indirect connections, and the positional relationship between entities may be direct or indirect positional relationships. Furthermore, the various tasks and process steps described herein may be incorporated into a more comprehensive procedure or process with additional steps or functions not described in detail herein.

[0114] One or more methods described herein can be implemented using any one or a combination of the following techniques well known in the art: discrete logic circuit(s) having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0115] For the sake of brevity, this document may or may not describe in detail conventional techniques associated with the manufacture and use of aspects of the invention. Specifically, various aspects of computing systems and specific computer programs used to implement the different technical features described herein are well known. Consequently, for the sake of brevity, many conventional implementation details are only briefly mentioned or omitted entirely herein, without providing well-known system and / or process details.

[0116] In some embodiments, different functions or actions may occur at a given location and / or in combination with the operation of one or more devices or systems. In some embodiments, a portion of a given function or action may be performed at a first device or location, and the remainder of the function or action may be performed at one or more additional devices or locations.

[0117] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. It should also be understood that when the terms “comprises” and / or “comprising” are used in this specification, they specify the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or combinations thereof.

[0118] All means or steps in the following claims, plus the corresponding structures, materials, actions, and equivalents of the functional elements, are intended to include any structure, material, or action for performing the function in combination with other claimed elements as specifically claimed. This disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. These embodiments were chosen and described in order to best explain the principles and practical application of this disclosure, and to enable others skilled in the art to understand this disclosure with respect to different embodiments having different modifications suitable for the particular intended use.

[0119] The figures described herein are illustrative. Many variations may be made to the figures or steps (or operations) described herein without departing from the spirit of this disclosure. For example, actions may be performed in a different order, or actions may be added, deleted, or modified. Furthermore, the term "coupled" describes a signal path between two elements and does not imply a direct connection between elements without intermediate elements / connections. All such variations are considered part of this disclosure.

[0120] The following definitions and abbreviations will be used to interpret the claims and specification. As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having,” “contains,” or “containing,” or any other variations thereof, are intended to cover non-exclusive inclusion. For example, a composition, mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or apparatus.

[0121] Furthermore, the term "exemplary" is used herein to mean "serving as an example, illustration, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments or designs. The terms "at least one" and "one or more" should be understood to include any integer greater than or equal to one, i.e., one, two, three, four, etc. The term "multiple" should be understood to include any integer greater than or equal to two, i.e., two, three, four, five, etc. The term "connection" can include both indirect "connection" and direct "connection."

[0122] The terms “about,” “substantially,” “approximately,” and their variations are intended to include the degree of error associated with a measurement based on a specific quantity of equipment available at the time of application submission. For example, “about” could include a range of ±8%, 5%, or 2% of a given value.

[0123] This invention can be a system, method, and / or computer program product with any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0124] Computer-readable storage media can be tangible devices capable of retaining and storing instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0125] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the corresponding computing / processing device.

[0126] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry in order to perform aspects of this invention.

[0127] This document describes various aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0128] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, wherein the computer-readable storage medium storing the instructions comprises an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0129] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, thereby causing the instructions to be executed on the computer, other programmable apparatus, or other device to perform the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than indicated in the figures. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0131] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements to technologies found in the market, or to enable those skilled in the art to understand the embodiments described herein.

Claims

1. A computer-implemented method, comprising: The worker node virtual machine determines that the volume can be attached to the worker node virtual machine; The middleware of the worker node virtual machine is used to attach the volume to the Pod container storage interface; and In response to attaching the volume to the Pod virtual machine, the intermediate software of the worker node virtual machine is used to enable the Pod container storage interface to mount the volume to the Pod virtual machine, making the volume available to the Pod virtual machine, wherein the intermediate software is configured to use an isolated computing environment on the worker node virtual machine as a proxy for mounting the volume on the Pod virtual machine.

2. The computer-implemented method according to claim 1, wherein, The volume is a persistent volume.

3. The computer-implemented method according to claim 1, wherein, The volume includes file storage.

4. The computer-implemented method according to claim 1, wherein, The volume includes block storage.

5. The computer-implemented method according to claim 1, wherein: The worker node virtual machine is configured to receive requests to attach the volume; and The request triggers the intermediate software to attach the volume to the Pod virtual machine instead of attaching the volume to the worker node virtual machine.

6. The computer-implemented method according to claim 1, wherein: The worker node virtual machine is configured to receive requests to mount the volume; and The request triggers the intermediate software to mount the volume to the Pod virtual machine, instead of mounting the volume to the worker node virtual machine.

7. A computer system, comprising: Memory, containing computer-readable instructions; as well as One or more processors are configured to execute the computer-readable instructions, which control the one or more processors to perform the computer-implemented method of any one of claims 1-6.

8. A computer program product comprising computer program instructions executable by one or more processors to cause the one or more processors to perform the computer-implemented method of any one of claims 1-6.

Images