A method and device for running and managing unikernels based on libvirt and containerd

Abstract

The present invention discloses a method for running and managing unikernels based on libvirt and containerd: parsing the upper-layer configuration file and creating a unikernel structure, saving status information and files, starting the daemon process and pausing it; restoring the unikernel structure and the daemon process from the status file, converting the configuration file into an XML configuration file to request libvirt to start the unikernel process; the daemon process monitors the unikernel process until it exits; terminating the unikernel process, and the daemon process performs cleanup work; deleting unikernel-related information, as well as status information and files. This method and device enable unikernels to be seamlessly integrated into containerd and further integrated into Kubernetes, significantly improving the isolation and security of containers.

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud computing, and particularly relates to a method and device for running and managing unikernels based on libvirt and containerd. Background Art

[0002] With the rapid development of modern computing technology, cloud services and cloud computing have become indispensable. To deploy and run applications more effectively, securely, and efficiently, container technology has emerged and gradually become a popular solution in the current industry. Container technology allows developers to bundle application programs with their running environments, simplifies the deployment process, enhances migration flexibility, and optimizes resource utilization efficiency.

[0003] Among them, kubernetes, with its powerful capabilities, as an open-source container orchestration platform, helps developers and operation and maintenance personnel manage application programs gracefully in various environments. It brings unparalleled convenience to distributed systems and opens the door to convenient large-scale application deployment, expansion, and management. At the same time, the scalability of kubernetes enables it to be seamlessly integrated with other systems and platforms. For example, Chinese Patent with publication number CN116301943A discloses a method for building images of a kubernetes cluster based on containerd containers, including the following steps: obtaining a base image by inheriting the image information of podman containers, building application images of at least one programming language based on the base image; preparing a first configuration file of podman, the first configuration file including several image repositories and the storage locations of each image repository; preparing a second configuration file of at least one programming language, the second configuration file including the configuration environments relied on by at least one programming language; mounting the directories specified by the first configuration file and the second configuration file to a host directory or a distributed shared storage; writing an operation and maintenance feature configuration file for the application, the operation and maintenance feature configuration file including setting the basic operation and maintenance features of the application; performing the CI process and CD process of the application in the application deployment pipeline to achieve deployment in the kubernetes cluster.

[0004] However, not everything is perfect. Although container technology has excellent performance in multiple dimensions, it has inherent defects in security and isolation. This is mainly because container technology relies on operating system-level virtualization, which may become vulnerable in the face of various complex and evolving security threats.

[0005] To improve the performance in these two aspects, the industry has turned to unikernel. Unikernel is a lightweight operating system designed specifically for a single application. By eliminating unnecessary parts and tightly integrating the application with OS services, it ensures maximum performance and security. This gives unikernel significant advantages in terms of security, efficiency, and startup time.

[0006] Unikraft is a building and running framework designed for unikernel, which further simplifies the development and deployment of unikernel and provides developers with a set of friendly APIs and tool sets. However, it is still a problem to combine unikernel with mainstream container technologies, especially with containerd and kubernetes.

[0007] Currently, solutions such as Kata Containers attempt to solve this problem by using lightweight virtual machines to improve the isolation and security of containers. For example, Chinese Patent No. CN110569111A discloses a method for implementing a virtual machine based on a traditional container, including: when a startup request of a container is detected, starting the virtual machine required for the container using Kata technology; creating and running the container based on the virtual machine. In the above implementation method, when a startup request of a container is detected, the virtual machine of the container is started using Kata technology, and the container runs in the virtual machine. During the startup and running process of the container, it depends on the resources in its corresponding virtual machine and does not need to share the hardware resources and operating system in the host, reducing the resource consumption of the host. Further, the containers are isolated by different virtual machines, ensuring the security and isolation between containers.

[0008] However, how to perfectly integrate unikernel into this ecosystem remains a problem. In addition, although there are tools such as qemu-guest available for running and managing unikernel, they often only provide limited functions and cannot meet more diverse and complex application scenarios.

[0009] Libvirt is an excellent open-source tool library that provides a unified interface for various virtualization technologies, making the management and operation of resources such as virtual machines, storage, and networks more convenient and supporting numerous virtualization solutions.

[0010] In summary, the existing technologies in this field currently mainly have the following disadvantages:

[0011] (1) Although container technology has shown obvious advantages in the efficiency of deployment and migration and the elastic expansion of services, its security and isolation capabilities are relatively weak when facing complex and continuously evolving security risks.

[0012] (2) Although solutions such as Kata Containers have strengthened the security and isolation features of containers, the problem of integrating unikernel remains unresolved, preventing the full utilization of the high security and lightweight features of unikernel.

[0013] (3) Currently, the existing management and optimization methods for container runtimes show certain deficiencies when applying VMfunc technology, limiting the full exploration of the potential of VMfunc in communication acceleration.

[0014] (4) Although qemu-guest can be directly used to run and manage unikernel, it provides few functions and cannot meet complex application requirements. Summary of the Invention

[0015] The purpose of the present invention is to provide a method and device for running and managing unikernel based on libvirt and containerd, enabling unikernel to be seamlessly integrated into containerd and further incorporated into kubernetes, significantly enhancing the isolation and security of containers.

[0016] The present invention provides the following technical solutions:

[0017] A method for running and managing unikernel based on libvirt and containerd, the method comprising the following steps:

[0018] (1) Parse the upper-level configuration file and create a unikernel structure, save the status information and files, start the daemon process and pause it;

[0019] (2) Restore the unikernel structure and the daemon process from the status file, convert the configuration file into an XML configuration file to request libvirt to start the unikernel process;

[0020] (3) The daemon process monitors the unikernel process until it exits;

[0021] (4) Terminate the unikernel process, and the daemon process performs cleanup work;

[0022] (5) Delete the unikernel-related information, as well as the status information and files.

[0023] In step (1), obtain the running information of the unikernel through the configuration file, including: startup command, resource limits, network settings, file system, mounts, and namespaces, etc.; the configuration file includes: the ID of the unikernel and the type information of the unikernel, etc.

[0024] Specifically, obtain the key running information through a JSON configuration file designed specifically for unikernels.

[0025] In step (1), the daemon process is responsible for the startup of the unikernel and the configuration of various environments, specifically including:

[0026] (1-1) Configure the unikernel network environment to ensure its access to the CNI plugin network namespace provided by Kubernetes;

[0027] (1-2) Verify the integrity and format of the unikernel file system, and handle the relevant mount items at the same time;

[0028] (1-3) Set the unikernel startup parameters, including the kernel image file, file mounts, network configuration, and resource limits;

[0029] (1-4) Activate the daemon process, and send a command to it to pause the unikernel startup process. Subsequently, the PID of the daemon process and the unikernel status are persistently saved.

[0030] In step (2), the specific method is as follows:

[0031] (2-1) Recover the corresponding unikernel structure from the status file of the specified unikernel ID;

[0032] (2-2) Send a command to the daemon process to resume running;

[0033] (2-3) Convert the configuration file into an XML configuration file that libvirt can accept;

[0034] (2-4) Request the libvirt daemon to start the corresponding unikernel according to the configuration.

[0035] Convert the configuration file into an XML configuration file that libvirt can accept, so as to prepare for subsequent communication with libvirt

[0036] In step (4), call different libvirt termination methods to terminate the unikernel process.

[0037] In step (4), the daemon process will be responsible for cleaning up all resources required for the unikernel process to run, including network, storage, and computing resources. Specifically, the following operations need to be performed:

[0038] (4-1) Stop the subprocesses related to the unikernel;

[0039] (4-2) Send a signal to containerd indicating that the subprocesses have been stopped;

[0040] (4-3) Handle the logout logic of the unikernel service;

[0041] (4-4) Clean up the running environment after the end of the unikernel process lifecycle. After completion of the cleanup, the daemon process will report the end status of the unikernel process to containerd.

[0042] The method includes initializing the virtualization environment of the unikernel: using the APIs provided by libvirt, initialize the virtualization environment of the unikernel by sending an XML configuration file, including setting resources such as CPU, memory, and network; trigger the start of the unikernel process by calling the APIs of libvirt, and the unikernel process will start running in the initialized virtualization environment.

[0043] The method includes creating a containerd container, including a unikernel process and a daemon process, both of which communicate with the libvirt background process to ensure the normal operation and management of the unikernel process.

[0044] In containerd, each running unit is a container. Therefore, the present invention creates a new container to host the unikernel process. This container contains / integrates a unikernel process and a daemon process. Among them, the daemon process is responsible for communicating with the libvirt background process to ensure the normal operation and management of the unikernel process.

[0045] The method includes pausing and resuming a running unikernel, creating a snapshot of a running unikernel, or restoring a snapshot.

[0046] The present invention also provides a unikernel operation and management device based on libvirt and containerd, including a memory and one or more processors. An executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the above-mentioned unikernel operation and management method based on libvirt and containerd.

[0047] The present invention also provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it is used to implement the above-mentioned unikernel operation and management method based on libvirt and containerd.

[0048] Compared with the prior art, the present invention has the following remarkable advantages:

[0049] (1) By integrating the lightweight unikernel into containerd and Kubernetes, the method provided by the present invention not only inherits the inherent advantages of unikernel, such as high isolation and security, but also ensures that unikernel can better adapt to modern containerization and microservices architectures, improving the robustness and efficiency of overall deployment.

[0050] (2) In the method provided by the present invention, a specific daemon process can manage and maintain the life cycle of unikernel, and at the same time ensure that unikernel can perfectly integrate into the Kubernetes environment, thereby improving the response speed and stability of container services.

[0051] (3) The method provided by the present invention is based on libvirt, providing a brand-new and efficient runtime environment for unikernel, thus further improving the resource utilization rate and performance advantages.

[0052] Generally speaking, the present invention provides a more efficient, secure and robust operation method for modern containerized applications, thus meeting the growing performance and security requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of a unikernel operation and management method based on libvirt and containerd provided by the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0055] In a specific embodiment, the method provided by the present invention enables unikraft to implement a CRI runtime called runlibvirt, which can handle the lifecycle of unikernels and allows them to be deployed in a kubernetes environment through the containerd method.

[0056] Runlibvirt is a container runtime client based on runc and developed in golang. Its main function is to encapsulate the functions of libvirt and runc to manage operations such as the creation, deletion, pause, and resume of unikernels. By improving the container runtime runc and combining it with libvirt-go, runlibvirt is created.

[0057] The specific running method includes the following steps:

[0058] Step 1, Creation: Parse the upper-layer configuration file and create a unikernel structure, save the status information and files, start the daemon process and pause

[0059] Step 1-1, runlibvirt create will parse the upper-layer JSON configuration file and create a unikernel structure containing all information, and save it to a status file; obtain the running information of the unikernel through the configuration file, including: startup command, resource limits, network settings, file system, mounts, and namespaces, etc.; the configuration file includes: the ID of the unikernel and the type information of the unikernel, etc.

[0060] Step 1-2, runlibvirt starts the runlibvirt-init daemon process, which is responsible for the startup of the unikernel and the configuration of various environments.

[0061] Specifically, this process includes:

[0062] Step 1-2-1, Configure the unikernel network environment to ensure that it accesses the CNI plugin network namespace provided by kubernetes.

[0063] Step 1-2-2: Verify the integrity and format of the unikernel file system, and process relevant mount items.

[0064] Step 1-2-3: Set unikernel startup parameters, such as the kernel image file, file mounts, network configuration, and resource limits.

[0065] Step 1-2-4: runlibvirt activates the runlibvirt-init daemon process via the cmd startup command and sends it a SIGSTOP command to pause the unikernel startup process; subsequently, the PID of this daemon process and the unikernel status are persistently saved.

[0066] Step 2: Startup: Restore the unikernel structure and the daemon process from the status file, convert the configuration file into an XML configuration file to request libvirt to start the unikernel process

[0067] Step 2-1: When runlibvirt start is executed, it will restore the status of the specified unikernel ID. Specifically, runlibvirt will restore the corresponding structure from the status file of the specified unikernel ID;

[0068] Step 2-2: Subsequently, send a SIGCONT command to the daemon process to resume its operation;

[0069] Step 2-3: The daemon process will convert it into an XML configuration file that can be accepted by libvirt according to the saved status information;

[0070] Step 2-4: Request the libvirt daemon to start the corresponding unikernel according to the configuration, and in this way, the real unikernel process is successfully started.

[0071] Before startup, it is also necessary to interact with libvirt through runlibvirt to initialize the virtualization environment of the unikernel and reserve resources for its subsequent operation. The specific method is as follows: By using runlibvirt, which is a modified version of runc, we can interact with libvirt through it. Using the API provided by libvirt, we can initialize the virtualization environment of the unikernel by sending an XML configuration file, including setting resources such as CPU, memory, and network. By calling the API of libvirt, the startup of the unikernel process is triggered, and the unikernel process will start running in the initialized virtualization environment. Then, through runlibvirt, the API of libvirt is called to trigger the startup of the unikernel process. At this time, the unikernel process will start running in the previously initialized virtualization environment.

[0072] Step 3, Monitoring: The daemon process monitors the unikernel process until it exits

[0073] After the unikernel starts, its daemon process monitors the unikernel process. Whether it exits abnormally or ends normally, the daemon process will trigger subsequent operations. runlibvirt can use the ListDomains method of libvirt-go to implement the list unikernel function to query all active unikernel instances.

[0074] By interacting with libvirt through runlibvirt, the lifecycle of the unikernel process is managed, such as pausing, resuming, saving the state, etc. Specifically, during the running process, the lifecycle of the unikernel can be managed through various APIs provided by libvirt (such as pausing, resuming, saving the state, etc.). These management functions are completed by the cooperation of runlibvirt and the libvirt background process.

[0075] In this embodiment, runlibvirt pause and runlibvirt resume can be used to pause and resume the running unikernel respectively. These two commands are implemented by calling the libvirt API, enabling the unikernel to be paused and resumed during the running process, improving resource utilization and system response speed.

[0076] In this embodiment, runlibvirt snapshot can be used to create or restore a snapshot of a running unikernel. This command is implemented by calling the libvirt API, which can create a snapshot of the unikernel at any time and restore the running state from the snapshot when needed, improving the fault tolerance and reliability of the system.

[0077] Step 4, Termination: Call different libvirt termination methods to terminate the unikernel process, and the daemon process performs cleanup work

[0078] The specific process of Step 4-1, calling different libvirt termination methods to terminate the unikernel process is as follows: Using the runlibvirt kill command can achieve the orderly termination of the unikernel and handle relevant information before and after the process exits. Specifically, this command will determine which method of libvirt-go to call by parsing the semaphore provided on the command line. For example, for SIGTERM, the Shutdown command will be executed, and for SIGKILL, the Destroy command will be executed.

[0079] Step 4-2, The daemon process will be responsible for cleaning up all resources required for the unikernel process to run, including network, storage, and computing resources. The following specific operations need to be performed:

[0080] Step 4-2-1, Stop the subprocesses related to the unikernel;

[0081] Step 4-2-2, Send a signal that the subprocess has been stopped to containerd / send a SIGCHLD signal to containerd-shim;

[0082] Step 4-2-3, Handle the logout logic of the unikernel service;

[0083] Step 4-2-4, Clean up the running environment after the lifecycle of the unikernel process ends. After the cleanup is completed, the daemon process will report the end status of the unikernel process to containerd.

[0084] Step 5, Deletion: Delete unikernel-related information, status information, and files

[0085] Through the runlibvirt delete command, the specified unikernel can be completely removed, including its status file. This process mainly relies on the Undefine method of libvirt-go to delete the configuration and metadata of the unikernel.

[0086] Through the five key steps of leveraging runlibvirt, the above embodiments completely manage the lifecycle of unikernel and ensure its seamless integration with containerd and Kubernetes.

[0087] In summary, the above running and management method provided by this embodiment enables unikernel to be seamlessly integrated into containerd and further integrated into Kubernetes, thereby not only significantly improving the isolation and security of containers, but also endowing them with more functions such as pause, resume, and snapshot.

[0088] The embodiment of the present invention also provides a unikernel running and management device based on libvirt and containerd, including one or more processors. There is executable code stored in the memory. When the processor executes the executable code, it is used to implement the unikernel running and management method based on libvirt and containerd in the above embodiments. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor of any device with data processing capabilities reading the corresponding computer program instructions in the non-volatile memory into the memory and running. In terms of hardware, in addition to the processor, memory, network interface, and non-volatile memory, any device with data processing capabilities where the device in the embodiment is located usually also includes other hardware according to the actual functions of the any device with data processing capabilities, which will not be elaborated here.

[0089] The embodiment of the present invention also provides a computer-readable storage medium, on which there is a program. When the program is executed by the processor, it implements the unikernel running and management method based on libvirt and containerd in the above embodiments: The computer-readable storage medium can be the internal storage unit of any device with data processing capabilities described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any device with data processing capabilities, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a Flash8 Card, etc. equipped on the device. Further, the computer-readable storage medium can also include both the internal storage unit of any device with data processing capabilities and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store the data that has been output or will be output.

[0090] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A method for running and managing unikernels based on libvirt and containerd, characterized in that, The method includes the following steps: (1) Parse the upper-layer configuration file and create a unikernel structure, save the status information and files, start the daemon process and pause it; (2) Restore the unikernel structure and the daemon process from the status file, convert the configuration file into an XML configuration file to request libvirt to start the unikernel process; (3) The daemon process monitors the unikernel process until it exits; (4) Terminate the unikernel process, and the daemon process performs cleanup work; (5) Delete the unikernel-related information, as well as the status information and files; In step (1), the daemon process is responsible for the startup of unikernel and the configuration of various environments, specifically including: (1-1) Configure the unikernel network environment to ensure its access to the CNI plugin network namespace provided by kubernetes; (1-2) Verify the integrity and format of the unikernel file system, and handle the relevant mount items at the same time; (1-3) Set the unikernel startup parameters, including the kernel image file, file mounts, network configuration, and resource limits; (1-4) Activate the daemon process, and send a command to it to pause the unikernel startup process. Subsequently, the PID of the daemon process and the unikernel status are persistently saved; In step (4), the daemon process will be responsible for cleaning up all resources required during the operation of the unikernel process, including network, storage, and computing resources. Specifically, the following operations need to be performed: (4-1) Stop the subprocesses related to unikernel; (4-2) Send a signal that the subprocess has been stopped to containerd; (4-3) Handle the logout logic of the unikernel service; (4-4) Clean up the running environment after the end of the unikernel process life cycle. After the cleanup is completed, the daemon process will report the end status of the unikernel process to containerd.

2. The unikernel running and management method based on libvirt and containerd according to claim 1, wherein In step (1), obtain the running information of unikernel through the configuration file, including: startup command, resource limits, network settings, file system, mounts, and namespaces; the configuration file includes: the ID of unikernel and the type information of unikernel.

3. The unikernel running and management method based on libvirt and containerd according to claim 1, characterized in that, In step (2), the specific method is as follows: (2-1) Restore the corresponding unikernel structure from the status file with the specified unikernel ID; (2-2) Send a command to the daemon process to resume running; (2-3) Convert the configuration file into an XML configuration file that libvirt can accept; (2-4) Request the libvirt daemon to start the corresponding unikernel according to the configuration.

4. The unikernel running and management method based on libvirt and containerd according to claim 1, characterized in that, The method includes initializing the virtualization environment of the unikernel: Using the API provided by libvirt, initialize the virtualization environment of the unikernel by sending an XML configuration file, including setting the CPU, memory, and network; By calling the API of libvirt, trigger the start of the unikernel process, and the unikernel process will start running in the initialized virtualization environment.

5. The unikernel running and management method based on libvirt and containerd according to claim 1, characterized in that The method includes creating a containerd container, and the containerd container includes a unikernel process and a daemon process, both of which communicate with the libvirt background process to ensure the normal operation and management of the unikernel process.

6. The unikernel running and management method based on libvirt and containerd according to claim 1, characterized in that The method includes pausing and resuming a running unikernel, creating a snapshot of a running unikernel, or restoring a snapshot.

7. A unikernel running and management device based on libvirt and containerd, including a memory and one or more processors, and executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the unikernel running and management method based on libvirt and containerd according to any one of claims 1-6.

8. A computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it is used to implement the unikernel running and management method based on libvirt and containerd according to any one of claims 1-6.

Images