Snapshotter adaptation apparatus and method

Abstract

Provided are a snapshotter adaptation apparatus and method. The apparatus comprises: a snapshotter feature library, a listening service module, and a containerd client; the snapshotter feature library comprises snapshotters mapped to different manifest types; the listening service module is configured to acquire, in response to an image pull request from kubelet, a manifest file corresponding to a target container image to be pulled, and determine, on the basis of the snapshotter feature library and the manifest type of the manifest file, a target snapshotter adapted to the target container image; and the containerd client is configured to forward the image pull request to containerd, and specify the target snapshotter, so as to pull the target container image by means of the containerd.

Description

A snapshotter adapter and method

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411782570.0, filed on December 5, 2024, entitled “A snapshotter adapter and method”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of container image services, specifically to a snapshotter adapter and method. Background Technology

[0004] In the development of container technology, Docker (containers), as one of the earliest containerization platforms, introduced technologies such as namespaces and control groups, enabling applications to run in isolated environments. With the emergence of Kubernetes (K8S, a container orchestration engine), container orchestration and management have become more efficient and flexible. Kubernetes was initially tightly coupled with Docker, but as container runtimes diversified, Kubernetes gradually shifted to using containerd (Container Daemon) as its container runtime. As a high-performance container management tool, containerd provides a better pluggable architecture and storage driver support, with snapshotter being a crucial component responsible for managing container file system snapshots.

[0005] Although containerd supports multiple snapshotters, in Kubernetes, when kubelet calls containerd through CRI (Container Runtime Interface), it can only globally specify one snapshotter. This limits the flexibility of using different snapshotters for different images, especially when some snapshotters are incompatible with specific images, leading to wasted resources and management complexity. Summary of the Invention

[0006] This disclosure provides a snapshotter adapter device and method, which aims to solve the problems existing in the background art.

[0007] To solve the above-mentioned technical problems, this disclosure is implemented as follows:

[0008] In a first aspect, embodiments of this disclosure provide a snapshotter adapter device, the device including: a snapshotter signature library, a listening service module, and a containerd client;

[0009] The snapshotter feature library includes snapshotters mapped to different manifest (image manifest) types;

[0010] The listening service module is used to respond to image pull requests from kubelet (Kubernetes node agent), obtain the manifest file corresponding to the target container image to be pulled, and determine the target snapshotter that is compatible with the target container image based on the manifest type of the manifest file according to the snapshotter feature library.

[0011] The containerd client is used to forward image pull requests to containerd and specify the target snapshotter to pull the target container image through containerd.

[0012] Optionally, the snapshotter feature library has a pre-defined first mapping relationship between known manifest types and corresponding snapshotters, and the first mapping relationship is loaded into the snapshotter feature library through internal encoding.

[0013] Optionally, the snapshotter feature library has a second mapping relationship in addition to the first mapping relationship. The second mapping relationship is defined and loaded into the snapshotter feature library through environment variables or command line parameters.

[0014] Optionally, the listening service module is also used to determine the attribute information of the target container image based on the image pull request, and to determine the target snapshotter that is compatible with the target container image based on the snapshotter feature library and the attribute information of the target container image; the attribute information includes at least: the namespace to which the target container image belongs, the image repository to which the target container image belongs, the tag suffix of the target container image, and the tag prefix of the target container image.

[0015] Optionally, the listening service module includes an unknown service processor;

[0016] The listening service module is also used to respond to other interface requests from other containerd clients, and to forward other interface requests to containerd through an unknown service handler.

[0017] Optionally, the listening service module is used to record the original containerd socket file and move the original containerd socket file to a new location; and create a new socket file with the same filename as the original containerd socket file.

[0018] The listening service module is used to listen for image pull requests from kubelet and other interface requests from other containerd clients via a new socket file.

[0019] The listening service module is used to respond to image pull requests from kubelet and other interface requests from other containerd clients. It sends image pull requests to containerd through the existing containerd socket file.

[0020] Optionally, the listening service module is used to create a symbolic link pointing to the original containerd socket file based on the original containerd socket file.

[0021] Symbolic links are used to forward requests from kubelet or other containerd clients to the existing containerd socket file when the snapshotter adapter stops working.

[0022] Optionally, the device also includes a verification module;

[0023] The verification module is used to write the mapping relationship between the first verification image and the corresponding snapshotter into the snapshotter feature library. The first verification image is an accelerated version of the container image.

[0024] The verification module is also used to create and run a second verification image and check if any new content has been generated in the root directory where the first verification image is located. The second verification image is a normal version of the container image.

[0025] If no new content is generated in the root directory where the first verification image is located, create and run the first verification image, and check whether the first verification image runs successfully.

[0026] If the first verification image is found to be running successfully, the snapshotter adapter is confirmed to have passed verification.

[0027] Secondly, this disclosure provides a snapshotter adaptation method, applied to a snapshotter adaptation device, the method comprising:

[0028] In response to an image pull request from kubelet, retrieve the manifest file corresponding to the target container image to be pulled;

[0029] Based on the snapshotter feature library, the target snapshotter that is compatible with the target container image is determined according to the manifest type of the manifest file.

[0030] Forward the image pull request to containerd and specify the target snapshotter to pull the target container image through containerd.

[0031] Optionally, based on the snapshotter signature library, the target snapshotter adapted to the target container image is determined according to the manifest type of the manifest file, including:

[0032] Parse the manifest file to determine its manifest type;

[0033] Search the snapshotter feature library for mappings that match the manifest type;

[0034] If a mapping relationship matching the manifest type is found, the snapshotter corresponding to that mapping relationship is determined as the target snapshotter.

[0035] If no mapping is found that matches the manifest type, the default specified snapshotter will be used as the target snapshotter.

[0036] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects:

[0037] This disclosure introduces a snapshotter signature library and a listening service module to achieve dynamic adaptation to different manifest types, significantly improving the flexibility and efficiency of container image pulling. It can intelligently select the most suitable snapshotter based on the specific manifest type of the image to be pulled, thus avoiding compatibility issues caused by globally specifying snapshotters. Through the adaptation mechanism provided by this disclosure, users can fully leverage the advantages of various snapshotters in the Kubernetes environment, thereby improving resource utilization and overall system performance. This not only increases the success rate of image pulling but also reduces errors caused by incompatibility, lowering operational costs. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 is a schematic diagram of a snapshotter adapter architecture provided in an embodiment of this disclosure;

[0040] Figure 2 is a schematic diagram of the pre-configuration process of a snapshotter adapter device in one embodiment of the present disclosure;

[0041] Figure 3 is a schematic diagram of the steps of a snapshotter adaptation method provided in an embodiment of this disclosure;

[0042] Figure 4 is a schematic diagram of the image retrieval process in one embodiment of this disclosure. Detailed Implementation

[0043] Kubernetes, as a mainstream container orchestration platform, is widely used in various enterprise applications. However, with the continuous evolution of container technology, Kubernetes faces a series of compatibility and flexibility issues when handling different types of container images. The traditional containerd snapshotter mechanism requires specifying a single snapshotter in the global configuration. This approach often leads to incompatibility in environments with multiple images and snapshotters, limiting user choice and optimal resource utilization. To address this deficiency, this disclosure aims to introduce a snapshotter feature library and a listening service module to achieve dynamic adaptation to different manifest types, intelligently identify the manifest type of the image to be pulled, and select the most suitable snapshotter based on its characteristics, thereby effectively solving the compatibility problems caused by globally specifying a snapshotter.

[0044] Figure 1 is a schematic diagram of a snapshotter adapter device architecture provided in an embodiment of this disclosure. As shown in Figure 1, the device includes: a snapshotter signature library, a listening service module, and a containerd client.

[0045] The snapshotter feature library includes snapshotters mapped to different manifest types.

[0046] The snapshotter feature library is the core data storage module of a snapshotter adapter provided in this disclosure. Its main function is to maintain the mapping relationship between different manifest types and corresponding snapshotters. The manifest file is the metadata of the image, containing the image's hierarchical structure, content hash value, and mediaType describing the image type. Each manifest type represents an image with a specific format or characteristics, and the manifest type can essentially characterize the image's features. The snapshotter is the tool or plugin used to process these images. The snapshotter feature library allows users to flexibly select the appropriate snapshotter in different use cases, thereby avoiding compatibility issues caused by globally specifying the snapshotter.

[0047] The listening service module is used to respond to image pull requests from kubelet, obtain the manifest file corresponding to the target container image to be pulled, and determine the target snapshotter that is compatible with the target container image based on the manifest type of the manifest file according to the snapshotter feature library.

[0048] The listening service module is responsible for monitoring and responding to image pull requests from kubelet. When kubelet initiates an image pull request, the listening service module first obtains the manifest file corresponding to the target container image to be pulled. Then, using the mapping relationship in the snapshotter signature library, it analyzes the type of the manifest file and determines the target snapshotter that is most suitable for the target container image, ensuring that the best snapshotter is selected for each image pull based on the specific circumstances.

[0049] The containerd client is used to forward image pull requests to containerd and specify the target snapshotter to pull the target container image through containerd.

[0050] The containerd client acts as a bridge between this device and containerd, responsible for forwarding processed image pull requests to containerd. When forwarding a request, the containerd client specifies the previously determined target snapshotter so that containerd uses the correct snapshotter to pull the target container image.

[0051] Referring to Figure 1, when the kubelet in the Kubernetes cluster needs to pull a container image, it initiates an image pull request to the device provided in this embodiment. The image pull request is sent to the listening service module of this device.

[0052] Upon receiving an image pull request, the listening service module sends a request to the image service to obtain the manifest file of the target container image. The manifest file contains detailed information about the image, including its hierarchical structure, content hash value, and required snapshotter.

[0053] When the monitoring service module obtains the manifest file, it passes it to the snapshotter signature library. At this point, the snapshotter signature library uses the information in the manifest file to determine the manifest type, that is, to identify the manifest type corresponding to the target container image. It then searches its internal mapping for the target snapshotter of that type.

[0054] After determining the target snapshotter, the listening service module returns this information and specifies the target snapshotter as the target for subsequent pull operations. Next, the listening service module constructs a complete image pull request and forwards it to containerd via the built-in containerd client. At this point, the containerd client uses the previously determined target snapshotter as a parameter to ensure that containerd uses the correct snapshotter when performing the pull operation. Finally, upon receiving the request, containerd begins pulling the target container image based on the specified target snapshotter.

[0055] It's important to note that in Kubernetes, CRI (Container Runtime Interface) is the standardized interface between the container runtime and Kubernetes, responsible for image management, container lifecycle management, and other functions. ImageService is the core component of CRI, used for pulling and managing images. This disclosure dynamically adapts to different types of images based on the functionality provided in the device and adds necessary special parameters for specific image types. Specifically, in the PullImage method, a request is first sent to the image repository through the OCI (Open Container Initiative) standard v2 interface to pull the manifest file of the target container image. Assuming the target container image address is registry.example.com / my-image:latest, its manifest file can be pulled through the OCI v2 interface (Docker Registry HTTP API V2, the second version of the Docker repository HTTP interface):

[0056] GET https: / / registry.example.com / v2 / my-image / manifests / latest

[0057] For example, the returned manifest file can contain the following:

[0058] After obtaining the manifest file of the target container image, the manifest type (i.e., the value of the mediaType field) that characterizes the image is extracted. Then, the manifest type is compared with the mapping relationship in the snapshotter feature library to determine which snapshotter to use to process the image. For example, if the value of mediaType in the manifest file is application / vnd.oci.image.nydus.v1+json, and the mapping relationship is defined in the mapping library as follows:

[0059] {

[0060] "application / vnd.oci.image.nydus.v1+json":"nydus-snapshotter"

[0061] }

[0062] Then nydus-snapshotter (Nydus snapshot manager) is matched as the target snapshotter for processing the target container image.

[0063] The actual image pull task is performed by calling the `PullImage` method of `containerd` through the `containerd` client inside this device. The pull request needs to include additional parameters required for specific image types, which can be obtained from the `containerd` configuration file ` / etc / containerd / config.toml`.

[0064] After constructing the image pull request, the request is sent to the containerd client, specifying the adapted snapshotter and related parameters. The containerd Pull API (Pull Application Interface) is then invoked. Based on the specified snapshotter and parameters, containerd interacts with the Nydus (High-Performance Container Image Acceleration Framework) backend to complete the image pull and decompression. Through this process, the device implements dynamic adaptation of the PullImage method in CRI's ImageService interface. Based on the image type and the mapping relationship of the feature library, a suitable snapshotter is selected and the necessary special parameters are added.

[0065] This disclosure introduces a snapshotter signature library and a listening service module to achieve dynamic adaptation to different manifest types, significantly improving the flexibility and efficiency of container image pulling. It can intelligently select the most suitable snapshotter based on the specific manifest type of the image to be pulled, thus avoiding compatibility issues caused by globally specifying snapshotters. Through the adaptation mechanism provided by this disclosure, users can fully leverage the advantages of various snapshotters in the Kubernetes environment, thereby improving resource utilization and overall system performance. This not only increases the success rate of image pulling but also reduces errors caused by incompatibility, lowering operational costs.

[0066] In one optional implementation, the snapshotter feature library has a pre-defined first mapping relationship between known manifest types and corresponding snapshotters, and the first mapping relationship is loaded into the snapshotter feature library through internal encoding.

[0067] Figure 2 is a schematic diagram of the pre-configuration process of a snapshotter adapter device according to an embodiment of this disclosure. Referring to Figure 2, the step of loading the mapping relationship into the snapshotter feature library is one of the pre-configuration processes after the device is initialized. The first mapping relationship refers to the mapping between the preset, known image manifest types and their corresponding snapshotters in the snapshotter feature library. Through the configured mapping relationship, the snapshotter feature library can quickly find and select a suitable snapshotter when it receives an image pull request.

[0068] The initial mapping relationships are stored in a structured form within the code of the Snapshotter feature library, such as using dictionaries, hash tables, or other data structures; that is, they are loaded into the Snapshotter feature library through internal encoding. This allows the Snapshotter feature library to efficiently search and match at runtime. For example, the mapping relationships can be represented using key-value pairs:

[0069] snapshotter_mapping = {

[0070] "application / vnd.docker.distribution.manifest.v2+json":"overlayfs-snapshotter",

[0071] "application / vnd.oci.image.manifest.v1+json":"nydus-snapshotter",

[0072] "application / vnd.oci.image.index.v1+json":"squashfs-snapshotter"

[0073] In this example, the snapshotter_mapping dictionary defines three known manifest types and their corresponding snapshotters.

[0074] In one optional implementation, the snapshotter feature library has a second mapping relationship in addition to the first mapping relationship. The second mapping relationship is defined and loaded into the snapshotter feature library through environment variables or command line parameters.

[0075] In addition to the preset first mapping relationship, the snapshotter feature library also supports defining and loading a second mapping relationship through environment variables or command line parameters, which aims to improve the flexibility of the system and enable users to dynamically adjust the snapshotter mapping relationship according to specific needs.

[0076] The second mapping refers to the mapping between image manifest types and snapshotters that users can customize through environment variables or command-line arguments. The second mapping allows users to adjust the snapshotter selection at runtime to suit different image requirements or specific runtime environments. Users can define the second mapping by setting environment variables or passing command-line arguments when starting the program. For example, a user can set the following environment variables when starting a Kubernetes cluster or related services:

[0077] SNAPSHOTTER_MAPPING="application / vnd.custom.image.manifest.v1+json=custom-snapshotter"

[0078] In this example, the manifest type of application / vnd.custom.image.manifest.v1+json is mapped to a snapshotter named custom-snapshotter.

[0079] Users can also define a second mapping relationship via command-line arguments. For example, when starting the service, the following command can be used:

[0080] In the command `. / my_service --snapshotter-mapping "application / vnd.custom.image.manifest.v1+json=custom-snapshotter"`, the `--snapshotter-mapping` parameter is used to pass the mapping relationship, parse it, and load it into the snapshotter feature library, resulting in the second mapping relationship entered in the command line argument.

[0081] It's important to note that the second mapping relationship takes precedence over the first. This means that when both the first and second mapping relationships exist in the snapshotter feature library, the monitoring service module will prioritize using the second mapping relationship defined by the user through environment variables or command-line arguments. Users can override the default behavior through simple environment variable settings or command-line arguments, thus achieving personalized configuration.

[0082] By supporting the loading of second mapping relationships, the Snapshotter feature library can flexibly adapt to different user needs and operating environments. This not only enhances the overall adaptability of the device but also allows users to optimize the image retrieval process according to actual conditions, thereby improving overall efficiency and performance.

[0083] In this embodiment, to facilitate Kubernetes (via kubelet) management of node disk resources, especially when assessing disk pressure or triggering container image garbage collection (GC), the root directory path of the snapshotter needs to be explicitly specified. This device uniformly specifies the root directories of all snapshotters covered by the snapshotter feature library to the default root directory of the snapshotter overlayfs. The Kubernetes kubelet periodically checks node disk pressure and assesses storage space usage. When disk usage exceeds a threshold, the kubelet triggers container image garbage collection (GC) via CRI using containerd. By unifying the snapshotter root directory path, the kubelet can directly access and calculate all snapshot data stored in the overlayfs root directory, simplifying the logic of disk usage analysis. The core of image garbage collection lies in removing unused images and snapshotters. With a unified path, the GC logic can quickly locate all snapshot data and determine its status without having to query or process scattered storage paths individually.

[0084] In one optional implementation, the listening service module is further configured to determine the attribute information of the target container image based on the image pull request, and determine the target snapshotter that is compatible with the target container image based on the snapshotter feature library and the attribute information of the target container image; the attribute information includes at least: the namespace to which the target container image belongs, the image repository to which the target container image belongs, the tag suffix of the target container image, and the tag prefix of the target container image.

[0085] When the listening service module receives an image pull request from kubelet, it parses the request, extracts the attribute information of the target container image, and determines which snapshotter to use to handle the image pull based on the attribute information.

[0086] When selecting a suitable snapshotter, the listening service module considers one or more of the following attributes: the namespace to which the target container image belongs, the image repository to which the target container image belongs, the tag suffix of the target container image, and the tag prefix of the target container image.

[0087] A namespace is a logical grouping identifier for a container image, used to distinguish different projects or applications. For example, `myrepo` in the image `myrepo / myapp:latest` is the namespace. Namespaces help the system identify the source and purpose of an image. An image repository is the specific location where container images are stored, typically a URL (Uniform Resource Locator) or path to a registry. For example, `docker.io` is the image repository for Docker Hub. An image tag is used to identify the image version or variant. The tag suffix is ​​the part separated by a colon after the image name; for example, in `myrepo / myapp:latest`, `latest` is the tag suffix. Different tags may correspond to different snapshotters, especially when using specific build or optimization strategies. In some cases, the image tag prefix also contains useful information; for example, `v1.0.0` in `myrepo / myapp:v1.0.0-alpha` can be considered a tag prefix.

[0088] After extracting the attribute information of the target container image, the monitoring service module matches this attribute information against the mapping relationships in the snapshotter signature library. Specifically, based on the extracted attribute information, the monitoring service module queries the first and second mapping relationships in the snapshotter signature library to find snapshotters that match these attributes. If a matching snapshotter is found in the signature library, the monitoring service module designates it as the target snapshotter. If no match is found, a default snapshotter (such as overlayfs (Overlay File System)) can be selected to handle the request.

[0089] In one alternative implementation, the listening service module includes an unknown service processor; the listening service module is also used to respond to other interface requests from other containerd clients and forward the other interface requests to containerd through the unknown service processor.

[0090] Please refer to Figure 1. In addition to image pull requests from kubelet, there are also other interface requests from other containerd clients. In Figure 1, solid arrows indicate the processing path for image pull requests to kubelet, while dashed arrows indicate the processing path for other interface requests to other containerd clients.

[0091] Unknown service handlers are used to intercept additional interface requests from other containerd clients and forward these requests to the containerd service. Specifically, when the listening service module receives a request from another containerd client (such as nerdctl (containerd CTL, a Docker-compatible command-line tool designed specifically for containerd) or ctr (containerd CLI, containerd's native command-line tool)), the unknown service handler first identifies the type of the request. If the request is not an image pull request but another type of request (e.g., querying container status, managing container lifecycles), it is considered an unknown service request. For these unknown service requests, the unknown service handler forwards them to the upstream containerd service. This means that the unknown service handler is responsible for passing requests from the listening service module to containerd, enabling containerd to process these requests and return the corresponding results. By using unknown service handlers, the listening service module can remain transparent to external clients. Clients do not need to be aware of the existence of the listening service module. They can send requests as if they were accessing containerd normally, and the listening service module will process these requests in the background to ensure that the requests are forwarded and responded to correctly.

[0092] The following is a simplified workflow example illustrating how the listening service module and the unknown service handler work together. When a containerd client (such as ctr) sends a request to query the status of currently running containers, the listening service module receives this request and recognizes that it is not an image pull request but an unknown service request. The listening service module passes the request to the unknown service handler. The unknown service handler is responsible for forwarding the request to the upstream containerd service. After receiving the request, containerd performs the corresponding operation (such as querying the container status) and returns the result to the unknown service handler. The unknown service handler returns the containerd response to the original requesting client (such as ctr), completing the entire request processing.

[0093] In one alternative implementation, the listening service module is used to record the original containerd socket file and move the original containerd socket file to a new location; and create a new socket file with the same filename as the original containerd socket file.

[0094] Please refer to Figure 2. During device initialization, the listening service module identifies and records the location of the currently used containerd socket file. The original containerd socket file is the main interface for containerd to communicate with external clients (such as kubelet and other container management tools). The listening service module obtains the path of the original socket file and saves it for use in subsequent operations.

[0095] To avoid conflicts between newly created socket files and existing socket files, the listening service module moves the existing socket files to a new location. Optionally, it moves the existing containerd.sock file (the containerd socket file, the local communication interface of the container runtime service) to a backup directory or renames it to ensure it is no longer used, while retaining its contents for future reference. Unix Socket (Unix Domain Socket) is a file system-based inter-process communication (IPC) mechanism. Compared to network sockets, Unix sockets offer higher performance and are suitable for low-latency, local communication scenarios. The socket files disclosed herein belong to the Unix Socket category.

[0096] To maintain compatibility with external clients, the listening service module creates a new socket file with the same name as the existing socket file (e.g., still containerd.sock). This means a new socket file with the same name is created in the same location as the original socket file. This way, external clients can still use the same filename when sending requests.

[0097] The listening service module is used to listen for image pull requests from kubelet and other interface requests from other containerd clients via a new socket file.

[0098] The new socket file will be used to listen for and receive image pull requests from kubelet. The listening service module sets up a listener through the new socket file to wait for image pull requests from kubelet, as well as other interface requests from other containerd clients.

[0099] The listening service module is used to respond to image pull requests from kubelet and other interface requests from other containerd clients. It sends image pull requests to containerd through the existing containerd socket file.

[0100] When the listening service module receives an image pull request from kubelet, it forwards the request to the containerd service to perform the actual image pull operation. The listening service module sends the image pull request to containerd using the original containerd socket file (which has been moved to a new location). containerd processes the request and returns the result. Similarly, the listening service module forwards requests from other containerd clients to the containerd service.

[0101] In one optional implementation, the listening service module is used to establish a soft link pointing to the original containerd socket file based on the original containerd socket file.

[0102] Symbolic links are used to forward requests from kubelet or other containerd clients to the existing containerd socket file when the snapshotter adapter stops working.

[0103] Considering that the snapshotter adapter may stop operating under certain circumstances due to maintenance, malfunction, or other reasons, requests from kubelet or other clients will not be processed, potentially leading to system failure or service interruption. Therefore, during the adapter's operation, the listening service module can use the operating system's symbolic link functionality to create a new symbolic link pointing to the existing containerd socket file. This symbolic link can be named the same as the original socket file for client convenience.

[0104] When this device stops running, all requests from kubelet or other containerd clients will be forwarded to the original containerd socket file via this symbolic link. Even if the adapter no longer processes requests, it can still respond to external requests normally. Because a symbolic link is created, external clients (such as kubelet and other containerd clients) do not need to know the existence or status of the snapshotter adapter; they can continue to communicate through the same socket file.

[0105] In one alternative implementation, the apparatus further includes a verification module;

[0106] The verification module is used to write the mapping relationship between the first verification image and the corresponding snapshotter into the snapshotter feature library. The first verification image is an accelerated version of the container image.

[0107] The verification module is also used to create and run a second verification image and check if any new content has been generated in the root directory where the first verification image is located. The second verification image is a normal version of the container image.

[0108] If no new content is generated in the root directory where the first verification image is located, create and run the first verification image, and check whether the first verification image runs successfully.

[0109] If the first verification image is found to be running successfully, the snapshotter adapter is confirmed to have passed verification.

[0110] The verification module verifies the functionality of the snapshotter adapter, ensuring it can correctly handle image pull requests. The module verifies the mapping relationship between different container image versions (accelerated and normal versions) and monitors their running status. The verification module records the mapping relationship between the first verified image (the accelerated version of the container image) and its corresponding snapshotter in the snapshotter signature library.

[0111] Before verifying this device, a verification environment needs to be set up to support the functionality of the verification module. A specific example is provided below, requiring a virtual machine configured with 4 CPU (Central Processing Unit) cores (4C) and 8GB of memory (8G). For the operating system, a Linux operating system with kernel version 4.19 or higher is required. Additionally, a Kubernetes cluster with version 1.23 or higher needs to be deployed on the virtual machine.

[0112] Two test images need to be prepared: nginx:latest and nginx:latest-nydus. nginx:latest is the first verification image for the normal version, and nginx:latest-nydus is the second verification image for the accelerated version, which uses nydus technology.

[0113] To start nydus-snapshotter as a system service, use the following command:

[0114] containerd-nydus-grpc

[0115] nydus-snapshotter can run as a service and interact with containerd. Locate the containerd configuration file, usually located at ` / etc / containerd / config.toml`. Add the following lines to the configuration file to add nydus as a proxy plugin:

[0116] After modifying the configuration, you need to restart the containerd service for the changes to take effect. After restarting, execute the following command to verify that nydus snapshotter has been successfully imported:

[0117] ctr plugins ls|grep nydus

[0118] If the output contains the word "ok", it means that nydus snapshotter has been successfully imported.

[0119] Start this device and, upon completion, check the directory / run / containerd to confirm the existence of the following two Unix socket files: containerd.sock and containerd-real.sock. If these two socket files exist, it indicates that the device has successfully started and is running normally.

[0120] After the verification environment is set up, the first step is for the verification module to start the second verification image and check if any new content has been generated in the root directory of the first verification image. This step is to verify whether the pull of the normal version image is using the default snapshotter, because the snapshotter signature library does not define a mapping relationship for the second verification image. If new content is generated in the root directory of the first verification image, it means that the normal version image is using the default snapshotter.

[0121] In the second step, the verification module creates and runs the first verification image and checks its running status. If the first verification image runs successfully, it means that the accelerated version image has found a matching snapshotter based on the mapping relationship defined in the snapshotter feature library, and the verification module will determine that the snapshotter adapter device has passed verification. Conversely, if the first verification image fails to run, it means that the accelerated version image has failed to achieve snapshotter adaptation based on this device, and the verification module will determine that the snapshotter adapter device has failed verification.

[0122] Figure 3 is a schematic diagram of the steps of a snapshotter adapter method provided in an embodiment of this disclosure. As shown in Figure 3, the method is applied to a snapshotter adapter device, and the method includes:

[0123] Step S101: In response to the image pull request from kubelet, obtain the manifest file corresponding to the target container image to be pulled.

[0124] When a kubelet needs to pull a container image, it sends an image pull request. In response to this request, the kubelet first identifies the name and version of the target container image to be pulled. Then, it requests the image's manifest file from the image repository via the OCI standard v2 interface. The manifest file contains the image's metadata, including layer information, configuration, etc., to characterize the target container image or its corresponding manifest type.

[0125] A mirror pull request can be sent to the mirror repository via an HTTP GET (HyperText Transfer Protocol GET, to retrieve a specified resource from the server) request, and the mirror repository will return the corresponding manifest file.

[0126] Step S102: Based on the snapshotter feature library, determine the target snapshotter that is compatible with the target container image according to the manifest type of the manifest file.

[0127] Parse the manifest file obtained from the image repository and extract the manifest type (e.g., application / vnd.docker.distribution.manifest.v2+json).

[0128] Based on the extracted manifest type, a matching snapshotter is searched in the feature library. If a matching snapshotter is found, it is recorded as the target snapshotter for subsequent image pull requests. If no matching snapshotter is found, a default snapshotter (such as overlayfs) is selected.

[0129] Step S103: Forward the image pull request to containerd and specify the target snapshotter to pull the target container image through containerd.

[0130] The obtained target snapshotter information is forwarded to containerd along with the image pull request. During forwarding, the selected target snapshotter is specified in the request so that containerd can use that snapshotter to pull the image. After receiving the request, containerd will perform the image pull operation according to the specified snapshotter. Specifically, containerd calls the relevant interfaces of the target snapshotter and uses the characteristics of the target snapshotter to handle the image pull process.

[0131] Figure 4 is a schematic diagram of the image pull process in one embodiment of this disclosure. Referring to Figure 4, when a container is created or started, Kubernetes sends an image pull request to containerd. Upon receiving the image pull request, the manifest file of the target container image is obtained. After obtaining the manifest file, the manifest type representing the image characteristics is compared with a preset snapshotter feature library to find a snapshotter that matches the image type. Once a snapshotter matching the image characteristics is found, it is determined as the target snapshotter. If no match is found, a default snapshotter (e.g., overlayfs) is selected. After determining the target snapshotter, the containerd client will use the target snapshotter to pull the target container image. Specifically, the containerd client constructs a pull request and specifies the target snapshotter in the request. By calling the relevant containerd API, the client sends the request to containerd, and containerd will perform the pull operation for the target container image according to the specified snapshotter. The target snapshotter will be responsible for processing the image layers, downloading and storing them in the local file system, ensuring that the image can be used correctly.

[0132] In one optional implementation, based on a snapshotter signature library, the target snapshotter adapted to the target container image is determined according to the manifest type of the manifest file, including:

[0133] Step S201: Parse the manifest file and determine the manifest type to which the manifest file belongs.

[0134] The manifest file is parsed to extract key information. Specifically, the mediaType field in the manifest file is located, as this field indicates the type of the manifest.

[0135] By parsing, the specific type of the manifest file can be determined; for example, it might be...

[0136] application / vnd.docker.distribution.manifest.v2+json、

[0137] application / vnd.oci.image.manifest.v1+json, etc.

[0138] Step S202: Search the snapshotter feature library for mapping relationships that match the manifest type.

[0139] As mentioned earlier, the snapshotter feature library contains the mapping relationship between different manifest types and snapshotters. The snapshotter feature library can be a static mapping table or a dynamically loaded configuration file.

[0140] Using the parsed manifest type as the query condition, the system searches the snapshotter feature library for a matching mapping. If a matching mapping is found, the corresponding snapshotter is recorded.

[0141] Step S203: If a mapping relationship matching the manifest type is found, the snapshotter corresponding to the mapping relationship is determined as the target snapshotter.

[0142] If a mapping relationship matching the manifest type is found in the feature library, the target snapshotter is determined based on the mapping relationship, and the specified target snapshotter will be used to handle the image pull request.

[0143] Save the target snapshotter's information for use in subsequent image pull requests.

[0144] Step S204: If no mapping relationship matching the manifest type is found, the default specified snapshotter is determined as the target snapshotter.

[0145] If no mapping relationship matching the manifest type is found in the feature library, an alternative approach will be adopted. In this case, the default snapshotter will be used as the target snapshotter. The default snapshotter can be a general snapshotter; in this embodiment, the default snapshotter can be an overlayfs, suitable for most image pull requirements. The information of the default snapshotter will be recorded for use in subsequent requests.

[0146] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, apparatus, electronic devices, and storage media. Therefore, embodiments of this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this disclosure can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM (Compact Disc Read-Only Memory), optical storage, etc.) containing computer-usable program code.

[0147] This disclosure describes embodiments of methods and apparatus according to embodiments of this disclosure with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0148] While preferred embodiments of the present disclosure have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the present disclosure.

[0149] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element. The above provides a detailed description of a snapshotter adapter apparatus and method provided by this disclosure. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas; at the same time, for those skilled in the art, based on the ideas of this disclosure, there will be changes in specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A snapshotter adapter device, characterized by The device includes: a snapshotter signature library, a listening service module, and a containerd client; The snapshotter feature library includes snapshotters mapped to different manifest types; The listening service module is used to respond to image pull requests from kubelet, obtain the manifest file corresponding to the target container image to be pulled, and determine the target snapshotter that is compatible with the target container image based on the manifest type of the manifest file according to the snapshotter feature library. The containerd client is used to forward the image pull request to containerd and specify the target snapshotter so as to pull the target container image through containerd.

2. The apparatus of claim 1, wherein, The snapshotter feature library has a pre-defined first mapping relationship between known manifest types and corresponding snapshotters. This first mapping relationship is loaded into the snapshotter feature library through internal encoding.

3. The apparatus of claim 2, wherein, The snapshotter feature library has a second mapping relationship in addition to the first mapping relationship. The second mapping relationship is defined and loaded into the snapshotter feature library through environment variables or command line parameters.

4. The apparatus of claim 1, wherein, The monitoring service module is also used to determine the attribute information of the target container image based on the image pull request, and to determine the target snapshotter that is compatible with the target container image based on the snapshotter feature library and the attribute information of the target container image. The attribute information includes at least: the namespace to which the target container image belongs, the image repository to which the target container image belongs, the tag suffix of the target container image, and the tag prefix of the target container image.

5. The apparatus of claim 1, wherein, The monitoring service module includes an unknown service processor; The listening service module is also used to respond to other interface requests from other containerd clients and forward the other interface requests to the containerd client through the unknown service processor.

6. The apparatus of claim 5, wherein, The listening service module is used to record the original containerd socket file and move the original containerd socket file to a new location; and create a new socket file with the same filename as the original containerd socket file. The listening service module is used to listen for image pull requests from kubelet and other interface requests from other containerd clients through the new socket file. The listening service module is used to respond to image pull requests from kubelet and other interface requests from other containerd clients, and to send the image pull requests to the containerd client through the existing containerd socket file.

7. The apparatus of claim 6, wherein, The listening service module is used to establish a soft link pointing to the original containerd socket file based on the original containerd socket file. The soft link is used to forward requests from kubelet or other containerd clients to the original containerd socket file when the snapshotter adapter stops running.

8. The apparatus of claim 1, wherein, The device also includes a verification module; The verification module is used to write the mapping relationship between the first verification image and the corresponding snapshotter into the snapshotter feature library, wherein the first verification image is an accelerated version of the container image. The verification module is also used to create and run a second verification image and check whether new content is generated in the root directory where the first verification image is located. The second verification image is a normal version of the container image. If no new content is generated in the root directory where the first verification image is located, create and run the first verification image, and check whether the first verification image runs successfully. If the first verification image is found to be running successfully, the snapshotter adapter is determined to have passed verification.

9. A snapshotter adaptation method, characterized by, Applied to the snapshotter adapter as described in any one of claims 1-8, the method comprises: In response to an image pull request from kubelet, retrieve the manifest file corresponding to the target container image to be pulled; Based on the snapshotter feature library, a target snapshotter that is compatible with the target container image is determined according to the manifest type of the manifest file. The image pull request is forwarded to containerd, and the target snapshotter is specified so that the target container image can be pulled through containerd.

10. The method according to claim 9, characterized in that, The step of determining the target snapshotter that is compatible with the target container image based on the snapshotter feature library and the manifest type of the manifest file includes: Parse the manifest file to determine the manifest type to which the manifest file belongs; Search the snapshotter feature library for a mapping relationship that matches the manifest type; If a mapping relationship matching the manifest type is found, the snapshotter corresponding to the mapping relationship is determined as the target snapshotter. If no mapping relationship matching the manifest type is found, the default specified snapshotter will be determined as the target snapshotter.

Images