Snapshot adaptation device and method

Abstract

The invention provides a snapshotter adaptation device and method, and relates to the field of container mirror image services, and the device comprises a snapshotter feature library, a monitoring service module and a containerd client. The snapshotter feature library comprises a snapshotter which is mapped to different manifest types, and the snapshotter feature library comprises a snapshotter which is mapped to different manifest types; the monitoring service module is used for responding to a mirror image pulling request from the kubelet, obtaining a mannest file corresponding to a target container mirror image to be pulled, and determining a target snapshotter matched with the target container mirror image according to a mannest type of the mannest file on the basis of a snapshotter feature library; and the containerd client is used for forwarding the mirror image pulling request to the containerd, and specifying a target snapshotter so as to pull the mirror image of the target container through the containerd. According to the method and the device, the compatibility problem caused by global designation of the snapshotter is avoided, and errors caused by incompatibility are reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of container image services, and in particular to a snapshotter adaptation device and method. Background Art

[0002] In the development of container technology, Docker, as one of the earliest containerization platforms, introduced technologies such as namespaces and control groups, allowing applications to run in isolated environments. With the emergence of Kubernetes (K8S), container orchestration and management have become more efficient and flexible. Kubernetes was initially tightly coupled with Docker, but with the diversification of container runtimes, Kubernetes gradually turned to using containerd as its container runtime. As a high-performance container management tool, containerd provides better plug-in architecture and storage driver support, of which snapshotter is an important component responsible for managing container file system snapshots.

[0003] Although containerd supports multiple snapshotters, in Kubernetes, kubelet can only specify one snapshotter globally when calling containerd through CRI. This limits the flexibility of using different snapshotters for different images, especially when some snapshotters are incompatible with specific images, resulting in resource waste and management complexity. Summary of the invention

[0004] The embodiments of the present disclosure provide a snapshotter adaptation device and method, aiming to solve the problems existing in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the present disclosure is implemented as follows: In a first aspect, an embodiment of the present disclosure provides a snapshotter adaptation device, the device comprising: a snapshotter feature library, a monitoring service module and a containerd client; The snapshotter feature library includes snapshotters mapped to different manifest types; The monitoring service module is used to respond to the image pull request from kubelet, obtain the manifest file corresponding to the target container image to be pulled, and determine the target snapshotter adapted to the target container image based on the snapshotter feature library and the manifest type of the manifest file; The containerd client is used to forward the image pull request to containerd and specify the target snapshotter to pull the target container image through the containerd.

[0006] Optionally, the snapshotter feature library is preset with a first mapping relationship between a known manifest type and a corresponding snapshotter, and the first mapping relationship is loaded into the snapshotter feature library by means of internal coding.

[0007] Optionally, the snapshotter feature library is preset with a second mapping relationship in addition to the first mapping relationship, and the second mapping relationship is defined and loaded into the snapshotter feature library in the form of an environment variable or a command line parameter.

[0008] Optionally, the listening service module is also used to determine the attribute information of the target container image according to the image pull request, and based on the snapshotter feature library, determine the target snapshotter that is compatible with the target container image according to the attribute information of the target container image; the attribute information includes at least: the namespace to which the target container image belongs, the image repository to which the target container image belongs, the tag suffix of the target container image, and the tag prefix of the target container image.

[0009] Optionally, the monitoring service module includes an unknown service processor; The monitoring service module is also used to respond to other interface requests from other containerd clients, and forward the other interface requests to the containerd through the unknown service processor.

[0010] Optionally, the monitoring service module is used to record an original containerd socket file and transfer the original containerd socket file to a new location; create a new socket file with the same file name as the original containerd socket file; The monitoring service module is used to monitor the image pull request from kubelet and other interface requests from other containerd clients through the new socket file; The monitoring service module is used to respond to image pull requests from kubelet and other interface requests from other containerd clients, and send the image pull request to the containerd through the original containerd socket file.

[0011] Optionally, the monitoring service module is used to establish a soft link pointing to the original containerd socket file according to the original containerd socket file; The soft link is used to forward requests from kubelet or other containerd clients to the original containerd socket file when the snapshotter adapter stops running.

[0012] Optionally, the device further comprises a verification module; The verification module is used to write a mapping relationship between a first verification image and a corresponding snapshotter in the snapshotter feature library, wherein the first verification image is an accelerated version of a container image; The verification module is further used to create and run a second verification image, and check whether new content is generated in the root directory where the first verification image is located, and the second verification image is a common version of the container image; In the case where no new content is generated in the root directory where the first verification image is located, creating and running the first verification image, and checking whether the first verification image runs successfully; In the case where it is checked that the first verification image runs successfully, it is determined that the snapshotter adapter device passes the verification.

[0013] In a second aspect, an embodiment of the present disclosure provides a snapshotter adaptation method, which is applied to a snapshotter adaptation device, and the method includes: In response to the image pull request from kubelet, obtain the manifest file corresponding to the target container image to be pulled; Based on the snapshotter feature library, determine a target snapshotter that is compatible with the target container image according to the manifest type of the manifest file; The image pull request is forwarded to containerd, and the target snapshotter is specified to pull the target container image through the containerd.

[0014] Optionally, the determining, based on the snapshotter feature library and according to the manifest type of the manifest file, a target snapshotter that is compatible with the target container image includes: Parsing the manifest file to determine the manifest type to which the manifest file belongs; Searching the snapshotter feature library for a mapping relationship that matches the manifest type; When a mapping relationship matching the manifest type is found, the snapshotter corresponding to the mapping relationship is determined as the target snapshotter; When no mapping relationship matching the manifest type is found, the default snapshotter is determined as the target snapshotter.

[0015] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: The present disclosure realizes dynamic adaptation of different manifest types by introducing snapshotter feature library and monitoring service module, which significantly improves the flexibility and efficiency of container image pulling. It can intelligently select the most suitable snapshotter according to the specific manifest type of the image to be pulled, thereby avoiding compatibility issues caused by global specification of snapshotter. Through the adapter device provided by the present disclosure, users can make full use of the advantages of various snapshotters in the Kubernetes environment, thereby improving resource utilization and overall system performance. It not only improves the success rate of image pulling, but also reduces errors caused by incompatibility and reduces operation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 is a schematic diagram of a snapshotter adapter architecture provided by an embodiment of the present disclosure; Figure 2 It is a schematic diagram of a pre-configuration process of a snapshotter adapter device in one embodiment of the present disclosure; Figure 3 It is a schematic diagram of the steps of a snapshotter adaptation method provided by an embodiment of the present disclosure; Figure 4 It is a schematic diagram of the process of image pulling in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0019] As a mainstream container orchestration platform, Kubernetes is widely used in various enterprise-level applications. However, with the continuous evolution of container technology, Kubernetes faces a series of compatibility and flexibility issues when dealing with different types of container images. The traditional containerd snapshotter mechanism requires a single snapshotter to be specified in the global configuration. This practice often leads to incompatibility in an environment where multiple images and snapshotters coexist, limiting user choices and optimal utilization of resources. To address this shortcoming, the present disclosure aims to achieve dynamic adaptation of different manifest types by introducing a snapshotter feature library and a monitoring service module, intelligently identify the manifest type of the image to be pulled, and select the most suitable snapshotter according to its characteristics, thereby effectively solving the compatibility issues caused by globally specifying the snapshotter.

[0020] Figure 1 is a schematic diagram of a snapshotter adapter architecture provided by an embodiment of the present disclosure, such as Figure 1 As shown, the device includes: a snapshotter feature library, a monitoring service module and a containerd client.

[0021] The snapshotter feature library includes snapshotters mapped to different manifest types.

[0022] The snapshotter feature library is a core data storage module of a snapshotter adapter device provided by the present disclosure. Its main function is to maintain the mapping relationship between different manifest types and corresponding snapshotters. The manifest file is the metadata of the image, which contains the hierarchical structure of the image, the content hash value, and the mediaType describing the image type. Each manifest type represents an image of a specific format or feature. The manifest type is equivalent to the feature that can characterize the image. The snapshotter is a tool or plug-in for processing these images. The snapshotter feature library allows users to flexibly select the appropriate snapshotter in different usage scenarios, thereby avoiding compatibility issues caused by globally specifying the snapshotter.

[0023] The monitoring service module is used to respond to the image pull request from kubelet, obtain the manifest file corresponding to the target container image to be pulled, and determine the target snapshotter adapted to the target container image based on the snapshotter feature library and the manifest type of the manifest file.

[0024] The monitoring service module is responsible for monitoring and responding to image pull requests from kubelet. When kubelet initiates an image pull request, the monitoring service module first obtains the manifest file corresponding to the target container image to be pulled. Then, using the mapping relationship in the snapshotter feature library, the type of the manifest file is analyzed, and the target snapshotter that best matches the target container image is determined, ensuring that the best snapshotter can be selected according to the specific situation each time the image is pulled.

[0025] The containerd client is used to forward the image pull request to containerd and specify the target snapshotter to pull the target container image through the containerd.

[0026] The containerd client is a bridge between this device and containerd, and is responsible for forwarding the processed image pull request to containerd. When forwarding the request, the containerd client specifies the previously determined target snapshotter so that containerd uses the correct snapshotter to pull the target container image.

[0027] See also Figure 1When the kubelet in the Kubernetes cluster needs to pull a container image, an image pull request is initiated to the device provided by the embodiment of the present disclosure. The image pull request is sent to the monitoring service module of the device.

[0028] After receiving the image pull request, the listening service module initiates a request to the image service to obtain the manifest file of the target container image. The manifest file contains detailed information about the image, including the image's hierarchy, content hash value, and its required snapshotter.

[0029] When the monitoring service module obtains the manifest file, it passes the manifest file to the snapshotter feature library. At this time, the snapshotter feature library will determine the manifest type of the manifest file based on the information in the manifest file, that is, determine the manifest type corresponding to the target container image. And search for the target snapshotter corresponding to this type in the internal mapping relationship of the snapshotter feature library.

[0030] After determining the target snapshotter, the listening service module returns this information and specifies the target snapshotter as the target for subsequent pull operations. Next, the listening service module will build a complete image pull request and forward the request to containerd through the built-in containerd client. At this time, the containerd client uses the previously determined target snapshotter as a parameter to ensure that containerd can use the correct snapshotter when performing the pull operation. Finally, after receiving the request, containerd starts pulling the target container image according to the specified target snapshotter.

[0031] It should be noted that in Kubernetes, CRI (Container Runtime Interface) is a standardized interface between the container runtime and Kubernetes, which is responsible for image management, container lifecycle management and other functions. ImageService is the core component of CRI, which is used to pull and manage images. The present invention dynamically adapts to different types of images based on the functions provided in the device, and adds special parameters required for specific types of images. Specifically, in the PullImage method, a request is first initiated to the image repository through the v2 interface of the OCI (Open Container Initiative) standard to pull the manifest file of the target container image. Assuming that the target container image address is registry.example.com / my-image:latest, its manifest file can be pulled through the v2 interface of OCI: GET https: / / registry.example.com / v2 / my-image / manifests / latest For example, the returned manifest file can contain the following content: { "schemaVersion": 2, "mediaType": "application / vnd.oci.image.nydus.v1+json", "config": { "mediaType": "application / vnd.oci.image.config.v1+json", "digest": "sha256:xxxxxx" }, "layers": [ { "mediaType": "application / vnd.oci.image.layer.nydus.blob.v1", "digest": "sha256:yyyyyy" } ] } After obtaining the manifest file of the target container image, extract the manifest type (i.e., the value of the mediaType field) that represents the image characteristics. Then compare the manifest type with the mapping relationship in the snapshotter feature library to determine which snapshotter to use to process the image. If the value of mediaType in the manifest file is application / vnd.oci.image.nydus.v1+json, and the following mapping relationship is defined in the mapping library: { "application / vnd.oci.image.nydus.v1+json": "nydus-snapshotter" } Then nydus-snapshotter is matched as the target snapshotter for processing the target container image.

[0032] The PullImage method of containerd is called by the containerd client inside this device to perform the actual image pull task. The pull request needs to contain additional parameters required for a specific type of image, which can be obtained from the containerd configuration file / etc / containerd / config.toml.

[0033] After completing the construction of the image pull request, send the request to the containerd client and specify the adapted snapshotter and related parameters. Call the Pull API of containerd, and containerd interacts with the Nydus backend according to the specified snapshotter and parameters to complete the image pull and decompression. Through the above process, the device implements the dynamic adaptation of the PullImage method in the ImageService interface of CRI. According to the mapping relationship between the image type and the feature library, select the appropriate snapshotter and add the required special parameters.

[0034] The present disclosure realizes dynamic adaptation of different manifest types by introducing snapshotter feature library and monitoring service module, which significantly improves the flexibility and efficiency of container image pulling. It can intelligently select the most suitable snapshotter according to the specific manifest type of the image to be pulled, thereby avoiding compatibility issues caused by global specification of snapshotter. Through the adapter device provided by the present disclosure, users can make full use of the advantages of various snapshotters in the Kubernetes environment, thereby improving resource utilization and overall system performance. It not only improves the success rate of image pulling, but also reduces errors caused by incompatibility and reduces operation and maintenance costs.

[0035] In an optional implementation, the snapshotter feature library is preset with a first mapping relationship between a known manifest type and a corresponding snapshotter, and the first mapping relationship is loaded into the snapshotter feature library by means of internal coding.

[0036] Figure 2 This is a schematic diagram of a pre-configuration process of a snapshotter adapter device in one embodiment of the present disclosure, see Figure 2 The step of loading the mapping relationship into the snapshotter feature library belongs to one of the pre-configuration processes after the initialization of the device. The first mapping relationship refers to the mapping between the preset and known image manifest types and their corresponding snapshotters in the snapshotter feature library. Through the configured mapping relationship, the snapshotter feature library can quickly find and select the appropriate snapshotter when receiving an image pull request.

[0037] The first mapping relationship is stored in a structured form in the code of the snapshotter feature library, such as using a dictionary, a hash table or other data structure, that is, loaded into the snapshotter feature library by internal encoding. In this way, the snapshotter feature library can efficiently search and match at runtime. For example, the mapping relationship can be represented in the form of a key-value pair: snapshotter_mapping = { "application / vnd.docker.distribution.manifest.v2+json":"overlayfs-snapshotter", "application / vnd.oci.image.manifest.v1+json": "nydus-snapshotter", "application / vnd.oci.image.index.v1+json": "squashfs-snapshotter" In this example, the snapshotter_mapping dictionary defines three known manifest types and their corresponding snapshotters.

[0038] In an optional implementation, the snapshotter feature library is preset with a second mapping relationship in addition to the first mapping relationship, and the second mapping relationship is defined and loaded into the snapshotter feature library in the form of an environment variable or a command line parameter.

[0039] In addition to the preset first mapping relationship, the snapshotter feature library also supports defining and loading the second mapping relationship through environment variables or command line parameters, aiming to improve the flexibility of the system and enable users to dynamically adjust the snapshotter mapping relationship according to specific needs.

[0040] The second mapping relationship refers to the mapping between the image manifest type and the snapshotter that the user can customize through environment variables or command line parameters. The second mapping relationship allows the user to adjust the snapshotter selection at runtime to adapt to different image requirements or specific operating environments. Users can define the second mapping relationship by setting environment variables or passing command line parameters when starting the program. For example, when starting a Kubernetes cluster or related services, users can set the following environment variables: export SNAPSHOTTER_MAPPING="application / vnd.custom.image.manifest.v1+json=custom-snapshotter" In this example, the manifest type of application / vnd.custom.image.manifest.v1+json is mapped to the snapshotter named custom-snapshotter.

[0041] Users can also define the second mapping relationship through command line parameters. For example, when starting the service, you can use the following command: . / my_service--snapshotter-mapping "application / vnd.custom.image.manifest.v1+json=custom-snapshotter" In this command, the --snapshotter-mapping parameter is used to pass the mapping relationship, parse it and load it into the snapshotter feature library to obtain the second mapping relationship typed by the command line parameter.

[0042] It should be noted that the second mapping relationship has a higher priority than the first mapping relationship, that is, when the first mapping relationship and the second mapping relationship exist in the snapshotter feature library at the same time, the monitoring service module will give priority to the second mapping relationship defined by the user through environment variables or command line parameters. Users can override the default behavior through simple environment variable settings or command line parameters to achieve personalized configuration.

[0043] By supporting the loading of the second mapping relationship, the snapshotter feature library can flexibly respond to different user needs and operating environments. It not only enhances the adaptability of the overall device, but also enables users to optimize the image pulling process according to actual conditions, thereby improving overall efficiency and performance.

[0044] In this embodiment, in order to facilitate Kubernetes (through kubelet) to manage node disk resources, especially when judging disk pressure or triggering container image garbage collection (GC), the root directory path of snapshotter needs to be explicitly specified. This device uniformly specifies the root directory of the relevant snapshotter covered by the snapshotter feature library to the root directory of the default snapshotter overlayfs. Kubernetes' kubelet regularly checks the node disk pressure and evaluates the storage space usage. When the disk usage exceeds the threshold, kubelet calls containerd through CRI to trigger garbage collection (GC) of the container image. By unifying the snapshotter root directory path, kubelet can directly access and calculate all snapshot data stored in the overlayfs root directory, simplifying the logic of disk usage analysis. The core of image garbage collection is to remove images and snapshotters that are no longer used. After unifying the path, the GC logic can quickly locate all snapshot data and determine its status without querying or processing scattered storage paths one by one.

[0045] In an optional embodiment, the listening service module is also used to determine the attribute information of the target container image according to the image pull request, and based on the snapshotter feature library, determine the target snapshotter that is adapted to the target container image according to the attribute information of the target container image; the attribute information includes at least: the namespace to which the target container image belongs, the image repository to which the target container image belongs, the tag suffix of the target container image, and the tag prefix of the target container image.

[0046] When the monitoring service module receives an image pull request from kubelet, it parses the request, extracts the attribute information of the target container image, and decides which snapshotter to use to process the image pull based on the attribute information.

[0047] When selecting a suitable snapshotter, the monitoring service module considers one or more of the following attribute information: the namespace to which the target container image belongs, the image repository to which the target container image belongs, the tag suffix of the target container image, and the tag prefix of the target container image.

[0048] A namespace is a logical grouping identifier for container images that is used to distinguish different projects or applications. For example, myrepo in the image myrepo / myapp:latest is a namespace. Namespaces help systems identify the source and purpose of images. An image repository is a specific location where container images are stored, which is a URL or path to a registry. For example, docker.io is the image repository of Docker Hub. The tag of an image is used to identify the version or variant of the image. The tag suffix refers to the part separated by a colon after the image name. For example, in myrepo / myapp:latest, latest is the tag suffix. Different tags may correspond to different snapshotters, especially when using specific build or optimization strategies. In some cases, the tag prefix of an image also contains useful information. For example, v1.0.0 in myrepo / myapp:v1.0.0-alpha can be regarded as a tag prefix.

[0049] After extracting the attribute information of the target container image, the listening service module will match the attribute information with the mapping relationship in the snapshotter feature library. Specifically, the listening service module queries the first mapping relationship and the second mapping relationship in the snapshotter feature library based on the extracted attribute information to find a snapshotter that matches the attribute information. If a matching snapshotter is found in the feature library, the listening service module will designate it as the target snapshotter. If no match is found, a default snapshotter (such as overlayfs) can be selected to handle the request.

[0050] In an optional implementation, the monitoring service module includes an unknown service processor; the monitoring service module is further used to respond to other interface requests from other containerd clients, and forward the other interface requests to the containerd through the unknown service processor.

[0051] See also Figure 1 In addition to image pull requests from kubelet, it also includes other interface requests from other containerd clients. Figure 1 In the figure, the solid arrows show the image pull request processing path for kubelet, and the dotted arrows show the processing paths for other interface requests of other containerd clients.

[0052] The unknown service processor is used to take over other interface requests from other containerd clients and forward these other interface requests to the containerd service. Specifically, the function of the unknown service processor is that when the listening service module receives a request from other containerd clients (such as nerdctl, ctr, etc.), it first identifies the type of request. If the request is not an image pull request, but other types of requests (for example, querying container status, managing container lifecycle, etc.), it will be regarded as an unknown service request. For these unknown service requests, the unknown service processor will forward them to the upstream containerd service. This means that the unknown service processor is responsible for passing requests from the listening service module to containerd, so that containerd can process these requests and return the corresponding results. By using the unknown service processor, the listening service module can remain transparent to external clients. Clients do not need to know the existence of the listening service module. They can send requests as if they were accessing containerd normally, and the listening service module will process these requests in the background to ensure that the requests are forwarded and responded to correctly.

[0053] The following is a simple workflow example to illustrate how the listening service module and the unknown service processor work together. When a containerd client (such as ctr) sends a request to query the status of the currently running container. The listening service module receives this request and recognizes that it is not an image pull request, but an unknown service request. The listening service module passes the request to the unknown service processor. The unknown service processor is responsible for forwarding the request to the upstream containerd service. After receiving the request, containerd performs the corresponding operation (such as querying the container status) and returns the result to the unknown service processor. The unknown service processor returns the response of containerd to the original request client (such as ctr), completing the entire request processing process.

[0054] In an optional implementation, the monitoring service module is used to record the original containerd socket file, and transfer the original containerd socket file to a new location; create a new socket file with the same file name as the original containerd socket file.

[0055] See also Figure 2 When the device is initialized, the monitoring service module identifies and records the location of the containerd socket file currently in use. The original containerd socket file is the main interface for containerd to communicate with external clients (such as kubelet and other container management tools). The monitoring service module obtains the path of the original socket file and saves it for use in subsequent operations.

[0056] In order to avoid conflicts between the newly created socket file and the original socket file, the listening service module transfers the original socket file to a new location. Optionally, the original containerd.sock file is moved to a backup directory or renamed to ensure that it is no longer used, while retaining its content for subsequent reference. Unix Socket is an inter-process communication (IPC) mechanism based on the file system. Compared with network sockets, Unix Socket has higher performance and is suitable for low-latency, local communication scenarios. The socket file described in the present disclosure belongs to a Unix Socket socket file.

[0057] In order to maintain compatibility with external clients, the listening service module will create a new socket file with the same name as the original socket file (for example, still containerd.sock), that is, a new socket file with the same name is created at the location of the original socket file. In this way, external clients can still use the same file name when sending requests.

[0058] The monitoring service module is used to monitor the image pull request from kubelet and other interface requests from other containerd clients through the new socket file.

[0059] The new socket file will be used to listen and receive image pull requests from kubelet. The listening service module sets up a listener through the new socket file, waiting for image pull requests from kubelet, as well as other interface requests from other containerd clients.

[0060] The monitoring service module is used to respond to image pull requests from kubelet and other interface requests from other containerd clients, and send the image pull request to the containerd through the original containerd socket file.

[0061] When the listening service module receives an image pull request from kubelet, it forwards the request to the containerd service to perform the actual image pull operation. The listening service module sends the image pull request to containerd through the original containerd socket file (which has been moved to the new location). Containerd processes the request and returns the result. Similarly, the listening service module forwards requests from other containerd clients to the containerd service.

[0062] In an optional implementation, the monitoring service module is used to establish a soft link pointing to the original containerd socket file according to the original containerd socket file; The soft link is used to forward requests from kubelet or other containerd clients to the original containerd socket file when the snapshotter adapter stops running.

[0063] Considering that in some cases, the snapshotter adapter may stop running due to maintenance, failure or other reasons. At this time, requests from kubelet or other clients will not be processed, which may cause system failure or service interruption. To this end, during the operation of this device, the monitoring service module can use the soft link function provided by the operating system to create a new soft link, which points to the original containerd socket file. This soft link can be named the same as the original socket file for easy use by the client.

[0064] When this device stops running, all requests from kubelet or other containerd clients will be forwarded to the original containerd socket file through this soft link, and even if the adapter no longer processes requests, it can still respond to external requests normally. Because a soft link is created, external clients (such as kubelet and other containerd clients) do not need to know the existence or status of the snapshotter adapter, and they can continue to communicate through the same socket file.

[0065] In an optional embodiment, the device further includes a verification module; The verification module is used to write a mapping relationship between a first verification image and a corresponding snapshotter in the snapshotter feature library, wherein the first verification image is an accelerated version of a container image; The verification module is further used to create and run a second verification image, and check whether new content is generated in the root directory where the first verification image is located, and the second verification image is a common version of the container image; In the case where no new content is generated in the root directory where the first verification image is located, creating and running the first verification image, and checking whether the first verification image runs successfully; In the case where it is checked that the first verification image runs successfully, it is determined that the snapshotter adapter device passes the verification.

[0066] The verification module is used to verify whether the snapshotter adapter functions normally and ensure that it can correctly process the image pull request. The verification module verifies the mapping relationship between different versions of container images (accelerated version and normal version) and monitors their running status. The verification module records the mapping relationship between the first verification image (accelerated version of the container image) and its corresponding snapshotter in the snapshotter feature library.

[0067] Before verifying this device, you need to build a verification environment for the functions of the verification module. Here is a specific example. A virtual machine is required, configured with 4 CPU cores (4C) and 8GB of memory (8G). In terms of the operating system, install the Linux operating system, and the kernel version must be 4.19 or above. In addition, deploy a Kubernetes cluster on the virtual machine, and the version must be 1.23 or above.

[0068] You need to prepare two test images: nginx:latest and nginx:latest-nydus. nginx:latest is the first verification image of the normal version, and nginx:latest-nydus is the second verification image of the accelerated version, which uses the nydus technology.

[0069] Start nydus-snapshotter as a system service. The specific command is: containerd-nydus-grpc nydus-snapshotter can be run as a service and interact with containerd. Find the configuration file of containerd, usually located at / etc / containerd / config.toml. Add the following lines to the configuration file to add nydus as a proxy plugin: [proxy_plugins] [proxy_plugins.nydus] type = "snapshot" address = " / run / containerd / containerd-nydus-grpc.sock" After modifying the configuration, you need to restart the containerd service for the configuration to take effect. After restarting, execute the following command to verify whether nydus snapshotter is successfully introduced: ctr plugins ls | grep nydus If the output contains "ok", it means that nydus snapshotter has been successfully introduced.

[0070] Start the device and check the directory / run / containerd when the startup is complete to confirm whether the following two unix socket files exist: containerd.sock and containerd-real.sock. If these two socket files exist, it means that the device has been successfully started and is running normally.

[0071] After the above verification environment is built, the first step is for the verification module to start the second verification image and check whether there is new content generated in the root directory of the first verification image. This step is to verify whether the pull of the normal version image is the default snapshotter, because the snapshotter feature library does not define a mapping relationship for the second verification image. If there is new content generated in the root directory of the first verification image, it means that the normal version image uses the default snapshotter.

[0072] In the second step, the verification module will create and run the first verification image and check its running status. If the first verification image is checked to run successfully, it means that the accelerated version of the image has found a matching snapshotter based on the mapping relationship defined by the snapshotter feature library, and the verification module will determine that the snapshotter adapter has passed the verification. On the contrary, if the first verification image fails to run, it means that the accelerated version of the image has not been able to implement the snapshotter adaptation based on this device, and the verification module will determine that the snapshotter adapter has not passed the verification.

[0073] Figure 3 is a schematic diagram of a snapshotter adaptation method step provided by an embodiment of the present disclosure, such as Figure 3 As shown, the method is applied to a snapshotter adapter device, and the method includes: Step S101, in response to the image pull request from kubelet, obtain the manifest file corresponding to the target container image to be pulled.

[0074] When kubelet needs to pull a container image, it sends an image pull request. In response to the image pull request, it is first necessary to identify the name and version of the target container image to be pulled. Through the OCI standard v2 interface, request the image manifest file from the image repository. The manifest file contains the metadata of the image, including the image's layer information, configuration, etc., to characterize the characteristics of the target container image or the corresponding manifest type.

[0075] An image pull request can be sent to the image repository via an HTTP GET request, and the image repository will return the corresponding manifest file.

[0076] Step S102: Based on the snapshotter feature library and according to the manifest type of the manifest file, a target snapshotter that is compatible with the target container image is determined.

[0077] Parse the manifest file obtained from the image repository and extract the manifest type (for example, application / vnd.docker.distribution.manifest.v2+json).

[0078] According to the extracted manifest type, search for a snapshotter that matches it in the feature library. If a matching snapshotter is found, it will be recorded as the target snapshotter for subsequent image pull requests. If no matching snapshotter is found, a default snapshotter (such as overlayfs) is selected.

[0079] Step S103: forward the image pull request to containerd, and specify the target snapshotter to pull the target container image through containerd.

[0080] The acquired target snapshotter information is forwarded to containerd together with the image pull request. During the forwarding process, the selected target snapshotter is specified in the request so that containerd can use the snapshotter to pull the image. After containerd receives the request, it performs the image pull operation according to the specified snapshotter. Specifically, containerd calls the relevant interface of the target snapshotter and uses the characteristics of the target snapshotter to handle the image pull process.

[0081] Figure 4 This is a schematic diagram of the image pulling process in one embodiment of the present disclosure, please refer to Figure 4, when creating or starting a container, Kubernetes will send an image pull request to containerd. After receiving the image pull request, the manifest file of the target container image is obtained. After obtaining the manifest file, the manifest type that characterizes the image characteristics is compared with the preset snapshotter feature library to find a snapshotter that matches the image type. Once a snapshotter that matches the image characteristics is found, the snapshotter is determined to be the target snapshotter. If no match is found, a default snapshotter (such as overlayfs) is selected. After the target snapshotter is determined, the containerd client will use the target snapshotter to pull the target container image. Specifically, the containerd client constructs a pull request and specifies the target snapshotter in the request. By calling the relevant API of containerd, the client sends the request to containerd, and containerd performs a pull operation on the target container image according to the specified snapshotter. The target snapshotter will be responsible for processing the image's layers, downloading and storing them in the local file system, ensuring that the image can be used correctly.

[0082] In an optional implementation, the determining, based on the snapshotter feature library and according to the manifest type of the manifest file, a target snapshotter that is compatible with the target container image includes: Step S201, parsing the manifest file to determine the manifest type to which the manifest file belongs.

[0083] Parse the manifest file and extract key information. Specifically, find the mediaType field in the manifest file, which indicates the type of the manifest.

[0084] By parsing, determine the specific type of the manifest file, for example, it may be application / vnd.docker.distribution.manifest.v2+json、 application / vnd.oci.image.manifest.v1+json, etc.

[0085] Step S202: searching the snapshotter feature library for a mapping relationship that matches the manifest type.

[0086] As mentioned above, the snapshotter feature library contains the mapping relationship between different manifest types and snapshotters. The snapshotter feature library can be a static mapping table or a dynamically loaded configuration file.

[0087] Use the manifest type obtained by parsing as the query condition to find out whether there is a mapping relationship matching this type in the snapshotter feature library. If a matching mapping relationship is found, the snapshotter corresponding to the mapping relationship will be recorded.

[0088] Step S203: When a mapping relationship matching the manifest type is found, the snapshotter corresponding to the mapping relationship is determined as the target snapshotter.

[0089] If a mapping relationship matching the manifest type is found in the feature library, the target snapshotter is determined based on the mapping relationship, and the specified target snapshotter will be used to process the pull request for the image.

[0090] Save the target snapshotter's information for use in subsequent image pull requests.

[0091] Step S204: if no mapping relationship matching the manifest type is found, a default snapshotter is determined as the target snapshotter.

[0092] If no mapping relationship matching the manifest type is found in the feature library, a backup plan will be taken. In this case, the default snapshotter is used as the target snapshotter. The default snapshotter can be a general snapshotter. In the embodiment of the present disclosure, the default snapshotter can be overlayfs, which is suitable for pulling most images. The information of the default snapshotter is recorded for use in subsequent requests.

[0093] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, devices, electronic devices, and storage media. Therefore, the embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] The embodiments of the present disclosure are described with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0095] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present disclosure.

[0096] Finally, it should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. The above is a detailed introduction to a snapshotter adapter device and method provided by the present disclosure. The principles and implementation methods of the present disclosure are explained in this article using specific examples. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present disclosure; at the same time, for those of ordinary skill in the art, according to the idea of ​​the present disclosure, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A snapshotter adapter device, characterized in that: The device comprises: a snapshotter feature library, a monitoring service module and a containerd client; The snapshotter feature library includes snapshotters mapped to different manifest types; The monitoring service module is used to respond to the image pull request from kubelet, obtain the manifest file corresponding to the target container image to be pulled, and determine the target snapshotter adapted to the target container image based on the snapshotter feature library and the manifest type of the manifest file; The containerd client is used to forward the image pull request to containerd and specify the target snapshotter to pull the target container image through the containerd.

2. The device according to claim 1, characterized in that The snapshotter feature library is preset with a first mapping relationship between a known manifest type and a corresponding snapshotter, and the first mapping relationship is loaded into the snapshotter feature library by means of internal coding.

3. The device according to claim 2, characterized in that The snapshotter feature library is preset with a second mapping relationship in addition to the first mapping relationship, and the second mapping relationship is defined and loaded into the snapshotter feature library in the form of environment variables or command line parameters.

4. The device according to claim 1, characterized in that The monitoring service module is further used to determine the attribute information of the target container image according to the image pull request, and determine the target snapshotter adapted to the target container image according to the attribute information of the target container image based on the snapshotter feature library; The attribute information includes at least: a namespace to which the target container image belongs, an image repository to which the target container image belongs, a tag suffix of the target container image, and a tag prefix of the target container image.

5. The device according to claim 1, characterized in that The monitoring service module includes an unknown service processor; The monitoring service module is also used to respond to other interface requests from other containerd clients, and forward the other interface requests to the containerd through the unknown service processor.

6. The device according to claim 5, characterized in that The monitoring service module is used to record the original containerd socket file and transfer the original containerd socket file to a new location; create a new socket file with the same file name as the original containerd socket file; The monitoring service module is used to monitor the image pull request from kubelet and other interface requests from other containerd clients through the new socket file; The monitoring service module is used to respond to image pull requests from kubelet and other interface requests from other containerd clients, and send the image pull request to the containerd through the original containerd socket file.

7. The device according to claim 6, characterized in that The monitoring service module is used to establish a soft link pointing to the original containerd socket file according to the original containerd socket file; The soft link is used to forward requests from kubelet or other containerd clients to the original containerd socket file when the snapshotter adapter stops running.

8. The device according to claim 1, characterized in that The device also includes a verification module; The verification module is used to write a mapping relationship between a first verification image and a corresponding snapshotter in the snapshotter feature library, wherein the first verification image is an accelerated version of a container image; The verification module is further used to create and run a second verification image, and check whether new content is generated in the root directory where the first verification image is located, and the second verification image is a common version of the container image; In the case where no new content is generated in the root directory where the first verification image is located, creating and running the first verification image, and checking whether the first verification image runs successfully; In the case where it is checked that the first verification image runs successfully, it is determined that the snapshotter adapter device passes the verification.

9. A snapshotter adaptation method, characterized in that: Applied to the snapshotter adapter device according to any one of claims 1 to 8, the method comprises: In response to the image pull request from kubelet, obtain the manifest file corresponding to the target container image to be pulled; Based on the snapshotter feature library, determine a target snapshotter that is compatible with the target container image according to the manifest type of the manifest file; The image pull request is forwarded to containerd, and the target snapshotter is specified to pull the target container image through the containerd.

10. The method according to claim 9, characterized in that The determining, based on the snapshotter feature library and according to the manifest type of the manifest file, a target snapshotter adapted to the target container image includes: Parsing the manifest file to determine the manifest type to which the manifest file belongs; Searching the snapshotter feature library for a mapping relationship that matches the manifest type; When a mapping relationship matching the manifest type is found, the snapshotter corresponding to the mapping relationship is determined as the target snapshotter; When no mapping relationship matching the manifest type is found, the default snapshotter is determined as the target snapshotter.

Images