A container deployment acceleration method based on delayed loading and P2P distribution

Abstract

The present invention realizes a container deployment acceleration method based on delayed loading and P2P distribution through the method in the field of distributed systems. By defining a new image storage format stored in a local object storage pool using an extensible distributed object storage architecture or a hierarchical object location tracking architecture, file redundancy between images is removed, existing data is reused in upgrade and other scenarios, and delayed loading and P2P distribution are realized on the basis of the image storage format; and based on the image storage format, a server consisting of two modules based on the image storage format is designed to finally return the corresponding data from the local object storage pool according to the name of the object. The method provided by the present invention solves the problems of delayed loading of image data, file redundancy and non-reusability in image storage, and at the same time distributes files in the image through extensible distributed object storage, reducing the bottleneck of image warehouses during large-scale distribution.

Description

Technical Field

[0001] The present invention relates to the field of network technology, and in particular to a container deployment acceleration method based on delayed loading and P2P distribution. Background Art

[0002] Traditional container deployment methods have problems such as slow cold start, slow large-scale distribution, large storage redundancy, and low data reuse. Cold start refers to deploying a container on a node where the image does not yet exist. At this time, the node needs to pull the image from the warehouse before starting the container. A study analyzed commonly used images and showed that the image pulling process accounts for an average of 76% of the container cold start time, and only 6.4% of the data is actually used. When a large number of nodes need to pull images from the warehouse, the image warehouse is likely to become a bottleneck for large-scale deployment. In addition, due to the layer-by-layer packaging format of the image, a large number of files are stored redundantly. A study analyzed a large number of images on the Docker Hub and found that only 3% of the files in the image are unique. In the current image format, redundant files between images cannot be reused, and existing data may be downloaded repeatedly when the image version is upgraded.

[0003] Existing container deployment acceleration methods can be divided into three categories:

[0004] The first type is image repositories based on distributed storage, such as Slacker, Cider, and CernVM. This type of method pre-unpacks the image and stores it in a distributed file system, implements delayed loading through protocols such as NFS, and implements redundancy removal through the characteristics of the file system. The disadvantage of this type of method is that the scale of the image repository is pre-set. If the scale is too large, it will cause waste, and if the scale is too small, it will easily become a bottleneck.

[0005] The second type is to share images through cache, such as Wharf. It allows multiple nodes to share an image cache, and use the image data in the cache through protocols such as NFS to achieve delayed loading. This is a compromise solution. The process of pulling and unpacking images from the image repository is consistent with the original image distribution mode. Therefore, the effect of cold start acceleration and redundancy removal is still limited.

[0006] The third category is to define new image storage formats, such as Stargz, DADI, D4C, etc. Among them, Stargz implements delayed loading through an addressable compressed package format. DADI uses the block device ZFile format to delay loading while combining tree-shaped P2P distribution to reduce the bottleneck of the image warehouse. D4C reorganizes the hierarchical order of the image so that data can be downloaded according to the pre-measured priority. This type of solution is more flexible. Perhaps due to design priority issues, their solutions do not take into account the three issues mentioned above at the same time.

[0007] The above methods often only address one or two of the problems, and are incompatible with each other and cannot be simply combined. This method also belongs to the third category, hoping to solve the three problems of slow cold start, redundant and difficult to reuse data, and image warehouse bottleneck at the same time. Summary of the invention

[0008] To this end, the present invention first proposes a container deployment acceleration method based on delayed loading and P2P distribution, which defines a new image storage format and stores it in a local object storage pool using a scalable distributed object storage architecture or a hierarchical object location tracking architecture, thereby removing file redundancy between images, reusing existing data in upgrade scenarios, and implementing delayed loading and P2P distribution based on the format;

[0009] And based on the image storage format, design the local and remote structures;

[0010] The local end includes a local image management unit and a local object management unit, and the remote end includes a remote image repository and a remote object management unit;

[0011] The remote image warehouse stores image layer compression packages and image description files containing metadata files; the remote object management unit includes a remote object storage service client, a remote object storage client and an object location management unit;

[0012] The local object management unit includes a local object storage client and a local object storage service client;

[0013] The local image management unit pulls data from the image layer compression package and the image description file through the daemon container process, and transmits the data to the snapshot management program, and the snapshot management program sends the image layer metadata to the virtual file system, and the virtual file system reads the data in the local object storage pool and sends an object loading request to the local object storage client;

[0014] The local object storage client receives the object loading request, writes the object into the local object storage pool, and reports and queries the object location management unit, and the remote object storage service client obtains the object from the local object storage client;

[0015] The remote object storage client reports and queries the object location management unit, and the local object storage service client obtains the object from the remote object storage client and reads the object from the local object storage pool.

[0016] The new image storage format is as follows: the image contains configuration information and hierarchical information, the hierarchical information contains file metadata and directory tree information, the contents of directories and symbolic links are encoded in the file metadata, and the hash value of the file content in the file metadata is used as the key to find the file content from the object storage.

[0017] The scalable distributed object storage architecture is specifically as follows: the object location tracking service tracker set by sharding is responsible for tracking a subset of objects respectively, and the mirror repository registry and the node host are set to be equal, so that when a node needs to obtain a certain object, it can select a node to obtain data;

[0018] The hierarchical object location tracking architecture is based on a scalable distributed object storage architecture, and divides the object location tracking service tracker into a local search object tracking service Local tracker and a global search object tracking service Global tracker. The node first searches for objects in the local search object tracking service Local tracker. When the corresponding object is not found locally, it searches for the global search object tracking service Global tracker.

[0019] The specific method of the object location tracking service tracker tracking a subset of objects is that when a query from a client is received, only a subset of the object locations is returned so that the client has enough information to find the required object. When a report from the client is received, the object location information needs to be updated, and a heartbeat connection is maintained with the client. When a client cannot be contacted, the client is shielded from the location information. The object location tracking service tracker is also responsible for traffic management to prevent a large number of nodes from simultaneously requesting objects from a small number of object stores, and chooses to let the subsequent clients wait for the previous clients to complete, and obtain objects from more sources; etcd is used for dynamic configuration management. When a node of the object location tracking service tracker fails, the load of the node is migrated to other nodes, and the nodes in the cluster will communicate with etcd regularly to obtain the latest configuration

[0020] The local object tracking service Local tracker regularly communicates with the global object tracking service Globaltracker to report object location information. When a client cannot find an object on the local object tracking service Localtracker, it queries the global object tracking service Global tracker for the object location. At the same time, the local object tracking service Local tracker records that the client is obtaining the object to avoid other local clients from querying the global object tracking service Global tracker at the same time.

[0021] The client is responsible for adding, deleting and reporting objects. When a new object needs to be obtained, it obtains a location list from the object location tracking service tracker, then obtains the object from the target location according to certain rules and records the change. In regular heartbeat messages, the change is updated to the object location tracking service tracker; the frequency of change updates is negatively correlated with the number of copies of the object. When the number of copies of the object is relatively small, the object location is required to be updated faster. When the number of copies of the object is relatively large, the location update can be delayed to save communication.

[0022] The virtual file system is implemented using FUSE, and a read-only file system is mounted according to the metadata information of the image layer. When a process requests to open a file, the virtual file system opens the corresponding file from the local object pool according to the file hash value. If the file does not exist in the local object pool, it is necessary to first request the object from the object loading module, and use FUSE passthrough to reduce the data reading overhead caused by using FUSE.

[0023] The technical effects to be achieved by the present invention are:

[0024] The present invention proposes an image storage format that supports delayed loading of image data and solves the problems of file redundancy and non-reusability in image storage. It also proposes an object distribution method that distributes files in images through scalable distributed object storage, thereby reducing the bottleneck of image warehouses during large-scale distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Overall architecture diagram;

[0026] Figure 2 New image format diagram

[0027] Figure 3 Scalable distributed object storage architecture diagram

[0028] Figure 4Layered object position tracking architecture diagram DETAILED DESCRIPTION

[0029] The following is a preferred embodiment of the present invention and in conjunction with the accompanying drawings, further describing the technical solution of the present invention, but the present invention is not limited to this embodiment.

[0030] The present invention proposes a container deployment acceleration method based on delayed loading and P2P distribution. The method first defines a new image storage format, which can remove file redundancy between images and reuse existing data in upgrade scenarios; and implements delayed loading and P2P distribution based on this format. Figure 1 This is the overall outline of the solution, which consists of two main parts: local snapshot management module and distributed object distribution module. The local snapshot management module is responsible for providing support for new image formats and implementing delayed loading. The distributed object distribution module implements the distribution of actual image data. The overall architecture of the solution is as follows: Figure 1 As shown, the remote end in the figure can be an image repository or a peer node.

[0031] Local image management module

[0032] The image format in the present invention is defined as follows Figure 2 As shown in the figure. The image contains configuration information and hierarchical information, where each layer of the image only contains file metadata and directory tree information, and no longer contains file data blocks, which greatly reduces the amount of data that needs to be downloaded when pulling the image. The contents of the directory and symbolic link are encoded in the metadata, and the hash value (sha256) of the file content is stored in the metadata of the ordinary file, which can be used as a key to find the file content from the object storage.

[0033] Virtual file system:

[0034] A virtual file system is implemented using FUSE, which can mount a read-only file system based on the metadata information of the image layer. When a process requests to open a file, it will open the corresponding file from the local object pool based on the file hash value. If the file does not exist in the local object pool, it is necessary to request the object from the object loading module first. FUSE passthrough is used in this system to minimize the data reading overhead caused by using FUSE.

[0035] Snapshot Manager:

[0036] Implement a snapshot manager for the file system. During the image unpacking process, the snapshot manager will store the unpacked metadata in a specific location. When the layer is requested, the snapshot manager uses the above virtual file system to mount the layer for the container to use. When creating a container, the snapshot manager will create an additional container read-write layer, and then use overlayfs to combine all image layers into a unified file system view for the container to use. All modifications will occur on the container read-write layer in the form of copy on write.

[0037] Scalable distributed object storage

[0038] The present invention proposes a scalable distributed object storage, whose architecture is similar to HDFS, and aims to reduce the bottleneck of image warehouse in large-scale container deployment. Figure 3 Describes the architecture of this module.

[0039] In larger-scale scenarios (such as the cloud), a hierarchical object location tracking service can be organized, such as Figure 4 As shown in the figure, the node will first search for objects locally, and only search for objects in other locations if the corresponding objects cannot be found locally.

[0040] Object Location Tracking Service:

[0041] The object location tracking service (tracker) is responsible for object location management. When it receives a query from a client, it only needs to return a subset of the object location so that the client has enough information to find the object it needs. When it receives a report from the client, it needs to update the object location information. In addition, it maintains a heartbeat connection with the client, and when a client cannot be contacted, it will block the client from the location information. The object location tracking service is also responsible for traffic management to prevent a large number of nodes from simultaneously requesting objects from a small number of object stores. It can choose to let the later clients wait for the previous clients to complete and then obtain objects from more sources.

[0042] In a layered architecture, the local tracker also needs to communicate with the global tracker regularly to report object location information. When a client cannot find an object on the local tracker, it will query the global tracker for the object location. At the same time, the local tracker will record that the client is getting the object to avoid other local clients from querying the global tracker at the same time.

[0043] Object storage client:

[0044] The client is responsible for adding, deleting, and reporting objects. When a new object needs to be retrieved, it obtains a list of locations from the object location tracking service, then retrieves the object from the target location according to certain rules and records the changes. In regular heartbeat messages, it updates the object location tracking service with these changes.

[0045] The frequency of change updates is negatively correlated with the number of copies of the object. When the number of copies of the object is relatively small, the object location needs to be updated faster. When the number of copies of the object is relatively large, the location update can be delayed to save communication.

[0046] Object storage service client:

[0047] The server is responsible for responding to object requests from other clients and returning the corresponding data from the local object storage pool according to the object name (key). The image repository service can be provided by multiple nodes, each of which can store a subset of all objects and can also store multiple copies of more popular objects.

[0048] Dynamic Configuration Management

[0049] In order to prevent the metadata service from being unavailable due to a single point failure, etcd is used for dynamic configuration management. When an object location tracking node fails, the load of the node can be automatically migrated to other nodes. The nodes in the cluster will communicate with etcd regularly to obtain the latest configuration.

Claims

1. A container deployment acceleration method based on delayed loading and P2P distribution, characterized in that: By defining a new image storage format and storing it in a local object storage pool using a scalable distributed object storage architecture or a hierarchical object location tracking architecture, file redundancy between images can be removed, existing data can be reused in upgrade scenarios, and lazy loading and P2P distribution can be implemented on this format. And based on the image storage format, design the local and remote structures; The local end includes a local image management unit and a local object management unit, and the remote end includes a remote image repository and a remote object management unit; The remote image warehouse stores image layer compression packages and image description files containing metadata files; the remote object management unit includes a remote object storage service client, a remote object storage client and an object location management unit; The local object management unit includes a local object storage client and a local object storage service client; The local image management unit pulls data from the image layer compression package and the image description file through the daemon container process, and transmits the data to the snapshot management program, and the snapshot management program sends the image layer metadata to the virtual file system, and the virtual file system reads the data in the local object storage pool and sends an object loading request to the local object storage client; The local object storage client receives the object loading request, writes the object into the local object storage pool, and reports and queries the object location management unit, and the remote object storage service client obtains the object from the local object storage client; The remote object storage client reports and queries the object location management unit, and the local object storage service client obtains the object from the remote object storage client and reads the object from the local object storage pool; The new image storage format is as follows: the image contains configuration information and hierarchical information, the hierarchical information contains file metadata and directory tree information, the contents of directories and symbolic links are encoded in the file metadata, and the hash value of the file content in the file metadata is used as the key to find the file content from the object storage.

2. A container deployment acceleration method based on delayed loading and P2P distribution as claimed in claim 1, characterized in that: The scalable distributed object storage architecture is specifically as follows: the object location tracking service tracker set by sharding is responsible for tracking a subset of objects respectively, and the mirror repository registry and the node host are set to be equal, so that when a node needs to obtain a certain object, it can select a node to obtain data; The hierarchical object location tracking architecture is based on a scalable distributed object storage architecture, and divides the object location tracking service tracker into a local search object tracking service Local tracker and a global search object tracking service Global tracker. The node first searches for objects in the local search object tracking service Local tracker. When the corresponding object is not found locally, it searches for the global search object tracking service Global tracker.

3. A container deployment acceleration method based on delayed loading and P2P distribution as claimed in claim 2, characterized in that: The specific method of the object location tracking service tracker tracking a subset of objects is that when receiving a query from an object storage client, only a subset of the object locations is returned, so that the object storage client has enough information to find the required object. When receiving a report from the object storage client, the object location information needs to be updated, and a heartbeat connection is maintained with the object storage client. When an object storage client cannot be contacted, the object storage client is shielded from the location information. The object location tracking service tracker is also responsible for traffic management to prevent a large number of nodes from simultaneously requesting objects from a small number of object stores, and chooses to let the subsequent object storage clients wait for the previous object storage clients to complete, and then obtain objects from more sources; etcd is used for dynamic configuration management. When a node of the object location tracking service tracker fails, the load of the node is migrated to other nodes, and the nodes in the cluster will regularly communicate with etcd to obtain the latest configuration; The local object tracking service Local tracker regularly communicates with the global object tracking service Globaltracker to report object location information. When an object storage client cannot find an object on the local object tracking service Local tracker, it queries the global object tracking service Global tracker for the object location. At the same time, the local object tracking service Local tracker records that the object storage client is obtaining the object to avoid other local clients from querying the global object tracking service Global tracker at the same time.

4. A container deployment acceleration method based on delayed loading and P2P distribution as claimed in claim 3, characterized in that: The object storage client is responsible for adding, deleting and reporting objects. When a new object needs to be obtained, it obtains a location list from the object location tracking service tracker, then obtains the object from the target location according to the rules, and records the change; updates the changes to the object location tracking service tracker in regular heartbeat messages; the frequency of change updates is negatively correlated with the number of copies of the object. When the number of copies of the object is relatively small, the object location is required to be updated faster; when the number of copies of the object is relatively large, the location update can be delayed to save communication.

5. A container deployment acceleration method based on delayed loading and P2P distribution as claimed in claim 4, characterized in that: The virtual file system is implemented using FUSE, and a read-only file system is mounted according to the metadata information of the image layer. When a process requests to open a file, the virtual file system opens the corresponding file from the local object pool according to the file hash value. If the file does not exist in the local object pool, it is necessary to first request the object from the object loading module, and use FUSE passthrough to reduce the data reading overhead caused by using FUSE.

Images