IPFS-based batch type mirror distribution method and system, and storage medium

Abstract

Disclosed in the present invention are an IPFS-based batch type mirror distribution method and system, and a storage medium, belonging to the field of cloud computing containers. The method comprises: acquiring a list of cluster nodes about to download a mirror, calculating the cache ratio of mirror layers of a mirror to be distributed of each node, and comparing the cache ratio of each node with a threshold value of a preset node, so as to obtain a complete mirror; adding nodes having the complete mirror into a source node list, adding the nodes about to download the mirror into a demand node list, constructing an initial IPFS network on the basis of the nodes in the source node list, and exporting and sharing a mirror file; using an optimization algorithm to calculate target demand nodes, inputting the target demand nodes into the IPFS network for mirror downloading, importing the downloaded mirror file and starting a container; and removing from the demand node list demand nodes that have finished downloading, and adding same into the source node list until the demand node list is empty. The present invention can improve transmission efficiency, and increase the robustness of transmission processes.

Description

IPFS-based batch image distribution method, system, and storage medium Technical Field

[0001] This invention relates to a batch image distribution method, system, and storage medium based on IPFS, belonging to the field of cloud computing container technology. Background Technology

[0002] In container clusters, as services scale, tasks need to be scheduled across multiple nodes to achieve load balancing and efficient resource utilization. Therefore, multiple nodes need to download the same container image. In scenarios with strict requirements for service response speed, such as e-commerce platform promotions or streaming services releasing new trending content, the speed of image distribution becomes crucial to service quality. These scenarios need to reduce user wait times while ensuring the gradual scaling of the service. In the initial stage, to quickly respond to sudden increases in user traffic, a small number of nodes need to rapidly start containers to handle early traffic and prevent users from waiting too long. In subsequent stages, to meet the ever-increasing traffic demands, it is essential to ensure that the remaining majority of nodes can also start quickly.

[0003] However, in the aforementioned scenarios, the default image distribution method, where multiple nodes download images from a central image repository in parallel, can lead to network resource congestion in the repository, impacting the download speed and service startup latency of each node. This is particularly detrimental in high-concurrency scenarios requiring rapid scaling. To address the network congestion issue of the central repository, the industry commonly utilizes P2P transmission protocols, such as the traditional IPFS, for image distribution. This allows image data to be transmitted directly between nodes, reducing reliance on the central image repository and improving overall download speed.

[0004] In image distribution scenarios, while the traditional IPFS image distribution process improves the overall distribution speed, it cannot meet the rapid response needs of some nodes and still has shortcomings in transmission efficiency and robustness. First, while P2P transmission improves the overall speed of image distribution, all nodes complete image downloads and start container services almost simultaneously, resulting in service unavailability for a period initially, increasing user wait time and impacting service quality. Second, when transmitting images, IPFS needs to search for nodes with the required images level by level in the network via DHT. This is especially problematic when the number of source nodes is relatively small, as most nodes in the cluster act as intermediate routing nodes. These nodes occupy a large portion of the network, not only failing to provide the required files but also making the network topology larger and more complex, thus slowing down the node discovery process. Finally, in the initial download phase, many requesting nodes can only obtain image file blocks from a limited selection of download sources, leading to competition among downloaders for source node network resources, making transmission quality susceptible to fluctuations in the initial node network.

[0005] Furthermore, although IPFS enables nodes to share the required file blocks immediately after downloading them, thus providing more download sources for subsequent download requests, the download order of mirrored blocks is random. Therefore, it is difficult to guarantee that the required file blocks exist on multiple nodes. When the initial source node leaves the cluster, other nodes cannot obtain the remaining required file blocks, which can cause the transmission process to become congested.

[0006] Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a batch-based image distribution method, system, and storage medium based on IPFS, thereby solving the problems of slow transmission speed and long transmission latency caused by bandwidth contention.

[0008] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0009] In a first aspect, the present invention provides a batched image distribution method based on IPFS, comprising:

[0010] The container scheduling result is obtained using the container orchestration engine, and based on the container scheduling result, a list of cluster nodes for the image to be downloaded is obtained;

[0011] Based on the cluster node list, calculate the cache ratio of the image layer of the image to be distributed that each node already owns, compare the cache ratio of each node with the preset threshold of the node, and select the node whose cache ratio exceeds the threshold to obtain the complete image from the image repository.

[0012] Nodes with complete images are added to the source node list, and nodes with images to be downloaded are added to the demand node list. An initial IPFS network is constructed based on each node in the source node list, and image files are exported and shared based on the IPFS network.

[0013] An optimization algorithm is used to calculate the target required nodes, and the target required nodes are input into the IPFS network in batches according to the optimization algorithm results for image download. The downloaded image files are then imported and the container is started.

[0014] Remove the currently downloaded demand node from the demand node list and add it to the source node list, then continue with the mirror download of the next batch of demand nodes until the demand node list is empty.

[0015] In conjunction with the first aspect, further, the calculation of the cache ratio of the image layer for each node to be distributed includes:

[0016] Get the cached image layer information of each node in the cluster node list of the image to be downloaded, as well as the image layer information of the image to be distributed, where the image layer information includes the image layer digest and image layer size;

[0017] The cached image layer digest of each node in the cluster node list of the image to be downloaded is compared with the image layer digest of the image to be distributed to determine whether each node in the cluster node list has a cached image layer of the image to be distributed.

[0018] If some nodes have image layer caches for images to be distributed, then the image layer cache ratio of the images to be distributed is calculated based on the image layer size.

[0019] In conjunction with the first aspect, the process of acquiring the mirror layer information further includes:

[0020] Based on the image name and version number, obtain the corresponding image manifest file from the image repository, and extract the image layer information based on the image manifest file.

[0021] In conjunction with the first aspect, the expression for the image layer cache ratio of the image to be distributed is as follows:

[0022] Among them, R i Indicates the cache ratio of node i; N represents the total number of mirror layers; S j H represents the size of the j-th mirror layer; j This indicates whether the j-th image layer has been cached on the node.

[0023] In conjunction with the first aspect, further, if the image layer caching ratio of the images to be distributed does not exceed the threshold of the preset nodes, then any node can be selected to obtain the complete image from the image repository.

[0024] In conjunction with the first aspect, further, exporting and sharing image files based on the IPFS network includes:

[0025] Each node in the source node list starts the IPFS daemon process;

[0026] The container runtime of each node in the source node list exports the image file and shares the image file to the IPFS network using the add instruction.

[0027] In conjunction with the first aspect, further, the requesting node joins the IPFS network for mirror download, including:

[0028] The upload bandwidth of each node in the source node list and the download bandwidth of each node in the demand point list are obtained using a bandwidth detection tool.

[0029] The target nodes required for the current batch of transmission are calculated using an optimization algorithm;

[0030] The target node adds the IP address and network ID of each node in the source node list to the IPFS network configuration;

[0031] The target node starts the IPFS daemon process and downloads the required image file based on the hash value of the image file and the get command.

[0032] Based on the first aspect, the expression for the target demand node for the current batch of transmission is calculated as follows:

[0033] Where N represents the number of demand nodes; M represents the number of current source nodes; D i B represents the download bandwidth required by node i; j Represents the upload bandwidth of source node j; S represents the required image size for the node; R i Indicates the proportion of the existing image layer cache for demand node i; X i The decision variable represents whether a node with a demand will download in this batch; α represents the weighted value of the average download latency; β represents the weighted value of the unused source node bandwidth.

[0034] Secondly, an IPFS-based batch image distribution system includes:

[0035] The node list acquisition module is used to obtain the container scheduling result using the container orchestration engine, and based on the container scheduling result, obtain the cluster node list of the image to be downloaded;

[0036] The node information collection module is used to calculate the cache ratio of the image layer of the image to be distributed that each node already owns based on the cluster node list, compare the cache ratio of each node with a preset node threshold, and select nodes whose cache ratio exceeds the threshold to obtain the complete image from the image repository.

[0037] The strategy execution module is used to add nodes with complete images to the source node list and nodes with images to be downloaded to the demand node list, and to build an initial IPFS network based on each node in the source node list, and to export and share image files based on the IPFS network.

[0038] The distribution coordination module is used to calculate the target demand nodes using an optimization algorithm, and input the target demand nodes into the IPFS network in batches according to the optimization algorithm results for image download, and import the downloaded image files and start the container;

[0039] The list update module is used to remove the currently downloaded demand node from the demand node list and add it to the source node list, and continue to download the next batch of demand nodes in the mirror until the demand node list is empty, thus completing the mirror download of all demand nodes.

[0040] Thirdly, a computer-readable storage medium having a computer program stored thereon, characterized in that, when executed by a processor, the program implements the steps of the method as described in the first aspect.

[0041] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0042] Based on the container scheduling results, this invention determines the cluster nodes that need to download images, calculates the cache ratio of image layers that each node already has for the images to be distributed based on the cluster node list, and obtains complete images from the image repository for nodes whose cache ratio exceeds a preset threshold. By selecting nodes that exceed the preset threshold, these nodes have more image layer caches, thereby enabling the construction of more source nodes at a faster speed.

[0043] An optimization algorithm is used to calculate the target demand nodes, and these nodes are then input into the IPFS network in batches for mirror transmission according to the algorithm results. Since existing nodes have already obtained the complete image before new nodes are added to the IPFS network, these nodes can start containers earlier, thus accelerating service response. Furthermore, subsequent demand nodes can stably download the required image files from multiple source nodes, reducing bandwidth resource competition for the same source node network. In the later stages of IPFS transmission, by continuously updating the source node list and the demand node list, the number of source nodes is kept at a relatively high proportion. Newly added nodes can discover source nodes with fewer queries and select the optimal transmission path, improving transmission efficiency. Attached Figure Description

[0044] Figure 1 is a flowchart of the IPFS-based batch image distribution method provided in an embodiment of the present invention;

[0045] Figure 2 is a schematic diagram of the batch image distribution method based on IPFS provided in an embodiment of the present invention;

[0046] Figure 3 is a block diagram of the IPFS-based batch image distribution system provided in an embodiment of the present invention;

[0047] Figure 4 is a flowchart of the IPFS-based batch image distribution system provided in an embodiment of the present invention. Detailed Implementation

[0048] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0049] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0050] Example 1

[0051] Figure 1 is a flowchart of a batch image distribution method based on IPFS according to Embodiment 1 of the present invention. This flowchart only shows the logical order of the method described in this embodiment. Under the premise of no conflict, in other possible embodiments of the present invention, the steps shown or described may be performed in a different order than that shown in Figure 1.

[0052] Referring to Figure 1, the method of this implementation specifically includes the following steps:

[0053] Step S1

[0054] Use the container orchestration engine to obtain container scheduling results, and based on the container scheduling results, obtain a list of cluster nodes for the image to be downloaded;

[0055] Step S2: Based on the cluster node list, calculate the cache ratio of the image layer of the image to be distributed on each node, and set the threshold of the node. Compare the cache ratio of each node with the preset node threshold, and select the nodes that exceed the threshold to obtain the complete image from the image repository.

[0056] Specifically, the steps for calculating the cache ratio of the image layer for each node to be distributed are as follows:

[0057] Step S21: Obtain the cached image layer information of each node in the cluster node list of the image to be downloaded, as well as the image layer information of the image to be distributed;

[0058] The image layer information includes image layer digest and image layer size. Image layer digest represents the hash value obtained by hashing the image layer file. This hash value uniquely corresponds to the image layer. Image layer size represents the storage volume occupied by the image layer.

[0059] Furthermore, the process of obtaining image layer information includes:

[0060] Based on the image name and version number, retrieve the corresponding image manifest file from the image repository, and extract the image layer information from the image manifest file.

[0061] It should be noted that the image manifest file is a file generated when the image is stored in the image repository. This file includes image information metadata, which includes image layer information.

[0062] Step S22: Compare the cached image layer digest of each node in the cluster node list of the image to be downloaded with the image layer digest of the required (to be distributed) image, and determine whether each node in the cluster node list has a cached image layer of the image to be distributed.

[0063] Step S23: If some nodes have image layer caches for the images to be distributed, calculate the proportion of image layer caches for the images to be distributed based on the image layer size.

[0064] The expression for calculating the cache ratio of this image layer is as follows:

[0065] Among them, R iIndicates the cache ratio of node i; N represents the total number of mirror layers; S j H represents the size of the j-th mirror layer; j This indicates whether the j-th image layer has been cached on the node.

[0066] The cache ratio calculated by formula (1) is used to ensure that nodes whose cache ratio exceeds the set threshold download the complete image from the image repository. If no node in the image layer has a cache ratio exceeding the preset threshold, then any node is selected to obtain the complete image from the image repository.

[0067] In practical applications, during the initial stage of P2P transmission, due to the limited number of source nodes, other requesting nodes can only download the required images from a limited number of source nodes, leading to bandwidth competition and slower transmission speeds. The image distribution method of this application selects nodes with high cache levels, i.e., nodes exceeding the set threshold, to obtain complete images from the image repository and add them to the source node list mentioned in step S3 as initial source nodes. This allows for the selection of as many initial source nodes as possible during the initial stage of P2P transmission. At the same time, these selected nodes already have a large number of image layer caches, so the process of obtaining complete images from the image repository is faster, and nodes that obtain complete images can start containers earlier, accelerating service response speed.

[0068] Step S3: Add the nodes with complete images to the source node list and the nodes with images to be downloaded to the demand node list. Build the initial IPFS network based on the nodes in the source node list and export and share image files based on the IPFS network.

[0069] Specifically, each node in the source node list starts the IPFS daemon process;

[0070] The container runtime of each node in the source node list exports the image to be distributed, i.e. the required image file, and shares the image file to the IPFS network through the add instruction.

[0071] It should be noted that the network ID of a node is a unique identifier for the node, usually a long string generated by hashing the node's public key. It is used for identification and communication between nodes, data verification, and network connection. After adding the network configuration information of other nodes, the node can start the IPFS daemon to communicate with the other configured nodes, upload, search, and download files.

[0072] Step S4: Use an optimization algorithm to calculate the target demand nodes for the current batch of transmission, input the target demand nodes into the IPFS network for image download (image transmission), import the downloaded image file into the container and start the container, and store the image file locally.

[0073] Specifically, the steps for inputting the target request node into the IPFS network for mirror download include:

[0074] Step S41: Obtain the upload bandwidth of each node in the source node list and the download bandwidth of each node in the demand node list using a bandwidth testing tool.

[0075] Step S42: Calculate the target demand nodes for the current batch of transmission using an optimization algorithm; Step S43: The target demand nodes add the IP addresses and network IDs of each node in the source node list to the IPFS network configuration;

[0076] Step S44: The request node starts the IPFS daemon process and downloads the required image file based on the image file's hash value and the get command.

[0077] The `get` command is used to retrieve and download files from the IPFS network based on the file's content identifier. It utilizes content addressing to retrieve files by their content, enabling users to access and download any public files shared on the IPFS network.

[0078] It should be noted that during the mirror transmission process, if some nodes are unable to provide downloads due to failures or other reasons, the requesting nodes can also download the mirror file from other nodes that have a copy of the mirror, increasing the robustness of the transmission process.

[0079] Furthermore, the expression for calculating the target demand node is as follows:

[0080] Where N represents the number of demand nodes; M represents the number of current source nodes; D i B represents the download bandwidth required by node i; j Represents the upload bandwidth of source node j; S represents the required image size for the node; R i Indicates the proportion of the existing image layer cache for demand node i; X i The decision variable represents whether a node with a demand will download in this batch; α represents the weighted value of the average download latency; β represents the weighted value of the unused source node bandwidth.

[0081] Furthermore, if the number of demanding nodes is small, the optimal solution of formula (2) can be calculated by exhaustive search. If the number of demanding nodes is large, the optimal solution of formula (2) can be calculated by heuristic algorithm, that is, the download time is calculated based on the bandwidth of the demanding nodes and the required download volume, and nodes are added to the IPFS network for transmission in order from low to high according to the required download time, until the download bandwidth of the demanding node is higher than the upload bandwidth of the source node, and then the addition of nodes to the IPFS network is stopped.

[0082] Furthermore, the constraint that the download bandwidth of the demand node is less than the upload bandwidth of the source node is as follows:

[0083] In this embodiment of the invention, the first half of the objective function represents the average download latency required by the node, and the second half represents the unused source node bandwidth. In the mirror allocation method of this application, in order to reduce the average transmission latency of the node and make the best use of the source node's upload bandwidth to avoid resource waste, the average download latency and the unused source node bandwidth are weighted to make the objective function value as small as possible. The constraint requires that the download bandwidth of the node currently included in the network transmission is less than the upload bandwidth of the source node, thereby avoiding network resource competition and congestion. Whether the node ultimately needs to download in this batch is determined by the decision variable X using the above formula 2. i If the decision variable X of node i is determined... i If the value is 1, then node i will be added to the IPFS network for transmission in this batch.

[0084] Step S5: Remove the downloaded requirement node from the requirement node list and add it to the source node list. Repeat steps S3 to S5 until the requirement node list is empty.

[0085] Specifically, compared to the traditional IPFS mirror distribution method, which in extreme cases requires searching the entire cluster network to find a node with the required image, resulting in long transmission delays, the application continuously updates the cluster node list. In the initial stage of IPFS transmission, although the number of source nodes in the cluster is small, only a small number of demand nodes are added, making the network nodes relatively simple and node discovery relatively fast. In the later stages of IPFS transmission, as demand nodes that have completed downloads are continuously moved to the source node list, the number of source nodes also reaches a considerable proportion, allowing newly added nodes to discover source nodes with fewer queries. At the same time, among many source nodes, the application selects the transmission path with the shortest distance and better network conditions, thereby improving transmission efficiency. Furthermore, during the transmission process, more and more nodes have complete image copies rather than just partial file blocks. Even if some nodes cannot provide downloads due to network failures or other reasons, the remaining nodes can download the image file from other nodes that have the copy, effectively increasing the robustness of the transmission process.

[0086] Example 2

[0087] This invention provides an image distribution method for use in a Kubernetes cluster. The implementation environment is as follows: there are 8 worker nodes in the cluster that require a specific image. Each node uses Docker as the container runtime, and the 8 nodes are in the same Kubernetes cluster. The network conditions of each node are exactly the same, and the upload bandwidth is equal to the download bandwidth. Each node has deployed IPFS-related components. One node has 60% of the image layer of the required image, while the other nodes do not need image-related caching.

[0088] The specific image distribution steps are as follows:

[0089] Step S211: After completing container scheduling, interact with the Kubernetes API Server to obtain a list of nodes where containers need to be deployed and their IP address information. Based on the node IP address information, interact with the IPFS API of these nodes via remote calls to obtain the node's network ID information.

[0090] Step S212: Based on the image name and version number of the image required by the container to be run, obtain the manifest file of the image from the image repository, and extract the image layer information (including image layer digest and size) from the manifest file.

[0091] Step S213: For the image to be distributed, obtain the image name and version number of the cached image of the 8 nodes in the cluster one by one by interacting with the Kubernetes API, and obtain the corresponding manifest file from the image repository according to the image name and version number. Extract all image layer digests cached by the node from the manifest file, compare them with the required image layer digest, and calculate the cache hit ratio of each node according to the image layer size.

[0092] For nodes with a cache ratio exceeding 50%, the `docker pull` command is used to download the complete image from the central image repository and then start the container.

[0093] Step S214: Add the node that currently has the complete image to the source node list, and add other nodes that need the image to the demand node list.

[0094] Step S215: Each node in the source node list starts the IPFS daemon process using the ipfs daemon command, specifies the image file to be exported using docker save, and shares the image file to the IPFS network using the ipfs add command.

[0095] Step S216: Decision on which nodes in this batch should be added to the IPFS network for mirror transmission using an optimization algorithm;

[0096] Step S217: The node selected by the optimization algorithm performs the following operations: Add the IP address and network ID information of each node in the source node list to its own IPFS network configuration; Execute the ipfs daemon command to start the IPFS daemon process; Download the required image file from the IPFS network using the ipfs get command; After the image file is downloaded, Import the image file into the local storage of the container runtime using the docker load command, and then start the container.

[0097] Step S218: After all nodes in the current batch have completed the image acquisition, remove the nodes that have acquired the image from the demand node list and add them to the source node list. Repeat steps S216 to S218 until the demand node list is empty, thus completing the image download for all demand nodes.

[0098] In this embodiment of the invention, since only a single node has a mirror layer cache exceeding 50% of the set threshold, initially only this node retrieves the complete image from the mirror repository and serves as the initial source node. The remaining 7 nodes are in the initial list of required nodes. Furthermore, since all nodes have identical network conditions and upload bandwidth equals download bandwidth, when calculating the number of nodes to be added in each batch using an optimization algorithm, an equal number of required nodes are randomly selected to join the IPFS network for mirror transmission based on the number of original nodes in the network at that time. As shown in Figure 2, initially there is only a single source node in the IPFS network, so one required node is added for mirror transmission in the first batch. After this node downloads the image, there are now two source nodes in the IPFS network, so two required nodes are added for transmission in the second batch. Similarly, the remaining 4 required nodes are added for transmission in the third batch. Once these nodes have completed their transmissions, the mirror transmission of all required nodes is complete.

[0099] Example 3

[0100] A batch image distribution system based on IPFS includes:

[0101] The node list acquisition module is used to obtain the container scheduling result using the container orchestration engine, and based on the container scheduling result, obtain the cluster node list of the image to be downloaded;

[0102] The node information collection module is used to calculate the cache ratio of the image layer of the image to be distributed that each node already owns based on the cluster node list, compare the cache ratio of each node with a preset node threshold, and select nodes whose cache ratio exceeds the threshold to obtain the complete image from the image repository.

[0103] The strategy execution module is used to add nodes with complete images to the source node list and nodes with images to be downloaded to the demand node list, and to build an initial IPFS network based on each node in the source node list, and to export and share image files based on the IPFS network.

[0104] The distribution coordination module is used to calculate the target demand nodes using an optimization algorithm, and input the target demand nodes into the IPFS network in batches according to the optimization algorithm results for image download, and import the downloaded image files and start the container;

[0105] The list update module is used to remove the currently downloaded demand node from the demand node list and add it to the source node list, and continue to download the next batch of demand nodes in the mirror until the demand node list is empty, thus completing the mirror download of all demand nodes.

[0106] Specifically, referring to Figure 3, the node list acquisition module includes a node list acquisition component, used to obtain a list of cluster nodes for the image to be downloaded; the node information collection module includes a node information collection component, used to query the IP address and network ID of each node and calculate the caching ratio of the image to be distributed; the distribution coordination module includes a distribution coordination component, used to monitor changes in the number of source nodes and the image download status of the requesting nodes in real time, and to coordinate the image distribution order of the nodes, adding the requesting nodes to the initial IPFS network in batches. This module also needs to execute corresponding actions according to the policy logic of the distribution coordination module, including notifying the requesting nodes to add the IP address and network ID of the source node to their own IPFS network configuration at appropriate times, and starting the IPFS daemon; the policy execution module includes a policy execution component, used to interact with the node's IPFS API and container runtime, and to execute network configuration, image export and import, IPFS service process startup, and image file sharing operations according to algorithm logic; the list update module includes a list update component, used to update the source node and requesting node lists.

[0107] Further, referring to Figure 4, during image distribution, the node information collection component interacts with the container orchestration engine to find nodes that possess the image and construct a source node list. If none of the nodes in the cluster node list possess the complete image, an arbitrary node is selected to obtain the complete image from the image repository. Next, the policy execution component interacts with the container runtime of the aforementioned nodes and the IPFS API via remote access to construct an initial IPFS network and export and share the image to be distributed (image file) based on the IPFS network. Then, the distribution coordination component determines the nodes to be transmitted in this batch and notifies the policy execution component to add the nodes that need to be transmitted in the current batch to the IPFS network for image transmission. After the transmission is completed, the policy execution component notifies the nodes in this batch to import the image file and start the container, while providing download services for subsequent nodes. Finally, the distribution coordination component updates the source node list and the demand node list and repeats the above operations until the demand node list is empty, completing the image acquisition.

[0108] It should be noted that the IPFS-based batch image distribution system provided by this invention can quickly obtain the IP information and image cache status of each node. Before transmission, it enables nodes with the required image to configure the network, start the transmission service and share the image through remote access. During the transmission process, it monitors the image download status of each node in the cluster in real time. After the nodes in the same batch have completed the download, it quickly coordinates the download of subsequent nodes, effectively reducing the overhead caused by manual configuration.

[0109] Example 4

[0110] This embodiment provides a computer-readable storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the steps of the methods described in Embodiments 1 and 2.

[0111] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0113] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0114] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0115] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A batch-based image distribution method based on IPFS, characterized in that, include: The container scheduling result is obtained using the container orchestration engine, and based on the container scheduling result, a list of cluster nodes for the image to be downloaded is obtained; Based on the cluster node list, calculate the cache ratio of the image layer of the image to be distributed that each node already owns, compare the cache ratio of each node with the preset threshold of the node, and select the node whose cache ratio exceeds the threshold to obtain the complete image from the image repository. Nodes with complete images are added to the source node list, and nodes with images to be downloaded are added to the demand node list. An initial IPFS network is constructed based on each node in the source node list, and image files are exported and shared based on the IPFS network. An optimization algorithm is used to calculate the target required nodes, and the target required nodes are input into the IPFS network in batches according to the optimization algorithm results for image download. The downloaded image files are then imported and the container is started. Remove the currently downloaded demand node from the demand node list and add it to the source node list. Continue to download the next batch of demand nodes in the mirror until the demand node list is empty, thus completing the mirror download of all demand nodes.

2. The batch image distribution method based on IPFS according to claim 1, characterized in that, The calculation of the cache ratio of the image layer for each node to be distributed includes: Get the cached image layer information of each node in the cluster node list of the image to be downloaded, as well as the image layer information of the image to be distributed, where the image layer information includes the image layer digest and image layer size; The cached image layer digest of each node in the cluster node list of the image to be downloaded is compared with the image layer digest of the image to be distributed to determine whether each node in the cluster node list has an image to be distributed. Image layer cache for sending images; If some nodes have image layer caches for images to be distributed, then the image layer cache ratio of the images to be distributed is calculated based on the image layer size.

3. The batch image distribution method based on IPFS according to claim 2, characterized in that, The process of obtaining the mirror layer information includes: Based on the image name and version number, obtain the corresponding image manifest file from the image repository, and extract the image layer information based on the image manifest file.

4. The batch image distribution method based on IPFS according to claim 2, characterized in that, The expression for the image layer caching ratio of the image to be distributed is as follows: Among them, R i Indicates the cache ratio of node i; N represents the total number of mirror layers; S j H represents the size of the j-th mirror layer; j This indicates whether the j-th image layer has been cached on the node.

5. The batch image distribution method based on IPFS according to claim 4, characterized in that, If the image layer caching ratio of the images to be distributed does not exceed the threshold of the preset node, then any node can be selected to obtain the complete image from the image repository.

6. The batch image distribution method based on IPFS according to claim 1, characterized in that, Exporting and sharing image files based on the IPFS network includes: Each node in the source node list starts the IPFSdaemon process; The container runtime of each node in the source node list exports the image file and shares the image file to the IPFS network using the add instruction.

7. The batch image distribution method based on IPFS according to claim 1, characterized in that, The requesting nodes join the IPFS network for mirror downloading, including: The upload bandwidth of each node in the source node list and the download bandwidth of each node in the demand point list are obtained using a bandwidth detection tool. The target nodes required for the current batch of transmission are calculated using an optimization algorithm; The target node adds the IP address and network ID of each node in the source node list to the IPFS network configuration; The target node starts the IPFS daemon process and downloads the required image file based on the hash value of the image file and the get command.

8. The batch image distribution method based on IPFS according to claim 7, characterized in that, The expression for calculating the target node requirement for the current batch of transmission is as follows: Where N represents the number of demand nodes; M represents the number of current source nodes; D i B represents the download bandwidth required by node i; j Represents the upload bandwidth of source node j; S represents the required image size for the node; R i Indicates the proportion of the existing image layer cache for demand node i; X i The decision variable represents whether a node with a demand will download in this batch; α represents the weighted value of the average download latency; β represents the weighted value of the unused source node bandwidth.

9. A batch image distribution system based on IPFS, characterized in that, include: The node list acquisition module is used to obtain the container scheduling result using the container orchestration engine, and based on the container scheduling result, obtain the cluster node list of the image to be downloaded; The node information collection module is used to calculate the cache ratio of the image layer of the image to be distributed that each node already owns based on the cluster node list, compare the cache ratio of each node with a preset node threshold, and select nodes whose cache ratio exceeds the threshold to obtain the complete image from the image repository. The strategy execution module is used to add nodes with complete images to the source node list and nodes with images to be downloaded to the demand node list, and to build an initial IPFS network based on each node in the source node list, and to export and share image files based on the IPFS network. The distribution coordination module is used to calculate the target demand nodes using an optimization algorithm, and input the target demand nodes into the IPFS network in batches according to the optimization algorithm results for image download, and import the downloaded image files and start the container; The list update module is used to remove the currently downloaded demand node from the demand node list and add it to the source node list, and continue to download the next batch of demand nodes in the mirror until the demand node list is empty, thus completing the mirror download of all demand nodes.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 8.

Images