Mirror Pull System for Container Image Distribution Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The deployment of large container application clusters faces significant bottlenecks in image distribution due to high server load and network bandwidth limitations, leading to inefficient image pulling processes, especially when using private repositories.
Innovation Solution
A method and system that redirect image pull requests from overloaded servers to available clients within the same subnet or closest subnets, optimizing image distribution by utilizing point-to-point transmission protocols and prioritizing requests based on subnet load and client availability, thereby maximizing network bandwidth usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If images are pulled from a private repository server in a large container application cluster, then image distribution can be centralized and managed, but the server experiences high load and network bandwidth bottlenecks causing slow image pulling efficiency
Solution Approach 1:
The patent segments the centralized image distribution task into multiple distributed peer-to-peer connections. Instead of all clients connecting to a single registry server, clients that have already pulled images share them with other clients directly within the same subnet, dividing the distribution load across multiple nodes rather than concentrating it on one server.
Solution Approach 2:
The patent combines centralized registry management with distributed peer-to-peer transmission. The system merges the advantages of both approaches: the registry server maintains the authoritative image repository and handles initial pulls, while simultaneously enabling clients to share images peer-to-peer, combining centralized control with distributed efficiency.
2Speed
If Bit Torrent protocol is used to accelerate image distribution through P2P sharing, then image distribution speed improves, but the system complexity increases due to file division and compression requirements
Solution Approach 1:
The patent uses simple, lightweight peer-to-peer image sharing without the complex file segmentation and compression mechanisms of Bit Torrent. Clients share complete image files directly through standard Docker pull protocols, avoiding the need for complex torrent metadata, file piece division, and compression/decompression operations.
Solution Approach 2:
The patent implements direct image file copying between peers using standard Docker image formats. Instead of converting images to torrent format with compression and segmentation, the system allows clients to pull complete image files directly from other clients' file systems through simple copy operations over the network.
3Quantity of substance
If image files are divided and compressed for P2P transmission, then network bandwidth utilization improves, but the time required for file synchronization and maintenance increases
Solution Approach 1:
The patent performs image pulling and storage in advance on client nodes that need them. Once a client has an image, it can immediately share it with peers without requiring real-time compression or segmentation. The preliminary action of pre-pulling images enables fast peer-to-peer distribution without additional processing time.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed are a mirror pull method and a system therefor. The method is applied to a serving end, and comprises: receiving a mirror pull request sent out by a requesting client; judging whether a current load state of the serving end is a high-load operating state; in the case where a judgment result is yes, determining a first client, wherein the first client is configured with a mirror corresponding to the mirror pull request; and sending a redirectional mirror pull message to the requesting client, wherein the redirectional mirror pull message is used for the requesting client to pull the mirror from the first client. The system is applied to a serving end and comprises: a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the computer program implements, when executed by the processor, steps of the mirror pull method provided in the present invention. The present invention solves the problem of a network and processing capability bottleneck of a service of a private warehouse, accelerating the pull of a cluster mirror.