Distributed Private Registry Hub Parallel Image Layer Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image registry management systems are inefficient due to serial transmission of image layers, leading to slow pushing and pulling of large images, and they become unreliable if the backend storage or registry server crashes.
Innovation Solution
Implementing a locally managed registry hub that pushes multiple replicas of images to remote registries, allowing simultaneous transmission of image layers across multiple registries, and storing metadata in a local cache for improved reliability and speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image layers are transmitted serially through a single registry server, then the system structure is simple, but the transmission speed is slow and productivity is reduced
Solution Approach 1:
The patent segments the image transmission process by dividing image layers into multiple parallel transmission streams across different registry servers. Instead of transmitting all layers sequentially through one server, the system splits the transmission task across multiple servers, enabling concurrent uploads and downloads of different layers, thereby significantly improving transmission speed while maintaining manageable system complexity through modular architecture
Solution Approach 2:
The patent transitions from a single-dimension serial transmission model to a multi-dimension parallel transmission model by introducing multiple registry servers that operate simultaneously. This dimensional expansion allows the system to perform multiple transmission operations in parallel across different servers, transforming the transmission architecture from linear to distributed, thus improving productivity without overwhelming complexity
2Reliability
If a single backend storage or registry server is used, then the system is simple to manage, but reliability deteriorates when the server crashes
Solution Approach 1:
The patent implements local quality by allowing different registry servers to store different subsets of image layers locally. Each server has specialized local storage for specific layers, creating a distributed storage architecture where no single server holds the complete image. This segmentation improves reliability because failure of one server does not result in complete data loss, while the distributed local storage approach manages complexity through clear separation of responsibilities
Solution Approach 2:
The patent applies beforehand cushioning by creating redundant copies of image layers across multiple registry servers before any potential failure occurs. The system proactively distributes replicas of critical layers to multiple servers, ensuring that if one server crashes, the layers are already available on other servers. This preventive measure enhances reliability without requiring complex real-time failover mechanisms
3Reliability
If multiple replicas of images are pushed to remote registries, then High Availability is achieved, but the quantity of data transmitted increases
Solution Approach 1:
The patent applies partial action by pushing only necessary replicas of image layers to remote registries rather than complete duplicates. The system intelligently determines which layers need replication based on demand patterns and criticality, transmitting partial copies or selective layers to multiple registries. This approach achieves High Availability for critical components while minimizing unnecessary data transmission volume
Solution Approach 2:
The patent utilizes parameter changes by dynamically adjusting the replication factor and data transmission parameters based on system conditions. The system modifies transmission parameters such as replica count, data compression levels, and transmission timing to optimize the balance between reliability and data volume. By changing these parameters adaptively, the system achieves HA when needed while reducing overall data transmission
Data Source
AI summary
An embodiment maps identifying information of a remote registry into a database within a local inventory at a local registry hub. An embodiment selects at least one remote registry from an index maintained in the local inventory in accordance with a policy received at a scheduler from an external client of the local registry hub. An embodiment selects a locally stored image in accordance with a policy received from an external client of the local registry hub. An embodiment uploads replicas of the selected image via one or more registry agents, each registry agent transmitting to its corresponding remote registry, transmitting constituent layers of the replica across multiple remote registries simultaneously such that a subset of the layers constituting the image are uploaded to each remote registry. An embodiment stores metadata for the uploaded image in a cache within a local metadata store.