Container Image Deduplication for Air-Gapped Transport
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The transport and processing of containers in air-gapped networks is inefficient due to the increasing prevalence and size of images, leading to bottlenecks in deployment and execution.
Innovation Solution
A computing system selectively compresses container images by generating an archive that stores only a single copy of each unique file or layer, verifies the files using checksums, and simulates reconstruction to ensure accuracy, transmitting the archive to a different network only if successful.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If container images are transported and processed in air-gapped networks, then application deployment is enabled, but processing and transport time increases due to image size
Solution Approach 1:
The container image is divided into multiple layers, with each layer representing a distinct set of files or dependencies. This segmentation allows the system to process and transmit only the necessary layers rather than the entire image, reducing overall transport time while maintaining deployment capability.
Solution Approach 2:
The system creates a compressed archive copy of the container image layers and transmits this archive instead of the original large image files. The archive contains redundant file information that allows for efficient reconstruction, significantly reducing transport time while enabling complete image restoration at the destination.
2Productivity
If container images are compressed into archives with redundant files removed, then transport efficiency improves, but reconstruction accuracy may be compromised
Solution Approach 1:
The system performs simulated reconstruction of container images from the compressed archive before actual deployment. This feedback mechanism verifies that the archive can be successfully reconstructed and contains all necessary files, ensuring image integrity is maintained while benefiting from compression efficiency.
Solution Approach 2:
The system removes redundant files from the archive in advance before transmission, performing the compression and deduplication work beforehand. This preliminary action reduces the size of data that needs to be transmitted and processed during deployment, improving transport efficiency while the pre-processed archive ensures reliable reconstruction.
Data Source
AI summary
Computing systems methods, and non-transitory storage media are provided for obtaining images, extracting layers from each of the images, extracting segments from each of the layers, generating a compressed version of the segments by storing a single copy of each segment and metadata to reconstruct the layers from the segments and the images from the layers, and simulating a reconstruction of the image from the compressed version.