Container Image Deduplication via Checksum Linking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The surge in demand for container image storage due to containerized services has put a strain on existing storage capacities, leading to rising costs for data centers and end users, as well as increased energy consumption and resource usage, primarily due to the duplication of files across multiple container images.
Innovation Solution
A system and method that extracts files from container image layers, determines their checksums, and creates links to existing files in a repository instead of duplicating them, thereby optimizing storage by eliminating redundant file copies across multiple container images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If files are copied across multiple container images, then each container image can be independently stored and deployed, but storage capacity requirements increase and costs rise due to file duplication
Solution Approach 1:
The patent merges file storage across multiple container images by implementing a shared file system where common files are stored once and referenced by multiple images. This combines the storage resources of multiple independent images into a unified structure, eliminating redundant file copies while maintaining image independence through symbolic links or hard links to shared files.
Solution Approach 2:
The patent creates a universal file storage mechanism where a single file can serve multiple container images simultaneously. The file system is designed to be multi-functional, allowing the same file to be accessed and utilized by different container images through references rather than physical copies, thus one file performs the storage function for multiple images.
2Quantity of substance
If files are duplicated across container images, then storage capacity increases, but operational costs and energy consumption increase
Solution Approach 1:
The patent merges the storage infrastructure into a unified file system that serves multiple container images. By combining file storage resources and eliminating redundant copies, the system reduces the total energy required for file operations, data transmission, and storage management across the container ecosystem.
3Reliability
If files are duplicated across container images, then each image has complete file sets, but resource usage increases due to redundancy
Solution Approach 1:
The patent uses symbolic links or hard links as a form of copying that references the original file without creating independent duplicate data. This allows container images to have complete file sets through references, ensuring reliability and file completeness while avoiding actual data duplication and reducing resource usage.
Data Source
AI summary
Systems and methods are included for matching a file's checksum to a file path and creating a link in a container image repository to the file at the file path, according to aspects of the present disclosure. A first container image layer comprising one or more files is received. Then, a first file of the one or more files from the first container image layer is extracted. A checksum of the first extracted file is determined, and the checksum is matched to a file path which includes the checksum, wherein the file path is associated with a second file accessible to a first container image repository via the file path. Responsive to the matching, a link to the second file is created in a second container image repository using the file path.