Container Image Deduplication via Checksum Repository
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The large size of container images and duplication of files across different instances lead to significant resource consumption in storage and transmission, as many container images share common files, resulting in inefficient use of resources.
Innovation Solution
A method where a client device calculates checksums for container image files and sends them to an image server, which determines duplicate files and only stores and transmits unique files, using checksums to reference existing files in a repository, thereby reducing storage and transmission of duplicates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all container image files are transmitted and stored completely, then file完整性 is ensured, but storage and network resources are consumed excessively due to duplicate files
Solution Approach 1:
Instead of transmitting and storing actual duplicate file copies, the system transmits and stores only checksum values (digests) that represent the files. The checksum acts as a reference copy that identifies existing files without duplicating their content, thus reducing storage and network resource consumption while maintaining file integrity through verification.
Solution Approach 2:
The checksum (digest) serves as an intermediary between the actual file content and the storage/transmission system. Rather than directly handling duplicate file content, the system uses checksums as mediators to identify, reference, and verify files, eliminating the need to store and transmit duplicate file data while ensuring integrity through checksum verification.
2Reliability
If duplicate files are stored and transmitted, then file availability is maintained, but processing resources are consumed unnecessarily
Solution Approach 1:
The system creates a reference copy of file identification information (checksum) rather than copying actual file content. This allows the system to maintain file availability by referencing existing files through their checksums while avoiding the consumption of processing resources that would be required to handle duplicate file content.
Solution Approach 2:
The system extracts only the essential identification information (checksum) from the file content and uses this extracted data for file management. By taking out only the necessary identification attributes and discarding redundant file content, the system maintains file availability while eliminating unnecessary processing resource consumption associated with duplicate files.
3Productivity
If checksums and file lists are maintained, then duplicate detection is enabled, but system complexity increases
Solution Approach 1:
The checksum mechanism serves multiple functions simultaneously: it identifies duplicate files, verifies file integrity, and enables efficient storage and transmission management. By making the checksum system multi-functional, the patent achieves duplicate file reduction without proportionally increasing system complexity, as a single mechanism handles multiple tasks.
Data Source
AI summary
Method and apparatus for reducing resource consumption in container image management. Embodiments include receiving directory information and checksum information corresponding to a plurality of files of a container image. Embodiments include determining, based on the directory information and the checksum information, whether each file of the plurality of files is present in a repository. For each respective file of the plurality of files that is determined to be present in the repository, embodiments include updating a file list for the container image to indicate a respective location in the repository of the respective file. For each given file of the plurality of files that is determined not to be present in the repository, embodiments include storing the given file at a given location in the repository and updating the file list for the container image to indicate the given location in the repository of the given file.