Cluster-Wide Container Storage Optimization via File Pooling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current container storage optimization is limited to the host level and does not effectively share common files across containers, leading to inefficiencies in storage usage and increased costs due to redundant file storage.
Innovation Solution
Implementing a method to identify common and unique files across containers, moving unique files into a shared pool and using pointers for common files, allowing for cluster-wide optimization by migrating containers to hosts with the most shared files, thereby reducing redundant storage and optimizing disk usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If containers share storage layers via union file system, then storage footprint is reduced, but containers must be co-located and layers must be precisely the same
Solution Approach 1:
The patent segments container files into three categories: common files shared across multiple containers, unique files specific to individual containers, and container-specific configuration files. This segmentation allows common files to be shared via union file systems while unique files remain container-specific, resolving the contradiction between storage optimization and placement flexibility.
Solution Approach 2:
The patent introduces a file migration service as an intermediary that automatically identifies, migrates, and manages common files across containers. This intermediary service enables containers to share common files without requiring precise matching of storage layers, thereby allowing greater placement flexibility while maintaining storage efficiency.
2Quantity of substance
If container storage is optimized at host level, then some storage redundancy is reduced, but cluster-wide optimization is not achieved
Solution Approach 1:
The patent implements a cluster-wide file migration service that operates across all hosts in the container cluster, providing universal optimization. This service identifies common files across all containers cluster-wide, not just per-host, and coordinates migration to achieve optimal storage utilization across the entire cluster, thereby improving overall storage optimization efficiency.
Solution Approach 2:
The patent implements a feedback mechanism where the file migration service continuously monitors container file structures, identifies new common files, and performs iterative migration. This feedback loop ensures ongoing optimization as containers are created, modified, or deleted, maintaining optimal storage efficiency across the cluster over time.
3Quantity of substance
If common files are shared across containers, then storage space is saved, but file access and management complexity increases
Solution Approach 1:
The patent implements a self-service file migration system that automatically identifies common files, determines optimal sharing candidates, performs migration, and updates container configurations without manual intervention. This automation eliminates the complexity of manual file management while preserving storage space through shared common files.
Solution Approach 2:
The patent implements a mechanism to discard duplicate common files from individual container storage and recover the storage space by replacing them with references to the shared copy in the union file system. This process automatically manages file sharing while simplifying storage structure by eliminating redundancy.
Data Source
AI summary
One embodiment includes identifying a common file associated with a first software container deployed on a host; adding a single copy of the common file to a common file pool maintained by the host, removing the common file from the first container and replacing it with a pointer to the copy of the common file in the shared file pool, and removing the common file from the first container and replacing it with a pointer to the copy of the common file in the shared file pool; identifying at least one unique file associated with the first container; and moving the unique file to the common file pool maintained by the host and removing the unique file from the first container and replacing it with a pointer to the copy of the unique file in the shared file pool.