Virtual Machine File Deduplication via Centralized Hash Store
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtualization environments, duplicate files across multiple virtual machines lead to wasted storage space and inefficient scanning operations, as each virtual machine scans multiple copies of the same files, consuming redundant computing resources.
Innovation Solution
Implementing a virtual machine template with deduplicated files, where a hash of each file's content is generated and stored locally, and the file content is moved to a central file store, allowing virtual machines to access the files through their hashes, thereby eliminating duplicate storage and improving scanning efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple virtual machines each contain separate copies of the same files, then each virtual machine can access files independently, but storage space is wasted and scanning operations become inefficient
Solution Approach 1:
The patent merges duplicate file copies from multiple virtual machines into a single centralized file store. Files that are identical across multiple VMs are stored only once in the centralized location, while the VMs maintain references to these files. This combining approach eliminates redundant storage while preserving the ability of each VM to access its required files independently through the centralized storage system.
2Reliability
If multiple virtual machines each contain separate copies of the same files, then each virtual machine can access files independently, but scanning operations repeat the same tasks multiple times
Solution Approach 1:
The patent merges the scanning operations into a single centralized process. Instead of each virtual machine performing separate scanning tasks on its local file copies, the system performs one scanning operation on the centralized file store. This single scanning effort benefits all virtual machines that reference files from the centralized location, thereby improving scanning efficiency while maintaining file access independence for each VM.
3Quantity of substance
If file content is moved to a central file store, then storage space is saved, but file access operations become more complex
Solution Approach 1:
The patent introduces a file access intermediary layer that manages the complexity of centralized file store access. This intermediary component handles the translation between virtual machine file access requests and the centralized storage system, managing hash-based lookups, file retrieval, and caching mechanisms. By placing this intermediary layer between the VMs and the centralized file store, the system saves storage space while the intermediary absorbs the operational complexity, presenting a simplified interface to the virtual machines.
4Quantity of substance
If a centralized file store is implemented, then duplicate files are eliminated, but the system architecture becomes more complex
Solution Approach 1:
The patent employs an intermediary file management layer that simplifies the overall system architecture despite implementing centralized storage. This intermediary manages the hash-based file identification system, handles file placement in the centralized store, and manages the mapping between virtual machines and their files. The intermediary absorbs the architectural complexity, allowing the virtual machines themselves to maintain their original simple interfaces while benefiting from the deduplication advantages of centralized storage.
Data Source
AI summary
A virtual machine template is created. The template includes a file system containing files to be deduplicated across multiple virtual machines. For each file to deduplicate, a hash of the content is generated and stored in association with the file. The content of the file is moved from the virtual machine template to a file store. The entry for the file in the store is indexed according to the hash. Multiple virtual machines are created by cloning the template, each containing a copy of its file system and the hashes stored locally in association with the corresponding deduplicated files. File access operations are monitored on each one of the multiple virtual machines, and attempts to access deduplicated file are detected. In response, the corresponding locally stored hash is used to retrieve the content of the file from the central file store, and provide it to the virtual machine.