Virtual Block Device Persistence via File Map Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage solutions for virtual machines face challenges in persisting data across restarts and hardware failures, and in optimizing disk performance by efficiently managing virtual block devices across networked physical storage resources.
Innovation Solution
The method involves maintaining a file map that associates virtual block devices with data files and sequence numbers, scanning for updates, and updating the file map entries to reflect new positions of data blocks, ensuring data persistence and efficient storage across virtual machine restarts and hardware failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in virtual block devices for virtual machines, then data persistence across restarts and hardware failures is improved, but tracking and updating data block positions becomes more complex
Solution Approach 1:
The patent introduces a file map as an intermediary data structure that mediates between virtual block devices and physical storage files. The file map tracks mappings between virtual block device positions and actual file positions, allowing the system to persist data across restarts while managing the complexity of position tracking centrally rather than scattered across storage operations.
Solution Approach 2:
The system implements feedback mechanisms by scanning files to detect when data blocks have been written to new positions not reflected in the file map. Sequence numbers provide feedback about the order of writes, enabling the system to identify outdated mappings and update the file map to reflect current data positions, ensuring consistency between virtual block devices and physical storage.
2Measurement precision
If virtual block devices are managed with detailed file map entries, then data position tracking accuracy is improved, but the time and resources required to scan and update file maps increase
Solution Approach 1:
The patent applies preliminary action by maintaining file map entries with sequence numbers that record the order of data writes. This preliminary tracking information is stored alongside file position data, allowing the system to quickly determine whether a scan is necessary by comparing sequence numbers without having to perform full file scans every time, thus reducing unnecessary processing time.
Solution Approach 2:
The system performs partial scanning of files rather than complete scans, focusing only on portions of files where changes are likely to have occurred. By using sequence numbers to identify which data blocks need verification, the system avoids the excessive action of scanning entire files, reducing the time and resources required while maintaining accurate position tracking.
3Productivity
If data blocks are written to new positions in storage files, then storage optimization is improved, but maintaining accurate file map entries becomes more difficult
Solution Approach 1:
The patent uses sequence numbers as a feedback mechanism to detect when data blocks have been written to new positions. When scanning files, the system compares sequence numbers to identify writes that occurred after the file map was last updated. This feedback allows the system to reliably identify and update outdated file map entries, maintaining accuracy even as data blocks are moved to optimize storage.
Solution Approach 2:
The system changes the parameter tracking approach by incorporating sequence numbers alongside file positions in file map entries. This additional parameter allows the system to detect and adapt to position changes in data blocks, enabling storage optimization through block movement while maintaining reliable tracking of current block locations through sequence number comparison.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for virtual block storage. In one aspect, a method includes receiving a request to initialize a virtual machine, the virtual machine having an associated virtual block device; accessing a file map comprising a plurality of file map entries; determining file map entries corresponding to blocks of data allocated to the virtual block device and one or more files in which the blocks of data allocated to the virtual block device are stored; determining that a particular one of the blocks allocated to the virtual block device has been written to a new position not associated with the particular block in the file map; and updating the position associated with the particular block to the new position.