Log-Structured Versioning for Virtual Computing I/O Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual computing systems face high I/O latency and low I/O throughput when serving queries for versioned objects due to the need to scan large amounts of unsorted data, leading to inefficient resource utilization and increased power consumption.
Innovation Solution
Implementing a log-structured object versioning system that maintains a special pointer for the latest version of objects, using multi-level indexing, tree structures, and bloom filters to efficiently locate and update versioned objects, thereby reducing I/O latency and increasing throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large amounts of unsorted data are scanned to locate versioned objects, then version queries can be served, but I/O latency increases and I/O throughput decreases
Solution Approach 1:
The patent applies preliminary action by pre-organizing versioned objects into a log-structured data structure with multi-level indexing before queries are issued. Objects are pre-sorted by version identifiers and organized into hierarchical index structures (root index, intermediate indexes, leaf indexes) that enable direct navigation to target versions without scanning all data. This pre-organization eliminates the need for runtime data scanning, thereby reducing I/O latency while maintaining query accuracy.
Solution Approach 2:
The patent introduces intermediary index structures that mediate between the query system and the actual versioned object data. Instead of directly scanning raw data, the system uses multiple levels of indexes (root index containing intermediate index pointers, intermediate indexes containing leaf index pointers, leaf indexes containing object pointers) as intermediaries to progressively narrow down and locate the target version. This intermediary layering significantly reduces the amount of data that must be read and processed, thereby decreasing I/O latency and increasing throughput.
2Measurement precision
If large amounts of unsorted data are scanned to locate versioned objects, then version queries can be served, but I/O throughput decreases
Solution Approach 1:
The patent applies preliminary action by pre-organizing versioned objects into a log-structured data structure with multi-level indexing before queries are issued. Objects are pre-sorted by version identifiers and organized into hierarchical index structures (root index, intermediate indexes, leaf indexes) that enable direct navigation to target versions without scanning all data. This pre-organization eliminates the need for runtime data scanning, thereby reducing I/O latency while maintaining query accuracy.
Solution Approach 2:
The patent applies segmentation by dividing the large dataset of versioned objects into smaller, organized segments through multi-level indexing. The data is segmented into hierarchical layers: root index segments, intermediate index segments, and leaf index segments. Each segment contains a manageable subset of data with its own index structure, allowing the system to process and navigate only the relevant segments rather than scanning the entire dataset. This segmentation increases I/O throughput by reducing the volume of data that must be read and processed for each query.
3Device complexity
If traditional data structures are used for versioning, then implementation is simpler, but resource utilization is less efficient and power consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-organizing versioned objects into a log-structured data structure with multi-level indexing before queries are issued. Objects are pre-sorted by version identifiers and organized into hierarchical index structures (root index, intermediate indexes, leaf indexes) that enable direct navigation to target versions without scanning all data. This pre-organization eliminates the need for runtime data scanning, thereby reducing I/O latency while maintaining query accuracy.
Solution Approach 2:
The patent applies dimensionality change by transitioning from a single-level, linear data structure to a multi-level hierarchical structure. The versioning system adds vertical dimensionality with multiple indexing levels (root level, intermediate levels, leaf level) rather than using a flat horizontal structure. This dimensional transformation enables the system to navigate through data in a tree-like hierarchy, dramatically reducing the search space and I/O operations required, thereby improving resource utilization and reducing power consumption despite increased structural complexity.
Data Source
AI summary
An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions to update a versioned object by appending a latest version of the versioned object to a data structure, read a predetermined version identifier corresponding to the latest version from a pointer entry, and look up the latest version of the versioned object, in the data structure, using the predetermined version identifier corresponding to the latest version of the versioned object.