Compound Data Prefetching via Metadata Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Wide-area network (WAN) performance bottlenecks, characterized by lower bandwidth and higher latency, significantly reduce application performance for data-intensive operations across distributed organizations, especially when accessing and manipulating compound data files that contain multiple types of data structures.
Innovation Solution
The solution involves analyzing metadata within compound data files to identify internal and external data streams, and proactively or reactively prefetching these streams to cache them locally, thereby masking WAN limitations by predicting future data access patterns and optimizing data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is accessed over WAN, then distributed organization collaboration is enabled, but bandwidth limitations reduce data transfer speed
Solution Approach 1:
The system performs preliminary actions by analyzing metadata to identify data streams that are likely to be accessed and prefetching them to cache before actual access occurs. This proactive approach prepares data in advance, reducing the impact of WAN bandwidth limitations when the data is actually needed.
Solution Approach 2:
The patent introduces a metadata analysis mechanism and prefetching cache as an intermediary between the WAN and the application. This intermediary layer analyzes compound data files, identifies required data streams, and caches them locally, thereby mediating the data transfer process and reducing the direct impact of WAN bandwidth constraints.
2Quantity of substance
If data is accessed over WAN, then remote storage utilization is achieved, but network latency increases access time
Solution Approach 1:
The system performs preliminary analysis of metadata to identify data streams that are likely to be accessed and prefetches them to cache before actual access occurs. This proactive caching reduces the time penalty of WAN latency by having data ready locally when needed.
Solution Approach 2:
The patent implements dynamic prefetching that adapts to access patterns. By continuously analyzing metadata and monitoring access behavior, the system dynamically adjusts which data streams to prefetch and when, optimizing the balance between remote storage utilization and access time based on actual usage patterns.
3Adaptability or versatility
If compound data files are accessed, then multiple data types are consolidated, but identifying and retrieving required data streams becomes complex
Solution Approach 1:
The patent extracts and analyzes metadata from compound data files to identify individual data streams and their characteristics. By separating the metadata analysis from the actual data retrieval process, the system simplifies the complexity of handling compound files with multiple data types.
Solution Approach 2:
The metadata analysis mechanism serves as an intermediary that simplifies the complexity of compound data files. By analyzing metadata first and identifying required data streams, the system creates a simplified view of the complex compound file structure, making retrieval more manageable.
4Speed
If WAN optimization devices prefetch data, then bandwidth limitations are masked, but network traffic analysis complexity increases
Solution Approach 1:
The patent extracts only the necessary metadata from compound data files for analysis, rather than analyzing entire files or all network traffic. This selective extraction reduces the complexity of network traffic analysis while still enabling effective prefetching decisions.
Solution Approach 2:
The system performs partial analysis by focusing only on metadata and identified data streams rather than comprehensive network traffic analysis. This partial action approach provides sufficient optimization without the full complexity of analyzing all network traffic patterns.
Data Source
AI summary
Access to compound data over a wide-area network is optimized by analyzing metadata within compound data to identify internal and external data streams to be prefetched. Upon receiving or intercepting a network packet including an access request for a data resource, metadata in this data resource is analyzed to identify associated data streams and their storage locations within and/or outside of the data resource. Data streams may be proactively or reactively prefetched. Proactive prefetching identifies and retrieves data streams or portions thereof likely to be accessed by a client based on attributes associated with the data resource. Reactive prefetching identifies portions of data streams associated with received access requests and retrieves additional portions of these data streams. Prefetched data streams or portions thereof are stored in a data storage on the same local network or near to the local network including the client.