Filesystem Replication via Data Variation Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional filesystem replication techniques are inefficient in managing increasing data volumes and meeting the demands of real-time data access and storage, leading to longer application response times and higher operational costs, especially in cloud storage systems.
Innovation Solution
The method determines patterns and properties of applications and queries to estimate the utility of storing data in different variations across replication sites, modifying the write system call to store data in variations at multiple sites if the estimated utility exceeds a predetermined threshold, optimizing application response times and storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated in exact copies across multiple locations, then system reliability and availability are improved, but application response time increases and storage efficiency deteriorates
Solution Approach 1:
The patent applies local quality by storing different variations of the same data at different replication sites. Instead of uniform exact copies, each site receives data transformed according to its specific characteristics (e.g., compression level, format optimization). This allows the system to maintain reliability through redundancy while improving response time by serving queries from the most appropriate variation at the nearest site.
Solution Approach 2:
The patent implements parameter changes by transforming data into multiple variations with different parameters before replication. The transformation process adjusts parameters such as compression ratio, data format, or organization structure based on the destination site's requirements. This enables the system to maintain data integrity and reliability while optimizing access performance for different application scenarios.
2Reliability
If data is replicated in exact copies across multiple locations, then system reliability is improved, but storage efficiency and operational costs worsen
Solution Approach 1:
By tailoring data variations to local site characteristics, the patent reduces redundant storage. Each replication site stores optimized versions of data suited to its specific access patterns and requirements, eliminating the need to store identical copies everywhere. This maintains reliability through diverse redundancy while improving storage efficiency.
Solution Approach 2:
The patent changes data parameters (compression, formatting, organization) before replication to optimize storage efficiency at each site. Different sites may receive differently compressed or formatted versions of the same data, reducing total storage requirements while maintaining data availability and integrity across the distributed system.
3Ease of manufacture
If conventional replication techniques are used, then implementation simplicity is maintained, but productivity and cost-effectiveness deteriorate
Solution Approach 1:
The patent applies preliminary action by pre-transforming data into multiple variations before replication occurs. The transformation process prepares data in advance for different access scenarios, so when queries arrive at replication sites, the optimized variations are already available. This eliminates the need for complex real-time processing and improves productivity without significantly complicating implementation.
Data Source
AI summary
A computer-implemented method according to one embodiment includes determining patterns of an application that utilizes a filesystem and/or properties of queries of the application. Data of the filesystem is stored across a plurality of replication sites of a data storage system. Based on the determined patterns of the application and/or the determined proper-ties of the queries of the application, a utility of storing at least some of the data of the filesystem in different variations at more than one of the replication sites is estimated. The estimated utility is compared against a predetermined utility threshold, and in response to a determination that the estimated utility is greater than the predetermined utility threshold, a write system call offered by the filesystem is modified to store the data in different variations at more than one of the replication sites.