Replication Control in Redundant Data Centers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Denormalized data storage systems face challenges in efficiently managing queries and data consistency across distributed systems, leading to increased computational effort, storage costs, and decreased efficiency due to the need for multiple queries and replicated indices, which can result in bottlenecks and reduced reliability.
Innovation Solution
The implementation of promoted properties and aggregated properties allows for the 'pre-joining' of data by replicating and updating properties across related entities, reducing the number of queries required and enhancing data coherency, thereby improving query performance and system reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple queries and replicated indices are used in distributed data storage systems, then data availability and redundancy are improved, but computational effort and storage costs increase
Solution Approach 1:
The system performs preliminary actions by pre-computing and materializing aggregate results and data joins before queries are executed. Aggregated data is pre-calculated and stored in intermediate tables, and data from multiple sources is pre-joined, so that when queries arrive, the system can retrieve pre-computed results rather than performing complex computations at query time. This reduces computational effort while maintaining data availability through redundancy.
Solution Approach 2:
The system creates copies of data and computational results by materializing aggregates and joining data across multiple tables into intermediate representations. Instead of computing aggregates and joins dynamically for each query, the system creates physical copies of these results and stores them for rapid retrieval. This copying approach maintains data redundancy for availability while reducing the computational burden of repeated queries.
2Loss of information
If multiple queries are executed to retrieve data from distributed systems, then data completeness is improved, but response time increases
Solution Approach 1:
The system merges multiple data sources and query operations into unified intermediate tables and aggregated results. By joining data from multiple tables and aggregating information into pre-computed structures, the system consolidates what would otherwise require multiple separate queries into single retrieval operations. This merging approach maintains data completeness by incorporating information from all sources while dramatically reducing response time through eliminated query multiplexing.
Solution Approach 2:
The system performs preliminary data consolidation and aggregation actions by pre-computing joins and materializing results in intermediate tables. This preliminary action ensures that when queries are executed, the necessary data is already prepared and accessible, eliminating the need for multiple sequential queries. The pre-computed aggregated data is immediately available, reducing response time while maintaining complete information through the pre-joined data structures.
3Measurement precision
If replicated indices are maintained in distributed storage systems, then query accuracy is improved, but storage costs increase
Solution Approach 1:
The system performs preliminary aggregation and index creation actions by pre-computing aggregate results and materializing them in intermediate tables. Instead of maintaining separate replicated indices for every possible query, the system pre-computes aggregates and stores them as ready-to-query data structures. This preliminary action provides query accuracy through pre-computed indices while reducing storage costs by eliminating redundant index structures and only retaining necessary aggregated data.
Solution Approach 2:
The system creates copies of aggregated data and pre-computed indices rather than maintaining full replicated indices for all data. By copying only the necessary aggregate results and storing them in intermediate tables, the system maintains query accuracy for the most important queries while significantly reducing storage requirements. The copied aggregated data provides indexed access to frequently queried information without the storage overhead of complete data replication.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, methods, and software for management of redundant data storage centers is provided herein. An exemplary method includes redundantly storing data records across the redundant data storage centers, with a first data storage center operating as a primary data storage center configured to respond to at least user input related to the data records. The method includes placing the first data storage center into an intermediate mode which comprises ceasing response to the user input by the first data storage center and executing pending operations related to previous user input received by the first data storage center, and designating the second data storage center as the primary data storage center configured to respond to at least further user input related to the data records.