Model-Based Data Replication for Heterogeneous Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data replication techniques are inadequate for handling heterogeneous data sources, particularly in distributed systems, as they often focus on preserving ACID properties rather than managing replicas efficiently, and struggle with reconciling diverged local and remote copies in applications that perform CRUD operations.
Innovation Solution
A model-based approach is introduced, where an information model is created with a specified schema, allowing for the replication of data across multiple data sources, with a consistency policy ensuring data integrity between replicas, and enabling applications to perform CRUD operations on a unified model tier.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data replication is performed using traditional techniques focused on ACID properties, then data consistency is preserved, but replication efficiency and applicability to heterogeneous data sources deteriorates
Solution Approach 1:
The patent introduces a model tier as an intermediary layer between the data tier and application tier. This model tier contains model instances that represent data from multiple heterogeneous sources in a unified format, enabling efficient replication without directly dealing with the complexity of heterogeneous data formats and ACID property preservation across diverse sources.
Solution Approach 2:
The patent segments the system into three distinct tiers: data tier (heterogeneous data sources), model tier (unified model instances), and application tier (CRUD operations). This segmentation allows each tier to handle specific concerns independently, with the model tier serving as a buffer that enables efficient replication while maintaining consistency.
2Adaptability or versatility
If data replication is performed using federated database approach, then heterogeneous data sources are aggregated, but management of replicas and synchronization deteriorates
Solution Approach 1:
The model tier acts as an intermediary that receives data from heterogeneous sources in the data tier and presents a unified view to applications. This intermediary layer simplifies replica management by providing a standardized interface and data structure, eliminating the need to manage complexity of heterogeneous formats across multiple replicas.
Solution Approach 2:
The patent transforms heterogeneous data from multiple sources into homogeneous model instances with a unified schema. This homogenization occurs in the model tier, where all data sources are represented using the same data structures and formats, greatly simplifying replication and synchronization operations.
3Productivity
If data replication is performed using distributed file systems, then data distribution is achieved, but higher level representations for efficient storage and retrieval deteriorates
Solution Approach 1:
The patent adds a new dimension (the model tier) between the data tier and application tier. This intermediate layer provides higher-level representations of data as model instances with rich metadata and structured formats, enabling efficient storage and retrieval operations while maintaining distributed data architecture.
Solution Approach 2:
The patent creates model instances as copies of data from heterogeneous sources, but these copies are not simple replicas. Instead, they are transformed into a unified model format that includes schema information, data types, and relationships, making them more efficient for storage and retrieval operations compared to raw file system copies.
4Speed
If local replicas are created for CRUD operations, then access latency is reduced, but reconciliation of diverged copies becomes non-trivial
Solution Approach 1:
By representing all data in the system as homogeneous model instances with a unified schema in the model tier, the patent simplifies reconciliation operations. When replicas diverge, the system can use the consistent model structure to detect conflicts and apply resolution policies, rather than dealing with format mismatches between heterogeneous data sources.
Solution Approach 2:
The patent implements a feedback mechanism where the model tier monitors changes to model instances and triggers synchronization operations when divergence is detected. This feedback loop automatically reconciles replicas by comparing them against the authoritative model instance and applying updates, reducing the complexity of manual reconciliation.
Data Source
AI summary
Techniques for replicating data in a system having a data tier with a plurality of data sources and an application tier with a plurality of applications are provided. Such techniques can include creating an instance of an information model of the data that is specified by a schema. The model can be created in a form that is suitable for manipulation by at least one of the applications and that forms at least a portion of a model tier that is intermediate the data tier and the application tier. A replica of an instance of the information model can be created in a form suitable for remote transmission, and a consistency policy can be obtained that specifies a level of consistency required between the instance of the information model and the replica.