Data Resolution System for Distributed Data Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed data management across multiple remote systems is costly and prone to data synchronization issues, making it difficult to determine which system is the system of record, especially when data replication is required for specialized operations like generating emails.
Innovation Solution
A data resolution system that manages data sources and schemas across multiple systems, allowing for the orchestration of data resolution expressions to facilitate data movement between applications and physical data centers, focusing on requesting data at runtime rather than design time, and minimizing data replication and synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is replicated between data centers to enable specialized operations, then data accessibility is improved, but cost and complexity increase
Solution Approach 1:
The patent introduces a data virtualization layer that acts as an intermediary between applications and distributed data sources. This layer provides unified data access without physical replication, using virtual schemas to abstract the complexity of data distribution while maintaining data accessibility across data centers.
2Adaptability or versatility
If data is replicated between data centers to enable specialized operations, then data accessibility is improved, but cost increases
Solution Approach 1:
The patent uses virtual copies of data schemas rather than physical data replication. The virtualization layer creates lightweight schema representations that enable data access without duplicating actual data, significantly reducing storage and network costs while maintaining accessibility.
3Reliability
If data replication is implemented across multiple systems, then data availability is improved, but data synchronization becomes difficult
Solution Approach 1:
The data virtualization layer serves as a mediator that manages data access across distributed systems without requiring synchronization of the actual data. It handles data resolution and coordination centrally, eliminating synchronization complexity while maintaining data availability.
4Reliability
If schema management is implemented across multiple systems, then data consistency is improved, but system complexity increases
Solution Approach 1:
The patent implements a universal virtual schema layer that provides a single, unified view of data across multiple systems. This universal schema manages data consistency for all data sources through a common interface, reducing complexity by consolidating schema management functionality.
Data Source
AI summary
System and methods are described for management of distributed data. In one implementation, a data resolution system receives a data resolution request from a data consumer. The system then identifies data sources based on a schema registry, and queries each of the data sources for data corresponding to the expressions. The system then generates a hydrated data object comprising the received data, and transmits the hydrated data object to the data consumer.