Data Access Directory for Distributed System Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face challenges in accessing and managing data that is replicated across multiple networks, requiring applications to know the location and freshness of data, while also handling read/write access and maintaining consistency, which can lead to outdated local copies and complexity in data retrieval.
Innovation Solution
A directory system that provides information on data ownership, location, access types, and quality, coupled with an information service that determines the best provider for data requests, allowing access via semantic levels and managing replication and caching processes to ensure data consistency and freshness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If applications store local copies of data for quick access, then access speed is improved, but data freshness and consistency deteriorate
Solution Approach 1:
The system performs preliminary actions by establishing subscription relationships and caching mechanisms in advance. When data is updated at the provider, the subscriber is automatically notified and receives updates without manual intervention, ensuring local copies stay synchronized with master data while maintaining fast access speeds
Solution Approach 2:
The system implements feedback mechanisms where data subscribers receive notifications when master data changes. This feedback loop ensures that local cached copies are automatically updated with the latest data from the provider, maintaining data freshness while preserving the benefits of local caching for quick access
2Productivity
If applications know the location of data, then data retrieval efficiency is improved, but system complexity and coupling increase
Solution Approach 1:
The patent introduces a data directory as an intermediary that decouples data consumers from data providers. Instead of applications directly knowing and managing data locations, they interact through the directory service which handles location resolution, subscription management, and data routing, thereby reducing system complexity while maintaining efficient data retrieval
Solution Approach 2:
The system segments data access functionality into separate components: a data directory for location and metadata management, provider systems for data storage and updates, and subscriber systems for data consumption. This segmentation allows each component to operate independently, reducing coupling and overall system complexity while improving retrieval efficiency through specialized functions
3Adaptability or versatility
If data is replicated across multiple networks, then data availability is improved, but maintaining consistency and handling read/write access becomes more difficult
Solution Approach 1:
The data directory acts as a central intermediary that coordinates replication across multiple networks. It maintains information about data locations, access types, and quality levels, enabling consistent read/write operations across distributed systems without requiring complex peer-to-peer consistency protocols
Solution Approach 2:
The system allows different data copies to have different quality attributes (master data vs. replicated data) while maintaining consistency through the directory's coordination. Each replica can be optimized for its local context (read-only or read-write access) while the directory ensures overall system consistency and proper update propagation
Data Source
AI summary
A directory includes information about various systems (e.g., applications, processes, tasks, objects, services) and data, and may include data ownership information. The directory may define existing systems, corresponding locations by address, and corresponding semantic names. The directory may also specify the role of each system, e.g., a consumer or provider role. The directory may also define a scope of data a provider system can provide. As such, the directory may also include information about whether and how such data is being replicated. When a user requests data, a service may, based on the directory, determine and select an appropriate provider system for the data. The service may further create a subscription process to provide the user a local cache of the requested data. With such directory information, a service can provide data access services that integrate subscription processes.