Data Propagator Service for Distributed Database Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, maintaining eventual consistency of data across multiple data repositories is challenging due to frequent modifications and the difficulty in determining the current state of data, leading to inconsistencies and potential downtime during data migration.
Innovation Solution
A data propagator service records updates in an event queue, determines aggregate updates based on data configuration rules specific to each data type, and applies these updates to ensure eventual consistency across repositories, using an event stream to manage and synchronize data changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is frequently modified across multiple distributed data repositories, then data scalability and flexibility are improved, but data consistency and reliability deteriorate
Solution Approach 1:
The patent introduces an event queue as an intermediary component between data repositories and the data propagator service. This event queue buffers and manages data update events, allowing the system to handle frequent modifications while maintaining consistency. The event queue decouples the production of update events from their processing, enabling scalable data modifications without compromising consistency.
Solution Approach 2:
The data propagator service implements a feedback mechanism by continuously monitoring the event queue, determining current data state, identifying inconsistencies, and propagating corrective updates back to the data repositories. This closed-loop feedback system ensures that despite frequent modifications, all repositories eventually reach a consistent state with the authoritative source.
2Ease of operation
If data state determination methods are simplified, then ease of operation is improved, but measurement precision and consistency detection capability worsen
Solution Approach 1:
The system performs preliminary actions by maintaining an event queue that pre-captures all data modification events before consistency checks are performed. This allows the data propagator service to determine data state by processing the event queue rather than querying multiple repositories, simplifying the operation while maintaining precise detection of inconsistencies through systematic event processing.
Solution Approach 2:
The patent replaces complex mechanical consistency checking mechanisms (comparing data across multiple repositories) with an event-driven processing system. Instead of mechanically querying and comparing data states across repositories, the system substitutes this with processing update events through the event queue, which is operationally simpler while maintaining precise inconsistency detection through the structured event processing pipeline.
3Adaptability or versatility
If data migration is performed in distributed systems, then adaptability and versatility are improved, but downtime and loss of service worsen
Solution Approach 1:
The patent enables continuous data migration by implementing an event-driven architecture that processes data updates asynchronously. The event queue continues to accept and process update events during migration, and the data propagator service maintains consistency propagation throughout the migration process. This allows data migration to occur without stopping service operations, eliminating downtime while maintaining adaptability for migrating data between repositories.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques for determining the state of data in database applications are described herein. Changes made to a first entry in a database table are recorded and an aggregate change is determined based at least in part on a data type of a second entry associated with the first entry. The second entry is then updated based at least in part on the aggregate change.