API Engine Synchronized Data Replication Delta Calculation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches to maintaining accurate synchronization of master data and its replication between a data warehouse and external consumers are complex and resource-intensive, particularly in scenarios where future changes need to be filtered out until they occur, such as employee status updates.
Innovation Solution
An Application Program Interface (API) engine processes queries with start and end dates and a last synchronization date to calculate and filter data changes, replicating only relevant data changes affecting the specified period, thereby simplifying delta calculation and reducing the need for additional middleware logic and storage resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If specialized data persistence in middleware is implemented to track changes and filter future changes, then data synchronization accuracy is improved, but device complexity and storage resource consumption increase
Solution Approach 1:
The patent extracts the data persistence and change tracking functionality from the middleware layer and relocates it to the database layer. The database now directly maintains version information and change timestamps for each data record, eliminating the need for specialized persistence mechanisms in the middleware. This extraction reduces middleware complexity while maintaining synchronization accuracy.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of structured query parameters that convey time-based filtering requirements from the API layer to the database layer. These parameters act as mediators, enabling the database to perform time-aware delta calculations without requiring complex middleware logic.
2Reliability
If specialized data persistence in middleware is implemented to track changes, then data synchronization accuracy is improved, but storage resource consumption increases
Solution Approach 1:
The patent extracts the change tracking functionality from middleware storage and relocates it to the database's native storage structures. By using the database's existing capacity to store version numbers and timestamps alongside data records, the system eliminates the need for separate middleware storage resources while maintaining full change tracking capability.
3Productivity
If conventional middleware approaches are used for delta calculation, then data replication can be achieved, but processing time and computational resources increase
Solution Approach 1:
The patent replaces the mechanical middleware-based delta calculation process with a database-native query mechanism. By leveraging SQL capabilities to directly compute deltas based on version numbers and timestamps stored in the database, the system eliminates the need for intermediate processing steps in the middleware, significantly reducing processing time while maintaining replication capability.
Data Source
AI summary
An Application Program Interface (API) manages synchronization and replication of master data of a system with different consumers. An API engine is configured to receive from the third party, a query including content information and date-relevant information including a start date, an end date, and a date of last synchronization. The API engine processes the query to access the database and produce a query result replicating only data changes affecting that period or a previous time. The consumer thus specifies the time period in which to consider changes to the stored database data, and the API calculates the delta according to that period. In this manner complex, time-dependent delta calculation and filtering logic can be implemented in the API without requiring substantial logic or additional data persistence in middleware. Particular embodiments may facilitate efficient synchronized replication of employment data to outside (e.g., payroll, benefits) providers.