Data Synchronization via Change Tracking Services Framework
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
MDM systems face performance degradation and scalability limitations when handling highly transactional and volatile data, particularly due to the need for multiple web services or scheduled jobs for each external system, and traditional 'full data' synchronization methods are costly in terms of performance.
Innovation Solution
The implementation of a method and system that creates a first and second database instance using an integration data model, along with a services framework comprising an entity definition repository, change tracking repository, and data export element, to synchronize data between the data management system and external systems, optimizing communication through an intermediate data management system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional full data synchronization is used between MDM system and external systems, then data consistency is maintained, but system performance degrades and synchronization cost increases
Solution Approach 1:
The patent extracts only the changed data portions from the MDM system using change tracking mechanisms (trigger-based or snapshot-based change detection) and synchronizes only these specific changes to external systems, rather than performing full data synchronization. This selective extraction maintains data consistency while significantly reducing the volume of data transmitted and processed.
Solution Approach 2:
Instead of performing complete full-data synchronization, the patent implements partial synchronization by identifying and transmitting only the necessary change portions (inserts, updates, deletes) from the MDM system to external systems. This partial action approach maintains data consistency for changed records while reducing overall synchronization overhead and improving system performance.
2Adaptability or versatility
If multiple web services or scheduled jobs are created for each external system, then data synchronization capability is provided, but device complexity and scalability are limited
Solution Approach 1:
The patent implements a universal data synchronization service that can handle multiple external systems through a single standardized interface. This service uses common change tracking mechanisms and a unified data export framework that works across different external systems, eliminating the need to create separate web services or scheduled jobs for each system while maintaining full synchronization capability.
Solution Approach 2:
The patent segments the synchronization architecture into independent modular components: change tracking layer (trigger-based or snapshot-based), data export layer (services framework with configurable data marts), and consumption layer (external systems). This segmentation allows each component to be developed, maintained, and scaled independently, reducing overall system complexity while improving adaptability to different external systems.
3Ease of manufacture
If data marts are used to solve specific business problems, then targeted data delivery is achieved, but flexibility is limited to specific business problems
Solution Approach 1:
The patent implements dynamic and configurable data marts within the services framework that can be easily modified to address different business problems. The data mart configuration is not hard-coded but can be dynamically adjusted through the unified services framework, allowing the same infrastructure to serve multiple business purposes by changing configuration parameters rather than creating fixed specialized systems.
Data Source
AI summary
A method, a system and a computer program of synchronizing data between a data management system and an external system are disclosed. The method includes creating a first database instance and a second database instance using an integration data model. The method further includes synchronizing data between the second database instance and the external system. A services framework is created and the first database instance and the second database instance are interfaced using the services framework which includes an entity definition repository, a change tracking repository, and a data export element. Also disclosed are a system and a computer program for synchronizing data between a data management system and an external system.