Multi-Tenant MDM Platform with Dynamic Golden Record Versioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Master Data Management (MDM) and Data Quality Management systems face inefficiencies in updating 'golden records' due to their linear process nature, requiring system shutdowns for integrating new datasets, which hampers scalability and real-time data access.
Innovation Solution
A multi-tenant platform with a three-layer information model architecture, enabling seamless data integration, scalable operations, and real-time data access through APIs, connectors, and event streaming, allowing for rapid deployment and integration of diverse data sources into a unified, trusted data environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a traditional linear process is used to update golden records in MDM systems, then data integrity is maintained through sequential processing, but system availability deteriorates requiring shutdowns for integration
Solution Approach 1:
The system segments the golden record into multiple versions (current version and new version) with different validity periods. The current version remains accessible for read operations while the new version is being integrated, allowing system availability to improve without compromising data integrity. This segmentation enables parallel processing of integration tasks.
Solution Approach 2:
The system dynamically manages version validity periods, allowing the current version to remain valid while a new version is being integrated. The validity period of the current version can be extended or adjusted during the integration process, enabling the system to maintain availability while processing updates without requiring complete shutdowns.
2Quantity of substance
If new datasets are integrated into an existing MDM system, then data completeness is improved, but integration time increases requiring system shutdowns
Solution Approach 1:
The system performs preliminary actions by creating a new version of the golden record with the integrated dataset before fully committing the integration. This allows validation and preparation of the new data structure in advance, reducing the actual integration time and enabling faster deployment of new datasets without requiring prolonged system shutdowns.
Solution Approach 2:
The system maintains continuity of useful action by allowing read operations to continue on the current version while write and integration operations proceed on the new version. This parallel operation enables data completeness to improve without interrupting system availability or requiring complete shutdowns during integration.
3Productivity
If a multi-version architecture is implemented, then system availability is improved through parallel processing, but system complexity increases
Solution Approach 1:
The system implements a universal version management mechanism that handles multiple versions of golden records through a standardized interface. The version manager provides unified functions for creating, validating, and managing different versions, reducing the apparent complexity by presenting a consistent abstraction layer over the multi-version architecture.
Solution Approach 2:
The version manager acts as an intermediary between the integration processes and the golden record storage. It mediates the complexity of multi-version management by providing standardized operations for version creation, validation, and activation, thereby simplifying the system architecture while enabling parallel processing and improved availability.
4Manufacturing precision
If validation rules are applied to new datasets, then data quality is improved, but integration speed decreases
Solution Approach 1:
Validation rules are applied in advance during the creation of the new version of the golden record, before the integration is fully committed. This preliminary validation ensures data quality requirements are met early in the process, allowing faster integration execution without compromising data quality, as the validation overhead is distributed across the version creation phase rather than blocking the entire integration.
Data Source
AI summary
Among other techniques, techniques for dynamic survivorship, cross-tenant matching, and lineage entity identifier (EID) promotion are described. A system utilizing these techniques can include an EID assignment engine, a legacy EID retention engine, and a data item update engine. The legacy EID retention engine can be at least conceptually characterized as comprising a legacy EID survivorship subengine and a lineage EID promotion subengine. A method utilizing these techniques can include assigning a first EID to a first data item, maintaining the first EID during survivorship and lineage EID promotion processes in a multitenant EID lineage-persistent relational database management system (RDBMS), and merging the first data item with a second data item that has a second EID, which triggers survivorship and lineage EID promotion.