Master Data Management System Using Record and Field Rules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current master data management consolidation processes are manual, time-consuming, and prone to errors, especially when dealing with large datasets and conflicting data values across multiple records.
Innovation Solution
A system and method that utilize record and field-based rules to automatically identify and merge duplicate records, using a two-phase process of duplicate detection and consolidation, where record-level rules select the master record based on reliability scores and field-level rules modify fields based on value criteria, such as length, frequency, and reliability scores, to create a single, accurate master record.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual consolidation process is used, then data accuracy can be maintained through human review, but time consumption and operational complexity increase significantly
Solution Approach 1:
The patent replaces the manual mechanical review process with an automated computer-based system that uses algorithms to detect duplicate records and determine consolidation actions. The system automatically compares data across systems, identifies duplicates based on matching criteria, and executes consolidation without human intervention, thereby eliminating time consumption while maintaining accuracy through systematic rule-based processing.
Solution Approach 2:
The consolidation system performs self-service by automatically detecting duplicates, evaluating consolidation criteria, and executing merges without requiring manual human review. The computer system independently evaluates data quality metrics, applies consolidation rules, and completes the entire process autonomously, freeing operators from time-consuming manual tasks while ensuring consistent application of consolidation standards.
2Adaptability or versatility
If manual consolidation process is used, then complex judgment can be applied, but error rate increases and scalability decreases
Solution Approach 1:
The system implements dynamic consolidation rules that can adapt to different data types, systems, and quality metrics. The computer-based platform allows configuration of various matching criteria, data quality weights, and consolidation strategies that can be adjusted based on specific business requirements. This dynamic rule engine provides flexibility in handling diverse consolidation scenarios while maintaining consistent, error-free execution through automated processing.
3Productivity
If automated consolidation is implemented, then productivity increases, but system complexity and rule configuration difficulty increase
Solution Approach 1:
The patent introduces an intermediary layer between raw data and consolidation outcomes in the form of a configurable rule engine. This intermediary component translates business requirements into executable consolidation rules, managing the complexity of automated decision-making. The rule engine serves as a mediator that handles the sophisticated logic of duplicate detection and consolidation strategy selection, shielding users from underlying system complexity while enabling high-productivity automated processing.
4Reliability
If comprehensive duplicate detection is performed, then data quality improves, but processing time and computational resources increase
Solution Approach 1:
The duplicate detection process is segmented into multiple stages: initial filtering using key identifiers, intermediate evaluation using data quality metrics, and final confirmation using comprehensive comparison criteria. This segmented approach processes records in hierarchical batches, applying increasingly sophisticated analysis only where needed. By dividing the detection process into discrete segments with different resource requirements, the system achieves comprehensive data quality validation while managing computational resource consumption efficiently.
Data Source
AI summary
According to some embodiments, a plurality of input records may be received from a plurality of sources, and each input record may include a plurality of fields. It may then be detected that a set of input records from different sources are related to each other (e.g., are duplicates). One of the set of input records may be automatically selected as a master record in accordance with a record level rule. At least one field in the master record may, according to some embodiments, be automatically modified based on a corresponding field in another input record in accordance with a field level rule. The modified master record could then be stored for subsequent use by other applications.