Cross-System Data Mapping Using Similarity Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems struggle to effectively identify and manage interrelations between datasets with varying fields and formats, leading to inconsistent and unreliable manual efforts in determining data relationships.
Innovation Solution
A computer-implemented method utilizing precision matching, concordance matching, and text analytics algorithms to generate similarity scores for data fields, creating a map that connects related data entries across disparate systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual methods are used to determine interrelations between data, then flexibility in analysis is maintained, but consistency and reliability deteriorate due to human error and inability to track changes
Solution Approach 1:
The system performs self-service by automatically executing mapping operations between datasets without requiring manual intervention. The computer system retrieves data from multiple sources, applies mapping algorithms, generates similarity scores, and produces mapped datasets autonomously, eliminating human error while maintaining high reliability
Solution Approach 2:
Manual mechanical analysis is replaced with automated computational systems. The patent substitutes human cognitive processes with computer-based mapping operations that use algorithms to determine data relationships, ensuring consistent and reliable results through deterministic computational methods rather than variable human judgment
2Reliability
If comprehensive mapping of vast amounts of data is performed, then completeness of data relationships is improved, but time and computational resources worsen
Solution Approach 1:
The mapping process is segmented into distinct operational phases: retrieving data from first and second systems, performing mapping operations on specific data fields, generating similarity scores for field pairs, and producing the final mapped dataset. This segmentation enables parallel processing and optimizes computational resource utilization across the vast dataset
Solution Approach 2:
The system performs preliminary actions by pre-retrieving data from multiple systems and pre-processing data fields before executing the main mapping operation. Similarity scores are generated in advance for data field comparisons, and the mapping framework is prepared beforehand, reducing the time required for the actual mapping execution
3Measurement precision
If multiple mapping algorithms are combined to improve accuracy, then mapping precision is improved, but system complexity worsens
Solution Approach 1:
Multiple mapping algorithms are merged into a unified mapping operation. The patent combines similarity score generation, data field comparison, and relationship determination into a single integrated mapping process that operates on both datasets simultaneously, reducing system complexity while maintaining high precision through algorithmic synergy
Data Source
AI summary
Disclosed are systems and method of mapping data entries originating in different systems. A plurality of data entries from different systems are normalized such that they can be compared to each other and mapped, even though the data entries are defined by data fields with differing phrases, descriptive details, and lengths of detail. Data entries may be filtered according to data fields before a mapping operation is employed for mapping. The mapping operation evaluates similarity scores based on the data fields using a combination of exact matching algorithms, dictionary matching algorithms, and text mining algorithms. The mapped data entries and data fields are displayed to a user.