Data Correlation Server for Disparate Source Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Different data repositories have unique structures and formats, making it difficult to combine and correlate data records representing the same entity across them, as they may contain different attributes and informational content.
Innovation Solution
A data correlation/integration server system retrieves source data records from various repositories, applies a data record matching rule to identify matching records, and generates an integrated data record that includes data from the matched records, linking it back to the original sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data records from different repositories are combined directly, then comprehensive data coverage is achieved, but data quality and consistency deteriorate due to structural and format differences
Solution Approach 1:
The patent introduces an intermediary integration layer that mediates between disparate data repositories. This integration layer applies transformation rules and mapping protocols to reconcile structural differences, allowing comprehensive data collection while maintaining consistency through standardized intermediate representations.
Solution Approach 2:
The system dynamically adjusts data parameters and transformation rules based on the specific characteristics of each data source. By changing parameters such as data format, structure, and mapping relationships, the system achieves both broad data coverage and consistent integration across different repositories.
2Reliability
If data repositories maintain unique structures and formats, then data integrity within each repository is preserved, but correlation between repositories becomes difficult
Solution Approach 1:
The patent segments the data integration process into distinct modules: data extraction, transformation, and loading. Each module handles specific aspects of correlation independently, allowing repositories to maintain their unique structures while the integration system systematically correlates records through structured segmentation of the integration workflow.
Solution Approach 2:
The integration system implements universal correlation mechanisms that can handle multiple data formats and structures simultaneously. The correlation engine serves multiple functions by adapting to different repository structures while applying consistent correlation logic, enabling ease of operation across diverse data sources.
3Measurement precision
If comprehensive matching rules are applied to identify matching records, then correlation accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent implements a multi-stage matching approach where partial matching rules are applied in sequence. Rather than applying all comprehensive rules simultaneously, the system uses progressive filtering with increasingly specific rules, achieving high correlation accuracy while managing processing complexity through staged partial actions.
Solution Approach 2:
The system performs preliminary data preprocessing and candidate record identification before applying comprehensive matching rules. By conducting preliminary actions such as data cleaning, normalization, and candidate selection, the system reduces the complexity of subsequent correlation operations while maintaining high accuracy through focused application of matching rules on pre-processed data.
Data Source
AI summary
Systems, methods and computer-readable media are disclosed for generating integrated data records by correlating source data records stored at different registry source data repositories. A set of source data records is retrieved based on execution of one or more search queries against a set of registry source data repositories. A data record matching rule is selected for execution on the set of source data records. The matching rule specifies one or more input properties, each of which specifies at least one data field designator, and optionally, a matching algorithm and an input property match threshold value. The matching rule is executed on pairwise combinations of source data records to obtain a set of matched source data records. An integrated data record is generated and populated with respective data from each of one or more of the matched source data records. The integrated data record is linked to each source record.