Metadata Value-Based Mapping for Data Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data integration services statically map data from source systems to target systems without considering actual column values, ignoring valuable information that could enhance data integration efficiency.
Innovation Solution
A metadata value-based mapping approach that compares data values from the source system with metadata from the target system to identify matching values, allowing for intuitive data loading into specific destinations based on relationships between data values and metadata descriptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional static mapping is used to map data from source systems to target systems, then the data integration process is simple to implement, but actual column values are ignored causing loss of valuable information that could enhance data integration efficiency
Solution Approach 1:
The system performs preliminary comparison between source data values and target metadata values before actual data loading. This preliminary action identifies matching values and determines destination locations in advance, allowing the data integration process to leverage actual data values for intelligent routing without adding significant complexity to the overall workflow
Solution Approach 2:
The patent introduces metadata as an intermediary layer between source data and target destinations. The metadata describes destinations at the target system and serves as a bridge for comparing source data values with target locations, enabling value-based mapping while maintaining system modularity and managing complexity through this intermediate representation
2Productivity
If metadata value-based mapping is implemented to compare data values with metadata, then data integration efficiency is improved, but the processing time and computational resources increase
Solution Approach 1:
The system extracts and compares only the relevant column values from the source data with corresponding metadata values, rather than processing entire datasets. This selective extraction of key identifying values reduces the computational burden while still enabling intelligent destination selection based on actual data content
Solution Approach 2:
The patent applies partial action by performing metadata comparison on a sample or subset of data values first, or only on critical identifier columns, rather than comparing all data values across all columns. This partial comparison provides sufficient information for destination determination while significantly reducing processing time and computational resources
Data Source
AI summary
The present embodiments relate to metadata value-based mapping during a data load in a data integration job. A computing device can receive a first data set from a source system and computer-readable instructions to load data into a target system. The device can receive a first metadata set from the target system that describe destinations. The computing device can identify a first data value of the first data set that matches a metadata value of the first metadata set. The device can receive a data integration mapping of the second data value of the first data set to a data field associated with the matching metadata value of the first metadata set. The device can load the second data value of the first data set from the source system into the target system pursuant to the mapping and the computer-readable instructions.