Schema Mapping Feature Store for Enterprise Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing schema mapping processes for cloud-hosted enterprise applications are time-consuming and error-prone, requiring human intervention and domain expertise, especially when different applications with different schemas need to exchange data.
Innovation Solution
A schema mapping feature store that automates schema mapping by using feature extraction, rule-based matching, and semantic matching through machine learning models, generating similarity scores and recommending mappings, thereby reducing the need for manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual schema mapping processes are used, then mapping accuracy can be maintained through human expertise, but the process becomes time-consuming and requires domain expertise
Solution Approach 1:
The patent introduces a feature store as an intermediary component that stores pre-extracted features from schemas. This feature store acts as a mediator between the source and target schemas, enabling automated matching while maintaining accuracy. The feature extraction process creates intermediate representations that can be efficiently compared and matched, resolving the contradiction between automated speed and accurate mapping.
Solution Approach 2:
The patent applies preliminary action by pre-extracting features from schemas and storing them in the feature store before actual mapping is needed. Schema parsing, feature extraction, and normalization are performed in advance, so that when mapping is required, the system can quickly retrieve and match pre-processed features rather than processing raw schemas from scratch, significantly reducing mapping time while maintaining accuracy.
2Productivity
If automated schema mapping is implemented, then processing speed increases, but accuracy and reliability decrease without human intervention
Solution Approach 1:
The patent implements feedback mechanisms where mapping results are evaluated and used to improve future mappings. The system learns from mapping outcomes, adjusting its feature matching strategies and weighting schemes. This feedback loop enables the automated system to improve its reliability over time while maintaining high processing speed, as the learned patterns are applied automatically without requiring human intervention for each mapping task.
Solution Approach 2:
The feature store enables self-service by automatically storing and retrieving schema features without human intervention. The system autonomously parses schemas, extracts features, and performs matching operations, eliminating the need for manual domain expertise while maintaining reliability through consistent automated processes. The feature store serves itself by managing its own data structure and retrieval mechanisms.
3Measurement precision
If feature extraction and semantic matching are added to improve mapping quality, then system complexity increases
Solution Approach 1:
The patent segments the schema mapping process into distinct modular components: schema parsing, feature extraction, feature storage in the feature store, and feature matching. Each component handles a specific aspect of the mapping process independently. This segmentation allows the system to incorporate sophisticated feature extraction and semantic matching techniques without overwhelming complexity, as each module can be developed and maintained separately with well-defined interfaces.
Solution Approach 2:
The feature store serves multiple functions: it stores extracted schema features, provides a retrieval mechanism for matching operations, and maintains a standardized format for feature representation. This multi-functionality reduces overall system complexity by consolidating what could be separate complex components into a single universal feature storage and retrieval system that supports various mapping operations.
Data Source
AI summary
Embodiments map a source schema to a target schema using a feature store. Embodiments receive a file including a plurality of source schema elements and a plurality of target schema elements, the file including a plurality of unmapped elements. Embodiments retrieve rule based mappings for the unmapped elements between the source schema elements and the target schema elements. Based on semantic matching of the source schema elements, embodiments retrieve feature store based mappings from the feature store for the unmapped elements between the source schema elements and the target schema elements. Embodiments then generate one or more similarity scores for mappings of the source schema elements to the target schema elements.