Graph-Based Declarative Mapping for Heterogeneous Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for facilitating communication between heterogeneous systems are inflexible and inefficient, particularly in handling complex data structures with multiple MIME types and dynamically changing content streams, often requiring metadata that limits their ability to process arbitrarily complex data structures in real-time.
Innovation Solution
A graph-based declarative mapping technique that generates target information sets by traversing a mapping graph, using declarative expressions and resolvers to map and transform content from multiple sources to multiple destinations without the need for pre-configured metadata, supporting conditional and hierarchical mapping, and allowing dynamic updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional techniques use pre-configured metadata repositories to facilitate communication between heterogeneous systems, then system communication is enabled, but flexibility and efficiency are reduced when handling complex data structures with multiple MIME types and dynamically changing content streams
Solution Approach 1:
The patent extracts the metadata dependency from the system architecture. Instead of requiring pre-configured metadata repositories, the system dynamically generates metadata during runtime by analyzing the actual data structures being processed. This extraction eliminates the burden of maintaining extensive metadata while preserving the ability to handle complex, heterogeneous data types through on-the-fly type inference and adaptation.
Solution Approach 2:
The system implements self-service by automatically generating and managing metadata without external configuration. The mapping engine autonomously analyzes source and target data structures, infers appropriate transformations, and adapts to new data types as they appear. This self-configuration capability eliminates the need for manual metadata repository maintenance while maintaining high adaptability to complex data structures.
2Adaptability or versatility
If conventional techniques process complex data structures with multiple MIME types, then comprehensive data handling is achieved, but performance efficiency deteriorates due to the need to maintain and reference extensive metadata
Solution Approach 1:
The system performs preliminary action by pre-compiling and caching the mapping graph structure before runtime execution. The mapping graph, which encodes transformation logic between data structures, is built in advance and stored in an optimized format. During runtime, the system traverses this pre-compiled graph rather than generating mappings on-the-fly, significantly improving processing efficiency while maintaining support for multiple MIME types through the graph's inherent flexibility.
Solution Approach 2:
The patent introduces dynamics by enabling the mapping graph to adapt to new data types and structures without requiring system reconfiguration. The graph structure allows runtime addition of new mapping nodes and edges as new MIME types or data formats are encountered. This dynamic adaptability maintains comprehensive data handling capability while avoiding the performance penalty of reprocessing entire metadata repositories when new types are introduced.
3Reliability
If conventional techniques require metadata to be configured and stored in advance for each data source and target, then structured data processing is enabled, but the ability to process arbitrarily complex data structures deteriorates
Solution Approach 1:
The patent applies segmentation by breaking down complex data structure mappings into discrete, composable mapping operations represented as nodes in a mapping graph. Each node handles a specific transformation task (e.g., type conversion, field mapping, nested structure traversal). This segmentation allows the system to process arbitrarily complex data structures by composing simple, reliable mapping operations, maintaining processing reliability while achieving high adaptability to new and complex data formats.
4Stability of the object's composition
If conventional techniques use stored metadata to guide content transformation, then transformation consistency is maintained, but responsiveness to dynamic changes in source content deteriorates
Solution Approach 1:
The system implements feedback by continuously monitoring source content changes and dynamically adjusting the mapping graph traversal accordingly. When source data structures change, the system uses type inference and schema validation to detect changes and automatically updates the mapping graph to reflect new data patterns. This feedback mechanism maintains transformation consistency through structured validation while achieving rapid responsiveness to dynamic changes by avoiding full metadata reconfiguration.
Data Source
AI summary
Techniques for mapping from one or more source information sets to one or more target information sets are provided. Mapping ontologies associate a first declarative expression with a first information set from which to read first information, and a second declarative expression with a second information set to which second information determined based upon the read first information is written. A mapping graph having a nodes arranged according to an execution flow for obtaining the one or more target information sets is generated, at least some of these nodes representing respective ones of the mapping ontologies. The one or more target information sets are generated, each including information determined from the one or more source information sets, by traversing the mapping graph according to the execution flow. Each of the first and second declarative expressions may be associated with a mapping resolver for accessing one or more information sets.