Federated Data Query Platform With Semantic Layer Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches for querying and analyzing diverse, heterogeneous data types across distributed data stores require users to have knowledge of data locations, formats, and query mechanisms, leading to inefficiencies in data storage, query performance, and network traffic.
Innovation Solution
A knowledge-driven federated big data query and analytics platform that provides a single logical interface for interacting with diverse data types, using a semantic model to link data across multiple data stores without relocating data, enabling query and analysis directly from the storage infrastructure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If users manually generate multiple queries to interrogate disparate data types across distributed sources, then data can be accessed from multiple locations, but the approach is burdensome in time, network capacity, and infrastructure, and requires user knowledge of data locations, storage formats, and query languages
Solution Approach 1:
The patent introduces a semantic layer as an intermediary between users and distributed data stores. This semantic layer provides a unified query interface that automatically maps user queries to the appropriate data sources, formats, and access mechanisms, eliminating the need for users to manually manage multiple queries and understand underlying data infrastructure
Solution Approach 2:
The semantic layer serves multiple functions: it provides a unified query interface, automatically routes queries to appropriate data sources, transforms queries into source-specific formats, and aggregates results. This single universal interface handles all data types and storage locations, replacing the need for multiple specialized query mechanisms
2Adaptability or versatility
If all different types of data are shoehorned into a common format within a single repository using ETL operations, then data integration is achieved, but data storage efficiency and query performance deteriorate due to forcing non-relational data into relational structures
Solution Approach 1:
The patent segments the data storage architecture into multiple specialized data stores, each optimized for specific data types (relational, non-relational, time-series, etc.). The semantic layer sits above these segmented stores and provides unified access without requiring physical data movement or format conversion, thus preserving storage efficiency and query performance for each data type
Solution Approach 2:
The semantic layer acts as a mediator that provides data integration functionality without requiring ETL operations. It virtualizes the access to segmented data stores, allowing integrated queries across diverse data types while each data type remains stored in its optimal format and location
3Adaptability or versatility
If different types of data are collected into a single NoSQL data store, then no assumptions about data format or structure are required, but data storage efficiency and query performance deteriorate because the system must relocate data and cannot benefit from structural optimizations
Solution Approach 1:
The patent segments data storage into multiple specialized stores including NoSQL stores for unstructured data, relational stores for structured data, and other specialized stores for different data types. Each segment maintains its structural optimizations while the semantic layer provides unified access, avoiding the performance penalties of consolidating all data into a single NoSQL store
Solution Approach 2:
The patent adds a semantic layer dimension above the physical data storage infrastructure. This additional layer provides flexibility and unified access without requiring changes to the underlying storage structure, allowing the system to benefit from both the flexibility of NoSQL and the performance optimizations of specialized storage formats
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A system for querying a federated data store includes a metadata knowledge graph describing the contents and relationships among one or more underlying data stores, an interactive user interface receiving requests from a data consumer, a predefined constrainable query ('nodegroup') store containing predefined constrainable queries that define data subsets of interest across one or more of the underlying data repositories, a knowledge-driven querying layer generating and executing queries against the federated data store and merging responsive results, a scalable analytic execution layer receiving the search results from the federated data store and applying machine learning/artificial intelligence techniques to analyze the results, and a user interface presenting visualizations of raw or analyzed results to the consumer. A method and a non-transitory computer-readable medium are also disclosed.