Polystore Integration Module for Semantic Query Coordination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current semantic web technologies are not optimized for storing and retrieving low-dimensional large data, such as time series and images, and struggle with interacting between multiple data sources, requiring manual and time-consuming processes to obtain data from external sources.
Innovation Solution
A computer-implemented method and system that uses an integration module to receive node groups containing requests for internal and external data, identifies meta-data nodes, determines their order of execution, and ingests results into a semantic datastore to generate a final query result, automating the retrieval of data from multiple sources using semantic technology and a service layer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If semantic web technologies are used to store and retrieve data, then data interlinking and semantic querying are improved, but storage efficiency for low-dimensional large data (time series, images) deteriorates
Solution Approach 1:
The system segments data into two categories: structured data stored in the semantic datastore (knowledge graph) and unstructured large data (images, time series) stored in external datastores. This segmentation allows each storage system to operate in its optimal domain, with the semantic layer providing query coordination without bearing the storage burden of large binary data
Solution Approach 2:
The patent introduces an integration module as an intermediary between the semantic datastore and external datastores. This mediator handles query routing, result aggregation, and coordination between multiple data sources, enabling semantic querying capabilities to extend to external data without requiring the semantic datastore itself to store the actual large data
2Loss of information
If data is retrieved from multiple external sources, then data completeness is improved, but query execution time and complexity increase
Solution Approach 1:
The system performs preliminary actions by pre-configuring metadata about external data sources in the semantic datastore, including data schemas, access endpoints, and relationship definitions. This preliminary structuring allows the integration module to efficiently route queries without ad-hoc exploration, reducing execution time while maintaining data completeness from multiple sources
Solution Approach 2:
The patent merges results from multiple external data sources by translating their heterogeneous data formats into a unified semantic representation. The integration module combines results from different sources according to the ontology model, presenting a consolidated view that maintains data completeness while abstracting away the complexity of multi-source retrieval
3Reliability
If manual processes are used to obtain external data, then data accuracy is improved through expert validation, but operational efficiency deteriorates
Solution Approach 1:
The system implements self-service by enabling automated query execution against external data sources through the integration module. Once metadata is configured, the system automatically retrieves, validates, and integrates external data without requiring manual intervention for each query, maintaining reliability through structured validation rules while dramatically improving operational efficiency
Solution Approach 2:
The patent changes parameters by transforming unstructured external data into structured semantic representations that conform to the ontology model. This parameter transformation includes validating data against schemas, converting formats, and enriching with semantic context, thereby maintaining data accuracy while enabling automated processing at scale
Data Source
AI summary
According to some embodiments, a system, method and non-transitory computer-readable medium are provided comprising receiving a node group at an integration module, wherein the node group includes one or more requests for internal data and external data, wherein internal data is data stored in an internal datastore and external data is data stored outside of the internal data store; identifying in a configuration data, one or more meta-data nodes from the node group, wherein each meta-data node indicates a request for external data; retrieving the internal data via execution of an internal data query; determining an order of execution for the one or more meta-data nodes; executing a first meta-data node based on the determined order to generate a first result; ingesting the first result into a semantic datastore; and executing a query to generate a final result, wherein the query includes the retrieved internal data and the ingested first result. Numerous other aspects are provided.