Temporal Semantic Data Integration for IoT Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in effectively processing and integrating timeseries data from IoT devices with time-dependent semantic data, where the semantic data often changes over time, leading to inaccuracies and inefficiencies in data analysis due to the lack of association with correct master data values.
Innovation Solution
The solution involves tracking changes to semantic data attributes and storing them with specific temporal ranges, allowing for the retrieval of accurate master data values relevant to the timeseries data, thereby ensuring correct data integration and aggregation through the use of a data model that combines timeseries and master data in a relational database, reducing the need for complex JOIN operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional JOIN operations are used to integrate timeseries data with semantic data, then data integration can be achieved, but computational resources and processing time are excessively consumed
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing the results of JOIN operations between timeseries data and semantic data in a materialized view or cache structure. When queries are executed, the system retrieves pre-computed results rather than performing complex JOIN operations in real-time, significantly reducing computational resources and processing time while maintaining data integration accuracy.
2Reliability
If semantic data is updated frequently to reflect current values, then data currency is improved, but query accuracy for historical analysis deteriorates
Solution Approach 1:
The patent resolves this contradiction by adding a temporal dimension to semantic data storage. Instead of storing only current values, the system maintains historical versions of semantic data with timestamps, creating a time-travel capability. This allows queries to retrieve the correct semantic data version corresponding to the timeseries data period, ensuring both data currency for current operations and query accuracy for historical analysis.
Solution Approach 2:
The patent introduces an intermediary temporal mapping layer that sits between the timeseries data and semantic data. This intermediary structure tracks the validity periods of semantic data and automatically selects the appropriate version for each query based on the timeseries data timestamp, ensuring that current and historical queries receive the correct data without conflict.
3Measurement precision
If complex JOIN operations are performed to ensure correct master data association, then data accuracy is improved, but system complexity and processing overhead increase
Solution Approach 1:
The patent extracts the complex JOIN logic from the query execution path and moves it to the data ingestion and storage phase. By pre-establishing the relationships between timeseries data and semantic data during data loading, and storing this association information in the database schema, the system eliminates the need for complex JOIN operations during querying, reducing both system complexity and processing overhead while maintaining data accuracy.
Data Source
AI summary
Techniques for processing combinations of timeseries data and time-dependent semantic data are provided. The timeseries data can be data from one or more Internet of things (IOT) devices having one or more hardware sensors. The semantic data can be master data. Disclosed techniques allow for time dependent semantic data to be used with the timeseries data, so that semantic data appropriate for a time period associated with the timeseries data can be used. Changes to semantic data are tracked and recorded, where the changes can represent a new value to be used going forward in time or an update to a value for a prior time period. Timeseries data and semantic data can be stored with identifiers that facilitate their combination, such as date ranges, identifiers of analog world objects, or identifiers for discrete sets of semantic data values.