Linked Data Event Streams with Semantic Context for Coherent Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data streaming systems fail to link data streams from complex systems coherently due to the lack of semantic meaning, preventing a consolidated view of the data.
Innovation Solution
A method and system for generating, publishing, and consuming linked data events and streams using an ordered node hierarchy map and modern data event schema, leveraging CSVW and JSON-LD to create a semantic web of data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data streams are shared by complex systems at high velocity, then real-time data transfer is achieved, but the ability to link data streams coherently deteriorates due to lack of semantic meaning
Solution Approach 1:
The patent introduces an intermediary component that processes data streams between production and consumption. This intermediary adds semantic context and links data streams coherently without reducing transfer velocity, resolving the contradiction between high-speed data transfer and preservation of semantic meaning.
Solution Approach 2:
The patent applies preliminary action by pre-processing data streams with semantic context before they are consumed. This allows data to be linked coherently in advance, ensuring that semantic meaning is preserved while maintaining high transfer velocities throughout the streaming pipeline.
2Productivity
If multiple data streams are processed independently, then processing efficiency is improved, but the ability to form a consolidated coherent view deteriorates
Solution Approach 1:
The patent merges multiple independently processed data streams by introducing semantic context and linking mechanisms. This allows efficient independent processing to continue while simultaneously consolidating views through the addition of semantic layers that connect the streams coherently.
3Productivity
If traditional data formats like Avro are used, then data interchange efficiency is improved, but support for linked data and semantic web deteriorates
Solution Approach 1:
The patent creates a composite data format that combines the efficiency of traditional formats like Avro with the semantic capabilities of linked data. This composite approach maintains binary format efficiency while incorporating JSON-LD or RDF structures to provide semantic meaning and linked data support simultaneously.
Data Source
AI summary
Methods and systems for generating, publishing, processing, and consuming linked data events and streams being shared by complex systems in order to construct a semantic web of the data and thereby form a consolidated coherent view of the data are provided. A method includes: receiving a set of metadata that relates to a set of events; generating, based on the metadata, an ordered node hierarchy map and a modern data event schema; receiving a set of tabular data; generating, based on the tabular data and the ordered node hierarchy map, the set of events; streaming the set of events over a streaming pipeline; processing the streamed set of events to generate a linked data media format; and outputting the linked data media format.