Data Stream Processing Device with Modular Semantization Platform
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques and tools are inadequate for managing and processing large-scale data streams, particularly 'hot' data that requires real-time processing, due to their inability to handle the volume and variety of data streams effectively.
Innovation Solution
A device comprising a knowledge base and a front-end communication device that captures and processes data streams, applying semantization, summarization, and interconnection of streams, with a modular platform for decision-making and storage, using RDF and SPARQL for data processing and management, enabling scalable and adaptive data handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current techniques and tools are used for processing data streams, then processing capability is limited, but handling large-scale and diverse data streams becomes impossible
Solution Approach 1:
The system segments data stream processing into multiple independent components: front-end communication devices capture streams, a platform performs various processings (semantisation, summarization, crossing), and a decision-making device analyzes results. This segmentation allows parallel processing of large volumes of data streams, overcoming the limitation of single-tool processing capabilities.
Solution Approach 2:
The platform is designed with multi-functional processing capabilities including semantisation, summarization, crossing of streams, and interconnection of streams. This universal platform can handle diverse data stream types and processing requirements simultaneously, enabling the system to process large-scale and varied data streams that current specialized tools cannot handle.
2Loss of information
If more processing operations are applied to data streams, then data insights improve, but system complexity increases
Solution Approach 1:
The system divides complex processing operations into distinct functional modules: semantisation unit, summarization unit, crossing unit, and decision-making unit. Each module performs a specific processing function, allowing the system to apply multiple processing operations while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The platform acts as an intermediary between data stream capture and decision-making. It provides standardized interfaces and common processing functions that simplify the integration of multiple processing operations, reducing overall system complexity while enabling comprehensive data analysis.
3Speed
If real-time processing of hot data is implemented, then response speed improves, but processing accuracy may deteriorate
Solution Approach 1:
The system performs preliminary semantisation and structuring of data streams in real-time at the platform level, preparing data for faster subsequent analysis by the decision-making device. This preliminary processing maintains real-time response capability while ensuring data quality and accuracy for critical decisions.
Solution Approach 2:
The system applies different processing depths to different data streams based on requirements: critical hot data receive full processing (semantisation, summarization, crossing) for accuracy, while less critical streams receive streamlined processing for speed. This selective processing approach balances real-time response with processing accuracy.
Data Source
AI summary
The present invention relates to a device for processing large-scale data streams (big data), comprising a knowledge base, a hardware and software assembly constituting a front-end communication device for capturing the streams of the external medium and if needed restoring data to this medium, the front-end delivering the streams to a platform where it undergoes different processings, traces are collected and stored in a storage and memory architecture during execution of processing operations, the platform producing data which supply a decision-making device comprising a hardware and software assembly defining decision rules for either triggering actions or initiating retroactions to the front-end communication device or to the knowledge base of said processing device.