Graph-Based Data Stream Search for IoT Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for correlating data streams from IoT sensors are inefficient, consuming excessive computing and memory resources and unable to perform effectively in distributed environments due to the need for indexing, which slows down search processes and lacks predictability.
Innovation Solution
A graph-based searching approach is introduced, where tuples from data streams are extracted and represented as nodes in a graph, with edges representing correlations based on predetermined queries, allowing for traversing of the graph to improve search speed and accuracy, enabling parallel and asynchronous processing in distributed environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional indexing methods are used for correlating data streams, then data correlation can be performed, but search speed decreases and computing/memory consumption increases
Solution Approach 1:
The patent creates a graph structure that copies and represents the correlation relationships between data tuples in a more efficient format. Instead of using traditional indexing mechanisms that require extensive computing and memory resources, the invention constructs a graph where nodes represent tuples and edges represent correlations, enabling faster traversal and search operations with reduced resource consumption
Solution Approach 2:
The patent replaces the traditional mechanical indexing system with a graph-based structure. The graph traversal mechanism substitutes for conventional index-based search methods, providing a more efficient approach to data correlation that reduces both computing power and memory requirements while improving search speed
2Adaptability or versatility
If traditional indexing methods are used for data stream correlation, then searching can be performed, but the system cannot effectively operate in distributed environments
Solution Approach 1:
The patent segments the data correlation problem into independent graph traversal operations that can be distributed across multiple nodes. The graph structure allows correlation searching to be divided into smaller tasks that can be executed in parallel in distributed environments, maintaining search effectiveness while enabling adaptability to distributed systems
Solution Approach 2:
The graph-based searching mechanism provides universal functionality that works both in centralized and distributed environments. The same graph structure and traversal algorithms can be applied across different system architectures, making the solution adaptable to various deployment scenarios while maintaining reliable search performance
3Productivity
If correlation searching is performed on data streams, then data analysis can be enabled, but computing and memory resources are excessively consumed
Solution Approach 1:
The patent performs preliminary action by pre-processing data tuples and pre-establishing the graph structure before actual correlation searching is needed. Tuples are extracted and organized into the graph structure in advance, with edges representing correlations predetermined based on query requirements. This preliminary organization enables fast traversal and search operations when data analysis is performed, significantly reducing the computing and memory resources required during actual query execution
Data Source
AI summary
A method, system, and computer program product for graph-based searching for one or more data streams is disclosed. A computer-implemented method comprises extracting a plurality of tuples from one or more data streams. The method further comprises generating a graph for the plurality of tuples in which a node represents a tuple of the plurality of tuples and an edge represents a correlation between the node and another node, and the edge is generated based at least partly on one or more predetermined queries for the one or more data streams. The method further comprises traversing the graph based on the one or more predetermined queries. Accordingly, embodiments of the present disclosure can improve the search speed by use of the graph-based searching for one or more data streams.