Dynamic Window Management for Stream Processing Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional Data Stream Management Systems (DSMS) lack the capability for semantic reasoning on dynamic data, leading to erroneous results from ad-hoc and registered queries due to inadequate window sizing and data selection for processing dynamic data streams from sensors.
Innovation Solution
A system and method that includes a processor and memory with modules for receiving datasets from sensors, resizing windows based on query types, and selecting maximum size windows to provide accurate data for resolving queries, using a data receiving module, logic manager, window manager, learning module, and data facilitating module to unify datasets and metadata, and prioritize data storage and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the window size is increased to include more data tuples for processing, then the accuracy of query results is improved, but the processing time and system resource consumption increase
Solution Approach 1:
The patent implements dynamic window sizing where the window size is not fixed but adapts based on query requirements and data characteristics. The system can resize windows dynamically during query execution to optimize between including sufficient data for accuracy and maintaining efficient processing speeds.
Solution Approach 2:
The system changes the parameter of window size based on different query types and requirements. For registered queries, a fixed window size may be used, while for ad-hoc queries, the window size can be adjusted dynamically. This parameter adaptation allows the system to balance data inclusion and processing efficiency for different scenarios.
2Productivity
If the window size is decreased to improve processing speed, then the processing time is reduced, but the query results become erroneous due to insufficient data
Solution Approach 1:
The system performs preliminary analysis of query requirements and data characteristics before executing queries. This allows the system to pre-determine appropriate window sizes and data subsets that will ensure query correctness while maintaining processing efficiency, avoiding the need to use excessively large windows.
Solution Approach 2:
The system incorporates feedback mechanisms that monitor query execution and results. When queries are executed on data subsets, the system can detect if results are erroneous or incomplete and adjust the window size or data selection accordingly, ensuring reliability while maintaining processing speed.
3Adaptability or versatility
If multiple windows are maintained to handle different query types, then the versatility of the system is improved, but the device complexity increases
Solution Approach 1:
The patent implements a universal window management system that can handle both registered queries and ad-hoc queries through a unified framework. The same window manager and data subset selection mechanisms serve multiple query types, reducing the need for separate specialized components for each query type while maintaining versatility.
Solution Approach 2:
The system segments the window management functionality into modular components that can be independently configured and reused. This includes separate modules for window creation, resizing, data subset selection, and query execution, which reduces overall system complexity by making each component simple and focused while allowing flexible combination for different query types.
4Quantity of substance
If data is continuously stored in windows for all possible queries, then the completeness of data for analysis is improved, but the memory consumption and storage requirements increase
Solution Approach 1:
The system extracts and stores only the necessary data subsets required for specific queries rather than maintaining complete data windows for all possible queries. The data subset selection module identifies and extracts only the relevant portions of data needed for each query type, significantly reducing storage requirements while maintaining data completeness for analysis purposes.
Solution Approach 2:
The system implements a discard and recover mechanism where data is not permanently stored in fixed windows but is selectively retained and discarded based on query requirements. When queries are executed, the system recovers the necessary data subsets from storage, processes them, and then discards them, allowing efficient use of storage space while ensuring data availability when needed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is a system and method for providing data required for resolving a query. A data receiving module may receive a dataset captured by a plurality of sensors. A logic manager module may create a space for storing the dataset in a window of a plurality of windows. The logic manager module may further receive the query from a user. In one aspect, the query may either be registered query or an ad-hoc query. A window manager module may determine the data based upon the query. The window manager module may further resize the window when the query is the registered query. A learning module may select a maximum size window having a maximum size amongst the plurality of windows when the query is the ad-hoc query. A data facilitating module may provide the data in order to resolve the query.