Continuous Query Executor for Stream Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current stream processing systems face inefficiencies due to high computational overhead from frequent database query setup and teardown, and they fail to leverage mature data management capabilities of database management systems (DBMS), leading to bottlenecks in operational business intelligence applications.
Innovation Solution
Implementing a single, long-standing continuous query in a DBMS that includes a stream data source as a function, allowing for continuous execution without repeated setup and teardown, utilizing a query executor and function manager to process unbounded stream data efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a DSMS executes frequent database queries on chunks of stream elements, then stream processing capability is provided, but computational overhead increases significantly
Solution Approach 1:
The system performs preliminary actions by establishing a single continuous query before data processing begins. The query remains active and ready to process stream elements as they arrive, eliminating the need for repeated query setup and teardown operations that would otherwise be required for each chunk of data.
Solution Approach 2:
The patent implements continuity of useful action by maintaining a single long-standing query that continuously processes stream elements without interruption. The query executor keeps the query active indefinitely, allowing seamless processing of data chunks as they flow through the stream, rather than repeatedly starting and stopping query execution.
2Adaptability or versatility
If separate DSMS and DBMS are deployed to access data from stream and database sources, then data access capability is provided, but system complexity increases
Solution Approach 1:
The patent merges the DSMS and DBMS into a single integrated system. The DBMS is extended to include native stream processing capabilities through the query executor and function manager, allowing both database and stream data sources to be accessed through a unified query interface, thereby reducing system complexity while maintaining full data access capability.
Solution Approach 2:
The extended DBMS achieves universality by providing multi-functional capabilities to handle both traditional database queries and stream processing operations through a single system. The query executor can process queries against persistent database data as well as continuous stream data sources, eliminating the need for separate specialized systems.
3Reliability
If streamed data is stored in a database first before processing, then data management capability is provided, but processing time increases
Solution Approach 1:
The system eliminates the batch store-then-process approach by implementing continuous processing where stream elements are processed immediately as they arrive through the active query. The query executor continuously fetches and processes stream elements in real-time, maintaining data management reliability while eliminating the time loss associated with batch writing to disk and subsequent reading for processing.
4Use of energy by moving object
If a single long-standing query is executed continuously, then computational overhead is reduced, but query control complexity increases
Solution Approach 1:
The patent introduces a query executor as an intermediary component that manages the single continuous query. This specialized executor handles the complexity of maintaining the query's active state, coordinating with the function manager to process stream elements, and managing the query lifecycle, thereby reducing the apparent control complexity for users while enabling the computational benefits of continuous execution.
Data Source
AI summary
Methods, database management systems (“DBMS”) and computer-readable media are provided for processing unbounded stream data using a traditional DBMS. Execution of a query that includes a data stream as a data source may be initiated. Tuples may be processed in accordance with the query as the tuples are received through the data stream until an indication is received that execution of the query should cease.