Query Execution Plan Distribution for Data Stream Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data stream management systems face significant processing loads due to the large number of queries from mobile devices, particularly in services like traffic flow and health care monitoring, where existing systems struggle to efficiently handle continuous data streams and distribute query execution effectively between terminals and servers.
Innovation Solution
An apparatus and method that determine and execute a query execution plan by dividing it into parts for terminal and server execution, estimating operation costs based on terminal-related information such as CPU performance, memory, and network access speed, and combining results accordingly to optimize processing load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all queries are executed on the server, then query processing completeness is improved, but server processing load increases
Solution Approach 1:
The query execution plan is segmented into two parts: a first part executed by the terminal and a second part executed by the server. The determination unit divides the query execution plan based on operation cost estimation, assigning appropriate portions to each execution environment to balance processing load while maintaining complete query processing.
2Power
If query execution is distributed to terminals, then server processing load is reduced, but processing efficiency may deteriorate
Solution Approach 1:
The system applies local quality by making the terminal's processing capabilities heterogeneous and adaptive. Each terminal's processing capacity is evaluated based on its specific attributes (CPU performance, memory size, network access speed), and query execution plans are customized accordingly. This allows the system to optimize for local processing efficiency while distributing load.
Solution Approach 2:
The system changes parameters by dynamically adjusting the division point of the query execution plan based on terminal attributes. The determination unit estimates operation costs for different query operators and modifies the execution plan distribution parameters to optimize overall processing efficiency while reducing server load.
3Productivity
If query execution plan is customized per terminal, then processing optimization is improved, but system complexity increases
Solution Approach 1:
The terminal performs self-service by executing the first part of the query execution plan locally based on its own attributes. The terminal autonomously processes its portion of the query without requiring constant server intervention, reducing system complexity while maintaining processing optimization.
Solution Approach 2:
The server performs preliminary action by determining and distributing the customized query execution plan before actual query processing. The determination unit pre-calculates the optimal division point based on terminal attributes, so that during query execution, the terminal can directly follow the pre-optimized plan without real-time complex decision-making.
Data Source
AI summary
An apparatus and method for processing data stream are provided. A query execution plan for processing data stream is shared between the server and the terminal. The server builds a query execution plan and determines a part of the built query execution plan to be executed in the terminal. The terminal executes the part of the built query execution plan and transfers a result of the execution. The server executes the remaining part of the built query execution plan based on the execution result transferred by the terminal. The part of the query execution plan to be executed in the terminal is determined depending on the performance of the terminal.