Cluster Query Processing for Data Stream Load Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data Stream Management Systems face high processing loads due to handling large volumes of dynamic data streams and numerous user queries, particularly in applications like network monitoring and mobile position sensing, where precise and real-time data processing is demanding.
Innovation Solution
The system employs a cluster query processing apparatus that generates clusters based on feature vectors from collected queries, extracts representative queries, and maps user queries to these clusters for efficient processing, reducing the load by executing representative queries instead of individual user queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system processes each user query individually, then query accuracy is maintained, but processing load increases significantly
Solution Approach 1:
The patent merges similar queries into clusters based on feature vector similarity. Queries with comparable characteristics are grouped together and processed using a single representative query, reducing the total number of processing operations while maintaining result accuracy for all queries in the cluster.
Solution Approach 2:
The system creates representative queries that copy the essential characteristics of multiple similar queries. Instead of processing each original query separately, the system executes the representative query once and reuses its results for all queries in the cluster, significantly reducing processing load.
2Productivity
If the system executes representative queries for clusters, then processing load is reduced, but query execution time may increase
Solution Approach 1:
The system performs preliminary actions by pre-computing feature vectors for queries and organizing them into clusters before actual execution. This preparation work is done in advance, allowing the representative queries to be executed efficiently without adding significant latency during the actual query processing phase.
3Productivity
If the system clusters queries based on feature vectors, then query processing efficiency improves, but system complexity increases
Solution Approach 1:
The system introduces feature vectors as an intermediary representation between raw queries and the clustering mechanism. By converting queries into standardized feature vectors, the system simplifies the comparison and clustering process, making the overall system more manageable despite the added processing step.
Data Source
AI summary
An apparatus and method for processing a data stream using a cluster query, are provided. Collected queries are clustered into a predetermined vector space based on a feature vector of the collected queries. In response to a query received from a user, the received query is classified to a cluster and may be replaced with a centroid query of the cluster to which the received query belongs. The data stream processing apparatus processes the centroid query and provides an approximate result to the user.