Approximate Query Processing via Machine Learning Plan Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the big data environment, existing query processing methods incur high costs and are inefficient, making it difficult for users to quickly obtain desired results, and lack sufficient means to express user requirements for accuracy and timeliness in approximate query processing.
Innovation Solution
A method and apparatus for processing approximate queries by extending query languages to include user-defined accuracy and timeliness requirements, generating and selecting optimal execution plans using machine learning models, and performing query processing on either raw or synthesized data based on these plans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exact query processing is performed by accessing raw data, then query accuracy is improved, but query processing time and cost increase significantly
Solution Approach 1:
The system pre-generates multiple candidate execution plans before query execution, including both exact and approximate query plans. This preliminary preparation allows the query processor to quickly select an appropriate plan without performing time-consuming analysis during actual query execution, thus reducing query processing time while maintaining accuracy options.
Solution Approach 2:
The system generates more execution plans than traditionally necessary, including multiple approximate query plans with varying accuracy levels. By providing excessive candidate plans, the system enables faster selection of an appropriate plan based on user requirements, trading off some generation overhead for significant query execution time savings.
2Productivity
If approximate query processing is used to reduce processing time, then query processing speed is improved, but query accuracy decreases
Solution Approach 1:
The system dynamically selects between exact and approximate query processing plans based on user-specified accuracy requirements and timeliness constraints. The query processor adapts its behavior by choosing the most appropriate execution plan from multiple candidates, allowing flexibility to balance accuracy and speed according to specific query needs rather than using a fixed approach.
Solution Approach 2:
The system changes the accuracy parameter of query results by selecting from multiple execution plans with different accuracy characteristics. Users can specify their desired accuracy level, and the system adjusts the query processing approach accordingly, transforming the static accuracy parameter into a dynamic, user-controllable variable that balances speed and precision.
3Productivity
If multiple candidate execution plans are generated and evaluated, then optimal plan selection is improved, but system complexity increases
Solution Approach 1:
The system uses cost estimation functions that provide approximate rather than exact cost calculations for execution plans. These inexpensive, rapid estimations allow the system to evaluate multiple candidate plans without incurring the high computational costs of precise analysis, enabling efficient plan selection while keeping the system relatively simple.
Solution Approach 2:
The system pre-calculates and stores cost estimation parameters and statistics about available execution plans before query processing. This preliminary preparation reduces the complexity of real-time plan evaluation by having reference information ready, allowing the query optimizer to make decisions based on pre-computed data rather than performing complex calculations during query execution.
4Adaptability or versatility
If user requirements for accuracy and timeliness are expressed in query language, then query specificity is improved, but query language complexity increases
Solution Approach 1:
The system uses an extended SQL query language that incorporates multiple functions within a familiar syntax framework. The same query language structure handles both exact and approximate queries, allowing users to express various requirements (accuracy, timeliness) using extended but familiar SQL constructs rather than requiring entirely new query languages for different query types.
Data Source
AI summary
Provided are a method and apparatus for processing an approximate query based on a machine learning model. When receiving a user query through an approximate query language extension interface, a processing apparatus parses a user query. The user query is an extended query form that includes information according to a user requirements. The processing apparatus generates a basic execution plan based on a parsing result and generates a plurality of executable candidate execution plans based on the basic execution plan. Then, an optimal final execution plan reflecting user requirements is selected from among the plurality of executable candidate execution plans, and query processing is performed on the user query based on a final execution plan.