Query Data Splitting for Federated Database Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated data environments, processing a large number of small queries leads to inefficient resource usage and slow performance due to excessive time and resource consumption during data searches in databases.
Innovation Solution
A method and system for query data splitting that detects and analyzes queries to identify relationships between them, using a query tree engine to determine whether one query is a generalized version of another, and exploiting results of general queries to benefit less general queries, with components like a cache, replicator, and router to optimize query processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of small queries are processed in federated data environments, then data search completeness is improved, but resource consumption increases and performance deteriorates
Solution Approach 1:
The patent combines multiple similar queries into a single query by detecting query relationships and grouping them. The query processor identifies when queries are duplicates or subsets of each other and merges them, reducing the total number of queries sent to the database while maintaining complete search results.
Solution Approach 2:
The patent performs preliminary query analysis and relationship detection before executing queries against the database. By analyzing query patterns, detecting duplicates, and identifying subset relationships in advance, the system prevents redundant queries from being executed, thereby reducing resource consumption while maintaining search completeness.
2Measurement precision
If a large number of small queries are processed in federated data environments, then data search completeness is improved, but processing time increases
Solution Approach 1:
The patent merges multiple similar queries into a single query execution, reducing the total time spent on query processing. By detecting when queries are duplicates or subsets and combining them, the system executes fewer queries sequentially, thereby reducing overall processing time while maintaining complete search results.
Solution Approach 2:
The patent performs preliminary query relationship detection and optimization before execution. By analyzing query patterns, identifying duplicates, and determining subset relationships in advance, the system eliminates redundant query executions, significantly reducing processing time while ensuring data search completeness is maintained.
3Productivity
If query relationships are detected and analyzed using a query tree engine, then query optimization is improved, but system complexity increases
Solution Approach 1:
The patent implements a query tree engine that serves multiple functions: storing query patterns, detecting relationships, identifying duplicates, and optimizing query execution. By consolidating these functions into a single multi-functional component, the system achieves improved query processing efficiency without proportionally increasing overall system complexity.
Data Source
AI summary
Implementations for query data splitting are disclosed. In one example, the method includes receiving, by a processing device, a first query and a second query, determining that the second query is a subset of the first query, determining a first set of attributes associated with the first query and a second set of attributes associated with the second query, and determining, by the processing device, whether to submit the second query or the first query to a database in view of the first set of attributes and the second set of attributes.