Runtime Join Pruning via Range Bloom Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional relational database management systems require extensive computing and storage resources, have limited scalability, and are costly to maintain, especially in terms of hardware and infrastructure, while also being susceptible to data loss during power outages or disasters.
Innovation Solution
The implementation of a cloud-based database system that utilizes a network-based database system with compute service managers, execution platforms, and databases, enabling efficient resource management, scalability, and reduced costs by leveraging cloud infrastructure and metadata-based pruning techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional relational database management systems are used, then data storage and retrieval functionality is provided, but extensive computing and storage resources are required and scalability is limited
Solution Approach 1:
The patent introduces a cloud-based database system as an intermediary layer between users and physical database infrastructure. This cloud system manages database operations remotely, allowing users to access and manipulate database tables without directly provisioning or managing the underlying computing and storage resources, thereby enabling scalability while reducing local resource requirements
Solution Approach 2:
The patent replaces traditional mechanical database management approaches (local hardware provisioning, on-premises server management) with cloud-based virtualized resource management. This substitution allows dynamic allocation and scaling of computing and storage resources through software-controlled cloud infrastructure rather than fixed physical systems
2Quantity of substance
If traditional database infrastructure is provisioned, then data storage capability is achieved, but significant capital investment in hardware and infrastructure is required
Solution Approach 1:
The cloud-based database system provides universal access to data storage and retrieval capabilities across multiple users and applications through a single shared infrastructure. This multi-functional platform serves diverse database needs without requiring each user to invest in separate hardware, thereby reducing overall capital expenditure while maintaining adequate storage capacity
Solution Approach 2:
The patent implements virtualized copies of database environments in the cloud, allowing users to access replicated database instances without owning physical storage infrastructure. This copying approach enables data storage capability through virtual representations rather than direct hardware investment
3Reliability
If full table scans are performed during join operations, then complete data coverage is achieved, but query execution time increases significantly
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing metadata about database tables, including minimum and maximum values for each column. This advance preparation enables the system to quickly determine which table partitions are relevant to a query without scanning entire tables, thus maintaining data coverage while reducing query execution time
Solution Approach 2:
Instead of performing excessive full table scans, the patent applies partial scanning by using metadata to identify and scan only the necessary table partitions that contain relevant data. This partial action approach achieves sufficient data coverage for join operations while significantly reducing the amount of data that must be processed
Data Source
AI summary
The subject technology receives a query, the query including a statement for performing a join operation on a first table and a second table. The subject technology executes, by an execution node using a generated query plan, the query. The subject technology performs, during execution of the query by the execution node, a runtime range pruning process. The subject technology determines a set of range sets for pruning, each range set including a set of columns from the first table, the set of columns being removed from undergoing a read operation as part of executing the query. The subject technology determines, based on a range bloom vector, a set of rows in a particular range set of the first table to avoid scanning in connection with the read operation. The subject technology performs the read operation based a remaining set of rows.