Systems and methods for optimizing database queries

Abstract

A relational database warehouse system with query optimization capabilities is described that allows for speedy identification of sets of records of interest from amongst tens of millions of records. The records of interest may include complex derived attributes, generated, at least in part, by aggregating data from a plurality of records in base data tables. In various embodiments, the query optimization capabilities allow the database warehouse system to identify conditions under which normal query execution may be replaced by one or more optimized execution methods, including, for example, eliminating unnecessary inner join operations on base data tables specified by a query, re-ordering the execution of group-by operations and left-outer join operations to greatly reduce the size of join tables produced while processing a query, and/or consolidating a set of segmentation queries for execution in one pass over the records of the database.

Description

FIELD OF THE INVENTION[0001]The invention relates generally to the field of database query optimization and, more specifically, to optimizations for batched queries on views in data warehouses.BACKGROUND OF THE INVENTION[0002]Databases have evolved from simple file systems to massive collections of data serving a variety of users and numerous distinct applications. Online database systems may be configured to receive an ongoing stream of new and updated information, to update their records accordingly, and to allow for relatively small queries of the up-to-date data in real-time. A database warehouse is typically a storage facility for large amounts of data that allows for a more in-depth analysis of the data, using derived and / or aggregated information based on the data to identify groups of records within the database that share common characteristics, for example.[0003]Some database warehouses are used for basic data that is relatively static over time. In such data warehouses, e...

AI Technical Summary

Benefits of technology

[0006]A database warehouse system with query optimization capabilities is described that allows for speedy identification of sets of records of interest from amongst a large number of records, such as tens of millions of records or more. The records of interest may include complex derived data values that may be presented in a view for use by a query and that may be generated, at least in part, by aggregating data from a plurality of records in base data tables. Views that present comprehensive access to the base data

Problems solved by technology

However, when a data warehouse stores data that is frequently updated, and when queries to the data warehouse demand very up-to-date information, it is not always feasible to pre-calculate and store the derived and aggregated data.

Furthermore, accessing and manipulating the massive views in memory in response to a query can take a prohibitively long amount of time.

Join operations are very expensive with respect to system memory and processing resources.

However, improper el

Images