A query optimization method based on join index in data warehouse

Abstract

The invention relates to a query optimization method based on a connection index in a data warehouse. The steps are: creating a connection index of a fact table foreign key based on a dimension table proxy key; creating a connection index update strategy; when there is a predicate condition on the dimension table, through the connection After the index directly accesses the dimension table records, it needs to judge the predicate, and refer to the bitmap connection index to generate the connection bitmap index through dynamic bitmap connection index calculation; when the selection rate on the dimension table is very low, the dimension table bitmap is further compressed to improve the dimensionality. The cache level of the table bitmap calculation; the calculation method of the reference bitmap connection index based on the coprocessor memory calculation is adopted, and only the array and the bitmap data structure are used to calculate the simple memory address access. The invention can effectively improve the computing performance of the bitmap index, and can be combined with high-performance coprocessors such as GPU and Xeon PhiTM to accelerate the computing performance of the index, and can be widely used in the field of data warehouses.

Description

technical field [0001] The invention relates to a query processing method in the field of various data warehouses, in particular to a query optimization method based on a join index in a data warehouse. Background technique [0002] A data warehouse is a complex data set based on a multidimensional data model. When processing a database-based OLAP (Relational OLAP, ROLAP) query, it is necessary to connect the fact table and multiple dimension tables to execute complex analysis and query commands. The performance of the connection operation has been the most important problem in analytical query processing (OLAP). A connection index is an index that creates a connection relationship between two or more table records through pre-connection, and the connection index records the address relationship of connection records between different tables. During query processing, the address of the connection record of the two tables can be directly obtained through the connection index...

AI Technical Summary

Problems solved by technology

Join index is an important technology to improve the performance of join operations between fact tables and dimension tables in data warehouses. Its main disadvantage is that the index storage space is expensive. When OLAP queries involve join operations between fact tables and multiple dimension tables, And when multiple attributes on the dimension table are involved in the query, the storage space cost of the join index will increase accordingly, and the increase in the number of values ​​on the dimension table attributes will also lead to an increase in the bitmap calculation cost in the bitmap join index

The characteristics of current data warehouse applications are transitioning from multi-dimensional query to high-dimensional query. OLAP query contains more and more join table and dimension table attributes, and the update frequency is getting higher and higher. Traditional join index technology faces huge storage and index maintenance. cost

Images