Query optimization method based on join index in data warehouse

Abstract

The invention relates to a query optimization method based on a join index in a data warehouse. The method comprises the following steps: creating a fact table foreign key join index based on a dimension table surrogate key; creating a join index update strategy; when a predicate condition exists on a dimension table, making a predicate judgment after direct access to dimension table record through the join index, and performing calculation through a dynamic bitmap join index by reference to a bitmap join index to generate a join bitmap index; when the selection rate on the dimension table is very low, further compressing a dimension table bitmap in order to raise the cache level of dimension table bitmap calculation; and calculating simple memory address access by only using an array-bitmap data structure with a reference bitmap join index calculation method based on co-processor memory calculation. Through adoption of the query optimization method, the calculation performance of the bitmap index can be effectively increased. Moreover, the query optimization method can be combined with high-performance co-processors such as GPU and Intel Xeon Phi TM to accelerate the index calculation performance, and can be widely applied to the field of data warehouses.

Description

technical field [0001] The invention relates to a query processing method in the field of various data warehouses, in particular to a query optimization method based on a join index in a data warehouse. Background technique [0002] A data warehouse is a complex data set based on a multidimensional data model. When processing a database-based OLAP (Relational OLAP, ROLAP) query, it is necessary to connect the fact table and multiple dimension tables to execute complex analysis and query commands. The performance of the connection operation has been the most important problem in analytical query processing (OLAP). A connection index is an index that creates a connection relationship between two or more table records through pre-connection, and the connection index records the address relationship of connection records between different tables. During query processing, the address of the connection record of the two tables can be directly obtained through the connection index...

AI Technical Summary

Problems solved by technology

Join index is an important technology to improve the performance of join operations between fact tables and dimension tables in data warehouses. Its main disadvantage is that the index storage space is expensive. When OLAP queries involve join operations between fact tables and multiple dimension tables, And when multiple attributes on the dimension table are involved in the query, the storage space cost of the join index will increase accordingly, and the increase in the number of values ​​on the dimension table attributes will also lead to an increase in the bitmap calculation cost in the bitmap join index

The characteristics of current data warehouse applications are transitioning from multi-dimensional query to high-dimensional query. OLAP query contains more and more join table and dimension table attributes, and the update frequency is getting higher and higher. Traditional join index technology faces huge storage and index maintenance. cost

Images