An olap query optimization method in memory-flash hybrid storage mode

Abstract

The invention relates to an OLAP (On-Line Analytical Processing) query optimization method in a memory and flesh memory hybrid storage mode, which comprises the following steps that a flash-aware storage model is adopted for OLAP storage, division is carried out between a relatively small DRAM (Dynamic Random Access Memory) and a relatively big flash memory according to locality of data access, and storage optimization is carried out two stages of heterogeneous memories; array storage is adopted for memory OLAP, each attribute line is stored in a continuous array unit, conventional connection operation is simplified into array index access (AIR), and OLAP query processing of an AIR algorithm is carried out, wherein AIR is array index access; OLAP query processing on the basis of array storage and the AIR algorithm is decomposed into three sequential data access processes; a designated measurement value of a flash storage measurement line is accessed to by a selection vector; K keyword connection bitmaps are optimally stored in two stages of DRAM-flash storage on the basis of keyword-based bitmap continuous indexing to form a two-stage connection bitmap indexing structure. According to the invention, storage cost performance of the memory, use efficiency of the memory and a CPU (Central Processing Unit) and data storage efficiency can be improved; the OLAP query optimization method can be widely applied to universal OLAP application scenes.

Description

technical field [0001] The invention relates to a storage optimization and OLAP (Online Analytical Processing) query optimization method in the field of databases, in particular to a two-level DRAM (Dynamic Random Access Memory) and Flash (flash memory) suitable for the memory database integrated machine platform OLAP query optimization method under storage-based memory-flash hybrid storage mode. Background technique [0002] In-memory analysis processing (in-memory OLAP) is an important technology for real-time analysis and processing of big data. With the support of large memory and parallel processing capabilities of multi-core processors, in-memory OLAP has excellent real-time analysis and processing capabilities, but compared to other storage devices, such as flash , disk, etc., memory is still a very expensive storage medium, and its storage energy consumption is an order of magnitude higher than that of flash (DRAM: ~100mW / GB, NAND flash: 1-10mW / GB), memory OLAP analy...

AI Technical Summary

Problems solved by technology

On the one hand, the application of high-speed flash memory cards provides cheap secondary storage expansion capabilities for expensive memory storage, but on the other hand, high-speed flash memory cards are mainly used as extended caches for databases on flash, expanding the capacity of memory caches (buffers), However, it has not been combined with in-memory OLAP storage optimization and query processing optimization technology, and has not implemented flash-aware OLAP optimization technology at the OLAP algorithm level

[0003] The current analytical in-memory database mainly uses DRAM as the main storage device, and flash is used as a backup storage or disk cache to replace the disk. Flash has not yet been incorporated into the OLAP algorithm design of the memory.

Images