Concurrency OLAP (On-Line Analytical Processing)-oriented test data hierarchy cluster query processing system and method

Abstract

The invention discloses a concurrency OLAP (On-Line Analytical Processing)-oriented test data hierarchy cluster query processing system and a concurrency OLAP (On-Line Analytical Processing)-oriented test data hierarchy cluster query processing method. The I / O price of a test data access in the concurrency OLAP is reduced by a series of DBMS (Data Base Management System) technologies of hierarchy test count, cluster, an index organization chart, large object-based storage and the like, so the processing capacity of concurrency query is imporved. According to the system and the method, the concurrency query processing and optimization techniques can be realized in a database management system by aiming at the I / O performance and concurrency OLAP processing performance, and the oriented I / O performance is supported to optimize and set a concurrency OLAP so as to process a load, thereby improving the predictable performance displayed by test data waveforms, and realizing the accelerated processing of the test data query of the large-scale concurrency OLAP.

Description

technical field [0001] The present invention relates to a database query processing system and method, in particular to a large-scale concurrent online analysis processing (OLAP)-oriented test data query processing system and method, which reduces the cost of reading in concurrent OLAP and improves concurrent query processing capabilities The method belongs to the technical field of test data management. Background technique [0002] Today, data processing can be roughly divided into two categories: on-line transaction processing (OLTP for short) and On-Line Analytical Processing (OLAP for short). OLTP is mainly about day-to-day transaction processing, such as bank transactions. OLAP is designed to meet specific query and reporting requirements for decision support or multidimensional environments. Many applications including OLAP drive the emergence and development of data warehouse technology; and data warehouse technology in turn promotes the development of OLAP technol...

AI Technical Summary

Problems solved by technology

[0004] However, the test data often has a data volume of more than 1G for a single physical quantity. During the test process, thousands of physical quantity values ​​are often generated at the same time point. When a large number of concurrent queries are performed, the overall data volume of concurrent query processing will be reduced If it is too large and the execution time is too long, the performance is unacceptable, and it is impossible to obtain acceptable performance of concurrent query of test data

In addition, in traditional disk databases, the storage of data heap tables is often controlled by the DBMS itself, and it is difficult for users to interfere with the specific physical location of each tuple storage. The physical storage of data that is often accessed at the same time is often stored in different physical data blocks. , which further degrades disk I / O performance

[0005] In the case of massive concurrent OLAP test data query, there are three key technical challenges faced by concurrent query processing: First, the test data of each physical quantity is composed of values ​​at multiple time points, often reaching tens of millions or even hundreds of millions. User access is often a full picture of one or more physical quantities in the entire test, so the resulting I / O cost is too high, and in the case of large-scale concurrent queries, the performance becomes unacceptable; the second is the usual storage form of commercial DBMS ( Heap table) It is difficult to physically store continuous test data on the same or adjacent data blocks to eliminate or reduce I / O contention caused by fact table scanning; the third is the usual storage form of commercial DBMS (heap The disorder of the table) makes it impossible to save test data with a simple fixed sampling rate. The time column must be added, so that the storage space is more than doubled (together with the index), and the storage space overhead needs to be reduced as much as possible while taking into account the query performance.

Images