Data query method and device for parallel database

Abstract

The invention discloses a data query method and device for a parallel database. The method comprises the steps of: respectively performing target data grouping aggregation on a target data table according to corresponding associated fields between the target data table and other data tables on each database node; respectively performing data re-partitioning on corresponding grouping aggregation results and corresponding other data tables in a Hash manner according to the corresponding associated fields on each database node; collecting data re-partitioning results of the grouping aggregation results and data re-partitioning results of the other data tables on each database node into a target database node; and performing target data connecting aggregation on the data re-partitioning results of the grouping aggregation results and the data re-partitioning results of the other data tables on the target database node. According to the method provided by the invention, data aggregation query can be realized, meanwhile, the parallelism of the query is improved, the resource utilization rate of a cluster is increased, the network cost is reduced, and the query performance is improved.

Description

technical field [0001] The invention relates to the field of parallel databases, in particular to a data query method and device for parallel databases. Background technique [0002] With the advent of the era of big data, compared with traditional data analysis, we now encounter greater challenges, on the one hand, the explosive growth of data volume, and on the other hand, the increase of data types. Faced with these challenges, Hadoop (a distributed system infrastructure developed by the Apache Foundation) emerged as the times require to solve the problem of offline data analysis, but for real-time data analysis requirements, Hadoop cannot meet the requirements of real-time data analysis due to its own characteristics. Therefore, parallel databases are still the main tool for real-time structured data analysis. [0003] In a parallel database system, aggregation and association query are the main methods for data analysis, and most of the analysis will involve the connec...

AI Technical Summary

Problems solved by technology

In order to ensure the correct determination of the result set, the current parallel database executes the aggregation query mainly by aggregating the data to a node for aggregation, but this method also brings a problem that the data aggregation process will greatly increase the network The communication overhead, which ultimately affects the performance of the query

[0005] For the above-mentioned problems in related technologies, no effective solution has been proposed yet

Images