Data online gathering method in cloud computing environment

Abstract

The invention relates to a data online gathering method in a cloud computing environment. The method comprises the steps of setting an SQL (Structured Query Language) query statement Q according to requirement of a user by an SQL interface module at an application interference layer, and transmitting the SQL query statement Q to a query processing layer; analyzing the received query statement Q through a grammar analyzing module, and transmitting an analyzing result to a MapReduce online performing plan module; starting MapReducer operation by the MapReduce online performing plan module if the analyzed query statement Q is query on account of a single form, starting two MapReduce operations by the MapReduce online performing plan module if the analyzed query statement Q is on account of query of a plurality of forms, processing a query request of the query statement Q through an online gathering processing layer according to one or two received MapReduce operations, and transmitting a processing result to the application interface layer for display. The data online gathering method can be widely applied to the cloud computing environment.

Description

technical field [0001] The invention relates to an online data gathering method, in particular to an online data gathering method under a cloud computing environment. Background technique [0002] Online aggregation was first proposed by Hellerstein et al. The earliest online aggregation research mainly focused on the realization of single-table online aggregation in relational databases. Subsequently, this problem has been studied to a certain extent in the field of relational databases. Extending the work of Hellerstein, Haas et al. provide methods for computing large sample-based confidence intervals and deterministic confidence intervals. Aiming at the online aggregation of multi-table joins, Haas et al. gave a series of ripple join algorithms. Ripple join is based on the nested join and hash join design in offline query processing, and its purpose is to get the estimated result as soon as possible under the premise of ensuring incremental calculation. Luo et al. impr...

AI Technical Summary

Problems solved by technology

This method assumes that the longer the processing time of the data block, the greater the aggregation value. However, this assumption is not true in all aggregation operations, and the implementation method is more complicated.

In addition, it only solves the single-table online aggregation problem composed of one MapReduce job, and does not consider the multi-table connection online aggregation based on multiple MapReduce jobs.

Images