Distributed OLAP analysis method and system based on pre-computation

Abstract

The present invention discloses a distributed OLAP analysis method and system based on pre-computation. The method mainly comprises: constructing a data model based on the distributed data warehouse, and defining data cubes according to the data model; starting pre-computation tasks for the given data cubes so as to pre-build the cubes in a parallel computation manner, and storing results in a distributed key storage system; through a series of steps, converting multi-dimensional analysis operations into key-value query operations on the data cubes, directing obtaining analysis results from the built cubes, and displaying the results in the form of rich and diversified charts; and using the NoSQL to carry out cache optimization on the OLAP query operation. According to the method and system disclosed by the present invention, the powerful processing performance of the Hadoop platform is fully exerted, and data cubes are pre-built, the problem that the query is slow due to that the traditional method needs a large amount of computation from the original data in each query is overcome, so that OLAP analysis efficiency and system performance are improved.

Description

technical field [0001] The present invention relates to the technical field of big data analysis, in particular to a distributed OLAP analysis method and system based on pre-calculation. Background technique [0002] With the continuous development and improvement of information technology, computer science and technology are widely used in all walks of life, and at the same time, massive amounts of data have been accumulated. How to extract effective information from these massive data and fully tap the value contained in it has become an important issue that many management decision makers are increasingly concerned about. For government agencies, big data analysis technology can improve the level of government information management. Through in-depth analysis of the massive data accumulated by governments at all levels, it can provide reference for the formulation of government policies and guidelines, and improve government management efficiency and macro decision-making...

AI Technical Summary

Problems solved by technology

[0005] Different from the traditional online analysis and processing method, the distributed OLAP analysis method and system based on pre-computation uses cluster parallel computing to analyze multidimensional data on the basis of Hadoop, thereby greatly improving the online analysis and processing capability and solving the problem that traditional analysis methods cannot solve. Efficient analysis and processing of massive data

Images