Distributed OLAP analysis method and system based on pre-computation

An analysis method and analysis system technology, applied in the field of distributed OLAP analysis method and system based on pre-computing, can solve problems such as the inability to efficiently analyze and process massive data, reduce disk IO operations, improve construction efficiency, and avoid repeated calculations. Effect

Inactive Publication Date: 2017-10-27
SOUTH CHINA UNIV OF TECH
View PDF3 Cites 26 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Different from the traditional online analysis and processing method, the distributed OLAP analysis method and system based on pre-computation uses cluster parallel computing to analyze multidimensional data on the basis of Hadoop, thereby greatly improving the online analysis and processing capability and solving the problem that traditional analysis methods cannot solve. Efficient analysis and processing of massive data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed OLAP analysis method and system based on pre-computation
  • Distributed OLAP analysis method and system based on pre-computation
  • Distributed OLAP analysis method and system based on pre-computation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] The present invention will be further described below in conjunction with examples of implementation.

[0035] see figure 1 As shown, the pre-computing-based distributed OLAP analysis method provided in this embodiment is specifically: first, build a Hadoop platform on the server cluster, and build a distributed data warehouse on the basis of HDFS; then select facts based on the distributed data warehouse Tables and a set of associated dimension tables construct a data model, define a data cube according to the data model; then start the data cube precomputation task for a given data cube, submit the job to the Hadoop cluster to run the data cube preconstruction, and in the construction process In real-time monitoring of the job running status, timely grasp the job running status, save the calculated intermediate results in the distributed key-value storage system for subsequent analysis and query; then convert the user's multi-dimensional operations into MDX statements...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention discloses a distributed OLAP analysis method and system based on pre-computation. The method mainly comprises: constructing a data model based on the distributed data warehouse, and defining data cubes according to the data model; starting pre-computation tasks for the given data cubes so as to pre-build the cubes in a parallel computation manner, and storing results in a distributed key storage system; through a series of steps, converting multi-dimensional analysis operations into key-value query operations on the data cubes, directing obtaining analysis results from the built cubes, and displaying the results in the form of rich and diversified charts; and using the NoSQL to carry out cache optimization on the OLAP query operation. According to the method and system disclosed by the present invention, the powerful processing performance of the Hadoop platform is fully exerted, and data cubes are pre-built, the problem that the query is slow due to that the traditional method needs a large amount of computation from the original data in each query is overcome, so that OLAP analysis efficiency and system performance are improved.

Description

technical field [0001] The present invention relates to the technical field of big data analysis, in particular to a distributed OLAP analysis method and system based on pre-calculation. Background technique [0002] With the continuous development and improvement of information technology, computer science and technology are widely used in all walks of life, and at the same time, massive amounts of data have been accumulated. How to extract effective information from these massive data and fully tap the value contained in it has become an important issue that many management decision makers are increasingly concerned about. For government agencies, big data analysis technology can improve the level of government information management. Through in-depth analysis of the massive data accumulated by governments at all levels, it can provide reference for the formulation of government policies and guidelines, and improve government management efficiency and macro decision-making...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/24534G06F16/248G06F16/283
Inventor 林育蓓古振威张星明梁桂煌陈霖吴世豪
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap