Data storage system and storage method thereof

Abstract

The invention discloses a data storage system and a storage method thereof. The storage system comprises a data acquisition unit, a data processing unit, a data storage unit, a data application unit and a display unit, the data acquisition unit is used for acquiring subsystem data from the business subsystem; the data processing unit is used for preprocessing the subsystem data to obtain preprocessed data and sending the preprocessed data to the data storage unit through a preset interface; a Kylin cluster is built on the data storage unit, and a storage database in the data storage unit comprises MySQL (Structured Query Language); the data storage unit is used for constructing a multi-dimensional table of the pre-processed data through the Kylin cluster and storing the pre-processed data and result data pre-queried and calculated through the Kylin cluster to MySQL; the data application unit is used for constructing an application program according to the multi-dimensional table and the result data stored in the MySQL; and the display unit is used for displaying the application program, the preprocessing data, the multi-dimensional table and the result data.

Description

technical field [0001] The present application relates to the field of databases, in particular to a data storage system and a storage method thereof. Background technique [0002] Data is an important product of the development of the modern Internet. With the business development of enterprises, a large amount of data is scattered in various sub-business systems, and the volume of data continues to increase, and the value of hidden data also increases. How to store large-scale data is a hot spot for researchers. [0003] At present, the more common large-scale data storage system is based on HBase+SparkSQL. The bottom layer of this system is the Hadoop ecosystem. By using the SparkSQL large-scale data computing engine on HBase, it is compatible with and optimizes the query performance of Hive. In this system, SparkSQL reads data into memory, and improves query and calculation performance through memory. If the data can be basically read into the memory, Spark's memory c...

AI Technical Summary

Problems solved by technology

[0004] This application provides a data storage system and its storage method, which solves the technical problem that the performance of the existing data storage system will be greatly reduced when Spark frequently reads and writes disks for ultra-large-scale data.

Images