Data storage method, device and device

A database and data technology, applied in the field of data warehousing, can solve the problems of fixed performance, high IO pressure affecting data warehousing performance, and warehousing performance, and achieve the effect of reducing IO pressure, ensuring performance, and reducing quantity
CN108628898BActive Publication Date: 2021-04-23HANDAN BRANCH OF CHINA MOBILE GRP HEBEI COMPANYLIMITED +1
5 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Patent Type
Patents(China)
Current Assignee / Owner
HANDAN BRANCH OF CHINA MOBILE GRP HEBEI COMPANYLIMITED
Publication Date
2021-04-23

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

A method, device and equipment for data storage, including: starting a high-speed Huffman Huffman compression algorithm including a quick sort algorithm to perform first data compression on written data to generate a first compressed file; the first compressed file write After entering the Hbase database table, call the double compression algorithm to realize the second data compression of the first compressed file, and output the second compressed file; based on the second compressed file, realize Hadoop distributed file system HDFS data storage. After adopting the embodiment of the present invention, the IO pressure of the network and the disk can be reduced, and the performance of data storage can be ensured.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the field of computers, in particular to a method, device and equipment for data storage. Background technique

[0002] According to the research results of the Internet Data Center (IDC) for many years, the global data volume doubles approximately every 2 years, and the annual data volume increases exponentially. The data growth rate conforms to Moore's law. It is estimated that by 2020, the global total data volume will be Reach 35ZB. How to effectively collect, load, analyze and process these massive data has become an important link and basis for big data applications.

[0003] In order to realize the rapid processing of massive data, an important prerequisite is to realize the rapid storage of massive data. However, as the volume of data becomes larger and larger, the pressure on the IO performance of the network and disk in the process of data storage increases sharply. In the existing network And under the performance ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More