Compression method of historical database

A compression method and database technology, which is applied in the compression field of historical databases, can solve problems such as complex algorithm structures, and achieve the effects of simple principles, easy operation of principles, and convenient code implementation

Active Publication Date: 2016-10-12
NANJING GUODIAN NANZI WEIMEIDE AUTOMATION CO LTD
View PDF5 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the structure of the algorithm is still relatively complicated, and the difficulty of code

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Compression method of historical database
  • Compression method of historical database
  • Compression method of historical database

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0033] The technical solutions of the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention, but the examples cited are not intended to limit the present invention.

[0034] A compression method of historical database, such as figure 1 As shown, including the following steps:

[0035] Step 1. Convert a row database to a column database. The methods of database row conversion are different, but the basic idea is to treat the original row database as a table with M rows and N columns and rotate the table 90 The conversion of ranks and columns forms a table with N rows and M columns. However, when actually converted to a columnar database suitable for compression, there are some problems in operation, the main problem is the limitation of the number of columns in sub-tables and general databases;

[0036] 1a) Sub-table:...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a compression method of a historical database. The compression method is characterized by comprising the following steps that 1, a line database is converted into a column-oriented database, and sharding and assembling are conducted; 2, piecewise linear fitting lossy compression is conducted; 3, data file sorting is conducted; 4, data file lossless compression is conducted, index sorting is conducted, and a complete two-stage compressed column-oriented database is formed. According to the characteristic of an existing database, compression is conducted by two stages, a piecewise linear fitting lossy compression mode is adopted firstly, a digital dictionary lossless compression mode is adopted later, data is compressed, the occupied space is saved, the requirement of a modern comprehensive monitoring system on longer time duration of saved historical data is met, and the error is kept within a certain error precision range.

Description

technical field [0001] The invention relates to a compression method of a historical database. Background technique [0002] The concurrency scale of data measurement points in modern integrated monitoring systems is getting larger and larger, and the frequency of data collection is getting higher and higher. In many application scenarios, it is necessary to store massive concurrent and high-frequency time-series data. The scale of raw data that needs to be stored every day It can reach hundreds of gigabytes, so the data needs to be compressed and stored. [0003] The system generally adopts lossy data compression algorithm and lossless compression algorithm for historical data compression. Common lossy data compression algorithms are divided into three categories: signal transformation method, vector quantization method, and piecewise linear method. The piecewise linear method is simple to implement and is more suitable for the compression of historical data. There are ma...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/221
Inventor 张迎金舒薛亮周金国董胜刚戴宏斌白忠贺杨荣昌
Owner NANJING GUODIAN NANZI WEIMEIDE AUTOMATION CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap