Columnar storage technology-based real-time data compression algorithm

A real-time data, columnar storage technology, applied in electrical digital data processing, special data processing applications, computing, etc., can solve problems such as low compression efficiency, and achieve the solution of huge storage space occupation, reducing storage space occupation, and saving deployment costs. Effect

Active Publication Date: 2017-09-22
浙江正泰中自控制工程有限公司
View PDF4 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The revolving door algorithm is better at compressing real-time data that has many data segments with the same changing trend, but for real-time data that is mainly jagged, the compre

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Columnar storage technology-based real-time data compression algorithm
  • Columnar storage technology-based real-time data compression algorithm
  • Columnar storage technology-based real-time data compression algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] The present invention will be further described in detail below in conjunction with the drawings.

[0039] This specific embodiment is only an explanation of the present invention, and it is not a limitation to the present invention. After reading this specification, those skilled in the art can make modifications to this embodiment without creative contribution as needed, but as long as the protection of the present invention The scope is protected by patent law.

[0040] A real-time data compression algorithm based on columnar storage technology. The algorithm includes steps:

[0041] (1) Read in real-time data packet Xi, i=1, 2,..., n, Xi is the i-th corresponding data value;

[0042] (2) Let △F be the precision value, if for the sequence Xi (a≤i≤b) in the column packet, there is a function Fi=f(Ti) defined on [Ta, Tb], such that:

[0043] |Xi-Fi|≤△F

[0044] |X(a-1)-F(a-1)|>△F

[0045] |X(n+1)-F(n+1)|>△F

[0046] Then call the interval [Ta, Tb] as the compressible interval for ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a columnar storage technology-based real-time data compression algorithm. According to the algorithm, trial calculation and assessment of a variable-step calculation formula are completed for a sequence in a specific column package, search and parameter calculation of a unit calculation formula are covered, and real-time data column packages in compression division can be compressed, so as to solve the problem of huge storage space occupancy, decrease the hardware deployment quantity, downsize the deployment environment and save the deployment cost. The algorithm disclosed by the invention is capable of realizing real-time high-frequency and large-data volume data compression and storage, and greatly decreasing the storage space occupancy problem.

Description

Technical field [0001] The present invention relates to the technical field of data compression, in particular to a real-time data compression algorithm based on columnar storage technology. Background technique [0002] The data structure design of real-time database is divided into two parts: memory and disk. The disk part is divided into short-term history and long-term history. Both the memory part and the disk part are columnar storage structures, but the physical location of the storage is different. The difference between the short-term history part and the long-term history part is that the former has no data compression, while the latter has data compression. [0003] For the memory part, what we have adopted is read-write separation technology. This technique allocates two kinds of memory buffers for data reading and writing. Among them, the read area can only be used to read data, but not write data; the write area can only be used to write data, but cannot read data....

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 黄永忠徐兵张建强曹福国
Owner 浙江正泰中自控制工程有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap