Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A real-time data compression method based on columnar storage technology

A real-time data and columnar storage technology, which is applied in database indexing, structured data retrieval, database design/maintenance, etc., can solve problems such as low compression efficiency, achieve huge storage space occupation, miniaturized deployment environment, and save deployment cost effect

Active Publication Date: 2019-09-20
浙江正泰中自控制工程有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The revolving door algorithm is better at compressing real-time data that has many data segments with the same changing trend, but for real-time data that is mainly jagged, the compression efficiency is relatively low
The reason is that the revolving door algorithm is a linear fitting algorithm, and the measurement data collected from industrial equipment is rarely linear

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A real-time data compression method based on columnar storage technology
  • A real-time data compression method based on columnar storage technology
  • A real-time data compression method based on columnar storage technology

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] The present invention will be further described in detail below in conjunction with the drawings.

[0039] This specific embodiment is only an explanation of the present invention, and it is not a limitation to the present invention. After reading this specification, those skilled in the art can make modifications to this embodiment without creative contribution as needed, but as long as the protection of the present invention The scope is protected by patent law.

[0040] A real-time data compression method based on columnar storage technology, the method includes the steps:

[0041] (1) Read in real-time data packet Xi, i=1, 2,..., n, Xi is the i-th corresponding data value;

[0042] (2) Let △F be the precision value, if for the sequence Xi (a≤i≤b) in the column packet, there is a function Fi=f(Ti) defined on [Ta, Tb], such that:

[0043] |Xi-Fi|≤△F

[0044] |X(a-1)-F(a-1)|>△F

[0045] |X(n+1)-F(n+1)|>△F

[0046] Then call the interval [Ta, Tb] as the compressible interval for f(...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a columnar storage technology-based real-time data compression algorithm. According to the algorithm, trial calculation and assessment of a variable-step calculation formula are completed for a sequence in a specific column package, search and parameter calculation of a unit calculation formula are covered, and real-time data column packages in compression division can be compressed, so as to solve the problem of huge storage space occupancy, decrease the hardware deployment quantity, downsize the deployment environment and save the deployment cost. The algorithm disclosed by the invention is capable of realizing real-time high-frequency and large-data volume data compression and storage, and greatly decreasing the storage space occupancy problem.

Description

Technical field [0001] The present invention relates to the field of data compression technology, in particular to a real-time data compression method based on columnar storage technology. Background technique [0002] The data structure design of real-time database is divided into two parts: memory and disk. The disk part is divided into short-term history and long-term history. Both the memory part and the disk part are columnar storage structures, but the physical location of the storage is different. The difference between the short-term history part and the long-term history part is that the former has no data compression, while the latter has data compression. [0003] For the memory part, what we have adopted is read-write separation technology. This technique allocates two kinds of memory buffers for data reading and writing. Among them, the read area can only be used to read data, but not write data; the write area can only be used to write data, but cannot read data. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22G06F16/21
Inventor 黄永忠徐兵张建强曹福国
Owner 浙江正泰中自控制工程有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com