Unlock instant, AI-driven research and patent intelligence for your innovation.

Time series data storage method and device

A time-series data and data technology, applied in database indexing, database update, electronic digital data processing, etc., can solve problems such as server crashes easily, achieve the effects of improving throughput, reducing read amplification, and reducing writing overhead

Inactive Publication Date: 2020-11-13
CHENGDU SEFON SOFTWARE CO LTD
View PDF7 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The purpose of the present invention is to provide a storage method and device for time-series data to solve the problem that the existing time-series database directly compresses a large amount of data and causes the server to crash easily

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Time series data storage method and device
  • Time series data storage method and device
  • Time series data storage method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0026] A method for storing time-series data includes the following steps: splitting original data into different data sets and writing them into different sub-tables; writing the sub-tables into a time-series database; to different sub-tables for compressed storage, reducing the total amount of compressed data, greatly reducing the write overhead, and improving the throughput of the time-series database as a whole; writing sub-tables into the time-series database uses the existing time-series data written into the time-series database The method is a mature prior art, so it will not be repeated here.

Embodiment 2

[0028] Such as figure 1 As shown, on the basis of Embodiment 1, this embodiment further divides the original data measurement columns into different measurement column subsets according to the set number of measurement columns after sorting the original data measurement columns according to the column name hash; the original data includes index columns Set and time columns, combine the index column set, time column and measurement column subsets into new data, write the data into sub-tables; write the data into different sub-tables after splitting and reorganizing, and then compress, reducing the data amount, and also reduces the total amount of compressed data.

Embodiment 3

[0030] Such as figure 2 As shown, this embodiment is further based on Embodiment 1. After the data is written into the sub-table, the key is extracted according to the set label, and the data is compressed according to the timestamp through the extracted key, and Compress_Block is generated after compression; ey and The following data block is written into the KV engine as a KV, and the last timestamp of each Compress_Block is used as the column name; the data is written into the subtable through the above method.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a time series data storage method and device, and mainly solves the problem that a server is easy to crash due to the fact that an existing time series database directly compresses a large amount of data in the prior art. The time series data storage method comprises the following steps: splitting original data into different data sets, and writing the data sets into different sub-tables; and writing the sub-tables into a time sequence database. By means of the scheme, tens of thousands of columns of measurement data are split into different sub-tables to be compressedand stored; meanwhile, a storage format for storing metadata of the sub-tables is provided, and the purpose of effectively managing the storage sub-tables is achieved.

Description

technical field [0001] The present invention relates to the technical field of data storage and query, in particular to a method and device for storing time series data. Background technique [0002] Time-series databases are mainly used to process data with time tags, that is, time-serialized data that changes in the order of time. Data with time tags is also called time-series data; existing time-series databases are faced with tens of thousands of columns of data When writing, the data is directly compressed. In the case of a huge amount of data in the original line, even after the batch data is compressed, the amount of data is massive, which puts the server under tremendous pressure and easily causes the server to crash. Contents of the invention [0003] The object of the present invention is to provide a method and device for storing time series data, so as to solve the problem that the existing time series database directly compresses a large amount of data and cau...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/23G06F16/2457G06F16/2458
CPCG06F16/2255G06F16/2322G06F16/24573G06F16/2474
Inventor 张艳清杨尧陈博胥莉君鲁燕
Owner CHENGDU SEFON SOFTWARE CO LTD