Unlock instant, AI-driven research and patent intelligence for your innovation.

Efficient time series data storing and reading system

A time-series data and high-efficiency technology, which is applied in database indexing, electronic digital data processing, structured data retrieval, etc., to achieve the effects of increased reading speed, fast reading and writing speed, and improved writing speed

Inactive Publication Date: 2018-05-11
AUTOMATION RES & DESIGN INST OF METALLURGICAL IND
View PDF6 Cites 24 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide an efficient storage and reading system for time series data, which solves the problems of efficient compression, efficient writing and efficient reading of various types of time series data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Efficient time series data storing and reading system
  • Efficient time series data storing and reading system
  • Efficient time series data storing and reading system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] like figure 1 As shown, the system includes three data storage forms of memory cache, log file, and data file, and three data processing behaviors of writing, compressing, and reading. When writing, the memory cache and the log file are written synchronously, the log file is periodically compressed into a data file, and the data file is gradually compressed into a larger data file. When reading, the system needs to determine whether to read from the memory cache or from the data file. If it is read from the data file, use the index in the file to quickly locate the data location.

[0019] like figure 2 As shown, the time series data includes point names, time stamps and numerical information. When writing a set of time series data, each point is traversed to determine whether the data type of the numerical value is an integer, floating point number, Boolean value or string, and the corresponding compression algorithm is called respectively. Compress, call the time st...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an efficient time series data storing and reading system, and belongs to the technical field of real-time databases. The system comprises one or more networking computers to form a system hardware platform; software of the system is operated on the computers, a data writing module, a data compressing module and a data reading module are included, the data writing module isresponsible for receiving new data, data is written into a memory cache and a log file respectively; the data compressing module is responsible for compressing the data of the log file according to adesigned compression algorithm and an index structure into the data file; the reading module responds to a reading request, and search results of the memory cache and the data file are synthesized andthen returned. The system has the advantages that compared with a relational database, the space occupied by a magnetic disk is small, and the reading-writing speed is high; the data is compressed, the space occupied by the magnetic disk is smaller, and only 35% of the magnetic disk space is occupied; the data write-in speed is higher, and compared with a Mysql database, the write-in speed is increased by three times; the data reading speed is higher, and compared with the Mysql database, the reading speed is increased by 20 times.

Description

technical field [0001] The invention belongs to the technical field of real-time databases, in particular to an efficient storage and reading system for time series data. Background technique [0002] Time series data refers to time series data, which refers to data with time tags (changes in the order of time, that is, time serialization). Time-series data is mainly collected and generated by various types of real-time monitoring, inspection and analysis equipment such as electric power, chemical industry, and metallurgy. The typical characteristics of these industrial data are: fast generation frequency (each monitoring point can generate multiple data), heavily dependent on the collection time (each piece of data requires a unique time), and a large amount of information from multiple measuring points (conventional real-time monitoring systems have tens of thousands of monitoring points, and the monitoring points generate data every second , generating dozens of gigabyte...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/22G06F16/2474
Inventor 徐化岩李勇
Owner AUTOMATION RES & DESIGN INST OF METALLURGICAL IND