Hierarchical storage method based on time series data cold and hot classification

A time-series data and hierarchical storage technology, applied in the input/output process of data processing, electrical digital data processing, instruments, etc., can solve the problem of high cost, achieve maximum utilization, ensure query performance, and maximize storage space Effect

Active Publication Date: 2018-07-10
北京航天云路有限公司
View PDF7 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At the same time, since all data is

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Hierarchical storage method based on time series data cold and hot classification
  • Hierarchical storage method based on time series data cold and hot classification
  • Hierarchical storage method based on time series data cold and hot classification

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention belong to the protection scope of the present invention.

[0016] Such as Figures 1 to 5 As shown, a hierarchical storage method based on hot and cold classification of time series data according to an embodiment of the present invention includes the following steps:

[0017] S1 Establish a UID system, and use UID to uniformly identify time series data;

[0018] S2 divides the data into real-time data, weekly data, historical data and cold data through specific time rules, and establishes real-time data layer, weekly data layer, historical data lay...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a hierarchical storage method based on time series data cold and hot classification. The method comprises following steps: establishing an UID system and using the UID to uniformly identify time series data; dividing data into real-time data, week and day data, historical data, and cold data through specific time rules. A real-time data layer, a week and day data layer, a historical data layer, and a cold data layer are respectively established and stored in the memory, the SSD storage hard disk, the HDD storage hard disk, and the distributed storage system HDFS. establishing a unified data insertion and query interface, and routing the query request to respective storage query interface through the time range of the query to provide different storage access for different data. The invention has the beneficial effects that the storage space of different access frequencies is hierarchically stored by tiered storage of data with different access frequencies, thereby maximizing the utilization of the storage space and ensuring the query performance, and also reducing the cost.

Description

technical field [0001] The present invention relates to the field of industrial internet of things, in particular to a hierarchical storage method based on hot and cold classification of time series data. Background technique [0002] Time is an important feature of time-series data. The more recent data, the more users pay attention to it, and the more queries are made. It belongs to hot data. The storage of this part of data needs to support high-concurrency and low-latency interactive queries, so higher storage is required. Cost; the longer the data is queried, the less it is, and it is cold data. These data are not as sensitive to the query delay as interactive scenarios, and have lower requirements for the read and write performance of the storage medium. According to the characteristics of time-series data storage, hierarchical storage will be a method to balance cost and performance. [0003] Beringei is an in-memory time-series database open sourced by Facebook. Ber...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F3/06
CPCG06F3/0607G06F3/0629G06F3/0674
Inventor 徐汕单酉刘强黄文峰童奥姜桥
Owner 北京航天云路有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap