A storage and retrieval method for IoT time series data in a flash granular array

Abstract

The invention discloses a method for storing and retrieving time-series data of the Internet of Things of a flash memory particle array, which is specially improved for reading and writing multi-type time-series data. According to the life cycle of various types of data, the present invention establishes respective block-level address mapping tables for the physical blocks allocated in advance from the flash memory particle array, and combines the data into a storage unit in the buffer area of ​​the memory by one page. A certain amount of data is then concurrently written into the pre-allocated physical block. When writing, one page of data is sequentially written from the first logical block to ensure that the same offset address of each block is written out in one round. Data; build a sensor change record table to record parameters such as data time point and logical page number, logical page number and logical block number, and calculate the target according to each parameter brought into the sensor change record table at the time point of the target data when querying data The address where the data resides gets the data.

Description

technical field [0001] The invention relates to the field of storage and retrieval of databases, in particular to a storage and retrieval method for time series data of the Internet of Things of a flash memory particle array. Background technique [0002] With the continuous popularization of Internet of Things technology and the continuous development of wireless sensor network technology, more and more sensors are used in the layout of sensor networks, the sampling rate of sensors is also higher and higher, and the amount of data generated is increasing. The most important feature of this kind of data is that it has time continuity and various types of data. How to quickly store and retrieve so many types of data has become a problem that needs to be solved. Especially for fast retrieval of specified data, common relational databases cannot meet the timeliness of data retrieval under large-scale data; while new non-relational databases cannot make good use of the time rela...

AI Technical Summary

Problems solved by technology

Existing time-series-based industrial data storage and retrieval methods have certain limitations in the storage and retrieval of time-series data. Existing methods only consider putting the same data in the same physical block and physically Blocks are combined into multiple data files to save data, and the physical address of data stored on the storage medium is not optimized for storage. For disk media, if multiple physical blocks in the file are too scattered on the disk, random The efficiency of reading and writing will be very poor. For flash memory arrays, if multiple physical blocks storing continuous data are concentrated in one flash particle, the reading and writing efficiency will also become very poor.

[0006] In addition, the existing method reserves data storage space to prevent the increase of collection points, and the increase of collection data lacks the effective use of storage space. If the number of collection points of the specified type does not increase, the overflow data area cannot be used to write other types of data , the storage space utilization efficiency is low

For data retrieval, the existing method uses first to find the data files of the corresponding time period, and then compares the time difference of each data from the data files to find the data of the corresponding time point. The existing method needs to find the data according to the time point from the file retrieval table Which file is the target data in? When the amount of data continues to increase, the increasing number of data files will lead to lower query efficiency

Images