Method and device for accessing mass time sequence data

A technology of time series and data, applied in the field of data cloud storage, can solve the problems of long storage time, high storage frequency, large amount of data, etc., and achieve the effect of improving access efficiency

Active Publication Date: 2015-09-02
广东省机电设备招标有限公司
View PDF4 Cites 27 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] With the extensive development and application of Internet of Things, cloud storage and other methods, how to store and retrieve massive time-related data has become a difficult problem for system applications. Time-series data refers to data closely related to time. Data access is not only related to the data itself In addition to keywords, it is also closely related to time. The storage structure is usually sorted by time, and the retrieval is usually performed according to the data keywords and time. In a specific field, massive time data has its own characteristics: data is highly correlated with ti

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for accessing mass time sequence data
  • Method and device for accessing mass time sequence data
  • Method and device for accessing mass time sequence data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0052] figure 1 It is a flowchart of a method for accessing massive time series data provided by this embodiment.

[0053] A method for accessing massive time series data, including:

[0054] S10. Receive time series data, the time series data including data source identifier DKey, data time Time and data information Value;

[0055] S20. Generate several bytes of the data source identifier through a hash algorithm, and establish several layers of directories corresponding to the several bytes;

[0056]S30. Store the time series data in the directory in the form of a data file DFile, the data file includes a data segment and an index segment, and the file name of the data file is generated by the data time;

[0057] S40. Receive a query request, where the query request includes a data source identifier and data time, and read corresponding time series data from the directory according to the query request and the index segment.

[0058] In this embodiment, the time-series da...

Embodiment 2

[0074] Such as figure 2 As shown, in this embodiment, the step S30 includes:

[0075] S31, search for all data files ending with .data in the directory, and search for the data file with the largest file name, if there is no data file ending with .data, then create a new data file, the file name of the new data file is: data Time time.data; record the basic information of the file at the tailOffset position of the file {startTime, 0, 0} where the start time is the data time, and the end time is 0 to indicate that it is not over. The current data segment is written to position 0 and recorded in the memory in the data structure of the file handle.

[0076] S32. If a data file ending with .data is found, read the end of file information, which includes the write offset offset; read the end of file record information {startTime, 0, offset}, wherein the start time is Data time, the end time is 0 means it is not over, the current data segment is written to the position offset, an...

Embodiment 3

[0082] Such as image 3 As shown, in this embodiment, the step S40 includes:

[0083] S41. Receive a query request, where the query request includes a data source identifier and a data time;

[0084] S42. Generate several bytes of the data source identifier of the query request through a hash algorithm, convert each byte into a hexadecimal string, and search for the corresponding directory;

[0085] S43. Sort the file names in the directory in reverse order, and find the first file name that is shorter than the data time of the query request;

[0086] S44. Open the data file corresponding to the file name, and read the index information at the end of the file;

[0087] S55. Read the data information of the time series data requested by the query according to the index information.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of data cloud storage, in particular to a method and a device for accessing mass time sequence data. The method includes: receiving time sequence data which include a data source identity, data time and data information; enabling the data source identity to generate a plurality of bytes through a Hash algorithm, and creating a plurality of catalogs corresponding to the bytes in a one-to-one manner; storing the time sequence data into the catalogs in a form of a data file which includes a data section and an index section, wherein a file name of the data file is generated by the data time; receiving an inquiry request which includes the data source identity and the data time, and reading corresponding time sequence data from the catalogs according to the inquiry request and the index section. The time sequence data are accessed by combining a means of the data file and index information, so that access efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of data cloud storage, in particular to a method and device for accessing massive time series data. Background technique [0002] With the extensive development and application of Internet of Things, cloud storage and other methods, how to store and retrieve massive time-related data has become a difficult problem for system applications. Time-series data refers to data closely related to time. Data access is not only related to the data itself In addition to keywords, it is also closely related to time. The storage structure is usually sorted by time, and the retrieval is usually performed according to the data keywords and time. In a specific field, massive time data has its own characteristics: data is highly correlated with time, Data is usually transmitted and stored in chronological order; the data itself is small, but the amount of data is huge, the storage time is long, and the storage frequency is h...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/2255G06F16/2474
Inventor 徐君
Owner 广东省机电设备招标有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap