Unlock instant, AI-driven research and patent intelligence for your innovation.

Time hierarchical merging method and device for time series data, equipment and medium

Pending Publication Date: 2022-04-29
AGRICULTURAL BANK OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, the existing data merging strategy of LSM-Tree does not fully consider the timing characteristics of the stored data, resulting in overlapping time ranges between the data files generated by the final merging, and cannot support data access and data deletion according to a specific time range wait for operation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Time hierarchical merging method and device for time series data, equipment and medium
  • Time hierarchical merging method and device for time series data, equipment and medium
  • Time hierarchical merging method and device for time series data, equipment and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0046] figure 1 Embodiment 1 of the present invention provides a flow chart of a method for time-layered merging of time-series data. This embodiment is applicable to the case of layered merging of time-series data content. The apparatus for time-layered merging of time-series data may be implemented in the form of hardware and / or software, and the apparatus for time-layered merging of time-series data may be configured in an electronic device with data processing capabilities. Such as figure 1 As shown, the method includes:

[0047] S110. Receive the generated time series data file through the layer to be processed.

[0048]The technical solution of this embodiment can be executed by a background server, and is suitable for working in a large number of updated usage scenarios, and can satisfy a usage scenario in which time-series data in cyberspace includes a certain proportion of data updates. Time-series data exists widely and has strong timeliness. It can include operat...

Embodiment 2

[0063] figure 2 It is a flow chart of a method for time-layered merging of time series data provided by Embodiment 2 of the present invention. This embodiment is optimized based on the foregoing embodiments. The specific optimization is: after retaining the time series data file in the layer to be processed, the method further includes: if it is detected that the layer to be processed satisfies the trigger condition of the number of time series data files or meets the trigger condition of the size of the time series data file, then The time series data files of the layer to be processed are merged according to the file size policy.

[0064] Such as figure 2 As shown, the method of this embodiment specifically includes the following steps:

[0065] S210. Receive the generated time series data file through the layer to be processed.

[0066] S220. If it is detected that the triggering condition of the number of time-series data files or the size of the time-series data file...

Embodiment 3

[0088] image 3 It is a schematic structural diagram of a device for time-layered merging of time-series data provided by Embodiment 3 of the present invention. The device can execute the method for time-layered merging of time-series data provided by any embodiment of the present invention, and has corresponding functional modules for executing the method and beneficial effects. Such as image 3 As shown, the device includes:

[0089] The file receiving module 310 is used to receive the time series data file generated by the layer to be processed;

[0090] The file division module 320 is used to determine the time range of the time series data file one by one, and if the time range is within the time span of a certain level obtained according to a predetermined division rule, write the time series data file into the hit target level;

[0091] The file caching module 330 is used to put the time-series data file into the cache area corresponding to the current level if the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses a time hierarchical merging method and device for time series data, equipment and a medium. The method comprises the following steps: receiving a generated time sequence data file through a to-be-processed layer; time ranges of the time sequence data files are determined one by one, and if the time ranges are within a time span of a certain hierarchy obtained according to a predetermined division rule, the time sequence data files are written into a hit target hierarchy; and if the time sequence data file of a certain hierarchy exceeds the time span of the current hierarchy due to time lapse, putting the time sequence data file into a cache region corresponding to the current hierarchy so as to be merged with the time sequence data file of the next hierarchy. According to the technical scheme, the requirements of time dimension analysis and life cycle management of the data can be effectively supported, and for a large-scale update use scene, the merging overhead can be reduced, so that the data files are distributed in sequence in a time range, and the overall read-write performance of the system is improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a method, device, equipment and medium for time-layered merging of time-series data. Background technique [0002] With the development of the mobile Internet and the Internet of Things, various industries have produced a large amount of time series data (TimeSeries Data, referred to as TSD or time series data). How to organize and manage time-series data and support efficient retrieval and analysis of data within a specific time range while ensuring efficient writing of data is a problem that big data management systems need to solve urgently. [0003] At present, the main solution uses the key-value storage based on LSM-Tree (log-structured merge tree) as the underlying storage engine to ensure the writing performance of high-speed sequential data. The compaction operation is continuously performed in the background to improve the efficiency of data reading by reorga...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/185G06F16/172G06F16/16
CPCG06F16/185G06F16/16G06F16/172
Inventor 辛明佳李瑞璨
Owner AGRICULTURAL BANK OF CHINA