Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Batch data time sequence transmission method and system based on database log file

A technology of log files and time series, applied in the field of data processing, to achieve the effect of improving call speed, improving retrieval efficiency, and improving transmission efficiency

Active Publication Date: 2022-07-08
HANGYIN CONSUMER FINANCE CO LTD
View PDF14 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The present invention provides a batch data time series transmission method and system based on database log files to solve the existing problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Batch data time sequence transmission method and system based on database log file
  • Batch data time sequence transmission method and system based on database log file
  • Batch data time sequence transmission method and system based on database log file

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0042] Embodiments of the method for batch data time series transmission based on database log files of the present invention, such as figure 1 As shown, the method includes:

[0043]S1. Obtain the log data corresponding to each log file in the database, and parse the log data into streaming data. Specifically, due to the massive storage occupation of the batch log file data in the database, the processing speed is extremely slow and ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of data processing, in particular to a batch data time sequence transmission method and system based on a database log file, and the method comprises the steps: obtaining log data, analyzing the log data into streaming data, obtaining a logic relation between the streaming data, and obtaining a data sequence; obtaining target data and target data sequences corresponding to the system non-periodic self-checking operation according to the operation time of data corresponding to the system periodic self-checking operation in the data sequence and the system self-checking period, dividing the target data sequences into a plurality of subsequences at equal intervals, and calculating the importance degree of each target data sequence, and calculating the size of a sliding window of the data sequence according to the importance degree, and compressing and transmitting the streaming data in each data sequence by using an LZ77 algorithm according to the size of the sliding window of the data sequence. Therefore, the data retrieval efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a method and system for batch data time series transmission based on database log files. Background technique [0002] In the world of information, all walks of life will establish their own databases in the process of managing production, and operations based on the database will generate a large number of log files, and these log files are stored locally. [0003] Since the log file data is dynamic data, a large amount of batch log data will be generated for a long time when the modification operation of the data is recorded by the log file in the database, and the large amount of batch log data not only occupies a large amount of storage resources, but also can be searched and reviewed in the database later. When the previous data modification operation is performed, it will be too difficult to find the log data required for backtracking, and the retrieval efficiency w...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/174G06F16/18G06F16/2458
CPCG06F16/1744G06F16/2474G06F16/1815Y02D10/00
Inventor 石杰廖家林伍星
Owner HANGYIN CONSUMER FINANCE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com