Unlock instant, AI-driven research and patent intelligence for your innovation.

A Fast Method of Acquiring File Increment Based on Memory Operation

A file and memory technology, which is applied in the field of fast acquisition of file increments based on memory calculations, can solve the problems of slow calculation speed, consumption, and manual intervention, and achieve fast incremental calculation speed, obvious incremental acquisition, and low memory consumption Effect

Active Publication Date: 2021-12-07
HEBEI GODSEND HIGH TECH
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In the ETL process, obtaining incremental data is a very critical operation link. The conventional method is to load new data into the database and perform incremental stripping operations. This method consumes a lot of expensive database resources, and the operation speed is slow and the interface is upgraded. It is necessary to modify the incremental stripping program in the database, which requires a lot of manual intervention and high development pressure

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Fast Method of Acquiring File Increment Based on Memory Operation
  • A Fast Method of Acquiring File Increment Based on Memory Operation
  • A Fast Method of Acquiring File Increment Based on Memory Operation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] The present invention is specifically described below in conjunction with accompanying drawing, as Figure 1-6 shown.

[0020] Step 1, select 2 files that need to get increments. (The old file is called A file, the new file is called B file, and the incremental file is called C file)

[0021] Step 2: Build a 32-point Tire Tree with a depth of D in the memory. The specific depth can be reasonably selected according to the file size or hardware environment. The recommended depth is 4-6.

[0022] The data structure of Tire Tree is as follows figure 1 Shown:

[0023] Step 3: Get the Hash value of file A line by line, select D (4~6) bits in the Hash value as the TireTree index according to the memory and file size, and find the leaf node pointer of the Tire Tree one by one according to the index.

[0024] TireTree as figure 2 Shown (N=32):

[0025] Step 4: Create a text information node at the leaf node, store the Hash value of each line of text and a part of the orig...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for quickly acquiring file increments based on memory operations. The incremental method includes: extracting feature values ​​from old files by row and writing them into a specific data structure in memory, and then extracting features from new files according to the same algorithm. Query after value in memory, algorithm of eigenvalue, design of data structure and avoid conflict. The invention has the advantages of simple structure and strong practicability.

Description

technical field [0001] The invention relates to the fields of G06F17 / 30, G06F17 / 00, G06F17, and G06F, in particular to a method for quickly acquiring file increments based on memory operation. Background technique [0002] In the ETL process, obtaining incremental data is a very critical operation link. The conventional method is to load new data into the database and perform incremental stripping operations. This method consumes a lot of expensive database resources, and the operation speed is slow and the interface is upgraded. It is necessary to modify the incremental stripping program in the database, which requires a lot of manual intervention and great development pressure. Contents of the invention [0003] The object of the present invention is to solve the above-mentioned problems, and designs a method for quickly acquiring file increments based on memory operation. [0004] The technical solution of the present invention to achieve the above object is a method f...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/25G06F16/2455G06F16/22
Inventor 柴磊原伟柳彦利杨峰马章焘王立强冯剑付斐郭峰刘改琴李扬刘晓霖
Owner HEBEI GODSEND HIGH TECH