Unlock instant, AI-driven research and patent intelligence for your innovation.

Memory operation-based incremental method for quickly obtaining files

A file and memory technology, applied in the field of rapid acquisition of file increments based on memory computing, can solve the problems of many manual interventions, large database resources, and slow computing speed, and achieve the effect of more manual interventions

Active Publication Date: 2018-11-13
HEBEI GODSEND HIGH TECH
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In the ETL process, obtaining incremental data is a very critical operation link. The conventional method is to load new data into the database and perform incremental stripping operations. This method consumes a lot of expensive database resources, and the operation speed is slow and the interface is upgraded. It is necessary to modify the incremental stripping program in the database, which requires a lot of manual intervention and high development pressure

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Memory operation-based incremental method for quickly obtaining files
  • Memory operation-based incremental method for quickly obtaining files
  • Memory operation-based incremental method for quickly obtaining files

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] The present invention is specifically described below in conjunction with accompanying drawing, as Figure 1-6 shown.

[0020] Step 1, select 2 files that need to get increments. (The old file is called A file, the new file is called B file, and the incremental file is called C file)

[0021] Step 2: Build a 32-point Tire Tree with a depth of D in the memory. The specific depth can be reasonably selected according to the file size or hardware environment. The recommended depth is 4-6.

[0022] The data structure of Tire Tree is as follows figure 1 Shown:

[0023] Step 3: Get the Hash value of file A line by line, select D (4~6) bits in the Hash value as the TireTree index according to the memory and file size, and find the leaf node pointer of the Tire Tree one by one according to the index.

[0024] TireTree as figure 2 Shown (N=32):

[0025] Step 4: Create a text information node at the leaf node, store the Hash value of each line of text and a part of the orig...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a memory operation-based incremental method for quickly obtaining files. The incremental method comprises the steps of extracting eigenvalues of old files by row and writing the eigenvalues into a specific data structure in a memory; extracting eigenvalues of new files by the same algorithm and performing a query in the memory; and realizing the eigenvalue algorithm, data structure design and conflict avoidance. The method has the beneficial effects of simple structure and high practicality.

Description

technical field [0001] The invention relates to the fields of G06F17 / 30, G06F17 / 00, G06F17, and G06F, in particular to a method for quickly acquiring file increments based on memory operation. Background technique [0002] In the ETL process, obtaining incremental data is a very critical operation link. The conventional method is to load new data into the database and perform incremental stripping operations. This method consumes a lot of expensive database resources, and the operation speed is slow and the interface is upgraded. It is necessary to modify the incremental stripping program in the database, which requires a lot of manual intervention and great development pressure. Contents of the invention [0003] The object of the present invention is to solve the above-mentioned problems, and designs a method for quickly acquiring file increments based on memory operation. [0004] The technical solution of the present invention to achieve the above object is a method f...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 柴磊原伟柳彦利杨峰马章焘王立强冯剑付斐郭峰刘改琴李扬刘晓霖
Owner HEBEI GODSEND HIGH TECH