Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Increment data capturing method and system

An incremental data and data technology, applied in the field of incremental data capture methods and systems, can solve the problems of metadata definition deviation, serious time-consuming, and occupying large memory space, so as to reduce time and space complexity and save computer systems. resources, the effect of enhancing generality

Inactive Publication Date: 2008-05-21
NEUSOFT CORP
View PDF0 Cites 53 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The algorithm also includes a large number of IO operations, which makes the time-consuming more serious; and if the number of IO operations is reduced to reduce the time-consuming, it will take up a lot of memory space
In the case of limited memory and a large amount of snapshot data, the execution efficiency of the algorithm will be greatly affected
[0006] 2. It is necessary to maintain database metadata information, which limits the versatility of the algorithm to a certain extent
Therefore, the incremental data capture using files as snapshots must additionally maintain this part of the lost metadata, which increases the complexity of snapshot comparison incremental extraction
Moreover, there are deviations in the definition of metadata between different types of databases, and to provide support for different types of databases requires an extended algorithm, which limits the versatility of the algorithm to a certain extent.
[0008] 3. Does not provide the recovery function after the incremental update of the target database

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Increment data capturing method and system
  • Increment data capturing method and system
  • Increment data capturing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0063] In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0064] Referring to Fig. 1, it shows a flowchart of a first embodiment of an incremental data capture method of the present invention, including steps:

[0065] Step 101, create a relational database snapshot table corresponding to the target table according to the structure information of the target table.

[0066]The snapshot table is a relational database table structure that is the same as the target table or fully compatible with all data in the target table.

[0067] The specific process of creating the snapshot table is as follows: first obtain the structural information of the target table for incremental capture, then construct the SQL CREATE TABLE statement of the relational database table according to the structural inf...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an incremental data acquisition method and an incremental data acquisition system; wherein, the method comprises that: a relational database snapshot list corresponding to a target list is created according to the structure information of the target list; extraction of snapshots in the target list with time points appointed is completed with the created snapshot list; and snapshot intercomparison is performed according to the snapshots with different time points so as to capture incremental data. The invention has the advantages that the snapshot list is adopted to store snapshots, and maintenance of metadata information of the incremental data is not needed, thus versatility is enhanced; furthermore, intercomparison time of snapshot record is shortened, computer system computation is decreased, and computer system resources are saved through creating the cluster index of the snapshot list.

Description

technical field [0001] The invention relates to the technical field of databases, in particular to an incremental data capture method and system in the technical field of database data integration. Background technique [0002] Incremental data capture (Change Data Capture, CDC) refers to the process of capturing the incremental data generated in the data source and providing it to other enterprise application software. CDC is one of the main problems to be solved in the process of ETL (Extract Transform Load) in data integration. CDC is used to capture data insertion, deletion, and update operations in business databases, and provides incremental data capture services for the synchronization of enterprise application database data such as OLAP databases, report databases, data warehouses, and business intelligence databases. [0003] The snapshot (Snapshot) comparison method is one of the main methods of incremental data capture in the prior art, and the incremental data i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 许力牟晓光
Owner NEUSOFT CORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com