Implementation method based on sqlldr2 and og data synchronization

An implementation method and data synchronization technology, which is applied in the field of data processing, can solve problems such as difficulty in cross-platform data import, frequent data import and export, and inability to add or delete according to requirements, so as to achieve the effect of reducing workload

Pending Publication Date: 2021-11-26
辽宁振兴银行股份有限公司
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, the current implementation method of sqlldr2 and ogg data synchronization has the problem of frequent data import and export, the difficulty of cross-platform data import, and the inability to add or delete tables that need to be synchronized according to requirements.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Implementation method based on sqlldr2 and og data synchronization

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] see figure 1 , in the embodiment of the present invention, a kind of realization method based on sqlldr2 and ogg data synchronization, described realization method comprises Oracle database, sqlldr2 client, ogg synchronization tool, HDFS distributed file system;

[0032] The sqlldr2 client is used to export the Oracle database data, and the data is exported in txt form, and transmitted to the HDFS distributed file system;

[0033] The ogg synchronization tool is used to synchronously export the log files of the Oracle database. After the redo log in the Oracle database loads the modified log files from the disk into the memory, the archive log mode archives and backs up the modified log files. When the data is exported , the source end of the ogg synchronization tool reads the redo log in real time and parses it into SQL statements and transmits it to the target end in the form of a binary file. The process of the ogg synchronization tool process on the target end reads...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of data processing, and discloses a data synchronization implementation method based on sqlldr2 and ogg. The sqlldr2 client side is used for exporting Oracle database data, exporting the data in a txt mode and transmitting the data to an HDFS distributed file system, an ogggog synchronization tool source side reads redo logs in real time and analyzes the redo logs into sql statements to be transmitted to a target side in a binary file mode, and the target end ogg synchronization tool process reads the file and imports the file into an HDFS (Hadoop Distributed File System). According to the method, the sqlldr2 tool is used for outputting the required data into the txt file in a formatting manner according to different requirements, and the og tool is used for specifying and collecting the data of the specific table and the specific field, so that the Oracle data can be synchronously backed up and stored into the HDFS; the problems that data are frequently imported and exported, data are difficult to import in a cross-platform mode, and tables needing to be synchronized cannot be newly added or deleted according to requirements are solved, automatic synchronization of incremental data is achieved, and the workload of manual synchronization every time is reduced.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to an implementation method based on sqlldr2 and ogg data synchronization. Background technique [0002] Sqlldr2: Sqlldr2 is a client tool that can export the data in the oracle database into a txt file according to a specific format. [0003] Ogg: also known as goldengate, is a data synchronization tool composed of source and target. The source reads redolog in real time and parses it into SQL statements, which are then transmitted to the target in the form of binary files. [0004] The functions realized by the two of them: sqlldr2 is used for data initialization from Oracle to Hadoop, and ogg is used for real-time synchronization of incremental data, which not only ensures the integrity of the initial data but also ensures the timeliness of the incremental data. [0005] However, the current implementation method of sqlldr2 and ogg data synchronization has the problem of...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/178G06F16/182G06F16/27
CPCG06F16/1794G06F16/182G06F16/27
Inventor 陈典银张德权
Owner 辽宁振兴银行股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap