Large object initialization method and device based on log analysis synchronization

An initialization method and large object technology, applied in the computer field, can solve the problem of low table initialization efficiency, and achieve the effect of preventing the accumulation of archived logs, improving storage efficiency, and reducing time-consuming

Active Publication Date: 2021-03-26
WUHAN DAMENG DATABASE
View PDF5 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] When there is large object data in the table to be initialized, because the database large object fields cannot be batch-operated by binding parameters when extracting and storing, the initialization efficiency of the table containing large objects is extremely low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Large object initialization method and device based on log analysis synchronization
  • Large object initialization method and device based on log analysis synchronization
  • Large object initialization method and device based on log analysis synchronization

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0047] The data types of the database system include non-large object data (such as INT, CHAR, etc.) and large object data (such as CLOB or BLOB) types. The database supports multi-line batch query and storage of non-large object data. When building a data synchronization system The performance is faster when initializing a table that does not contain large objects; however, the database only supports single row single large object operations when querying and warehousing large object data, which greatly affects the initialization efficiency, and the long-term result set extraction and warehousing operations increase The risk of database error reporting during the initialization process, resulting in initialization failure.

[0048] For this reason, the present invention uses a two-stage initialization scheme for non-large object data and large object data when initializing data, and adopts a method of segmented parallel initialization of large object data to solve the problem ...

Embodiment 2

[0078] The above-mentioned embodiment 1 introduces the main steps of a large object initialization method based on log parsing and synchronization. On this basis, the embodiment of the present invention further introduces the complete steps of the transaction merge execution method in embodiment 1 in detail.

[0079] combine figure 2 with image 3 , a large object initialization method based on log parsing and synchronization provided by an embodiment of the present invention. in, figure 2 It mainly corresponds to the initialization process of the first stage (ie steps 101-110), image 3 It mainly corresponds to the initialization process of the second stage (ie steps 111-116), as follows:

[0080] Step 101, deploying synchronization systems on the source database and the destination database respectively. Wherein, the source-end data synchronization system is used to read logs from the source-end database, and the destination-end data synchronization system is used to a...

Embodiment 3

[0119] On the basis of Embodiment 1 and Embodiment 2 above, the embodiment of the present invention provides a specific example of a large object initialization method based on log parsing and synchronization, so as to better understand the entire transaction merge execution process.

[0120] In this specific example, it is assumed that there is a table T to be initialized containing large object data in the source database, and the database supports a maximum in-line data length of 3900 bytes for large objects:

[0121] CREATE TABLE T(C1 INT,C2 CLOB,C3 CLOB);

[0122] INSERT INTO T(C1,C2,C3) VALUES(1,NULL,'B');

[0123] INSERT INTO T(C1,C2,C3)VALUES(2,NULL,REPLICATE('B',4000));

[0124] INSERT INTO T(C1,C2,C3)VALUES(3,REPLICATE('A',4000),REPLICATE('B',4000));

[0125] Step 301, initialize non-large object data in the first stage, and the source data synchronization system executes the following query statement:

[0126] SELECT ROWID,C1,C2,C3 FROM T;

[0127] Step 302, ext...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of computers, in particular to a large-object initialization method and device based on log analysis synchronization, and the method comprises the steps ofquerying a to-be-initialized table containing large-object data in a full-column manner through a source-end data synchronization system, and identifying a large-object column and a non-large-objectcolumn; initializing the non-large object column data to a destination end database in a multi-row batch manner, and storing a mapping relationship between data rows of the source end database and thedestination end database by adopting ROWID; after the source end data synchronization system confirms that the non-large object column data in the table is initialized in the destination end database, starting log synchronization of the table; and according to the stored ROWID information, supplementing and initializing the large object column data in the table to a destination end database in asegmented parallel mode. According to the scheme, the initialization speed of the data containing the large object table can be effectively increased, the establishment of a data synchronization environment is quickly completed, and the probability that a source end database report'snapshots too much 'is reduced.

Description

【Technical field】 [0001] The invention relates to the field of computer technology, in particular to a large object initialization method and device based on log parsing and synchronization. 【Background technique】 [0002] At present, the heterogeneous database replication technology based on database log analysis is widely used. This technology captures the incremental data of the database at the source and sends it to the destination. At the destination, the incremental data is applied to the destination through a common database access interface. Database for data replication. Because this technology uses a common database access interface, it supports heterogeneous database system replication, supports heterogeneous operating system environments, and the destination standby database system is readable and writable. It is a "active-active" system. [0003] When building a data synchronization system, the destination database is often an empty database. It is necessary to...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/27G06F16/2458G06F16/22
CPCG06F16/27G06F16/2458G06F16/221G06F16/2282G06F16/2219
Inventor 孙峰付铨彭青松余院兰
Owner WUHAN DAMENG DATABASE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap