Method for quickly maintaining data consistency between slave and host in master-slave replication mode of OpenGauss database

A technology of copying mode and host data, applied in database model, database update, structured data retrieval, etc., can solve problems such as database performance degradation and playback speed slowdown, so as to improve external service capabilities, overcome data inconsistency, and reduce WAL. cumulative effect

Pending Publication Date: 2022-04-05
MASSIVE CLOUD BEIJING DATA TECH
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, as the pressure on the main database increases, a large amount of WAL will be generated, and the WAL playback speed of the standby database will slow down, resulting in WAL accumulation. When the difference between the WAL of the main database and the standby database reaches the set threshold, the database performance will drop significantly.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for quickly maintaining data consistency between slave and host in master-slave replication mode of OpenGauss database
  • Method for quickly maintaining data consistency between slave and host in master-slave replication mode of OpenGauss database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0038] Embodiment 1: A kind of method (referring to figure 1), this method splits the work of the backup playback thread from a single thread playback into multi-thread parallel playback by transforming the existing WAL playback thread, and distinguishes the WAL type into a transaction log and a non-transaction log; the transaction log includes transaction operation related Logs, such as CSN (COMMIT SEQUENCE NUM) logs and transaction commit logs; non-transactional logs include data operation-related logs, such as insert operation logs and update operation logs; for transaction logs, this method still uses a separate thread playback to avoid out-of-order events For non-transactional logs, this method uses multi-threaded parallel playback to speed up WAL playback.

[0039] Specifically, this method includes the following steps:

[0040] Step 1: The log receiving thread writes the WAL sent by the main library to the disk;

[0041] Step 2: The Startup thread reads the WAL from t...

Embodiment 2

[0047] Embodiment 2: When using multi-threaded parallel playback processing for non-transactional logs in this method, if there is a dependency relationship between non-transactional logs, then the dependencies are divided into intra-transaction dependencies and inter-transaction dependencies, and for different types of dependencies Relationships are handled differently as follows:

[0048] (1) For intra-transaction dependencies, when there is WAL that operates multiple times on the same operation object (such as the same table) in a transaction (for example, do INSERT first, then do DELETE), parallel playback is based on the information obtained from WAL The numerical label that identifies the physical location of the operation object (the relfilenode of the table can be obtained according to the WAL information) calculates the number of the worker thread that plays back this log. Since the digital label that identifies the physical location of the operation object is unique, ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of relational database management and operating systems, and particularly relates to a method for quickly keeping data consistency between a slave and a host in a master-slave replication mode of an OpenGauss database and application of the method. According to the method, an existing WAL playback thread is transformed, standby library playback thread work is split into multi-thread parallel playback from single-thread playback, WAL types are divided into transaction logs and non-transaction logs, single-thread playback is still adopted for the transaction logs so as to avoid disorder events, multi-thread parallel playback is adopted for the non-transaction logs so as to avoid unordered events, and therefore the WAL type is divided into the transaction logs and the non-transaction logs. Therefore, the WAL playback speed is accelerated. By means of the method, the standby database can rapidly complete WAL playback, WAL accumulation is reduced, the WAL difference value of the main database and the standby database is kept within the set threshold value range all the time, the external service capacity of the OpenGauss database in the master-slave copy mode is remarkably improved, and technical guarantee is provided for normal exertion of database performance.

Description

technical field [0001] The invention belongs to the technical field of relational database management and operating system, and in particular relates to a method for quickly maintaining data consistency between a slave and a master in the master-slave replication mode of an OpenGauss database and its application. Background technique [0002] OpenGauss is an open source relational database management system. Its core is derived from PostgreSQL. As an open source and free database platform, it aims to encourage community contributions and cooperation. At present, the OpenGauss database already supports streaming physical replication (streaming replication) of the pre-written log (WAL). Database users can use streaming replication to achieve WAL reception from the main database while replaying the received WAL during the backup process. The standby database continues to receive and replay WAL from the main database to complete the construction of the standby database and data ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23G06F16/28G06F9/46
Inventor 秋明明黄晓涛
Owner MASSIVE CLOUD BEIJING DATA TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap