A master-slave library stream replication acceleration method, device, equipment and medium

Abstract

The embodiments of the present invention relate to the field of database technology and disclose a master-slave library stream replication acceleration method, apparatus, device and medium. The method comprises: periodically reading data files from a disk and loading them into an operating system cache based on replayed records in a log, obtaining records to be updated from the log, searching for data file pages corresponding to the records to be updated from the operating system cache based on the records to be updated; and using multiple threads to perform parallel merging of the records to be updated and their corresponding data file pages. The embodiments of the present invention can significantly increase the log playback speed, reduce the log playback delay, and improve the consistency of master-slave library stream replication data by obtaining the data files of the log to be updated from the operating system cache and using multiple threads to replay the log in parallel.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a method and apparatus, device, and medium for accelerating master-slave library stream replication. Background Art

[0002] To ensure data consistency in streaming replication between PostgreSQL master and slave databases, after receiving the master's WAL log, a single startup process currently replays the log. This process extracts data block changes from the WAL record, reads the corresponding complete data pages from the data files, merges the data into the shared buffer, and finally asynchronously writes the dirty pages from the shared buffer to the data files. Although the PostgreSQL slave database uses the shared buffer to reduce write operations, this operation actually results in scattered file reads and writes when the BG writer writes the shared buffer to the data files. If the master database generates a large amount of WAL logs, executes time-consuming operations, or certain slave query operations conflict with the apply (merge) playback operation, data inconsistency between the master and slave databases may occur. This inconsistency is particularly pronounced when there are multiple slaves.

[0003] At present, the industry has proposed many solutions to the above problems. The commonly used methods and their shortcomings are as follows:

[0004] 1. The disadvantage of not enabling checksum for data files is that it can easily cause damage to data files.

[0005] 2. Delete unnecessary indexes. The disadvantage is that it is difficult to define which indexes are not needed.

[0006] 3. Increase the shared buffer size of the backup database. However, this has the disadvantage of a maximum size limit.

[0007] 4. The standby database uses a machine with higher IOPS capabilities and lower IO latency (do not use RAID 5), but this will be limited by resource conditions.

[0008] 5. Increase the number of files that a single process can open, but there is an upper limit on the number of files that a single process can open.

[0009] 6. Turning off IO time tracking can only be used as a temporary measure.

[0010] In summary, existing stream replication methods still have problems such as low efficiency and easy jamming.

[0011] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. Summary of the Invention

[0012] The purpose of the embodiments of the present invention is to provide a method and apparatus, device and medium for accelerating stream replication between a master and a slave library, so as to solve the problem that the existing stream replication speed between the master and the slave library is limited and the slave library may be stuck.

[0013] To solve the above technical problems, in a first aspect, an embodiment of the present invention provides a master-slave library stream replication acceleration method, which is applied to a slave library of a distributed database, and the method includes:

[0014] Periodically read data files from disk based on the replayed records in the log and load them into the operating system cache;

[0015] Obtaining a record to be updated from the log, and searching the operating system cache for a data file page corresponding to the record to be updated according to the record to be updated;

[0016] Multithreading is used to merge the records to be updated and their corresponding data file pages in parallel.

[0017] In addition, data files are periodically read from disk and loaded into the operating system cache based on the replayed records in the log, including:

[0018] According to the preset number of records in the log that have been recently replayed, the corresponding data files are periodically read from the disk and loaded into the operating system cache.

[0019] Additionally, the method further comprises:

[0020] The data files in the operating system cache are updated using an LRU algorithm.

[0021] In addition, the periodic reading of data files from the disk and loading into the operating system cache according to the replayed records in the log includes:

[0022] Parsing the block number and offset of the data file of the replayed record;

[0023] The data file is loaded into the operating system cache according to the block number and the offset.

[0024] In addition, searching the operating system cache for a data file page corresponding to the record to be updated includes:

[0025] The data file page corresponding to the record to be updated is searched from the operating system cache. If not found, the data file page corresponding to the record to be updated is further searched from the disk data file.

[0026] In addition, multiple threads are used to merge the records to be updated and their corresponding data file pages in parallel, including:

[0027] Read the record identifier of each record to be updated;

[0028] Adding the record corresponding to the record identifier to the processing queue of the corresponding thread according to the record identifier and the number of threads;

[0029] Each thread merges the records in its own processing queue based on the shared cache.

[0030] In addition, after each thread merges the records in its own processing queue based on the shared cache, the process further includes:

[0031] Write dirty data pages in the shared cache to disk data files.

[0032] In a second aspect, an embodiment of the present invention further provides a master-slave database stream replication acceleration device, which is applied to a slave database in a master-slave database system. The device includes:

[0033] The data preloading module is used to periodically read data files from the disk according to the replayed records in the log and load them into the operating system cache;

[0034] A search module is used to obtain the record to be updated from the log, and search the data file page corresponding to the record to be updated from the operating system cache according to the record to be updated, and

[0035] The merging module is used to merge the to-be-updated records and their corresponding data file pages in parallel using multiple threads.

[0036] In a third aspect, an embodiment of the present invention further provides a device comprising: a memory and a processor, wherein the memory stores a computer program, and the processor runs the computer program to implement the master-slave library stream replication acceleration method as described above.

[0037] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present invention.

[0038] The master-slave library stream replication acceleration method and device of the embodiment of the present invention, on the one hand, searches for the data file page corresponding to the record to be updated from the operating system cache, which greatly improves the reading speed of the data file page compared to the existing random reading of the disk I / O that needs to wait for sequential execution to obtain the data file page corresponding to the record to be updated during data playback; on the other hand, it uses multiple threads to parallel merge the record to be updated and its corresponding data file page, which can improve the data playback speed compared to the existing single-thread merging, and effectively avoid data update delays caused by the slave library being stuck. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of a master-slave library stream replication acceleration method provided by the first embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of the application process of the master-slave library stream replication acceleration method provided by the first embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the structure of a master-slave library stream replication acceleration device provided by the second embodiment of the present invention;

[0042] Figure 4 is a schematic structural diagram of a device provided in Embodiment 3 of the present invention; DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions, and advantages of the present invention more clear, the following will refer to the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions of the present invention through implementation methods. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] Figure 1 This is a flow chart of the master-slave library stream replication acceleration method provided by the first embodiment of the present invention. The technical solution of this embodiment can be applied to the slave library in the master-slave database system. The method can be executed by a master-slave library stream replication acceleration device provided by the embodiment of the present invention. The device can be implemented in software and / or hardware and configured in the slave library. Figure 1 As shown, the method specifically includes steps 101 to 104.

[0045] Step 101: Periodically read data files from the disk according to the replayed records in the log and load them into the operating system cache.

[0046] The slave's disk stores log files sent by the master. These files contain both replayed records and pending updates (records to be replayed). The slave replays the pending updates to maintain consistency between the master and slave databases.

[0047] like Figure 2 As shown in the figure, taking the PostgreSQL master-slave database as an example, the process of obtaining the WAL log file from the slave database is described as follows:

[0048] S1: The slave starts the wal log receiving (walreceiver) process, which sends a connection request to the master.

[0049] S2: After receiving the slave connection request, the master starts the WalSender process and establishes a TCP connection with the WalReceiver process. Note that the master can start multiple WalSender processes. The number of WalSender processes is configured by the max_wal_senders setting in postgresql.conf. The WalSender process is responsible for sending Wal logs to the WalReceiver process of the slave.

[0050] S3: The slave wal receiver process sends the latest wal lsn to the master.

[0051] S4: The master database compares the WAL LSN. The walsender process calls the exec_replication_command function to send the undelivered WAL logs to the slave database. At the same time, it calls the SyncRepWaitForLSN() function to obtain the latch and wait for the slave database to respond.

[0052] S5: The walreceiver process in the slave library is responsible for receiving the wal log sent by the walsender, writing the wal to the file cache through the write() function, and then calling the operating system fsync() function to flush the wal to the disk.

[0053] In this embodiment, a new process pg_preload is enabled in the slave library to use pg_xlogdump or pg_waldump (postgres 10.x and above) to periodically read the replayed wal records from the slave library disk, so as to load the required data files into the operating system cache in advance.

[0054] Specifically, periodically reading data files from the disk and loading them into the operating system cache according to the replayed records in the log may include: parsing the block number and offset of the data file from the replayed records; and loading the data file into the operating system cache according to the block number and offset.

[0055] Specifically, pg_waldump asynchronously reads the replayed Postgres WAL log records, parses the replayed WAL log, and obtains the data in the heap table's blkref field. Based on this data, it calculates the data file's page number. Based on the obtained PostgreSQL data file's page number, disk I / O operations are performed to read the data file and load it into the operating system cache. In other words, the WAL log itself provides the location and offset of the data block. The Pg_Preload process periodically reads and parses the received walrecord log data to find the block number and offset of the data file corresponding to the WAL record. The slave library can increase the size of the operating system cache, locate the data file block based on the block number, and load it into the operating system cache. Once a portion of the data file is loaded into the operating system cache, the startup process retrieves the data file page from the operating system cache instead of randomly reading the operating system file. This eliminates discrete I / O reads and seeks, significantly improving data read speed.

[0056] Step 101, periodically reading data files from disk and loading them into the operating system cache based on the replayed records in the log, may include: periodically reading corresponding data files from disk based on a preset number of recently replayed records in the log and loading them into the operating system cache. Data updates in a database are often continuous, meaning that the same data file may be updated continuously. Therefore, loading the data files corresponding to the most recently replayed records into the operating system cache helps improve the hit rate of the data files in the operating system cache when replaying new records to be updated. The data file loading cycle can be adjusted based on the hit rate of the data files in the operating system cache to maintain a high hit rate. The size of the data files loaded each time can be set based on the size of the operating system cache. Each file loaded into the operating system cache may replace some data files in the operating system cache. For example, a currently loaded data file may replace a data file loaded into the operating system cache earlier, thereby dynamically updating the data files in the operating system cache to maintain a high hit rate. It is understood that other strategies can also be used to load data files into the operating system cache, as long as they ensure a high hit rate for the operating system cache.

[0057] In some examples, an LRU (Least Recently Used) algorithm can be used to update the data file pages in the operating system cache to optimize the hit rate of the data file pages in the operating system cache. Given the limited number of operating system caches, the LRU algorithm is used to replace the operating system cache. This algorithm makes decisions based on the usage of data files after they are loaded into the operating system cache, eliminating the least recently used data files. However, this is not limiting, and other methods can also be used to optimize the operating system cache hit rate.

[0058] Step 102: Obtain the record to be updated from the log, and search the operating system cache for the data file page corresponding to the record to be updated according to the record to be updated.

[0059] Among them, the data file page corresponding to the record to be updated is searched from the operating system cache. If it is not found, the data file page corresponding to the record to be updated is continued to be searched from the disk data file. For example, when the startup process of the PostgresSQL slave library replays the WAL log, it directly reads the data file page from the operating system cache instead of addressing and loading the data file page from the disk database file. Only when the required data file page is not found in the operating system cache, the data file page is searched and loaded from the disk database file. That is, there is a hit rate for the startup process to hit the operating system cache. If there is no hit, the data file page still needs to be loaded from the data file in a discrete read manner.

[0060] Step 103: Use multiple threads to merge the records to be updated and their corresponding data file pages in parallel.

[0061] Multiple threads are used to perform a parallel merge of the records to be updated and their corresponding data file pages, including: reading the record identifier of each record to be updated; adding the record corresponding to the record identifier to the corresponding thread's processing queue based on the record identifier and the number of threads; and each thread merging the records in its respective processing queue based on a shared cache. Specifically, the slave database can launch multiple threads to process the WAL log, and the number of threads can be specified in the configuration process. The database kernel startup process is modified to continue using a single process to read the WAL log, while simultaneously using multiple threads to process the read WAL records in parallel. The record identifier can use the unique relfilenode of each table. After reading the WAL log, the single startup process determines the thread's memory queue to place the WAL record based on the relfilenode in the WAL record and the number of threads. Each thread then reads the WAL record from its own memory queue to perform data playback. When multiple threads process the WAL record in parallel, they share the data file pages in the operating system cache. The processing logic is the same as that of the open source PostgreSQL, namely, merging the WAL log and data file pages into a shared cache. Even if a thread is blocked, it will not affect the data recovery of other threads. The blocked thread will continue to complete the recovery after the blocking factor is eliminated, such as the end of a large conflict query.

[0062] After each thread merges the records in its own processing queue based on the shared cache, the process also includes writing the dirty data pages in the shared cache to the disk data file. Specifically, the bgwriter process can write the dirty data pages of the shared buffer to the data file.

[0063] The open-source PostgreSQL database replays data based on the Wal log. If the current data page is not a full page, the data page is retrieved from the data file corresponding to the Wal log, merged with the Wal log, and overwritten with the corresponding data file page to restore the data. When the master database generates a large volume of Wal logs, existing slave database Wal log replay uses a single process and single thread, and data files are read and written discretely, resulting in data delays between the slave and master databases.

[0064] After adopting the stream replication acceleration method of this embodiment, the new slave process pg_preload uses pg_xlogdump or pg_waldump to periodically read the most recently replayed WAL records, preloading the required data files into the operating system cache. When the slave startup process replays the WAL, it reads the required data file pages from the operating system cache instead of directly searching for and loading the data file pages from disk. This reduces the IO wait associated with reading data file pages during playback and speeds up slave data recovery. Furthermore, the slave startup process is changed from a single process with a single thread to a single process with multiple threads. A single process reads the WAL sequentially, while multiple threads concurrently merge the WAL log and data file pages, further improving data playback speed.

[0065] The method of the embodiment of the present invention pre-places data file pages into the operating system cache area, and directly obtains the data file pages from the operating system cache area during playback to achieve data playback, thereby effectively avoiding the very time-consuming operation of discretely reading data file pages from data files on the disk in the prior art during log playback. In the WAL log playback processing step, single-process multi-thread parallel processing is adopted, which further improves the slave database log playback speed compared with the original single-process single-thread method, accelerates the master-slave database stream replication speed, and reduces the occurrence of master-slave database query data inconsistencies.

[0066] Figure 3 This is a block diagram of a master-slave stream replication acceleration device provided in Example 2 of the present invention. This embodiment of the present invention also provides a master-slave stream replication acceleration device 300, which is configured in a slave database and is used to execute the master-slave stream replication acceleration method provided in Example 1. Device 300 includes a data preloading module 301, a search module 302, and a merge module 303.

[0067] The data preloading module 301 is used to periodically read data files from the disk according to the replayed records in the log and load them into the operating system cache.

[0068] The search module 302 is configured to obtain the record to be updated from the log, and search the operating system cache for the data file page corresponding to the record to be updated according to the record to be updated.

[0069] The merging module 303 is used to merge the to-be-updated records and their corresponding data file pages in parallel using multiple threads.

[0070] Optionally, the data preloading module 301 is specifically configured to periodically read corresponding data files from a disk according to a preset number of recently replayed records in the log and load the data files into an operating system cache.

[0071] The device 300 may further include: a cache update module, configured to update the data files in the operating system cache using an LRU algorithm.

[0072] Optionally, the data preloading module 301 may include:

[0073] A parsing submodule, configured to parse the replayed record to determine the block number and offset of the data file;

[0074] The recording submodule is used to load the data file into the operating system cache according to the block number and the offset.

[0075] Optionally, the search module 302 is configured to search the operating system cache for the data file page corresponding to the record to be updated. If the page is not found, the search module 302 continues to search the disk data file for the data file page corresponding to the record to be updated.

[0076] Optionally, the merging module 303 may include:

[0077] The record identifier reading submodule is used to read the record identifier of each record to be updated;

[0078] An enqueue submodule, configured to add the record corresponding to the record identifier to the processing queue of the corresponding thread according to the record identifier and the number of threads;

[0079] The parallel merging submodule is used by each thread to merge the records in its respective processing queue based on the shared cache.

[0080] The apparatus 300 may further include a disk write module configured to write dirty data pages in the shared cache into a disk data file.

[0081] The stream replication acceleration device of the embodiment of the present invention pre-places data files into the operating system cache area, and directly obtains the data files from the operating system cache area during playback to achieve data playback, thereby effectively avoiding the very time-consuming operation of discretely reading data file pages from the data files on the disk in the prior art during log playback; single-process multi-threaded parallel processing is adopted in the WAL log playback processing step, which further improves the slave database log playback speed compared with the original single-process single-thread, accelerates the master-slave database stream replication speed, and reduces the occurrence of inconsistent master and slave database query data.

[0082] The third embodiment of the present invention also provides a device that can be used as a slave library in a master-slave data system. Figure 4 As shown, the device includes: a memory 502, a processor 501;

[0083] The memory 502 stores instructions that can be executed by the at least one processor 501 , and the instructions are executed by the at least one processor 501 to implement the master-slave library stream replication acceleration method described in the above embodiment.

[0084] The device includes one or more processors 501 and a memory 502. Figure 4 A processor 501 is taken as an example. The processor 501 and the memory 502 may be connected via a bus or other means. Figure 4 In the example, a bus connection is used. Memory 502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. Processor 501 executes the non-volatile software programs, instructions, and modules stored in memory 502 to execute various functional applications and data processing of the device, thereby implementing the aforementioned master-slave library stream replication acceleration method.

[0085] The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function. In addition, the memory 502 may include a high-speed random access memory and a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device.

[0086] One or more modules are stored in the memory 502 , and when executed by one or more processors 501 , execute the master-slave library stream replication acceleration method in any of the above method implementations.

[0087] The above-mentioned device can execute the method provided by the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0088] The stream replication acceleration device of the embodiment of the present invention pre-places data files into the operating system cache area, and directly obtains data file pages from the operating system cache area during playback to achieve data playback, thereby effectively avoiding the very time-consuming operation of discretely reading data file pages from data files on the disk in the prior art during log playback; single-process multi-threaded parallel processing is adopted in the WAL log playback processing step, which further improves the slave database log playback speed compared with the original single-process single-thread, accelerates the master-slave database stream replication speed, and reduces the occurrence of master-slave database query data inconsistency.

[0089] A fourth embodiment of the present invention further relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0090] That is, those skilled in the art will understand that all or part of the steps in the above-described method embodiments can be implemented by instructing related hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps in the method embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0091] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present invention, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A master-slave library stream replication acceleration method, characterized in that: Applied to a slave database in a PostgreSQL master-slave database system, the method includes: Parse the most recently replayed preset number of records in the WAL log to obtain the block number and offset of the preloaded data file, and periodically load the data file from the disk to the operating system cache based on the block number and offset; Obtaining a record to be updated from the WAL log, and searching the operating system cache for a data file page corresponding to the record to be updated according to the record to be updated; Multithreading is used to merge the records to be updated and their corresponding data file pages in parallel.

2. The master-slave library stream replication acceleration method according to claim 1, characterized in that: The method further comprises: The data files in the operating system cache are updated using an LRU algorithm.

3. The master-slave library stream replication acceleration method according to claim 1, characterized in that: The searching the operating system cache for the data file page corresponding to the record to be updated includes: The data file page corresponding to the record to be updated is searched from the operating system cache. If not found, the data file page corresponding to the record to be updated is further searched from the disk data file.

4. The master-slave library stream replication acceleration method according to claim 1, characterized in that: The records to be updated and their corresponding data file pages are merged in parallel using multiple threads, including: Read the record identifier of each record to be updated; Adding the record corresponding to the record identifier to the processing queue of the corresponding thread according to the record identifier and the number of threads; Each thread merges the records in its own processing queue based on the shared cache.

5. The master-slave library stream replication acceleration method according to claim 4, characterized in that: After each thread merges the records in its own processing queue based on the shared cache, the process further includes: Write dirty data pages in the shared cache to disk data files.

6. A master-slave library stream replication acceleration device, characterized in that: Applied to a slave database in a PostgreSQL master-slave database system, the device comprises: The data preloading module is used to parse the pre-replayed records of the latest preset number of replayed records in the WAL log to obtain the block number and offset of the pre-loaded data file, and periodically load the data file from the disk into the operating system cache according to the block number and offset; A search module is used to obtain the record to be updated from the WAL log, and search the data file page corresponding to the record to be updated from the operating system cache according to the record to be updated, and The merging module is used to merge the to-be-updated records and their corresponding data file pages in parallel using multiple threads.

7. A device, characterized in that include: A memory and a processor, wherein the memory stores a computer program and the processor runs the computer program to implement the master-slave library stream replication acceleration method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that Used to store a computer-readable program, wherein the computer-readable program is used for a computer to execute the master-slave library stream replication acceleration method according to any one of claims 1 to 5.

Images