Parallel processing method of database redo log and related product

Abstract

The application provides a database redo log parallel processing method and related products, wherein the database redo log parallel processing method comprises the following steps: a transaction process of a master database writes generated redo log data into a common log buffer; the transaction process awakens a log disk flushing process and a stream replication sending process of the master database in parallel; the log disk flushing process writes the redo log data to be persisted from the common log buffer to a disk log file of the master database; and the stream replication sending process sends the persisted redo log data from the common log buffer to a slave database. According to the scheme, the working starting point of the stream replication sending process is advanced from after the log disk flushing to during the log disk flushing, the overlap of network transmission time and disk I / O time is realized, the redo log data can flow to the slave database in real time, the replication delay between the master-slave cluster is shortened, and the real-time availability of data is improved.

Description

Technical Field

[0001] This invention relates to database technology, and in particular to a parallel processing method for database redo logs, a computer-readable storage medium, a computer program product, and a computer device. Background Technology

[0002] In a primary / standby database cluster, existing redo log synchronization follows a serial path of "persistence first, then transmission": the redo log is first flushed from the primary database to the local physical disk, then read from the disk and sent to the slave database. This design forcibly couples network transmission to disk I / O, causing the synchronization link to go through a lengthy process of "memory → disk → network," introducing significant disk access latency and waiting overhead. Summary of the Invention

[0003] One objective of this invention is to provide a parallel processing scheme for database redo logs to reduce the performance overhead introduced by the network transmission of redo logs relying on local physical disk persistence.

[0004] A further objective of this invention is to reduce the end-to-end latency of master-slave synchronization.

[0005] Specifically, this invention provides a parallel processing method for database redo logs, the method comprising: In response to a transaction commit, the primary database's transaction process writes the generated redo log data to a common log buffer. In response to the completion of the write operation, the transaction process simultaneously wakes up the log flushing process and the streaming replication sending process of the primary database. The log flushing process flushes the redo log data to be persisted from the public log buffer to the disk log file of the main database. In parallel with the write operation of the log flushing process, the streaming replication sending process sends the persisted redo log data from the public log buffer to the slave database.

[0006] Optionally, the step of having the primary database's transaction process write the generated redo log data to a common log buffer includes: The transaction process determines the starting position of the redo log data in the common log buffer; Determine if the remaining space in the public log buffer is sufficient for writing. If so, copy the redo log data from the starting position; If not, the transaction process triggers a log refresh and enters a waiting state until the remaining space meets the write requirements.

[0007] Optionally, the steps to determine whether the remaining space in the common log buffer is sufficient for writing include: Obtain the values ​​of the write pointer and the reclamation pointer associated with the public log buffer; Determine the remaining space in the logical address space between the value of the write pointer and the value of the garbage pointer; The remaining space in the public log buffer is compared with the amount of redo log data to determine whether the remaining space meets the write requirements.

[0008] Optionally, after the redo log data has been fully copied to the public log buffer, the method further includes: The write pointer associated with the public log buffer is atomically updated to reflect the newly written data.

[0009] Optionally, the step of having the transaction process simultaneously wake up the log flushing process and the streaming replication sending process of the primary database includes: Set a synchronization flag that is shared by the log flushing process and the stream replication sending process; The log flushing process and the stream replication sending process are woken up in parallel by the transaction process after detecting the change in the synchronization flag bit through polling.

[0010] Optionally, the log flushing process flushes the redo log data to be persisted from the common log buffer to the disk log file of the primary database, including: Obtain the disk flush pointer associated with the public log buffer; The data between the position indicated by the disk flushing pointer and the position indicated by the write pointer is determined as the first range of redo log data to be persisted in the public log buffer. The log flushing process reads the redo log data within the first range and flushes the redo log data within the first range to the disk log file of the main database.

[0011] Optionally, the streaming replication sending process sends the persisted redo log data from the common log buffer to the slave database, including: Obtain the send pointer associated with the public log buffer; The data between the position indicated by the send pointer and the position indicated by the flush pointer is determined as the second range of the persisted redo log data in the public log buffer. The streaming replication sending process reads the redo log data within the second range and sends the redo log data within the second range to the slave database.

[0012] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the parallel processing method for database redo logs described above.

[0013] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the steps of any of the above-described parallel processing methods for database redo logs.

[0014] According to another aspect of the present invention, a computer device is also provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any parallel processing method for database redo logs.

[0015] The parallel processing method for database redo logs in this invention advances the starting point of the streaming replication sending process from "log flushing completed" to "log flushing in progress," thereby overlapping network transmission time and disk I / O time. This allows redo log data to flow to the slave database in near real-time, shortening the replication latency between the master and slave clusters and improving real-time data availability. Following the principle that "the streaming replication sending process does not precede the log flushing process," only log data already persisted to the master database disk is sent. Even in the event of an extreme failure in the master database, the data sent to the slave database is guaranteed to be persisted, thus improving data reliability.

[0016] Furthermore, the parallel processing method for database redo logs of the present invention, by relying on flush pointers and write pointers, enables the log flushing process to atomically determine the first range that needs to be flushed in O(1) time complexity, thereby reducing the decision overhead of log flushing operations.

[0017] Furthermore, the parallel processing method for database redo logs of the present invention, by limiting the second range of sent data to between the send pointer and the flush pointer, eliminates the possibility of the streaming replication sending process reading or sending any redo log data that has not yet been persisted to the main database disk, thus ensuring data reliability.

[0018] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0019] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic flowchart of a parallel processing method for database redo logs according to an embodiment of the present invention; Figure 2 A schematic diagram of the process of writing redo log data to a common log buffer in a parallel processing method for database redo logs according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the write requirement judgment process of a parallel processing method for database redo logs according to an embodiment of the present invention. Figure 4 This is a flowchart illustrating the parallel wake-up of transaction processes in a parallel processing method for database redo logs according to an embodiment of the present invention. Figure 5 This is a flowchart illustrating the log flushing process in a parallel processing method for database redo logs according to an embodiment of the present invention. Figure 6 This is a flowchart illustrating the streaming replication and sending process in a parallel processing method for database redo logs according to an embodiment of the present invention. Figure 7 This is a timing diagram of a parallel processing method for database redo logs according to an embodiment of the present invention; Figure 8 This is a schematic diagram of a computer program product according to an embodiment of the present invention; Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; and Figure 10 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0020] In a primary / standby database cluster architecture, stream replication serves as a means to ensure data consistency and support high business availability. Its design logic and operational efficiency directly determine the cluster's service stability, data reliability, and fault recovery capabilities. This primary / standby cluster architecture uses the redo logs generated by the primary database during transaction execution as a data carrier to record data change details and ensure transaction atomicity and durability. These logs are then transmitted to the standby database, where they are used for log replay and data reconstruction. Ultimately, this achieves complete data synchronization between the primary and standby databases, ensuring that the standby database can quickly take over business operations in the event of a primary database failure, preventing data loss and service interruption. Therefore, the transmission efficiency, synchronization latency, and reliability of redo logs between the primary and standby databases not only directly affect the overall performance of the cluster but also relate to the timeliness of data synchronization and the time target for fault recovery, making them crucial indicators for measuring the high availability level of a primary / standby cluster.

[0021] Existing redo log synchronization generally adopts a serial processing path of "first local persistence, then remote transmission": after the master database generates redo logs in memory, it first persists the redo log data to the local physical disk through a disk flushing operation; after the local disk persistence is confirmed, the redo log data is reread from the local physical disk through disk I / O operations (Input / Output operations), and the network transmission process is started to send the read redo log data to the slave database through the cluster network link.

[0022] The existing redo log synchronization network transmission link is rigidly coupled with the local physical disk I / O operation. This forces the redo log synchronization link to go through a lengthy process of "memory generation → local physical disk persistence → disk data reading → network remote transmission". The network remote transmission link is forced to be placed after the local physical disk I / O operation, and must wait for the local disk flushing and disk reading process to be completed before it starts, resulting in excessive waiting overhead in the network transmission link. The read and write characteristics of the local physical disk (such as mechanical seek and head rotation) and the inherent latency of I / O operation introduce significant disk access latency into the redo log synchronization link. The combination of these two factors directly reduces the overall synchronization efficiency of redo logs and restricts the data synchronization throughput of the master-slave cluster.

[0023] As database workload increases, the rate of redo log generation continues to rise, further amplifying the performance bottleneck of I / O operations. This not only leads to a continuous increase in log synchronization latency among databases in the master-slave cluster, thus affecting data consistency, but also indirectly reduces the business processing performance of the slave database due to excessive I / O resource consumption of the master database, making it difficult to meet the synchronization requirements of the master-slave cluster in high-concurrency, low-latency business scenarios.

[0024] The parallel processing method for database redo logs and related products of the present invention are described below with reference to the accompanying drawings.

[0025] Figure 1 This is a schematic flowchart of a parallel processing method for database redo logs according to an embodiment of the present invention, as shown below. Figure 1 As shown, the parallel processing method for the database redo logs can generally include: In step S101, in response to the transaction commit, the transaction process of the primary database writes the generated redo log data into the common log buffer. In step S102, in response to the completion of the write operation, the transaction process wakes up the log flushing process and the stream replication sending process of the primary database in parallel. Step S103: The log flushing process flushes the redo log data to be persisted from the public log buffer to the disk log file of the main database. In step S104, in parallel with the write operation of the log flushing process, the streaming replication sending process sends the persisted redo log data from the common log buffer to the slave database.

[0026] In this embodiment, in response to the business transaction commit instruction received by the main database, the transaction process of the main database retrieves the redo log data (including transaction operation type, data modification content, transaction identifier, global time stamp, etc.) generated during the execution phase of the transaction. Following the database's preset redo log encoding format, data organization rules, and transaction sequence, the redo log data is completely written into a pre-allocated public log buffer in the main database. This public log buffer is a globally shared memory area configured for the main database, supporting high-concurrency writing by multiple transaction processes. It is globally accessible to the main database's log flushing process and streaming replication sending process. Internally, it employs a ring caching mechanism to manage data and preset write watermarks and protection thresholds, ensuring both unified aggregation and sequential temporary storage of redo log data from multiple transactions while preventing memory overflow. After the transaction process completes the log data writing, it marks the batch of redo log data in the public log buffer as completed (this completion mark is used to associate the transaction identifier and time stamp), confirming the integrity and validity of the data writing.

[0027] In this embodiment, after the transaction process detects that the redo log data has been written to the common log buffer and confirms through data verification that the redo log data is neither lost nor erroneous, the transaction process triggers wake-up commands in parallel to the log flushing process and the streaming replication sending process of the main database through the inter-process communication mechanism of the main database (such as semaphores or message queues). The wake-up of these two types of processes by the transaction process is a non-blocking trigger operation. After the trigger is completed, the relevant resources of the transaction process are released immediately without waiting for the wake-up response or subsequent operation feedback from any process. Only the redo log data that has been processed is marked with a "pending flush" or "pending transmission" status in the common log buffer and associated with a wake-up timestamp. This avoids the redo log data of this batch being processed repeatedly and provides a basis for tracing the status of subsequent process operations. In some scenarios, if multiple transaction processes complete the redo log writing consecutively, the wake-up commands will be triggered in superimposed order according to the transaction time sequence to ensure that the processing order of the redo log data by the two types of processes is consistent with the transaction commit order.

[0028] In this embodiment, after receiving the wake-up command, the log flushing process locates the redo log data range to be persisted based on the write completion flag and "pending flush" status indicator in the public log buffer. Following a batch read + sequential flush rule, it reads the corresponding batch of redo log data from the public log buffer. In the reading scenario, the log flushing process uses a lock-free memory read method (e.g., memory mapping mmap) to avoid lock contention with the memory access of the streaming replication sending process and to prevent subsequent transaction processes from blocking log write operations to the public log buffer. The log flushing process then flushes the read redo log data to the primary database's disk log file (including the local physical disk and storage media built on the local physical disk) according to the database redo log file storage specifications, completing local persistence.

[0029] After the write operation is completed, the log flushing process updates the status flag of the corresponding redo log data in the public log buffer, changing "pending flushing" to "flushed", and records the flushing completion timestamp and disk storage location information. At the same time, it sends a flushing completion notification to the log status management module of the main database, thus realizing the status closure of the local persistence operation.

[0030] In this embodiment, after the streaming replication sending process and the log flushing process receive the wake-up command synchronously, they start in parallel and execute independently with the local flushing operation of the log flushing process, and there is no execution timing dependency between the streaming replication sending process and the log flushing process. Based on the write completion flag and the "pending transmission" status flag in the common log buffer, the streaming replication sending process and the log flushing process respectively locate the same batch of redo log data range and read the redo log data from the common log buffer using a lock-free reading method originating from the same source as the flushing process. The reading process of the streaming replication sending process and the memory read operation of the log flushing process do not interfere with each other. After the streaming replication sending process reads the persisted redo log data, it first performs lightweight data verification (including checking data integrity and timing correctness), and then encapsulates the persisted redo log data into transmission data packets according to the master-slave cluster's preset streaming replication communication protocol (such as TCP / IP protocol), and sends them to the log receiving buffer of the slave database in real time. During transmission, the streaming replication process continuously reports the transmission progress to the master database. If an anomaly occurs in the master-slave network link, the streaming replication process will trigger a resume mechanism, resuming transmission from the breakpoint based on the timestamp of the previously transmitted redo log data, thus preventing data loss. When the slave database successfully receives the redo log data, it returns a reception confirmation signal to the master database. The streaming replication process updates the status flag of the corresponding redo log data in the common log buffer, changing "pending transmission" to "transmitted," and records the transmission completion timestamp, slave database node identifier, and other information. It then sends a transmission completion signal back to the master database's log status management module.

[0031] It should be noted that after the redo log data has been written to the common log buffer, and the log flushing process and streaming replication sending process have been awakened and started working, the primary database reaches the "commit waiting point." This "commit waiting point" branches according to the preset replication strategy. If the replication strategy is synchronous replication mode, the primary database simultaneously waits for "local persistence confirmation from the log flushing process, indicating that the redo log data has been successfully flushed to the primary database's disk log file" and "remote synchronization confirmation from the streaming replication sending process, indicating that at least one slave database has successfully received the persisted redo log data." The primary database only completes the transaction commit after receiving both the local persistence confirmation and the remote synchronization confirmation.

[0032] If the replication strategy is asynchronous replication mode, the master database only waits for "local persistence confirmation from the log flushing process, indicating that the redo log data has been successfully flushed to the master database disk log file". When the local persistence confirmation is received, the master database immediately completes the transaction commit without waiting for the slave database response.

[0033] Through the above steps, the database advances the starting point of the streaming replication sending process from "log flushing completed" to "log flushing in progress," achieving overlap between network transmission time and disk I / O time. This allows redo log data to flow to the slave database in near real-time, shortening the replication latency between the master and slave clusters and improving real-time data availability. By ensuring that the streaming replication sending process does not precede the log flushing process and only sends log data already persisted to the master database disk, even in the event of an extreme failure in the master database, the data sent to the slave database is guaranteed to be persisted, thereby improving data reliability.

[0034] Figure 2 A schematic diagram illustrating the process of writing redo log data to a common log buffer in a parallel processing method for database redo logs according to an embodiment of the present invention is shown below. Figure 2 As shown, the process by which the transaction process of the primary database writes the generated redo log data to the common log buffer may include: Step S201: The transaction process determines the starting position of the redo log data in the common log buffer; Step S202: Determine whether the remaining space in the public log buffer meets the writing requirements; Step S203: If yes, copy the redo log data from the starting position; Step S204: If not, the transaction process triggers log refresh and enters a waiting state until the remaining space meets the write requirements.

[0035] Through the above steps, the transaction process of the main database avoids space overflow caused by blind writing by pre-judging the remaining space in the buffer, ensuring that the storage resources of the public log buffer are allocated in an orderly and reasonable manner, and improving the utilization efficiency of memory resources; when space is insufficient, it triggers log refresh and waits, avoiding the risk of transaction write failure or data overwriting due to insufficient space in the public log buffer, ensuring that redo log data can be written completely and safely to the public log buffer, and guaranteeing the atomicity of transactions and data consistency.

[0036] Figure 3 This is a flowchart illustrating the write requirement judgment process of a parallel processing method for database redo logs according to an embodiment of the present invention, as shown below. Figure 3 As shown, the process of determining whether the remaining space in the common log buffer is sufficient for writing can include: Step S301: Obtain the values ​​of the write pointer and the recycling pointer associated with the public log buffer; Step S302: Determine the remaining space in the logical address space between the value of the write pointer and the value of the garbage collector; Step S303: By comparing the remaining space with the amount of redo log data, it is determined whether the remaining space of the public log buffer meets the writing requirements.

[0037] By following the steps above, the actual remaining space of the public log buffer is located (rather than a rough estimate) based on the logical address difference between the write pointer and the reclamation pointer. This avoids errors in space judgment and ensures a good match between the write requirements of redo log data and the capacity of the public log buffer. The calculation of the pointer's logical address reflects the dynamic available space of the public log buffer in real time (unaffected by physical storage fragmentation), allowing transaction processes to fully utilize the effective space of the public log buffer, reducing premature refreshes or waiting caused by space misjudgments, and optimizing memory resource turnover efficiency.

[0038] When the remaining space in the public log buffer meets the write requirements, the transaction process of the primary database writes the generated redo log data to the public log buffer. After the redo log data has been completely copied to the public log buffer, the write pointer associated with the public log buffer is atomically updated to reflect the newly written data, thus determining that the operation of writing redo log data to the public log buffer has been completed.

[0039] Figure 4 This is a flowchart illustrating the parallel wake-up of transaction processes in a parallel processing method for database redo logs according to an embodiment of the present invention, as shown below. Figure 4 As shown, the steps for the transaction process to simultaneously wake up the primary database's log flushing process and streaming replication sending process can include: Step S401: Set a synchronization flag that is shared and monitored by the log flushing process and the stream replication sending process; In step S402, the log flushing process and the stream replication sending process are woken up in parallel by the transaction process after detecting the change in the synchronization flag bit through polling.

[0040] By setting a shared synchronization flag in the primary database through the above steps, the transaction process can achieve a one-time, non-blocking parallel wake-up of the disk flushing and sending processes with extremely low atomic write operation overhead, thereby immediately starting subsequent persistence and transmission tasks. Through polling detection, sub-microsecond response latency is achieved, ensuring almost zero delay in task startup and minimizing the window between log writing and log transmission.

[0041] Figure 5 This is a flowchart illustrating the log flushing process in a parallel processing method for database redo logs according to an embodiment of the present invention, as shown below. Figure 5 As shown, the steps of the log flushing process to write the redo log data to be persisted from the common log buffer to the disk log file of the primary database include: Step S501: Obtain the disk flush pointer associated with the public log buffer; Step S502: The data between the position indicated by the disk flush pointer and the position indicated by the write pointer is determined as the first range of the redo log data to be persisted in the common log buffer. In step S503, the log flushing process reads the redo log data within the first range and flushes the redo log data within the first range to the disk log file of the main database.

[0042] Through the above steps, relying on the flush pointer and write pointer, the log flushing process can atomically determine the first range that needs to be flushed in O(1) time complexity, reducing the decision-making overhead of log flushing operations, ensuring that each flushing operation has a clear target and data integrity, avoiding full buffer scans, and significantly reducing computation and judgment overhead. Subsequently, single or batch I / O writes are performed directly based on the continuous first range, optimizing the granularity of disk operations, reducing the number of I / O operations and addressing overhead, and achieving high throughput and low latency performance of the log persistence path with extremely low CPU and memory management costs.

[0043] Figure 6 This is a flowchart illustrating the streaming replication and sending process in a parallel processing method for database redo logs according to an embodiment of the present invention, as shown below. Figure 6 As shown, the steps of the streaming replication sending process to send persistent redo log data from the common log buffer to the slave database include: Step S601: Obtain the send pointer associated with the public log buffer; Step S602: The data between the position indicated by the sending pointer and the position indicated by the flush pointer is determined as the second range of the persisted redo log data in the public log buffer; In step S603, the streaming replication sending process reads the redo log data in the second range and sends the redo log data in the second range to the slave database.

[0044] By introducing the corresponding send pointer and comparing it with the flush pointer, the network transmission progress is decoupled from the local physical disk I / O operation progress. By limiting the second range of the sent data to between the send pointer and the flush pointer, the possibility of the streaming replication sending process reading or sending any redo log data that has not yet been persisted to the main database disk is eliminated, thus ensuring data reliability.

[0045] As an example, Figure 7 This is a timing diagram of a parallel processing method for database redo logs according to an embodiment of the present invention.

[0046] like Figure 7As shown, in the normal process, in response to a transaction commit, the transaction process writes redo log data to a shared log buffer. After writing, the transaction process concurrently wakes up the primary database's log flushing process and streaming replication sending process. The log flushing process flushes the redo log data to be persisted to the primary database's disk log file. After flushing, the flushing pointer is updated. The redo log data between the sending pointer and the flushing pointer is determined to be safe to send. The streaming replication sending process sends the persisted redo log data to the slave database. The slave database returns an ACK (acknowledgment) to the streaming replication sending process. The streaming replication sending process reports the sending progress to the log flushing process.

[0047] like Figure 7 As shown, when a transmission failure or timeout (network interruption) occurs, the stream replication sending process pauses transmission and records the current breakpoint position (i.e., the position of the send pointer); the stream replication sending process initiates a retry mechanism and waits to restore the connection from the database; after the connection is restored from the database, the stream replication sending process retransmits the redo log data from the previously recorded breakpoint position to avoid data duplication or omission.

[0048] like Figure 7 As shown, by waking up the log flushing process and the streaming replication sending process in parallel, the existing solution is broken, reducing the master-slave synchronization latency from "flushing time + network time" to the maximum of the two, significantly improving data synchronization real-time performance and system throughput. Through the coordinated mechanism of the flushing pointer and the sending pointer, it is ensured that all data transmitted over the network is persisted, guaranteeing strong data consistency. The breakpoint recording and retransmission mechanism in exception handling achieves reliable recovery and zero data loss under network failures, significantly optimizing the performance of the database master-slave cluster while ensuring data reliability.

[0049] The flowcharts provided in this embodiment are not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0050] It should be understood that in some embodiments, the components may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0051] This embodiment also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. For example... Figure 8As shown, the computer program product 10 includes a computer program 11, which, when executed by the processor 32, implements the steps of any of the above-described methods for inserting semi-structured data into a database. Figure 9 As shown, a computer-readable storage medium 20 stores the aforementioned computer program 11, which, when executed by the processor 32, implements the steps of the method for inserting semi-structured data into a database according to any of the above embodiments. Figure 10 As shown, the computer device 30 may include a memory 31, a processor 32, and a computer program 11 stored on the memory 31 and running on the processor 32.

[0052] The computer program 11 used to perform the operations of this invention may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, Field-Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing state information from computer-readable program instructions.

[0053] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0054] For the purposes of this embodiment, the computer-readable storage medium 20 is a tangible device capable of holding and storing a computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting the computer program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0055] Computer device 30 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 30 can be a cloud computing node. Computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 30 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0056] Computer device 30 may include a processor 32 adapted to execute stored instructions and a memory 31 that provides temporary storage space for the operation of said instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0057] Computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows external devices that can be connected to the computer device to input and output data. The network adapter / interface provides communication between the computer device and a network, typically represented as a communication network.

[0058] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A parallel processing method for database redo logs, comprising: In response to a transaction commit, the primary database's transaction process writes the generated redo log data to a common log buffer. In response to the completion of the write operation, the transaction process simultaneously wakes up the log flushing process and the streaming replication sending process of the primary database. The log flushing process flushes the redo log data to be persisted from the public log buffer to the disk log file of the main database. In parallel with the write operation of the log flushing process, the streaming replication sending process sends the persisted redo log data from the public log buffer to the slave database.

2. The parallel processing method for database redo logs according to claim 1, wherein, The steps by which the primary database's transaction process writes the generated redo log data to the common log buffer include: The transaction process determines the starting position of the redo log data in the common log buffer; Determine if the remaining space in the public log buffer is sufficient for writing. If so, copy the redo log data from the starting position; If not, the transaction process triggers a log refresh and enters a waiting state until the remaining space meets the write requirements.

3. The parallel processing method for database redo logs according to claim 2, wherein, The steps to determine whether the remaining space in the common log buffer is sufficient for writing requirements include: Obtain the values ​​of the write pointer and the reclamation pointer associated with the public log buffer; Determine the remaining space in the logical address space between the value of the write pointer and the value of the garbage pointer; The remaining space in the public log buffer is compared with the amount of redo log data to determine whether the remaining space meets the write requirements.

4. The parallel processing method for database redo logs according to claim 1, wherein, After the redo log data has been fully copied to the public log buffer, the process also includes: The write pointer associated with the public log buffer is atomically updated to reflect the newly written data.

5. The parallel processing method for database redo logs according to claim 1, wherein, The steps by which the transaction process concurrently wakes up the log flushing process and the streaming replication sending process of the primary database include: Set a synchronization flag that is shared by the log flushing process and the stream replication sending process; The log flushing process and the stream replication sending process are woken up in parallel by the transaction process after detecting the change in the synchronization flag bit through polling.

6. The parallel processing method for database redo logs according to claim 1, wherein, The steps of the log flushing process, which flushes the redo log data to be persisted from the public log buffer to the disk log file of the main database, include: Obtain the disk flush pointer associated with the public log buffer; The data between the position indicated by the disk flushing pointer and the position indicated by the write pointer is determined as the first range of redo log data to be persisted in the public log buffer. The log flushing process reads the redo log data within the first range and flushes the redo log data within the first range to the disk log file of the main database.

7. The parallel processing method for database redo logs according to claim 1, wherein, The streaming replication sending process sends the persisted redo log data from the common log buffer to the slave database, including the following steps: Obtain the send pointer associated with the public log buffer; The data between the position indicated by the send pointer and the position indicated by the flush pointer is determined as the second range of the persisted redo log data in the public log buffer. The streaming replication sending process reads the redo log data within the second range and sends the redo log data within the second range to the slave database.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the parallel processing method for database redo logs as described in any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the parallel processing method for database redo logs as described in any one of claims 1 to 7.

10. A computer device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the parallel processing method for database redo logs according to any one of claims 1 to 7.

Images