Log replication method, apparatus, and storage medium

Abstract

Embodiments of the present application provide a log replication method and device and storage medium, relating to the technical field of database, comprising: receiving a write request sent by a client, the write request being sent by the client through RDMA multicast, the write request being used to instruct a standby end to copy a log from a host end, reading metadata of a log corresponding to the write request from a ring array of the host end through an RDMA read operation according to the write request, the log being generated by the host end according to the write request received from the client, and copying the log according to the metadata of the log, wherein the client, the host and the standby are in the same multicast. The client, the host and the standby are arranged in the same multicast, and the standby directly reads log data of the host through RDMA multicast, thereby avoiding multiple negotiation communications with the host in the replication process, improving the replication efficiency and reducing the CPU and memory resource overhead of the host.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a log replication method, apparatus and storage medium. Background Technology

[0002] Replication is a crucial aspect of database technology, enabling data to be kept synchronized between servers.

[0003] In existing technologies, write-ahead log-based replication is a commonly used solution for server data synchronization. It involves the primary server sending write-ahead logs to the standby server, and the standby server receiving the write-ahead logs to achieve data synchronization.

[0004] During this process, the primary and backup machines need to communicate and confirm multiple times, resulting in low replication efficiency. Furthermore, the write-ahead logs need to be copied multiple times before being sent to the backup machine, leading to significant CPU and memory resource overhead on the primary machine. Summary of the Invention

[0005] This application provides a log replication method, apparatus, and storage medium. By employing RDMA multicast for log replication, the overhead of host CPU and memory resources during the log replication process is reduced.

[0006] In a first aspect, embodiments of this application provide a log replication method applied to a standby machine, the method comprising:

[0007] Receive a write request sent by a client, the write request being sent by the client via Remote Direct Memory Access (RDMA) multicast, the write request being used to instruct the standby machine to copy logs from the master machine;

[0008] Based on the write request, the host reads the metadata of the log corresponding to the write request from the ring array on the host side via an RDMA read operation. The log is generated by the host side based on the write request received from the client.

[0009] The logs are replicated based on the log metadata, wherein the client, the host, and the backup machine are in the same multicast.

[0010] Optionally, the step of replicating the log based on the log's metadata includes:

[0011] The status of the log is obtained based on the metadata;

[0012] If the log is in a committed state, then the metadata is verified for integrity based on the checksum in the metadata; wherein, the committed state indicates that the host has generated the log;

[0013] If the integrity verification passes, the logs are read from the host's storage according to the log pointer in the metadata, and the read logs are stored in the local storage of the standby machine.

[0014] Optionally, after storing the read logs in the local storage of the standby machine, the method further includes:

[0015] The standby machine sends a reply message to the host machine via an RDMA write operation. The reply message indicates that the standby machine has completed the log replication, which enables the host machine to set the status of the log metadata in the ring array to expired.

[0016] Optionally, after receiving the write request sent by the client, the method further includes:

[0017] The write request is cached in the request list;

[0018] Before reading the metadata of the log corresponding to the write request from the ring array on the host side via an RDMA read operation based on the write request, the method further includes:

[0019] The write request is retrieved from the request list according to the first-in, first-out (FIFO) principle.

[0020] Secondly, embodiments of this application provide a log replication method applied to a host, the method comprising:

[0021] Receive a write request sent by a client, the write request being sent by the client via Remote Direct Memory Access (RDMA) multicast, the write request being used to instruct the standby machine to copy logs from the master machine;

[0022] Logs are generated according to the write request, and the metadata of the logs is written to a circular array so that the standby machine can read the metadata of the logs from the circular array through RDMA read operations and obtain the logs from the host based on the metadata. The client, the host, and the standby machine are in the same multicast.

[0023] Optionally, after generating the log based on the write request, the method further includes:

[0024] Set the status of the log in the metadata to a committed state. The committed state indicates that the host has generated the log and that the standby machine can read the log from the host.

[0025] Optionally, a reply message is received from the standby machine, the reply message indicating that the standby machine has completed the log replication;

[0026] Set the log status in the metadata to an expired state, which is used to instruct the standby machine to complete the replication of the log;

[0027] A write response is sent to the client, indicating that the write request has been completed.

[0028] Thirdly, embodiments of this application provide a log replication device applied to a standby machine, comprising:

[0029] The receiving module is used to receive a write request sent by a client, wherein the write request is sent by the client via Remote Direct Memory Access (RDMA) multicast, and the write request is used to instruct the standby machine to copy logs from the master machine.

[0030] The acquisition module is used to read the metadata of the log corresponding to the write request from the ring array on the host side through an RDMA read operation, wherein the log is generated by the host side based on the write request received from the client;

[0031] The replication module is used to replicate the logs based on the log's metadata, wherein the client, the host, and the standby machine are in the same multicast.

[0032] Optionally, the log replication apparatus described above can perform the log replication method described in any of the first aspects.

[0033] Fourthly, embodiments of this application provide a log replication device, applied to a host, comprising:

[0034] The receiving module is used to receive a write request sent by a client, wherein the write request is sent by the client via Remote Direct Memory Access (RDMA) multicast, and the write request is used to instruct the standby machine to copy logs from the master machine.

[0035] The writing module is used to generate logs according to the write request and write the metadata of the logs into a circular array, so that the standby machine can read the metadata of the logs from the circular array through RDMA read operation and obtain the logs from the host based on the metadata. The client, the host, and the standby machine are in the same multicast.

[0036] Optionally, the log replication apparatus described above can perform the log replication method described in any of the second aspects.

[0037] Fifthly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0038] The memory is used to store computer instructions; the processor is used to execute the computer instructions stored in the memory to implement the method of any one of the first and / or second aspects.

[0039] In a sixth aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method described in any one of the first and / or second aspects.

[0040] In a seventh aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any one of the first and / or second aspects.

[0041] The log replication method, apparatus, and storage medium provided in this application receive write requests sent by clients via RDMA multicast. These write requests instruct a standby machine to replicate logs from the master machine. Based on the write request, metadata of the corresponding log is read from a circular array on the master machine via RDMA read operations. The logs are generated by the master machine based on the write requests received from the clients. The logs are replicated based on their metadata. The client, master machine, and standby machine are all in the same multicast group. By placing the client, master machine, and standby machine in the same multicast group, the standby machine can directly read the master machine's log data via RDMA multicast, avoiding multiple negotiation communications with the master machine during replication, thus improving replication efficiency and reducing the CPU and memory resource overhead of the master machine. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of a scenario provided for an embodiment of this application;

[0043] Figure 2 A flowchart illustrating the log replication method provided in this application embodiment. Figure One ;

[0044] Figure 3 This is a schematic diagram of the structure of the costume changing array provided in the embodiments of this application;

[0045] Figure 4 A flowchart illustrating the log replication method provided in this application embodiment. Figure Two ;

[0046] Figure 5 A flowchart illustrating the log replication method provided in this application embodiment. Figure Three ;

[0047] Figure 6 Schematic diagram of the log replication device provided in the embodiments of this application Figure One ;

[0048] Figure 7 Schematic diagram of the log replication device provided in the embodiments of this application Figure Two ;

[0049] Figure 8 This is a schematic diagram of the structure of the electronic log replication settings provided in an embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0051] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0052] 1) Remote Direct Memory Access: Remote Direct Memory Access (RDMA) is a remote direct data storage technology. RDMA can quickly move data from local to remote memory over a network without getting caught in the operating system kernel calls, which can reduce the CPU and memory consumption caused by data copying.

[0053] 2) Write-ahead Log: The write-ahead log (WAL) is a series of techniques used in relational databases to ensure atomicity and durability. In a system using WAL, all data modifications need to be written to the WAL file before they are committed.

[0054] 3) Multicast: Multicast, also known as multi-target broadcast, is a message transmission method used in networks. It allows a message to be sent to a selected subset of all possible destinations, i.e., sending information to multiple explicitly specified addresses. Specifically, in multicast, when a source host (master) requests to send information to multiple target hosts (backup hosts), the source host sends only one copy of the data. The destination address of the data is the multicast group address, so that all members of that group can receive a copy of the data sent by the source host.

[0055] 4) Other terms

[0056] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect, without limiting their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0057] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0058] In the database field, replication is a crucial technology. The fundamental problem it solves is keeping the data on one server synchronized with that of another server—a process known as master-slave replication. Write-ahead log (WAP)-based replication is a common solution for achieving master-slave synchronization.

[0059] In existing technologies, most master-slave replication solutions communicate via traditional TCP-IP networks. The master actively sends data, and the slave receives it. This process involves extensive negotiation communication between the master and slave, leading to low replication efficiency. For example, in MySQL master-slave replication, both ends establish a TCP connection, communicating between the master and slave. During this communication, the master obtains the log range to be sent, performs multiple memory copies, and then sends the data to the slave via system calls. This incurs significant CPU and memory resource overhead on the master.

[0060] In view of this, this application provides a log replication method that uses RDMA multicast technology, in which the host and standby are in the same multicast. When performing log replication, the standby can directly read the host's log data through RDMA read operations without needing to negotiate with the host, thus reducing the host's CPU and memory resource overhead.

[0061] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be implemented independently or in combination with each other. The same or similar concepts or processes may not be described again in some embodiments.

[0062] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this application, such as... Figure 1 As shown, it includes a host terminal 101, a standby terminal 102, and a client terminal 103.

[0063] The host terminal 101, the standby terminal 102, and the client terminal 103 together form a system that can communicate with each other via a network.

[0064] In this embodiment of the application, the client 103 generates data during operation, and the host 101 writes the data to the local storage of the host 101. After receiving the information that the client 103 generates data, the standby 102 reads the data from the storage of the host 101 and persists the data to its own storage to complete the data synchronization between the host 101 and the standby 102.

[0065] In this embodiment of the application, the host 101 writes the data generated by the client 103 to the local storage of the host 101 in the manner of writing a log ahead. Therefore, the standby 102 realizes data synchronization with the host 101 by copying the write-ahead log in the host 101.

[0066] In this embodiment, when the host 101, the standby 102, and the client 103 establish an initial connection (initialization), they register with the same multicast. Message transmission between the host 101, the standby 102, and the client 103 is achieved through their respective queue pairs (QPs). Each QP includes a send queue (SQ) and a receive queue (RQ). The SQ is used to send messages, and the RQ is used to receive messages. The host 101, the standby 102, and the client 103 can transmit messages through their respective QPs.

[0067] Specifically, during initialization, the host 101, the standby 102, and the client 103 each initialize a QP and a monitoring event completion channel. The monitoring event completion channel is used to monitor the status of each node (host 101, standby 102, and client 103). Each node registers its corresponding QP and monitoring event completion channel to the same multicast according to its configuration file. The monitoring event completion channel is used to determine whether each node has successfully joined the same multicast. For example, the monitoring event completion channel can determine whether each node has successfully joined by sending a message and checking whether a reply is received.

[0068] It is understood that there can be multiple backup terminals in this application embodiment. When it is determined that each node has successfully joined the multicast, the system initialization is completed. If there are multiple backup terminals, and one of the backup terminals has not joined the multicast, it will not affect the system initialization. The host terminal can send a message to prompt the backup terminal to leave the multicast.

[0069] The application scenarios of the embodiments of this application have been briefly described above. The following section describes applications in... Figure 1 Taking the standby machine as an example, the log replication method provided in this application embodiment will be described.

[0070] Figure 2 A flowchart illustrating the log replication method provided in this application embodiment. Figure One In this embodiment of the application, the execution entity is the standby machine, such as... Figure 2 As shown, it includes the following steps:

[0071] S201. Receive a write request sent by the client. The write request is sent by the client via RDMA multicast and is used to instruct the standby machine to copy the logs from the master machine.

[0072] In this embodiment of the application, a write request refers to a request from the client to write the generated data to the storage on the host side during operation. At the same time, the write request is also used to instruct the standby side to copy the data written to the storage on the host side from the host side. The data is stored in the form of a write-ahead log (hereinafter referred to as log). Therefore, copying data from the host side means copying the corresponding log from the host side.

[0073] In this embodiment, the client, host, and standby are in the same multicast. Therefore, the client can directly send write requests to the host and standby via RDMA multicast, i.e., write the corresponding write request into the client's SQ. The SQ sends the write request to the host and standby based on the corresponding address handle (AH). The host and standby can receive the write request from their respective RQs. The address handle is a set of information used to locate a remote node; that is, a pointer to a remote node. The client can obtain the corresponding address handle during initialization based on the configuration file.

[0074] After receiving a write request from the client, the standby machine can execute the subsequent process of replicating logs from the master machine.

[0075] S202. Based on the write request, read the metadata of the log corresponding to the write request from the ring array on the host side through RDMA read operation. The log is generated by the host side based on the write request received from the client.

[0076] In this embodiment, the ring array is a storage structure set in the host memory for storing log metadata.

[0077] like Figure 3 As shown, in this embodiment, the ring array is divided into four regions: active region A, submitted region B, expired region C, and idle region D. Each region is used to store the metadata of the corresponding log. The size of each region can be set according to actual needs, and this embodiment does not limit this.

[0078] In this embodiment of the application, metadata refers to a data structure used to represent log attributes, including the log's status and pointer. The log's status is divided into active status, committed status, and expired status, and the pointer refers to the physical address of the log.

[0079] The active status of a log indicates that data is being written to the log. The committed status of a log indicates that the data has been written, but the standby machine has not replicated the log. The expired status of a log indicates that the standby machine has completed the replication of the log.

[0080] In this embodiment of the application, after receiving a write request from the client, the standby machine reads the metadata of the log corresponding to the write request from the ring array on the host machine through an RDMA read operation, and performs the subsequent log replication process based on the metadata.

[0081] S203. Replicate the logs based on their metadata, where the client, host, and standby are in the same multicast.

[0082] In this embodiment, after obtaining the log's metadata, the system determines whether the log can be replicated based on its status in the metadata. If it can, the standby machine replicates the log from the host's storage using the log pointer in the metadata. For example, if the log's status is "committed," the system obtains the log's storage address from the metadata and replicates the log based on that address.

[0083] It is understandable that copying a log file can mean copying the entire log file or copying only a portion of the data within the log file.

[0084] Optionally, the replication of logs in the committed state is an asynchronous replication method. The replication in this embodiment can also be synchronous replication, that is, the logs in the active state are replicated by obtaining the data written to the logs by the host in real time, thus realizing the replication of the logs.

[0085] The log replication method provided in this application receives a write request sent by a client via RDMA multicast. The write request instructs the standby machine to replicate logs from the master machine. Based on the write request, the metadata of the log corresponding to the write request is read from the ring array on the master machine via an RDMA read operation. The logs are generated by the master machine based on the write request received from the client. The logs are replicated based on their metadata. The client, master machine, and standby machine are all in the same multicast group. By placing the client, master machine, and standby machine in the same multicast group, the standby machine can directly read the master machine's log data via RDMA multicast, avoiding multiple negotiation communications with the master machine during replication, thus improving replication efficiency and reducing the CPU and memory resource overhead of the master machine.

[0086] The log replication method provided in this application embodiment has been described above using a standby machine as an example. The log replication method provided in this application embodiment will be described below using a host machine as an example.

[0087] Figure 4 A flowchart illustrating the log replication method provided in this application embodiment. Figure Two In this application embodiment, the execution entity is the host side, such as... Figure 4 As shown, it includes the following steps:

[0088] S401. Receive a write request sent by the client. The write request is sent by the client via RDMA multicast and is used to instruct the standby machine to copy the logs from the master machine.

[0089] The implementation method of step S401 in this application is the same as Figure 2 In the illustrated embodiment, the implementation of S201 differs only in the execution subject; the rest is similar and can be executed by referring to the implementation of S201. It will not be elaborated here.

[0090] S402. Generate a log according to the write request and write the log metadata into a ring array so that the standby machine can read the log metadata from the ring array through RDMA read operation and obtain the log from the host based on the metadata. The client, host, and standby machine are in the same multicast.

[0091] In this embodiment of the application, after the host receives the write request sent by the client, it will write the data generated by the client into the log according to the write request, and write the metadata of the log into a circular array in memory.

[0092] When writing data, the host can set the log status in the metadata to active and write the metadata to the active area of ​​the circular array in memory. When the host completes the data writing in the log, it will change the status of the log.

[0093] Specifically, the status of the log in the metadata is set to committed. The committed status indicates that the host has generated the log, and the committed status indicates that the standby server can read the log from the host.

[0094] After the host changes the log status in the metadata to a committed state, it also migrates the metadata to the committed zone.

[0095] Optionally, after the master server completes log replication on the standby server, it will change the status of the logs again and send a message reply to the client.

[0096] Specifically, it receives a reply message from the standby server, which indicates that the standby server has completed log replication; it sets the log status in the metadata to an expired state, which indicates that the standby server has completed log replication; and it sends a write response to the client, which indicates that the write request has been completed.

[0097] In this embodiment of the application, after receiving the reply message sent by the standby machine, the host will set the log status in the corresponding metadata to an expired state and migrate the metadata to the expired area. The reply message is sent via RDMA multicast, that is, the host receives the reply message sent by the standby machine via SQ through RQ.

[0098] When the host migrates the metadata to the expiration zone, it sends a write response to the client, indicating that the write request sent by the client has been completed. Similarly, the write response is also sent via RDMA multicast.

[0099] It is understood that the process of setting the log status in the metadata on the host side can be executed after the host side sends a write response to the client, or it can be executed at the same time as the host side sends a write response to the client. This application embodiment does not restrict the timing of the host side executing the above two processes.

[0100] Optionally, the host can delete metadata in the expired region based on the size of the metadata in the expired region of the ring array. For example, if the size of the metadata in the expired region exceeds 85% of the size of the expired region, the metadata in it will be deleted. The specific deletion size can be set according to actual needs.

[0101] The log replication method provided in this application receives a write request sent by a client via RDMA multicast. The write request instructs the standby server to replicate logs from the master server. Logs are generated based on the write request, and the log metadata is written to a circular array. The standby server then reads the log metadata from the circular array via RDMA read operations and retrieves the logs from the master server based on the metadata. The client, master server, and standby server are in the same multicast. The master server only needs to send the log metadata to the circular array for the standby server to retrieve the logs via RDMA multicast, eliminating the need for multiple negotiation communications during the replication process, thus improving replication efficiency. Furthermore, when there are multiple standby servers, the master server only needs to send the metadata once to enable multiple standby servers to replicate logs, alleviating the performance bottleneck of the master server.

[0102] Figure 5 This is a flowchart illustrating the log replication method provided in the embodiments of this application. Figure Three ,exist Figure 2Based on the illustrated embodiment, the process of log replication on the standby machine is further explained, such as... Figure 5 As shown, it includes the following steps:

[0103] S501. Cache the write request in the request list. The write request is sent by the client via remote RDMA multicast.

[0104] In this embodiment of the application, the request list refers to a list used by the standby machine to store multiple write requests, and the write requests in the list are arranged in order of their generation time.

[0105] During operation, the client generates multiple write requests. Correspondingly, the standby server performs log replication from the master server based on these multiple write requests. The number of write requests may exceed the number of threads that the standby server can run simultaneously. Therefore, when the standby server receives a write request from the client, it caches the write request in a request list. When the standby server has available threads, it reads the write request from the request list to perform the log replication process from the master server.

[0106] Understandably, when reading or writing requests, write requests are retrieved from the request list according to a first-in, first-out (FIFO) principle. That is, write requests that were generated earlier are retrieved from the request list first.

[0107] S502. Obtain the status of the log based on the metadata, and verify the integrity of the metadata based on the checksum in the metadata.

[0108] In this embodiment of the application, errors may occur when the host writes metadata into the ring array. In order to ensure the correctness of the log copied based on the metadata, the standby device needs to verify the integrity of the read metadata.

[0109] For example, the host machine verifies the metadata to be written according to a preset verification algorithm, generates a verification code, and writes the verification code into the metadata. After reading the metadata, the standby machine verifies the metadata according to the same verification algorithm and compares the generated verification result with the verification result in the metadata. If they are the same, the metadata is complete; if they are different, the metadata is incomplete, and corresponding incomplete information is generated. The verification algorithm in this embodiment can be MD4, MD5, or other verification algorithms, and this embodiment does not limit this.

[0110] In this embodiment, after the standby machine obtains a write request from the request list, it reads the metadata of the log corresponding to the write request from the ring array on the host machine through an RDMA read operation. When the log status in the metadata is "committed", the metadata integrity is verified by the checksum in the metadata. When the metadata is complete, the subsequent replication process is executed.

[0111] Optionally, if the standby machine performs log replication as synchronous replication, then the integrity of the metadata corresponding to the log status being active is verified. If the metadata is complete, the subsequent replication process is executed.

[0112] S503. Based on the log pointer in the metadata, read the log from the host's storage and store the read log in the standby's local storage.

[0113] In this embodiment, after verifying the integrity of the corresponding metadata, the standby machine reads the log corresponding to the write request from the corresponding location in the host storage according to the log pointer in the metadata. Then, it writes the log to the local storage of the standby machine, thus performing persistent processing on the log.

[0114] S504. Send a reply message to the master unit via RDMA write operation. The reply message is used to indicate that the standby unit has completed the log replication.

[0115] In this embodiment, after completing log persistence, the standby machine sends a reply message to the master machine, so that the master machine knows that the standby machine has completed the log replication corresponding to the write request. This reply message is sent via RDMA multicast; that is, the standby machine writes the reply message to the SQ (Server Queued Message), and the master machine can receive the reply message through the RQ.

[0116] The log replication method provided in this application caches write requests in a request list. Write requests are sent by the client via remote RDMA multicast. The log status is obtained based on metadata, and the integrity of the metadata is verified using a checksum in the metadata. Based on the log pointer in the metadata, the log is read from the host's storage and stored in the standby's local storage. A reply message is sent to the host via an RDMA write operation, indicating that the standby has completed log replication. By verifying the integrity of the metadata during the log replication process, the accuracy of log replication is improved.

[0117] Based on the above-described log replication method embodiments, this application also provides a log replication apparatus.

[0118] Figure 6 This is a schematic diagram of the structure of the log replication device 60 provided in the embodiments of this application, which is applied to the standby machine. Figure 6As shown, it includes:

[0119] The receiving module 601 is used to receive write requests sent by the client. The write requests are sent by the client via Remote Direct Memory Access (RDMA) multicast and are used to instruct the standby machine to copy logs from the master machine.

[0120] The acquisition module 602 is used to read the metadata of the log corresponding to the write request from the ring array on the host side through RDMA read operation according to the write request. The log is generated by the host side according to the write request received from the client.

[0121] The replication module 603 is used to replicate logs based on the log's metadata, where the client, host, and standby are in the same multicast.

[0122] Optionally, the acquisition module 602 is also used to obtain the status of the log based on the metadata; if the status of the log is committed, the integrity of the metadata is verified based on the check code in the metadata; wherein, the committed status is used to indicate that the host has generated the log.

[0123] Optionally, the acquisition module 602 is also used to cache write requests in a request list and retrieve write requests from the request list according to the first-in-first-out principle.

[0124] Optionally, the log replication device 60 may also include a sending module 604.

[0125] The sending module 604 is used to send a reply message to the master end via RDMA write operation. The reply message is used to indicate that the standby end has completed the log replication and to enable the master end to set the status of the log metadata in the ring array to the expired state.

[0126] The log replication apparatus provided in this application embodiment can perform the above-described... Figure 2 and Figure 5 The technical solution of the log replication method embodiment shown is similar in principle and effect, and will not be described again here.

[0127] Figure 7 This is a schematic diagram of the structure of the log replication device 70 provided in the embodiments of this application, applied to the host side, such as... Figure 7 As shown, it includes:

[0128] The receiving module 701 is used to receive write requests sent by the client. The write requests are sent by the client via Remote Direct Memory Access (RDMA) multicast and are used to instruct the standby machine to copy logs from the master machine.

[0129] The write module 702 is used to generate logs according to write requests and write the log metadata into a ring array so that the standby machine can read the log metadata from the ring array through RDMA read operations and obtain the logs from the master machine based on the metadata. The client, master machine and standby machine are in the same multicast.

[0130] Optionally, the receiving module 701 is also configured to set the status of the log in the metadata to a committed status, which indicates that the host has generated the log and that the standby machine can read the log from the host.

[0131] Optionally, the write module 702 is further configured to receive a reply message from the standby machine, the reply message indicating that the standby machine has completed the replication of the log; set the log status in the metadata to an expired status, the expired status indicating that the standby machine has completed the replication of the log; and send a write response to the client, the write response indicating that the write request has been completed.

[0132] The log replication apparatus provided in this application embodiment can perform the above-described... Figure 4 The technical solution of the log replication method embodiment shown is similar in principle and effect, and will not be described again here.

[0133] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software through processing element calls; they can be implemented entirely in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware.

[0134] Figure 8 This is a schematic diagram of the structure of the log replication electronic device provided in an embodiment of this application. Figure 8 As shown, the log replication electronic device 80 provided in this embodiment may include:

[0135] Processor 801.

[0136] Memory 802 is used to store executable instructions for the terminal device.

[0137] The processor is configured to execute the technical solution of the above log replication method embodiment by executing executable instructions. Its implementation principle and technical effect are similar, and will not be described again here.

[0138] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the technical solution of the above-described log replication method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0139] In one possible implementation, a computer-readable medium may include random access memory (RAM), read-only memory (ROM), compact discread-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. The above combinations should also be included within the scope of computer-readable media.

[0140] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the technical solution of the above-described log replication method embodiment. Its implementation principle and technical effects are similar, and will not be repeated here.

[0141] In the specific implementation of the aforementioned terminal device or server, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0142] Those skilled in the art will understand that all or part of the steps in any of the above method embodiments can be implemented by hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, all or part of the steps in the above method embodiments are performed.

[0143] If the technical solution of this application is implemented in software form and sold or used as a product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of this application can be embodied in the form of a software product, which is stored in a storage medium and includes a computer program or several instructions. This computer software product causes a computer device (which may be a personal computer, server, network device, or similar electronic device) to execute all or part of the steps of the method described in the embodiments of this application.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A log replication method, characterized in that, Applications on the standby end include: Receive a write request sent by the client, the write request being sent by the client via Remote Direct Memory Access (RDMA) multicast, the write request being used to instruct the standby machine to copy logs from the master machine; Based on the write request, the host reads the metadata of the log corresponding to the write request from the ring array on the host side via an RDMA read operation. The log is generated by the host side based on the write request received from the client. The logs are replicated based on the log metadata, wherein the client, the host, and the backup machine are in the same multicast. The step of replicating the log based on its metadata includes: The status of the log is obtained based on the metadata; If the log is in a committed state, then the metadata is verified for integrity based on the checksum in the metadata; wherein, the committed state indicates that the host has generated the log; If the integrity verification passes, the logs are read from the host's storage according to the log pointer in the metadata, and the read logs are stored in the local storage of the standby machine.

2. The method according to claim 1, characterized in that, After storing the read logs in the local storage of the standby machine, the method further includes: The standby machine sends a reply message to the host machine via an RDMA write operation. The reply message indicates that the standby machine has completed the log replication, which enables the host machine to set the status of the log metadata in the ring array to expired.

3. The method according to claim 1, characterized in that, After receiving the write request sent by the client, the method further includes: The write request is cached in the request list; Before reading the metadata of the log corresponding to the write request from the ring array on the host side via an RDMA read operation based on the write request, the method further includes: The write request is retrieved from the request list according to the first-in, first-out (FIFO) principle.

4. A log replication method, characterized in that, Applied to the host side, including: Receive a write request sent by the client, the write request being sent by the client via Remote Direct Memory Access (RDMA) multicast, the write request being used to instruct the standby machine to copy logs from the master machine; Logs are generated according to the write request, and the metadata of the logs is written to a circular array, so that the standby machine can read the metadata of the logs from the circular array through RDMA read operations and obtain the status of the logs according to the metadata. If the status of the logs is committed, the integrity of the metadata is verified according to the checksum in the metadata. The committed status is used to indicate that the host has generated the logs. If the integrity verification passes, the logs are read from the storage of the host according to the log pointer in the metadata, and the read logs are stored in the local storage of the standby machine. The client, the host, and the standby machine are in the same multicast.

5. The method according to claim 4, characterized in that, The submitted status indicates that the standby machine reads the logs from the master machine.

6. The method according to claim 5, characterized in that, The method further includes: Receive a reply message from the standby machine, the reply message indicating that the standby machine has completed the log replication; Set the log status in the metadata to an expired state, which is used to instruct the standby machine to complete the replication of the log; A write response is sent to the client, indicating that the write request has been completed.

7. A log copying device, characterized in that, Applications on the standby end include: The receiving module is used to receive a write request sent by the client, the write request being sent by the client via Remote Direct Memory Access (RDMA) multicast, the write request being used to instruct the standby machine to copy logs from the master machine; The acquisition module is used to read the metadata of the log corresponding to the write request from the ring array on the host side through an RDMA read operation, wherein the log is generated by the host side based on the write request received from the client; The replication module is used to replicate the logs based on the log's metadata, wherein the client, the host, and the backup machine are in the same multicast. The replication module is specifically used to obtain the status of the log based on the metadata; If the log is in a committed state, then the metadata is verified for integrity based on the checksum in the metadata; wherein, the committed state indicates that the host has generated the log; If the integrity verification passes, the logs are read from the host's storage according to the log pointer in the metadata, and the read logs are stored in the local storage of the standby machine.

8. A log copying device, characterized in that, Applied to the host side, including: The receiving module is used to receive a write request sent by a client, wherein the write request is sent by the client via Remote Direct Memory Access (RDMA) multicast, and the write request is used to instruct the standby machine to copy logs from the master machine. The write module is used to generate logs according to the write request and write the metadata of the logs into a circular array, so that the standby machine can read the metadata of the logs from the circular array through RDMA read operations and obtain the status of the logs according to the metadata. If the status of the logs is committed, the integrity of the metadata is verified according to the checksum in the metadata. The committed status is used to indicate that the host has generated the logs. If the integrity verification passes, the logs are read from the storage of the host according to the log pointer in the metadata and stored in the local storage of the standby machine. The client, the host, and the standby machine are in the same multicast.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method of any one of claims 1-3 or 4-6.

10. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which is executed by a processor to implement the method of any one of claims 1-3 or 4-6.

11. A computer program product, characterized in that, The invention includes a computer program, characterized in that, when executed by a processor, the computer program implements the method of any one of claims 1-3 or 4-6.

Images