Database synchronization method, apparatus, and readable medium

Abstract

The application discloses a database synchronization method and device and a readable medium, and relates to the field of databases. The method comprises the following steps: receiving a write command sent by a programmable gateway, the programmable gateway being used for sending the write command to a master instance and a replica instance of a database; generating a digest value corresponding to the write command; writing the digest value into a binary log file corresponding to the write command; and synchronizing the binary log file to the replica instance, the replica instance being used for matching the digest value with the received write command and then executing the write command corresponding to the digest value. After the programmable gateway sends the write command to the master instance and the replica instance, the master instance sends the digest value corresponding to the write command to the replica instance. Since the digest length of the digest value is smaller than the command length of the write command, the problem that a large amount of data interaction and a large data transmission pressure are caused by the need to transfer the complete write command between the master instance and the replica instance is avoided, and the amount of data interaction is reduced through the digest value.

Description

Technical Field

[0001] This application relates to the field of databases, and in particular to a database synchronization method, apparatus, and readable medium. Background Technology

[0002] In the database field, synchronizing data from the primary server to the backup server is a major disaster recovery method. The databases on the backup and primary servers remain consistent, allowing operations to continue directly from the backup server if the primary server's data is corrupted or crashes. The primary and backup servers can be implemented as physical devices or as virtual devices in the cloud.

[0003] Typically, a client modifies data in the database on the primary machine through database transactions. After the primary machine receives several database transactions submitted by the client, it writes the database transactions to a binary log file and sends the binary log file to the standby machine. The standby machine parses the binary log file to obtain the corresponding database transactions and executes the database transactions on the standby machine to ensure that the database on the primary machine and the database on the standby machine are consistent.

[0004] However, when performing disaster recovery in the above manner, if the host machine is equipped with multiple standby machines, the host machine's central processing unit needs to perform a large amount of computation to transfer the binlog file to each standby machine. This puts a heavy burden on the host machine's data processing and affects the efficiency of disaster recovery. Summary of the Invention

[0005] This application provides a database synchronization method, apparatus, and readable medium, which can improve the efficiency of data interaction between primary and backup databases and reduce the data transmission pressure between them. The technical solution is as follows:

[0006] On the one hand, a database synchronization method is provided, applied to the database master instance, the method comprising:

[0007] The system receives a write command sent by a programmable gateway, which is used to send the write command to the database master instance and replica instance.

[0008] Generate a digest value corresponding to the write command, wherein the digest length of the digest value is less than the command length of the write command;

[0009] Write the digest value to the binary log file corresponding to the write command;

[0010] The binary log file is synchronized to the replica instance, which is used to match the digest value with the received write command and then execute the write command corresponding to the digest value.

[0011] On the other hand, a database synchronization method is provided for use in replica instances, the method comprising:

[0012] The system receives a write command sent by a programmable gateway, which is used to send the write command to the database master instance and the replica instance.

[0013] The write command is written to a replay file, which is used to control the execution of the write command.

[0014] The system receives a binary log file synchronized by the database master instance. The binary log file includes a digest value, which is a digest generated by the database master instance based on the received write command. The digest length of the digest value is less than the command length of the write command.

[0015] After matching the digest value with the write command in the replay file, the write command corresponding to the digest value is executed.

[0016] On the other hand, a database synchronization device is provided, the device comprising:

[0017] A receiving module is used to receive write commands sent by a programmable gateway, wherein the programmable gateway is used to send the write commands to the database master instance and the replica instance;

[0018] A generation module is used to generate a digest value corresponding to the write command, wherein the digest length of the digest value is less than the command length of the write command;

[0019] The writing module is used to write the digest value into the binary log file corresponding to the write command;

[0020] The synchronization module is used to synchronize the binary log file to the replica instance. The replica instance is used to match the digest value with the received write command and then execute the write command corresponding to the digest value.

[0021] On the other hand, a database synchronization device is provided, the device comprising:

[0022] A receiving module is used to receive write commands sent by a programmable gateway, wherein the programmable gateway is used to send the write commands to the database master instance and the replica instance;

[0023] The writing module is used to write the write command to the replay file, and the replay file is used to control the execution of the write command;

[0024] The receiving module is further configured to receive the binary log file synchronized by the database master instance. The binary log file includes a digest value, which is a digest generated by the database master instance based on the received write command. The digest length of the digest value is less than the command length of the write command.

[0025] The execution module is used to match the digest value with the write command in the replay file, and then execute the write command corresponding to the digest value.

[0026] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one program, the at least one program being loaded and executed by the processor to implement the database synchronization method as described in any of the embodiments of this application above.

[0027] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a database synchronization method as described in any of the embodiments of this application above.

[0028] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the database synchronization method described in any of the above embodiments.

[0029] The beneficial effects of the technical solutions provided in this application include at least the following:

[0030] After a write command is sent to the primary and replica instances via the programmable gateway, the primary instance sends a digest value corresponding to the write command to the replica instances to instruct them to execute the corresponding write command. This achieves synchronization between the primary and replica instances. Since the digest value is shorter than the write command length, the problem of large data interaction and high data transmission pressure caused by transmitting the complete write command between the primary and replica instances is avoided. By using the digest value, the amount of data interaction between the primary and replica instances is reduced, alleviating the data transmission pressure between the primary and replica instances, thereby indirectly improving the accuracy and efficiency of database synchronization. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram illustrating the implementation environment of a database synchronization method provided in an exemplary embodiment of this application;

[0033] Figure 2 This is a flowchart of a database synchronization method provided in an exemplary embodiment of this application;

[0034] Figure 3 Based on Figure 2 The illustrated embodiment provides a schematic diagram of the working principle of a programmable gateway;

[0035] Figure 4 Based on Figure 2 The illustrated embodiment provides a schematic diagram of the working principle of a programmable gateway;

[0036] Figure 5 This is a flowchart of a database synchronization method provided in another exemplary embodiment of this application;

[0037] Figure 6 Based on Figure 5 A schematic diagram illustrating the asynchronous replication method provided in the illustrated embodiment;

[0038] Figure 7 Based on Figure 5 A schematic diagram of the semi-synchronous method provided in the illustrated embodiment;

[0039] Figure 8 Based on Figure 5 The illustrated embodiment provides a schematic diagram of a multi-threaded approach;

[0040] Figure 9 Based on Figure 5 A schematic diagram illustrating the group copying method provided in the illustrated embodiment;

[0041] Figure 10 This is a flowchart of a database synchronization method provided in another exemplary embodiment of this application;

[0042] Figure 11 This is a flowchart of a database synchronization method provided in another exemplary embodiment of this application;

[0043] Figure 12 Based on Figure 11 The illustrated embodiment provides a schematic diagram of the database distribution structure;

[0044] Figure 13 Based on Figure 11 The illustrated embodiment provides a schematic diagram of the working principle of the programmable gateway;

[0045] Figure 14 Based on Figure 11 The illustrated embodiment provides a schematic diagram of the structure of a data write request;

[0046] Figure 15 Based on Figure 11 The illustrated embodiment provides a schematic diagram of the structure of a MySQL message;

[0047] Figure 16 This is a structural block diagram of a database synchronization device provided in an exemplary embodiment of this application;

[0048] Figure 17 This is a structural block diagram of a database synchronization device provided in another exemplary embodiment of this application;

[0049] Figure 18 This is a structural block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0051] First, the following terms used in the embodiments of this application will be explained:

[0052] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0053] Database: Simply put, it can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, with minimal redundancy, and independent of application programs.

[0054] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as Structured Query Language (SQL) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, simultaneously supporting multiple query languages.

[0055] In this embodiment of the application, a primary database (also known as a primary instance) and a backup database (also known as a replica instance) are involved.

[0056] Primary instance: In practical applications, this is the main database instance used, specifically the master node responsible for read and write tasks. Read and write tasks created by the client are first completed on the primary instance, while for write tasks, the primary instance also needs to synchronize the write tasks to the slave instances. The primary instance can be implemented on a physical host or a virtual host in the cloud; this embodiment does not limit this.

[0057] A replica instance is an instance that has a data synchronization relationship with the primary instance. Replica instances include at least one of the following instance types: slave instance, read-only instance, etc. A slave instance is an implementation that is attached to the primary instance and exists as a backup, maintaining a data synchronization relationship with the primary instance. When the primary instance fails, the slave instance automatically switches to provide read and write services to clients. A read-only instance is an instance that is attached to the primary instance and can provide read-only services to clients, but does not participate in failover when the primary instance fails.

[0058] Optionally, the replica instance can be implemented on a physical standby machine or on a virtual standby machine in the cloud; this application embodiment does not limit this.

[0059] A programmable gateway is a gateway device with computing capabilities at the application layer. In this embodiment, the programmable gateway acts as a relay device between the client and the database instance, forwarding write commands between them. In some embodiments, read commands sent by the client can be directly transmitted to the primary instance, which then provides the data read result. Write commands sent by the client are forwarded by the programmable gateway to the primary / backup instances, where the write operation is performed. Alternatively, the programmable gateway can forward both read and write commands sent by the client. In other words, the programmable gateway needs to determine the operation type of the commands sent by the client and, based on the operation type determination, determine the target address for command forwarding.

[0060] In related technologies, the forwarding rules of gateway devices are relatively fixed. Typically, when a gateway receives an instruction, it determines the target address from the instruction and forwards the instruction to the target address. However, in this embodiment, since the gateway device forwards commands between the client and the master-slave instances, and different forwarding scenarios exist for different database service types (e.g., forwarding only to the master instance, forwarding to both the master and replica instances simultaneously), a programmable gateway needs to be set up. Command forwarding rules are configured at the application layer of the programmable gateway to forward commands to the master and replica instances based on these rules.

[0061] Database transaction: A database transaction is a logical unit of work in database operation. Typically, a database transaction can perform one operation on the database, such as an insert operation, a modify operation, or a delete operation. Optionally, a database transaction is one or a group of Structured Query Language (SQL) statements.

[0062] Binary log file: refers to a file that stores database transactions in binary form.

[0063] First, the conceptual design of the embodiments of this application will be introduced in conjunction with the implementation environment involved in the embodiments of this application. Please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram illustrating the implementation environment of a database synchronization method provided in an exemplary embodiment of this application, such as... Figure 1 As shown, the operating environment for this database synchronization method includes: client 110, programmable gateway 120, host 130, and standby 140;

[0064] The client 110 is used to initiate data read / write requests. When initiating a read / write request, the client 110 sends the request to the programmable gateway 120. To illustrate, the client 110 displays a data display interface, where users can perform operations such as adding, deleting, modifying, and querying data. Taking the add operation as an example, the user selects the data add control in the data display interface and enters the data to be added to the database. Then, the client 110 sends a data write request to the programmable gateway 120.

[0065] Programmable gateway 120 is used to forward commands between client 110 and master / slave instances 130 and 140. Illustratively, when programmable gateway 120 receives a data write request from client 110, it determines the target address corresponding to the instance requiring the data write operation and forwards the write command to the target address. In some embodiments, for a write command, programmable gateway 120 forwards the write command to both the master and replica instances simultaneously.

[0066] Host 130 includes a primary instance, which is the master node responsible for read and write tasks. When host 130 receives a write command from programmable gateway 120, it executes the write command and writes it to the binlog file. During the writing process, a digest value is extracted from the write command and written to the binlog file. Because the digest length is less than the length of the write command itself, the amount of data transferred between the primary and replica instances is reduced.

[0067] The primary server 130 synchronizes binlog files with the standby server 140. The standby server 140 includes a replica instance, which can be a slave instance or a read-only instance. When the programmable gateway 120 sends a write command, it simultaneously sends the write command to both the primary and replica instances. Upon receiving a write command, the replica instance writes the write command to the relay log file. When the replica instance receives the binlog file synchronized by the primary instance, it retrieves the digest value corresponding to the write command from the binlog file and matches the digest value with the write commands already stored in the relay log file. If a write command matches the digest value in the relay log file, the matching write command is executed.

[0068] This ensures that the same write commands are executed in both the primary and replica instances, thus maintaining synchronization.

[0069] It is worth noting that the aforementioned client 110, programmable gateway 120, host 130 and standby 140 are connected to each other through a communication network. This communication network can be a wired network or a wireless network, including networks in the form of local area networks and wide area networks. This application embodiment does not limit this.

[0070] The embodiments of this application will be described in conjunction with the terms used in the above embodiments and the implementation environment of the embodiments of this application, such as... Figure 2 As shown, Figure 2 This is a flowchart illustrating a database synchronization method provided in an exemplary embodiment of this application, demonstrating the application of this database synchronization method in, for example,... Figure 1 Taking the primary instance on the host shown as an example, the database synchronization method includes:

[0071] Step 201: Receive the write command sent by the programmable gateway.

[0072] Programmable gateways are used to send write commands to the primary and replica instances of a database. Specifically, a programmable gateway is a gateway device that forwards commands between the client and the database.

[0073] In related technologies, the forwarding rules of gateway devices are relatively fixed. Typically, when a gateway device receives an instruction, it determines the target address from the instruction and forwards the instruction to the target address. However, in this embodiment, since the gateway device forwards commands between the client and the master-slave instances, and different forwarding scenarios exist for different database service types (e.g., forwarding only to the master instance, forwarding to both the master and replica instances simultaneously), a programmable gateway needs to be set up. Command forwarding rules are configured at the application layer of the programmable gateway to forward commands to the master and replica instances based on these rules.

[0074] In some embodiments, after receiving a data write request from a client, the programmable gateway matches the data write request with command forwarding rules, thereby determining the instance that needs to receive the write command based on the matching result, and forwarding the write command to the primary instance and replica instance that need to receive the write command.

[0075] The programmable gateway provided in the embodiments of this application will be described in an illustrative manner, please refer to the following: Figure 3After the command forwarding rule 300 is written, it is input into the compiler 310 in the programmable gateway. The compiler 310 compiles the command forwarding rule 300 to obtain the hardware logic control file 321 and the interface control file 322. The control plane 331 understands the interface control file 322 and performs corresponding rule control. The control plane 331 is implemented as the central processing unit (CPU) in the programmable gateway. The runtime module 332 is used to execute the rule control of the interface control file 322 based on the understanding of the interface control file 322 by the control plane 331. The execution process of the interface control file 322 reaches the driver layer 333, and finally, together with the hardware logic control file 321, controls the hardware 334 in the programmable gateway. For example, it controls the input and output interfaces in the programmable gateway to send commands.

[0076] In some embodiments, the command forwarding rule 300 is written in P4 language. For the rule control process of the interface control file 322, please refer to... Figure 4 .like Figure 4 As shown, taking a write operation as an example, the received data write request is parsed by the programmable parser 400. After passing through the pre-switching stage 410 and the post-switching stage 420, the rule matching result corresponding to the data write request is flowed back to the pre-switching stage 410 and the post-switching stage 420. The rule matching result is then processed by the control plane 430.

[0077] Both the pre-switching phase 410 and the post-switching phase 420 include matching logic and action logic. The matching logic includes logical matching processes such as hybrid lookup tables, counters, flow meters, and general hash tables for Static Random-Access Memory (SRAM) and Ternary Content Addressable Memory (TCAM). The action logic includes execution logic processes such as standard Boolean and arithmetic operations, header modification operations, and hash operations using Arithmetic Logic Units (ALUs). In other words, the pre-switching phase mainly implements the matching between data write requests and forwarding rules; the post-switching phase mainly implements the forwarding of write commands to various instances.

[0078] As an illustration, the command forwarding rule written in P4 language defines an entry procedure (corresponding to the pre-switching phase mentioned above) and an exit procedure (corresponding to the post-switching phase mentioned above). The entry procedure is illustrated in the following function:

[0079]

[0080] Here, `key` represents the matching field; `hdr.ipv4.dstAddr:lpm` indicates that the matching field is the IP destination address in the header of the data write request packet, and `lpm` specifies that the matching mode is Longest Prefix Match. The `lpm` matching mode can also be implemented as an exact match or a ternary match. `actions` represents the set of matching action types; `ipv4_forward` represents the forwarding action that needs to be defined, i.e., the outgoing action; `drop` represents the drop action, i.e., dropping the received data request; `NoAction` represents an empty action, i.e., no further processing; `size=1024` indicates the number of matching items that can be accommodated.

[0081] After the programmable gateway matches the forwarding rules corresponding to the data request sent by the client through the above process, it forwards the command based on the forwarding rules.

[0082] As an illustration, after the client sends a data write request to the programmable gateway, the programmable gateway matches the data write request through the above process and determines that the target instances corresponding to the data write request include: primary instance A, replica instance b, and replica instance c, and then sends write commands to primary instance A, replica instance b, and replica instance c.

[0083] In some embodiments, a write command not only refers to a command to add data to the database, but also includes commands to delete data or modify data in the database. This application does not limit the specific business type of the write command.

[0084] Step 202: Generate a digest value corresponding to the write command.

[0085] The digest length of the digest value is less than the command length of the write command. In some embodiments, when the master instance receives a write command, it first executes the write command and then writes the write command to the binlog file; or, the master instance first writes the write command to the binlog file and then executes the write command; or the master instance synchronously implements the processes of executing the write command and writing the write command to the binlog file.

[0086] In some embodiments, when generating a digest value corresponding to a write command, the main instance first determines the command length of the write command and extracts a digest of the write command based on the command length.

[0087] Indicatively, in response to a command length less than or equal to a length threshold, a digest is extracted from the write command to obtain a digest value; in response to a command length reaching (i.e., greater than or equal to) the length threshold, a command fragment is extracted from the write command, and a digest is extracted from the command fragment to obtain a digest value.

[0088] When extracting command fragments from a write command, at least one of the following situations applies:

[0089] First, extract a segment of a preset length from a preset position as the command segment;

[0090] Second, starting from the first preset position, a segment of the first preset length is extracted as the first command segment; starting from the second preset position, a segment of the second preset length is extracted as the second command segment; after combining the first command segment and the second command segment, a command segment corresponding to the write command is obtained.

[0091] Schematic illustration: A first command segment of a first preset length is extracted from the beginning of the write command, and a second command segment of a second preset length is extracted from the end of the write command. The first and second command segments are combined to obtain the command segment of the write command. The first and second preset lengths may be equal or unequal.

[0092] In some embodiments, the combination of the first command fragment and the second command fragment includes an addition method or a concatenation method, that is, adding the first command fragment and the second command fragment to obtain the command fragment of the write command; or concatenating the first command fragment and the second command fragment to obtain the command fragment of the write command.

[0093] To illustrate, taking a length threshold of 1KB as an example, in order to improve data transmission efficiency, if the length of the write command is less than (or equal to) 1KB, the digest of the write command is directly extracted to obtain the digest value; if the length of the write command is greater than (or equal to) 1KB, the first 256 bytes and the last 256 bytes of the write command are added together to obtain a command fragment, and the digest value is extracted from the command fragment.

[0094] In some embodiments, the process of extracting the digest value of a write command is the process of performing digest calculation on the write command using the MD5 Message-Digest Algorithm; similarly, the process of extracting the digest value of a command fragment is the process of performing digest calculation on the command fragment using the MD5 algorithm.

[0095] It is worth noting that the above-described method of extracting command fragments is illustrated by taking one fragment and two fragments as examples. In this embodiment, the command fragments can also be three, four or more, and this embodiment does not limit this.

[0096] The MD5 algorithm is a cryptographic hash function that produces a 128-bit (16-byte) hash value, used to determine whether a write command received and executed by the primary instance matches a write command received and to be executed by the replica instance. Because the digest length of the hash value generated by the MD5 algorithm is shorter than the command length, the amount of data exchange between the primary and replica instances can be reduced through the hash value.

[0097] Step 203: Write the digest value to the binary log file corresponding to the write command.

[0098] The primary instance writes the digest value to the binlog file, which is a binary file used for synchronization between the primary and replica instances.

[0099] Optionally, the binary log file includes command padding bits. After the digest value is converted into binary format, it is written into the command padding bits to obtain the binlog file.

[0100] In some embodiments, the binlog file includes not only the digest value corresponding to the write command, but also the transaction identifier, command length, and other contents. This application embodiment does not limit this.

[0101] Step 204: Synchronize the binary file to the replica instance.

[0102] In some embodiments, the primary instance synchronizes binlog files to the replica instance via a programmable gateway; or, the primary instance directly synchronizes binlog files to the replica instance via other routing devices.

[0103] The replica instance is used to match the digest value with the received write command and then execute the write command corresponding to the digest value.

[0104] Because the aforementioned programmable gateway forwards write commands to both the primary instance and the replica instance simultaneously, and the replica instance caches the write command upon receiving it instead of executing it directly, after the primary instance sends the binlog file to the replica instance, the binlog file includes the digest value corresponding to the write command executed by the primary instance. In other words, the binlog file is used to indicate that the primary instance has determined to execute the write command. Therefore, the replica instance needs to execute the write command synchronously with the primary instance. After obtaining the digest value from the binlog file, the replica instance matches the digest value with the currently pending write command, thereby determining the write command corresponding to the digest value from the pending write commands and executing it, ensuring synchronization between the primary instance and the replica instance.

[0105] In summary, the database synchronization method provided in this application sends a write command to the primary instance and replica instance through a programmable gateway. The primary instance then sends a digest value corresponding to the write command to the replica instance to instruct the replica instance to execute the corresponding write command, thereby achieving synchronization between the primary instance and replica instance. Since the digest length of the digest value is less than the command length of the write command, the problem of large data interaction and high data transmission pressure caused by needing to transmit the complete write command between the primary instance and replica instance is avoided. By reducing the amount of data interaction between the primary instance and replica instance through the digest value, the data transmission pressure between the primary and replica instances is alleviated, thereby indirectly improving the database synchronization accuracy and efficiency.

[0106] In some embodiments, the binlog file also includes a transaction identifier and command length. Figure 5 This is a flowchart of a database synchronization method provided in another exemplary embodiment of this application, taking the application of the method to a master instance in a host as an example, such as... Figure 5 As shown, the method includes:

[0107] Step 501: Receive the write command sent by the programmable gateway.

[0108] Programmable gateways are used to send write commands to the primary and replica instances of a database. Specifically, a programmable gateway is a gateway device that forwards commands between the client and the database.

[0109] In some embodiments, after receiving a data write request from a client, the programmable gateway matches the data write request with command forwarding rules, thereby determining the instance that needs to receive the write command based on the matching result, and forwarding the write command to the primary instance and replica instance that need to receive the write command.

[0110] Step 502: Determine the command length for the write command.

[0111] The command length indicates the length of the write command. For example, when a write command occupies 1KB (1024 bytes), the command length is 1024 bytes. This 1024 bytes is converted to binary and written to the binary log file (binlog).

[0112] It is worth noting that this embodiment uses the example of expressing the command length in bytes. In some embodiments, the command length can also be expressed in bits, kilobytes, etc. This application does not limit the way the command length is expressed.

[0113] Step 503: Generate a digest value corresponding to the write command.

[0114] The length of the digest value is less than the length of the write command.

[0115] In some embodiments, when generating a digest value corresponding to a write command, the main instance first determines the command length of the write command and extracts a digest of the write command based on the command length.

[0116] Indicatively, in response to a command length less than or equal to a length threshold, a digest is extracted from the write command to obtain a digest value; in response to a command length reaching (i.e., greater than or equal to) the length threshold, a command fragment is extracted from the write command, and a digest is extracted from the command fragment to obtain a digest value.

[0117] Step 504: Determine the transaction identifier corresponding to the write command.

[0118] Transaction identifiers are determined based on the order in which database transactions corresponding to write commands are generated. A database transaction is a logical unit of work in database operation. Typically, a database transaction can perform one operation on the database, such as an insert, update, or delete operation. Optionally, a database transaction can be a single SQL statement or a group of SQL statements.

[0119] Starting with MySQL 5.6, a Global Transaction Identifier (GTID) was introduced for master-slave synchronization, which is the transaction identifier involved in this application embodiment. Due to the introduction of GTID, the uniqueness of database transactions is guaranteed regardless of which node generates the transaction. GTID consists of two parts: source_id and sequence_id. source_id represents the identifier of the environment where the database node resides, such as a physical machine or virtual machine, while sequence_id represents the identifier of the database transaction, generated sequentially by the system. Since only the master node generates this sequence_id, both the uniqueness of the GTID and unidirectional growth are guaranteed.

[0120] After a GTID is generated on the primary instance, it is recorded in the binlog file. The binlog file is synchronized to the server corresponding to the replica instance (i.e., the standby server) and stored in the relay log file. The replica instance reads the database transaction corresponding to the GTID, determines whether it has been executed, and if not, executes the database transaction and writes the data to the database engine.

[0121] In related technologies, based on GTID, synchronization between the primary and replica instances is typically achieved through asynchronous replication, semi-synchronous replication, multi-threading, or group replication. These four methods will be briefly introduced below:

[0122] 1. Asynchronous replication method

[0123] This is illustrative; please refer to it. Figure 6 Asynchronous replication refers to writing database transactions on the primary instance 610 without waiting for data in the binlog file to be synchronized to the secondary instances 620 and 630, and directly sending a response packet to the client. The client is unaware whether the database transactions have been synchronized to the secondary instances when it receives the response packet. If the primary instance fails at this time, and the data has not been synchronized to the secondary instances, the database transactions may be lost.

[0124] 2. Semi-synchronous mode

[0125] This is illustrative; please refer to it. Figure 7 The semi-synchronous method means that after the binlog file on the master instance 710 is successfully written, the binlog file needs to be synchronized to the slave instances 720 and 730. Only after receiving the response packet from the slave instances will it be returned to the client, ensuring that the data written by the database transaction is stored on both machines.

[0126] 3. Multi-threaded approach

[0127] This is illustrative; please refer to it. Figure 8 Multithreading refers to the parallel transmission of non-conflicting database transactions, where they do not need to wait for each other. For example... Figure 8 As shown, transactions 811 and 812 are transmitted using thread 801; transactions 821 and 822 are transmitted using thread 802; and transactions 831 and 832 are transmitted using thread 803.

[0128] 4. Group copying method

[0129] This is illustrative; please refer to it. Figure 9 Group replication means that the master instance 910 simultaneously sends the SQL statement to be executed to multiple slave instances, rather than sending binlog files. For example... Figure 9 As shown, the master instance 910 sends the SQL statement to both the slave instance 920 and the slave instance 930 simultaneously.

[0130] However, while methods 1 and 2 above guarantee the consistency and reliability of data synchronization with GTID, the data synchronization speed is relatively slow, which may lead to increased master-slave latency.

[0131] Methods 3 and 4 described above improve the speed of data transfer between master and slave servers while maintaining the same data volume. However, when the master or slave server's CPU is busy, it requires further CPU consumption, leading to increased CPU utilization and consequently, slower response times when executing complex SQL statements.

[0132] It should be noted that the above steps 503 and 504 are two parallel steps. One can execute step 503 first, or step 504 first, or both steps 503 and 504 can be executed simultaneously.

[0133] Step 505, write the digest value, command length, and transaction identifier into the binary log file.

[0134] The master instance writes the digest value, command length, and transaction identifier into the value binlog file. The binlog file is a binary file for synchronization between the master instance and the replica instance.

[0135] Optionally, the binary log file includes command padding bits. After converting the digest value into binary format, write it into these command padding bits to obtain the binlog file.

[0136] In some embodiments, when the master instance executes the write operation corresponding to the write command, the write operation also corresponds to an auto-increment identifier, which is used to represent the operation sequence of the write operation. In some embodiments, the write operation itself includes at least two operation steps. For example, if the write operation is implemented as a data addition operation, and the data to be added includes 3 sub-data in a complete piece of data, namely the number data 003, the name data "Little A", and the attribute data "female", then for the 3 sub-data, based on the increasing order of number data, name data, and attribute data, the process of adding the number data 003 corresponds to the auto-increment identifier 1, the process of adding the name data "Little A" corresponds to the auto-increment identifier 2, and the process of adding the attribute data "female" corresponds to the auto-increment identifier 3. The above auto-increment identifier is illustrated by incrementing by one by one. The incrementing method of the auto-increment identifier can also be other custom methods, such as incrementing by two by one, incrementing by any value, etc. The embodiments of the present application do not limit this.

[0137] Write the correspondence between the write operation and the auto-increment identifier into the binary log file. In some embodiments, write the correspondence between the sub-operations of the write operation and the auto-increment identifier into the binlog file.

[0138] Step 506, synchronize the binary file to the replica instance.

[0139] The replica instance is used to match the digest value with the received write command and then execute the write command corresponding to the digest value.

[0140] In some embodiments, after the digest value and the write command match successfully, it is also necessary to compare the command length of the write command with the command length in the binlog file. When the comparison result indicates that the command lengths are the same, then execute the write command corresponding to the digest value.

[0141] In some embodiments, when a replica instance receives a write command from a programmable gateway, it writes the write command to the replay relaylog file. When it receives a binlog file from the primary instance, it matches the digest value in the binlog file with the write command in the relaylog file and executes the write command that matches the digest value.

[0142] The binlog file also includes a transaction flag. When a replica instance writes a write command to the relaylog file, the flag is set to control the execution of the write command. When the digest value in the binlog file matches a write command in the relaylog file, the transaction flag corresponding to the digest value is written to the flag corresponding to the write command in the relaylog file, thus executing the write command based on the flag.

[0143] In summary, the database synchronization method provided in this application sends a write command to the primary instance and replica instance through a programmable gateway. The primary instance then sends a digest value corresponding to the write command to the replica instance to instruct the replica instance to execute the corresponding write command, thereby achieving synchronization between the primary instance and replica instance. Since the digest length of the digest value is less than the command length of the write command, the problem of large data interaction and high data transmission pressure caused by needing to transmit the complete write command between the primary instance and replica instance is avoided. By reducing the amount of data interaction between the primary instance and replica instance through the digest value, the data transmission pressure between the primary and replica instances is alleviated, thereby indirectly improving the database synchronization accuracy and efficiency.

[0144] The method provided in this embodiment fills the GTID into the binlog file, so that after the digest value and the write command are successfully matched, the execution process of the write command in the replica instance is controlled by the GTID, avoiding the problem of low synchronization accuracy caused by the disorder of the execution order between multiple write commands.

[0145] The method provided in this embodiment fills the binlog file with an auto-incrementing identifier. After the digest value matches the write command successfully, the replica instance executes the write operations in the write command sequentially based on the auto-incrementing identifier during the execution of the write command. This avoids the problem of low synchronization accuracy caused by the execution order of multiple write operations in the write command being inconsistent with that of the primary instance.

[0146] Based on the above explanation, the method for synchronizing the database on the replica instance side will be described. Figure 10 This is a flowchart of a database synchronization method provided in another exemplary embodiment of this application, taking the application of this method to a replica instance in a standby machine as an example. Figure 10 As shown, the method includes:

[0147] Step 1001: Receive the write command sent by the programmable gateway.

[0148] Programmable gateways are used to send write commands to the primary and replica instances of a database. Specifically, a programmable gateway is a gateway device that forwards commands between the client and the database.

[0149] In some embodiments, after receiving a data write request from a client, the programmable gateway matches the data write request with command forwarding rules, thereby determining the instance that needs to receive the write command based on the matching result, and forwarding the write command to the primary instance and replica instance that need to receive the write command.

[0150] Step 1002: Write the write command to the replay file.

[0151] The replay file is used to control the execution of hostile write commands. That is, when a replica instance receives a write command, it does not execute the write command immediately, but instead saves the write command to the replay file and waits for the primary instance's binlog file to indicate when to execute the write command.

[0152] In some embodiments, a write command is written from the command padding bits of the replay file. The flag bits in the replay file are set to null values. These flag bits control the execution of the write command; when the flag bit is null, the write command waits in a loop and is not executed; when the flag bit has a GTID value, the write command is executed.

[0153] Step 1003: Receive the binary log file synchronized from the database master instance. The binary log file includes a summary value.

[0154] The digest value is a digest generated by the database master instance based on the received write command, and the digest length is less than the command length of the write command.

[0155] The replica instance obtains the digest value from the binlog file.

[0156] Step 1004: After matching the digest value with the write command in the replay file, execute the write command corresponding to the digest value.

[0157] In some embodiments, the replica instance extracts a reference digest value from the write command in the replay file, matches the digest value extracted from the binlog file with the reference digest value, and executes the write command corresponding to the target digest value if a target digest value matches the digest value extracted from the binlog file. The method for extracting the reference digest value is the same as the method for extracting the digest value from the binlog file.

[0158] Optionally, the binlog file also includes the transaction identifier GTID corresponding to the write command. In response to the presence of a target digest value in the reference digest value that matches the digest value extracted from the binlog file, the transaction identifier GTID is filled into the flag bit; in response to looping through the replay file to the write command and finding that the transaction identifier GTID exists in the flag bit corresponding to the write command, the write command is executed.

[0159] It is worth noting that when the binlog file of the primary instance is synchronized to the replica instance, if a record with an empty GTID and a matching MD5 value cannot be found in the relaylog file, the subsequent logic will be executed in a master-slave semi-synchronous or asynchronous configuration.

[0160] In summary, the database synchronization method provided in this application sends a write command to the primary instance and replica instance through a programmable gateway. The primary instance then sends a digest value corresponding to the write command to the replica instance to instruct the replica instance to execute the corresponding write command, thereby achieving synchronization between the primary instance and replica instance. Since the digest length of the digest value is less than the command length of the write command, the problem of large data interaction and high data transmission pressure caused by needing to transmit the complete write command between the primary instance and replica instance is avoided. By reducing the amount of data interaction between the primary instance and replica instance through the digest value, the data transmission pressure between the primary and replica instances is alleviated, thereby indirectly improving the database synchronization accuracy and efficiency.

[0161] Based on the above, the database synchronization method provided in this application embodiment will be described from a system perspective. For illustrative purposes, please refer to [reference needed]. Figure 11 It illustrates a flowchart of a database synchronization method provided in an exemplary embodiment of this application, such as... Figure 11 As shown, the method includes:

[0162] Step 1101: The programmable gateway receives a data write request sent by the client.

[0163] When a client has a data write requirement, it sends a data write request to the programmable gateway to request that data content be written to the database.

[0164] Step 1102: The programmable gateway determines the primary instance and replica instance that will receive the write command based on the data write request.

[0165] Indicative, such as Figure 12 As shown, the databases are distributed across different racks, including different types of databases such as primary instance 1210, slave instance 1220, and read-only instance 1230. At the same time, there may be other business types that are not databases under the same rack, such as virtual machines, object storage, and other services.

[0166] At the P4 layer of the programmable gateway, it is necessary to identify the packets corresponding to the database and perform data replication and forwarding. Please refer to the example provided. Figure 13 Control information exchange occurs between the control layer 1310 and the data layer 1320. During the processing in the data layer 1320, data write requests are matched; that is, the data write requests are analyzed through a matching phase and an action phase, and finally, the write command corresponding to the data write request is sent to the matched primary and replica instances. Specifically, in the protocol matching layer, the TCP protocol is first determined, and the fields specified by the database are identified in the TCP packet body to determine the type of operation accessed by the data write request. The write command is then forwarded. Figure 14 As shown, the data write request includes a type field 1410, which indicates the data operation type corresponding to the current data write request, such as read, write, delete, etc.

[0167] When the database machine is installed, the programmable gateway determines the list of Internet Protocol (IP) addresses and the database communication port for that machine. The P4 program contains the configuration information for this IP address set and the database communication port, as well as matching and forwarding rules. When a client request matches the IP address and port of a specific instance, it distinguishes the command type of the data write request and coordinates with the database kernel to implement the appropriate action. For illustration, MySQL messages are divided into a header and a body, such as... Figure 15 As shown, MySQL message 1500 includes message header 1510 and message body 1520. Message header 1510 consists of message length 1511 and sequence number 1512, and message body 1520 includes message data.

[0168] The message length 1511, in bytes, marks the actual data length of the current write request message. The sequence number 1512 ensures correct message order during a complete request / response interaction; it is incremented from 0 each time a client initiates a request. The message body 1520 stores the content of the write request and the response data; its length is determined by the message length 1511 in the message header 1510.

[0169] The programmable gateway first needs to identify whether the received request is a login authentication command, because all command execution requires authorization first. If it is identified as a login command, it will abandon forwarding it to the replica instance and hand it over to the host where the primary instance resides for processing.

[0170] In some embodiments, during the database login phase, after the primary instance successfully logs in, the primary instance needs to broadcast the authentication session success information to all replica instances to ensure that subsequent replica instances can successfully authenticate when they receive read requests.

[0171] Step 1103: The programmable gateway sends a write command to the primary instance.

[0172] Programmable gateways are used to send write commands to the primary and replica instances of a database. Specifically, a programmable gateway is a gateway device that forwards commands between the client and the database.

[0173] Step 1104: The programmable gateway sends a write command to the replica instance.

[0174] Step 1105: The replica instance writes the write command to the replay file.

[0175] The replay file is used to control the execution of hostile write commands. That is, when a replica instance receives a write command, it does not execute the write command immediately, but instead saves the write command to the replay file and waits for the primary instance's binlog file to indicate when to execute the write command.

[0176] In some embodiments, a write command is written from the command padding bits of the replay file. The flag bits in the replay file are set to null values. These flag bits control the execution of the write command; when the flag bit is null, the write command waits in a loop and is not executed; when the flag bit has a GTID value, the write command is executed.

[0177] Step 1106: The main instance generates a digest value corresponding to the write command.

[0178] The length of the digest value is less than the length of the write command.

[0179] In some embodiments, when generating a digest value corresponding to a write command, the main instance first determines the command length of the write command and extracts a digest of the write command based on the command length.

[0180] Indicatively, in response to a command length less than or equal to a length threshold, a digest is extracted from the write command to obtain a digest value; in response to a command length reaching (i.e., greater than or equal to) the length threshold, a command fragment is extracted from the write command, and a digest is extracted from the command fragment to obtain a digest value.

[0181] Step 1107: The primary instance writes the digest value to the binary log file corresponding to the write command.

[0182] The primary instance writes the digest value to the binlog file, which is a binary file used for synchronization between the primary and replica instances.

[0183] In some embodiments, the binlog file includes not only the digest value corresponding to the write command, but also the transaction identifier, command length, and other contents. This application embodiment does not limit this.

[0184] Step 1108: The primary instance synchronizes the binary log file to the replica instance.

[0185] Step 1109: After matching the digest value with the write command in the replay file, the replica instance executes the write command corresponding to the digest value.

[0186] In some embodiments, the replica instance extracts a reference digest value from the write command in the replay file, matches the digest value extracted from the binlog file with the reference digest value, and executes the write command corresponding to the target digest value if a target digest value matches the digest value extracted from the binlog file. The method for extracting the reference digest value is the same as the method for extracting the digest value from the binlog file.

[0187] Optionally, the binlog file also includes the transaction identifier GTID corresponding to the write command. In response to the presence of a target digest value in the reference digest value that matches the digest value extracted from the binlog file, the transaction identifier GTID is filled into the flag bit; in response to looping through the replay file to the write command and finding that the transaction identifier GTID exists in the flag bit corresponding to the write command, the write command is executed.

[0188] In summary, the database synchronization method provided in this application sends a write command to the primary instance and replica instance through a programmable gateway. The primary instance then sends a digest value corresponding to the write command to the replica instance to instruct the replica instance to execute the corresponding write command, thereby achieving synchronization between the primary instance and replica instance. Since the digest length of the digest value is less than the command length of the write command, the problem of large data interaction and high data transmission pressure caused by needing to transmit the complete write command between the primary instance and replica instance is avoided. By reducing the amount of data interaction between the primary instance and replica instance through the digest value, the data transmission pressure between the primary and replica instances is alleviated, thereby indirectly improving the database synchronization accuracy and efficiency.

[0189] Figure 16 This is a structural block diagram of a database synchronization device provided in an exemplary embodiment of this application, such as... Figure 16 As shown, the device includes:

[0190] The receiving module 1610 is used to receive a write command sent by the programmable gateway, which is used to send the write command to the database master instance and the replica instance.

[0191] The generation module 1620 is used to generate a digest value corresponding to the write command, wherein the digest length of the digest value is less than the command length of the write command;

[0192] The writing module 1630 is used to write the digest value into the binary log file corresponding to the write command;

[0193] The synchronization module 1640 is used to synchronize the binary log file to the replica instance, and the replica instance is used to match the digest value with the received write command and then execute the write command corresponding to the digest value.

[0194] In an optional embodiment, the generation module 1620 is further configured to determine the command length of the write command;

[0195] The generation module 1620 is further configured to extract a digest from the write command in response to the command length being less than a length threshold, thereby obtaining the digest value;

[0196] The generation module 1620 is further configured to, in response to the command length reaching the length threshold, extract a command fragment from the write command; and perform a digest extraction on the command fragment to obtain the digest value.

[0197] In an optional embodiment, the generation module 1620 is further configured to: extract a first command segment of a first preset length from the beginning of the write command; extract a second command segment of a second preset length from the end of the write command; and combine the first command segment and the second command segment to obtain the command segment of the write command.

[0198] In an optional embodiment, the generation module 1620 is further configured to add the first command fragment and the second command fragment to obtain the command fragment of the write command;

[0199] or,

[0200] The generation module 1620 is further configured to concatenate the first command fragment and the second command fragment to obtain the command fragment of the write command.

[0201] In an optional embodiment, the writing module 1630 is further configured to determine the command length of the write command; determine the transaction identifier corresponding to the write command, the transaction identifier being determined based on the generation order of the database transaction corresponding to the write command; and write the digest value, the command length, and the transaction identifier into the binary log file.

[0202] In an optional embodiment, the writing module 1630 is further configured to execute the write operation corresponding to the write command; in response to the existence of an incrementing identifier during the execution of the write operation, the correspondence between the write operation and the incrementing identifier is written to the binary log file, wherein the incrementing identifier is used to indicate the operation order of the write operation.

[0203] Figure 17 This is a structural block diagram of a database synchronization device provided in an exemplary embodiment of this application, such as... Figure 17 As shown, the device includes:

[0204] The receiving module 1710 is used to receive a write command sent by the programmable gateway, which is used to send the write command to the database master instance and the replica instance.

[0205] The writing module 1720 is used to write the write command to a replay file, and the replay file is used to control the execution of the write command;

[0206] The receiving module 1710 is further configured to receive a binary log file synchronized by the database master instance. The binary log file includes a digest value, which is a digest generated by the database master instance based on the received write command. The digest length of the digest value is less than the command length of the write command.

[0207] The execution module 1730 is used to match the digest value with the write command in the replay file and then execute the write command corresponding to the digest value.

[0208] In an optional embodiment, the execution module 1730 is further configured to extract a reference digest value of a write command in the replay file; match the digest value with the reference digest value; and execute a write command corresponding to the target digest value in response to the presence of a target digest value that matches the digest value in the reference digest value.

[0209] In an optional embodiment, the method for extracting the reference digest value is consistent with the method for extracting the digest value.

[0210] In an optional embodiment, the binary log file further includes a transaction identifier corresponding to the write command;

[0211] The writing module 1720 is further configured to write the write command into the command padding bit of the replay file; set the identifier bit of the replay file to an empty value, the identifier bit being used to control the execution status of the write command; wherein, when the identifier bit is an empty value, the write command waits in a loop, and when the identifier bit has a value, the write command is executed.

[0212] In an optional embodiment, the writing module 1720 is further configured to fill the identifier into the identifier bit in response to the presence of a target digest value that matches the digest value in the reference digest value;

[0213] The write module 1720 is further configured to execute the write command in response to the loop to the write command in the replay file, and the transaction identifier is present in the identifier bit corresponding to the write command.

[0214] In summary, the database synchronization device provided in this application sends a write command to the primary instance and replica instance through a programmable gateway. The primary instance then sends a digest value corresponding to the write command to the replica instance to instruct the replica instance to execute the corresponding write command, thereby achieving synchronization between the primary instance and replica instance. Since the digest length of the digest value is less than the command length of the write command, the problem of large data interaction and high data transmission pressure caused by needing to transmit the complete write command between the primary instance and replica instance is avoided. By reducing the amount of data interaction between the primary instance and replica instance through the digest value, the data transmission pressure between the primary and replica instances is alleviated, thereby indirectly improving the database synchronization accuracy and efficiency.

[0215] It should be noted that the database synchronization device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the database synchronization device and the database synchronization method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0216] Figure 10 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown. This computer device can be implemented as a server as described above. Specifically:

[0217] Computer device 1800 includes a central processing unit (CPU) 1801, a system memory 1804 including random access memory (RAM) 1802 and read-only memory (ROM) 1803, and a system bus 1805 connecting the system memory 1804 and the central processing unit 1801. Computer device 1800 also includes a mass storage device 1806 for storing an operating system 1813, application programs 1814, and other program modules 1815.

[0218] Mass storage device 1806 is connected to central processing unit 1801 via a mass storage controller (not shown) connected to system bus 1805. Mass storage device 1806 and its associated computer-readable media provide non-volatile storage for computer device 1800. That is, mass storage device 1806 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drive.

[0219] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1804 and mass storage device 1806 described above can be collectively referred to as memory.

[0220] According to various embodiments of this application, the computer device 1800 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1800 can be connected to a network 1812 via a network interface unit 1811 connected to the system bus 1805, or the network interface unit 1811 can be used to connect to other types of networks or remote computer systems (not shown).

[0221] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU.

[0222] This application also provides a computer device, which includes a memory and a processor. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads the at least one instruction, at least one program, code set, or instruction set to implement the database synchronization method described in the above embodiments.

[0223] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the database synchronization method as described in the above embodiments.

[0224] Embodiments of this application also provide a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the database synchronization methods described in the above embodiments.

[0225] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0226] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0227] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A database synchronization method, characterized in that, When applied to the primary database instance, the method includes: The system receives a write command sent by a programmable gateway, which is used to send the write command to the database master instance and replica instance. Generate a digest value corresponding to the write command, wherein the digest value is shorter than the command length of the write command; wherein the digest value is obtained by extracting the digest of the write command after determining the command length, in response to the command length being less than a length threshold; or in response to the command length reaching the length threshold, truncating a first command segment of a first preset length from the beginning of the write command and truncating a second command segment of a second preset length from the end of the write command, and combining the first command segment and the second command segment; The digest value is written to the binary log file corresponding to the write command. After determining the transaction identifier corresponding to the write command, the digest value, the command length, and the transaction identifier are written to the binary log file. The transaction identifier is determined based on the generation order of the database transaction corresponding to the write command. The binary log file is synchronized to the replica instance, which is used to match the digest value with the received write command and then execute the write command corresponding to the digest value.

2. The method according to claim 1, characterized in that, The step of combining the first command fragment and the second command fragment to obtain the write command includes: The first command fragment and the second command fragment are added together to obtain the command fragment of the write command; or, The first command fragment and the second command fragment are concatenated to obtain the command fragment of the write command.

3. The method according to claim 1, characterized in that, After receiving the write command sent by the programmable gateway, the method further includes: Execute the write operation corresponding to the write command; In response to the presence of an incrementing identifier during the execution of the write operation, the correspondence between the write operation and the incrementing identifier is written to the binary log file, wherein the incrementing identifier is used to indicate the operation order of the write operation.

4. A database synchronization method, characterized in that, When applied to a replica instance, the method includes: The system receives a write command sent by a programmable gateway, which is used to send the write command to the database master instance and the replica instance. The write command is written to a replay file, which is used to control the execution of the write command. The system receives a binary log file synchronized by the database master instance. The binary log file includes a digest value, which is a digest generated by the database master instance based on a received write command. The digest value's length is less than the command length of the write command. The digest value is obtained by the database master instance extracting the write command digest after determining the command length, in response to the command length being less than a length threshold; or in response to the command length reaching the length threshold, by truncating a first command segment of a first preset length from the beginning of the write command and a second command segment of a second preset length from the end of the write command, and combining the first and second command segments. Furthermore, the database master instance, after determining the transaction identifier corresponding to the write command, writes the digest value, the command length, and the transaction identifier into the binary log file. The transaction identifier is determined based on the generation order of the database transactions corresponding to the write command. After matching the digest value with the write command in the replay file, the write command corresponding to the digest value is executed.

5. The method according to claim 4, characterized in that, The step of matching the digest value with the write command in the replay file and then executing the write command corresponding to the digest value includes: Extract the reference digest value of the write command from the replay file; Match the summary value with the reference summary value; In response to the presence of a target digest value that matches the digest value among the reference digest values, a write command corresponding to the target digest value is executed.

6. The method according to claim 5, characterized in that, The method for extracting the reference summary value is the same as the method for extracting the summary value.

7. The method according to claim 5, characterized in that, The binary log file also includes a transaction identifier corresponding to the write command; The step of writing the write command to the replay file includes: The write command is written to the command padding bits of the replay file; The flag bit of the replay file is set to an empty value. The flag bit is used to control the execution of the write command. When the flag bit is empty, the write command waits in a loop. When the flag bit has a value, the write command is executed.

8. The method according to claim 7, characterized in that, The response that a target digest value exists in the reference digest values ​​that matches the digest value, executing a write command corresponding to the target digest value, includes: In response to the presence of a target digest value that matches the digest value in the reference digest values, the transaction identifier is filled into the identifier bit; In response to the loop to the write command in the replay file, and the presence of the transaction identifier in the identifier bit corresponding to the write command, the write command is executed.

9. A database synchronization device, characterized in that, The device includes: A receiving module is used to receive write commands sent by a programmable gateway, wherein the programmable gateway is used to send the write commands to the database master instance and the replica instance; A generation module is used to generate a digest value corresponding to the write command, wherein the digest value has a digest length that is less than the command length of the write command; wherein the digest value is obtained by extracting a digest of the write command after determining the command length, in response to the command length being less than a length threshold; or in response to the command length reaching the length threshold, a first command segment of a first preset length is truncated from the beginning of the write command, and a second command segment of a second preset length is truncated from the end of the write command, and the first command segment and the second command segment are combined. The writing module is used to write the digest value into the binary log file corresponding to the write command. After determining the transaction identifier corresponding to the write command, the digest value, the command length, and the transaction identifier are written into the binary log file. The transaction identifier is determined based on the generation order of the database transaction corresponding to the write command. The synchronization module is used to synchronize the binary log file to the replica instance. The replica instance is used to match the digest value with the received write command and then execute the write command corresponding to the digest value.

10. A database synchronization device, characterized in that, The device includes: A receiving module is used to receive write commands sent by a programmable gateway, which is used to send the write commands to the database master instance and replica instances. The writing module is used to write the write command to a replay file, and the replay file is used to control the execution of the write command; The receiving module is further configured to receive a binary log file synchronized by the database master instance. The binary log file includes a digest value, which is a digest generated by the database master instance based on the received write command. The digest length of the digest value is less than the command length of the write command. The digest value is obtained by the database master instance extracting the write command digest after determining the command length, in response to the command length being less than a length threshold; or in response to the command length reaching the length threshold, extracting a first command segment of a first preset length from the beginning of the write command and a second command segment of a second preset length from the end of the write command, and combining the first command segment and the second command segment. The database master instance is configured to write the digest value, the command length, and the transaction identifier into the binary log file after determining the transaction identifier corresponding to the write command. The transaction identifier is determined based on the generation order of the database transaction corresponding to the write command. The execution module is used to match the digest value with the write command in the replay file, and then execute the write command corresponding to the digest value.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the database synchronization method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The storage medium stores at least one program segment, which is loaded and executed by a processor to implement the database synchronization method as described in any one of claims 1 to 8.

Images