Data synchronization method and system, electronic equipment and storage medium

Abstract

The invention discloses a data synchronization method and system, electronic equipment and a storage medium. The method comprises the steps that a first method identifier and a change parameter corresponding to a first change operation are obtained, and the first change operation comprises an operation executed by a first server in response to a data change request for a first database; the first method identifier is an operation method identifier corresponding to the first change operation, and the first method identifier is used for acquiring an operation method identifier of a second change operation for a second database; creating a change event based on the first method identifier and the change parameter; the change event is sent to a second server side, so that the second server side executes second change operation based on the change event, and the second change operation is used for synchronizing data between the first database and the second database. According to the embodiment of the invention, the technical problem of poor universality and flexibility of multi-database environment data synchronization in related technologies can be solved.

Description

Data synchronization methods, systems, electronic devices, and storage media Technical Field

[0001] This application relates to the field of data synchronization technology, specifically to a data synchronization method, system, electronic device, and storage medium. Background Technology

[0002] With the rapid development of information technology and network communication technology, the demand for distributed systems, cloud computing environments, and big data applications is increasing. Enterprise application system architectures often require the deployment of database nodes across multiple geographical locations to support service requirements such as high availability, load balancing, and disaster recovery. In multi-database environments, ensuring data consistency has become a critical issue.

[0003] To ensure data consistency across multiple databases, related technologies typically use replication functionality between databases of the same type for data synchronization, or employ third-party data synchronization tools to synchronize data between different types of databases. However, these solutions are only compatible with certain database types, limiting their applicability and resulting in poor versatility, performance, and flexibility in data synchronization. Summary of the Invention

[0004] In view of the above problems, this application provides a data synchronization method, system, electronic device and storage medium to at least solve the technical problem of poor universality and flexibility of data synchronization in multi-database environments in related technologies.

[0005] According to a first aspect of the embodiments of this application, a data synchronization method is provided, applied to a first server, comprising: obtaining a first method identifier and change parameters corresponding to a first change operation, wherein the first change operation includes an operation performed by the first server in response to a data change request for a first database; the first method identifier is an operation method identifier corresponding to the first change operation, and the first method identifier is used to obtain an operation method identifier for a second change operation for a second database; creating a change event based on the first method identifier and the change parameters; and sending the change event to a second server, so that the second server performs the second change operation based on the change event, wherein the second change operation is used to synchronize data between the first database and the second database.

[0006] According to a second aspect of the embodiments of this application, a data synchronization method is provided, applied to a second server. The data synchronization method includes: acquiring a change event sent by a first server, wherein the change event is created by the first server based on a first method identifier and change parameters corresponding to a first change operation, the first change operation including an operation performed by the first server in response to a data change request for a first database; the first method identifier is an operation method identifier corresponding to the first change operation; acquiring the first method identifier from the change event, determining a second method identifier corresponding to the first method identifier; the second method identifier is an operation method identifier for a second change operation for the second database; and executing the second change operation based on the change parameters in the change event to synchronize data between the first database and the second database.

[0007] According to a third aspect of the embodiments of this application, a data synchronization apparatus is provided, comprising: an acquisition unit, configured to acquire a first method identifier and change parameters corresponding to a first change operation, wherein the first change operation includes an operation performed by a first server in response to a data change request for a first database, the first method identifier being an operation method identifier corresponding to the first change operation, and the first method identifier being used to acquire an operation method identifier for a second change operation for a second database; a creation unit, configured to create a change event based on the first method identifier and the change parameters; and a sending unit, configured to send the change event to a second server, so that the second server performs the second change operation based on the change event, wherein the second change operation is used to synchronize data between the first database and the second database.

[0008] According to a fourth aspect of the embodiments of this application, a data synchronization apparatus is provided, comprising: an acquisition unit, configured to acquire a change event sent by a first server, wherein the change event is created by the first server based on a first method identifier and change parameters corresponding to a first change operation, the first change operation including an operation performed by the first server in response to a data change request for a first database, and the first method identifier being an operation method identifier corresponding to the first change operation; a determination unit, configured to acquire the first method identifier from the change event and determine a second method identifier corresponding to the first method identifier; the second method identifier being an operation method identifier for a second change operation for a second database; and an execution unit, configured to execute the second change operation based on the change parameters in the change event to synchronize data between the first database and the second database.

[0009] According to a fifth aspect of the embodiments of this application, a data synchronization system is also provided, including: a first server and a second server; the first server is configured to obtain a first method identifier and change parameters corresponding to a first change operation, wherein the first method identifier is an operation method identifier corresponding to the first change operation, and the first change operation includes an operation performed by the first server in response to a data change request for the first database; create a change event based on the first method identifier and change parameters; and send the change event to the second server;

[0010] The second server is configured to obtain a change event sent by the first server; obtain the first method identifier from the change event; determine a second method identifier corresponding to the first method identifier; the second method identifier is an operation method identifier for a second change operation on the second database; and execute the second change operation based on the change parameters in the change event to synchronize the data between the first database and the second database.

[0011] According to a sixth aspect of the embodiments of this application, a data synchronization system is also provided, including: a first cloud server, a second cloud server, an upstream database, and a downstream database;

[0012] The first cloud server is configured to obtain a first method identifier and change parameters corresponding to the first change operation, wherein the first method identifier is an operation method identifier corresponding to the first change operation, and the first change operation includes an operation performed by the first cloud server in response to a data change request for the upstream database; create a change event based on the first method identifier and change parameters; and send the change event to the second cloud server.

[0013] The second cloud server is used to obtain the change event sent by the first cloud server; obtain the first method identifier from the change event, and determine the second method identifier corresponding to the first method identifier; the second method identifier is the operation method identifier of the second change operation for the downstream database; and execute the second change operation based on the change parameters in the change event to synchronize the data between the upstream database and the downstream database.

[0014] According to a seventh aspect of the present application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the data synchronization method of the first aspect or the data synchronization method of the second aspect through the computer program.

[0015] According to an eighth aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the data synchronization method of the first aspect or the data synchronization method of the second aspect when running.

[0016] According to a ninth aspect of the embodiments of this application, a computer program product is also provided, including a computer program, which is executed by a processor using the data synchronization method of the first aspect described above, or the data synchronization method of the second aspect described above.

[0017] In this embodiment, a first server obtains a first method identifier and change parameters corresponding to a first change operation. The first change operation includes an operation performed by the first server in response to a data change request for the first database; a change event is created based on the first method identifier and change parameters; the change event is sent to a second server; the second server obtains the change event sent by the first server; the second server determines a second change operation for the second database corresponding to the change event; and the second change operation is executed to synchronize data between the first and second databases. Compared to existing technologies, this application, by performing a first change operation on the first database and then executing a second change operation corresponding to the first change operation on the second database, not only enables the first and second databases to efficiently complete data synchronization tasks, but also improves the universality and flexibility of data synchronization between databases, with fewer limitations and higher applicability. This solves the technical problem of poor universality and flexibility in data synchronization across multiple database environments in related technologies. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0019] Figure 1 is a schematic diagram of an application environment for an optional data synchronization method according to an embodiment of this application;

[0020] Figure 2 is a flowchart illustrating an optional data synchronization method according to an embodiment of this application;

[0021] Figure 3 is a schematic diagram of the architecture of an optional data synchronization method according to an embodiment of this application;

[0022] Figure 4 is a schematic diagram of the architecture of another optional data synchronization method according to an embodiment of this application;

[0023] Figure 5 is a flowchart illustrating another optional data synchronization method according to an embodiment of this application;

[0024] Figure 6 is a schematic diagram of the architecture of another optional data synchronization method according to an embodiment of this application;

[0025] Figure 7 is a flowchart illustrating another optional data synchronization method according to an embodiment of this application;

[0026] Figure 8 is a schematic diagram of the architecture of another optional data synchronization method according to an embodiment of this application;

[0027] Figure 9 is a flowchart illustrating another optional data synchronization method according to an embodiment of this application;

[0028] Figure 10 is a flowchart illustrating another optional data synchronization method according to an embodiment of this application;

[0029] Figure 11 is a schematic diagram of the architecture of another optional data synchronization method according to an embodiment of this application;

[0030] Figure 12 is a schematic diagram of the architecture of another optional data synchronization method according to an embodiment of this application;

[0031] Figure 13 is a schematic diagram of a data synchronization device according to an embodiment of this application;

[0032] Figure 14 is a schematic diagram of another data synchronization device provided in an embodiment of this application;

[0033] Figure 15 is a schematic diagram of the structure of a data synchronization system provided in an embodiment of this application;

[0034] Figure 16 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0035] Figure 17 is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0036] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0038] Optionally, according to one aspect of the embodiments of this application, a data synchronization method is provided. As an optional implementation, the above-described data synchronization method can be applied to, but is not limited to, the application environment shown in FIG1. ​​This application environment may include, but is not limited to, a first server 102, a network 110, and a second server 112. The first server 102 includes a database 104 and a processing engine 106. The database is used to store various types of data and corresponding change data. The processing engine 106 is used to obtain a first method identifier and change parameters corresponding to a first change operation. The first change operation includes an operation performed by the first server in response to a data change request for the database 104. The first method identifier is an operation method identifier corresponding to the first change operation. A change event is created based on the first method identifier and the change parameters. The change event is sent to the second server 112 so that the second server 112 performs a second change operation for the database 114 based on the change event. The second change operation is used to synchronize data between the database 104 and the database 114. Furthermore, the second server 112 includes a database 114 and a processing engine 116. The database is used to store various types of data and corresponding change data. The processing engine 116 is used to obtain a change event sent by the first server 102, the change event being created by the first server 102 based on a first method identifier and change parameters corresponding to a first change operation, the first change operation including an operation performed by the first server in response to a data change request for database 104; determine a second change operation for database 114 corresponding to the change event, and execute the second change operation to synchronize the data between database 104 and database 114.

[0039] The specific process is as follows: The first server 102 executes steps S102-S106 to obtain the first method identifier and change parameters corresponding to the first change operation. The first change operation includes the operation performed by the first server in response to the data change request for database 104. The first method identifier is the operation method identifier corresponding to the first change operation. The first method identifier is used to obtain the operation method identifier of the second change operation for database 114. A change event is created based on the first method identifier and the change parameters. The change event is sent to the second server 112 so that the second server 112 performs the second change operation for database 114 based on the change event. The second change operation is used to synchronize the data between database 104 and database 114. The second server 112 executes steps S108-S112, obtaining a change event sent by the first server 102. The change event is created by the first server 102 based on a first method identifier and change parameters corresponding to a first change operation. The first change operation includes an operation performed by the first server 102 in response to a data change request for database 104. The second server 112 determines a second change operation for database 114 corresponding to the change event and executes the second change operation to synchronize data between database 104 and database 114. Feedback information on data synchronization is sent to the first server 102.

[0040] Optionally, the first server 102 and the second server 112 can be mobile phones, laptops, tablets, PDAs, MIDs (Mobile Internet Devices), desktop computers, smart TVs, etc. The networks can include, but are not limited to, wired networks and wireless networks. The wired networks include local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). The wireless networks include Bluetooth, Wi-Fi, and other networks that enable wireless communication. The first server 102 and the second server 112 can also be a single server, a server cluster consisting of multiple servers, or a cloud server. The cloud server includes, but is not limited to, private cloud servers or public cloud servers. The above is merely an example, and no limitations are imposed in this embodiment.

[0041] In multi-database environments, ensuring data consistency is paramount. To guarantee consistency, relevant technologies typically employ replication of databases of the same type for data synchronization, or utilize third-party data synchronization tools to synchronize data across different database types. However, these solutions are only compatible with certain database types, limiting their applicability and offering limited versatility and flexibility in data synchronization.

[0042] To address the aforementioned technical problems, as an optional implementation method, as shown in Figure 2, this application provides a data synchronization method applied to a first server. The method includes the following steps:

[0043] S202, obtain the first method identifier and change parameters corresponding to the first change operation, the first change operation includes the operation performed by the first server in response to the data change request for the first database; the first method identifier is the operation method identifier corresponding to the first change operation, and the first method identifier is used to obtain the operation method identifier of the second change operation for the second database.

[0044] Specifically, in this embodiment, the first server can be a single server interacting with the client, a server cluster consisting of multiple servers, or a cloud server, including but not limited to private cloud servers or public cloud servers. The first method identifier is the operation method identifier corresponding to the first change operation. That is, when the first change operation is a data addition operation, the first method identifier can be the method identifier corresponding to the data addition method; when the first change operation is a data deletion operation, the first method identifier can be the method identifier corresponding to the data deletion method; when the first change operation is a data modification operation, the first method identifier can be the method identifier corresponding to the data modification method. Since the first method identifier can correspond to the change operation method identifiers of various types of databases, it can be used to obtain the operation method identifier for the second change operation on the second database. For example, if a data addition operation is performed on the first database, the second method identifier for the data addition operation on the second database can be obtained based on the first method identifier corresponding to the data addition operation. Then, the corresponding data addition operation can be performed on the second database based on the second method identifier, thereby synchronizing the data between the first and second databases. It should be noted that the first and second databases can be of the same type or different types.

[0045] As shown in Figure 3, taking a shopping platform as an example, suppose upstream merchant A modifies the attribute information of its product 1 in the merchant client (first client), for example, adjusting the price of product 1 (product ID 1001). This requires modifying the table containing the attribute information of product 1 stored in the first database (database A). The first server responds to the client's data change request for the first database by executing the first change operation (price adjustment operation). In one example, the price of product 1 is modified through a database operation command: `update products set price = new price where id = 1001`. Here, `products` is the table containing the attribute information of product 1. It should be noted that after the price of product 1 in database A is modified, the price of product 1 in the second database (database B) needs to be updated synchronously so that users can see the latest price of product 1 in the second client.

[0046] After the first server completes the price adjustment operation for product 1, it obtains the first method identifier and change parameter fields1 corresponding to the price adjustment operation. Assuming the first method identifier is modify-A, the change parameter fields1 includes the product's attribute information "1001" and "new price", etc.

[0047] S204, Create a change event based on the first method identifier and the change parameters.

[0048] Specifically, in this embodiment of the application, after the first server completes the price adjustment operation for product 1, it creates a change event event1, for example, based on the first method identifier modify-A, the change parameter "1001", and the product attributes such as "new price". That is, event1 contains the first method identifier modify-A, the change parameter "1001", and the "new price".

[0049] S206, the change event is sent to the second server so that the second server performs the second change operation based on the change event. The second change operation is used to synchronize the data between the first database and the second database.

[0050] Specifically, in this embodiment of the application, as shown in FIG3, for example, the first server sends a change event event1 to the second server. Based on the change event event1, the second server performs a second change operation on the second database (database B) to modify the table containing the attribute information of product 1 stored in database B, thereby ensuring the consistency of data between the first database and the second database.

[0051] In one example, the second server, based on the method identifier `modify-A` in change event `event1`, looks up the corresponding method identifier `modify-B`. Then, based on the change method and change parameters "1001" and "new price" corresponding to `modify-B`, it modifies the price of product 1 stored in database B. After the price of product 1 in the second database (database B) is adjusted to the new price, downstream users can see the latest price of product 1 in the second client. It should be noted that there can be a unit correspondence between the new price and the price of product 1 stored in database B. For example, the new price may be in US dollars, while the original price of product 1 stored in database B may be in Chinese yuan (RMB). The conversion between the two can be performed according to the exchange rate between US dollars and RMB.

[0052] In other embodiments, the data synchronization method provided in this application can also be used in master-slave server architecture scenarios. As shown in Figure 4, the first server is the master server, corresponding to the master database (database A), and the second and third servers are slave servers. The programs running on the second and third servers are the same as those on the first server. When data in database A changes, the first server synchronizes the changed data to database B corresponding to the second server, and synchronizes the changed data to database C corresponding to the third server. When the first server fails, such as crashing, the second server and database B can be enabled to provide the same service as the first server, or the third server and database C can be enabled to provide the same service as the first server.

[0053] In this embodiment, a first server obtains a first method identifier and change parameters corresponding to a first change operation. The first change operation includes an operation performed by the first server in response to a data change request for the first database; a change event is created based on the first method identifier and change parameters; the change event is sent to a second server; the second server obtains the change event sent by the first server; the second server determines a second change operation for the second database corresponding to the change event; and the second change operation is executed to synchronize data between the first and second databases. Compared to existing technologies, this application, by performing a first change operation on the first database and then executing a second change operation corresponding to the first change operation on the second database, not only enables the first and second databases to efficiently complete data synchronization tasks, but also improves the universality and flexibility of data synchronization between databases, with fewer limitations and higher applicability. This solves the technical problem of poor universality and flexibility in data synchronization across multiple database environments in related technologies.

[0054] As an optional implementation, as shown in FIG5, this application embodiment provides a data synchronization method, the method comprising the following steps:

[0055] S502, obtain the first method identifier and change parameters corresponding to the first change operation, wherein the first change operation includes the operation performed by the first server in response to the data change request for the first database.

[0056] S504, Create a change event based on the first method identifier and change parameters;

[0057] S506, the change event is sent to a message queue so that the second server reads the change event from the message queue and performs a second change operation on the second database based on the change event. The second change operation is used to synchronize data between the first database and the second database. The message queue includes at least one partition.

[0058] Specifically, in this embodiment, the message queue can be a memory queue deployed on a single machine, or it can be deployed in a distributed manner to form multiple message service clusters. The aforementioned message queue may or may not have partitioning functionality. The message queue includes, but is not limited to, memory queues, distributed stream processing platforms such as Kafka, or open-source distributed message middleware such as RocketMQ. As shown in Figure 6, the first server sends the change event to the message queue, and the second server reads the change event from the message queue and performs a second change operation on the second database (database B) based on the change event. This application reduces the dependency between different systems by storing and transmitting change events used for data synchronization through a message queue. Furthermore, because the message queue persistently stores change events until they are successfully processed, even if the sender or receiver is temporarily unavailable, the change events will not be lost. Therefore, this application significantly improves the reliability of the data synchronization process.

[0059] Steps S502 and S504 have been explained above and will not be repeated here.

[0060] As an optional implementation, as shown in FIG7, this application embodiment provides a data synchronization method, the method comprising the following steps:

[0061] S702, obtain the first method identifier and change parameters corresponding to the first change operation, wherein the first change operation includes the operation performed by the first server in response to the data change request for the first database.

[0062] S704, a change event is created based on the first method identifier and the change parameters.

[0063] S706, Calculate the target partition corresponding to the change event based on the partition key corresponding to the change event and the number of partitions in the message queue;

[0064] S708, the change event is routed to the target partition so that the second server reads the change event from the target partition and performs a second change operation on the second database based on the change event. The second change operation is used to synchronize data between the first database and the second database.

[0065] In this embodiment, to improve data synchronization efficiency, as shown in Figure 8, a message queue comprising multiple partitions is used to store multiple change events created by the first server, and these multiple change events are transmitted to the second server. Each partition can synchronously transmit different change events to the second server. The concurrency of change event synchronization is the same as the number of partitions.

[0066] The sharding key is a field in a change event. In online shopping platforms, the sharding key can be user ID, order number, etc.; in domain name resolution, it can be domain name information, IP address, etc. All change events with the same sharding key are sent to the same sharding key. In one example, suppose the change event created by the first server is Event1, and its corresponding sharding key is shardingKey1. The number of sharding keys in the message queue is N, where N is a positive integer. For example, the sharding key can be calculated using a hash algorithm: P(N-1) = hash(shardingKey1)%N. The first server routes the change event to the target sharding key with sharding key P(N-1). The second server reads the change event from the target sharding key and performs a second change operation on the second database based on the change event.

[0067] Steps S702 and S704 have been explained above and will not be repeated here.

[0068] In one or more embodiments, the change parameters further include execution parameters and an idempotency identifier, and the creation of a change event based on the first method identifier and the change parameters includes:

[0069] A change event is created based on the first method identifier, the execution parameters, and the idempotency identifier corresponding to the first change operation; the idempotency identifier is used to ensure that the second server obtains the same result when repeatedly executing the same change operation.

[0070] Specifically, in this embodiment, to avoid inconsistencies in synchronized data caused by performing the same change operation multiple times on the database when the network is unstable or the system has a retry mechanism, an idempotency identifier is introduced to solve the above-mentioned technical problem. This idempotency identifier can be a character sequence randomly generated based on the current change operation. The execution parameters may include, for example, product attribute information. In one example, the first server creates a change event based on the first method identifier `modify-A`, the execution parameters, the product ID and product price, and the idempotency identifier corresponding to the first change operation.

[0071] As an optional implementation, as shown in FIG9, this application embodiment provides a data synchronization method applied to a second server, the method comprising the following steps:

[0072] S902, Obtain a change event sent by the first server. The change event is created by the first server based on the first method identifier and change parameters corresponding to the first change operation. The first change operation includes the operation performed by the first server in response to a data change request for the first database. The first method identifier is the operation method identifier corresponding to the first change operation.

[0073] S904, obtain the first method identifier from the change event, and determine the second method identifier corresponding to the first method identifier; the second method identifier is the operation method identifier for the second change operation on the second database;

[0074] S906, based on the change parameters in the change event, perform the second change operation to synchronize the data between the first database and the second database.

[0075] Specifically, in this embodiment, the second server can be a single server interacting with the client, a server cluster consisting of multiple servers, or a cloud server. The aforementioned cloud server includes, but is not limited to, private cloud servers or public cloud servers. As shown in Figure 3, taking a shopping platform as an example, suppose upstream merchant A modifies the attribute information of its product 1 in the merchant client (first client), for example, adjusting the price of product 1 (product ID 1001). This requires modifying the table containing the attribute information of product 1 stored in the first database (database A). The first server responds to the client's data change request for the first database by executing a first change operation (price adjustment operation). In one example, the price of product 1 is modified through a database operation command: `updateproducts set price = new price where id = 1001`. Here, `products` is a table containing the attribute information of product 1. After the price of product 1 in database A is modified, the second server synchronously updates the price of product 1 in the second database (database B), allowing the user to see the latest price of product 1 in the second client.

[0076] After the first server completes the price adjustment operation for product 1, it creates a change event event1, for example, based on the first method identifier modify-A, the change parameter "1001", and the product attributes such as "new price". That is, event1 contains the first method identifier modify-A, the change parameter "1001", and "new price". The first server sends change event event1 to the second server. Based on change event event1, the second server executes a second change operation on the second database (database B), modifying the table containing the attribute information of product 1 stored in database B accordingly, thereby ensuring data consistency between the first and second databases.

[0077] During the second server's execution of the second change operation on the second database (database B) based on change event 1, the second server uses the method identifier `modify-A` in change event 1 to find the corresponding method identifier `modify-B`. Based on the change method and change parameters "1001" and "new price" corresponding to `modify-B`, the server modifies the price of product 1 stored in database B. After the price of product 1 in the second database (database B) is adjusted to the new price, downstream users can see the latest price of product 1 in the second client. It should be noted that the correspondence between the method identifiers `modify-A` and `modify-B` can be stored on the second server, in the second database, or in a cache cluster.

[0078] In this embodiment, a first server obtains a first method identifier and change parameters corresponding to a first change operation. The first change operation includes an operation performed by the first server in response to a data change request for the first database; a change event is created based on the first method identifier and change parameters; the change event is sent to a second server; the second server obtains the change event sent by the first server; the second server determines a second change operation for the second database corresponding to the change event; and the second change operation is executed to synchronize data between the first database and the second database. Compared with the prior art, this application, by performing a first change operation on the first database and then executing a second change operation corresponding to the first change operation on the second database, enables the first database and the second database to efficiently complete the data synchronization task. The data synchronization method of this application can improve the universality and flexibility of data synchronization between databases, and has few limitations and high universality.

[0079] In one or more embodiments, as shown in FIG10, the step of performing the second change operation corresponding to the second method identifier based on the change parameters in the change event includes:

[0080] S1, if the types of the first database and the second database are different, convert the data format of the change parameter to the data format of the second database to obtain the target change parameter;

[0081] S2, based on the target change parameters, execute the second change operation corresponding to the second method identifier.

[0082] Specifically, in this embodiment, it is assumed that the first database is a first-type database and the second database is a second-type database. The change parameter `fields1` contained in the change event `event1` received by the second server is a first-type parameter corresponding to the first database. At this point, according to the data conversion rules between the first-type and second-type databases, the data format of `fields1` needs to be converted to the second-type data format to obtain the target change parameter `fields2` matching the second type. Based on the target change parameter `fields2`, the second server executes the second change operation corresponding to `modify-B` identified by the second method. This data format conversion includes conversions between fields and conversions between data types.

[0083] In one or more embodiments, obtaining the change event sent by the first server includes:

[0084] The change event is read from the target partition of the message queue; the target partition is the partition calculated by the first server based on the partition key corresponding to the change event and the number of partitions in the message queue.

[0085] Specifically, to reduce dependencies between different systems and improve the security of the data synchronization process, this application uses a message queue to store and transmit change events used for data synchronization. As shown in Figure 6, the first server sends change events to the message queue, and the second server reads the change events from the message queue and performs a second change operation on the second database (database B) based on the change events. The message queue includes one or more partitions.

[0086] In this embodiment, it is assumed that the change event created by the first server is Event1, its corresponding partition key is shardingKey1, and the number of partitions in the message queue is N, where N is a positive integer. For example, the partition number P(N-1) corresponding to the target partition of the change event can be calculated using a hash algorithm: P(N-1) = hash(shardingKey1) % N. The first server routes the change event to the target partition with partition number P(N-1), and the second server reads the change event from the target partition and performs a second change operation on the second database based on the change event.

[0087] In one or more embodiments, performing the second change operation includes:

[0088] Record the consumption location information of the change events in the target partition that are consumed;

[0089] Once it is determined that the second server has recovered from the fault state to the normal state, the second change operation is continued based on the consumption location information.

[0090] Specifically, in the embodiments of this application, the aforementioned consumption location information refers to the location information where a change event occurring in the target partition is successfully consumed, and is also commonly referred to as a "consumption location" or "offset". For example, the consumption location can be identified by Offset, and the system executes an Offset+1 operation for each successful consumption of a change event in the target partition.

[0091] As shown in Figure 11, suppose the target partition is partition 1, which stores 5 change events, event1 to event5. Events 1 to event5 are sequentially entered into partition 1 and then consumed by the second server. When event1 is consumed by the second server, the consumption offset is recorded as 0. When event4 is consumed by the second server, the consumption offset is recorded as 3. Suppose the second server encounters a failure while consuming event4. After recovering from the failure, the second server, based on the consumption offset of 3, determines that the current change event has been consumed as event4. The second server then retrieves event4 from partition 1 and continues to execute the second change operation.

[0092] In one or more embodiments, the changed parameter includes an idempotency flag, and the data synchronization method further includes: saving the idempotency flag carried in the changed event to a cache;

[0093] The step of performing the second change operation corresponding to the second method identifier based on the change parameters in the change event includes:

[0094] If an idempotency flag corresponding to the current change event exists in the cache, delete the idempotency flag and execute the second change operation corresponding to the second method flag;

[0095] If the cache does not contain an idempotency flag corresponding to the current change event, an error message is generated indicating that the same change operation has been executed repeatedly.

[0096] Specifically, in this embodiment, to avoid inconsistencies in synchronized data caused by performing the same change operation on the database multiple times when the network is unstable or the system has a retry mechanism, an idempotency identifier is introduced to solve the above-mentioned technical problem. This idempotency identifier can be a character sequence randomly generated based on the current change operation. In one example, the first server creates a change event `event1` based on the first method identifier `modify-A`, execution parameters, product ID and product price, and the idempotency identifier `token1` corresponding to the first change operation. After receiving the change event `event1`, the second server saves the idempotency identifier `token1` carried in the change event to a cache; this cache can be, for example, a Tair cache cluster.

[0097] When the second server executes a change operation, it checks the idempotency flag stored in the cache. For example, if an idempotency flag corresponding to the current change event `event1` exists in the cache, it deletes the idempotency flag `token1` and executes the second change operation corresponding to the second method flag `modify-B`. In one example, after change event `event1` is consumed, if it is consumed again, the idempotency flag `token1` in the cache has been deleted, and there is no idempotency flag corresponding to the current change event `event1` in the cache. The second server will not execute the second change operation corresponding to the second method flag `modify-B` again, and will then generate an error message indicating that the same change operation has been executed repeatedly.

[0098] As an optional implementation, as shown in FIG12, this application embodiment provides a data synchronization method, the method comprising the following steps:

[0099] Step 1: Application 1 (first server) receives a change request for database A.

[0100] Step 2: Application 1 executes database A change operation (such as add, update, delete, etc.), with the corresponding method name being method1 and the parameter list being fields1 (change parameters).

[0101] Step 3: Construct database A change event event1 using application 1. The main information of event1 includes: key information of the data change method, such as method name method1, execution parameter list fields1, partition key shardingKey (e.g., user id, domain id, DNS record id, etc.) and idempotency flag.

[0102] Step 4: Application 1 calculates which partition of the message queue the event should be routed to for sending based on the partition key shardingKey of the event event1 in database A and the number of partitions N in the message queue. For example, the partition number P(N-1) is calculated as P(N-1) = hash(shardingKey) % N, where N is a positive integer.

[0103] Step 6: Application 2 (second server) concurrently receives messages from each partition of the message queue, where partition P(N-1) receives the database change event event1.

[0104] Step 7: Based on the key information of the execution data change method in the change event event1, Application 2 finds the corresponding execution database change method method2 for this application.

[0105] Step 8: Using the execution parameter list fields1 in the change event event1 as parameters, execute the method method2 found in step 7. This will synchronously replay the operations on database A in step 2 on database B, ensuring that the data in database A and database B remain consistent.

[0106] Step 9: During the data synchronization process, Application 2 records the message consumption position information of partition P(n). If Application 2 crashes and recovers, it can continue to perform change operations through the message position tracing function, thereby ensuring data consistency and integrity.

[0107] This application's embodiments do not rely on the database's own replication or log file functions for data synchronization between databases, and can support synchronization between any type of database, demonstrating versatility. Furthermore, it abstracts the database synchronization process into a multi-producer-consumer model, transforming system database update operations into message events for partitioned concurrent sending and consumption, thereby increasing the concurrency of data synchronization and improving data synchronization performance.

[0108] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0109] According to another aspect of the embodiments of this application, a data synchronization device is also provided, the device being applied to a first server, as shown in FIG13, the device comprising:

[0110] The acquisition unit 1302 is used to acquire the first method identifier and change parameters corresponding to the first change operation, wherein the first change operation includes the operation performed by the first server in response to the data change request for the first database; the first method identifier is the operation method identifier corresponding to the first change operation, and the first method identifier is used to acquire the operation method identifier of the second change operation for the second database;

[0111] Creation unit 1304 is used to create a change event based on the first method identifier and the change parameters;

[0112] The sending unit 1306 is used to send the change event to the second server so that the second server can perform the second change operation based on the change event. The second change operation is used to synchronize the data between the first database and the second database.

[0113] In this embodiment, a first server obtains a first method identifier and change parameters corresponding to a first change operation. The first change operation includes an operation performed by the first server in response to a data change request for the first database; a change event is created based on the first method identifier and change parameters; the change event is sent to a second server; the second server obtains the change event sent by the first server; the second server determines a second change operation for the second database corresponding to the change event; and the second change operation is executed to synchronize data between the first database and the second database. Compared with the prior art, this application, by performing a first change operation on the first database and then executing a second change operation corresponding to the first change operation on the second database, enables the first database and the second database to efficiently complete the data synchronization task. The data synchronization method of this application can improve the universality and flexibility of data synchronization between databases, and has few limitations and high universality.

[0114] In one or more embodiments, the transmitting unit 1306 described above includes:

[0115] A sending module is used to send the change event to a message queue so that the second server can read the change event from the message queue; the message queue includes at least one partition.

[0116] In one or more embodiments, the change parameters include a partition key, where the same partition key corresponds to the same change data object, and the change data object is the data targeted by each of the first change operations. The sending module includes:

[0117] The calculation subunit is used to calculate the target partition corresponding to the change event based on the partition key corresponding to the change event and the number of partitions in the message queue;

[0118] A routing subunit is used to route the change event to the target partition, so that the second server can read the change event from the target partition.

[0119] In one or more embodiments, the change parameters further include execution parameters and an idempotency flag, and the creation unit 1304 includes:

[0120] A creation module is used to create a change event based on the first method identifier, the execution parameters, and the idempotency identifier corresponding to the first change operation; the idempotency identifier is used to ensure that the second server obtains the same result when repeatedly executing the same change operation.

[0121] According to another aspect of the embodiments of this application, a data synchronization apparatus for implementing the above-described data synchronization method is also provided, applied to a second server. As shown in FIG14, the apparatus includes:

[0122] The acquisition unit 1402 is used to acquire a change event sent by the first server. The change event is created by the first server based on the first method identifier and change parameters corresponding to the first change operation. The first change operation includes the operation performed by the first server in response to a data change request for the first database.

[0123] The determining unit 1404 is configured to obtain the first method identifier from the change event and determine a second method identifier corresponding to the first method identifier; the second method identifier is an operation method identifier for a second change operation on the second database.

[0124] The execution unit 1406 is configured to perform the second change operation based on the change parameters in the change event to synchronize the data between the first database and the second database.

[0125] In one or more embodiments, the execution unit 1406 includes:

[0126] The conversion subunit is used to convert the data format of the change parameter to the data format of the second database when the types of the first database and the second database are different, so as to obtain the target change parameter;

[0127] The execution subunit is used to perform the second change operation corresponding to the second method identifier based on the target change parameters.

[0128] In one or more embodiments, the acquisition unit 1402 includes:

[0129] The reading module is used to read the change event from the target partition of the message queue; the target partition is the partition calculated by the first server based on the partition key corresponding to the change event and the number of partitions in the message queue.

[0130] In one or more embodiments, the execution unit 1406 includes:

[0131] The recording module is used to record the consumption location information of the change events consumed in the target partition;

[0132] The second execution module is used to determine that the second server has recovered from the fault state to the normal state, and to continue to execute the second change operation based on the consumption location information.

[0133] In one or more embodiments, the change parameter includes an idempotency flag, and the data synchronization device further includes: a cache unit for saving the idempotency flag carried in the change event to a cache;

[0134] The execution unit 1406 includes:

[0135] The deletion subunit is used to delete the idempotency identifier if an idempotency identifier corresponding to the current change event exists in the cache, and to execute the second change operation corresponding to the second method identifier.

[0136] A generation subunit is used to generate an error message indicating that the same change operation is being executed repeatedly if there is no idempotency identifier corresponding to the current change event in the cache.

[0137] According to another aspect of the embodiments of this application, a data synchronization system for implementing the above-described data synchronization method is also provided. As shown in FIG15, the system includes a first server, a second server, a first database, and a second database;

[0138] The first server is configured to obtain a first method identifier and change parameters corresponding to the first change operation, wherein the first method identifier is an operation method identifier corresponding to the first change operation, and the first change operation includes an operation performed by the first server in response to a data change request for the first database; create a change event based on the first method identifier and change parameters; and send the change event to the second server.

[0139] The second server is configured to obtain a change event sent by the first server; obtain the first method identifier from the change event; determine the second method identifier corresponding to the first method identifier; the second method identifier is an operation method identifier for a second change operation on the second database; and execute the second change operation based on the change parameters in the change event to synchronize the data between the first database and the second database.

[0140] According to another aspect of the embodiments of this application, a data synchronization system for implementing the above-described data synchronization method is also provided, the system comprising: a first cloud server, a second cloud server, an upstream database, and a downstream database;

[0141] The first cloud server is configured to obtain a first method identifier and change parameters corresponding to the first change operation, wherein the first method identifier is an operation method identifier corresponding to the first change operation, and the first change operation includes an operation performed by the first cloud server in response to a data change request for the upstream database; create a change event based on the first method identifier and change parameters; and send the change event to the second cloud server.

[0142] The second cloud server is used to obtain a change event sent by the first cloud server; obtain the first method identifier from the change event, and determine the second method identifier corresponding to the first method identifier; the second method identifier is an operation method identifier for a second change operation for the downstream database; and execute the second change operation based on the change parameters in the change event to synchronize the data between the upstream database and the downstream database.

[0143] Specifically, in this embodiment of the application, taking a shopping platform as an example, the first database can be a database for merchants to store data, the second database can be a database for users to retrieve data, the first cloud server can provide online sales services for merchants, and the second cloud server can provide shopping services for users.

[0144] As shown in Figure 3, suppose upstream merchant A modifies the attribute information of its product 1 in the merchant client (first client), for example, adjusting the price of product 1 (product ID 1001). This requires modifying the table containing the attribute information of product 1 stored in the first database (database A). The first server responds to the client's data change request for the first database by executing the first change operation (price adjustment operation). In one example, the price of product 1 is modified via a database operation command: `update products set price = new price where id = 1001`. Here, `products` is the table containing the attribute information of product 1. After the price of product 1 in database A is modified, the second server synchronously updates the price of product 1 in the second database (database B), so that users can see the latest price of product 1 in the second client.

[0145] In one or more embodiments, the first cloud server is specifically configured to send the change event to a message queue, so that the second cloud server reads the change event from the message queue; the message queue includes at least one partition.

[0146] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described data synchronization method is also provided. This electronic device may be the first server or the second server shown in FIG1. ​​This embodiment uses the electronic device as the first server as an example for illustration. As shown in FIG16, the electronic device includes a memory 1602 and a processor 1604. The memory 1602 stores a computer program, and the processor 1604 is configured to execute the steps of any of the above-described method embodiments through the computer program.

[0147] Optionally, in this embodiment, the aforementioned electronic device may be at least one of a plurality of network devices in a computer network.

[0148] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0149] S11, obtain the first method identifier and change parameters corresponding to the first change operation, the first change operation includes the operation performed by the first server in response to the data change request for the first database; the first method identifier is the operation method identifier corresponding to the first change operation, and the first method identifier is used to obtain the operation method identifier of the second change operation for the second database;

[0150] S12, create a change event based on the first method identifier and the change parameters;

[0151] S13, the change event is sent to the second server so that the second server performs the second change operation based on the change event. The second change operation is used to synchronize the data between the first database and the second database.

[0152] Optionally, those skilled in the art will understand that the structure shown in FIG16 is merely illustrative. FIG16 does not limit the structure of the electronic device described above. For example, the electronic device may also include more or fewer components (such as network interfaces) than shown in FIG16, or have a different configuration than that shown in FIG16.

[0153] The memory 1602 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data synchronization method and apparatus in this embodiment. The processor 1604 executes various functional applications and data synchronization by running the software programs and modules stored in the memory 1602, thereby realizing the aforementioned data synchronization method. The memory 1602 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1602 may further include memory remotely located relative to the processor 1604, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1602 may be used, but is not limited to, to store the first method identifier and change parameters.

[0154] As an example, as shown in FIG16, the memory 1602 may include, but is not limited to, the acquisition unit 1302, the creation unit 1304, and the sending unit 1306 in the data synchronization device. Furthermore, it may include, but is not limited to, other module units in the data synchronization device, which will not be elaborated upon in this example.

[0155] Optionally, the transmission device 1606 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1606 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1606 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0156] In addition, the aforementioned electronic device also includes a connection bus 1610 for connecting various module components in the aforementioned electronic device.

[0157] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described data synchronization method is also provided. This electronic device may be the first server or the second server shown in FIG1. ​​This embodiment uses the electronic device as the second server as an example for illustration. As shown in FIG17, the electronic device includes a memory 1702 and a processor 1704. The memory 1702 stores a computer program, and the processor 1704 is configured to execute the steps of any of the above-described method embodiments through the computer program.

[0158] Optionally, in this embodiment, the aforementioned electronic device may be at least one of a plurality of network devices in a computer network.

[0159] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0160] S21, Obtain a change event sent by the first server. The change event is created by the first server based on the first method identifier and change parameters corresponding to the first change operation. The first change operation includes the operation performed by the first server in response to a data change request for the first database. The first method identifier is the operation method identifier corresponding to the first change operation.

[0161] S22, obtain the first method identifier from the change event, and determine the second method identifier corresponding to the first method identifier; the second method identifier is the operation method identifier for the second change operation on the second database;

[0162] S23, based on the change parameters in the change event, perform the second change operation to synchronize the data between the first database and the second database.

[0163] Optionally, those skilled in the art will understand that the structure shown in Figure 17 is merely illustrative, and the electronic device may also be a cloud server or a physical server, or a cloud server cluster or a physical server cluster, etc. Figure 17 does not limit the structure of the aforementioned electronic device. For example, the electronic device may also include more or fewer components (such as network interfaces) than shown in Figure 17, or have a different configuration than that shown in Figure 17.

[0164] The memory 1702 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data synchronization method and apparatus in this embodiment. The processor 1704 executes various functional applications and data synchronization by running the software programs and modules stored in the memory 1702, thereby realizing the aforementioned data synchronization method. The memory 1702 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1702 may further include memory remotely located relative to the processor 1704, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1702 may be used, but is not limited to, to store a first method identifier, a second method identifier, and change parameters.

[0165] As an example, as shown in FIG17, the memory 1702 may include, but is not limited to, the acquisition unit 1402, the determination unit 1404, and the execution unit 1406 in the data synchronization device. Furthermore, it may include, but is not limited to, other module units in the data synchronization device, which will not be elaborated upon in this example.

[0166] Optionally, the transmission device 1706 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1706 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1706 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0167] In addition, the aforementioned electronic device also includes a connection bus 1710 for connecting various module components within the electronic device.

[0168] In other embodiments, the aforementioned electronic device can be a node in a distributed system, which can be a blockchain system. This blockchain system is formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, and any type of computing device, such as a server or terminal, can become a node in the blockchain system by joining this peer-to-peer network.

[0169] In one or more embodiments, this application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the power fault detection method described above. The computer program is configured to execute the steps of any of the method embodiments described above when running.

[0170] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0171] S11, obtain the first method identifier and change parameters corresponding to the first change operation, the first change operation includes the operation performed by the first server in response to the data change request for the first database; the first method identifier is the operation method identifier corresponding to the first change operation, and the first method identifier is used to obtain the operation method identifier of the second change operation for the second database;

[0172] S12, create a change event based on the first method identifier and the change parameters;

[0173] S13, the change event is sent to the second server, so that the second server performs the second change operation based on the change event. The second change operation is used to synchronize data between the first database and the second database. Alternatively, the computer-readable storage medium described above may also be configured to store a computer program for performing the following steps:

[0174] S21, Obtain a change event sent by the first server. The change event is created by the first server based on the first method identifier and change parameters corresponding to the first change operation. The first change operation includes the operation performed by the first server in response to a data change request for the first database. The first method identifier is the operation method identifier corresponding to the first change operation.

[0175] S22, obtain the first method identifier from the change event, and determine the second method identifier corresponding to the first method identifier; the second method identifier is the operation method identifier for the second change operation on the second database;

[0176] S23, based on the change parameters in the change event, perform the second change operation to synchronize the data between the first database and the second database.

[0177] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0178] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0179] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0180] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0185] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

Claims

1. A data synchronization method, characterized in that, Applied to a first server, the method includes: obtaining a first method identifier and change parameters corresponding to a first change operation, wherein the first change operation includes an operation performed by the first server in response to a data change request for a first database, the first method identifier being an operation method identifier corresponding to the first change operation, and the first method identifier being used to obtain an operation method identifier for a second change operation for a second database; creating a change event based on the first method identifier and the change parameters; and sending the change event to a second server, so that the second server performs the second change operation based on the change event, wherein the second change operation is used to synchronize data between the first database and the second database.

2. The data synchronization method according to claim 1, characterized in that, Sending the change event to the second server includes: sending the change event to a message queue so that the second server reads the change event from the message queue; the message queue includes at least one partition.

3. The data synchronization method according to claim 2, characterized in that, The change parameters include a partition key, with the same partition key corresponding to the same change data object. The change data object is the data targeted by each of the first change operations. Sending the change event to a message queue so that the second server can read the change event from the message queue includes: calculating the target partition corresponding to the change event based on the partition key corresponding to the change event and the number of partitions in the message queue; and routing the change event to the target partition so that the second server can read the change event from the target partition.

4. The data synchronization method according to any one of claims 1 to 3, characterized in that, The change parameters also include execution parameters and an idempotency flag. The step of creating a change event based on the first method flag and the change parameters includes: creating a change event based on the first method flag, the execution parameters, and the idempotency flag corresponding to the first change operation; the idempotency flag is used to ensure that the second server obtains the same result when repeatedly executing the same change operation.

5. A data synchronization method, characterized in that, Applied to a second server, the method includes: obtaining a change event sent by a first server, wherein the change event is created by the first server based on a first method identifier and change parameters corresponding to a first change operation, the first change operation including an operation performed by the first server in response to a data change request for a first database; the first method identifier being an operation method identifier corresponding to the first change operation; obtaining the first method identifier from the change event, determining a second method identifier corresponding to the first method identifier; the second method identifier being an operation method identifier for a second change operation for a second database; and performing the second change operation based on the change parameters in the change event to synchronize data between the first database and the second database.

6. The data synchronization method according to claim 5, characterized in that, The step of performing the second change operation based on the change parameters in the change event includes: when the types of the first database and the second database are different, converting the data format of the change parameters to the data format of the second database to obtain the target change parameters; and performing the second change operation corresponding to the second method identifier based on the target change parameters.

7. The data synchronization method according to any one of claims 5 to 6, characterized in that, The step of obtaining the change event sent by the first server includes: reading the change event from the target partition of the message queue; the target partition is the partition calculated by the first server based on the partition key corresponding to the change event and the number of partitions in the message queue.

8. The data synchronization method according to claim 7, characterized in that, The execution of the second change operation includes: recording the consumption point information of the change event consumed in the target partition; determining that the second server has recovered from the fault state to the normal state, and continuing to execute the second change operation based on the consumption point information.

9. The data synchronization method according to claim 5, characterized in that, The change parameters include an idempotency flag, and the data synchronization method further includes: saving the idempotency flag carried in the change event to a cache; the step of executing the second change operation corresponding to the second method flag based on the change parameters in the change event includes: if an idempotency flag corresponding to the current change event exists in the cache, deleting the idempotency flag and executing the second change operation corresponding to the second method flag; if an idempotency flag corresponding to the current change event does not exist in the cache, generating an error message indicating repeated execution of the same change operation.

10. A data synchronization system, characterized in that, The system includes: a first server and a second server; the first server is configured to obtain a first method identifier and change parameters corresponding to a first change operation, wherein the first method identifier is an operation method identifier corresponding to the first change operation, and the first change operation includes an operation performed by the first server in response to a data change request for a first database; create a change event based on the first method identifier and change parameters; and send the change event to the second server; the second server is configured to obtain the change event sent by the first server; obtain the first method identifier from the change event, and determine a second method identifier corresponding to the first method identifier; the second method identifier is an operation method identifier for a second change operation for a second database; and execute the second change operation based on the change parameters in the change event to synchronize data between the first database and the second database.

11. A data synchronization system, characterized in that, The system includes: a first cloud server, a second cloud server, an upstream database, and a downstream database; the first cloud server is used to obtain a first method identifier and change parameters corresponding to a first change operation, wherein the first method identifier is an operation method identifier corresponding to the first change operation, and the first change operation includes an operation performed by the first cloud server in response to a data change request for the upstream database; creating a change event based on the first method identifier and change parameters; and sending the change event to the second cloud server; the second cloud server is used to obtain the change event sent by the first cloud server; obtain the first method identifier from the change event, and determine a second method identifier corresponding to the first method identifier; the second method identifier is an operation method identifier for a second change operation for the downstream database; and execute the second change operation based on the change parameters in the change event to synchronize data between the upstream database and the downstream database.

12. The system according to claim 11, characterized in that, The first cloud server is specifically configured to send the change event to a message queue, so that the second cloud server can read the change event from the message queue; the message queue includes at least one partition.

13. An electronic device, characterized in that, The invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method as described in any one of claims 1 to 4 or 5 to 9.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1 to 4 or 5 to 9.

15. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the method of any one of claims 1 to 4 or 5 to 9.

Images