Data synchronization method and device

Abstract

The invention provides a data synchronization method and device, which are applied to the technical field of data processing and are used for improving data synchronization efficiency and realizing rapid switching of clusters. The method is applied to data synchronization servers of a first cluster and a second cluster, the first cluster comprises a first queue, the second cluster comprises a second queue, and the method comprises the following steps: determining all to-be-synchronized service messages in the first queue based on synchronization point information of the first queue, reading each to-be-synchronized service message from the first queue according to the sequence of the indexes from small to large; analyzing any service message to be synchronized, and obtaining metadata of any service message to be synchronized; determining a data synchronization rate of any to-be-synchronized service message according to the metadata of any to-be-synchronized service message and the metadata of the last to-be-synchronized service message in the first queue; and synchronizing any service message to be synchronized to the second queue according to the data synchronization rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a data synchronization method and device. BACKGROUND

[0002] With the development of the Internet and computer technology, the business volume and user volume continue to grow. As an important part of the interaction between application systems, the message queue that can be persisted needs to have high-availability backup and disaster recovery capability. Usually, the message queue product is deployed in a cluster mode. By setting a primary cluster and a disaster recovery cluster, when the primary cluster fails, the disaster recovery cluster takes over the primary cluster to provide external services. When the primary cluster resumes service, the disaster recovery cluster stops service, and synchronizes the new business messages in the message queue of the disaster recovery cluster to the message queue of the primary cluster. The existing open source technology provides a basic data synchronization tool MirrorMaker to realize data synchronization between message queues. However, the tool can only realize basic data replication, and the data synchronization efficiency is low.

[0003] Therefore, how to improve the data synchronization efficiency and realize the rapid switching of the cluster is a problem to be solved. SUMMARY

[0004] The present application provides a data synchronization method and device for improving the efficiency of data synchronization and realizing the rapid switching of the cluster.

[0005] In a first aspect, the present application provides a data synchronization method applied to a data synchronization server of a first cluster and a second cluster. The first cluster includes a first queue, and the second cluster includes a second queue. The second cluster is a primary cluster that resumes service, and the first cluster is a disaster recovery cluster that stops service. The method comprises the following steps: determining all the business messages to be synchronized in the first queue based on synchronization point information of the first queue, and reading the business messages to be synchronized in the first queue in turn according to the index from small to large; the synchronization point information of the first queue indicates that the latest business message synchronized from the second queue in the first queue is the index; the business message to be synchronized is the new business message written into the first queue from the next index of the synchronization point information of the first queue when the second cluster is abnormal and the first cluster provides service; analyzing any business message to be synchronized to obtain the metadata of the any business message to be synchronized; determining the data synchronization rate of the any business message to be synchronized according to the metadata of the any business message to be synchronized and the metadata of the last business message to be synchronized in the first queue; and synchronizing the any business message to be synchronized to the second queue according to the data synchronization rate.

[0006] In the embodiment, the synchronization point information of the first queue can accurately obtain the newly added business messages in the first queue when the first cluster provides services in the second cluster exception, only the newly added business messages are synchronized, repeated synchronization is avoided, and the efficiency of data synchronization can be improved; and the data synchronization rate of any to-be-synchronized business message is determined according to the metadata of any to-be-synchronized business message and the metadata of the last to-be-synchronized business message in the first queue, the metadata of each to-be-synchronized business message is taken as a monitoring object, the data synchronization rate of each to-be-synchronized business message is adjusted in time, the speed and reliability of data synchronization are improved, the efficiency of data synchronization can be further improved, and fast switching of the cluster is realized.

[0007] In an optional implementation, the metadata includes an index of any to-be-synchronized business message in the first queue and a timestamp when the any to-be-synchronized business message is written into the first queue; and the data synchronization rate of the any to-be-synchronized business message is determined according to the metadata of the any to-be-synchronized business message and the metadata of the last to-be-synchronized business message in the first queue, including: obtaining a first data synchronization rate and a timestamp when data synchronization of the any to-be-synchronized business message starts; calculating a first difference value according to the index of the last to-be-synchronized business message in the first queue and the index of the any to-be-synchronized business message; calculating a second difference value according to the timestamp when data synchronization of the any to-be-synchronized business message starts and the timestamp when the any to-be-synchronized business message is written into the first queue; if the first difference value belongs to a first range and / or the second difference value belongs to a second range, determining that the data synchronization rate of the any to-be-synchronized business message is the first data synchronization rate; otherwise, adjusting the first data synchronization rate according to a preset rule, and determining that the data synchronization rate of the any to-be-synchronized business message is the adjusted first data synchronization rate.

[0008] In the embodiment, the first difference value can indicate the number of business messages that have not been synchronized, and the second difference value can indicate the waiting time length of the any to-be-synchronized business message from being written into the first queue to being synchronized from the first queue to the second queue. The first difference value and the second difference value are respectively judged, when the first difference value belongs to the first range and / or the second difference value belongs to the second range, it is determined that the current data synchronization progress is normal, and the any to-be-synchronized business message can be synchronized according to the predetermined first data synchronization rate; otherwise, it is determined that the current data synchronization progress is too slow or too fast according to the actual judgment, and then the first data synchronization rate is adjusted adaptively, and the any to-be-synchronized business message is synchronized according to the adjusted first data synchronization rate. In addition, if the first difference value seriously exceeds the first range and / or the second difference value seriously exceeds the second range, it is determined that the current data synchronization progress is abnormal, and an alarm can be triggered to ensure stable data synchronization.

[0009] In one optional implementation, the first data synchronization rate is a preset data synchronization rate or the data synchronization rate of the previous business message that has completed data synchronization in the first queue for any business message to be synchronized.

[0010] In this embodiment, if any business message to be synchronized is the first business message to be synchronized, the first data synchronization rate can be a preset data synchronization rate. If any business message to be synchronized is not the first business message to be synchronized, the first synchronization rate can be the data synchronization rate of the previous business message that has been synchronized. In this way, the data synchronization process of any business message to be synchronized can be more adapted to the current data synchronization progress, thereby improving the efficiency of data synchronization.

[0011] In one optional implementation, the metadata of any business message to be synchronized further includes indication information, which is used to indicate the source of any business message to be synchronized; before determining the data synchronization rate of any business message to be synchronized based on the metadata of any business message to be synchronized and the metadata of the last business message to be synchronized in the first queue, the method further includes: if the indication information indicates that the source of any business message to be synchronized is the second cluster, then data synchronization is not performed on any business message to be synchronized; if the indication information indicates that the source of any business message to be synchronized is the first cluster, then data synchronization is determined to be performed on any business message to be synchronized.

[0012] In this embodiment, the source of any business message to be synchronized is first determined by the indication information in the metadata of the business message to be synchronized. If it comes from the second cluster, it means that the business message to be synchronized already exists in the second cluster and does not need to be synchronized repeatedly. Data synchronization is only performed on the business messages to be synchronized from the first cluster, which can avoid repeated synchronization, improve the efficiency of data synchronization, and realize the rapid switching of clusters.

[0013] In one optional implementation, synchronizing any business message to be synchronized to the second queue according to the data synchronization rate includes: writing the any business message to be synchronized to the second queue and obtaining the index of the any business message to be synchronized in the second queue; updating the synchronization point information of the second queue based on the index of the any business message to be synchronized in the second queue; and storing the synchronization point information in a preset storage space in the form of queue name-index key-value pairs.

[0014] In this embodiment, after synchronizing any business message to be synchronized to the second queue, the synchronization point information of the second queue is updated. This allows the first queue to synchronize data from the second queue after the second cluster resumes normal service, thus avoiding repeated synchronization of business messages synchronized from the first queue to the second queue based on the synchronization point information of the second queue, thereby ensuring the reliability and efficiency of cluster data synchronization.

[0015] In one optional implementation, the first queue and the second queue provide reading services in ascending order of the business message indices; the first queue stores multiple business messages synchronized from the second queue, and the first queue provides these multiple business messages when the second cluster fails; before determining all business messages to be synchronized in the first queue based on the synchronization point information of the first queue, the method further includes: obtaining a first reading record of business messages in the first queue, the first reading record including a first index of the latest read business message in the first queue when the first cluster stops service; synchronizing the first reading record to the second queue to obtain a second reading record; the second reading record includes a second index mapping the latest read business message in the first queue to the corresponding business message in the second queue; when the second cluster resumes service, it starts providing reading services from the business message corresponding to the next index of the second index.

[0016] In this embodiment, when the second cluster resumes service, the first read record of the business message in the first queue is synchronized to the second queue to obtain the second read record. This facilitates the second cluster to take over the first cluster in a timely manner to provide uninterrupted read service during cluster switching, thereby improving the user experience while achieving rapid cluster switching.

[0017] Secondly, embodiments of this application provide a data synchronization device applied to data synchronization servers of a first cluster and a second cluster. The first cluster includes a first queue, and the second cluster includes a second queue. The second cluster is a primary cluster for service recovery, and the first cluster is a disaster recovery cluster for service shutdown. The device includes: The reading module is used to: determine all business messages to be synchronized in the first queue based on the synchronization point information of the first queue, and read each business message to be synchronized sequentially from the first queue in ascending order of index; the synchronization point information of the first queue indicates the index of the latest business message synchronized from the second queue in the first queue; the business message to be synchronized is the newly added business message written to the first queue starting from the next index of the synchronization point information of the first queue when the second cluster is abnormal and the first cluster provides service. The synchronization module is used to: parse any business message to be synchronized and obtain the metadata of the business message to be synchronized; determine the data synchronization rate of the business message to be synchronized based on the metadata of the business message to be synchronized and the metadata of the last business message to be synchronized in the first queue; and synchronize the business message to be synchronized to the second queue according to the data synchronization rate.

[0018] In one optional implementation, the metadata includes the index of any business message to be synchronized in the first queue and the timestamp when it was written to the first queue. When the synchronization module determines the data synchronization rate of any business message to be synchronized based on the metadata of the business message to be synchronized and the metadata of the last business message to be synchronized in the first queue, it specifically performs the following steps: obtaining a first data synchronization rate and the timestamp when data synchronization of the business message to be synchronized begins; calculating a first difference based on the index of the last business message to be synchronized in the first queue and the index of the business message to be synchronized; calculating a second difference based on the timestamp when data synchronization of the business message to be synchronized begins and the timestamp when the business message to be synchronized is written to the first queue; if the first difference belongs to a first range and / or the second difference belongs to a second range, then the data synchronization rate of the business message to be synchronized is determined to be the first data synchronization rate; otherwise, the first data synchronization rate is adjusted according to a preset rule, and the adjusted data synchronization rate of the business message to be synchronized is determined to be the first data synchronization rate.

[0019] In one optional implementation, the first data synchronization rate is a preset data synchronization rate or the data synchronization rate of the previous business message that has completed data synchronization in the first queue for any business message to be synchronized.

[0020] In one optional implementation, the metadata of any business message to be synchronized further includes indication information, which is used to indicate the source of any business message to be synchronized; before determining the data synchronization rate of any business message to be synchronized based on the metadata of any business message to be synchronized and the metadata of the last business message to be synchronized in the first queue, the synchronization module is further configured to: if the indication information indicates that the source of any business message to be synchronized is the second cluster, then not perform data synchronization on any business message to be synchronized; if the indication information indicates that the source of any business message to be synchronized is the first cluster, then determine to perform data synchronization on any business message to be synchronized.

[0021] In one optional implementation, when the synchronization module synchronizes any business message to be synchronized to the second queue according to the data synchronization rate, it is specifically used to: write the any business message to be synchronized to the second queue and obtain the index of the any business message to be synchronized in the second queue; update the synchronization point information of the second queue based on the index of the any business message to be synchronized in the second queue; and store the synchronization point information in a preset storage space in the form of a key-value pair of queue name-index.

[0022] In one optional implementation, the first queue and the second queue provide reading services in ascending order of the business message indices; the first queue stores multiple business messages synchronized from the second queue, and the first queue provides these multiple business messages when the second cluster fails; before determining all business messages to be synchronized in the first queue based on the synchronization point information of the first queue, the reading module is further configured to: obtain a first reading record of business messages in the first queue, the first reading record including the first index of the latest read business message in the first queue when the first cluster stops service; the synchronization module is further configured to: synchronize the first reading record to the second queue to obtain a second reading record; the second reading record includes a second index mapping the latest read business message in the first queue to the corresponding business message in the second queue; when the second cluster resumes service, it starts providing reading services from the business message corresponding to the next index of the second index.

[0023] Thirdly, embodiments of this application provide an electronic device including at least one processor, which, when executing a computer program stored in a memory, causes the method as described in the first aspect or any optional implementation of the first aspect to be implemented.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium for storing instructions that, when executed, cause the method as described in the first aspect or any optional implementation of the first aspect to be implemented.

[0025] Fifthly, embodiments of this application provide a computer program product, including computer program code, which, when executed on a computer, causes the method as described in the first aspect or any optional implementation of the first aspect to be implemented.

[0026] The technical effects or advantages of one or more technical solutions provided in the second, third, fourth and fifth aspects of this application can all be explained by the corresponding technical effects or advantages of one or more technical solutions provided in the first aspect. Attached Figure Description

[0027] Figure 1This application provides a schematic diagram of application deployment within a single data center, as illustrated in an embodiment of the present application. Figure 2 This application provides an abstract application deployment diagram as an embodiment of the present application. Figure 3 This is a schematic diagram of a master-slave cluster mode provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a data synchronization system provided in an embodiment of this application; Figure 5 A flowchart of a data synchronization method provided in an embodiment of this application; Figure 6 An example diagram of synchronization point information provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of another data synchronization system provided in the embodiments of this application; Figure 8 A structural diagram of a data synchronization device provided in an embodiment of this application; Figure 9 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The data collection, dissemination, and use in this application all comply with relevant national laws and regulations.

[0029] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0030] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solution of this application, rather than limitations on the technical solution of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0031] It should be understood that "multiple" in the description of the embodiments of this application refers to two or more. "First," "second," etc., in the embodiments of this application are used to distinguish different objects, not to describe a specific order. The term "and / or" in the embodiments of this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. In the embodiments of this application, a module refers to a part of a software system that has independent functionality.

[0032] To facilitate understanding of the technical solutions provided in the embodiments of this application, the relevant technical features involved in the embodiments of this application will be explained below.

[0033] It should be noted that these explanations are intended to make the embodiments of this application easier to understand, and should not be regarded as a limitation on the scope of protection claimed in this application.

[0034] 1. Message Queue.

[0035] A message queue is a middleware carrier used to store messages during message transmission. Message queues act as intermediaries, relaying messages from their source to their destination, providing routing and ensuring the orderly delivery of messages.

[0036] With the development of internet and computer technology, computer application systems are undertaking more and more business functions. To provide users with a more convenient experience, the business logic that application systems need to implement is becoming increasingly complex, causing business processing to consume more time. Furthermore, complex business processes often involve multiple subsystems collaborating through network communication. Because network communication can experience varying latency and disconnections, traditional processing methods can significantly impact the overall response time of the business. Therefore, in recent years, more and more application systems have introduced message queues. Message queues decouple the internal execution processes of a business, breaking down a complex business process into multiple relatively independent sub-business processes. When these sub-business processes need to communicate or call each other, a message queue can be used as middleware between the communicating parties to facilitate the transmission of messages or data. The upstream, as a message producer, is responsible for sending messages to the message queue, and the downstream, as a message consumer, can retrieve and use messages from the message queue. This approach simplifies application function design and enables the independent operation of each sub-business process. Most mainstream message queues today also support temporarily storing unconsumed messages in persistent media (such as hard drives) to cope with abnormal situations such as power outages or system crashes, and record the current consumption position of consumers, allowing consumers to stop or continue consuming at any time according to their processing capacity. With these persistent characteristics, message queues help application systems achieve data caching and flow control.

[0037] 2. Producers and consumers.

[0038] A producer is a participant in a message queue who generates a message and sends it to the message queue (equivalent to writing a business message); a consumer is a participant in a message queue who retrieves a message through the message queue (equivalent to reading a business message). In the embodiments of this application, "consumer" and "reader" have the same meaning below.

[0039] 3. Displacement (offset).

[0040] Data in a message queue is stored in an ordered manner. The message queue records the latest write position of the producer and the current consumption position of the consumer. These positions are often represented by a monotonically increasing integer called the offset. The offset is similar to an index, and any message in the message queue can be mapped using the offset.

[0041] 4. Application deployment within a single data center.

[0042] With the continuous growth of business volume and user base, persistent message queues, as a crucial link in the interaction between application systems, also need to consider high availability issues such as backup and disaster recovery. Generally, most common message queue products support cluster deployment, meaning that when the same data or message is written to the message queue, it is simultaneously stored on the storage media of multiple message queue servers. If one server fails, the others can continue to provide services to the application system. However, this mode often has high requirements for network communication latency between servers within the cluster; otherwise, data synchronization between servers will directly affect the producer's write speed. Therefore, in this high availability mode, the cluster is often deployed in the same data center or server room, which can only support server-level disaster recovery. If the data center or server room fails, it will lead to the interruption of external services.

[0043] See Figure 1 This is a schematic diagram illustrating application deployment within a single data center, provided in an embodiment of this application. Figure 1 As shown, an application with a message queue middleware refers to an application system S1 that performs specific business functions. Depending on actual needs, application system S1, during operation, acts as a message producer by writing specific business messages to message queue K (using Apache Kafka as an example). Application system S2, which performs another specific function, connects to message queue K to receive these business messages and executes corresponding functional actions based on the message content. Generally, application systems also rely on database products (DB) to read and write business data. Therefore, as... Figure 1 As shown, application systems S1 and S2 are each connected to a database DB. Thus, application systems S1 and S2, message queue K, and database DB together constitute a system. If any component in this system fails, the availability of the system's external services will be affected. Therefore, from the perspective of building high availability or disaster recovery capabilities for the application system, it is necessary to... Figure 1 The system shown is deployed in multiple data centers to implement a primary-backup mode.

[0044] 5. Application deployment of master-slave mode.

[0045] To address the potential issues that may arise in a single data center and achieve disaster recovery capabilities across data centers or at the data center level, it is generally necessary to build multiple clusters (forming a primary + backup model), with the backup cluster synchronizing data from the primary cluster asynchronously.

[0046] according to Figure 1As shown in the system diagram, from an overall perspective, both application system S1, which acts as the message producer, and application system S2, which acts as the message consumer, need to actively connect to the message queue middleware to manipulate business messages. Furthermore, in real-world scenarios, message producers and consumers may reside within the same application system. Figure 1 The system shown can also be abstracted as follows: Figure 2 The system architecture shown includes an application system S, a message queue K, and a database DB.

[0047] Generally, database products and message queue middleware both have the ability to synchronize data between clusters. A diagram illustrating the deployment of a master-slave cluster in two separate data centers is shown below. Figure 3 As shown. In Figure 3 In this scenario, a primary cluster is deployed in data center 1, and a backup cluster is deployed in data center 2. The primary cluster (also known as the production cluster) includes application system S, message queue K, and database DB; the backup cluster (also known as the disaster recovery cluster) includes application system S', message queue K', and database DB'. Figure 3 In the master-slave cluster mode shown, under normal conditions, the master cluster in data center 1 provides services to the outside world, while the backup cluster in data center 2 only synchronizes data from the master cluster.

[0048] When the primary cluster in data center 1 becomes unavailable, the backup cluster in data center 2 can replace the primary cluster in data center 1 to provide services. Data that needs to be synchronized between clusters includes queue message data, queue metadata, and consumer consumption positions. The default data synchronization direction is from the primary cluster (upstream) to the backup cluster (downstream). As a middleware system, the message queue generally only provides message sending, receiving, and temporary storage functions, without including specific business logic implementation. Therefore, from the perspective of the overall application, in the event of a data center-level failure, in addition to the message queue cluster needing to ensure rapid primary / backup cluster failover, the application systems using the message queue also need to coordinate the failover.

[0049] 6. A tool for data replication between Kafka clusters.

[0050] This data replication tool is a native data synchronization feature of Kafka, used to replicate data between Kafka clusters. It supports topic partitioning and configuration parameter synchronization; it can be implemented using Kafka MirrorMaker 2.0 (MM2). However, existing open-source Kafka tools only provide basic data synchronization tools, which can only replicate business data within message queues between clusters. This results in low data synchronization efficiency and cannot meet the needs of smooth and rapid switching of application systems.

[0051] Therefore, embodiments of this application are provided to improve data synchronization efficiency and enable rapid cluster switching.

[0052] See Figure 4 This is a schematic diagram of the structure of a data synchronization system provided in an embodiment of this application. The system includes a first cluster, a second cluster, and a data synchronization server. The first cluster includes a first queue, and the second cluster includes a second queue.

[0053] The first cluster is a backup cluster (i.e., a disaster recovery cluster), and the second cluster is the primary cluster (i.e., a production cluster). The data synchronization server has communication connections with both the first and second clusters.

[0054] It is understandable that this data synchronization server can operate independently of the first and second clusters (i.e., as shown in the image). Figure 4 As shown in the figure, it can also be located in the first cluster or the second cluster. Alternatively, a data synchronization server with the same function can be deployed in the first cluster and the second cluster respectively. In addition, the data synchronization server can be an independent server, or it can be a module or code with data synchronization function deployed on a general application server in the first cluster and / or the second cluster. This application embodiment does not limit this.

[0055] This application embodiment takes the example of a data synchronization system that includes only one data synchronization server. The working state of the data synchronization system can include the following examples.

[0056] Example 1: When the second cluster is providing services normally, users can read or write business messages in the second queue. The data synchronization server will synchronize the business messages in the second queue to the first queue and update the synchronization point information of the first queue in real time.

[0057] Example 2: When the second cluster malfunctions, the second cluster stops providing services, and the first cluster is started to provide external services. The current synchronization point information of the first queue is stored.

[0058] Example 3: The first cluster provides services, and users can read multiple business messages that have been synchronized from the second queue in the first queue, and can write new business messages at the end of the first queue; the data synchronization server stops synchronizing business messages from the second queue to the first queue.

[0059] Example 4: When the second cluster resumes service and the first cluster stops service, the data synchronization server needs to synchronize the newly added business messages in the first queue to the second queue.

[0060] This application's embodiments primarily focus on Example 4, providing a data synchronization method to improve data synchronization efficiency and achieve rapid cluster switching. This method is applied to a data synchronization server; see [link to relevant documentation]. Figure 5 The method includes steps S501 to S504: S501. Based on the synchronization point information of the first queue, determine all business messages to be synchronized in the first queue, and read each business message to be synchronized from the first queue in ascending order of index.

[0061] The synchronization point information of the first queue indicates the index of the latest business message synchronized from the second queue in the first queue; the business message to be synchronized is the new business message written to the first queue starting from the next index of the synchronization point information of the first queue when the second cluster is abnormal and the first cluster provides services.

[0062] For example, see Figure 6 This is an example of synchronization point information provided in an embodiment of this application. For example... Figure 6 As shown, when the second cluster is providing normal service, the data synchronization server synchronizes the business messages from the second queue to the first queue. When the second cluster malfunctions, assuming the data synchronization server has just completed synchronizing the business message at index M in the second queue to the first queue, this business message is the latest business message synchronized by the first queue from the second queue, and its index is O in the first queue. Subsequently, due to the second cluster malfunction, the first cluster will take over and provide services. At this time, the data synchronization server stops synchronizing the business messages from the second queue to the first queue, and the first queue starts storing new business messages written by users from the next index after index O, i.e., index P.

[0063] To facilitate a clearer understanding of which index the new business messages are written to the user's first queue from, this embodiment of the application sets synchronization point information, namely... Figure 6 The index O shown here facilitates the determination of all business messages to be synchronized when the second cluster resumes service and the first cluster stops service, based on this synchronization point information. Figure 6 The business messages corresponding to index P and subsequent indexes shown are the business messages to be synchronized.

[0064] In this way, only the new business messages written by users in the first queue can be synchronized to the second queue, avoiding duplicate synchronization and improving the efficiency of data synchronization.

[0065] In one possible example, the index is the identifier of each business message in the queue (such as the offset mentioned above). In this embodiment, the index of each business message is set from smallest to largest according to the order in which they are written. This index can also refer to the position number of the business message in the queue. Figure 6 The index is represented by uppercase English letters. In practical applications, other symbols such as numbers or combinations of numbers and letters can also be used as indexes. This application does not limit this.

[0066] It is understandable that the index of the same business message in the first queue and its index in the second queue may not be the same.

[0067] S502. Parse any business message to be synchronized and obtain the metadata of any business message to be synchronized.

[0068] In one possible embodiment, the metadata may include: the timestamp when any business message to be synchronized is written to the first queue, the data size, the index in the first queue, the importance, the priority, etc., which can be set according to the actual situation.

[0069] S503. Determine the data synchronization rate of any business message to be synchronized based on the metadata of the business message to be synchronized and the metadata of the last business message to be synchronized in the first queue.

[0070] In one possible embodiment, the metadata includes the index of any business message to be synchronized in the first queue and the timestamp when it was written to the first queue; the specific implementation of step S503 can be as follows: Obtain the first data synchronization rate and the timestamp when data synchronization begins for any business message to be synchronized; Calculate the first difference based on the index of the last business message to be synchronized in the first queue and the index of any business message to be synchronized. The second difference is calculated based on the timestamp when data synchronization of any business message to be synchronized begins and the timestamp when any business message to be synchronized is written into the first queue. If the first difference falls within the first range and / or the second difference falls within the second range, then the data synchronization rate of any business message to be synchronized is determined to be the first data synchronization rate; otherwise, the first data synchronization rate is adjusted according to a preset rule, and the data synchronization efficiency of any business message to be synchronized is determined to be the adjusted first data synchronization rate.

[0071] For example, taking the first and second queues as Kafka queues, a request can be sent to MirrorMaker via the JMX interface to obtain the timestamp when data synchronization of any business message to be synchronized begins and the first data synchronization rate.

[0072] It is understandable that the first difference indicates how many pending but not yet synchronized business messages remain in the first queue (i.e., first difference + 1); the second difference indicates the waiting time for any pending business message from being written to the first queue to being ready to be synchronized from the first queue to the second queue. Value ranges are set for the first and second differences. If the first difference falls within the first range and / or the second difference falls within the second range, it indicates that the current data synchronization progress is normal, and data synchronization of any pending business message can proceed according to the predetermined first data synchronization rate; otherwise, the first data synchronization rate needs to be adjusted, and data synchronization of any pending business message should proceed according to the adjusted first data synchronization rate. The specific values ​​of the first and second ranges can be set according to actual needs.

[0073] For example, if the first difference is less than the first range and / or the second difference is less than the second range, it indicates that the current data synchronization progress is relatively fast, and the first data synchronization rate can be appropriately reduced to decrease the burden on the data synchronization server. If the first difference is greater than the first range and / or the second difference is greater than the second range, it indicates that the current data synchronization progress is relatively slow, and the first data synchronization rate can be appropriately increased to improve the efficiency of data synchronization. If the first difference is significantly greater than the first range and / or the second difference is significantly greater than the second range, it indicates that the current data synchronization progress is abnormal, and an alarm can be triggered to prompt technicians to troubleshoot the fault.

[0074] The above-described scheme, which uses the first and second differences as criteria for adjusting the data synchronization rate, is merely one possible example provided by the embodiments of this application. In practice, other criteria can be used based on different metadata. For example, obtaining the index of the latest data to be written in the second queue, calculating the third difference between the index of the latest data to be written in the second queue and the index of the last business message to be synchronized in the first queue, and determining whether the third difference meets a preset range; or obtaining the data size of any business message to be synchronized and adjusting the first data synchronization rate based on the data size, etc. This application does not impose any limitations on this.

[0075] In one possible embodiment, the first data synchronization rate is a preset data synchronization rate or the data synchronization rate of the previous business message that has completed data synchronization in the first queue for any business message to be synchronized.

[0076] It is understandable that if any business message to be synchronized is the first business message to be synchronized, the first data synchronization rate can be a preset data synchronization rate. If any business message to be synchronized is not the first business message to be synchronized, the first data synchronization rate can be the data synchronization rate of the previous business message that has been synchronized. In this way, the data synchronization process of any business message to be synchronized can be more adapted to the current data synchronization progress, thereby improving the efficiency of data synchronization.

[0077] Of course, it is also possible to set the first data synchronization rate for all business messages to be synchronized to the preset data synchronization rate, and then adjust it based on the actual situation of each business message to be synchronized and the current data synchronization progress.

[0078] In one possible design, to further avoid duplicate synchronization of business messages, before step S503, this embodiment of the application also provides a method for intercepting duplicate business messages. The metadata of any business message to be synchronized further includes indication information, which indicates the source of the business message to be synchronized. The specific implementation of this method is as follows: If the indication information indicates that the source of any business message to be synchronized is the second cluster, then data synchronization will not be performed on any business message to be synchronized. If the indication information indicates that the source of any business message to be synchronized is the first cluster, then it is determined that data synchronization will be performed on any business message to be synchronized.

[0079] For example, this indication information can be a flag. For instance, if the metadata of any business message to be synchronized includes a flag with the key "sourceoffset" (i.e., the second cluster offset), it indicates that the source of any business message to be synchronized is the second cluster, and therefore, data synchronization will not be performed on that business message. This further improves the efficiency of data synchronization and avoids adding redundant data to the queue.

[0080] S504. Synchronize any business message to be synchronized to the second queue according to the data synchronization rate.

[0081] In one possible embodiment, step S504 can be implemented as follows: Write any business message to be synchronized to the second queue, and obtain the index of any business message to be synchronized in the second queue; The synchronization point information of the second queue is updated based on the index of any business message to be synchronized in the second queue; the synchronization point information is stored in the preset storage space in the form of key-value pairs of queue name-index.

[0082] For example, if the index of any business message to be synchronized in the second queue is the sourceoffset mentioned above, the synchronization point information of the updated second queue can be stored in local file storage, the database of the second cluster, or middleware with persistence capabilities such as Zookeeper, in the form of queue name as key and sourceoffset of any business message to be synchronized as value.

[0083] It is understood that in practical applications, the first queue and the second queue may each include multiple sub-queues. In this embodiment of the application, the synchronization point information is stored in the form of queue name-index to facilitate the management of the synchronization point information of each queue.

[0084] In this embodiment, the synchronization point information of the first queue can accurately obtain the newly added business messages in the first queue when the second cluster is abnormal and the first cluster is providing services. Only the newly added business messages are synchronized to avoid duplicate synchronization and improve the efficiency of data synchronization. Furthermore, the data synchronization rate of any business message to be synchronized is determined based on the metadata of any business message to be synchronized and the metadata of the last business message to be synchronized in the first queue. The metadata of each business message to be synchronized is used as the monitoring object, and the data synchronization rate of each business message to be synchronized is adjusted in a timely manner to improve the speed and reliability of data synchronization. This can further improve the efficiency of data synchronization and enable rapid cluster switching.

[0085] Furthermore, since simply synchronizing the business messages in the queue to the second cluster may not meet the actual needs, such as when application system S switches to the second cluster, it also needs to restore the effect of continuing to read the business messages at the point where reading was interrupted in the first cluster (instead of reading from the beginning and not skipping unread data), the reading position of the message queue in the first cluster also needs to be synchronized to the message queue in the second cluster to achieve a smooth migration effect.

[0086] Therefore, in one possible design, since the first queue stores multiple business messages synchronized from the second queue, and the first cluster provides services while the user provides the reading service for these multiple business messages, in order to ensure that the cluster can continuously and uninterruptedly provide services to the outside world, this application embodiment provides a read record synchronization method, the specific implementation of which can be as follows: Retrieve the first read record of the business message in the first queue. The first read record includes the first index of the latest read business message in the first queue when the first cluster stopped service. The first read record is synchronized to the second queue to obtain the second read record; the second read record includes a second index that maps the latest read business message in the first queue to the corresponding business message in the second queue; When the second cluster resumes service, it starts providing read services from the business messages corresponding to the next index after the second index.

[0087] In this way, before or at the same time as the synchronization of business messages, the first read record is synchronized to the second queue to obtain the second read record. This makes it easier for the second cluster to take over the first cluster and provide uninterrupted read services during cluster switching, thereby improving the user experience while achieving rapid cluster switching.

[0088] In another possible design, since the business messages read by users in the first queue are all business messages synchronized from the second queue to the first queue, the data synchronization server can also periodically update the second read record of the second queue based on the first read record of the first queue when the second cluster fails and the first cluster provides service. This makes it easier to know the current read position of users in a timely manner when the second cluster recovers service, and to provide uninterrupted service to users.

[0089] Similarly, when the second cluster fails and the first cluster starts providing services, the data synchronization server also needs to obtain the latest second read record from the second cluster and update the first read record based on the latest second read record. Alternatively, when the second cluster is operating normally and the first cluster is not operating, the data synchronization server can periodically update the first read record based on the second read record.

[0090] The above describes the method of this application embodiment using a data synchronization system that includes only one data synchronization server as an example. Similarly, in a data synchronization system, when the first cluster and the second cluster each include one data synchronization server, such as... Figure 7 As shown, the first cluster includes data synchronization server A, and the second cluster includes data synchronization server B. The operating status of this data synchronization system can be illustrated as follows.

[0091] Example 5, such as Figure 7 As shown in line 1, when the second cluster is providing normal service, users can read or write business messages in the second queue. Data synchronization server A will synchronize the business messages in the second queue to the first queue and update the synchronization point information of the first queue in real time.

[0092] Optionally, data synchronization server B can also periodically synchronize the second read record to data synchronization server A, and data synchronization server A updates the first read record based on the second read record.

[0093] Example 6: When the second cluster fails, the second cluster stops providing services, the first cluster starts providing external services, and stores the current synchronization point information of the first queue.

[0094] Example 7: The first cluster provides services, and users can read multiple business messages that have been synchronized from the second queue in the first queue, and can write new business messages at the end of the first queue; Data synchronization server A stops synchronizing business messages from the second queue to the first queue, and periodically sends the first read record of the first queue to data synchronization server B, and data synchronization server B updates the second read record of the second queue according to the first read record.

[0095] Example 8, such as Figure 7 As shown in line 2, when the second cluster resumes service and the first cluster stops service, the data synchronization server B synchronizes the newly added business messages in the first queue to the second queue according to the data synchronization method described above and the methods in various embodiments.

[0096] In this way, by deploying a data synchronization server in each cluster, the burden on each data synchronization server can be reduced, and the stability of data synchronization can be improved.

[0097] The methods provided in the embodiments of this application have been described above. The apparatus provided in the embodiments of this application will be described below.

[0098] Based on the same technical concept, embodiments of this application provide a data synchronization apparatus, which includes a module / unit / means for executing the method performed by the data synchronization server in the above-described method embodiments. This module / unit / means can be implemented in software, or in hardware, or implemented by hardware executing corresponding software.

[0099] See Figure 8 The device 800 is used as a data synchronization server for a first cluster and a second cluster. The first cluster includes a first queue, and the second cluster includes a second queue. The second cluster is the primary cluster for service recovery, and the first cluster is a disaster recovery cluster for service shutdown. The reading module 801 is used to: determine all business messages to be synchronized in the first queue based on the synchronization point information of the first queue, and read each business message to be synchronized sequentially from the first queue in ascending order of index; the synchronization point information of the first queue indicates the index of the latest business message synchronized from the second queue in the first queue; the business message to be synchronized is a newly added business message written to the first queue starting from the next index of the synchronization point information of the first queue when the second cluster is abnormal and the first cluster provides services. The synchronization module 802 is used to: parse any business message to be synchronized and obtain the metadata of the business message to be synchronized; determine the data synchronization rate of the business message to be synchronized based on the metadata of the business message to be synchronized and the metadata of the last business message to be synchronized in the first queue; and synchronize the business message to be synchronized to the second queue according to the data synchronization rate.

[0100] In one optional implementation, the metadata includes the index of any business message to be synchronized in the first queue and the timestamp when it was written to the first queue. When the synchronization module 802 determines the data synchronization rate of any business message to be synchronized based on the metadata of the business message to be synchronized and the metadata of the last business message to be synchronized in the first queue, it is specifically configured to: obtain a first data synchronization rate and the timestamp when data synchronization of the business message to be synchronized begins; calculate a first difference based on the index of the last business message to be synchronized in the first queue and the index of the business message to be synchronized; calculate a second difference based on the timestamp when data synchronization of the business message to be synchronized begins and the timestamp when the business message to be synchronized is written to the first queue; if the first difference belongs to a first range and / or the second difference belongs to a second range, then the data synchronization rate of the business message to be synchronized is determined to be the first data synchronization rate; otherwise, the first data synchronization rate is adjusted according to a preset rule, and the data synchronization rate of the business message to be synchronized is determined to be the adjusted first data synchronization rate.

[0101] In one optional implementation, the first data synchronization rate is a preset data synchronization rate or the data synchronization rate of the previous business message that has completed data synchronization in the first queue for any business message to be synchronized.

[0102] In one optional implementation, the metadata of any business message to be synchronized further includes indication information, which is used to indicate the source of any business message to be synchronized; before determining the data synchronization rate of any business message to be synchronized based on the metadata of any business message to be synchronized and the metadata of the last business message to be synchronized in the first queue, the synchronization module 802 is further configured to: if the indication information indicates that the source of any business message to be synchronized is the second cluster, then not perform data synchronization on any business message to be synchronized; if the indication information indicates that the source of any business message to be synchronized is the first cluster, then determine to perform data synchronization on any business message to be synchronized.

[0103] In one optional implementation, when the synchronization module 802 synchronizes any business message to be synchronized to the second queue according to the data synchronization rate, it is specifically used to: write the any business message to be synchronized to the second queue and obtain the index of the any business message to be synchronized in the second queue; update the synchronization point information of the second queue based on the index of the any business message to be synchronized in the second queue; and store the synchronization point information in a preset storage space in the form of a key-value pair of queue name-index.

[0104] In one optional implementation, the first queue and the second queue provide reading services in descending order of the service message indices; the first queue stores multiple service messages synchronized from the second queue, and the first queue provides these multiple service messages when the second cluster is abnormal; before determining all service messages to be synchronized in the first queue based on the synchronization point information of the first queue, the reading module 801 is further configured to: obtain a first reading record of the service messages in the first queue, the first reading record including a first index of the latest read service message in the first queue when the first cluster stops service; the synchronization module 802 is further configured to: synchronize the first reading record to the second queue to obtain a second reading record; the second reading record includes a second index of the latest read service message in the first queue mapped to the corresponding service message in the second queue; when the second cluster resumes service, it starts providing reading services from the service message corresponding to the next index of the second index.

[0105] It should be understood that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0106] Based on the same technical concept, see [link / reference] Figure 9 This application also provides an electronic device 900, comprising: At least one processor 901; and a communication interface 903 communicatively connected to the at least one processor 901; the at least one processor 901 executes instructions stored in memory 902, causing the electronic device 900 to execute the method steps performed by the data synchronization server in the above method embodiment through the communication interface 903.

[0107] Optionally, the memory 902 is located outside the electronic device 900.

[0108] Optionally, the electronic device 900 includes a memory 902 connected to the at least one processor 901, and the memory 902 contains instructions executable by the at least one processor 901. (See attached image) Figure 9 The dashed line indicates that the memory 902 is optional for the electronic device 900.

[0109] The at least one processor 901 and the memory 902 can be coupled through an interface circuit or integrated together, which is not limited here.

[0110] This application embodiment does not limit the specific connection medium between at least one processor 901, memory 902, and communication interface 903. This application embodiment... Figure 9 At least one processor 901, memory 902, and communication interface 903 are connected via a bus 904, and the bus 904 is in Figure 9 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. This bus section can be an address bus, data bus, control bus, etc. For ease of illustration, Figure 9 It is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0111] It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0112] For example, the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0113] It should be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which acts as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAM (DR RAM).

[0114] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0115] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0116] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium for storing instructions that, when executed, cause a computer to perform the method steps performed by any of the devices in the above method embodiments.

[0117] Based on the same technical concept, this application also provides a computer program product, including computer program code, which, when the computer program code is run on a computer, causes the method steps executed by any device in the above method embodiments to be implemented.

[0118] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 Figure 1 The steps of the function specified in one or more boxes.

[0122] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data synchronization method, characterized in that, A data synchronization server applied to a first cluster and a second cluster, wherein the first cluster includes a first queue, the second cluster includes a second queue, the second cluster is a primary cluster for service recovery, and the first cluster is a disaster recovery cluster for service shutdown, the method comprising: Based on the synchronization point information of the first queue, all business messages to be synchronized in the first queue are determined, and each business message to be synchronized is read from the first queue in ascending order of index; the synchronization point information of the first queue indicates the index of the latest business message synchronized from the second queue in the first queue; the business message to be synchronized is the new business message written to the first queue starting from the next index of the synchronization point information of the first queue when the second cluster is abnormal and the first cluster provides services. Parse any business message to be synchronized to obtain the metadata of the business message to be synchronized; The data synchronization rate of any business message to be synchronized is determined based on the metadata of any business message to be synchronized and the metadata of the last business message to be synchronized in the first queue. Synchronize any business message to be synchronized to the second queue according to the data synchronization rate.

2. The method as described in claim 1, characterized in that, The metadata includes the index of any business message to be synchronized in the first queue and the timestamp when it was written to the first queue; determining the data synchronization rate of any business message to be synchronized based on the metadata of the business message to be synchronized and the metadata of the last business message to be synchronized in the first queue includes: Obtain the first data synchronization rate and the timestamp when data synchronization begins for any of the business messages to be synchronized; Calculate the first difference based on the index of the last business message to be synchronized in the first queue and the index of any business message to be synchronized. The second difference is calculated based on the timestamp when data synchronization of any of the business messages to be synchronized begins and the timestamp when any of the business messages to be synchronized is written into the first queue. If the first difference falls within a first range and / or the second difference falls within a second range, then the data synchronization rate of any business message to be synchronized is determined to be the first data synchronization rate; otherwise, the first data synchronization rate is adjusted according to a preset rule, and the data synchronization rate of any business message to be synchronized is determined to be the adjusted first data synchronization rate.

3. The method as described in claim 2, characterized in that, The first data synchronization rate is a preset data synchronization rate or the data synchronization rate of the previous business message that has completed data synchronization for any business message to be synchronized in the first queue.

4. The method as described in claim 1, characterized in that, The metadata of any business message to be synchronized also includes indication information, which is used to indicate the source of any business message to be synchronized; Before determining the data synchronization rate of any business message to be synchronized based on its metadata and the metadata of the last business message to be synchronized in the first queue, the method further includes: If the indication information indicates that the source of any business message to be synchronized is the second cluster, then data synchronization will not be performed on any business message to be synchronized. If the indication information indicates that the source of any business message to be synchronized is the first cluster, then it is determined that data synchronization will be performed on any business message to be synchronized.

5. The method as described in claim 1, characterized in that, The step of synchronizing any business message to be synchronized to the second queue according to the data synchronization rate includes: Write any of the business messages to be synchronized into the second queue, and obtain the index of any of the business messages to be synchronized in the second queue; The synchronization point information of the second queue is updated based on the index of any business message to be synchronized in the second queue; the synchronization point information is stored in a preset storage space in the form of key-value pairs of queue name-index.

6. The method according to any one of claims 1-5, characterized in that, The first queue and the second queue provide read services in ascending order of the business message index; the first queue stores multiple business messages synchronized from the second queue, and the first queue provides read services for the multiple business messages when the second cluster is abnormal; Before determining all business messages to be synchronized in the first queue based on the synchronization point information of the first queue, the method further includes: Obtain the first read record of the business message in the first queue. The first read record includes the first index of the latest read business message in the first queue when the first cluster stops service. The first read record is synchronized to the second queue to obtain the second read record; the second read record includes a second index that maps the latest read business message in the first queue to the corresponding business message in the second queue; when the second cluster resumes service, it starts providing read service from the business message corresponding to the next index of the second index.

7. A data synchronization device, characterized in that, A data synchronization server applied to a first cluster and a second cluster, wherein the first cluster includes a first queue, the second cluster includes a second queue, the second cluster is a primary cluster for service recovery, and the first cluster is a disaster recovery cluster for service shutdown, the device comprising: The reading module is configured to: determine all business messages to be synchronized in the first queue based on the synchronization point information of the first queue, and read each business message to be synchronized sequentially from the first queue in ascending order of index; the synchronization point information of the first queue indicates the index of the latest business message synchronized from the second queue in the first queue; the business message to be synchronized is a newly added business message written to the first queue starting from the next index of the synchronization point information of the first queue when the second cluster is abnormal and the first cluster provides services; The synchronization module is used to: parse any business message to be synchronized, and obtain the metadata of the business message to be synchronized; determine the data synchronization rate of the business message to be synchronized based on the metadata of the business message to be synchronized and the metadata of the last business message to be synchronized in the first queue; and synchronize the business message to be synchronized to the second queue according to the data synchronization rate.

8. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps included in the method as described in any one of claims 1-6 according to the obtained program instructions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a computer, cause the method as described in any one of claims 1-6 to be implemented.

10. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the method described in any one of claims 1-6.

Images