Database synchronization method and system

Abstract

The embodiment of the invention discloses a database synchronization method and system. The method is applied to a first end, the first end is associated with a first database, and the method comprises the steps that for a first statement needing to be executed in the first database to conduct first data operation and a first parameter of the first statement, the first statement and the first parameter are obtained, and according to the first statement and the first parameter, the first data operation is executed in the first database; determining a first character string; the first character string is pushed to a first message queue, so that under the condition that a second end obtains the first character string from the first message queue, the first data operation is carried out in a second database according to the first character string, and the second end is associated with the second database. According to the technical scheme, the database load can be reduced.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a database synchronization method and system. Background Technology

[0002] With the strategic deployment of technological iteration and innovation in the financial sector, database replacement work is accelerating. Many systems are facing scenarios of database migration and other technology stack iteration updates. In this scenario, it is necessary to consider the ability to quickly switch back to normal traffic in the event of an anomaly after the database traffic switch, so as to minimize system downtime and ensure business continuity.

[0003] To ensure rapid traffic rollback in case of anomalies, it is crucial to maintain consistency of data in both the old and new databases during the parallel operation of the new and old systems after migration. Currently, database synchronization is often used to ensure the consistency of data in both databases.

[0004] However, current database synchronization methods place a heavy load on the database, making it difficult to perform database synchronization for extended periods. This may affect the ability to quickly switch back traffic in abnormal situations, and urgently needs to be addressed. Summary of the Invention

[0005] This invention provides a database synchronization method and system to reduce database load.

[0006] According to one aspect of the present invention, a database synchronization method is provided, applied to a first end, the first end being associated with a first database, the method comprising:

[0007] For the first statement to be executed in the first database to perform the first data operation and the first parameter of the first statement, obtain the first statement and the first parameter, and determine the first string based on the first statement and the first parameter;

[0008] The first string is pushed to the first message queue so that, if the second end retrieves the first string from the first message queue, the first data operation is performed in the second database based on the first string, wherein the second end is associated with the second database.

[0009] According to another aspect of the present invention, a database synchronization method is provided, applied to a second end, the second end being associated with a second database, the method comprising:

[0010] If the first string is retrieved from the first message queue, perform the first data operation in the second database based on the first string;

[0011] The first string is determined by the first end based on the first statement to be executed in the first database to perform the first data operation and the first parameter of the first statement, and is pushed to the first message queue. The first end is associated with the first database.

[0012] According to another aspect of the present invention, a database synchronization system is provided, which may include: a first end and a second end; wherein,

[0013] The first end is used to implement the database synchronization method provided in any embodiment of the present invention during execution, and the second end is used to implement the database synchronization method provided in any embodiment of the present invention during execution.

[0014] The technical solution of this invention is applied to a first end, which is associated with a first database. For a first statement and its first parameters to be executed in the first database for a first data operation, the first statement and first parameters are obtained. Based on the first statement and first parameters, a first string is determined, which is then used to perform the first data operation in a second database. This first string is then pushed to a first message queue. When the second end retrieves the first string from the first message queue, it performs the first data operation in the second database based on the first string, thereby achieving data synchronization between the first and second databases. The second end is associated with the second database. This technical solution, by pushing the first string to the first message queue as a buffer, avoids the problem of excessive database load caused by directly performing the first data operation in the second database based on the first string. In particular, it avoids the problem of excessive database load caused by directly performing the first data operation in the second database based on a large number of first strings in a short period of time, thus reducing the database load.

[0015] It should be understood that the description in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a database synchronization method provided according to an embodiment of the present invention;

[0018] Figure 2This is a flowchart of another database synchronization method provided according to an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram illustrating the publishing and listening of synchronization events in another database synchronization method provided by an embodiment of the present invention;

[0020] Figure 4 This is a flowchart of an optional example of another database synchronization method provided according to an embodiment of the present invention;

[0021] Figure 5 This is a flowchart of an optional example of another database synchronization method provided according to an embodiment of the present invention, which encapsulates the first statement and the first parameter as well as the synchronization event publication.

[0022] Figure 6 This is a flowchart of another database synchronization method provided according to an embodiment of the present invention;

[0023] Figure 7 This is a flowchart of an optional example of another database synchronization method provided according to an embodiment of the present invention;

[0024] Figure 8 This is a structural block diagram of a database synchronization system provided according to an embodiment of the present invention;

[0025] Figure 9 This is a structural block diagram of a database synchronization device according to an embodiment of the present invention;

[0026] Figure 10 This is a structural block diagram of another database synchronization device provided according to an embodiment of the present invention;

[0027] Figure 11 This is a schematic diagram of the structure of an electronic device that implements the database synchronization method of this invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The same applies to "target," "original," etc., and will not be repeated here. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Before introducing the embodiments of the present invention, we will first provide an exemplary description of the application scenarios of database synchronization, the implementation process of the current database synchronization solutions, and the reasons for the problem of their large database load, so as to better understand why the solution proposed in the embodiments of the present invention can reduce the database load.

[0031] With the strategic deployment of technological iteration and innovation in the financial sector, a series of measures have become important engines for maintaining financial information security and enhancing the industry's core competitiveness. Digital transformation of the financial industry has become a key development direction. Through continuous technological innovation and industrial upgrading, not only can financial risks be effectively prevented, but also solid technical support can be provided for building a strong financial nation. Against this backdrop, the replacement of databases and other technologies in banking applications is accelerating. Many systems are facing scenarios such as database migration for applications to the cloud and other technology stack iterations and updates. In this scenario, there will be a need for in-depth adjustments and transformations to the original system architecture. The transformation process needs to consider the ability to quickly switch back to normal traffic in the event of an anomaly after the database traffic switch, so as to minimize system downtime and ensure business continuity.

[0032] To ensure rapid traffic rollback in case of anomalies, it is crucial to maintain consistency of data in both the old and new databases during the parallel operation of the new and old systems after migration. Currently, database synchronization is often used to ensure the consistency of data in both databases.

[0033] However, current database synchronization methods place a heavy load on the database, making it difficult to perform database synchronization for extended periods, which may affect rapid traffic rollback in case of anomalies. For example, according to the technology stack and database system iteration plan, some systems will migrate from the existing on-premises platform deployment connected to the DB-1 database (hereinafter referred to as on-premises application) to the cloud platform deployment connected to the DB-2 database (hereinafter referred to as cloud application). The migration work is required to proceed smoothly according to the overall principles of phased implementation, traffic diversion, and rollback mechanisms. However, this migration work faces many challenges, including a large scale of changes in the migration architecture, the use of newer product technologies, a wide range of upstream and downstream involvement, and a relatively lack of practical experience accumulated by the project team. Looking back at previous database migration projects, many problems were exposed: when some early pilot migration projects switched business traffic to the DB-2 database, the performance differences between the old and new databases, slow SQL queries caused system instability, and extended interface response times led to business operations not being able to proceed normally, and upstream and downstream systems were also affected; compatibility issues of some statements in the DB-2 database caused service anomalies; other unknown situations caused system anomalies, such as incorrect base data, incorrect table creation, and inconsistent field types. Once the above problems occur, it is necessary to consider system rollback (i.e., traffic rollback). If the data in the old and new databases is inconsistent at this time, the business will be in chaos due to data loss after the rollback, which will incalculably prolong the business recovery time and inevitably impact business operations. Current database synchronization methods such as full data synchronization (data migration during downtime) and incremental synchronization still have many limitations in this scenario. For example, both full data synchronization and incremental synchronization place a large load on the database and are difficult to run for a long time, which will result in a short available synchronization window, which may affect the rapid traffic rollback in abnormal situations.

[0034] To address this, this embodiment of the invention pushes the first string to a first message queue, which serves as a buffer. This avoids the problem of excessive database load caused by directly performing first data operations on the second database based on the first string, especially avoiding the problem of excessive database load caused by directly performing first data operations on a large number of first strings on the second database in a short period of time, thereby reducing the database load. This will be explained in detail below.

[0035] Figure 1 This is a flowchart of a database synchronization method provided in an embodiment of the present invention. This embodiment is applicable to database synchronization situations. The method can be executed by the database synchronization device provided in this embodiment of the present invention. The device can be implemented by software and / or hardware, and can be integrated into an electronic device, which can be various user terminals or servers.

[0036] See Figure 1 The method of this embodiment of the invention is applied to a first end, which is associated with a first database. The method of this embodiment of the invention specifically includes the following steps:

[0037] S110. For the first statement to be executed in the first database to perform the first data operation and the first parameter of the first statement, obtain the first statement and the first parameter, and determine the first string based on the first statement and the first parameter.

[0038] The first end can be understood as the end that performs database synchronization operations.

[0039] In this embodiment of the invention, the first end may include a first application, and the second end may include a second application. The first application and the second application may be connected to the same first message queue instance. The incremental data corresponding to the first database is synchronized in real time to the second database connected to the second application through the first message queue via the first application, so as to achieve database synchronization.

[0040] In this embodiment of the invention, after the first application is started, the synchronization start / stop flag of the database synchronization in the configuration file can be detected. If the synchronization start / stop flag indicates that the database synchronization is in a closed state, then no database synchronization operation is performed. If the synchronization start / stop flag indicates that the database synchronization is in a closed state, then the database synchronization operation is performed. Furthermore, if database synchronization can also be achieved on the second end, including the second application, this means that dual-write is possible between the first and second databases (the first and second databases can synchronize with each other). After the second application starts, it can detect the dual-write start / stop flag in the configuration file. If the dual-write start / stop flag indicates that dual-write is off, no dual-write operation will be performed; if the flag indicates that dual-write is on, dual-write operation will be performed. This allows for quick and convenient start / stop of the database dual-write function and control of the dual-write data flow. For example, the dual-write send and receive start / stop flag can be configured as a function switch in the project's "application.properties" file. When the send switch is on, the data persistence layer interceptor runs normally and efficiently, fully processing the intercepted statements and parameters and sending them to the corresponding message queue in a timely manner. When the send switch is off, the data persistence layer interceptor stops dual-write related work and directly returns the return value of the original method. The receive switch is used to control whether the corresponding application on the other end listens to the corresponding message queue to obtain strings from it.

[0041] The first database can be understood as the database associated with the first end. The first database can also be understood as the database that needs to synchronize data to the second database; for example, the first database can be connected to the first end.

[0042] The first data operation can be understood as an operation performed in the first database that corresponds to the first statement and the first parameter; the first data operation may be at least one of the following: adding data, searching for data, updating data, and deleting data.

[0043] The first statement can be understood as a statement executed in the first database to perform the first data operation.

[0044] The first parameter can be understood as the parameter executed in the first database to perform the first data operation.

[0045] The first string can be understood as the string determined by the first statement and the first parameter; the first string can be, for example, a JSON string.

[0046] In this embodiment of the invention, the first string can be determined based on the obtained first statement and first parameter. For example, the first statement and first parameter can be used as the first string; or the first statement and first parameter can be converted into JSON strings to obtain the first string; and so on.

[0047] S120. Push the first string to the first message queue so that when the second end retrieves the first string from the first message queue, perform the first data operation in the second database based on the first string, wherein the second end is associated with the second database.

[0048] The first message queue can be understood as the message queue to which the first string is pushed.

[0049] Understandably, in distributed systems, message queues are a key technology for implementing asynchronous communication and decoupling system components. They are used to transmit and store messages in distributed systems, serving as a medium for data storage and forwarding. Their core function is to act as an intermediate container for messages, coordinating communication between producers and consumers to ensure reliable message transmission. The Remote Dictionary Server (Redis) message queue operates based on memory, achieving read / write speeds at the microsecond level. Its high performance meets the need for rapidly processing large volumes of messages, ensuring efficient system operation. Therefore, in this embodiment of the invention, a Redis message queue can be used as the first message queue. Importantly, using a Redis message queue as the first message queue allows the solution of this embodiment to not only possess the inherent advantages of Redis message queues but also achieve database synchronization without introducing additional components and dependencies, reducing learning and maintenance costs. Furthermore, when the solution of this embodiment employs a message publish / subscribe approach, the Redis message queue's functionality is sufficient to meet the requirements, eliminating the need for complex specialized message queues and avoiding over-design and increased learning costs.

[0050] In this embodiment of the invention, when there are certain requirements for the stability of the first message queue or in scenarios with a larger data scale, more specialized message queues such as Kafka (Apache Kafka), RabbitMQ, or RocketMQ can be selected.

[0051] The second end can be understood as the end associated with the second database that can retrieve the first string from the first message queue.

[0052] The second database can be understood as a database associated with a second end, or it can be understood as a database that needs to be synchronized with the first database.

[0053] In this embodiment of the invention, the first string can be pushed to the first message queue. For example, the jedisCluster.lpsuh() method can be used to push the first string to the first message queue so that when the second end retrieves the first string from the first message queue, the first data operation can be performed in the second database based on the first string.

[0054] The solution of this invention can reduce the database load to ensure long-term database synchronization, especially the ability to accurately and in real-time synchronize incremental data to the second database, ensuring bilateral data consistency. The time consumption is extremely low, at the millisecond level, allowing for longer use and minimal impact on the stability of the source application system. This avoids the need for rapid traffic rollback in abnormal situations, ensuring that after the business traffic is fully switched to the new database, the system can quickly and reliably switch between the old and new applications in the shortest possible time in the event of complex situations such as network failures, data conflicts, and program anomalies.

[0055] The technical solution of this invention is applied to a first end, which is associated with a first database. For a first statement and its first parameters to be executed in the first database for a first data operation, the first statement and first parameters are obtained. Based on the first statement and first parameters, a first string is determined, which is then used to perform the first data operation in a second database. This first string is then pushed to a first message queue. When the second end retrieves the first string from the first message queue, it performs the first data operation in the second database based on the first string, thereby achieving data synchronization between the first and second databases. The second end is associated with the second database. This technical solution, by pushing the first string to the first message queue as a buffer, avoids the problem of excessive database load caused by directly performing the first data operation in the second database based on the first string. In particular, it avoids the problem of excessive database load caused by directly performing the first data operation in the second database based on a large number of first strings in a short period of time, thus reducing the database load.

[0056] An optional technical solution, before determining the first string based on the first statement and the first parameter, the database synchronization method further includes: performing parameter conversion on the first parameter according to the parameter type corresponding to the first parameter in the second database, and updating the first parameter based on the obtained conversion result.

[0057] The parameter type can be understood as the type of the second database corresponding to the first parameter; the parameter type can be at least one of table name type, table partition type, and field type, etc., and correspondingly, the parameter conversion can be at least one of table name conversion, table partition conversion, and field conversion, etc.

[0058] The conversion result can be understood as the result obtained by converting the first parameter according to the parameter type.

[0059] In this embodiment of the invention, redundant whitespace characters in the first statement can also be converted into single spaces. For example, redundant whitespace characters such as long spaces, newlines, and tabs in the first statement can be converted into single spaces in turn, and the first statement can be updated based on the conversion results.

[0060] In this embodiment of the invention, the first parameter can be converted according to its type, and the first parameter can be updated based on the conversion result. Compared to related solutions that can only handle database synchronization between specific databases and cannot meet the database synchronization needs between other types of databases or more complex synchronization requirements, the above technical solution enables the updated first parameter to be applicable to execution in the second database, thereby ensuring universal and reliable data synchronization between different types of first and second databases, and thus meeting the database synchronization needs between other types of databases or more complex synchronization requirements.

[0061] Figure 2 This is a flowchart of another database synchronization method provided in this embodiment of the invention. This embodiment is based on the above-mentioned technical solutions and optimized. In this embodiment, optionally, the first statement is a statement under the target transaction; determining the first string according to the first statement and the first parameter includes: encapsulating the first statement and the first parameter into a list object corresponding to the target transaction according to the execution order of the first statement under the target transaction, and returning to the step of obtaining the first statement and the first parameter if the first statement is a statement other than a resource release statement; if the first statement is a resource release statement, determining the first string according to the list object.

[0062] The explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0063] See Figure 2 The method of this embodiment of the invention is applied to a first end, which is associated with a first database. The method of this embodiment of the invention specifically includes the following steps:

[0064] S210. For the first statement to be executed in the first database to perform the first data operation and the first parameter of the first statement, obtain the first statement and the first parameter, wherein the first statement is the statement under the target transaction.

[0065] The target transaction can be understood as the transaction in which the first statement is located; the target transaction may include at least one statement.

[0066] S220. According to the execution order of the first statement under the target transaction, encapsulate the first statement and the first parameter into a list object corresponding to the target transaction, and if the first statement is a statement other than a resource release statement, return to execute S210.

[0067] The execution order can be understood as the order in which the first statement is executed within the target transaction.

[0068] A list object can be understood as an object represented in list form.

[0069] A resource release statement can be understood as a statement that instructs the release of resources; a resource release statement can indicate that the first statement is the last statement to be executed under the target transaction; a resource release statement can be, for example, a statement corresponding to a closing method (such as the close method).

[0070] In this embodiment of the invention, the first statement and the first parameter can be encapsulated (or stored) in a list object according to the execution order, and if the first statement is a statement other than a resource release statement, the step of obtaining the first statement and the first parameter can be returned.

[0071] In this embodiment of the invention, if the first statement is the first statement under the target transaction, a list object corresponding to the thread-safe target transaction can be defined to encapsulate or store the statements and parameters under the target transaction.

[0072] S230. If the first statement is a resource release statement, determine the first string based on the list object.

[0073] In this embodiment of the invention, if the first statement is a resource release statement, the first string can be determined based on the list object.

[0074] In this embodiment of the invention, when the first statement is a resource release statement, before determining the first string based on the list object, it is also possible to check whether the list object is empty. If it is not empty, the first string is then determined based on the list object.

[0075] In this embodiment of the invention, after determining the first string based on the list object, the Executor interface can be closed, resources such as database connections occupied by the Executor interface can be released, and internal states and caches can be cleaned up.

[0076] In this embodiment of the invention, if it is determined that there is no exception when executing the first statement and the first parameter in the first database, the first string can be determined based on the list object; if it is determined that there is an exception when executing the first statement and the first parameter in the first database, a rollback can be automatically triggered by the transaction manager, and the first end can directly return the return value of the original method after capturing the exception, that is, without performing subsequent database synchronization operations such as determining the first string based on the list object.

[0077] S240. Push the first string to the first message queue so that when the second end retrieves the first string from the first message queue, perform the first data operation in the second database based on the first string, wherein the second end is associated with the second database.

[0078] In the technical solution of this invention, the first statement is a statement under the target transaction. Following the execution order of the first statement within the target transaction, the first statement and the first parameter are encapsulated into a list object corresponding to the target transaction. If the first statement is not a resource release statement, the step of obtaining the first statement and the first parameter is returned. If the first statement is a resource release statement, the first string is determined based on the list object. This technical solution allows statements under the same target transaction to be encapsulated in the same list object, ensuring that when subsequent data operations are performed in the second database, statements under the same target transaction are executed sequentially according to their execution order, thereby guaranteeing the accuracy of database synchronization.

[0079] An optional technical solution involves encapsulating the first statement and the first parameter into a list object corresponding to the target transaction according to the execution order of the first statement within the target transaction. This includes: if the first statement is a data update statement, encapsulating the first statement and the first parameter into a list object corresponding to the target transaction according to the execution order of the first statement within the target transaction; determining a first string based on the first statement and the first parameter; and further including: if the first statement is a data lookup statement, returning to the step of obtaining the first statement and the first parameter.

[0080] In this context, a data update statement can be understood as a statement that modifies data, object attributes, or system status in the database; a data update statement can be at least one of the following: INSERT, UPDATE, and DELETE statements.

[0081] A data lookup statement can be understood as a statement that searches for data in a database, and it does not cause changes to the data, object attributes, or system state in the database.

[0082] In this embodiment of the invention, if the first statement is a data update statement, the first statement and the first parameter can be encapsulated into a list object according to the execution order. If the first statement is a data lookup statement, the step of retrieving the first statement and the first parameter is returned. This technical solution only encapsulates the first statement and the first parameter if the first statement modifies data, object attributes, or system state in the database. If the first statement is a data lookup statement, the encapsulation of the first statement and the first parameter is not performed. This avoids the waste of resources associated with statements that do not need to be executed in the second database for database synchronization, thereby further reducing the database load.

[0083] Another alternative technical solution involves determining the first string based on the list object, including: using a data persistence layer interceptor to create and publish a synchronous event based on the list object using Spring's event mechanism; and converting the list object into the first string when the synchronous event is detected by the event listener component.

[0084] Understandably, the data persistence layer interceptor is one of the core extension mechanisms provided by the data persistence layer framework. It is implemented based on Java dynamic proxies and the chain of responsibility pattern, allowing the interception and enhancement of key content (the first statement and the first parameters) during the execution of Structured Query Language (SQL) to achieve customized processing of the first data operation.

[0085] As is understandable, Spring's event mechanism is an event-driven programming mechanism based on the observer pattern, allowing loosely coupled communication between program components through event publishing and event listening. From an implementation perspective, Spring's event mechanism relies on reflection and dynamic proxy technologies. When the container starts, Spring's event mechanism scans all classes annotated with `@Component`, identifies classes implementing the `ApplicationListener` interface and methods annotated with `@EventListener`, and registers them as event listeners. When an event is published, Spring's event mechanism can create an event object through reflection, find the corresponding listener based on the event type, and use dynamic proxy technology to call the listener's handling method. In addition, Spring's event mechanism also supports asynchronous event processing. By adding the `@Async` annotation to the listener method, the event processing logic can be placed in a separate thread to improve response speed and concurrency capabilities.

[0086] A synchronization event can be understood as an event that indicates the synchronization of the first database and the second database; more specifically, it can be understood as an event that indicates the synchronization of the first database to the second database.

[0087] In this embodiment of the invention, a data persistence layer interceptor can be used to create and publish synchronous events based on the list object using the Spring event mechanism. For example, a data persistence layer interceptor can be used to create synchronous events based on the list object using the Spring event mechanism and call the publishTransList() method to publish the synchronous events.

[0088] In this embodiment of the invention, after the synchronous event is published, the data persistence layer interceptor can close the database connection and clean up the occupied resources.

[0089] An event listener component can be understood as one that can listen for synchronous events. Since a transaction event listener can be configured with TransactionPhase.AFTER_COMMIT to specify that the first application should be executed after the target transaction is committed, ensuring that all statements within the same target transaction can be intercepted, an event listener component can be, for example, a transaction event listener, to ensure that all statements within the same target transaction can be intercepted, thereby ensuring that the written data of the first database and the second database are consistent.

[0090] In this embodiment of the invention, the event listening component can continuously listen to the published synchronous events. When a synchronous event is detected by the event listening component, the encapsulated list object (or synchronous data) corresponding to the synchronous event is converted into a first string.

[0091] For example, participate Figure 3 Custom synchronous events can be created and published based on the list object using Spring's event mechanism, through the data persistence layer interceptor in the first application that is the source application; a custom event listener component can listen for events and convert the list object into a first string when a synchronous event is detected.

[0092] In this embodiment of the invention, a data persistence layer interceptor creates and publishes a synchronous event based on the list object using the Spring event mechanism. Upon detecting the synchronous event through the event listener component, the list object is converted into a first string. This technical solution achieves loosely coupled communication between synchronous event publishing and listening.

[0093] Based on the above solution, another optional technical solution for obtaining the first statement and the first parameter includes: intercepting the first statement and the first parameter through a data persistence layer interceptor to obtain the first statement and the first parameter.

[0094] In this embodiment of the invention, the first statement and the first parameter can be intercepted by a data persistence layer interceptor. For example, the first statement and the first parameter can be intercepted by the data persistence layer interceptor to intercept the methods of the Executor interface that modify data, object properties or system state (such as the update method). (The intercepted first statement can be a data update statement.)

[0095] In this embodiment of the invention, the data persistence layer connector can continue the interception of the first statement and the first parameter after the return value of the original method is returned after the synchronization event is published.

[0096] In this embodiment of the invention, the first statement and the first parameter can be intercepted using a data persistence layer interceptor. The above technical solution enables the data persistence layer interceptor to obtain the first statement and the first parameter.

[0097] To better understand the technical solutions of the above embodiments of the present invention, an optional example is provided herein. For example, see... Figure 4 In the first application (producer), the data persistence layer interceptor can retrieve the first statement and its first parameters for execution in the first database (source database) to perform the first data operation. It then encapsulates these parameters into a list object, creates a synchronization event based on the list object, and publishes the event. When the event listener in the first application detects the synchronization event, it converts the list object into a first string and pushes it to the Redis message queue (the first message queue) (e.g., via the LPUSH command). In the second application (consumer), the message listener retrieves the first string from the Redis message queue (e.g., via the RPOP command) and performs the first data operation in the second database (the target database) based on the first string. The process of encapsulating the first statement and parameters into a list object corresponding to the target transaction, creating and publishing the synchronization event, can be found in [reference needed]. Figure 5 First, determine the statement type of the first statement. If the first statement is a data update statement, convert any extra whitespace characters in the first statement into single spaces. Then, perform parameter conversion on the first parameter according to the parameter type. Finally, encapsulate the first statement and the first parameter into a list object. If the first statement is a resource release statement, create a synchronization event based on the list object, publish the synchronization event, close the database connection, and clear the cache.

[0098] Figure 6This is a flowchart of another database synchronization method provided in this embodiment of the invention. This embodiment is applicable to database synchronization situations. The method can be executed by the database synchronization device provided in this embodiment of the invention. This device can be implemented in software and / or hardware, and can be integrated into an electronic device, which can be various user terminals or servers.

[0099] See Figure 6 The method of this embodiment of the invention is applied to a second end, which is associated with a second database. The method of this embodiment of the invention specifically includes the following steps:

[0100] S310. When the first string is retrieved from the first message queue, a first data operation is performed in the second database based on the first string. The first string is determined by the first end based on the first statement to be executed in the first database to perform the first data operation and the first parameter of the first statement, and is pushed to the first message queue. The first end is associated with the first database.

[0101] In this embodiment of the invention, the second end can continuously listen to the first message queue to perform string retrieval operations on the first message queue. For example, the second end can continuously listen to the first message queue to use the jedisCluster.rpop() method to retrieve a specified string from the first message queue. If the retrieved string is empty or undefined, the loop operation of retrieving the string continues (the blocking time can be customized according to the situation).

[0102] In this embodiment of the invention, the second end can retrieve the first string from the first message queue using an asynchronous method.

[0103] In this embodiment of the invention, when a first string is retrieved from a first message queue, a first data operation is performed in a second database based on the first string. For example, the first string can be parsed into an ordered list storing multiple JSON objects. <jsonobject>), and for List <jsonobject>Iterate through the List in order. <jsonobject>Each JSONObject is transformed to obtain the third statement corresponding to the first statement and the third parameter corresponding to the first parameter. The third statement and the third parameter are then executed in the second database (this can be achieved through methods such as jdbcTemplate.update()). Detailed log information can be recorded before and after execution for easy querying.

[0104] It is understood that the first end of the embodiments of the present invention can be used to execute the database synchronization method of any of the above embodiments.

[0105] The technical solution of this invention is applied to a second end, which is associated with a second database. When a first string is retrieved from a first message queue, a first data operation is performed in the second database based on the first string. The first string is determined by the first end based on a first statement to be executed in the first database for the first data operation and a first parameter of the first statement, and is pushed to the first message queue. The first end is associated with the first database. This technical solution, by pushing the first string to the first message queue as a buffer, avoids the problem of excessive database load caused by directly performing first data operations on the second database based on the first string. In particular, it avoids the problem of excessive database load caused by directly performing first data operations on a large number of first strings in a short period of time, thereby reducing the database load.

[0106] An optional technical solution, the database synchronization method, further includes: for a second statement to be executed in a second database to perform a second data operation and a second parameter of the second statement, obtaining the second statement and the second parameter, and determining a second string based on the second statement and the second parameter; pushing the second string to a second message queue, so that if the first end retrieves the second string from the second message queue, the second data operation is performed in the first database based on the second string.

[0107] The second data operation can be understood as an operation performed in the second database that corresponds to the second statement and the second parameter; the second data operation can be, for example, at least one of adding data, searching for data, updating data, and deleting data.

[0108] The second statement can be understood as a statement executed in the second database to perform second data operations.

[0109] The second parameter can be understood as the parameter executed in the second database to perform the second data operation.

[0110] The second string can be understood as the string determined by the second statement and the second parameter; the second string can be, for example, a JSON string.

[0111] The second message queue can be understood as the message queue to which the second string is pushed.

[0112] In this embodiment of the invention, the first message queue and the second message queue can be different message queues (using two message queues can ensure that after the dual-write direction is reversed, the messages in the original queue can continue to be listened to and processed, ensuring that each statement is not lost and can be executed smoothly). The first message queue and the second message queue can each correspond to a key. Different parameters can be configured for the message queues in the called jedisCluster.lpsuh() and jedisCluster.rpop() methods to specify the corresponding message queue. Combined with business traffic switching, the purpose of switching the sender and receiver and controlling the dual-write data flow can be achieved. That is, by switching the system business traffic in conjunction with message queues with different keys, the dual-write direction of the database (whether the first database is synchronizing to the second database or the second database is synchronizing to the first database) can be flexibly controlled, thereby achieving the purpose of bidirectional dual-write.

[0113] It should be noted that the process of synchronizing the second database to the first database described above may be the same as or different from the process of synchronizing the first database to the second database mentioned in any of the above embodiments, thereby realizing dual database writing.

[0114] In this embodiment of the invention, a second statement and a second parameter can be obtained for a second statement to be executed in the second database to perform a second data operation. A second string is then determined based on the second statement and the second parameter, and pushed to a second message queue. When the first end retrieves the second string from the second message queue, the second data operation is performed in the first database based on the second string. Compared to solutions with limited application scenarios that only allow forward database synchronization, where the synchronization direction between the first and second databases cannot be switched, and even after traffic switching, reverse synchronization is impossible, resulting in data generated after migration not being synchronized to the old application, and in case of anomalies during parallel migration, the inconsistency between the old and new databases prevents a switchback to the old application, the above technical solution, combined with data synchronization from the first database to the second database, enables dual-write between databases. This ensures that data generated after migration can be synchronized to the old application, and in case of anomalies during parallel migration, the consistency between the old and new databases allows a switchback to the old application.

[0115] Another optional technical solution, the database synchronization method, further includes: obtaining the business processing volume associated with the first database, and adjusting the number of threads used for database synchronization according to the business processing volume; and using the threads under the specified number of threads to process the first message queue.

[0116] Here, business processing volume can be understood as the number of business processes (business volume or traffic) associated with the first database.

[0117] The number of threads can be understood as the number of threads used for database synchronization.

[0118] In this embodiment of the invention, the number of threads can be adjusted according to the acquired business processing volume. For example, the number of threads can be increased when the business processing volume is greater than a preset threshold; or the number of threads can be adjusted to the number corresponding to the range in which the business processing volume is located; and so on.

[0119] A thread can be understood as a thread used for database synchronization.

[0120] In this embodiment of the invention, the first message queue can be processed using threads within the specified number of threads.

[0121] In this embodiment of the invention, the business processing volume associated with the second database can also be obtained, and the number of threads can be adjusted according to the business processing volume associated with the second database. For example, the number of threads can be reduced when the business processing volume is greater than a preset threshold; or the number of threads can be adjusted to the number corresponding to the range in which the business processing volume associated with the second database is located.

[0122] In this embodiment of the invention, the business processing volume is obtained, and the number of threads used for database synchronization is adjusted according to the business processing volume. Then, the threads under the given number of threads are used to process the first message queue. The above technical solution can ensure that the number of threads is sufficient for database synchronization.

[0123] To better understand the technical solutions of the above embodiments of the present invention, an optional example is provided herein. For example, see... Figure 7 The application connected to database DB-1 (the first database) is deployed on an on-premises platform as the first application (on-premises application), and the application connected to database DB-2 (the second database) is deployed on a cloud platform as the second application (cloud application). When the requirement is to synchronize database DB-1 to database DB-2, the first application is the source application (dual-write data producer), and the second application is the target application (dual-write data consumer). When the `jedisCluster.lpsuh()` and `jedisCluster.rpop()` methods are called with "db1tpdb2" as the key of the first message queue, the first application will intercept the first statement and first parameter through the data persistence layer interceptor. Based on the first statement and first parameter, and using the Spring event mechanism, it will determine the first string and then push (send) the first string to the Redis message queue with the specified key, thus allowing the first string to flow from the first application to the second application. The second application continuously listens to and consumes the first message queue. Synchronize to the connected DB-2 database; when the requirement is to write both DB-2 and DB-1 databases, the second application is the source application (dual-write data producer), and the first application is the target application (dual-write data consumer). When calling the jedisCluster.lpsuh() and jedisCluster.rpop() methods and specifying "db2todb1" as the key of the second message queue, the second application will intercept the second statement and the second parameter through the data persistence layer interceptor. Based on the second statement and the second parameter, it will use the Spring event mechanism to determine the second string and then push (send) the second string to the Redis message queue with the specified key as the second message queue, so that the second string flows from the second application to the first application. The first application continuously listens to and consumes the second message queue and synchronizes it to the connected DB-1 database; when there is no dual-write operation (i.e., no statement and parameters are sent to the message queue), the original method return value can be returned directly.

[0124] The above technical solution combines readily available data persistence layer interceptors, Spring event mechanisms, and Redis message queues to quickly and easily embed existing logic code, greatly reducing the learning and maintenance costs of introducing new components. By specifying message queues with different primary keys and combining them with business traffic switching for bidirectional dual-write, it achieves minimal intrusion into existing business logic code, almost no modification to the existing code structure, no need to introduce new dependencies, and only requires minimal configuration and simple code embedding to integrate lightweight database dual-write, effectively reducing development costs and risks. It also allows for adjustment of the dual-write direction to adapt to more scenarios of switching business traffic, effectively addressing data synchronization challenges during database migration, and can be applied to any heterogeneous or homogeneous database. In the migration process, it adapts to business scenarios of different types of database migration, supports cross-database or cross-platform data synchronization, and can realize real-time replication, asynchronous replication and batch replication of data. It can synchronize incremental data during the parallel operation of the old and new systems after migration, and ensure the consistency of data in the old and new databases (the first database and the second database). This allows for rapid application rollback in abnormal situations, thus fully ensuring business continuity and system stability. It focuses on solving the problem of being unable to rollback or experiencing business loss due to data inconsistency between the two databases in abnormal situations after system migration or database switching. Moreover, the above solution has high flexibility and scalability, few restrictions, strong universality and controllable direction, and can also avoid data conflicts and increased traffic and synchronization delays caused by multi-source writes.

[0125] Figure 8 This is a structural block diagram of a database synchronization system provided in an embodiment of the present invention. This embodiment is applicable to database synchronization scenarios.

[0126] See Figure 8 The application database synchronization system of this invention includes: a first terminal 410 and a second terminal 420; wherein,

[0127] The first terminal 410 is used to execute the database synchronization method provided in any embodiment of the present invention, and the second terminal 420 is used to execute the database synchronization method provided in any embodiment of the present invention.

[0128] The technical solution of this invention includes a first end and a second end. The first end is used to execute the database synchronization method provided in any embodiment of this invention, and the second end is used to execute the database synchronization method provided in any embodiment of this invention. The above technical solution pushes the first string to a first message queue, which serves as a buffer. This avoids the problem of excessive database load caused by directly performing first data operations on the second database based on the first string, especially avoiding the problem of excessive database load caused by directly performing first data operations on a large number of first strings on the second database in a short period of time, thereby reducing the database load.

[0129] Figure 9 This is a structural block diagram of a database synchronization device provided in an embodiment of the present invention. This device is used to execute the database synchronization method provided in any of the above embodiments. This device and the database synchronization methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the database synchronization device can be found in the embodiments of the above database synchronization methods. See also... Figure 9 The device is configured at the first end, which is associated with the first database. Specifically, the device may include: a first string determination module 510 and a first data operation module 520.

[0130] The first string determination module 510 is used to obtain the first statement and the first parameter for the first statement to be executed in the first database to perform the first data operation, and to determine the first string based on the first statement and the first parameter.

[0131] The first data operation module 520 is used to push the first string to the first message queue so that when the second end retrieves the first string from the first message queue, it can perform the first data operation in the second database based on the first string, wherein the second end is associated with the second database.

[0132] Optionally, the first statement is a statement within the target transaction;

[0133] The first string determination module 510 may include:

[0134] The first return execution submodule is used to encapsulate the first statement and the first parameter into a list object corresponding to the target transaction according to the execution order of the first statement under the target transaction, and return to the execution steps of obtaining the first statement and the first parameter if the first statement is a statement other than a resource release statement;

[0135] The first string determination submodule is used to determine the first string based on the list object when the first statement is a resource release statement.

[0136] Optionally, based on the above-described apparatus, the first return execution submodule may include:

[0137] The statement parameter encapsulation unit is used to encapsulate the first statement and the first parameter into a list object corresponding to the target transaction, according to the execution order of the first statement under the target transaction, when the first statement is a data update statement.

[0138] The first string determination module 510 may further include:

[0139] The second return execution submodule is used to return to the execution steps of retrieving the first statement and the first parameter if the first statement is a data lookup statement.

[0140] Optionally, based on the above-described apparatus, the first string determining submodule may include:

[0141] The Synchronous Event Publishing Unit is used to create and publish synchronous events based on the list object using Spring's event mechanism, through the data persistence layer interceptor.

[0142] The first string conversion unit is used to convert the list object into the first string when a synchronous event is detected by the event listener component.

[0143] Optionally, based on the above-described apparatus, the first string determination module 510 may include:

[0144] The first parameter acquisition submodule is used to intercept the first statement and the first parameter through the data persistence layer interceptor in order to obtain the first statement and the first parameter.

[0145] Optionally, the device may also include:

[0146] The first parameter update module is used to perform parameter conversion on the first parameter according to the parameter type corresponding to the first parameter in the second database before determining the first string based on the first statement and the first parameter, and update the first parameter based on the conversion result.

[0147] The database synchronization device provided in this embodiment of the invention is configured at a first end, which is associated with a first database. Through a first string determination module, a first statement and first parameters to be executed in the first database for a first data operation are obtained. Based on the first statement and first parameters, a first string is determined, which is then used to perform the first data operation in a second database. The first string is then pushed to a first message queue by a first first data operation module. When the second end retrieves the first string from the first message queue, the first data operation is performed in the second database based on the first string, thereby achieving data synchronization between the first and second databases. The second end is associated with the second database. By pushing the first string to the first message queue as a buffer, the device avoids the problem of excessive database load caused by directly performing first data operations in the second database based on the first string. In particular, it avoids the problem of excessive database load caused by directly performing first data operations in the second database based on a large number of first strings in a short period of time, thus reducing the database load.

[0148] The database synchronization device provided in this embodiment of the invention can execute the database synchronization method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0149] Figure 10 This is a structural block diagram of another database synchronization device provided in an embodiment of the present invention. This device is used to execute the database synchronization method provided in any of the above embodiments. This device and the database synchronization methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the database synchronization device can be found in the embodiments of the above database synchronization methods. See also Figure 10 The device is configured at the second end, which is associated with the second database. Specifically, the device may include: a second first data operation module 610.

[0150] The second first data operation module 610 is used to perform a first data operation in the second database based on the first string when the first string is retrieved from the first message queue.

[0151] The first string is determined by the first end based on the first statement to be executed in the first database to perform the first data operation and the first parameter of the first statement, and is pushed to the first message queue. The first end is associated with the first database.

[0152] Optionally, the device may also include:

[0153] The second string determination module is used to obtain the second statement and the second parameter of the second statement, which need to be executed in the second database to perform the second data operation, and to determine the second string based on the second statement and the second parameter.

[0154] The second data operation module is used to push the second string to the second message queue so that, when the first end retrieves the second string from the second message queue, it can perform the second data operation in the first database based on the second string.

[0155] Optionally, the device may also include:

[0156] The thread count adjustment module is used to obtain the business processing volume associated with the first database and adjust the number of threads used for database synchronization according to the business processing volume.

[0157] The first message queue processing module is used to process the first message queue using threads within a specified number of threads. The database synchronization device provided in this embodiment is configured at a second end, which is associated with a second database. Through a second first data operation module, when a first string is retrieved from the first message queue, a first data operation is performed in the second database based on the first string. The first string is determined by the first end based on a first statement to be executed in the first database for the first data operation and a first parameter of the first statement, and is pushed to the first message queue. The first end is associated with the first database. This device, by pushing the first string to the first message queue as a buffer, avoids the problem of excessive database load caused by directly performing first data operations on the second database based on the first string. In particular, it avoids the problem of excessive database load caused by directly performing first data operations on a large number of first strings in a short period of time, thereby reducing the database load.

[0158] The database synchronization device provided in this embodiment of the invention can execute the database synchronization method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0159] It is worth noting that in the above-described embodiments of the database synchronization device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0160] Figure 11 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0161] like Figure 11 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0162] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0163] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as database synchronization methods.

[0164] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0165] In some embodiments, the database synchronization method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the database synchronization method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the database synchronization method by any other suitable means (e.g., by means of firmware).

[0166] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0168] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0169] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0170] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0171] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0172] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0173] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.< / jsonobject> < / jsonobject> < / jsonobject>

Claims

1. A database synchronization method, characterized in that, Applied to a first end, which is associated with a first database, the method includes: For a first statement that needs to be executed in the first database to perform a first data operation and a first parameter of the first statement, obtain the first statement and the first parameter, and determine a first string based on the first statement and the first parameter; The first string is pushed to the first message queue so that, if the second end retrieves the first string from the first message queue, the first data operation is performed in the second database based on the first string, wherein the second end is associated with the second database.

2. The method according to claim 1, characterized in that, The first statement is a statement within the target transaction; Determining the first string based on the first statement and the first parameter includes: According to the execution order of the first statement under the target transaction, the first statement and the first parameter are encapsulated into a list object corresponding to the target transaction, and if the first statement is a statement other than a resource release statement, the step of obtaining the first statement and the first parameter is returned. If the first statement is a resource release statement, the first string is determined based on the list object.

3. The method according to claim 2, characterized in that, The step of encapsulating the first statement and the first parameter into a list object corresponding to the target transaction according to the execution order of the first statement under the target transaction includes: If the first statement is a data update statement, the first statement and the first parameter are encapsulated into a list object corresponding to the target transaction according to the execution order of the first statement under the target transaction. The step of determining the first string based on the first statement and the first parameter further includes: If the first statement is a data lookup statement, return to the step of obtaining the first statement and the first parameter.

4. The method according to claim 2, characterized in that, Determining the first string based on the list object includes: Using the data persistence layer interceptor, based on the list object, a synchronous event is created and published using the Spring event mechanism; If the synchronous event is detected by the event listener component, the list object is converted into the first string.

5. The method according to claim 4, characterized in that, The step of obtaining the first statement and the first parameter includes: The first statement and the first parameter are intercepted by the data persistence layer interceptor to obtain the first statement and the first parameter.

6. The method according to claim 1, characterized in that, Before determining the first string based on the first statement and the first parameter, the method further includes: Based on the parameter type corresponding to the first parameter in the second database, the first parameter is converted, and the first parameter is updated based on the conversion result.

7. A database synchronization method, characterized in that, Applied to a second end, which is associated with a second database, the method includes: If a first string is retrieved from the first message queue, a first data operation is performed in the second database based on the first string. The first string is determined by the first end based on the first statement to be executed in the first database to perform the first data operation and the first parameter of the first statement, and is pushed to the first message queue. The first end is associated with the first database.

8. The method according to claim 7, characterized in that, Also includes: For a second statement that needs to be executed in the second database to perform a second data operation and a second parameter of the second statement, obtain the second statement and the second parameter, and determine the second string based on the second statement and the second parameter; The second string is pushed to the second message queue so that, if the first end retrieves the second string from the second message queue, the second data operation is performed in the first database based on the second string.

9. The method according to claim 7, characterized in that, Also includes: Obtain the business processing volume associated with the first database, and adjust the number of threads used for database synchronization based on the business processing volume; The first message queue is processed using the number of threads specified.

10. A database synchronization system, characterized in that, include: The first end and the second end; among which, The first end is used to execute the database synchronization method as described in any one of claims 1-6, and the second end is used to execute the database synchronization method as described in any one of claims 7-9.

Images