A data synchronization method, device, apparatus, and storage medium

Abstract

The application discloses a data synchronization method and device, equipment and a storage medium. The method comprises the following steps: when a target timer in the archiving configuration file triggers, a connection between a subscription master library and a master library is established according to master library connection information in the archiving configuration file; a target LSN value is obtained, a synchronization request instruction is generated according to the target LSN value, and the synchronization request instruction is sent to the subscription master library; the target archiving log file corresponding to the target LSN value is received and sent by the subscription master library; and data synchronization is performed according to the subscription mode in the archiving configuration file and the target archiving log file. Through the technical scheme, the problem that the log replay speed of the backup library cannot catch up with the log generation speed of the master library due to too many archiving logs generated by the master library, and further leading to abnormality of the master-slave system, is solved, the independence of the subscription backup library and the flexibility of the subscription master library configuration are improved, and the subscription master library can continuously provide services externally.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, and particularly relate to a data synchronization method, device, equipment and storage medium. BACKGROUND

[0002] In order to ensure the security of data and prevent accidents, a database system is usually configured as a master-slave system. The user performs daily operations on the master database. After the master database modifies data, the modification is recorded in an archive log file, and then the archive log file is sent to the slave database. The slave database synchronously modifies data according to the archive log (this process is called log replay), thereby ensuring the consistency of data on the master database and the slave database. The above process is called data synchronization between the master database and the slave database.

[0003] An asynchronous master-slave system is a commonly used non-real-time master-slave system. In the asynchronous master-slave system, a timer, an archive configuration file, and a communication module configuration file need to be configured on the master database. An archive configuration file and a communication module configuration file also need to be configured on the slave database. In the asynchronous master-slave system, the master database maintains and initiates archive log synchronization. The master database sends archive logs to the slave database at regular intervals, and the slave database immediately performs log replay after receiving the archive logs. The communication module is used to establish a connection between the master database and the slave database and to perform message communication.

[0004] For the asynchronous master-slave system, if the master database generates a large number of archive logs, the log replay speed of the slave database may not catch up with the log generation speed of the master database, thereby causing the master-slave system to be abnormal. In addition, the master database needs to provide services continuously, but the configuration of the master-slave system requires the configuration of the archive log and the communication module on the master database, so that the master database cannot provide services continuously. SUMMARY

[0005] Embodiments of the present application provide a data synchronization method, device, equipment and storage medium. In the embodiments of the present application, the archive configuration file is configured on the subscription slave database, and the subscription master database and the subscription slave database do not rely on the communication module for communication, thereby solving the problem that the log replay speed of the slave database cannot catch up with the log generation speed of the master database due to a large number of archive logs generated by the master database, and further causing the master-slave system to be abnormal. The subscription slave database can improve the independence and the flexibility of the subscription master database configuration, and the subscription master database can provide services continuously.

[0006] According to an aspect of the present application, a data synchronization method is provided, which is executed by a subscription slave database, and the method comprises:

[0007] When a target timer in the archive configuration file triggers, a connection between the subscription master database and the subscription slave database is established according to master database connection information in the archive configuration file;

[0008] acquire a target LSN value, generate a synchronization request instruction according to the target LSN value, and send the synchronization request instruction to a subscription primary library;

[0009] receive a target archive log file corresponding to the target LSN value sent by the subscription primary library;

[0010] perform data synchronization according to a subscription mode in the archive configuration file and the target archive log file.

[0011] According to another aspect of the present application, a data synchronization method is provided, which is performed by a subscription primary library, and the method comprises:

[0012] receive a synchronization request instruction sent by a subscription secondary library, wherein the synchronization request instruction carries a target LSN value;

[0013] acquire a target archive log file corresponding to the target LSN value;

[0014] send the target archive log file corresponding to the target LSN value to the subscription secondary library, so that the subscription secondary library performs data synchronization according to a subscription mode in the archive configuration file and the target archive log file.

[0015] According to another aspect of the present application, a data synchronization device is provided, which comprises:

[0016] a connection establishment module configured to establish a connection with a subscription primary library according to primary library connection information in an archive configuration file when a target timer in the archive configuration file triggers;

[0017] a synchronization request instruction sending module configured to acquire a target LSN value, generate a synchronization request instruction according to the target LSN value, and send the synchronization request instruction to a subscription primary library;

[0018] an archive log file receiving module configured to receive a target archive log file corresponding to the target LSN value sent by the subscription primary library;

[0019] a data synchronization module configured to perform data synchronization according to a subscription mode in the archive configuration file and the target archive log file.

[0020] According to another aspect of the present application, a data synchronization device is provided, which comprises:

[0021] a synchronization request instruction receiving module configured to receive a synchronization request instruction sent by a subscription secondary library, wherein the synchronization request instruction carries a target LSN value;

[0022] a target archive log file acquiring module configured to acquire a target archive log file corresponding to the target LSN value;

[0023] a target archive log file sending module configured to send a target archive log file corresponding to the target LSN value to the subscription standby database, so that the subscription standby database performs data synchronization according to a subscription mode in the archive configuration file and the target archive log file.

[0024] According to another aspect of the present application, an electronic device is provided, which comprises:

[0025] at least one processor; and

[0026] a memory connected to the at least one processor in communication; wherein

[0027] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the data synchronization method according to any one of the embodiments of the present application.

[0028] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to perform the data synchronization method according to any one of the embodiments of the present application when executed by the processor.

[0029] The embodiments of the present application can establish a connection between the subscription primary database and the subscription standby database according to the primary database connection information in the archive configuration file when a target timer in the archive configuration file triggers; obtain a target LSN value, generate a synchronization request instruction according to the target LSN value, and send the synchronization request instruction to the subscription primary database; receive a target archive log file corresponding to the target LSN value sent by the subscription primary database; and perform data synchronization according to a subscription mode in the archive configuration file and the target archive log file, thereby solving the problem that the log replay speed of the standby database cannot catch up with the log generation speed of the primary database due to too many archive logs generated by the primary database, and further causing abnormality of the primary-standby system, improving the independence of the subscription standby database and the flexibility of the subscription primary database configuration, and enabling the subscription primary database to continuously provide services to the outside.

[0030] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0032] Figure 1 is a flow chart of a data synchronization method in an embodiment of the present application;

[0033] Figure 2 is a flow chart of another data synchronization method in an embodiment of the present application;

[0034] Figure 3 is a structural schematic diagram of a data synchronization device in an embodiment of the present application;

[0035] Figure 4 is a structural schematic diagram of another data synchronization device in an embodiment of the present application;

[0036] Figure 5 is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the person skilled in the art better understand the present application, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of the present application.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0039] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the use range, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained according to relevant laws and regulations.

[0040] Embodiment one

[0041] Figure 1 A flowchart of a data synchronization method provided by the embodiment of the present application, the embodiment can be applicable to the case of data synchronization, and the method can be executed by a data synchronization device configured in a subscription standby library in the embodiment of the present application. The device can be realized in the form of software and / or hardware. As shown in the figure, the method specifically includes the following steps: Figure 1

[0042] S110, when the target timer in the archiving configuration file triggers, a connection between the subscription standby library and the subscription master library is established according to the master library connection information in the archiving configuration file.

[0043] It should be noted that the subscription master-slave system only needs to be configured on the subscription standby library. The user needs to configure the archiving configuration file on the subscription standby library. In the archiving configuration file, the following information can be set: timer name (corresponding to the name in the timer configuration file), subscription mode, archiving directory (i.e. the storage path of the archiving log file), and master library connection information. The archiving configuration file can also set: archiving type (for example, in the embodiment of the present application, the archiving type is specified as subscription archiving). Many other extended attributes can also be configured in the archiving configuration file, which will not be described here. The user can also configure the timer configuration file on the subscription standby library, and the following information can be set in the timer configuration file: timer name, timer start time, timer end time, and interval time of each trigger.

[0044] The master library connection information can include: IP address, port number, username, and password of the subscription master library. The master library connection information can also be set as a service name or a wallet file address, etc. The subscription standby library can directly connect the subscription master library through the master library connection information, without relying on the communication module to establish the connection between the subscription master library and the subscription standby library.

[0045] ​Specifically, when the target timer in the archiving configuration file triggers, the way of establishing the connection between the subscription master and the subscription slave according to the master connection information in the archiving configuration file can be: querying the timer configuration file to obtain the trigger condition corresponding to the target timer in the archiving configuration file, and when the target timer meets the trigger condition and is triggered, establishing the connection between the subscription master and the subscription slave according to the master connection information in the archiving configuration file. When the target timer in the archiving configuration file triggers, the way of establishing the connection between the subscription master and the subscription slave according to the master connection information in the archiving configuration file can also be: obtaining the target timer name in the archiving configuration file; querying the timer configuration file according to the target timer name in the archiving configuration file to obtain the timer start time, the timer end time and the interval time of each trigger corresponding to the target timer name in the archiving configuration file; when the target timer is determined to trigger according to the timer start time, the timer end time and the interval time of each trigger, establishing the connection between the subscription master and the subscription slave according to the master connection information in the archiving configuration file.

[0046] It should be noted that after the archiving configuration file and the timer configuration file are configured, the archiving and the timer need to be started. Specifically, the archiving can be started or stopped through the archiving switch parameter, for example, setting the parameter to 0 indicates stopping the archiving, and setting the parameter to 1 indicates starting the archiving. After starting the archiving, a subscription thread will be started on the subscription slave to perform the data synchronization task between the master and the slave. Similarly, the timer can also be started or stopped through the timer switch parameter, for example, setting the parameter to 0 indicates stopping the timer, and setting the parameter to 1 indicates starting the timer. After starting the timer, a timer thread will be started on the subscription slave, which controls the timing trigger of the timer.

[0047] In a specific example, the INI parameter file, the timer configuration file and the archiving configuration file are pre-configured in the subscription slave, the INI parameter file, the timer configuration file and the archiving configuration file are read, and if the INI parameter file includes the start archiving parameter and the start timer parameter, when the target timer in the archiving configuration file triggers, the connection between the subscription master and the subscription slave is established according to the master connection information in the archiving configuration file. For example, if the INI parameter file includes (1, 1), it is determined that the archiving is started and the timer is started, and when the target timer in the archiving configuration file triggers, the connection between the subscription master and the subscription slave is established according to the master connection information in the archiving configuration file.

[0048] It should be noted that the subscription master library can be a single-node database or a database cluster (i.e., a database system composed of multiple database nodes, multiple nodes can access and operate the same database at the same time, and the database can normally provide services to the outside when any node fails). If the subscription master library is a database cluster, the connection information of all or part of the nodes in the cluster can be specified when the master library connection information is specified, and the subscription standby library connection subscription master library will automatically select an effective node for connection. Among them, the way to select an effective node can be: obtaining the state information of all or part of the nodes in the cluster, selecting a target node according to the state information of all or part of the nodes in the cluster, and the target node is an effective node.

[0049] S120, obtain the target LSN value, generate a synchronization request instruction according to the target LSN value, and send the synchronization request instruction to the subscription master library.

[0050] Among them, LSN (Log Sequence Number) is an integer value automatically maintained by the database system, which is globally increasing, and each LSN value represents a physical transaction generated inside the database system. Each log in the archive log file corresponds to a physical transaction, that is, an LSN value.

[0051] Among them, the synchronization request instruction carries the target LSN value, and the synchronization request instruction can also carry the identification information of the subscription standby library.

[0052] Specifically, the way to obtain the target LSN value and generate a synchronization request instruction according to the target LSN value and send the synchronization request instruction to the subscription master library can be: the subscription standby library obtains the maximum LSN value corresponding to the archive directory in the archive configuration file, and determines the maximum LSN value corresponding to the archive directory as the target LSN value. The subscription standby library generates a synchronization request instruction according to the target LSN value and sends the synchronization request instruction to the subscription master library. After receiving the synchronization request instruction, the subscription master library obtains the target archive log file according to the target LSN value carried by the synchronization request instruction.

[0053] S130, receiving the target archive log file corresponding to the target LSN value sent by the subscription master library.

[0054] Specifically, after the subscription master library receives the synchronization request instruction sent by the subscription standby library, the subscription master library queries the target archive log file according to the target LSN value carried by the synchronization request instruction, and sends the target archive log file to the subscription standby library corresponding to the identification information carried by the synchronization request instruction. The subscription standby library receives the target archive log file corresponding to the target LSN value sent by the subscription master library.

[0055] S140, synchronizing data according to the subscription mode in the archiving configuration file and the target archiving log file.

[0056] The subscription mode can include an archiving mode, a replay mode, and an archiving and replay mode. The archiving mode is a mode of only archiving operation, the replay mode is a mode of only replay operation, and the archiving and replay mode is a mode of both archiving and replaying.

[0057] The data synchronization can refer to storing the target archiving log file into the archiving directory of the archiving configuration file, can also refer to replaying the target archiving log file, and can also refer to storing the target archiving log file into the archiving directory of the archiving configuration file and replaying the target archiving log file.

[0058] Specifically, the way of synchronizing data according to the subscription mode in the archiving configuration file and the target archiving log file can be that, when in the archiving mode, if the target archiving log file passes the verification, the target archiving log file is stored into the archiving directory of the archiving configuration file. The way of synchronizing data according to the subscription mode in the archiving configuration file and the target archiving log file can also be that, when in the replay mode, if the target archiving log file passes the verification, the target archiving log file is replayed. The way of synchronizing data according to the subscription mode in the archiving configuration file and the target archiving log file can also be that, when in the archiving and replay mode, if the target archiving log file passes the verification, the target archiving log file is stored into the archiving directory of the archiving configuration file and is replayed.

[0059] In a specific example, the data synchronization process is as follows: when the target timer in the archiving configuration file triggers, the subscription thread of the subscription standby initiates a new round of data synchronization; the subscription standby parses the archiving configuration file to obtain the master connection information, first attempts to connect each node in turn according to the master connection information, selects any one valid node (non-fault node) to establish a final connection for data synchronization, and directly calls the database driver interface (the database driver is used for connecting and operating the database, and the commonly used ones include DPI driver, JDBC driver, etc.) when establishing the connection. After the connection is established, the subscription standby and the subscription master can interact directly through the connection; the subscription standby sends a synchronization request to the subscription master node according to the maximum LSN value in the local archiving log file (the archiving log file in the archiving directory can be scanned to determine the maximum LSN value); after receiving the synchronization request, the subscription master sends the target archiving log file to the subscription standby (the request initiator); after receiving the target archiving log file, the subscription standby synchronizes data according to the subscription mode specified in the archiving configuration file and the target archiving log file.

[0060] Optionally, the data synchronization according to the subscription mode in the archiving configuration file and the target archiving log file comprises:

[0061] When in the archiving mode, a maximum LSN value corresponding to the archiving directory in the archiving configuration file is acquired;

[0062] The target archiving log file is verified according to the maximum LSN value corresponding to the archiving directory in the archiving configuration file;

[0063] If the verification is passed, the target archiving log file is stored into the archiving directory of the archiving configuration file.

[0064] Specifically, the verification of the target archiving log file according to the maximum LSN value corresponding to the archiving directory in the archiving configuration file can be in the following manner: whether the difference between the LSN value corresponding to the target archiving log file and the maximum LSN value in the archiving log file corresponding to the archiving directory in the archiving configuration file is equal to a set threshold value, if yes, it is determined that the verification is passed, if no, it is determined that the verification is not passed, wherein the set threshold value can be 1 or other numerical value, and the present embodiment does not limit this.

[0065] It should be noted that the purpose of the verification of the target archiving log file is to ensure the continuity of the LSN value.

[0066] Optionally, the method further comprises:

[0067] An replay instruction is acquired, wherein the replay instruction comprises: an LSN value interval;

[0068] All archiving log files in the LSN value interval are replayed.

[0069] The LSN value interval can be determined according to the database system process SP_A, for example, the minimum LSN value and the maximum LSN value corresponding to the LSN value interval can be determined according to the parameters of the database system process SP_A.

[0070] Specifically, the replay of all archiving log files in the LSN value interval can be in the following manner: all archiving log files in the LSN value interval are replayed in a preset order. For example, the archiving log files in the LSN value interval can be replayed in the order of the LSN value from small to large.

[0071] It should be noted that when in the archiving mode, the archiving log files can be manually replayed: the archiving log files can be replayed by a database system process, for example, a database system process SP_A can be implemented, and a user can manually replay the archiving log by specifying an LSN value range. In a specific example, SP_A (10000, 11000) is executed to immediately replay all archiving log files with LSN values in the range of 10000 to 11000.

[0072] In a specific example, when in the archiving mode, the subscription standby database saves the target archiving log file obtained to the archiving directory without replaying, and a user can manually replay the archiving log at an appropriate time. The archiving mode can effectively solve the problem that the log replay speed of the standby database cannot catch up with the log generation speed of the primary database in an asynchronous primary-standby system. In the archiving mode, after the subscription standby database obtains the target archiving log file, the target archiving log file is checked according to the maximum LSN value corresponding to the archiving directory in the archiving configuration file, thereby ensuring the continuity of the LSN value.

[0073] Optionally, the data synchronization is performed according to the subscription mode in the archiving configuration file and the target archiving log file, and the data synchronization includes:

[0074] When in the replay mode, the LSN value corresponding to the archiving log file that has been successfully replayed is obtained.

[0075] The target archiving log file is checked according to the LSN value corresponding to the archiving log file that has been successfully replayed.

[0076] If the checking passes, the target archiving log file is replayed.

[0077] It should be noted that when in the replay mode, after the subscription standby database obtains the archiving log file, the target archiving log file is directly replayed without being saved to the archiving directory.

[0078] Specifically, the checking of the target archiving log file according to the LSN value corresponding to the archiving log file that has been successfully replayed can be performed in the following manner: whether the difference between the LSN value corresponding to the target archiving log file and the LSN value corresponding to the archiving log file that has been successfully replayed is equal to a set threshold value is determined, if yes, it is determined that the checking passes, and if no, it is determined that the checking does not pass, where the set threshold value can be 1 or another numerical value, and the embodiments of the present application do not limit this.

[0079] Optionally, the data synchronization is performed according to the subscription mode in the archiving configuration file and the target archiving log file, and the data synchronization includes:

[0080] When in the archiving and replay mode, the LSN value corresponding to the archiving log file that has been successfully replayed is obtained.

[0081] checking the target archived log file according to the LSN value corresponding to the successfully replayed archived log file;

[0082] if the checking is passed, storing the target archived log file into the archived directory of the archived configuration file;

[0083] replaying the target archived log file.

[0084] It should be noted that the archiving and replaying mode: the target archived log file acquired by the subscription standby library is saved into the archived directory, and the target archived log file is replayed. The subscription standby library records the maximum LSN value of the currently replayed log (the maximum LSN value of the currently replayed log is also the LSN value corresponding to the successfully replayed archived log file) after the log is replayed each time. In the only replaying mode and the archiving and replaying mode, the subscription standby library checks the acquired archived log file according to the maximum LSN value of the replaying above, to ensure the continuity of the LSN value.

[0085] In addition to the above, the embodiment of the present application also supports setting a flow control parameter in the archived configuration file, to control the maximum amount of archived logs acquired by the subscription standby library per second, so as to reduce the influence of the data synchronization work on the performance of the subscription master library.

[0086] In addition, the subscription standby library can perform the action of acquiring the archived log file in parallel while saving the archived log file into the archived directory or replaying the archived log.

[0087] Embodiment two

[0088] Figure 2 A flow chart of a data synchronization method provided by the embodiment of the present application, the embodiment is applicable to the case of data synchronization, and the method can be executed by a data synchronization device configured in the subscription master library in the embodiment of the present application. The device can be realized in the form of software and / or hardware, as shown in the figure, and the method specifically includes the following steps: Figure 2

[0089] S210, receiving a synchronization request instruction sent by a subscription standby library, wherein the synchronization request instruction carries a target LSN value.

[0090] The LSN (Log Sequence Number) is an integer type value automatically maintained by a database system, and is globally increased. Each LSN value represents a physical transaction generated in the database system. Each log in the archived log file corresponds to a physical transaction, that is, corresponds to an LSN value.

[0091] ​The target LSN value is a maximum LSN value corresponding to an archived directory in an archived configuration file in the subscription standby.

[0092] It should be noted that the synchronization request instruction carries subscription standby identification information, so that the subscription master can send the target archived log file to the target subscription standby, wherein the target subscription standby is the subscription standby sending the synchronization request instruction. That is, in the case that there are at least two subscription standbys, the synchronization request instruction sent by the subscription standby to the subscription master needs to carry the identification information of the subscription standby sending the request, so that the subscription master can feed back the target archived log file to the subscription standby sending the request. This can improve the accuracy of data interaction between the subscription master and the subscription standby.

[0093] S220, obtaining a target archived log file corresponding to the target LSN value.

[0094] The LSN value of the target archived log file can be the same as the target LSN value, or the difference between the LSN value of the target archived log file and the target LSN value can be equal to a set threshold, which is not limited by the embodiment of the application.

[0095] It should be noted that if the LSN value of the target archived log file is the same as the target LSN value, the data processed by the subscription master can be reduced, and the processing effect of the subscription master can be improved.

[0096] S230, sending the target archived log file corresponding to the target LSN value to the subscription standby, so that the subscription standby performs data synchronization according to the subscription mode in the archived configuration file and the target archived log file.

[0097] Specifically, the subscription master library sends the target archive log file corresponding to the target LSN value to the subscription standby library, and the subscription standby library performs data synchronization according to the subscription mode in the archive configuration file and the target archive log file. Wherein, the way that the subscription standby library performs data synchronization according to the subscription mode in the archive configuration file and the target archive log file can be: when in the archive mode, obtaining the maximum LSN value corresponding to the archive directory in the archive configuration file; verifying the target archive log file according to the maximum LSN value corresponding to the archive directory in the archive configuration file; if the verification is passed, storing the target archive log file into the archive directory of the archive configuration file. When in the archive mode, the archive log can also be manually replayed, and the specific way is: obtaining a replay instruction, wherein the replay instruction includes: an LSN value interval; replaying all archive log files in the LSN value interval. The way that the subscription standby library performs data synchronization according to the subscription mode in the archive configuration file and the target archive log file can also be: when in the replay mode, obtaining the LSN value corresponding to the archive log file that the current replay is successful; verifying the target archive log file according to the LSN value corresponding to the archive log file that the current replay is successful; if the verification is passed, replaying the target archive log file. The way that the subscription standby library performs data synchronization according to the subscription mode in the archive configuration file and the target archive log file can also be: when in the archive and replay mode, obtaining the LSN value corresponding to the archive log file that the current replay is successful; verifying the target archive log file according to the LSN value corresponding to the archive log file that the current replay is successful; if the verification is passed, storing the target archive log file into the archive directory of the archive configuration file; replaying the target archive log file.

[0098] The technical scheme of the embodiment configures the archive configuration file only on the subscription standby library, and the subscription master library and the subscription standby library do not rely on the communication module for communication. The subscription master library receives a synchronization request instruction sent by the subscription standby library, wherein the synchronization request instruction carries a target LSN value; obtains a target archive log file corresponding to the target LSN value; and sends the target archive log file corresponding to the target LSN value to the subscription standby library, so that the subscription standby library performs data synchronization according to the subscription mode in the archive configuration file and the target archive log file. This can solve the problem that the log replay speed of the standby library cannot catch up with the log generation speed of the master library due to too many archive logs generated by the master library, and further cause the master-standby system to be abnormal. This can improve the independence of the subscription standby library and the flexibility of the subscription master library configuration, and ensure that the subscription master library continuously provides services to the outside.

[0099] Embodiment three

[0100] Figure 3This is a schematic diagram of a data synchronization device provided in an embodiment of the present invention. This embodiment is applicable to data synchronization situations. The device can be implemented using software and / or hardware, and can be integrated into any device that provides data synchronization functionality, such as… Figure 3 As shown, the data synchronization device specifically includes: a connection establishment module 310, a synchronization request instruction sending module 320, an archived log file receiving module 330, and a data synchronization module 340.

[0101] The connection establishment module is used to establish a connection with the subscribed master database based on the master database connection information in the archive configuration file when the target timer in the archive configuration file is triggered.

[0102] A synchronization request instruction sending module is used to obtain a target LSN value, generate a synchronization request instruction based on the target LSN value, and send the synchronization request instruction to the subscribing master database;

[0103] The archive log file receiving module is used to receive the target archive log file corresponding to the target LSN value sent by the subscribing master database;

[0104] The data synchronization module is used to synchronize data according to the subscription mode in the archive configuration file and the target archive log file.

[0105] The above-described products can perform the methods provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for performing the methods.

[0106] Example 4

[0107] Figure 4 This is a schematic diagram of a data synchronization device provided in an embodiment of the present invention. This embodiment is applicable to data synchronization situations. The device can be implemented using software and / or hardware, and can be integrated into any device that provides data synchronization functionality, such as… Figure 4 As shown, the data synchronization device specifically includes: a synchronization request instruction receiving module 410, a target archived log file acquisition module 420, and a target archived log file sending module 430.

[0108] The synchronization request instruction receiving module is used to receive a synchronization request instruction sent by the subscribed backup library, wherein the synchronization request instruction carries a target LSN value.

[0109] The target archive log file acquisition module is used to acquire the target archive log file corresponding to the target LSN value;

[0110] The target archived log file sending module is configured to send a target archived log file corresponding to the target LSN value to the subscription standby database, so that the subscription standby database performs data synchronization according to a subscription mode in an archived configuration file and the target archived log file.

[0111] The product can perform the method provided by any embodiment of the application, has the corresponding function modules and beneficial effects of performing the method.

[0112] Embodiment five

[0113] Figure 5 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the applications described and / or claimed in this document.

[0114] As shown in Figure 5 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0115] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a speaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0116] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The processor 11 performs various methods and processes described above, such as the data synchronization method.

[0117] In some embodiments, the data synchronization method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the data synchronization method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the data synchronization method by any other suitable means, such as by means of firmware.

[0118] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0119] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0120] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0121] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0122] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0123] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0124] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0125] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data synchronization method, characterized in that, This method is applied to a database, which includes a subscription master database and a subscription slave database. The subscription slave database is configured with an archive configuration file. The data synchronization method is executed by the subscription slave database and includes: When the target timer in the archive configuration file is triggered, a connection is established with the subscribed master database based on the master database connection information in the archive configuration file; Obtain the target LSN value, generate a synchronization request instruction based on the target LSN value, and send the synchronization request instruction to the subscribing master database; Receive the target archive log file corresponding to the target LSN value sent by the subscription master database; Data synchronization is performed based on the subscription mode in the archive configuration file and the target archive log file.

2. The method according to claim 1, characterized in that, Data synchronization is performed based on the subscription mode in the archive configuration file and the target archive log file, including: When in archive mode, retrieve the maximum LSN value corresponding to the archive directory in the archive configuration file; The target archived log file is verified based on the maximum LSN value corresponding to the archived directory in the archived configuration file; If the verification passes, the target archived log file will be stored in the archive directory of the archive configuration file.

3. The method according to claim 2, characterized in that, Also includes: Obtain a replay instruction, wherein the replay instruction includes: an LSN value range; Replay all archived log files within the LSN value range.

4. The method according to claim 1, characterized in that, Data synchronization is performed based on the subscription mode in the archive configuration file and the target archive log file, including: When in replay mode, obtain the LSN value corresponding to the archived log file that was successfully replayed; The target archived log file is verified based on the LSN value corresponding to the currently successfully replayed archived log file; If the verification passes, the target archived log file will be replayed.

5. The method according to claim 1, characterized in that, Data synchronization is performed based on the subscription mode in the archive configuration file and the target archive log file, including: When in archive and replay mode, obtain the LSN value corresponding to the archived log file that was successfully replayed; The target archived log file is verified based on the LSN value corresponding to the currently successfully replayed archived log file; If the verification passes, the target archived log file will be stored in the archive directory of the archive configuration file; Replay the target archived log file.

6. A data synchronization method, characterized in that, The data synchronization method, executed by the subscribing master, includes: Receive a synchronization request instruction sent by the subscribed backup database, wherein the synchronization request instruction carries a target LSN value; Obtain the target archived log file corresponding to the target LSN value; The target archived log file corresponding to the target LSN value is sent to the subscription standby database so that the subscription standby database can synchronize data according to the subscription mode in the archive configuration file and the target archived log file.

7. A data synchronization device, characterized in that, The data synchronization device includes: The connection establishment module is used to establish a connection with the subscribed master database based on the master database connection information in the archive configuration file when the target timer in the archive configuration file is triggered. A synchronization request instruction sending module is used to obtain a target LSN value, generate a synchronization request instruction based on the target LSN value, and send the synchronization request instruction to the subscribing master database; The archive log file receiving module is used to receive the target archive log file corresponding to the target LSN value sent by the subscribing master database; The data synchronization module is used to synchronize data according to the subscription mode in the archive configuration file and the target archive log file.

8. A data synchronization device, characterized in that, The data synchronization device includes: A synchronization request instruction receiving module is used to receive a synchronization request instruction sent by the subscribed backup database, wherein the synchronization request instruction carries a target LSN value; The target archive log file acquisition module is used to acquire the target archive log file corresponding to the target LSN value; The target archive log file sending module is used to send the target archive log file corresponding to the target LSN value to the subscription standby database, so that the subscription standby database can synchronize data with the target archive log file according to the subscription mode in the archive configuration file.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data synchronization method according to any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data synchronization method of any one of claims 1-6.

Images