Disaster recovery data processing method and device

Abstract

The embodiment of the application provides a kind of off-site disaster recovery data processing method and device, it is related to distributed field, method includes: multiple links parallelly read the log file of production environment database, obtain the operation record of the production environment database, and send to distributed message middleware to generate the database execution statement of off-site disaster recovery environment;The database execution statement is sequentially sent to distributed storage system according to set frequency;Get the transaction record and copy timestamp that have been completed on different databases on the same service chain, determine the consistency mirror time of next copy transaction and write into the distributed storage system, so that each application system accesses the distributed storage system and carries out data uniform update operation according to the consistency mirror time;The application can effectively reduce the risk of data loss during off-site disaster recovery account, improve the high availability during account process.

Description

Technical Field

[0001] This application relates to the field of distributed systems, specifically to a method and apparatus for off-site disaster recovery data processing. Background Technology

[0002] Intra-city disaster recovery and reconciliation on open platforms typically employs a high-availability design, while off-site disaster recovery and reconciliation generally uses disk-level replication technology to assist manual reconciliation. This has a significant impact on transactions in the source database, resulting in low replication efficiency, lack of support for heterogeneous databases, and the potential for partial data loss. The reconciliation period carries high risks and affects the business continuity of the bank's external services. Summary of the Invention

[0003] To address the problems in the existing technology, this application provides a method and apparatus for off-site disaster recovery data processing, which can effectively reduce the risk of data loss during off-site disaster recovery and improve the high availability capability during the recovery process.

[0004] To solve at least one of the above problems, this application provides the following technical solution:

[0005] Firstly, this application provides a method for off-site disaster recovery data processing, including:

[0006] Multiple links read the log files of the production environment database in parallel to obtain the operation records of the production environment database, and send them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment;

[0007] The database execution statements are sent to the distributed storage system in sequence according to a set frequency.

[0008] The system retrieves completed transaction records and replication timestamps from different databases on the same business chain, determines the consistency mirror time of the next replication transaction, and writes it into the distributed storage system. This enables each application system to access the distributed storage system and perform unified data update operations based on the consistency mirror time.

[0009] Further, after obtaining the operation records of the production environment database, the process includes:

[0010] The binary code of the operation records in the production environment database is concatenated into a target string according to a set rule;

[0011] The target string is sent to the distributed message middleware through a preset producer program.

[0012] Furthermore, the database execution statements for generating the off-site disaster recovery environment include:

[0013] The target string is obtained from the distributed message middleware through a pre-defined consumer program;

[0014] The target string in the distributed message middleware is parsed to obtain the operation records of the production environment and concatenate them to generate the database execution statement for the off-site disaster recovery environment.

[0015] Furthermore, before obtaining completed transaction records and replicating timestamps from different databases on the same business chain, the process includes:

[0016] Start the multi-database timing control node;

[0017] The playback logs of each database in the off-site disaster recovery environment are accessed through the multi-database timing control node.

[0018] Furthermore, the step of accessing the distributed storage system and performing a unified data update operation based on the consistent mirror time includes:

[0019] Periodically access the distributed storage system to obtain the database execution statements to be executed and the next consistent mirror time;

[0020] Update the data from the current time point to the next consistent mirror time to the database to perform the batch of data replenishment operations.

[0021] Further, after accessing the distributed storage system and performing a unified data update operation based on the consistent mirror time, the process includes:

[0022] Determine if there is still data in the distributed storage system that needs to be replenished;

[0023] If so, begin copying a new batch of transaction data.

[0024] Secondly, this application provides an off-site disaster recovery data processing device, comprising:

[0025] The disaster recovery statement determination module is used to read the log files of the production environment database in parallel through multiple links, obtain the operation records of the production environment database, and send them to the distributed message middleware to generate the database execution statement for the off-site disaster recovery environment.

[0026] The sequential distribution module is used to send the database execution statements to the distributed storage system in a sequential manner according to a set frequency.

[0027] The data update module is used to obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction, and write it into the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time.

[0028] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the off-site disaster recovery data processing method.

[0029] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the off-site disaster recovery data processing method.

[0030] Fifthly, this application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the off-site disaster recovery data processing method.

[0031] As can be seen from the above technical solution, this application provides a method and apparatus for off-site disaster recovery data processing. It reads log files from the production environment database in parallel through multiple links to obtain the operation records of the production environment database, and sends them to a distributed message middleware to generate database execution statements for the off-site disaster recovery environment. The database execution statements are then sent to a distributed storage system in sequence according to a set frequency. Completed transaction records and replication timestamps from different databases on the same business chain are obtained, the consistency mirror time of the next replication transaction is determined, and written to the distributed storage system. This allows each application system to access the distributed storage system and perform unified data update operations based on the consistency mirror time, effectively reducing the risk of data loss during off-site disaster recovery and improving high availability during the recovery process. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is one of the flowcharts illustrating the off-site disaster recovery data processing method in the embodiments of this application;

[0034] Figure 2 This is the second flowchart illustrating the off-site disaster recovery data processing method in the embodiments of this application;

[0035] Figure 3 This is the third flowchart illustrating the off-site disaster recovery data processing method in the embodiments of this application;

[0036] Figure 4 This is the fourth flowchart illustrating the off-site disaster recovery data processing method in the embodiments of this application;

[0037] Figure 5 This is the fifth flowchart illustrating the off-site disaster recovery data processing method in the embodiments of this application;

[0038] Figure 6 This is the sixth flowchart illustrating the off-site disaster recovery data processing method in the embodiments of this application;

[0039] Figure 7 This is a structural diagram of the off-site disaster recovery data processing device in the embodiments of this application;

[0040] Figure 8 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0043] In view of the problems existing in the prior art, this application provides a method and apparatus for off-site disaster recovery data processing. It reads the log files of the production environment database in parallel through multiple links to obtain the operation records of the production environment database, and sends them to a distributed message middleware to generate database execution statements for the off-site disaster recovery environment. The database execution statements are then sent to a distributed storage system in sequence according to a set frequency. Completed transaction records and replication timestamps on different databases on the same business chain are obtained, the consistency mirror time of the next replication transaction is determined, and written to the distributed storage system. This allows each application system to access the distributed storage system and perform unified data update operations based on the consistency mirror time, effectively reducing the risk of data loss during off-site disaster recovery and improving high availability during the recovery process.

[0044] To effectively reduce the risk of data loss during off-site disaster recovery and remediation, and to improve high availability during the remediation process, this application provides an embodiment of an off-site disaster recovery data processing method, see [link to embodiment]. Figure 1 The off-site disaster recovery data processing method specifically includes the following:

[0045] Step S101: Multiple links read the log files of the production environment database in parallel to obtain the operation records of the production environment database, and send them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment.

[0046] Optionally, this application can read the log files of various databases (such as ORACLE, MySQL, etc.) in the production environment in parallel through multiple links, extract binary code from the log files, and parse the operation records (such as UPDATE, DELETE, INSERT, etc.) of the production environment database.

[0047] Optionally, this application can concatenate the binary code of the operation record according to certain rules to generate the target string. For example, if the binary code of the data inserted in the operation record corresponds to the fifth to tenth bits in the log file, then the binary code of the fifth to tenth bits can be converted into a target string (such as a JSON string) that can be recognized by the distributed message middleware. A producer program is written to send the generated multi-link target string to the distributed message middleware. Simultaneously, a unique index value can be used as the key to ensure the sequential delivery of data to the distributed message middleware.

[0048] Step S102: Send the database execution statements to the distributed storage system in sequence according to a set frequency.

[0049] Optionally, this application can write a consumer program to obtain the target string in the distributed message middleware. Based on predefined parsing conditions, the target string in the distributed message middleware is parsed to obtain the operation records from the production environment, and these records are concatenated to generate database execution statements (e.g., UPDATE, DELETE, INSERT) for the disaster recovery environment. If, during the parsing process, the target string is found to not meet the current parsing conditions, the user can choose whether to skip the operation record through a predefined configuration file and store the record in an exception table.

[0050] After parsing the target string, the resulting execution statements are sent sequentially to a distributed storage system (such as ETCD, ZooKeeper, etc.) at certain time intervals (e.g., 200MS).

[0051] Step S103: Obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction and write it into the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time.

[0052] Optionally, this application can activate multiple database timing control nodes to access the log replay units of each database in the off-site disaster recovery environment, obtain the completed transaction records (SQL) and replication timestamps on different databases along the same business chain, and take the latest replication timestamp as the next replication transaction consistency time point. If the latest replication timestamps of each replication link are completely consistent at this time, then the current latest replication timestamp plus a pre-set time interval is taken as the next replication transaction consistency time point. After obtaining the next replication transaction consistency time point, the result is written to the distributed storage system.

[0053] Optionally, this application allows the reconciliation module of each application system to periodically access the distributed storage system to obtain the database execution statements to be executed and the next consistent mirror time, and uniformly update the data from the current time point to the next consistent mirror time into the database to complete the reconciliation of this batch.

[0054] As described above, the off-site disaster recovery data processing method provided in this application embodiment can read the log files of the production environment database in parallel through multiple links to obtain the operation records of the production environment database, and send them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment; send the database execution statements to the distributed storage system in sequence according to a set frequency; obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction and write it into the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time, which can effectively reduce the risk of data loss during off-site disaster recovery and improve the high availability capability during the recovery process.

[0055] In one embodiment of the off-site disaster recovery data processing method of this application, see [link to relevant documentation]. Figure 2 It can also specifically include the following:

[0056] Step S201: Concatenate the binary code of the operation record in the production environment database into a target string according to the set rules.

[0057] Step S202: Send the target string to the distributed message middleware through a preset producer program.

[0058] Optionally, this application can concatenate the binary code of the operation record according to certain rules to generate the target string. For example, if the binary code of the data inserted in the operation record corresponds to the fifth to tenth bits in the log file, then the binary code of the fifth to tenth bits can be converted into a target string (such as a JSON string) that can be recognized by the distributed message middleware. A producer program is written to send the generated multi-link target string to the distributed message middleware. Simultaneously, a unique index value can be used as the key to ensure the sequential delivery of data to the distributed message middleware.

[0059] In one embodiment of the off-site disaster recovery data processing method of this application, see [link to relevant documentation]. Figure 3 It can also specifically include the following:

[0060] Step S301: Obtain the target string from the distributed message middleware through the preset consumer program.

[0061] Step S302: Parse the target string in the distributed message middleware, obtain the operation records of the production environment, and concatenate them to generate the database execution statement for the off-site disaster recovery environment.

[0062] Optionally, this application can write a consumer program to obtain the target string in the distributed message middleware. Based on predefined parsing conditions, the target string in the distributed message middleware is parsed to obtain the operation records from the production environment, and these records are concatenated to generate database execution statements (e.g., UPDATE, DELETE, INSERT) for the disaster recovery environment. If, during the parsing process, the target string is found to not meet the current parsing conditions, the user can choose whether to skip the operation record through a predefined configuration file and store the record in an exception table.

[0063] In one embodiment of the off-site disaster recovery data processing method of this application, see [link to relevant documentation]. Figure 4 It can also specifically include the following:

[0064] Step S401: Start the multi-database timing control node.

[0065] Step S402: Access the playback logs of each database in the off-site disaster recovery environment through the multi-database timing control node.

[0066] Optionally, this application can activate multiple database timing control nodes to access the log replay units of each database in the off-site disaster recovery environment, obtain the completed transaction records (SQL) and replication timestamps on different databases along the same business chain, and take the latest replication timestamp as the next replication transaction consistency time point. If the latest replication timestamps of each replication link are completely consistent at this time, then the current latest replication timestamp plus a pre-set time interval is taken as the next replication transaction consistency time point. After obtaining the next replication transaction consistency time point, the result is written to the distributed storage system.

[0067] In one embodiment of the off-site disaster recovery data processing method of this application, see [link to relevant documentation]. Figure 5 It can also specifically include the following:

[0068] Step S501: Periodically access the distributed storage system to obtain the database execution statements to be executed and the next consistent mirror time.

[0069] Step S502: Update the data from the current time point to the next consistent mirror time to the database to perform the batch of data replenishment operations.

[0070] Optionally, this application can periodically access the distributed storage system to obtain the database execution statements to be executed and the next consistent mirror time, and uniformly update the data from the current time point to the next consistent mirror time into the database to complete the replenishment of this batch.

[0071] In one embodiment of the off-site disaster recovery data processing method of this application, see [link to relevant documentation]. Figure 6 It can also specifically include the following:

[0072] Step S601: Determine whether there is any data in the distributed storage system that needs to be replenished.

[0073] Step S602: If so, begin copying a new batch of transaction data.

[0074] Optionally, after completing the current batch of data replenishment, this application can determine whether there is still data to be replenished in the distributed storage system. If so, it can start replicating a new batch of transaction data. By continuously advancing the consistent mirroring of all databases, the time consistency of all databases can be effectively guaranteed in the event of an unplanned disaster.

[0075] To effectively reduce the risk of data loss during off-site disaster recovery and remediation and improve high availability during the remediation process, this application provides an embodiment of an off-site disaster recovery data processing device for implementing all or part of the aforementioned off-site disaster recovery data processing method. See [link to embodiment]. Figure 7 The off-site disaster recovery data processing device specifically includes the following components:

[0076] The disaster recovery statement determination module 10 is used to read the log files of the production environment database in parallel through multiple links, obtain the operation records of the production environment database, and send them to the distributed message middleware to generate the database execution statement for the off-site disaster recovery environment.

[0077] The sequential distribution module 20 is used to send the database execution statements to the distributed storage system in a sequential manner according to a set frequency.

[0078] The data update module 30 is used to obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction, and write it into the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time.

[0079] As described above, the off-site disaster recovery data processing device provided in this application embodiment can read the log files of the production environment database in parallel through multiple links, obtain the operation records of the production environment database, and send them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment; send the database execution statements to the distributed storage system in sequence according to a set frequency; obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction, and write it into the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time, which can effectively reduce the risk of data loss during off-site disaster recovery and improve the high availability capability during the recovery process.

[0080] From a hardware perspective, in order to effectively reduce the risk of data loss during off-site disaster recovery and restoration and improve the high availability capability during the restoration process, this application provides an embodiment of an electronic device for implementing all or part of the off-site disaster recovery data processing method. The electronic device specifically includes the following components:

[0081] The system comprises a processor, memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to realize information transmission between the off-site disaster recovery data processing device and core business systems, user terminals, and related databases and other related devices; the logic controller can be a desktop computer, tablet computer, or mobile terminal, etc., and this embodiment is not limited to these. In this embodiment, the logic controller can be implemented with reference to the embodiments of the off-site disaster recovery data processing method and the off-site disaster recovery data processing device in the embodiments, the content of which is incorporated herein, and repeated details will not be repeated.

[0082] It is understood that the user terminal may include smartphones, tablet computers, network set-top boxes, portable computers, desktop computers, personal digital assistants (PDAs), in-vehicle devices, smart wearable devices, etc. Among these, the smart wearable devices may include smart glasses, smartwatches, smart bracelets, etc.

[0083] In practical applications, some parts of the off-site disaster recovery data processing method can be executed on the electronic device side as described above, or all operations can be completed in the client device. The choice can be made based on the processing capabilities of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed in the client device, the client device may further include a processor.

[0084] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0085] Figure 8 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 8 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 8 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0086] In one embodiment, the off-site disaster recovery data processing method function can be integrated into the central processing unit 9100.

[0087] The central processing unit 9100 can be configured to perform the following controls:

[0088] Step S101: Multiple links read the log files of the production environment database in parallel to obtain the operation records of the production environment database, and send them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment.

[0089] Step S102: Send the database execution statements to the distributed storage system in sequence according to a set frequency.

[0090] Step S103: Obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction and write it into the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time.

[0091] As described above, the electronic device provided in this application embodiment reads the log files of the production environment database in parallel through multiple links to obtain the operation records of the production environment database, and sends them to a distributed message middleware to generate database execution statements for the off-site disaster recovery environment; the database execution statements are sent to the distributed storage system in sequence according to a set frequency; the completed transaction records and replication timestamps on different databases on the same business chain are obtained, the consistency mirror time of the next replication transaction is determined, and written to the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time, which can effectively reduce the risk of data loss during off-site disaster recovery and improve the high availability capability during the recovery process.

[0092] In another embodiment, the off-site disaster recovery data processing device can be configured separately from the central processing unit 9100. For example, the off-site disaster recovery data processing device can be configured as a chip connected to the central processing unit 9100, and the off-site disaster recovery data processing method function can be realized through the control of the central processing unit.

[0093] like Figure 8 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 8 All components shown; in addition, the electronic device 9600 may also include Figure 8 For components not shown, please refer to existing technologies.

[0094] like Figure 8 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0095] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0096] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0097] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0098] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device's communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0099] The communication module 9110 is a transmitter / receiver 9110 that transmits and receives signals via the antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0100] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 9130 is coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored sound via the speaker 9131.

[0101] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the off-site disaster recovery data processing method with a server or client as the execution subject in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the off-site disaster recovery data processing method with a server or client as the execution subject in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0102] Step S101: Multiple links read the log files of the production environment database in parallel to obtain the operation records of the production environment database, and send them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment.

[0103] Step S102: Send the database execution statements to the distributed storage system in sequence according to a set frequency.

[0104] Step S103: Obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction and write it into the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time.

[0105] As described above, the computer-readable storage medium provided in this application embodiment reads the log files of the production environment database in parallel through multiple links to obtain the operation records of the production environment database, and sends them to a distributed message middleware to generate database execution statements for the off-site disaster recovery environment; the database execution statements are sent to the distributed storage system in sequence according to a set frequency; the completed transaction records and replication timestamps on different databases on the same business chain are obtained, the consistency mirror time of the next replication transaction is determined and written to the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time, which can effectively reduce the risk of data loss during off-site disaster recovery and improve the high availability capability during the recovery process.

[0106] Embodiments of this application also provide a computer program product capable of implementing all steps of the off-site disaster recovery data processing method with the execution subject being a server or client in the above embodiments. When this computer program / instruction is executed by a processor, it implements the steps of the off-site disaster recovery data processing method. For example, the computer program / instruction implements the following steps:

[0107] Step S101: Multiple links read the log files of the production environment database in parallel to obtain the operation records of the production environment database, and send them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment.

[0108] Step S102: Send the database execution statements to the distributed storage system in sequence according to a set frequency.

[0109] Step S103: Obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction and write it into the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time.

[0110] As described above, the computer program product provided in this application reads the log files of the production environment database in parallel through multiple links to obtain the operation records of the production environment database, and sends them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment; the database execution statements are sent to the distributed storage system in sequence according to a set frequency; the completed transaction records and replication timestamps on different databases on the same business chain are obtained, the consistency mirror time of the next replication transaction is determined and written to the distributed storage system, so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time, which can effectively reduce the risk of data loss during off-site disaster recovery and improve the high availability capability during the recovery process.

[0111] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0113] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0114] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0115] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for processing off-site disaster recovery data, characterized in that, The method includes: Multiple links read the log files of the production environment database in parallel to obtain the operation records of the production environment database, and send them to the distributed message middleware to generate database execution statements for the off-site disaster recovery environment; The database execution statements are sent to the distributed storage system in sequence according to a set frequency. Obtain completed transaction records and replication timestamps from different databases on the same business chain, determine the consistency mirror time of the next replication transaction, and write it into the distributed storage system so that each application system can access the distributed storage system and perform unified data update operations based on the consistency mirror time. Prior to obtaining completed transaction records and replica timestamps from different databases on the same business chain, the process includes: Start the multi-database timing control node; Access the playback logs of each database in the off-site disaster recovery environment through the multi-database timing control node; This includes obtaining completed transaction records and replication timestamps from different databases on the same business chain to determine the consistency mirror time of the next replication transaction, including: Obtain completed transaction records and replication timestamps from different databases on the same business chain, and take the latest replication timestamp as the consistency mirror time for the next replication transaction. If the latest replication timestamps of each replication chain are completely consistent at this time, add the current latest replication timestamp to the preset time interval as the consistency mirror time for the next replication transaction.

2. The off-site disaster recovery data processing method according to claim 1, characterized in that, After obtaining the operation records of the production environment database, the following is included: The binary code of the operation records in the production environment database is concatenated into a target string according to a set rule; The target string is sent to the distributed message middleware through a preset producer program.

3. The off-site disaster recovery data processing method according to claim 1, characterized in that, The database execution statements for generating the off-site disaster recovery environment include: The target string is obtained from the distributed message middleware through a pre-defined consumer program; The target string in the distributed message middleware is parsed to obtain the operation records of the production environment and concatenate them to generate the database execution statement for the off-site disaster recovery environment.

4. The off-site disaster recovery data processing method according to claim 1, characterized in that, The operation of accessing the distributed storage system and performing unified data updates based on the consistent mirror time includes: Periodically access the distributed storage system to obtain the database execution statements to be executed and the next consistent mirror time; Update the data from the current time point to the next consistent mirror time to the database to perform the batch of data replenishment operations.

5. The off-site disaster recovery data processing method according to claim 1, characterized in that, After accessing the distributed storage system and performing a unified data update operation based on the consistent mirror time, the process includes: Determine if there is still data in the distributed storage system that needs to be replenished; If so, begin copying a new batch of transaction data.

6. A remote disaster recovery data processing device, characterized in that, include: The disaster recovery statement determination module is used to read the log files of the production environment database in parallel through multiple links, obtain the operation records of the production environment database, and send them to the distributed message middleware to generate the database execution statement for the off-site disaster recovery environment. The sequential distribution module is used to send the database execution statements to the distributed storage system in a sequential manner according to a set frequency. The data update module is used to obtain the completed transaction records and replication timestamps on different databases on the same business chain, determine the consistency mirror time of the next replication transaction, and write it into the distributed storage system so that each application system can access the distributed storage system and perform unified data update operations according to the consistency mirror time. The off-site disaster recovery data processing device is also used for: Start the multi-database timing control node; Access the playback logs of each database in the off-site disaster recovery environment through the multi-database timing control node; Specifically, the data update module is used for: Obtain completed transaction records and replication timestamps from different databases on the same business chain, and take the latest replication timestamp as the consistency mirror time for the next replication transaction. If the latest replication timestamps of each replication chain are completely consistent at this time, add the current latest replication timestamp to the preset time interval as the consistency mirror time for the next replication transaction.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the off-site disaster recovery data processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the off-site disaster recovery data processing method according to any one of claims 1 to 5.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the off-site disaster recovery data processing method according to any one of claims 1 to 5.

Images