Remote data disaster recovery system and data backup method

Abstract

The embodiment of the invention relates to a remote data disaster recovery system and a data backup method, the system comprises a source end and a target end, and the source end and the target end are connected through a virtual private network; the source end comprises a source end server, a physical backup module and a source end database; the target end comprises a target end server and a target end database; the physical backup module is used for generating a backup strategy according to the importance and change frequency of the data and generating backup data based on the backup strategy; and the virtual private network is used for establishing a data synchronization channel of data from the source end database to the target end database. According to the technical scheme provided by the embodiment of the invention, the change event of the database can be efficiently captured and transmitted to the disaster recovery end, and real-time or quasi-real-time updating of the data is realized. The risk of data loss can be reduced, the protection level of data can be further enhanced through physical isolation, the safety of sensitive information is ensured, and enterprises are helped to comply with related compliance standards.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a remote data disaster recovery system and a data backup method. Background Art

[0002] As businesses rapidly expand, the volume of business data is experiencing a non-linear and dramatic growth trend. This data has become an indispensable core digital asset for enterprises. To ensure business continuity and data security, local online hot backup storage strategies are often implemented to address data protection needs during daily operations. However, in the face of unforeseen circumstances such as natural disasters, hardware failures, or human error, ensuring uninterrupted data services and maintaining business continuity becomes a pressing issue. Summary of the Invention

[0003] Based on the above situation of the prior art, the purpose of the embodiments of the present invention is to provide an off-site data disaster recovery system and data backup method, which can efficiently capture database change events and transmit them to the disaster recovery end, thereby realizing real-time or quasi-real-time update of data.

[0004] To achieve the above object, according to one aspect of the present invention, there is provided a remote data disaster recovery system, comprising a source end and a target end, wherein the source end and the target end are connected via a virtual private network;

[0005] The source end includes a source end server, a physical backup module and a source end database; the target end includes a target end server and a target end database;

[0006] The physical backup module is configured to generate a backup strategy based on the importance and change frequency of the data, and generate backup data based on the backup strategy;

[0007] The virtual private network is used to establish a data synchronization channel from the source database to the target database.

[0008] Furthermore, the source end further includes a data encryption module and a synchronization link module;

[0009] The data encryption module is used to generate encryption keys and compression algorithm parameters to encrypt and compress the data to be transmitted;

[0010] The synchronization link module is connected to the source database and is used to obtain data change information from the source database. The data change information is used to verify the update status and integrity of the data to be transmitted.

[0011] Furthermore, the data change information includes a first timestamp of the data information change and a first change operation of the data information;

[0012] The backup data includes a second timestamp of the data backup and a second change operation of the data;

[0013] Verifying the update status and integrity of the data to be transmitted includes: comparing the first timestamp with the second timestamp to determine whether the backup data is the latest data; and comparing the first change operation with the second change operation to determine whether the backup data is complete.

[0014] Furthermore, a backup strategy is generated based on the importance and change frequency of the data, including:

[0015] For core business data, a full backup is performed based on the first time interval, and an incremental backup is performed during the off-peak period of the first time interval.

[0016] For general business data, a full backup is performed based on the second time interval, and an incremental backup is performed at a preset time point in the second time interval;

[0017] The first time interval is smaller than the second time interval.

[0018] Furthermore, a data synchronization channel is established from the source database to the target database, including:

[0019] Configuring the VPN gateway device at the source end, and adding the target end connection configuration information to the VPN gateway device at the source end;

[0020] Configuring the VPN gateway device at the target end, and adding the source end connection configuration information to the VPN gateway device at the target end;

[0021] Configure IPsec policies on the source and target ends;

[0022] Configure routing on both the source and destination sides.

[0023] Furthermore, configure IPsec policies on the source and target ends, including:

[0024] Setting an encrypted access control list, wherein the encrypted access control list is used to determine the traffic path transmitted through the VPN;

[0025] Setting an IPsec transform set, wherein the IPsec transform set is used to determine the encryption algorithm and authentication method;

[0026] The encryption access control list and the IPsec transform set are associated with each other.

[0027] Furthermore, establishing a data synchronization channel from the source database to the target database also includes:

[0028] Connect the SFTP server VM to the VPN network;

[0029] Configure the SFTP server application, including the port, account password, account permissions, target end whitelist IP, and SSH key.

[0030] Furthermore, the system also includes an automatic backup data transmission module, which is used to detect the generated backup data and transmit the backup data to the target end through the SFTP server virtual machine.

[0031] According to another aspect of the present invention, a data backup method based on an off-site data disaster recovery system is provided, comprising the steps of:

[0032] Based on the physical backup module, a backup strategy is generated according to the importance and change frequency of the data, and backup data is generated based on the backup strategy;

[0033] Use a virtual private network to establish a data synchronization channel from the source database to the target database;

[0034] The backup data is transferred from the source database to the target database through a data synchronization channel.

[0035] Furthermore, the steps include:

[0036] Obtain data change information from the source database, which is used to verify the update status and integrity of the data to be transmitted;

[0037] The data to be transmitted is encrypted and compressed before transmission.

[0038] In summary, an embodiment of the present invention provides an off-site data disaster recovery system, comprising a source end and a target end, wherein the source end and the target end are connected via a virtual private network; the source end comprises a source server, a physical backup module, and a source database; the target end comprises a target server and a target database; the physical backup module is used to generate a backup strategy according to the importance and change frequency of the data, and to generate backup data based on the backup strategy; the virtual private network is used to establish a data synchronization channel from the source database to the target database. The technical solution provided by the embodiment of the present invention can efficiently capture database change events and transmit them to the disaster recovery end, thereby realizing real-time or quasi-real-time updates of data. It can not only reduce the risk of data loss, but also further enhance the protection level of data through physical isolation, ensure the security of sensitive information, and help enterprises comply with relevant compliance standards. When a data disaster occurs, the off-site disaster recovery system can quickly start the data recovery process, greatly shortening the disaster recovery time (RTO) and data loss window (RPO), thereby minimizing the impact of data loss and business interruption. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the overall structure of the remote data disaster recovery system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present invention.

[0041] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprising" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0042] The technical solution of the present invention is described in detail below with reference to the accompanying drawings. Figure 1 FIG. 1 shows a schematic diagram of the overall structure of the remote data disaster recovery system according to an embodiment of the present invention. Figure 1 As shown, the system includes a source and a target, connected via a virtual private network. The source includes a source server, a physical backup module, and a source database; the target includes a target server and a target database. The source server selects appropriate hardware and configures a high-performance file system, such as EXT4 or XFS, to provide stable and efficient underlying support for MySQL database operations and data storage. For example, the source and target databases can be MySQL databases. Standardized deployment of MySQL databases ensures high performance and high availability of data processing. On the target side, the configuration of the target server and target database can be consistent with the architecture of the source server and source database, providing infrastructure for data reception and recovery.

[0043] The physical backup module is used to generate a backup strategy based on the importance and change frequency of the data, and to generate backup data based on the backup strategy. This strategy includes: for core business data, performing a full backup based on a first time interval, and performing incremental backups during low periods within the first time interval; for general business data, performing a full backup based on a second time interval, and performing incremental backups at preset time points within the second time interval; the first time interval is shorter than the second time interval.

[0044] According to some optional embodiments, the physical backup module may also generate a backup strategy based on dynamic perception of data value, which may be implemented by the following steps:

[0045] (1) A model is trained using annotated historical data. The annotations are based on the mapping relationship between data and related businesses and the frequency of data changes, resulting in a multidimensional data value classification model. This model classifies the value of input data through hierarchical analysis of business relevance (data and related business mapping) and timeliness statistics (change frequency monitoring). The data to be backed up is input into the multidimensional data value classification model to obtain a multidimensional data value classification result for each data. The classification result is represented as, for example, T0, T1, and T2 levels, in descending order of data value.

[0046] (2) Based on the multi-dimensional data value classification results and historical risk event records of the data, a backup strategy is generated, which includes the backup mode, frequency and number of copies corresponding to each data classification level.

[0047] To perform a backup based on the generated backup policy, you can use the following steps:

[0048] Based on the generated backup strategy and real-time network bandwidth data, a data shard queue with priority tags is obtained. This data shard queue includes a hot data shard set and a cold data shard set, and a shard transmission route is set for each shard queue. For example, when the real-time network bandwidth is 80Mbps, the hot data shard set splits the T0-level data into small shards of fixed size (for example, 10MB per shard), and adds a high-priority tag (for example, level 1) to each shard. The cold data shard set merges the T2-level data into batch packages of variable size (for example, 500MB per package) and adds a low-priority tag (for example, level 3). Independent transmission channels (for example, dedicated VPN tunnels) are allocated for hot data shards, and cold data shards use a public bandwidth pool.

[0049] By generating backup and transmission strategies through the above dynamic perception method based on data value, refined management and control of data resources are achieved, which can flexibly respond to risks, thereby ensuring the real-time nature of disaster recovery data while saving storage space and bandwidth.

[0050] When generating backup data, the physical backup module also records the backup start timestamp as a secondary timestamp and the sequence of operations involved in the data, such as insert, update, and delete, as a secondary change operation. You can use shell scripts to automate database physical backups, and schedule backups with scheduled tasks to ensure regular and automated data backup operations.

[0051] A virtual private network (VPN) is used to establish a data synchronization channel from the source database to the target database. VPN technology is used to build a secure VPN between the source and target databases, ensuring the privacy and integrity of data during transmission. In an embodiment of the present invention, the data synchronization channel from the source database to the target database can be established according to the following steps:

[0052] (1) The VPN gateway device at the source end is configured, and target end connection configuration information is added to the VPN gateway device at the source end.

[0053] (2) The VPN gateway device at the target end is configured, and the source end connection configuration information is added to the VPN gateway device at the target end.

[0054] (3) Configuring IPsec policies at the source and target ends, including: setting an encryption access control list, which is used to determine the traffic path transmitted through the VPN; setting an IPsec transform set, which is used to determine the encryption algorithm and authentication method; and associating the encryption access control list and the IPsec transform set.

[0055] (4) Configure routing at the source and destination ends.

[0056] According to some optional embodiments, an SFTP server can be configured on the source end and SSH key authentication can be used to enhance security, ensuring that the target end can securely access and transfer backup data through an encrypted virtual private network. The SFTP server configuration can be based on the following steps:

[0057] (1) Create an SFTP server VM resource and specify the VPC and VPN network to connect the SFTP server VM to the VPN network.

[0058] (2) Configure the SFTP server application, including the port, account password, account permissions, target end whitelist IP, and SSH key.

[0059] (3) Start the SFTP server program.

[0060] (4) Configure the SFTP server security group to allow access from the specified target end whitelist IP.

[0061] (5) Target-side application connectivity test.

[0062] (6) Start the target application.

[0063] According to certain optional embodiments, the source end may also include a data encryption module and a synchronization link module. The synchronization link module connects to the source database and is used to obtain data change information from the source database. The data change information is used to verify the update status and integrity of the data to be transmitted. The synchronization link module can use MySQL Binlog to establish a data synchronization mechanism from the source database to the target database, achieving real-time or near-real-time capture and transmission of data changes, minimizing the RPO. This data synchronization mechanism can be based on the following steps:

[0064] (1) Configure the source database and target database information, including access links, account numbers, and passwords.

[0065] (2) Configure the task object, including synchronization type, such as full backup and incremental backup; synchronization topology type, such as one-way synchronization; source database object, synchronization rate, and other information.

[0066] (3) Save the task and pre-check the task.

[0067] (4) Start the task and monitor it.

[0068] The synchronization link module obtains data change information from the source, records the timestamp of each data change as the first timestamp, and also records the sequence of operations involved in the data change, such as insert, update, and delete. This sequence is then converted into an operation sequence, which serves as the first change operation. The first timestamp can be compared with the second timestamp of the backup data to determine whether the backup data is up to date. The first change operation can also be compared with the second change operation to determine whether the backup data is complete.

[0069] The data encryption module is used to generate encryption keys and compression algorithm parameters to encrypt and compress the data to be transmitted. The data encryption module can use tools to generate symmetric encryption keys and combine them with compression algorithm parameters, such as the gzip algorithm and the lz4 algorithm, to ensure the security and efficiency of backup data transmission. Data encryption can be performed according to the following steps:

[0070] (1) Generate a random-length key using the preset encryption algorithm.

[0071] (2) Configure the encryption key parameters for backup data.

[0072] (3) Configure the compression algorithm parameters for backup data.

[0073] (4) Start the encryption program.

[0074] According to certain optional embodiments, the system further includes an automatic backup data transmission module configured to detect generated backup data and transmit the backup data to a target end via an SFTP server virtual machine. The automatic backup data transmission module can implement automated physical backup of the database using a shell script and schedule backups using a scheduled task to ensure regular and automated data backup operations.

[0075] According to another aspect of the present invention, a data backup method based on an off-site data disaster recovery system is provided, comprising the following steps:

[0076] Based on the physical backup module, a backup strategy is generated according to the importance and change frequency of the data, and backup data is generated based on the backup strategy;

[0077] Use a virtual private network to establish a data synchronization channel from the source database to the target database;

[0078] The backup data is transferred from the source database to the target database through a data synchronization channel.

[0079] Furthermore, the method may include the following steps:

[0080] Obtain data change information from the source database, which is used to verify the update status and integrity of the data to be transmitted;

[0081] The data to be transmitted is encrypted and compressed before transmission.

[0082] The specific implementation method of each step of the data backup method in the embodiment of the present invention is the same as the specific implementation of each module in the above-mentioned remote data disaster recovery system of the present invention, and its repeated description will be omitted here.

[0083] In summary, an embodiment of the present invention relates to an off-site data disaster recovery system, comprising a source end and a target end, wherein the source end and the target end are connected via a virtual private network; the source end comprises a source server, a physical backup module, and a source database; the target end comprises a target server and a target database; the physical backup module is used to generate a backup strategy according to the importance and change frequency of the data, and to generate backup data based on the backup strategy; the virtual private network is used to establish a data synchronization channel from the source database to the target database. The technical solution provided by the embodiment of the present invention can efficiently capture database change events and transmit them to the disaster recovery end, thereby realizing real-time or quasi-real-time updates of data. It can not only reduce the risk of data loss, but also further enhance the protection level of data through physical isolation, ensure the security of sensitive information, and help enterprises comply with relevant compliance standards. When a data disaster occurs, the off-site disaster recovery system can quickly start the data recovery process, greatly shortening the disaster recovery time (RTO) and data loss window (RPO), thereby minimizing the impact of data loss and business interruption.

[0084] It should be understood that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples; under the thinking of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity. The above specific embodiments of the present invention are merely used to illustrate or explain the principles of the present invention and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the scope of protection of the present invention. In addition, the claims appended to the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

Claims

1. A remote data disaster recovery system, characterized in that: comprising a source end and a target end, wherein the source end and the target end are connected via a virtual private network; The source end includes a source end server, a physical backup module and a source end database; the target end includes a target end server and a target end database; The physical backup module is configured to generate a backup strategy based on the importance and change frequency of the data, and generate backup data based on the backup strategy; The virtual private network is used to establish a data synchronization channel from the source database to the target database.

2. The system according to claim 1, wherein: The source end also includes a data encryption module and a synchronization link module; The data encryption module is used to generate encryption keys and compression algorithm parameters to encrypt and compress the data to be transmitted; The synchronization link module is connected to the source database and is used to obtain data change information from the source database. The data change information is used to verify the update status and integrity of the data to be transmitted.

3. The system according to claim 2, characterized in that The data change information includes a first timestamp of the data information change and a first change operation of the data information; The backup data includes a second timestamp of the data backup and a second change operation of the data; Verifying the update status and integrity of the data to be transmitted, including: comparing the first timestamp with the second timestamp to determine whether the backup data is the latest data; The first change operation is compared with the second change operation to determine whether the backup data is complete.

4. The system according to claim 1, wherein: Generate a backup strategy based on the importance and change frequency of the data, including: For core business data, a full backup is performed based on the first time interval, and an incremental backup is performed during the off-peak period of the first time interval. For general business data, a full backup is performed based on the second time interval, and an incremental backup is performed at a preset time point in the second time interval; The first time interval is smaller than the second time interval.

5. The system according to claim 1, wherein: Establish a data synchronization channel from the source database to the target database, including: Configuring the VPN gateway device at the source end, and adding the target end connection configuration information to the VPN gateway device at the source end; Configuring the VPN gateway device at the target end, and adding the source end connection configuration information to the VPN gateway device at the target end; Configure IPsec policies on the source and target ends; Configure routing on both the source and destination sides.

6. The system according to claim 5, characterized in that Configure IPsec policies on the source and target ends, including: Setting an encrypted access control list, wherein the encrypted access control list is used to determine the traffic path transmitted through the VPN; Setting an IPsec transform set, wherein the IPsec transform set is used to determine the encryption algorithm and authentication method; The encryption access control list and the IPsec transform set are associated with each other.

7. The system according to claim 5, characterized in that Establishing a data synchronization channel from the source database to the target database also includes: Connect the SFTP server VM to the VPN network; Configure the SFTP server application, including the port, account password, account permissions, target end whitelist IP, and SSH key.

8. The system according to claim 1, wherein: The system further comprises an automatic backup data transmission module, which is used to detect the generated backup data and transmit the backup data to the target end through the SFTP server virtual machine.

9. A data backup method based on an off-site data disaster recovery system, characterized in that: Including steps: Based on the physical backup module, a backup strategy is generated according to the importance and change frequency of the data, and backup data is generated based on the backup strategy; Use a virtual private network to establish a data synchronization channel from the source database to the target database; The backup data is transferred from the source database to the target database through a data synchronization channel.

10. The method according to claim 9, characterized in that Also includes the steps: Obtain data change information from the source database, which is used to verify the update status and integrity of the data to be transmitted; The data to be transmitted is encrypted and compressed before transmission.

Images