System and method for quickly backing up and recovering data

Abstract

The invention discloses a system and a method for quickly backing up and recovering data, which are applied to the technical field of data backup. The system comprises a service front end, a backup and recovery application end and a database server, the service front end provides a visual interface for a user, the backup and recovery application end processes backup and recovery of a database, and the database server stores backup data; the backup and recovery application end comprises a scheduling and strategy processing module, a database connection processing module, a backup data processing module, a backup execution module, a backup file management module and a backup data recovery module. According to the method, the database backup efficiency is improved, the backup time is shortened, and the resource consumption is reduced; the backup process is simplified, the operation complexity is reduced, and the user experience is improved; the security and reliability of data management are enhanced, and the comprehensive coverage and rapid recovery of data are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of data backup, and more particularly to a system and method for rapid data backup and recovery. Background Art

[0002] With the advent of the big data era, the backup of TB-level databases has become an important challenge in data management. Currently, there is still a lack of a dedicated backup system for TB-level databases in the market, and traditional backup methods usually have problems such as long backup time, high resource consumption, and complex operations. Therefore, how to provide a system and method for rapid data backup and recovery is an urgent problem for those skilled in the art. Summary of the Invention

[0003] In view of this, the present invention provides a system and method for rapid data backup and recovery, which solves the problems of long backup time, high resource consumption, and complex operations in the prior art.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A system for rapid data backup and recovery includes a service front end, a backup and recovery application end, and a database server. The service front end provides a visual interface for users. The backup and recovery application end processes the backup and recovery of the database. The database server stores the backed-up data. The backup and recovery application end includes a scheduling and policy processing module, a database connection processing module, a backup data processing module, a backup execution module, a backup file management module, and a backup data recovery module. The scheduling and policy processing module processes the scheduling and policy information configured by the user. The database connection processing module processes the connection information of the backup database source. The backup data processing module divides the data to be backed up through a data segmentation algorithm. The backup execution module performs the backup of the data according to the scheduling and policy information configured by the user. The backup file management module processes the backed-up files. The backup data recovery module verifies the integrity of the backed-up data and performs the backup data recovery.

[0006] Optionally, the data segmentation algorithm of the backup data processing module is specifically as follows: According to the backup policy configured by the user, determine the data to be backed up. After reading the corresponding backup data, convert the data into a corresponding hash value through the MD5 algorithm, and compare the hash value with the hash value of the configuration file through the consistent_hash function to complete the allocation of the database server address for storage.

[0007] Optionally, the consistent_hash function is specifically as follows: convert the key into a SHA-256 hash value, convert it into a hexadecimal string, convert the hexadecimal hash value into an integer, and take the modulus of 2^32 to limit the range. Traverse the list of database server nodes. For each node in the list of database server nodes, if the current hash value is less than or equal to the hash range of the node, output the IP address of the node.

[0008] Optionally, the backup process of the backup execution module is specifically as follows: encrypt and compress the hash values of the data to be backed up in sequence, and store the processed data in the specified backup database server; the recovery process of the backup data recovery module is specifically as follows: decompress, decrypt, convert the hash value, and verify the data integrity of the backup data in sequence, and then restore the processed data to the target database server.

[0009] Optionally, the encryption process is specifically as follows: create a Fernet encryption object using a preset key, convert the input data to be backed up from a string to a byte sequence encoded in UTF-8, and encrypt it using the Fernet object to output the encrypted data.

[0010] Optionally, the compression process is specifically as follows: encode the input encrypted data string into a UTF-8 byte string byte data, compress the UTF-8 byte string byte data using the zlib library, and output the compressed data.

[0011] Optionally, the decompression process is specifically as follows: decompress the input data using the zlib library, decode the decompressed byte data into a UTF-8 string, and output the decompressed and decoded UTF-8 string data.

[0012] Optionally, the decryption process is specifically as follows: create a Fernet decryption object using a preset key, decrypt the given encrypted data using the Fernet object, and convert the result back from a byte sequence to a UTF-8 encoded string to output the decrypted data.

[0013] Optionally, during the backup and recovery process, a parallel task allocation algorithm is adopted, specifically as follows: create a process pool, calculate the server resources according to the backup policy configured by the user, allocate resources based on the calculation results, calculate the number of threads that the existing server resources can support, output the number of threads, submit the backup or recovery task to the process pool and then close the process pool, and wait to collect the task results after the process tasks are completed.

[0014] A method for rapid data backup and recovery, applying the above system for rapid data backup and recovery, includes the following steps:

[0015] S1. The user configures the backup policy, backup scheduling information, and backup method through the server front-end;

[0016] S2. Connect to the corresponding database to be backed up, convert the extracted backup data into a hash value, and complete the allocation of the database server address for storage;

[0017] S3. Encrypt and compress the hash values of the data to be backed up in sequence, and store the processed data in the specified backup database server;

[0018] S4. When recovery is needed, decompress, decrypt, and perform hash value conversion on the backed-up data in sequence, and then restore the processed data to the target database server.

[0019] As can be seen from the above technical solutions, compared with the prior art, the present invention provides a system and method for rapid data backup and recovery, which has the following beneficial effects: The present invention improves the efficiency of database backup, shortens the backup time, and reduces resource consumption; simplifies the backup process, reduces the operation complexity, and improves the user experience; enhances the security and reliability of data management, and ensures the comprehensive coverage and rapid recovery of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0021] Figure 1 It is the system schematic diagram for rapid data backup and recovery of the present invention;

[0022] Figure 2 It is the data backup flow chart of the present invention;

[0023] Figure 3 It is the data recovery flow chart of the present invention;

[0024] Figure 4 It is the database server address allocation flow chart of the present invention;

[0025] Figure 5 It is the data encryption flow chart of the present invention;

[0026] Figure 6 It is the data decryption and integrity verification flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0028] An embodiment of the present invention discloses a system for rapid data backup and recovery, as Figure 1 shown, which includes a service front end, a backup and recovery application end, and a database server. The service front end provides a visual interface for users. The backup and recovery application end processes the backup and recovery of the database, and the database server stores the backed-up data. The backup and recovery application end includes a scheduling and policy processing module, a database connection processing module, a backup data processing module, a backup execution module, a backup file management module, and a backup data recovery module. The scheduling and policy processing module processes the scheduling and policy information configured by the user. The database connection processing module processes the connection information of the backup database source. The backup data processing module divides the data to be backed up through a data splitting algorithm. The backup execution module performs the backup of the data according to the scheduling and policy information configured by the user. The backup file management module processes the backed-up files. The backup data recovery module performs integrity verification on the backed-up data and restores the backup data.

[0029] In the embodiment of the present invention, the visual interface provided by the service front end can perform operations such as backup scheduling and policy setting, controlling backup, managing backup files, and performing backup recovery. If scheduled backup is configured, the system service performs scheduled backup according to the configured time. If manual backup is used, the user needs to start the backup operation on the front-end interface, connect to the corresponding database to be backed up, convert the extracted backup data into a hash value, and after encrypting and compressing the hash value, store it in the specified backup server.

[0030] The backup file management module can process the backed-up files, such as compression and naming, and process the historical backup file management operations.

[0031] Furthermore, as Figure 4 shown, the data splitting algorithm of the backup data processing module is specifically as follows: according to the backup policy configured by the user, determine the data to be backed up. After reading the corresponding backup data, convert the data into a corresponding hash value through the MD5 algorithm, and compare the hash value with the hash value of the configuration file through the consistent_hash function to complete the allocation of the database server address for storage.

[0032] In an embodiment of the present invention, the MD5 algorithm is specifically as follows: Create a new md5_hash object, update the data of the hash object, convert the input data from a string to a byte sequence encoded in UTF-8, use the converted byte sequence to update the md5_hash object, and output the hexadecimal string representation of the hash object.

[0033] Further, the consistent_hash function is specifically as follows: Convert the key to a SHA-256 hash value, convert it to a hexadecimal string, convert the hexadecimal hash value to an integer, and take the modulus of 2^32 to limit the range. Traverse the list of database server nodes. For each node in the list of database server nodes, if the current hash value is less than or equal to the hash range of the node, output the IP address of the node.

[0034] If no matching node is found, output a blank value.

[0035] Further, as Figure 2 shown, the backup process of the backup execution module is specifically as follows: Encrypt and compress the hash values of the data to be backed up in sequence, and store the processed data in the specified backup database server; as Figure 3 shown, the recovery process of the backup data recovery module is specifically as follows: After decompressing, decrypting, hash value conversion processing, and data integrity verification of the backup data in sequence, restore the processed data to the target database server.

[0036] Further, as Figure 5 shown, the encryption processing is specifically as follows: Create a Fernet encryption object using a preset key, convert the input data to be backed up from a string to a byte sequence encoded in UTF-8, and encrypt it using the Fernet object to output the encrypted data.

[0037] After converting the backup data into a hash value through the MD5 algorithm, encrypt the corresponding backup data converted into a hash value through encryption processing (aes_encrypt encryption function), and then store it in the backup server according to the policy rules

[0038] Further, the compression processing is specifically as follows: Encode the input encrypted data string into a UTF-8 byte string byte data, compress the UTF-8 byte string byte data using the zlib library, and output the compressed data. In an embodiment of the present invention, the compression processing uses the function gzip_compress, and the compressed data = zlib.compress(byte data).

[0039] Further, the decompression process is specifically as follows: Use the zlib library to decompress the input data, decode the decompressed byte data into a UTF-8 string, and output the decompressed and decoded UTF-8 string data. In the embodiment of the present invention, the decompression process uses the function gzip_decompress, and the decompressed data = decode(decompressed byte data, 'utf-8').

[0040] Further, as Figure 6 shown, the decryption process is specifically as follows: Create a Fernet decryption object using a preset key, use the Fernet object to decrypt the given encrypted data, and convert the result back from a byte sequence to a UTF-8 encoded string, and output the decrypted data.

[0041] During the data recovery process, first select the backup data to be recovered, decrypt the backup data through the decryption process (aes_decrypt decryption function) to obtain the hash value of the backup data. Through data integrity verification, if the hash value integrity is true, perform the data recovery operation. If the hash value integrity is false, it is necessary to manually check the backup process log to check the problems of the backup data.

[0042] Further, during the backup and recovery process, a parallel task allocation algorithm is adopted, specifically as follows: Create a process pool, calculate the server resources according to the backup policy configured by the user, perform resource allocation based on the calculation results, calculate the number of threads supported by the existing server resources, output the number of threads, submit the backup or recovery task to the process pool and then close the process pool, and wait to collect the task results after the process tasks are completed.

[0043] The backup task uses the function backup_work(hash_data, server_ip): Store the data finally encrypted by the aes_encrypt function and the backup server addresses where each data is stored obtained by the consistent_hash function into the corresponding backup server according to the backup policy, and output the storage result;

[0044] The recovery task uses the function recovery_work(data_range, target_server_ip, source_server_ip): According to source_server_ip, read the recovery data range from the backup storage server according to data_range, determine the database address of the data to be recovered according to the input target_server_ip, decrypt the encrypted data through the aes_decrypt function, and perform database data recovery, and output the recovery result.

[0045] With Figure 1Corresponding to the described system, an embodiment of the present invention also discloses a method for rapid data backup and recovery. Applying the above-mentioned system for rapid data backup and recovery, it includes the following steps:

[0046] S1. The user configures the backup strategy, backup scheduling information, and backup method through the server front end;

[0047] S2. Connect to the corresponding database to be backed up, convert the extracted backup data into a hash value, and complete the allocation of the database server address for storage;

[0048] S3. Encrypt and compress the hash value of the data to be backed up in sequence, and store the processed data in the specified backup database server;

[0049] S4. When recovery is required, decompress, decrypt, and perform hash value conversion on the backed-up data in sequence, and then restore the processed data to the target database server.

[0050] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple. For the relevant parts, refer to the description of the system part.

[0051] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A system for rapid data backup and recovery, characterized in that, It includes a service front end, a backup and recovery application end, and a database server. The service front end provides a visual interface for users. The backup and recovery application end processes the backup and recovery of the database. The database server stores the backed-up data. The backup and recovery application end includes a scheduling and policy processing module, a database connection processing module, a backup data processing module, a backup execution module, a backup file management module, and a backup data recovery module. The scheduling and policy processing module processes the scheduling and policy information configured by the user. The database connection processing module processes the backup database source connection information. The backup data processing module divides the data to be backed up through a data splitting algorithm. The backup execution module performs the backup of the data according to the scheduling and policy information configured by the user. The backup file management module processes the backed-up files. The backup data recovery module performs integrity verification on the backed-up data and restores the backup data.

2. The system for rapid data backup and recovery according to claim 1, characterized in that, The data splitting algorithm of the backup data processing module is specifically as follows: According to the backup policy configured by the user, determine the data to be backed up. After reading the corresponding backup data, convert the data into the corresponding hash value through the MD5 algorithm. Compare the hash value with the hash value of the configuration file through the consistent_hash function to complete the allocation of the database server address for storage.

3. A system for rapid data backup and recovery according to claim 2, characterized in that, The consistent_hash function is specifically as follows: Convert the key into a SHA-256 hash value and convert it into a hexadecimal string. Convert the hexadecimal hash value into an integer and take the modulus of 2^32 to limit the range. Traverse the list of database server nodes. For each node in the list of database server nodes, if the current hash value is less than or equal to the hash range of the node, output the IP address of the current node.

4. A system for rapid data backup and recovery according to claim 1, characterized in that, The backup process of the backup execution module is specifically as follows: Encrypt and compress the hash values of the data to be backed up in sequence, and store the processed data in the specified backup database server. The recovery process of the backup data recovery module is specifically as follows: After decompressing, decrypting, hash value conversion processing, and data integrity verification of the backed-up data in sequence, restore the processed data to the target database server.

5. A system for rapid data backup and recovery according to claim 4, characterized in that, The encryption process is specifically as follows: Create a Fernet encryption object using a preset key. Convert the input data to be backed up from a string to a byte sequence encoded in UTF-8, and encrypt it using the Fernet object to output the encrypted data.

6. A system for rapid data backup and recovery according to claim 4, characterized in that, The compression process is specifically as follows: Encode the input encrypted data string into a UTF-8 byte string byte data, and compress the UTF-8 byte string byte data using the zlib library to output the compressed data.

7. A system for rapid data backup and recovery according to claim 4, characterized in that, The decompression process is specifically as follows: Use the zlib library to decompress the input data, decode the decompressed byte data into a UTF-8 string, and output the decompressed and decoded UTF-8 string data.

8. A system for rapid data backup and recovery according to claim 4, characterized in that The decryption process is specifically as follows: Create a Fernet decryption object using a preset key, decrypt the given encrypted data using the Fernet object, and convert the result from a byte sequence back to a UTF-8 encoded string to output the decrypted data.

9. A system for rapid data backup and recovery according to claim 1, characterized in that, During the backup and recovery process, a parallel task allocation algorithm is adopted, specifically: creating a process pool, calculating server resources according to the backup policy configured by the user, allocating resources based on the calculation results, calculating the number of threads supported by the existing server resources, outputting the number of threads, submitting backup or recovery tasks to the process pool and then closing the process pool, and collecting task results after the process tasks are completed.

10. A method for rapid data backup and recovery, characterized in that, Applying the system for rapid data backup and recovery according to any one of claims 1-9, comprising the following steps: S1. The user configures the backup policy, backup scheduling information, and backup method through the server front end; S2. Connect to the corresponding database to be backed up, convert the extracted backup data into a hash value, and complete the allocation of the database server address for storage; S3. Encrypt and compress the hash values of the data to be backed up in sequence, and store the processed data in the specified backup database server; S4. When recovery is required, decompress, decrypt, and convert the hash value of the backup data in sequence, and restore the processed data to the target database server.

Images