A data backup and rapid recovery processing method and system

Abstract

The present invention relates to the field of data backup technology, and in particular, to a data backup and rapid recovery processing method and system. It includes a data acquisition module, an evaluation matrix establishment module, a predefined recovery module and a data verification module. The present invention collects data change frequency, remaining storage space and business cycle information through the data acquisition module, and establishes a backup strategy evaluation matrix, and selects a corresponding backup strategy based on the information of the corresponding dimension transmitted by the data acquisition module, so as to achieve the most suitable backup type under different data conditions, improve the backup efficiency, and use the backup index to quickly locate the storage location of the corresponding backup type. At the same time, the parallel recovery technology is adopted, and the data of different parts are restored simultaneously by multi-threading or multi-process, so as to improve the recovery speed and significantly shorten the recovery time.

Description

Technical Field

[0001] The present invention relates to the technical field of data backup, and in particular to a data backup and rapid recovery processing method and system. Background Art

[0002] Data backup is the basis of disaster recovery. It refers to the process of copying all or part of the data set from the hard disk or array of the application host to other storage media to prevent data loss due to system operation errors or system failures. Common data backup types include: full backup: full backup refers to backing up the entire data set, including all data files and databases. When restoring, only the latest full backup is needed; incremental backup: incremental backup only backs up data that has been changed or added since the last backup. When restoring data, all related incremental backup files and the most recent full backup are required; differential backup: differential backup will back up all data that has changed since the last full backup. When restoring, only the most recent full backup and the most recent differential backup are needed.

[0003] At present, when backing up data, it is necessary to select the corresponding backup type according to the data update frequency, the idle time of the backup resources and the business cycle. Full backup is used for important data to ensure that all data can be backed up. However, full backup will take up a large amount of backup storage space. If full backup is always used, it will lead to insufficient memory and then backup failure. Considering that incremental backup and differential backup are based on full backup, in order to be able to switch the data backup type according to the actual situation such as data update frequency, backup resource space and business cycle, so as to better back up the data and quickly restore the backup data, we propose a data backup and rapid recovery processing method and system. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the existing data backup method cannot effectively balance factors such as data update frequency, storage space resources and business cycle, resulting in insufficient memory or backup failure when backing up key data. In particular, when full backup takes up a large amount of storage space, in order to flexibly switch the appropriate backup type under different data environments, improve backup efficiency, and ensure rapid recovery of backup data.

[0005] To achieve the above object, the present invention proposes a data backup and rapid recovery processing system that can dynamically select a backup strategy according to actual conditions, specifically comprising a data acquisition module, an evaluation matrix establishment module, a predefined recovery module and a data verification module;

[0006] The data acquisition module identifies data changes, including newly added, modified and deleted data content, through the data change tracking function, so as to directly obtain the frequency of data changes, use the system monitoring tool to detect the remaining storage space of the backup data storage device, and judge the backup data cycle by analyzing the database log according to the previous business cycle. The data acquisition module transmits the data change frequency, remaining storage space and business cycle information to the evaluation matrix establishment module;

[0007] The evaluation matrix establishment module uses data change frequency, remaining storage space and business cycle information as three dimensions to establish a backup strategy evaluation matrix. For the evaluation results of each backup strategy under each dimension, a comprehensive evaluation score is calculated according to the weight, and the comprehensive evaluation scores of different backup strategies are compared. The backup strategy with the highest score is the optimal choice under the current evaluation environment, and the backup strategy corresponding to different combinations of each dimension is obtained. The corresponding backup strategy is selected according to the information of the corresponding dimension transmitted by the data acquisition module, and the data is backed up according to the corresponding backup type. The backup type is one of full backup, incremental backup and differential backup.

[0008] When backing up data, the predefined recovery module creates a backup index for the backup file and records the checksum of the backup data and the redundant information of the backup data. When restoring data, the backup index is used to quickly locate the storage location of the corresponding backup type. At the same time, parallel recovery technology is adopted to simultaneously restore different parts of data using multi-threading or multi-process, thereby improving the recovery speed and significantly shortening the recovery time, and the restored data is passed to the data verification module;

[0009] The data verification module compares the checksum of the restored data with the checksum of the backup data. If the two are the same, the restored data is complete and correct. If the two are different, the restored data is lost or tampered with. At this time, the backup data is restored based on its redundant information to ensure the accuracy of data recovery.

[0010] As a further improvement of the technical solution, the evaluation matrix establishment module includes a matrix establishment unit and a type selection unit;

[0011] The matrix establishment unit further subdivides the three dimensions of data change frequency, remaining storage space and business cycle information, and subdivides each dimension into different categories, thereby obtaining more detailed evaluation results and establishing a more accurate backup strategy evaluation matrix;

[0012] The type selection unit selects the corresponding backup type according to the detection information of each dimension transmitted by the data acquisition module and the backup strategy evaluation matrix, and starts to back up the data.

[0013] As a further improvement of the technical solution, when establishing the backup strategy evaluation matrix, the matrix establishment unit assigns weights to each evaluation dimension according to the business focus and resource status of the enterprise, according to the formula: Get the comprehensive evaluation score of the corresponding backup type;

[0014] Among them, S is the comprehensive evaluation score, A is the score of the data update frequency dimension, is the weight of the data update frequency dimension, B is the score of the remaining storage space dimension, is the weight of the remaining storage space dimension, C is the score of the business cycle dimension, is the weight of the business cycle dimension.

[0015] As a further improvement of the present technical solution, the type selection unit records the data status when the data backup type is switched to ensure that the data at the switching point is complete and accurate. At the same time, a consistency check is performed on the log files and data files of the database at the switching time to ensure data consistency.

[0016] As a further improvement of the technical solution, the predefined recovery module includes an index creation unit and a parallel recovery unit. The index creation unit creates a table-level index for the data when backing up the data. The table-level index records the name of each table, the table structure definition, including the column name, data type, primary key and foreign key, etc., as well as the storage location in the backup file, so as to facilitate rapid positioning when restoring the data.

[0017] The parallel recovery unit decomposes the recovery task into multiple subtasks. Each thread or process is responsible for one subtask. One thread may be responsible for recovering the table structure, and another thread may be responsible for recovering the data records. Multiple threads run simultaneously to improve recovery efficiency.

[0018] As a further improvement of the present technical solution, when creating a table-level index, the index creation unit uses a custom script in the relational database MySQL combined with the database's information model information_schema to create a backup index. These tables provide detailed information about the database structure and storage. At the same time, the backup index is regularly maintained and updated.

[0019] As a further improvement of the present technical solution, when the parallel recovery unit performs multi-threaded or multi-process recovery on data, a lock mechanism is used to ensure that only one thread can access and modify shared resources at any time, and data is coordinated and shared through an inter-process communication mechanism to ensure data consistency.

[0020] As a further improvement of the technical solution, the data verification module includes a comparison checksum unit and a redundancy recovery unit, wherein the comparison checksum unit calculates the checksum of the backup data by using the MD5 algorithm, compares the checksum of the restored data with the checksum of the backup data, and verifies the consistency of the restored data;

[0021] The redundant recovery unit locates the timestamp, transaction ID and other information in the binary log BinaryLog of the lost data by using the redundant information of the backup data to find the position of the last correct transaction before the data was lost.

[0022] As a further improvement of the present technical solution, the redundant recovery unit uses the "mysqlbinlog" tool in the relational database MySQL to read the binary log and apply the transaction operations therein to the database to restore the database to the state before the data was lost.

[0023] A second object of the present invention is to provide a data backup and rapid recovery processing method, including any one of the data backup and rapid recovery processing systems described above, comprising the following steps:

[0024] S1. Collect data change frequency, remaining storage space and business cycle information through the data acquisition module;

[0025] S2. Use the evaluation matrix establishment module to establish a backup strategy evaluation matrix with data change frequency, remaining storage space and business cycle information as dimensions, select a corresponding backup strategy according to the information of the corresponding dimension transmitted by the data acquisition module, and back up the data according to the corresponding backup type, where the backup type is one of full backup, incremental backup and differential backup;

[0026] S3. Establish a backup index for the backup data through a predefined recovery module, and record the checksum of the backup data and the redundant information of the backup data. At the same time, use parallel recovery technology to simultaneously restore different parts of data;

[0027] S4. Use the data verification module to compare the checksum of the restored data with the checksum of the backup data to verify the consistency of the restored data.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. The data backup and rapid recovery processing method and system collects data change frequency, remaining storage space and business cycle information through a data acquisition module, and uses an evaluation matrix establishment module to establish a backup strategy evaluation matrix with data change frequency, remaining storage space and business cycle information as dimensions, calculates a comprehensive evaluation score according to weights, compares the comprehensive evaluation scores of different backup strategies, and the backup strategy with the highest score is the optimal choice under the current evaluation environment. The backup strategy corresponding to different combinations of each dimension is obtained, and the corresponding backup strategy is selected according to the information of the corresponding dimension transmitted by the data acquisition module, so as to realize the selection of the most appropriate backup type under different data conditions and improve the backup efficiency;

[0030] 2. When restoring the backup data, create a backup index for the backup file, and record the checksum and redundant information of the backup data. When restoring the data, use the backup index to quickly locate the storage location of the corresponding backup type. At the same time, use parallel recovery technology to use multi-threading or multi-process to restore different parts of the data at the same time, improve the recovery speed, and significantly shorten the recovery time;

[0031] 3. After the backup data is restored, the checksum of the restored data is compared with the checksum of the backup data through the data verification module to verify the consistency of the restored data. For the lost or tampered backup data segments, they are restored according to the redundant information of the backup data to ensure the accuracy of data recovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall principle of the present invention;

[0033] Figure 2 It is a schematic diagram of the overall detailed principle of the present invention;

[0034] Figure 3 This is a schematic diagram of the principle of the data verification module of the present invention;

[0035] Figure 4 It is a schematic diagram of the method flow of the present invention.

[0036] The meaning of each number in the figure is:

[0037] 100, data acquisition module; 200, evaluation matrix establishment module; 210, matrix establishment unit; 220, type selection unit; 300, predefined recovery module; 310, index creation unit; 320, parallel recovery unit; 400, data verification module; 410, comparison checksum unit; 420, redundant recovery unit. DETAILED DESCRIPTION

[0038] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] At present, data backup mainly includes the following types: full backup, incremental backup and differential backup. Full backup is to back up all data, but it takes up a lot of storage space and is not suitable for high frequency. Incremental backup and differential backup are based on the results of the last backup, only backing up updated or changed data. Although it reduces the storage space occupied, the recovery process is more complicated.

[0040] Traditional data backup methods fail to flexibly optimize and select based on multiple factors such as data change frequency, storage space margin, and business cycle, and often fail to balance backup efficiency and storage space utilization. In addition, existing recovery methods are mostly serial recovery, which results in slow recovery speed and difficulty in ensuring data consistency.

[0041] Therefore, the present invention proposes a data backup and rapid recovery processing method and system, which obtains data change frequency, storage space margin and business cycle information through a data acquisition module, and establishes a multi-dimensional evaluation matrix to dynamically select the most suitable data backup strategy, thereby improving backup efficiency and reducing storage requirements. At the same time, during the recovery process, the system adopts parallel recovery technology, combined with index rapid positioning and data verification mechanism, to accelerate recovery and ensure data consistency and integrity, meeting the efficient demand for data backup and recovery in complex application environments.

[0042] The details are as follows:

[0043] See also Figure 1 As shown, it includes a data acquisition module 100, an evaluation matrix establishment module 200, a predefined recovery module 300 and a data verification module 400;

[0044] The data acquisition module 100 identifies the data changes, including the newly added, modified and deleted data contents, through the data change tracking function, and directly obtains the frequency of data changes. The system monitoring tool is used to detect the remaining storage space of the backup data storage device, and the backup data cycle is determined by analyzing the database log according to the previous business cycle. The data acquisition module 100 transmits the data change frequency, remaining storage space and business cycle information to the evaluation matrix establishment module 200;

[0045] The evaluation matrix establishment module 200 establishes a backup strategy evaluation matrix with data change frequency, remaining storage space and business cycle information as three dimensions. For the evaluation results of each backup strategy under each dimension, a comprehensive evaluation score is calculated according to the weight, and the comprehensive evaluation scores of different backup strategies are compared. The backup strategy with the highest score is the optimal choice under the current evaluation environment, and the backup strategies corresponding to different combinations of each dimension are obtained. The corresponding backup strategy is selected according to the information of the corresponding dimension transmitted by the data acquisition module 100, and the data is backed up according to the corresponding backup type, which is one of full backup, incremental backup and differential backup.

[0046] When backing up data, the predefined recovery module 300 creates a backup index for the backup file, and records the checksum of the backup data and the redundant information of the backup data. When restoring data, the backup index is used to quickly locate the storage location of the corresponding backup type. At the same time, the parallel recovery technology is used to restore different parts of data simultaneously using multi-threading or multi-process, thereby improving the recovery speed and significantly shortening the recovery time, and the restored data is transmitted to the data verification module 400.

[0047] The data verification module 400 compares the checksum of the restored data with the checksum of the backup data. If the two are the same, the restored data is complete and correct. If the two are different, the restored data is lost or tampered with. At this time, the backup data is restored based on its redundant information to ensure the accuracy of data recovery.

[0048] like Figure 2 As shown, the evaluation matrix establishment module 200 includes a matrix establishment unit 210 and a type selection unit 220;

[0049] The matrix establishment unit 210 further subdivides the three dimensions of data change frequency, remaining storage space and business cycle information, and subdivides each dimension into different categories, thereby obtaining more detailed evaluation results and establishing a more accurate backup strategy evaluation matrix;

[0050] The type selection unit 220 selects the corresponding backup type according to the detection information of each dimension transmitted by the data acquisition module 100 and the backup strategy evaluation matrix, and starts to back up the data;

[0051] Among them, the business cycle is divided into daily, weekly, monthly, quarterly and other categories; the data update frequency is divided into high, medium and low levels; the backup resource idle time is divided into sufficient, moderate and insufficient levels. The corresponding level of each dimension is used to refine the entire backup strategy evaluation matrix to ensure the accuracy of the backup strategies corresponding to the combinations of different categories, so that the corresponding backup type can be selected more accurately.

[0052] In order to better calculate the comprehensive evaluation score of the corresponding backup type, the matrix establishment unit 210 assigns weights to each evaluation dimension according to the business focus and resource status of the enterprise when establishing the backup strategy evaluation matrix, according to the formula: Get the comprehensive evaluation score of the corresponding backup type;

[0053] Among them, S is the comprehensive evaluation score, A is the score of the data update frequency dimension, is the weight of the data update frequency dimension, B is the score of the remaining storage space dimension, is the weight of the remaining storage space dimension, C is the score of the business cycle dimension, is the weight of the business cycle dimension;

[0054] Assign weights to each evaluation dimension based on business needs and data characteristics. For example, if the data frequency changes greatly, assign a weight to the data change frequency dimension ( ) is 50%, and the weight of the remaining storage space dimension ( ) is 25%, and the weight of the business cycle information dimension ( ) is 25%, and the backup strategy is scored in each dimension. The scoring standard for data update dimension A is established: Full backup (FA) consumes a lot of resources for data with high frequency of update, and is rated 3 points; data with medium frequency of update is rated 5 points; data with low frequency of update is rated 8 points;

[0055] Incremental backup (IA) is rated 8 points for data with high frequency of update, as it can efficiently record changes. For data with medium frequency of update, it is rated 6 points. For data with low frequency of update, it is rated 2 points.

[0056] Differential backup (DA) is rated 6 points for data with high frequency of update, 5 points for data with medium frequency of update, and 3 points for data with low frequency of update.

[0057] Similarly, the remaining storage space dimension B and the business cycle information dimension C establish the same type of scoring criteria, and substitute the weight ratio of the corresponding dimension and the scoring value under the corresponding dimension into the formula: The corresponding comprehensive evaluation scores are obtained and sorted according to the comprehensive evaluation scores. The backup type with the highest comprehensive evaluation score is the most appropriate backup option.

[0058] In order to ensure the integrity of the data when the data backup type is switched, the type selection unit 220 records the data status when the data backup type is switched to ensure that the data at the switching point is complete and accurate. At the same time, a consistency check is performed on the log file and the data file of the database at the switching time to ensure the consistency of the data;

[0059] When performing a consistency check on the database's log files and data files, first check the size and timestamp of the log files. Under normal circumstances, the size of the log files should gradually increase with the operation of the database, and the timestamp should also reflect the most recent write time. If the log file size is abnormal (such as suddenly becoming smaller or not growing) or the timestamp does not match expectations, it may indicate a problem. Then check whether the end mark of the log file is complete. Some database systems will add specific marks at the end of the log file to indicate the normal end of the file. At the same time, you can calculate the checksum of the log file (such as the CRC checksum) and compare it with the checksum recorded during the last backup. If the checksum does not match, it means that the log file may have data loss or damage during transmission or storage.

[0060] The predefined recovery module 300 includes an index creation unit 310 and a parallel recovery unit 320. The index creation unit 310 creates a table-level index for the data when backing up the data. The table-level index records the name of each table, the table structure definition, including the column name, data type, primary key and foreign key, etc., as well as the storage location in the backup file, so as to facilitate rapid positioning of the data when restoring the data.

[0061] The parallel recovery unit 320 decomposes the recovery task into multiple subtasks. Each thread or process is responsible for one subtask. One thread may be responsible for recovering the table structure, and another thread may be responsible for recovering the data records. Multiple threads run simultaneously to improve recovery efficiency.

[0062] In order to better create a backup index, the index creation unit 310 uses a custom script in the relational database MySQL in combination with the information mode information_schema of the database to create a backup index when creating a table-level index. These tables provide detailed information about the database structure and storage. At the same time, the backup index is regularly maintained and updated;

[0063] Among them, the relational database MySQL is an open source relational database management system, which is widely used in various scenarios. It can not only store and manage a large amount of data, support various data types, and provide efficient data storage and retrieval mechanisms, but also can be used to store and analyze a large amount of log data. Through the SQL query language, complex analysis and statistics of log data can be performed;

[0064] Among them, information_schema is an information database in the MySQL database. It provides a standard way to access database metadata. Metadata is data about data, including definitions and properties of objects such as databases, tables, columns, and indexes. By querying the views in information_schema, you can obtain detailed information on the database structure and create backup indexes based on this information.

[0065] In order to improve the recovery speed of backup data, when the parallel recovery unit 320 performs multi-thread or multi-process recovery on data, a lock mechanism is used to ensure that only one thread can access and modify shared resources at any time, and data is coordinated and shared through an inter-process communication mechanism to ensure data consistency;

[0066] Among them, the mutex lock is the most common lock mechanism. Its basic principle is that at the same time, only one thread can obtain the lock. When a thread obtains the mutex lock, other threads trying to obtain the lock will be blocked until the thread holding the lock releases the lock. Parallel recovery technology refers to the use of multiple threads, processes or resources to process different parts of data at the same time during data recovery to speed up recovery. It breaks the traditional sequential recovery mode and makes full use of the system's multi-core processors, multi-disk I / O channels and other hardware resources. The data recovery task is divided into multiple subtasks for parallel execution. By analyzing the structure and content of the backup data, the parts that can be recovered independently are determined. For example, when restoring a backup set containing multiple files, the files can be divided into multiple groups, and each group is assigned a recovery thread. For database recovery, the recovery tasks can be divided according to tables, indexes or data blocks. These parallel recovery tasks can be performed simultaneously. They are independent of each other, but they work together to complete the recovery of the entire data.

[0067] like Figure 3 As shown, the data verification module 400 includes a comparison checksum unit 410 and a redundancy recovery unit 420. The comparison checksum unit 410 calculates the checksum of the backup data by using the MD5 algorithm, compares the checksum of the recovery data with the checksum of the backup data, and verifies the consistency of the recovery data;

[0068] The redundancy recovery unit 420 locates the timestamp, transaction ID and other information in the binary log BinaryLog of the lost data by using the redundant information of the backup data to find the position of the last correct transaction before the data was lost.

[0069] Among them, the checksum is a method for verifying data integrity. It is a fixed-length value calculated by applying a specific algorithm to the data.

[0070] Among them, the MD5 algorithm is a widely used hash function, which is used to convert data of any length into a 128-bit (16-byte) hash value, usually represented by a 32-bit hexadecimal number. First, it fills the input message (data) to a multiple of 512 bits by adding a 1 and several 0s after the message until the message length satisfies the remainder of 448 after taking the modulus of 512, and then adds a 64-bit original message length representation, so that the entire message becomes a multiple of 512 bits in length. Next, the padded message is divided into multiple 512-bit data blocks. The algorithm uses four 32-bit link variables (A, B, C, D), which are initialized to fixed values. Through four rounds of loop operations, each round contains 16 steps, and complex bit operations (including Boolean operations, shift operations, etc.) are performed on each data block. These operations will continuously update the values ​​of the link variables. Finally, the four link variables are spliced ​​together to obtain a 128-bit MD5 hash value, thereby realizing the comparison of the checksum of the restored data with the checksum of the backup data, and then verifying the consistency of the restored data.

[0071] In order to better recover the lost backup data, the redundancy recovery unit 420 uses the "mysqlbinlog" tool in the relational database MySQL to read the binary log and apply the transaction operations therein to the database to restore the database to the state before the data loss;

[0072] mysqlbinlog is a command line tool provided by MySQL database, which is mainly used to read and process binary log files. Binary logs record all operations that modify data in MySQL database, including but not limited to INSERT, UPDATE, and DELETE statements, as well as database structure modifications, such as CREATE TABLE and ALTER TABLE.

[0073] Use the --start-datetime and --stop-datetime parameters to extract transaction operations within the specified time range and save them to the recovery.sql file. After confirming that the transaction operations in the recovery.sql file meet the recovery requirements, you can apply these operations to the database. Use the mysql command to execute the SQL statements in the recovery.sql file and apply the transaction operations to the database to restore the database to the state before the data was lost.

[0074] like Figure 4As shown, the second object of the present invention is to provide a data backup and rapid recovery processing method, including any one of the above data backup and rapid recovery processing systems, including the following steps:

[0075] S1. Collect data change frequency, remaining storage space and business cycle information through the data collection module 100;

[0076] S2, using the evaluation matrix establishment module 200 to establish a backup strategy evaluation matrix with data change frequency, remaining storage space and business cycle information as dimensions, selecting a corresponding backup strategy according to the information of the corresponding dimension transmitted by the data acquisition module 100, and backing up the data according to the corresponding backup type, wherein the backup type is one of full backup, incremental backup and differential backup;

[0077] S3, establishing a backup index for the backup data through the predefined recovery module 300, and recording the checksum and redundancy information of the backup data, and at the same time, using parallel recovery technology to simultaneously restore different parts of data;

[0078] S4. Use the data verification module 400 to compare the checksum of the restored data with the checksum of the backup data to verify the consistency of the restored data.

[0079] In summary, the data backup and rapid recovery processing method and system collects data change frequency, remaining storage space and business cycle information through the data acquisition module 100, and uses the evaluation matrix establishment module 200 to establish a backup strategy evaluation matrix with data change frequency, remaining storage space and business cycle information as dimensions, calculates a comprehensive evaluation score according to the weight, compares the comprehensive evaluation scores of different backup strategies, and the backup strategy with the highest score is the best choice under the current evaluation environment, obtains the backup strategy corresponding to different combinations of each dimension, and selects the corresponding backup strategy according to the information of the corresponding dimension transmitted by the data acquisition module, thereby realizing the ability to select the most appropriate backup strategy under different data conditions. The backup efficiency is improved. When restoring the backup data, a backup index is created for the backup file, and the checksum of the backup data and the redundant information of the backup data are recorded. When restoring the data, the backup index is used to quickly locate the storage location of the corresponding backup type. At the same time, parallel recovery technology is adopted, and different parts of the data are restored simultaneously by multi-threading or multi-process, which improves the recovery speed and significantly shortens the recovery time. After the backup data is restored, the checksum of the restored data is compared with the checksum of the backup data through the data verification module 400 to verify the consistency of the restored data. For the lost or tampered backup data segment, it is restored according to the redundant information of the backup data to ensure the accuracy of data recovery.

[0080] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A data backup and rapid recovery processing system, characterized in that: It comprises a data collection module (100), an evaluation matrix establishment module (200), a predefined recovery module (300) and a data verification module (400); The data acquisition module (100) identifies data changes, including newly added, modified and deleted data content, through a data change tracking function, thereby directly obtaining the frequency of data changes, and uses a system monitoring tool to detect the remaining storage space of a backup data storage device, and determines the backup data cycle by analyzing database logs based on previous business cycles. The data acquisition module (100) transmits the data change frequency, remaining storage space and business cycle information to the evaluation matrix establishment module (200); The evaluation matrix establishment module (200) establishes a backup strategy evaluation matrix with data change frequency, remaining storage space and business cycle information as three dimensions, calculates a comprehensive evaluation score according to the weight for the evaluation results of each backup strategy under each dimension, compares the comprehensive evaluation scores of different backup strategies, and the backup strategy with the highest score is the optimal choice under the current evaluation environment, obtains the backup strategy corresponding to different combinations of each dimension, selects the corresponding backup strategy according to the information of the corresponding dimension transmitted by the data acquisition module (100), and backs up the data according to the corresponding backup type, wherein the backup type is one of full backup, incremental backup and differential backup; The predefined recovery module (300) creates a backup index for the backup file when backing up data, and records the checksum and redundancy information of the backup data; when restoring data, the backup index is used to quickly locate the storage location of the corresponding backup type, and the parallel recovery technology is used to simultaneously restore different parts of data, and the restored data is transmitted to the data verification module (400); The data verification module (400) compares the checksum of the restored data with the checksum of the backup data. If the two are the same, the restored data is complete and correct. If the two are different, the restored data is lost or tampered with. In this case, the backup data is restored based on its redundant information.

2. The data backup and rapid recovery processing system according to claim 1, characterized in that: The evaluation matrix establishment module (200) comprises a matrix establishment unit (210) and a type selection unit (220); The matrix establishment unit (210) further subdivides the three dimensions of data change frequency, remaining storage space and business cycle information, and subdivides each dimension into different categories, thereby obtaining detailed evaluation results and establishing a backup strategy evaluation matrix; The type selection unit (220) selects a corresponding backup type according to the detection information of each dimension transmitted by the data acquisition module (100) and according to the backup strategy evaluation matrix, and starts backing up the data.

3. The data backup and rapid recovery processing system according to claim 2, characterized in that: When establishing the backup strategy evaluation matrix, the matrix establishment unit (210) assigns a weight to each evaluation dimension according to the business focus and resource status of the enterprise, according to the formula: Get the comprehensive evaluation score of the corresponding backup type; Among them, S is the comprehensive evaluation score, A is the score of the data update frequency dimension, is the weight of the data update frequency dimension, B is the score of the remaining storage space dimension, is the weight of the remaining storage space dimension, C is the score of the business cycle dimension, is the weight of the business cycle dimension.

4. The data backup and rapid recovery processing system according to claim 2, characterized in that: The type selection unit (220) records the data status when the data backup type is switched, and at the same time, performs a consistency check on the log files and data files of the database at the switching time.

5. The data backup and rapid recovery processing system according to claim 1, characterized in that: The predefined recovery module (300) comprises an index creation unit (310) and a parallel recovery unit (320). When backing up data, the index creation unit (310) creates a table-level index for the data. The table-level index records the name of each table, the table structure definition, including the column name, data type, primary key and foreign key, and the storage location in the backup file. The parallel recovery unit (320) decomposes the recovery task into multiple subtasks, each thread or process is responsible for one subtask, one thread is responsible for recovering the table structure, and another thread is responsible for recovering the data records.

6. The data backup and rapid recovery processing system according to claim 5, characterized in that: When creating a table-level index, the index creation unit (310) uses a custom script in the relational database MySQL in combination with the information mode information_schema of the database to create a backup index. These tables include detailed information on the database structure and storage. At the same time, the backup index is regularly maintained and updated.

7. The data backup and rapid recovery processing system according to claim 5, characterized in that: When the parallel recovery unit (320) performs multi-thread or multi-process recovery on data, a lock mechanism is used to ensure that only one thread can access and modify shared resources at any time.

8. The data backup and rapid recovery processing system according to claim 1, characterized in that: The data verification module (400) comprises a comparison checksum unit (410) and a redundancy recovery unit (420), wherein the comparison checksum unit (410) calculates the checksum of the backup data by using the MD5 algorithm, compares the checksum of the recovery data with the checksum of the backup data, and verifies the consistency of the recovery data; The redundant recovery unit (420) locates the timestamp and transaction ID information in the binary log of the lost data by using the redundant information of the backup data, and finds the position of the last correct transaction before the data is lost.

9. The data backup and rapid recovery processing system according to claim 8, characterized in that: The redundancy recovery unit (420) uses the mysqlbinlog tool in the relational database MySQL to read the binary log and apply the transaction operations in the binary log to the database, so as to restore the database to the state before the data is lost.

10. A method for implementing data backup and rapid recovery processing, applied to the data backup and rapid recovery processing system according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1, collecting data change frequency, remaining storage space and business cycle information through a data collection module (100); S2, using the evaluation matrix establishment module (200) to establish a backup strategy evaluation matrix with data change frequency, remaining storage space and business cycle information as dimensions, selecting a corresponding backup strategy according to the information of the corresponding dimension transmitted by the data acquisition module (100), and backing up the data according to the corresponding backup type, wherein the backup type is one of full backup, incremental backup and differential backup; S3, establishing a backup index for the backup data through a predefined recovery module (300), and recording the checksum of the backup data and redundant information of the backup data, and at the same time, using a parallel recovery technology to simultaneously restore different parts of data; S4. Using the data verification module (400), the checksum of the restored data is compared with the checksum of the backup data to verify the consistency of the restored data.

Images