A data processing method, device and storage medium

Abstract

The embodiment of the present application discloses a data processing method, device and storage medium, and the data processing method includes: obtaining a backup file of a database, the backup file is obtained by backing up a production instance of the database; performing backup recovery processing on the database according to the backup file to obtain a recovery instance of the database; extracting business data in the production instance and business data in the recovery instance according to business verification rules, and verifying the consistency between the business data in the production instance and the business data in the recovery instance; if the business data in the production instance is consistent with the business data in the recovery instance, confirming that the backup file is a valid file. By adopting the embodiment of the present application, the reliability of the backup file validity verification process can be improved, and the accuracy of the backup file validity verification result can be further improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to the field of data recovery technology, and in particular to a data processing method, a data processing device, and a computer-readable storage medium. Background Art

[0002] With the development of science and the advancement of technology, various business platforms (such as payment, video playback and other business platforms) have gradually increased their requirements for data reliability. As the requirements for data reliability of various business platforms gradually increase, it is particularly important to back up the data in the business platform database. Once a business platform fails, the backup files obtained can quickly restore important data in the business platform database.

[0003] In addition, in the process of using backup files for data recovery, the validity of the backup files is equally important; if the backup file is an invalid file (for example, the backup file is damaged, the data in the backup file is lost, etc.), then the data restored based on the backup file will be unreliable, and the invalid backup file does not meet the data recovery requirements of the business platform; therefore, how to verify the validity of the backup file has become a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, device and storage medium, which can improve the reliability of the backup file validity verification process and further improve the accuracy of the backup file validity verification results.

[0005] On the one hand, an embodiment of the present application provides a data processing method, the data processing method comprising:

[0006] Get the backup file of the database. The backup file is obtained by backing up the production instance of the database.

[0007] Perform backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database;

[0008] Extract the business data in the production instance and the business data in the recovery instance according to the business verification rules, and verify the consistency between the business data in the production instance and the business data in the recovery instance;

[0009] If the business data in the production instance is consistent with the business data in the recovery instance, the backup file is confirmed to be a valid file.

[0010] On the one hand, an embodiment of the present application provides a data processing device, the data processing device comprising:

[0011] A management unit configured to schedule an execution unit to perform a backup and recovery task;

[0012] an execution unit, configured to obtain a backup file of the database according to the scheduling of the management unit, and perform backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database; and

[0013] A management unit configured to schedule an execution unit to perform a backup verification task;

[0014] An execution unit is configured to extract business data in the production instance and business data in the recovery instance according to the scheduling of the management unit and the business verification rule, and verify the consistency between the business data in the production instance and the business data in the recovery instance, and return the data verification result to the management unit;

[0015] The management unit is configured to confirm that the backup file is a valid file if the data verification result indicates that the business data in the production instance is consistent with the business data in the recovery instance.

[0016] In one implementation, the data processing device further includes:

[0017] A storage unit, the storage unit includes a backup storage unit and a task storage unit; the backup storage unit is configured to store backup files of the database, and the task storage unit is configured to store multiple tasks in the database; the backup storage unit is equipped with a distributed file system, and the task storage unit is equipped with a relational database management system cluster;

[0018] The registration unit is configured to perform registration processing on the execution unit.

[0019] In one implementation, there are multiple execution units; when performing scheduling, the management unit selects an idle execution unit from the multiple execution units to execute the backup recovery task or the backup verification task.

[0020] On the one hand, an embodiment of the present application provides another data processing device, the data processing device comprising:

[0021] a processor adapted to implement a computer program; and,

[0022] A computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned data processing method.

[0023] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned data processing method.

[0024] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above-mentioned data processing method.

[0025] In the embodiment of the present application, the backup file of the database is obtained by backing up the production instance of the database, and the database is backed up and restored according to the backup file of the database to obtain the restored instance of the database; the business data in the production instance and the business data in the restored instance are extracted according to the business verification rules, and the business data extracted from the production instance and the business data extracted from the restored instance are verified to determine whether the business data in the production instance is consistent with the business data in the restored instance; if the business data in the production instance is consistent with the business data in the restored instance, the backup file is confirmed to be a valid file. In the above process, the backup file of the database is restored through a complete recovery process to obtain the restored instance of the database. If the business data restored in the restored instance of the database is consistent with the business data actually produced by the production instance of the database, then the backup file is a valid file, and the recovery process of the backup file is also valid, which improves the reliability of the backup file validity verification process and further improves the accuracy of the backup file validity verification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A schematic diagram of a data processing architecture provided by an exemplary embodiment of the present application is shown;

[0028] Figure 2 A schematic diagram of the structure of a data processing device provided by an exemplary embodiment of the present application is shown;

[0029] Figure 3 A functional schematic diagram of a data processing device provided by an exemplary embodiment of the present application is shown;

[0030] Figure 4 A schematic diagram of a data processing method provided by an exemplary embodiment of the present application is shown;

[0031] Figure 5A flowchart of a backup and recovery process provided by an exemplary embodiment of the present application is shown;

[0032] Figure 6 A schematic diagram of the architecture of a data processing process provided by an exemplary embodiment of the present application is shown;

[0033] Figure 7 A schematic diagram showing a data verification time range provided by an exemplary embodiment of the present application is shown;

[0034] Figure 8 A schematic diagram showing a data verification time range provided by another exemplary embodiment of the present application is shown;

[0035] Fig. 9 A flowchart of a backup verification process provided by an exemplary embodiment of the present application is shown;

[0036] Fig.10 A schematic structural diagram of a data processing device provided by another exemplary embodiment of the present application is shown.

[0037] Specific implementation

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] The embodiments of the present application relate to a database, which is usually set in a business platform to support the normal operation of the business platform and manage and save relevant business data of the business platform. Here, the business platform may include but is not limited to any of the following: an instant messaging business platform, a game business platform, an e-commerce business platform, a payment platform, a video playback platform, etc. Business data refers to data used for business interaction in a business platform, such as payment data and order data of users of each platform in a payment platform; or, video data on demand of users of each platform in a video playback platform, etc. In general, a database is provided with a data backup mechanism. The so-called data backup refers to the process of copying and duplicating the production instance of the database to obtain a backup file in order to prevent data loss caused by operational errors or failures in the database. When a data loss failure occurs in the database, important business data in the database can be quickly restored through the backup file to ensure the normal operation of the business platform.

[0040] As mentioned above, the backup file is an important file that supports the normal operation of the business platform when data loss or other failures occur in the database. In order to enable the backup file to function normally, the embodiment of the present application provides a data processing solution, which can be used to provide a complete backup file validity verification service to the database of the business platform to ensure that the backup file providing the service is a valid file, thereby ensuring the normal operation of the business platform.

[0041] See also Figure 1 , Figure 1 A schematic diagram of a data processing architecture provided by an exemplary embodiment of the present application is shown, and the data processing architecture includes a service platform 101 and a data processing device 102. Among them:

[0042] The database of the business platform 101 is used to support the normal operation of the business platform and to manage and save the relevant business data of the business platform. The database involved in the embodiment of the present application may be MySQL (a relational database management system (RDMS)). The database of the business platform may include a master node and at least one slave node (for example, the database includes a master node and N slave nodes, N is a positive integer); when the master node of the database operates normally, the master node of the database provides relevant business services for the business platform; each slave node in the database corresponds to a switching priority. When the master node of the database operates abnormally, the database needs to execute the master-slave node switching. The database selects and starts a slave node from at least one slave node in the order of switching priority from high to low, and the selected slave node continues to provide business services for the business platform. In one implementation, the switching priority of at least one slave node included in the database can be divided according to the performance of each slave node. For example, if the CPU (Central Processing Unit) processing performance of slave node 1 is better than the CPU processing performance of slave node 2, then the switching priority of slave node 1 is higher than the switching priority of slave node 2; for another example, if the network connection rate between slave node 2 and the business platform is better than the network connection rate between slave node 3 and the business platform, then the switching priority of slave node 2 is higher than the switching priority of slave node 3.

[0043] The data processing device 102 is used to execute the data processing solution proposed in the embodiment of the present application, and provide a complete backup file validity verification service to the database of the business platform. Figure 2-Figure 3 The illustrated embodiment provides a detailed introduction to a data processing device provided in an embodiment of the present application.

[0044] Figure 2A schematic diagram of the structure of a data processing device provided by an exemplary embodiment of the present application is shown. The data processing device 102 may include:

[0045] (1) Task storage unit 201 (system library)

[0046] The task storage unit generates business tasks at fixed intervals. The business tasks generated by the task storage unit may include but are not limited to at least one of the following: full backup tasks, incremental backup tasks, permanent backup (also known as permanent incremental backup) tasks, backup recovery tasks, and backup verification tasks; the task storage unit stores the task data of each business task, and the execution results of each business task are also stored in the task storage unit. Among them, the task storage unit involved in the embodiment of the present application is equipped with a relational database management system cluster (MySQL cluster), which is built by MGR (MySQL Group Replication); MGR is a MySQL plug-in that provides high-availability, high-scalability, and high-reliability MySQL cluster services for data processing equipment.

[0047] (2) Management unit 202 (Manager)

[0048] The management unit reads the business task generated by the task storage unit from the task storage unit, and schedules the execution unit 203 to execute the business task; after the execution unit executes the business task, the management unit also writes the execution result of the business task received by the execution unit into the task storage unit.

[0049] In one implementation, there are multiple management units in the data processing device, and the data processing device includes a master management unit and at least one slave management unit, such as Figure 2 As shown, the data processing device includes a main management unit and two slave management units (respectively, slave management unit 1 and slave management unit 2); usually, the main management unit schedules the execution unit to perform the business task; when the main management unit fails, the coordination unit 206 selects a slave management unit from at least one slave management unit included in the data processing device, and the selected slave management unit schedules the execution unit to perform the business task; for example, the data processing device includes a main management unit and two slave management units (respectively, slave management unit 1 and slave management unit 2), when the main management unit fails, the coordination unit selects an idle slave management unit 1 from the two slave management units included in the data processing device, and the slave management unit 1 schedules the execution unit to perform the business task; the so-called idle slave management unit 1 refers to the current system time when the slave management unit 1 does not perform the scheduling operation, and the current system time refers to the time displayed when the coordination unit is running when selecting the slave management node.

[0050] In one implementation, the data processing device includes multiple execution units, and the management unit selects an idle execution unit from the multiple execution units to execute a business task when scheduling; the so-called idle execution unit refers to an execution unit that has not executed a business task at the current system time, and the current system time refers to the time displayed by the data processing device when the management unit performs scheduling.

[0051] (3) Execution unit 203 (Actuator)

[0052] The execution unit executes the business task generated by the task storage unit according to the scheduling of the management unit, and returns the execution result of the business task to the management unit; for example, the execution unit executes the backup and recovery task according to the scheduling of the management unit, and returns the execution result of the backup and recovery task to the management unit, and the execution result of the backup and recovery task can be used to indicate the success of the backup and recovery or to indicate the failure of the backup and recovery; or, the execution unit executes the backup verification task according to the scheduling of the management unit, and returns the execution result of the backup verification task to the management unit, and the execution result of the backup verification task can be used to indicate the success of the backup verification or to indicate the failure of the backup verification.

[0053] (4) Backup storage unit 204 (HDFS)

[0054] In order to prevent the loss of data due to operational errors or failures in the database, which may lead to the stagnation of the business platform operation, the database will generally perform data backup. The so-called data backup refers to the process of copying and duplicating the production instance of the database; the production instance of the database refers to the database instance that maintains the normal operation of the database; the production instance of the database is backed up to obtain the backup file. Data backup can include two processes: full backup and incremental backup. Full backup (FullBackup) refers to the consistent backup of the production instance of the database through the full backup tool (such as mydumper), and incremental backup (Incremental Backup) refers to the backup of the binary log (binlog) pulled by the log analysis process (such as mysqlbinlog server). The binary log records all the edits (including additions, modifications, and deletions) performed on the database after the last backup (i.e. the last full backup or incremental backup); accordingly, the backup files obtained by the backup may include full files and incremental files. The full file is obtained by performing a full backup of the production instance of the database, and the incremental file is obtained by performing an incremental backup of the production instance of the database.

[0055] In an embodiment of the present application, the backup file can be stored in a backup storage unit of the data processing device, that is, the backup storage unit can be used to store the backup file; when the backup file includes a full file and an incremental file, the backup storage unit divides different storage spaces for the full file and the incremental file, that is, the storage locations of the full file and the incremental file in the backup storage unit are different. In one implementation, when the execution unit executes the data backup task according to the scheduling of the management unit, if the execution result of the data backup task indicates that the execution unit has successfully backed up the database of the data processing device, the execution unit returns the execution result of the data backup task to the management unit, and the execution unit also sends the backup file obtained by backing up the database to the backup storage unit, and the backup storage unit stores the backup file. Among them, the backup storage unit involved in the embodiment of the present application is equipped with a distributed file system (Hadoop Distributed File System, HDFS), which refers to a distributed file system designed to run on general-purpose hardware (Commodity Hardware). In one implementation, the data processing device may be a distributed device integrated by a plurality of hardware units, and the backup storage unit is an independent hardware unit in the data processing device; when the backup storage unit is an independent hardware unit in the data processing device, the backup storage unit may be a node in the blockchain network, and the backup storage unit may package the backup files into blocks and upload the blocks to the blockchain network; wherein, the blockchain refers to a decentralized infrastructure with distributed storage characteristics, specifically a data structure that organizes data blocks in a time sequence in a manner similar to a linked list, which can safely store data that has a sequential relationship and can be verified within the system, and cryptographically ensure that the data cannot be tampered with or forged; the blockchain network is composed of a plurality of nodes, and each node in the blockchain network is networked in a P2P (Peer-to-Peer, point-to-point transmission) manner, and the nodes communicate with each other according to the P2P protocol, and each node jointly follows the broadcast mechanism, the consensus mechanism (including the POW (Proof of Work, proof of work) mechanism, and the POS (Proof of Stake, proof of equity) mechanism and other core mechanisms) jointly ensure that the data on the blockchain cannot be tampered with or forged, while realizing the decentralization and trustlessness of the blockchain. The backup storage unit uploads the backup file to the blockchain network, and the backup file cannot be tampered with, making the backup file safe and reliable.

[0056] (5) Registration Unit 205 (Eureka)

[0057] The registration unit is used to perform registration processing on multiple execution units in the data processing device; for example, the registration center can register the execution units in the data processing device that are not executing business tasks as idle execution units, so that the management unit can select idle execution units from multiple execution units to execute business tasks; or, the registration center can register execution units that have abnormalities in the process of executing business tasks as abnormal execution units, so as to avoid the management unit from mistakenly selecting abnormal execution units as idle execution units when selecting idle execution units from multiple execution units.

[0058] (6) Coordination unit 206 (ZooKeeper)

[0059] The coordination unit provides coordination services for the master and slave management units of the data processing device; the so-called coordination service for the master and slave management units refers to a service for selecting a slave management unit from at least one slave management unit included in the data processing device when an abnormality occurs in the master management unit. Among them, the coordination unit involved in the embodiment of the present application is equipped with a distributed application coordination service (ZooKeeper).

[0060] (7) Transmission unit 207 (Upload)

[0061] The transmission unit is used to upload the backup file to the backup storage unit or download the backup file from the backup storage unit.

[0062] It should be noted that the above seven units (i.e., task storage unit, management unit, execution unit, backup storage unit, registration unit, coordination unit and transmission unit) are Figure 2 The data processing device shown is an important component unit for implementing the data processing solution; the data processing device also includes multiple application programming interfaces 208 (Application Programming Interface, API) related to the data processing process, for example, the data processing device also includes an application programming interface (FMHA-API) for calling the instance information of a database instance (such as a production instance of a database or a recovery instance of a database), an application programming interface (FHMA) for calling authorization / package management, and an application programming interface (Aegis) for calling the data verification involved in the backup verification process. In addition, Figure 2 The data processing device shown also provides a front-end page 209 (web page), which provides network services (such as network services provided by nginx) through corresponding application program interfaces, and can access the management unit of the data processing device through the front-end page to implement task scheduling. Figure 3 As shown, Figure 3A functional schematic diagram of a data processing device provided by an exemplary embodiment of the present application is shown, in which the front-end page includes UI (User Interface) elements, a module packager (webpack) and ECMAScript (European Computer Manufacturers Association Script, a script program for the front-end page); wherein the UI elements include various types of controls (such as menu bars, edit boxes, etc.); the module packager is used to package ECMAScript script programs.

[0063] Based on the above Figure 2-Figure 3 The structure of the data processing device shown in the figure can execute the data processing scheme proposed in the embodiment of the present application, and provide a complete backup file validity verification service to the database of the business platform. The validity verification service may include data backup, backup recovery and backup verification services. The so-called backup recovery refers to the process of obtaining a recovery instance of a database through backup file recovery. The so-called backup verification refers to the process of performing consistency verification on the business data in the production instance and the business data in the recovery instance after obtaining a recovery instance of a database through backup file recovery; if the business data in the production instance is consistent with the business data in the recovery instance, the backup file is a valid file; if the business data in the production instance is inconsistent with the business data in the recovery instance, the backup file is an invalid file; a valid backup file can be used to restore valid and reliable business data. When an operational error or failure occurs in the database of the business platform, resulting in data loss, the data processing device can quickly restore important business data in the database through a valid backup file to support the normal operation of the business platform.

[0064] The following describes the data backup process, backup recovery process, and backup verification process respectively:

[0065] (1) Data backup process

[0066] During the data backup process of the data processing device, the task storage unit generates a data backup task, and the task data of the data backup task is stored in the task storage unit. The task data of the data backup task may include but is not limited to any of the following: instance information of the production instance of the database to be backed up, and the time period for the production instance in the database to be backed up. The management unit reads the data backup task from the task storage unit, and selects an idle execution unit (such as a target execution unit) from multiple execution units included in the data processing device; the management unit schedules the target execution unit to execute the data backup task. The target execution unit backs up the production instance of the database according to the instance information of the production instance of the database to be backed up or the time period for the production instance in the database to be backed up as scheduled by the management unit.

[0067] In one implementation, if the target execution unit is successful in backing up the database of the data processing device, the target execution unit returns the execution result of the data backup task to the management unit, the execution result of the data backup task is used to indicate that the execution unit has successfully backed up the database of the data processing device, and the management unit writes the execution result of the data backup task into the task storage unit; and the target execution unit sends the backup file obtained by the backup to the backup storage unit, and the backup storage unit stores and saves the backup file. In another implementation, if the target execution unit fails to back up the database of the data processing device, the execution unit returns the execution result of the data backup task to the management unit, the execution result of the data backup task is used to indicate that the database backup of the data processing device has failed, and the management unit writes the execution result of the data backup task into the task storage unit.

[0068] (2) Backup and recovery process

[0069] During the backup and recovery process of the data processing device, the task storage unit generates a backup and recovery task, and the task data of the backup and recovery task is stored in the task storage unit. The task data of the backup and recovery task may be, but is not limited to, any of the following: the file identifier of the backup file to be restored, the timestamp of the backup file to be restored. The management unit reads the backup and recovery task from the task storage unit, and selects an idle execution unit (such as a target execution unit) from a plurality of execution units included in the data processing device; the management unit schedules the target execution unit to execute the backup and recovery task. The target execution unit obtains the backup file of the database from the backup storage unit according to the file identifier of the backup file to be restored or the timestamp of the backup file to be restored as scheduled by the management unit, and performs backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database.

[0070] (3) Backup verification process

[0071] In the backup verification process of the data processing device, the task storage unit generates a backup verification task, the task data of the backup verification task is stored in the task storage unit, and the task data of the backup verification task may include a business verification rule. The management unit reads the backup verification task from the task storage unit, and selects an idle execution unit (such as a target execution unit) from multiple execution units contained in the data processing device; the management unit schedules the target execution unit to execute the backup verification task. The target execution unit extracts the business data in the production instance and the business data in the recovery instance according to the business verification rule according to the scheduling of the management unit, and verifies the consistency between the business data in the production instance and the business data in the recovery instance; if the business data in the production instance is consistent with the business data in the recovery instance, the target execution unit returns the execution result of the backup verification task (i.e., the data verification result) to the management unit, and the execution result of the backup verification task is used to indicate that the business data in the production instance is consistent with the business data in the recovery instance; if the business data in the production instance is inconsistent with the business data in the recovery instance, the target execution unit returns the execution result of the backup verification task to the management unit, and the execution result of the backup verification task is used to indicate that the business data in the production instance is inconsistent with the business data in the recovery instance. The management node device writes the execution result of the backup verification task into the task storage unit.

[0072] In one implementation, if the execution result of the backup verification task indicates that the business data in the production instance is consistent with the business data in the recovery instance, the management unit confirms that the backup file is a valid file, and the management unit can schedule the backup storage unit to add a valid tag to the valid backup file, and the valid tag is used to indicate that the backup file is a valid file. In another implementation, if the execution result of the backup verification task indicates that the business data in the production instance is inconsistent with the business data in the recovery instance, the management unit confirms that the backup file is an invalid file, and the management unit can schedule the backup storage unit to add an invalid tag to the invalid backup file, and the invalid tag is used to indicate that the backup file is an invalid file; the management unit can also schedule the backup storage unit to delete the invalid backup file from the backup storage unit.

[0073] In an embodiment of the present application, a task storage unit in a data processing device is used to generate business tasks (such as data backup tasks, backup recovery tasks, backup verification tasks, and other business tasks) and store task data of the business tasks; a management unit in the data processing device is used to read business tasks from the task storage unit, schedule execution units to execute business tasks, and write the execution results of business tasks returned by the execution units into the task storage unit; the execution units in the data processing device are used to execute business tasks according to the scheduling of the management unit; the backup storage unit in the data processing device is used to store backup files; the registration unit in the data processing device is used to register multiple execution units in the data processing device; the coordination unit in the data processing device is used to provide coordination services when the master and slave management units of the data processing device are switched; the transmission unit in the data processing device is used to upload backup files to the backup storage unit or download backup files from the backup storage unit; through the collaboration and cooperation between the various units in the data processing device, a complete set of backup file validity verification processes is provided, the reliability of the backup file validity verification process is improved, and the accuracy of the backup file validity verification results is further improved.

[0074] It can be understood that the data processing device described in the embodiment of the present application is for more clearly illustrating the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can know that with the evolution of the device structure and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0075] Based on the above description, see Figure 4 , Figure 4 A schematic diagram of a data processing method provided by an exemplary embodiment of the present application is shown. The method can be performed by Figure 2 The data processing device in the embodiment shown in the figure performs the data processing method, which includes the following steps S401 to S405:

[0076] Step S401, obtaining a backup file of the database.

[0077] In one implementation, the backup file of the database is obtained by backing up a production instance of the database, and the backup file of the database is stored in a backup storage unit of a data processing device.

[0078] In one implementation, after obtaining the backup file of the database, the integrity of the backup file needs to be verified; if the verification succeeds, that is, the backup file is a complete backup file, then the database is backed up and restored according to the backup file to obtain a restored instance of the database; if the verification fails, that is, the backup file is an incomplete backup file, then the backup file is deleted from the backup storage unit. Generally, the database of the data processing device will prevent the backup and restoration process of an incomplete backup file. Specifically, the backup file is obtained from the backup storage unit, and the backup file may include a full file and an incremental file. The integrity verification process of the backup file may include at least one of the following:

[0079] (1) Check whether the backup file is damaged. If the backup file is not damaged, the verification is successful. If the backup file is damaged, the verification fails. Specifically, the embodiment of the present application checks whether the backup file is damaged by RMAN (Recovery Manager) verification. In the process of checking whether the backup file is damaged by RMAN verification, the following five operations may exist: ① Closing the access rights of the backup file during the recovery of the backup file; ② Only one recovery operation is allowed at a time during the recovery of the backup file; ③ Ensuring the orderliness of the incremental files in the backup file during the recovery of the backup file; ④ In the process of recovering the backup file, storing the storage information used to check whether the backup file is damaged in the backup file; ⑤ In the process of recovering the backup file, checking each read and write operation performed on the data blocks in the backup file to facilitate checking whether there are damaged data blocks in the backup file.

[0080] (2) Check whether the backup file is correct. If it is correct, the verification is successful. If it is not correct, the verification fails. Specifically, compare the summary information of the acquired backup file with the summary information of the backup file stored in the backup storage unit to see if they are the same. If they are the same, the backup file is correct and the verification is successful. If they are not the same, the backup files are different and the verification fails. The summary information of the backup file is calculated by a hash algorithm; wherein the hash algorithm may include MD5 (Message-Digest Algorithm 5) algorithm, SHA (Secure Hash Algorithm) 256 algorithm, SHA512 algorithm, etc.

[0081] (3) Verify whether the incremental files in the backup file are played back normally. If so (for example, playback is not interrupted), the verification is successful. If not (for example, playback is interrupted), the verification fails. Specifically, the application log is used to verify whether the incremental files in the backup file are played back normally.

[0082] (4) Check whether the data page of the backup file is complete. If it is complete (for example, the data in the data page of the backup file is not lost), the verification is successful. If it is incomplete (for example, the data in the data page of the backup file is lost), the verification fails. Specifically, the check table (for example, CHECK TABLE) syntax function provided by MySQL is used to check whether each data page in the backup file is complete.

[0083] Step S402: Perform backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database.

[0084] In one implementation, the backup file includes a full file; the specific implementation method of performing backup and recovery processing on the database based on the full file can be: determining the recovery time point for backing up and restoring the full file; checking whether the full file exists in the backup storage unit; if so, determining an idle execution unit and downloading the full file from the backup storage unit; creating a new instance of the database; decompressing the full file, and importing the decompressed full file into the new instance through a backup and recovery tool (such as myloader) to obtain a recovery instance of the database.

[0085] In another implementation, the backup file includes a full file and an incremental file; the process of performing backup and recovery processing on the database according to the full file and the incremental file can be seen in Figure 5 , Figure 5 A flowchart of a backup and recovery process provided by an exemplary embodiment of the present application is shown. The specific implementation includes the following steps S501 to S506:

[0086] Step S501, determine the recovery time point for backing up and restoring the full files and incremental files; the recovery time point can be determined based on the business recovery requirements of the business platform. For example, the recovery time point is set within the idle time period of the business platform. At this time, less business data is produced in the production instance of the database of the business platform, which reduces the data processing pressure of the data processing device when performing the leveling replication processing between the target slave node and the cascade node of the database.

[0087] Step S502, checking whether the full file and the incremental file exist in the backup storage unit.

[0088] Step S503: If yes, determine the idle execution unit.

[0089] Step S504, downloading the full file and the incremental file from the backup storage unit.

[0090] Step S505, creating a new instance of the database, decompressing the full file, and importing the decompressed full file into the new instance through a backup and recovery tool.

[0091] Step S506, copy the binary log (binlog) in the incremental file to the relay log (relaylog) of the new instance; replay the relay log in the new instance after the copy is completed to obtain the restored instance of the database.

[0092] In one implementation, the database includes a master node and at least one slave node, each slave node corresponds to a switching priority, and when the database needs to perform a master-slave node switch, the slave node is selected and started in descending order of the switching priority; Figure 6 As shown, Figure 6 An architectural diagram of a data processing process provided by an exemplary embodiment of the present application is shown, wherein the database includes a master node and two slave nodes (slave node 1 and slave node 2), the switching priority of slave node 1 is higher than the switching priority of slave node 2, and when the database needs to perform a master-slave node switch, slave node 1 is selected and started in order of switching priority from high to low. In one implementation, after performing backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database, it is also necessary to select a target slave node from at least one slave node in order of switching priority from low to high, the production instance of the database is located in the target slave node, and the recovery instance is attached to the target slave node, a cascade node of the target slave node is generated, and a leveling replication process is performed between the target slave node and the cascade node. Figure 6 As shown, after performing backup and recovery processing on the database according to the backup file to obtain the recovery instance of the database, select slave node 2 from the two slave nodes in the order of switching priority from low to high, the production instance of the database is located in slave node 2, and the recovery instance is attached to slave node 2, a cascade node of slave node 2 is generated, and a leveling replication process is performed between slave node 2 and the cascade node. The so-called leveling replication process refers to the process of copying the binary log in the target slave node to the relay log of the new instance, and replaying the relay log of the new instance after the replication is completed.

[0093] Step S403: extracting the business data in the production instance and the business data in the recovery instance according to the business verification rule.

[0094] In one implementation, the business verification rules include a data verification time range; the business data in the production instance and the business data in the recovery instance both contain a timestamp; the specific implementation method for extracting the business data in the production instance and the business data in the recovery instance according to the business verification rules can be: extracting the first business data whose timestamp is within the data verification time range from the production instance, and extracting the second business data whose timestamp is within the data verification time range from the recovery instance.

[0095] In another implementation, the business verification rules include a data verification time range and a data verification type; the business data in the production instance and the business data in the recovery instance both contain a timestamp; the specific implementation method for extracting the business data in the production instance and the business data in the recovery instance according to the business verification rules can be: extracting the first business data whose timestamp is within the data verification time range and whose business type is the data verification type from the production instance, and extracting the second business data whose timestamp is within the data verification time range and whose business type is the data verification type from the recovery instance.

[0096] Among them, the production instance or the recovery instance may include business data of multiple business types, and the data verification type refers to any one or more business types included in the production instance or the recovery instance; if the business platform supported by the data processing device is a payment platform, the data verification type may include at least one of the following: the user name of the platform user of the payment platform, the payment order number, and the payment amount; if the business platform supported by the data processing device is a video playback platform, the data verification type may include at least one of the following: the user name of the platform user of the video playback platform, and the video information requested by the platform user (such as video name, video duration, video clarity, etc.).

[0097] In one implementation, the backup file includes a full file, and the data verification time range when the backup file includes the full file includes at least one of the following:

[0098] (1) Full verification time range. The full verification time range is used to verify the validity of the business data in the recovery instance obtained through full file recovery; wherein, the full verification time range refers to the time range formed by the target time period before and after the time point N days before the full backup snapshot time point, the full backup snapshot time point refers to the time point when the full file is snapshotted after the full backup is completed, and the snapshot refers to the record of the data status of the full file; N is an integer greater than 0, and the value of N can be set according to an empirical value, for example, N is 1; the target time period can be set according to an empirical value, for example, the target time period is 5 minutes. Figure 7 A schematic diagram of a data verification time range provided by an exemplary embodiment of the present application is shown as follows: Figure 7 As shown in the figure, the full verification time range refers to the time range formed by 5 minutes before and after the time point 1 day before the full backup snapshot time point (time point 1).

[0099] (2) The time range for checking the leveling replication. The time range for checking the leveling replication is used to verify the validity of the business data in the recovery instance after the leveling replication is performed between the target slave node and the cascade node; the time range for checking the leveling replication refers to the time range formed by the target time period before and after the leveling replication time point. Figure 7As shown, the time range for balancing replication and verification refers to the time range formed by 5 minutes before and after the balancing replication time point (time point 2).

[0100] In this implementation, the first business data extracted from the production instance and having a timestamp within the data verification time range includes: the business data in the production instance with a timestamp within the full verification time range and the business data in the production instance with a timestamp within the leveling copy verification time range. The second business data extracted from the recovery instance and having a timestamp within the data verification time range includes: the business data in the recovery instance with a timestamp within the full verification time range and the business data in the recovery instance with a timestamp within the leveling copy verification time range.

[0101] In this way, the data verification time range covers important critical time points in the backup and recovery process and the backup verification process (such as the time point of successful full backup and recovery and the time point of catching up with the replication). By performing consistency verification on the business data in the production instance and the business data in the recovery instance extracted within the data verification time range, the accuracy of the backup file validity verification results is further improved.

[0102] In another implementation, the backup file includes a full file and an incremental file, and the data verification time range when the backup file includes the full file and the incremental file includes at least one of the following:

[0103] (1) Time range for full verification. Figure 8 A schematic diagram of a data verification time range provided by another exemplary embodiment of the present application is shown. Figure 8 As shown in the figure, the full verification time range refers to the time range formed by 5 minutes before and after the time point 1 day before the full backup snapshot time point (time point 1).

[0104] (2) The critical verification time range between the full backup and the incremental backup. The critical verification time range between the full backup and the incremental backup refers to the time range formed by the target time period before and after the full backup snapshot time point; the full backup ends within the time range formed by the target time period before the full backup snapshot time point, and the incremental backup just begins within the time range formed by the target time period after the full backup snapshot time point; Figure 8 As shown in the figure, the critical verification time range between the full and incremental data refers to the time range formed by 5 minutes before and after the full snapshot time point (time point 2). The critical verification time range between the full and incremental data is used to verify the validity of the business data in the recovery instance obtained by full file recovery, and to verify the validity of the business data in the recovery instance obtained by full file and incremental file (within the target time period).

[0105] (3) The critical verification time range between incremental and catch-up replication. The critical verification time range between incremental and catch-up replication refers to the time range formed by the target time period before and after the incremental backup end time point; the incremental backup ends within the time range formed by the target time period before the incremental backup end time point, and the catch-up replication process just begins within the time range formed by the target time period after the incremental backup end time point; Figure 8 As shown, the critical verification time range between incremental and catch-up replication refers to the time range formed by 5 minutes before and after the incremental backup end time point (time point 3). The critical verification time range between incremental and catch-up replication is used to verify the validity of business data in the recovery instance obtained by restoring the full file and incremental file, and to verify the validity of business data in the recovery instance obtained by performing catch-up replication processing (within the target time period) between the target slave node and the cascade node.

[0106] (4) To match the time range of the copy verification. Figure 8 As shown, the time range for balancing replication and verification refers to the time range formed by 5 minutes before and after the balancing replication time point (time point 4).

[0107] In this implementation, the first business data extracted from the production instance and having a timestamp within the data verification time range includes: business data in the production instance with a timestamp within the full verification time range, business data in the production instance with a timestamp within the critical verification time range between the full and incremental, business data in the production instance with a timestamp within the critical verification time range between the incremental and the leveling copy, and business data in the production instance with a timestamp within the leveling copy verification time range. The second business data extracted from the recovery instance and having a timestamp within the data verification time range includes: business data in the recovery instance with a timestamp within the full verification time range, business data in the recovery instance with a timestamp within the critical verification time range between the full and incremental, business data in the recovery instance with a timestamp within the critical verification time range between the incremental and the leveling copy, and business data in the recovery instance with a timestamp within the leveling copy verification time range.

[0108] In this way, the data verification time range can cover important critical time points in the backup and recovery process and the backup verification process (such as the time point when the full backup and recovery is successful, the time point when the incremental backup and recovery starts, the time point when the incremental backup and recovery ends, and the time point when the replication is caught up). By performing consistency verification on the business data in the production instance and the business data in the recovery instance extracted within the data verification time range, the accuracy of the backup file validity verification results can be further improved.

[0109] Step S404: Check the consistency between the business data in the production instance and the business data in the recovery instance.

[0110] Specifically, the specific implementation method for checking the consistency between the business data in the production instance and the business data in the recovery instance can be: calculating the summary information of the first business data through a hash algorithm, and calculating the summary information of the second business data through a hash algorithm; comparing the summary information of the first business data with the summary information of the second business data.

[0111] In one implementation, the data verification time range includes a full verification time range and a catch-up copy verification time range. According to the data verification time range, the business data verification process can be divided into the following two data verification stages:

[0112] (1) Full verification phase. The full verification phase refers to verifying the consistency between the business data in the production instance whose timestamps fall within the full verification time range and the business data in the recovery instance whose timestamps fall within the full verification time range.

[0113] (2) The leveling replication verification phase. The leveling replication verification phase refers to the consistency between the business data in the production instance whose timestamps are within the leveling replication verification time range and the business data in the recovery instance whose timestamps are within the leveling replication verification time range.

[0114] In this implementation, the two data verification stages in the business data verification process can be parallel, that is, the data verification process in the full verification stage and the data verification process in the leveling and copying verification stage are executed at the same time; the two verification stages in the business data verification process can also be serial, for example, the data verification process in the full verification stage is executed first, and then the data verification process in the leveling and copying verification stage is executed, or the data verification process in the leveling and copying verification stage is executed first, and then the data verification process in the full verification stage is executed.

[0115] In another implementation, the data verification time range includes a full verification time range, a critical verification time range between the full and incremental data, a critical verification time range between the incremental data and the catch-up copy, and a catch-up copy verification time range. According to the data verification time range, the verification process of the business data can be divided into the following four data verification stages:

[0116] (1) Full verification phase. The full verification phase refers to verifying the consistency between the business data in the production instance whose timestamps fall within the full verification time range and the business data in the recovery instance whose timestamps fall within the full verification time range.

[0117] (2) The critical verification phase between the full volume and the incremental volume. The critical verification phase between the full volume and the incremental volume refers to verifying the consistency between the business data with timestamps in the critical verification time range between the full volume and the incremental volume in the production instance and the business data with timestamps in the critical verification time range between the full volume and the incremental volume in the recovery instance.

[0118] (3) Critical verification phase between incremental and catch-up replication. The critical verification phase between incremental and catch-up replication refers to verifying the consistency between the business data in the production instance whose timestamps are within the critical verification time range between the incremental and catch-up replication and the business data in the recovery instance whose timestamps are within the critical verification time range between the incremental and catch-up replication.

[0119] (4) The leveling replication verification phase. The leveling replication verification phase refers to the consistency between the business data in the production instance whose timestamps are within the leveling replication verification time range and the business data in the recovery instance whose timestamps are within the leveling replication verification time range.

[0120] In this implementation, the four data verification stages in the business data verification process can be parallel, and the two verification stages in the business data verification process can also be serial; the case where the business data verification process includes four data verification stages is similar to the case where the business data verification process includes two data verification stages, and will not be repeated here.

[0121] Step S405: If the business data in the production instance is consistent with the business data in the recovery instance, the backup file is confirmed to be a valid file.

[0122] In one implementation, if the summary information of the first business data is the same as the summary information of the second business data, it is confirmed that the business data in the production instance is consistent with the business data in the recovery instance, and the backup file is confirmed to be a valid file; during the data processing process, if an operational error or failure occurs in the database, resulting in the loss of business data in the database, the valid backup file can be used for backup and recovery processing to restore reliable business data. For the two implementations in step S404, the summary information of the first business data is the same as the summary information of the second business data, which means that the summary information of the business data in the production instance and the summary information of the business data in the recovery instance are the same in two data verification stages (i.e., the full verification stage and the leveling copy verification stage); or the summary information of the first business data is the same as the summary information of the second business data, which means that the summary information of the business data in the production instance and the summary information of the business data in the recovery instance are the same in four data verification stages (i.e., the full verification stage, the critical verification stage between the full and the incremental, the critical verification stage between the incremental and the leveling copy, and the leveling copy verification stage).

[0123] In another implementation, if the summary information of the first business data is different from the summary information of the second business data, it is confirmed that the business data in the production instance is inconsistent with the business data in the recovery instance, and the backup file is confirmed to be an invalid file. In one implementation, an alarm message can also be output in the data processing device, and the alarm message is used to indicate that the backup file is an invalid file; the backup storage unit of the data processing device responds to the alarm message and processes the backup file stored in the backup storage unit, for example, the backup storage unit adds an invalid tag to the invalid backup file, and the invalid tag is used to indicate that the backup file is an invalid file. The backup storage unit can also delete the invalid backup file from the backup storage unit; or, the equipment maintenance personnel of the data processing device checks the data in the production instance of the database according to the alarm message and the production data in the recovery instance. For the two implementations in step S404, the summary information of the first business data is different from the summary information of the second business data, which means that the summary information of the business data in the production instance is different from the summary information of the business data in the recovery instance in any one of the two data verification stages, or the summary information of the business data in the production instance is different from the summary information of the business data in the recovery instance in both data verification stages. Alternatively, the summary information of the first business data is different from the summary information of the second business data, which means that the summary information of the business data in the production instance is different from the summary information of the business data in the recovery instance in any one or more of the four data verification stages.

[0124] Take the backup file including full file and incremental file as an example. Fig. 9 A schematic diagram of a backup verification process provided by an exemplary embodiment of the present application is shown. Fig. 9 As shown, the backup verification process includes the following steps S901 to S906:

[0125] Step S901, performing backup and recovery processing on the database according to the backup file (including the full file and the incremental file) to obtain a recovery instance of the database.

[0126] Step S902, attach the recovery instance to the target slave node (also known as the online asynchronous standby machine), generate a cascade node (also known as the cascade standby machine) of the target slave node, the production instance is located in the target slave node, and perform a leveling replication process between the target slave node and the cascade node.

[0127] Step S903, extract the business data in the production instance and the business data in the recovery instance respectively within four data verification time ranges (full verification time range, critical verification time range between full and incremental, critical verification time range between incremental and leveling copy, and leveling copy verification time range).

[0128] Step S904-Step S905: Check the consistency between the business data in the production instance and the business data in the recovery instance respectively within four data verification time ranges.

[0129] Step S906, if the business data in the production instance is consistent with the business data in the recovery instance within the four data verification time ranges, the verification is successful and the backup file is confirmed to be a valid file; if the business data in the production instance is inconsistent with the business data in the recovery instance within any of the four data verification time ranges, the verification fails, and the backup file is confirmed to be an invalid file, and an alarm message is output.

[0130] In the embodiment of the present application, the backup file of the database is obtained by backing up the production instance of the database, and the database is backed up and restored according to the backup file of the database to obtain the restored instance of the database; the business data in the production instance and the business data in the restored instance are extracted according to the business verification rules, and the business data extracted from the production instance and the business data extracted from the restored instance are verified to determine whether the business data in the production instance is consistent with the business data in the restored instance; if the business data in the production instance is consistent with the business data in the restored instance, the backup file is confirmed to be a valid file. In the above process, the backup file of the database is restored through a complete recovery process to obtain the restored instance of the database. If the business data restored in the restored instance of the database is consistent with the business data actually produced by the production instance of the database, then the backup file is a valid file, and the recovery process of the backup file is also valid, which improves the reliability of the backup file validity verification process and further improves the accuracy of the backup file validity verification results. In addition, the business data used for data verification is extracted according to the business verification rules. The business verification rules include the data verification time range or the data verification type. The data verification time range can cover the important critical time points in the backup recovery process and the backup verification process. By performing consistency verification on the business data in the production instance and the business data in the recovery instance extracted within the data verification time range, the accuracy of the backup file validity verification results can be further improved.

[0131] See also Fig.10 , Fig.10A schematic diagram of the structure of a data processing device provided by another exemplary embodiment of the present application is shown, and the data processing device 100 includes at least a processor 1001 and a computer-readable storage medium 1002. The processor 1001 and the computer-readable storage medium 1002 may be connected via a bus or other means. The computer-readable storage medium 1002 may be stored in a memory, the computer-readable storage medium 1002 is used to store a computer program, and the processor 1001 is used to execute the computer program stored in the computer-readable storage medium 1002. The processor 1001 (or CPU (Central Processing Unit)) is the computing core and control core of the data processing device 100, which is suitable for implementing a computer program, and is specifically suitable for loading and executing a computer program to implement a corresponding method flow or a corresponding function.

[0132] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in the data processing device 100 for storing programs and data. It can be understood that the computer-readable storage medium 1002 here can include both the built-in storage medium in the data processing device 100 and the extended storage medium supported by the data processing device 100. The computer-readable storage medium provides a storage space, which stores the operating system of the data processing device 100. In addition, computer programs suitable for being loaded and executed by the processor 1001 are also stored in the storage space, and these computer programs can be program codes. It should be noted that the computer-readable storage medium 1002 here can be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor 1001.

[0133] In one implementation, the computer readable storage medium 1002 stores a computer program; the processor 1001 loads and executes the computer program stored in the computer readable storage medium 1002 to implement Figure 4 Corresponding steps in the method embodiment shown; in a specific implementation, the computer program in the computer-readable storage medium 1002 is loaded by the processor 1001 and executes the following steps:

[0134] Get the backup file of the database. The backup file is obtained by backing up the production instance of the database.

[0135] Perform backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database;

[0136] Extract the business data in the production instance and the business data in the recovery instance according to the business verification rules, and verify the consistency between the business data in the production instance and the business data in the recovery instance;

[0137] If the business data in the production instance is consistent with the business data in the recovery instance, the backup file is confirmed to be a valid file.

[0138] In one implementation, the backup file includes a full file; when the computer program in the computer-readable storage medium 1002 is loaded by the processor 1001, the following steps are specifically performed:

[0139] Create a new instance of the database;

[0140] Import the full file into the new instance to obtain the restored instance of the database.

[0141] In one implementation, the backup file includes a full file and an incremental file; when the computer program in the computer-readable storage medium 1002 is loaded by the processor 1001, the following steps are specifically performed:

[0142] Create a new instance of the database;

[0143] Import the full file into the new instance and copy the incremental file into the relay log of the new instance;

[0144] Replay the relay log of the new instance after the replication is completed to obtain the restored instance of the database.

[0145] In one implementation, the database includes a master node and at least one slave node, each slave node corresponds to a switching priority, and when the database needs to perform a master-slave node switch, the slave node is selected and started in a descending order of the switching priority; when the computer program in the computer-readable storage medium 1002 is loaded by the processor 1001, the following steps are also executed:

[0146] A target slave node is selected from at least one slave node in the order of the switching priority from low to high, and the production instance is located in the target slave node;

[0147] Attach the recovery instance to the target slave node to generate a cascade node of the target slave node;

[0148] Perform catch-up replication between the target slave node and the cascade nodes.

[0149] In one implementation, both the business data in the production instance and the business data in the recovery instance include a timestamp; the business verification rule includes a data verification time range, and the computer program in the computer-readable storage medium 1002 specifically executes the following steps when loaded by the processor 1001:

[0150] Extracting first business data whose timestamp is within the data verification time range from the production instance, and extracting second business data whose timestamp is within the data verification time range from the recovery instance;

[0151] Alternatively, the business verification rule includes a data verification time range and a data verification type, and the computer program in the computer-readable storage medium 1002 specifically executes the following steps when loaded by the processor 1001:

[0152] Extract first business data whose timestamp is within the data verification time range and whose business type is the data verification type from the production instance, and extract second business data whose timestamp is within the data verification time range and whose business type is the data verification type from the recovery instance.

[0153] In one implementation, the data verification time range includes at least one of the following: a full verification time range, a critical verification time range between the full and incremental verification time range, a critical verification time range between the incremental and the catch-up copy, and a catch-up copy verification time range.

[0154] In one implementation, when the computer program in the computer-readable storage medium 1002 is loaded by the processor 1001, the following steps are specifically performed:

[0155] Calculating summary information of the first business data and summary information of the second business data;

[0156] If the summary information of the first business data is the same as the summary information of the second business data, it is determined that the business data in the production instance is consistent with the business data in the recovery instance; otherwise, it is determined that the business data in the production instance is inconsistent with the business data in the recovery instance.

[0157] In one implementation, when the computer program in the computer-readable storage medium 1002 is loaded by the processor 1001, the following steps are further performed:

[0158] Verify the integrity of the backup file;

[0159] If the verification is successful, the step of performing backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database is triggered.

[0160] In one implementation, the backup file includes a full file and an incremental file; and verifying the integrity of the backup file includes at least one of the following:

[0161] Check whether the backup file is damaged. If the backup file is not damaged, the verification is successful;

[0162] Check whether the backup file is correct. If it is correct, the verification is successful.

[0163] Verify whether the incremental file is played back normally. If it is normal, the verification is successful.

[0164] Check whether the data pages of the backup file are complete. If they are complete, the verification is successful.

[0165] In one implementation, when the computer program in the computer-readable storage medium 1002 is loaded by the processor 1001, the following steps are specifically performed:

[0166] If the business data in the production instance is inconsistent with the business data in the recovery instance, an alarm message is output, which is used to indicate that the backup file is an invalid file.

[0167] In the embodiment of the present application, the backup file of the database is obtained by backing up the production instance of the database, and the database is backed up and restored according to the backup file of the database to obtain the restored instance of the database; the business data in the production instance and the business data in the restored instance are extracted according to the business verification rules, and the business data extracted from the production instance and the business data extracted from the restored instance are verified to determine whether the business data in the production instance is consistent with the business data in the restored instance; if the business data in the production instance is consistent with the business data in the restored instance, the backup file is confirmed to be a valid file. In the above process, the backup file of the database is restored through a complete recovery process to obtain the restored instance of the database. If the business data restored in the restored instance of the database is consistent with the business data actually produced by the production instance of the database, then the backup file is a valid file, and the recovery process of the backup file is also valid, which improves the reliability of the backup file validity verification process and further improves the accuracy of the backup file validity verification results.

[0168] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the data processing method provided in the above-mentioned various optional manners.

[0169] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: Obtain a backup file of the database, wherein the backup file is obtained by backing up a production instance of the database, and the backup file is used to support the continuous normal operation of the business platform when data loss or failure occurs in the database, and the database includes a master node and at least one slave node; Performing backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database; Selecting a target slave node from the at least one slave node, wherein the production instance is located in the target slave node; Mounting the recovery instance to the target slave node to generate a cascade node of the target slave node; Performing a leveling replication process between the target slave node and the cascade node; Extracting the business data in the production instance and the business data in the recovery instance according to the business verification rule, and verifying the consistency between the business data in the production instance and the business data in the recovery instance; If the business data in the production instance is consistent with the business data in the recovery instance, the backup file is confirmed to be a valid file.

2. The method according to claim 1, characterized in that The backup file includes a full file; and performing backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database includes: creating a new instance of said database; Import the full file into the new instance to obtain a restored instance of the database.

3. The method according to claim 1, characterized in that The backup file includes a full file and an incremental file; performing backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database includes: creating a new instance of said database; Import the full file into the new instance, and copy the incremental file into the relay log of the new instance; The relay log of the new instance after the replication is completed is replayed to obtain a restored instance of the database.

4. The method according to claim 1, characterized in that Each slave node corresponds to a switching priority. When the database needs to perform a master-slave node switch, the slave nodes are selected and started in descending order of the switching priorities. The selecting a target slave node from the at least one slave node comprises: A target slave node is selected from the at least one slave node in the order of the switching priority from low to high.

5. The method according to any one of claims 1 to 4, characterized in that: The business data in the production instance and the business data in the recovery instance both include a timestamp; The business verification rule includes a data verification time range, and the extracting of the business data in the production instance and the business data in the recovery instance according to the business verification rule includes: Extracting first business data whose timestamp is within the data verification time range from the production instance, and extracting second business data whose timestamp is within the data verification time range from the recovery instance; Alternatively, the business verification rule includes the data verification time range and the data verification type, and the extracting the business data in the production instance and the business data in the recovery instance according to the business verification rule includes: Extract the first business data whose timestamp is within the data verification time range and whose business type is the data verification type from the production instance, and extract the second business data whose timestamp is within the data verification time range and whose business type is the data verification type from the recovery instance.

6. The method according to claim 5, characterized in that The data verification time range includes at least one of the following: a full verification time range, a critical verification time range between a full and an incremental verification time range, a critical verification time range between an incremental and a catch-up copy, and a catch-up copy verification time range.

7. The method according to claim 5, characterized in that The checking the consistency between the business data in the production instance and the business data in the recovery instance includes: Calculating summary information of the first service data and summary information of the second service data; If the summary information of the first business data is the same as the summary information of the second business data, it is determined that the business data in the production instance is consistent with the business data in the recovery instance; otherwise, it is determined that the business data in the production instance is inconsistent with the business data in the recovery instance.

8. The method according to claim 1, characterized in that After obtaining the backup file of the database, the method further includes: Verifying the integrity of the backup file; If the verification is successful, the step of performing backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database is triggered.

9. The method according to claim 8, characterized in that The backup file includes a full file and an incremental file; and the verifying the integrity of the backup file includes at least one of the following: Check whether the backup file is damaged, if the backup file is not damaged, the verification is successful; Check whether the backup file is correct, if correct, the verification is successful; Verify whether the incremental file is played back normally, if so, the verification is successful; Check whether the data pages of the backup file are complete, if so, the verification is successful.

10. The method according to claim 1, characterized in that The method further comprises: If the business data in the production instance is inconsistent with the business data in the recovery instance, an alarm message is output, where the alarm message is used to indicate that the backup file is an invalid file.

11. A data processing device, characterized in that: The device comprises a management unit and an execution unit; The management unit is configured to schedule the execution unit to perform the backup and recovery task; The execution unit is configured to obtain a backup file of the database according to the scheduling of the management unit, and perform backup and recovery processing on the database according to the backup file to obtain a recovery instance of the database, wherein the backup file is obtained by backing up a production instance of the database, and the backup file is used to support the continuous normal operation of the business platform when data loss or failure occurs in the database, and the database includes a master node and at least one slave node; and The management unit is configured to schedule the execution unit to perform the backup verification task; The execution unit is configured to select a target slave node from the at least one slave node, the production instance being located in the target slave node; Mounting the recovery instance to the target slave node to generate a cascade node of the target slave node; Performing a leveling replication process between the target slave node and the cascade node; The execution unit is configured to extract the business data in the production instance and the business data in the recovery instance according to the scheduling of the management unit and the business verification rule, and verify the consistency between the business data in the production instance and the business data in the recovery instance, and return the data verification result to the management unit; The management unit is configured to confirm that the backup file is a valid file if the data verification result indicates that the business data in the production instance is consistent with the business data in the recovery instance.

12. The device according to claim 11, characterized in that The device also includes: A storage unit, the storage unit comprising a backup storage unit and a task storage unit; the backup storage unit is configured to store backup files of the database, and the task storage unit is configured to store multiple tasks in the database; the backup storage unit is equipped with a distributed file system, and the task storage unit is equipped with a relational database management system cluster; The registration unit is configured to perform registration processing on the execution unit.

13. The device according to any one of claims 11 to 12, characterized in that The number of the execution units is multiple; When performing scheduling, the management unit selects an idle execution unit from the multiple execution units to execute the backup recovery task or the backup verification task.

14. A data processing device, characterized in that: The data processing device comprises: a processor adapted to implement a computer program; and, A computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by the processor and executing the data processing method according to any one of claims 1 to 10.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and the computer program is suitable for being loaded by a processor and executing the data processing method according to any one of claims 1 to 10.

Images