Data processing method and device, equipment and medium

Abstract

The embodiment of the invention provides a data processing method and device, equipment and a medium. The method comprises the steps that when local storage is in a first state, a current full-amount snapshot is obtained according to a first full-amount snapshot and an incremental snapshot in an object storage service, and the first full-amount snapshot is an initial full-amount snapshot uploaded when local data is in a second state; the incremental snapshots are determined according to difference data between historical full-amount snapshots and initial full-amount snapshots, and the historical full-amount snapshots are full-amount snapshots uploaded after the initial full-amount snapshots are uploaded by the local data; performing first data recovery on a to-be-recovered disk in the database according to the current full-amount snapshot to obtain an initial recovery disk; and performing second data recovery on the initial recovery disk according to the physical incremental backup and the transaction log in the object storage service to obtain a target disk. The method is used for improving the data recovery effect.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, equipment and medium. Background Art

[0002] With the development of information technology and the acceleration of digital transformation of enterprises, data, as an important asset of enterprises, has become increasingly important. In order to ensure the security and integrity of data, database backup and recovery have become an indispensable part of enterprise IT operation and maintenance management. Especially in the scenario of large data volume, how to efficiently perform data backup, recovery drills, and quickly restore services in the event of a failure has become one of the key indicators to measure the level of enterprise data management.

[0003] At present, database backup and recovery technology is quite mature, providing a variety of solutions to meet different business needs, including physical backup, logical backup, incremental backup and differential backup. Physical backup involves copying the entire database file system, logical backup is to export database objects and data, and incremental backup and differential backup reduce backup time and storage space requirements by backing up only the data that has changed since the last full backup. In addition, some modern database management systems have introduced snapshot technology, which can quickly generate backups without affecting the performance of the production system, and can be used for disaster recovery and data recovery drills, greatly shortening the time for data recovery.

[0004] However, when performing data recovery in existing databases, there is a technical problem of poor recovery effect. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, apparatus, device and medium to solve the technical problem that the existing database has a poor recovery effect when performing data recovery.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0007] When the local storage is in the first state, the current full snapshot is obtained according to the first full snapshot and the incremental snapshot in the object storage service, where the first full snapshot is the initial full snapshot uploaded when the local data is in the second state, and the incremental snapshot is determined according to the difference data between the historical full snapshot and the initial full snapshot, and the historical full snapshot is the full snapshot uploaded by the local data after uploading the initial full snapshot;

[0008] According to the current full snapshot, perform the first data recovery on the disk to be recovered in the database to obtain the initial recovery disk;

[0009] According to the physical incremental backup and transaction log in the object storage service, a second data recovery is performed on the initial recovery disk to obtain the target disk.

[0010] In a possible implementation, performing a second data recovery on the initial recovery disk according to the physical incremental backup and transaction log in the object storage service to obtain the target disk includes:

[0011] If there is a physical incremental backup in the object storage service, a second data recovery is performed on the initial recovery disk according to the consistent snapshot time point corresponding to the physical incremental backup to obtain the initial target disk;

[0012] If there is a transaction log in the object storage service, a second data recovery is performed on the initial target disk according to the time point corresponding to the transaction log to obtain the target disk.

[0013] In a possible implementation, when the database receives the service request, the method further includes:

[0014] Determine the target data block according to the business request;

[0015] According to the physical incremental backup and / or transaction log corresponding to the target data block, a second data recovery is performed on the initial recovery disk to obtain a target disk.

[0016] In a possible implementation, after performing a second data recovery on the initial recovery disk according to the physical incremental backup and the transaction log in the object storage service to obtain the target disk, the method further includes:

[0017] Mount the target disk to the database;

[0018] When the database receives a business access, it determines whether the data block corresponding to the business access is restored;

[0019] If the data block corresponding to the business access has been recovered, the database access operation is performed;

[0020] If the data block corresponding to the business access is not restored, the target disk is restored based on the physical incremental backup and / or transaction log corresponding to the data block;

[0021] After the data recovery on the target disk is complete, perform the database access operation.

[0022] In a possible implementation, the method further includes:

[0023] When the local storage is in the second state, data is restored on the disk to be restored in the database according to the second full snapshot in the local storage to obtain an initial restored disk;

[0024] According to the physical incremental backup and transaction log in the object storage service, a second data recovery is performed on the initial recovery disk to obtain the target disk.

[0025] In a possible implementation, before obtaining the current full snapshot according to the first full snapshot and the incremental snapshots in the object storage service, the method further includes:

[0026] Obtain snapshot backup information according to a preset time period;

[0027] The full snapshot in the snapshot backup information is stored in the local storage to obtain a second full snapshot;

[0028] The second full snapshot, the physical incremental backup in the snapshot backup information, and the transaction log are uploaded to the object storage service to generate the first full snapshot, the incremental snapshot, the physical incremental backup, and the transaction log in the object storage service.

[0029] In a possible implementation, the method further includes:

[0030] If the transaction log in the snapshot backup information does not exist, a snapshot is taken of the transaction log, and the snapshotted transaction log is uploaded to the object storage service.

[0031] In a second aspect, an embodiment of the present application provides a hidden crack identification device, including:

[0032] A first determination module, used to determine a first crack to be identified in a first view of the object to be identified, and a second crack to be identified in a second view;

[0033] a second determination module, configured to determine, when the position of the first crack to be identified does not correspond to the position of the second crack to be identified, a hidden crack to be identified in the first crack to be identified according to a pixel change of the first crack to be identified in the first view;

[0034] The obtaining module is used to identify the hidden crack to be identified according to the positional relationship between the first hidden crack to be identified and the second hidden crack to be identified, and the positional relationship with other hidden cracks to be identified in other views of the object to be identified, and obtain a hidden crack identification result.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0036] Memory stores computer-executable instructions;

[0037] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] Figure 1 A schematic diagram of the scenario of the data method provided for this application;

[0042] Figure 2 Schematic diagram of the data processing method provided for this application Figure 1 ;

[0043] Figure 3 Schematic diagram of the data processing method provided for this application Figure 2 ;

[0044] Figure 4 Schematic diagram of the data processing method provided for this application Figure 3 ;

[0045] Figure 5 A schematic diagram of the structure of the data processing device provided by this application;

[0046] Figure 6 A schematic diagram of the structure of the electronic device provided in this application.

[0047] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] First, the terms involved in this application are explained:

[0050] OBS (Object Storage Service) is an object-based mass storage service that provides massive, secure, highly reliable, and low-cost data storage capabilities, including the creation, modification, and deletion of buckets, as well as the upload, download, and deletion of objects. OBS is an Internet-accessible service that provides a Web service interface based on the HTTP / HTTPS protocol. Data stored in OBS can be accessed and managed through the OBS management console or client. OBS supports SDK and API interfaces, which allows users to easily manage data and develop various types of upper-level business applications.

[0051] EVS (Elastic Volume Service), also known as cloud hard disk, is a virtual block storage service that mainly provides block storage space for ECS (Elastic Cloud Server) and BMS (Bare Metal Server). Users can create cloud hard disks online and mount them to instances. The use of cloud hard disks is exactly the same as that of traditional server hard disks. At the same time, cloud hard disks have higher data reliability, higher I / O throughput, and are easier to use. They are suitable for file systems, databases, or other system software or applications that require block storage devices.

[0052] In the existing technology, traditional database backup and recovery methods face significant challenges in large data volume scenarios. Due to the huge amount of data, the daily backup process takes a long time, which seriously affects the availability and response speed of the system. In addition, when performing recovery drills or actual fault recovery, the long data recovery process is difficult to meet the strict recovery point objective (RPO) and recovery time objective (RTO) requirements, which makes it impossible for enterprises to quickly resume normal operations when faced with data loss or system failures. At the same time, complex backup and recovery operations also increase the workload of operation and maintenance personnel, easily introduce human errors, and reduce the stability and security of the overall system.

[0053] The data processing method, apparatus, device and medium provided in the embodiments of the present application, by combining the initial full snapshot and subsequent incremental snapshots in the object storage to construct the current full snapshot when the local storage is in the first state, first perform preliminary data recovery to obtain the initial recovery disk, then use the physical incremental backup and transaction log to perform refined recovery, and finally complete the means of comprehensive data recovery of the target disk, thereby improving the data recovery effect.

[0054] Figure 1 A schematic diagram of the scenario of the data method provided for this application, such as Figure 1As shown, the specific application scenarios of the present application may include a database, a local server and a cloud server, and the local server and the cloud server are respectively connected to the database, wherein the local server and the cloud server can be mobile phones, tablet computers, smart Bluetooth devices, laptops, or personal computers (PC) and other devices; the server can be a single server or a server cluster composed of multiple servers.

[0055] The embodiments of the present application do not restrict the execution subject, as long as the execution subject can obtain the current full snapshot according to the first full snapshot and incremental snapshots in the object storage service when the local storage is in the first state, the first full snapshot is the initial full snapshot uploaded when the local data is in the second state, the incremental snapshot is determined according to the difference data between the historical full snapshot and the initial full snapshot, the historical full snapshot is the full snapshot uploaded after the initial full snapshot is uploaded; according to the current full snapshot, perform a first data recovery on the disk to be recovered in the database to obtain an initial recovery disk; according to the physical incremental backup and transaction log in the object storage service, perform a second data recovery on the initial recovery disk to obtain the target disk.

[0056] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0057] Figure 2 Schematic diagram of the data processing method provided for this application Figure 1 ,like Figure 2 As shown, the method includes:

[0058] S201. When the local storage is in a first state, a current full snapshot is obtained according to a first full snapshot and incremental snapshots in the object storage service, where the first full snapshot is an initial full snapshot uploaded when the local data is in a second state, and the incremental snapshots in the object storage service are determined according to the difference data between the historical full snapshot and the initial full snapshot, and the historical full snapshot is a full snapshot uploaded after the initial full snapshot is uploaded.

[0059] Among them, local storage can refer to a data storage system located within a user or enterprise, usually including a hard disk, a solid-state drive or other types of storage devices. In the embodiment of the present application, the local storage in the first state can refer to a state where the local storage is unavailable, that is, the local data storage system cannot be accessed or work normally for some reason, and the local data in the second state can refer to a state where the local storage is available, that is, the local data storage system can access and read data normally.

[0060] Object storage service may refer to a cloud-based service for storing and retrieving large amounts of unstructured data, such as pictures, videos, documents, etc. It provides access through an API interface and has high availability and scalability. In an embodiment of the present application, the object storage service may be OBS object storage.

[0061] The first full snapshot can refer to a complete backup of the entire database or data set at a certain point in time. The full snapshot contains the state of all data and can be used to restore the data state to that point in time.

[0062] The initial full snapshot refers to the full snapshot uploaded to the object storage service for the first time, which is used to record the complete status of local data at a certain point in time.

[0063] A historical full snapshot may refer to a full snapshot uploaded subsequently after the initial full snapshot.

[0064] An incremental snapshot can refer to backing up only the changed data between two full snapshots. Incremental snapshots can significantly reduce the time and storage space required for backup. Incremental snapshots are determined based on the difference between the historical full snapshot and the initial full snapshot.

[0065] In an embodiment of the present application, when the local storage is in a first state (i.e., local data is unavailable), a current full snapshot reflecting the current latest data state can be generated by combining the first full snapshot and incremental snapshots in the object storage service. The first full snapshot is uploaded when the local data is in a second state (i.e., local data is available), and records the complete data state at that point in time. The incremental snapshot is generated by comparing the difference data between the historical full snapshot and the initial full snapshot, and records the data that has changed during this period of time. The historical full snapshot is uploaded after the initial full snapshot, and is used to record the complete state of local data at different points in time. In this way, data can be effectively managed and restored when local data is frequently updated, while reducing the storage space and time required for backup.

[0066] S202: Perform a first data recovery on the disk to be recovered in the database according to the current full snapshot to obtain an initial recovery disk.

[0067] The disk to be recovered in the database may refer to a database storage device that needs to perform data recovery, and is usually a disk that needs to be recovered due to data loss, damage, or system failure.

[0068] The first data recovery may refer to a process of restoring the backup data to a target storage device, with the purpose of restoring the integrity and availability of the data.

[0069] The initial recovery disk may refer to a disk containing the latest data status after recovery through a full snapshot.

[0070] In the embodiment of the present application, when performing data recovery, the current full snapshot file can be downloaded from the object storage service, and then the database recovery tool (such as the MySQL client of MySQL or the pg_restore tool of PostgreSQL) is used to restore the data in the snapshot file to the disk to be restored, thereby obtaining the initial recovery disk.

[0071] S203: Perform a second data recovery on the initial recovery disk according to the physical incremental backup and transaction log in the object storage service to obtain a target disk.

[0072] The physical incremental backup in the object storage service may refer to the incremental backup data stored in the object storage service, which records the data blocks that have changed since the last full backup. The physical incremental backup includes the changes in the data blocks.

[0073] The transaction log may refer to an XLOG log file. An XLOG (Write-Ahead Log) log file may refer to a transaction log file in a database, which records all transaction operations in the database, including inserts, deletes, and updates. The XLOG log file may be used to ensure data consistency and integrity during data recovery.

[0074] The second data recovery may refer to a data recovery operation that is further performed using a physical incremental backup and a transaction log on the basis of the initial recovery disk to ensure data integrity and consistency.

[0075] The target disk can refer to the disk after all recovery operations are completed or the disk of the physical incremental backup and transaction log in the current object storage service.

[0076] In the embodiment of the present application, the second data recovery is performed on the initial recovery disk according to the physical incremental backup and transaction log in the object storage service to obtain the target disk, including:

[0077] If there is a physical incremental backup in the object storage service, a second data recovery is performed on the initial recovery disk according to the consistent snapshot time point corresponding to the physical incremental backup to obtain the initial target disk;

[0078] If there is a transaction log in the object storage service, a second data recovery is performed on the initial target disk according to the time point corresponding to the transaction log to obtain the target disk.

[0079] The consistency snapshot time point may refer to the data consistency time point corresponding to when the physical incremental backup is generated. At this time point, the data state of the database is consistent and there is no uncommitted transaction.

[0080] In an embodiment of the present application, a second data recovery is performed on the initial recovery disk according to the consistency snapshot time point corresponding to the physical incremental backup, and the initial target disk can refer to: if a physical incremental backup exists in the object storage service, the physical incremental backup file is downloaded from the object storage service, the consistency snapshot time point of each backup file is determined, the database service is stopped, and the data block changes in the physical incremental backup file are sequentially applied using a recovery tool to verify the correctness of the data recovery, and finally an initial target disk containing the latest data status is generated.

[0081] In an embodiment of the present application, the initial target disk is subjected to a second data recovery according to the time point corresponding to the transaction log, and the target disk obtained may refer to: if there is a transaction log (XLOG log file) in the object storage service, the transaction log file is downloaded from the object storage service, the time point of each log file is determined, the database service is stopped, the recovery configuration file is edited to specify the location of the log file and the time point of recovery, the database service is started for recovery, the correctness of the data recovery is verified, and finally a target disk containing the latest, complete and consistent data is generated. This ensures the consistency and integrity of the data during the recovery process. Among them, in an embodiment of the present application, when the database receives a business request, the method also includes:

[0082] Determine the target data block according to the business request;

[0083] According to the physical incremental backup and / or transaction log corresponding to the target data block, a second data recovery is performed on the initial recovery disk to obtain a target disk.

[0084] The time when the database receives the service request may refer to the time when the initial recovery disk is in the process of performing the second data recovery, or may refer to the time after the initial recovery disk completes the second data recovery.

[0085] Therefore, in the embodiment of the present application, the initial recovery disk is first obtained through full snapshot recovery, and then the second data recovery is performed on the initial recovery disk based on the physical incremental backup and transaction log in the object storage service. In this process, since both the physical incremental backup and the transaction log have corresponding time points, specific data blocks can be loaded only when needed, thereby reducing the initial loading time and resource consumption.

[0086] In the embodiment of the present application, after performing a second data recovery on the initial recovery disk according to the physical incremental backup and transaction log in the object storage service to obtain the target disk, the method further includes:

[0087] Mount the target disk to the database;

[0088] When the database receives a business access, it determines whether the data block corresponding to the business access is restored;

[0089] If the data block corresponding to the business access has been recovered, the database access operation is performed;

[0090] If the data block corresponding to the business access is not restored, the target disk is restored based on the physical incremental backup and / or transaction log corresponding to the data block;

[0091] After the data recovery on the target disk is complete, perform the database access operation.

[0092] Mounting the target disk to the database may refer to mounting the target disk recovered through the full snapshot and the physical incremental backup to the database server to use the target disk.

[0093] When the database receives a business access, if the data block corresponding to the business access has been restored, the database access operation is directly executed. If the data block corresponding to the business access has not been restored, the data is restored on the target disk according to the physical incremental backup and / or transaction log corresponding to the data block, and then the database access operation is executed. Thus, by checking and restoring the data block on demand through the received business access, the efficiency and response speed of database recovery can be significantly improved, the integrity and consistency of the data are ensured, and unnecessary data loading and processing time are reduced. This allows the database to be restored to normal operation in the shortest time, ensuring business continuity and stability.

[0094] For example, in an embodiment of the present application, in the process of processing database recovery, the original data disk is first unloaded by calling the EVS interface, and then the secondary full snapshot and incremental snapshot in the OBS object storage are merged to generate the latest secondary full snapshot. Subsequently, the recovery task of the snapshot is started in the background to restore the data to the data disk. If the physical incremental backup is turned on, the physical incremental backup that matches the consistency snapshot time point is selected for recovery; at the same time, if there are XLOG archive logs, these logs will be used to restore the database to the most recent point in time. Once the secondary snapshot recovery task completes the recovery of the EVS snapshot metadata, the EVS interface is called to remount the data disk and start the database external service. For the received business request, if the requested data block has not yet completed the snapshot recovery, the Lazy loading technology can be used to restore the corresponding database block to the data disk on demand to ensure that the database execution result can be returned in time.

[0095] Figure 3Schematic diagram of the data processing method provided for this application Figure 2 ,like Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment, the data processing method is described in detail, and the method includes:

[0096] S301. When the local storage is in the second state, data is restored on the disk to be restored in the database according to the second full snapshot in the local storage to obtain an initial restored disk.

[0097] S302: Perform a second data recovery on the initial recovery disk according to the physical incremental backup and transaction log in the object storage service to obtain a target disk.

[0098] When the local storage is available, the disk to be restored can be restored based on the second full snapshot in the local storage. Then, the physical incremental backup and transaction log in the object storage service are combined to perform the second data recovery on the initial recovery disk to obtain the target disk.

[0099] In the embodiment of the present application, the second full snapshot and the first full snapshot may be the same or different.

[0100] For example, in an embodiment of the present application, in order to restore the database, the EVS interface can be called first to unmount the original data disk, and then the most recent level 1 full snapshot can be restored to the data disk. If physical incremental backup is enabled, the data corresponding to the consistency snapshot time point is further selected for restoration. In addition, if there are XLOG archive logs, the database will be restored to the most recent time point by applying these logs. After completing the above steps, call the EVS interface to mount the data disk, and start the database to provide external services.

[0101] Figure 4 Schematic diagram of the data processing method provided for this application Figure 3 ,like Figure 4 As shown, in this embodiment Figure 2 Based on the embodiment, a method before obtaining the current full snapshot according to the first full snapshot and the incremental snapshot in the object storage service is described in detail, and the method includes:

[0102] S401, obtaining snapshot backup information according to a preset time period;

[0103] S402, storing the full snapshot in the snapshot backup information in local storage to obtain a second full snapshot;

[0104] S403: Upload the second full snapshot, the physical incremental backup in the snapshot backup information, and the transaction log to the object storage service to generate the first full snapshot, the incremental snapshot, the physical incremental backup, and the transaction log in the object storage service.

[0105] In order to support the need to restore to any point in time, it is necessary to pre-configure regular physical backup of the database and XLOG archive log backup to the object storage service.

[0106] When performing snapshot backup, you can perform snapshot backup regularly according to the preset backup strategy. Before backup, the system will perform a verification check and record the starting playback point of the xlog log in the metadata. Next, call the EVS interface to create a consistent full snapshot of the database data disk, and store the consistent full snapshot in the EVS local disk (full snapshot in local storage). After the backup is completed, the system will check whether the snapshot contains the xlog log file. If not, you need to take a snapshot of the xlog log again. After that, the full snapshot in the local storage will be uploaded to the object storage service on the cloud to form a full snapshot of the object storage service. If there is an old version snapshot in the object storage service storage, the system will capture the difference data between the old and new snapshots based on ROW (Redirect-On-Write) technology and upload it to the object storage service to form an incremental snapshot of the object storage service.

[0107] Therefore, the data processing method provided by the embodiment of the present application can not only realize multi-level backup of local and cloud (object storage service) by utilizing multi-level snapshot technology, improve the reliability of backup, but also achieve the effect of rapid database recovery with the help of the first-level snapshot. On this basis, the lazy loading technology of the second-level snapshot can load data blocks on demand, further speeding up the recovery speed of the database service. At the same time, combined with the XLOG archive log backup, it ensures that the database can be restored to any time point, greatly enhancing the flexibility and efficiency of data recovery.

[0108] Figure 5 A schematic diagram of the structure of the data processing device provided in this application, such as Figure 5 As shown, the data processing device 50 provided in this embodiment includes:

[0109] Obtaining module 501, used for obtaining the current full snapshot according to the first full snapshot and incremental snapshots in the object storage service when the local storage is in the first state, wherein the first full snapshot is the initial full snapshot uploaded when the local data is in the second state, and the incremental snapshot is determined according to the difference data between the historical full snapshot and the initial full snapshot, and the historical full snapshot is the full snapshot uploaded by the local data after uploading the initial full snapshot;

[0110] A first recovery module 502 is used to perform a first data recovery on the disk to be recovered in the database according to the current full snapshot to obtain an initial recovery disk;

[0111] The second recovery module 503 is used to perform a second data recovery on the initial recovery disk according to the physical incremental backup and transaction log in the object storage service to obtain a target disk.

[0112] In a possible implementation, the second recovery module 503 may also be specifically configured to:

[0113] If there is a physical incremental backup in the object storage service, a second data recovery is performed on the initial recovery disk according to the consistent snapshot time point corresponding to the physical incremental backup to obtain the initial target disk;

[0114] If there is a transaction log in the object storage service, a second data recovery is performed on the initial target disk according to the time point corresponding to the transaction log to obtain the target disk.

[0115] In a possible implementation, when the database receives the service request, the second recovery module 503 may also be specifically configured to:

[0116] Determine the target data block according to the business request;

[0117] According to the physical incremental backup and / or transaction log corresponding to the target data block, a second data recovery is performed on the initial recovery disk to obtain a target disk.

[0118] In a possible implementation, the second recovery module 503 may also be specifically configured to:

[0119] Mount the target disk to the database;

[0120] When the database receives a business access, it determines whether the data block corresponding to the business access is restored;

[0121] If the data block corresponding to the business access has been recovered, the database access operation is performed;

[0122] If the data block corresponding to the business access is not restored, the target disk is restored based on the physical incremental backup and / or transaction log corresponding to the data block;

[0123] After the data recovery on the target disk is complete, perform the database access operation.

[0124] In a possible implementation, the second recovery module 503 may also be specifically configured to:

[0125] When the local storage is in the second state, data is restored on the disk to be restored in the database according to the second full snapshot in the local storage to obtain an initial restored disk;

[0126] According to the physical incremental backup and transaction log in the object storage service, a second data recovery is performed on the initial recovery disk to obtain the target disk.

[0127] In a possible implementation, the second recovery module 503 may also be specifically configured to:

[0128] Obtain snapshot backup information according to a preset time period;

[0129] The full snapshot in the snapshot backup information is stored in the local storage to obtain a second full snapshot;

[0130] The second full snapshot, the physical incremental backup in the snapshot backup information, and the transaction log are uploaded to the object storage service to generate the first full snapshot, the incremental snapshot, the physical incremental backup, and the transaction log in the object storage service.

[0131] In a possible implementation, the second recovery module 503 may also be specifically configured to:

[0132] If the transaction log in the snapshot backup information does not exist, a snapshot is taken of the transaction log, and the snapshotted transaction log is uploaded to the object storage service.

[0133] The data processing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0134] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the electronic device 60 further includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus 604.

[0135] In a specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0136] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0137] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0138] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0139] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0140] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0141] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0142] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0143] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0144] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0147] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0148] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0149] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: The method comprises: When the local storage is in a first state, a current full snapshot is obtained according to a first full snapshot and an incremental snapshot in the object storage service, wherein the first full snapshot is an initial full snapshot uploaded when the local data is in a second state, and the incremental snapshot is determined according to difference data between a historical full snapshot and the initial full snapshot, and the historical full snapshot is a full snapshot uploaded by the local data after uploading the initial full snapshot; Performing a first data recovery on the disk to be recovered in the database according to the current full snapshot to obtain an initial recovery disk; According to the physical incremental backup and transaction log in the object storage service, a second data recovery is performed on the initial recovery disk to obtain a target disk.

2. The method according to claim 1, characterized in that The method of performing a second data recovery on the initial recovery disk according to the physical incremental backup and the transaction log in the object storage service to obtain a target disk includes: If the physical incremental backup exists in the object storage service, performing a second data recovery on the initial recovery disk according to the consistent snapshot time point corresponding to the physical incremental backup to obtain an initial target disk; If the transaction log exists in the object storage service, a second data recovery is performed on the initial target disk according to the time point corresponding to the transaction log to obtain a target disk.

3. The method according to claim 2, characterized in that When the database receives a service request, the method further includes: Determine a target data block according to the service request; According to the physical incremental backup and / or transaction log corresponding to the target data block, a second data recovery is performed on the initial recovery disk to obtain a target disk.

4. The method according to any one of claims 1 to 3, characterized in that After performing a second data recovery on the initial recovery disk according to the physical incremental backup and the transaction log in the object storage service to obtain the target disk, the method further includes: Mounting the target disk to the database; When the database receives a service access, determining whether a data block corresponding to the service access is restored; If the data block corresponding to the business access has been restored, then executing the database access operation; If the data block corresponding to the business access is not restored, recovering the data on the target disk according to the physical incremental backup and / or transaction log corresponding to the data block; After the data recovery of the target disk is completed, the access operation of the database is performed.

5. The method according to claim 1, characterized in that: The method further comprises: When the local storage is in the second state, data is restored on the disk to be restored in the database according to the second full snapshot in the local storage to obtain an initial restored disk; According to the physical incremental backup and transaction log in the object storage service, a second data recovery is performed on the initial recovery disk to obtain a target disk.

6. The method according to claim 1, characterized in that Before obtaining the current full snapshot according to the first full snapshot and the incremental snapshots in the object storage service, the method further includes: Obtain snapshot backup information according to a preset time period; The full snapshot in the snapshot backup information is stored in local storage to obtain a second full snapshot; The second full snapshot, the physical incremental backup in the snapshot backup information, and the transaction log are uploaded to the object storage service to generate the first full snapshot, the incremental snapshot, the physical incremental backup, and the transaction log in the object storage service.

7. The method according to claim 6, characterized in that The method further comprises: If the transaction log in the snapshot backup information does not exist, a snapshot is taken of the transaction log, and the transaction log after the snapshot is uploaded to the object storage service.

8. A data processing device, characterized in that: include: An obtaining module is used to obtain a current full snapshot according to a first full snapshot and an incremental snapshot in an object storage service when the local storage is in a first state, wherein the first full snapshot is an initial full snapshot uploaded when the local data is in a second state, and the incremental snapshot is determined according to difference data between a historical full snapshot and the initial full snapshot, and the historical full snapshot is a full snapshot uploaded by the local data after uploading the initial full snapshot; A first recovery module, configured to perform a first data recovery on the disk to be recovered in the database according to the current full snapshot, to obtain an initial recovery disk; The second recovery module is used to perform a second data recovery on the initial recovery disk according to the physical incremental backup and transaction log in the object storage service to obtain a target disk.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images