Data backup method, device and equipment and computer readable storage medium

By combining full backup with real-time log incremental information, and using the log cache disk for data merging and deletion, the problems of data loss and large volume in data backup are solved, and real-time data recovery and storage optimization are achieved.

CN118939481APending Publication Date: 2024-11-12ZHENGZHOU INSPUR DATA TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411156699.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing technologies suffer from data loss and massive data volumes during data backup, especially when the host crashes during delayed backups and data cannot be recovered. Furthermore, real-time backups can result in excessively large data volumes, making them unsuitable for large-scale scenarios.

Method used

A method combining full backup and real-time log incremental information is adopted. Real-time backup is performed using a log cache disk, and incremental data is merged with full backup data by timestamp and data merging period to form target backup data. At the same time, incremental information is deleted according to the deletion time period.

Benefits of technology

It enables data to be restored to any point in time, reduces the amount of data on the storage disk, avoids excessive data, and enables real-time incremental backup without increasing the storage load.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939481A_ABST
    Figure CN118939481A_ABST
Patent Text Reader

Abstract

The invention discloses a data backup method, device and equipment and a computer readable storage medium, which are applied to the technical field of data processing, and the method comprises the following steps: carrying out full backup on data of a virtual machine, and storing the full backup data to a storage disk; backing up the real-time log increment information and storing the real-time log increment information in a log cache disk; according to a timestamp in the real-time log increment information, carrying out data merging on the real-time increment data in the log cache disk and the total backup data to obtain target backup data; and deleting the data in the log cache disk according to the data deletion time period. When data backup is carried out, real-time backup can be carried out by utilizing the log cache disk, so that the data can be recovered to any time point, the incremental data and the full backup data are merged and stored in the storage disk, the data volume stored in the storage disk is reduced, the real-time incremental data can be deleted regularly, and the data backup efficiency is improved. The data volume stored in the log cache disk is reduced, and excessive data volume is prevented.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data backup method, device, equipment and computer-readable storage medium. Background Art

[0002] Currently, when backing up data, full backup and incremental backup are mainly used. However, full backup and incremental backup are performed at regular intervals, and there is a delay in the backup. If the host fails during the delay period, this part of the data will be lost, and the only time point that can be restored is the time to be backed up. If real-time backup is performed, the amount of data will be huge and it is not suitable for large-scale scenarios.

[0003] Therefore, how to reduce the loss of data during backup without causing a huge amount of data is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a data backup method, device, equipment and computer-readable storage medium, which can solve the technical problems of data loss and huge data volume.

[0005] To solve the above technical problems, an embodiment of the present invention provides a data backup method, which may include:

[0006] Perform a full backup of the data of the virtual machine to obtain full backup data, and store the full backup data in a storage disk;

[0007] Backing up the real-time log incremental information and storing it in the log cache disk; wherein the real-time log incremental information is the incremental information corresponding to the full backup data;

[0008] Merging the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle to obtain target backup data; wherein the storage location of the target backup data is the storage disk;

[0009] The real-time log incremental information in the log cache disk is deleted according to the data deletion time period.

[0010] On the one hand, the full backup data is the data generated from the creation of the virtual machine to the full backup; or the full backup data is the data corresponding to the creation of the virtual machine to the merging of the last real-time incremental data into the storage disk.

[0011] On the one hand, the log cache disk is an input / output log disk, and the input / output log disk includes the first basic data and / or the second basic data;

[0012] Wherein, the first basic data includes the timestamp and actual incremental data; the actual incremental data is the input and output data actually sent; the actual incremental data includes the real-time incremental data;

[0013] The second basic data includes at least one of log header data, virtual machine log data, quantity, length, first offset, second offset, third offset, data block size and sequence number; the log header data indicates the quantity of virtual machine log data recorded by the input / output log disk; the virtual machine log data indicates the input / output data recorded each time; the quantity indicates how much data is recorded in the log disk; the length indicates the current size of the log disk; the first offset indicates the offset of the first data in the input / output log disk; the second offset indicates the offset of the last recorded data in the input / output log disk; the third offset indicates the offset of the input / output data located in the storage disk; the data block size indicates the size of the input / output data in the storage disk; the sequence number indicates the number of the data in the input / output log disk;

[0014] Accordingly, after backing up the real-time log incremental information and storing it in the log cache disk, it also includes:

[0015] Sorting all real-time log incremental information according to the timestamp, the first offset, the second offset and the sequence number in the input and output log disk;

[0016] Determine the space size of the current input-output log disk according to the length in the input-output log disk, and if the space size exceeds a preset space threshold, determine to create a new input-output log disk;

[0017] Analyzing the virtual machine log data in the input and output log disk to determine the cause of the failure of the failed virtual machine;

[0018] The log is located based on the log header data in the input and output log disk.

[0019] On the one hand, the number of the log cache disks is multiple, and the real-time log incremental information is backed up and stored in the log cache disk, including:

[0020] Generate a new log cache disk according to the log cache disk generation cycle;

[0021] Correspondingly, the real-time incremental data in the log cache disk is merged with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle to obtain the target backup data, including:

[0022] According to the data merging cycle, the real-time incremental data in the real-time log incremental information in the log cache disk corresponding to the previous log cache disk generation cycle is merged with the full backup data in the storage disk to obtain the target backup data.

[0023] On the one hand, after generating a new log cache disk according to the log cache disk generation cycle, the method further includes:

[0024] All the log cache disks are merged according to the timestamps to obtain a data backup chain including multiple data recovery areas; wherein the data in each data recovery area corresponds to a different data recovery period.

[0025] On the one hand, the multiple data recovery areas include a coarse-grained recovery area, a relatively coarse-grained recovery area and a fine-grained recovery area; wherein the data recovery period of the coarse-grained recovery area is in months, the data recovery period of the relatively coarse-grained recovery area is in days, and the data recovery period of the fine-grained recovery area is in seconds.

[0026] On the one hand, after merging the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle to obtain the target backup data, the method further includes:

[0027] Data recovery is performed based on the target backup data in the storage disk and the current real-time backup data in the log cache disk.

[0028] The present invention also provides a data backup device, comprising:

[0029] A full backup module, used for performing a full backup of the data of the virtual machine, obtaining full backup data, and storing the full backup data in a storage disk;

[0030] A real-time incremental data backup module, used to back up real-time log incremental information and store it in a log cache disk; wherein the real-time log incremental information is incremental information corresponding to the full backup data;

[0031] A backup data merging module, configured to merge the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle, to obtain target backup data; wherein the storage location of the target backup data is the storage disk;

[0032] The data deletion module is used to delete the real-time log increment information in the log cache disk according to the data deletion time period.

[0033] The present invention also provides a data backup device, comprising:

[0034] Memory for storing computer programs;

[0035] A processor is used to execute the computer program to implement the steps of the above data backup method.

[0036] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned data backup method are implemented.

[0037] The data backup method provided by the present invention may include: performing a full backup of the data of a virtual machine to obtain full backup data, and storing the full backup data in a storage disk; performing a back-up of real-time log incremental information and storing it in a log cache disk; wherein the real-time log incremental information is incremental information corresponding to the full backup data; merging the real-time incremental data in the log cache disk with the full backup data in the storage disk according to a data merging cycle and a timestamp in the real-time log incremental information to obtain target backup data; wherein the storage location of the target backup data is the storage disk; and deleting the real-time log incremental information in the log cache disk according to a data deletion time cycle.

[0038] It can be seen from the above technical scheme that the beneficial effect of the present invention is that when performing data backup, a log cache disk will be used for real-time backup, so that the data can be restored to any time point, and because when the data is merged into the storage disk, only the incremental data itself is copied to the storage disk, and other incremental information is not copied to the storage disk, the amount of data stored in the storage disk is reduced, and the incremental data is deleted according to the data deletion time period, so that real-time incremental backup can be performed without causing too much data in the log cache disk. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of a data backup method provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of an input and output log disk provided by an embodiment of the present invention;

[0042] Figure 3A schematic diagram of a data backup chain provided by an embodiment of the present invention;

[0043] Figure 4 An example flowchart of a data backup method provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of redirection provided by an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of the structure of a data backup device provided by an embodiment of the present invention;

[0046] Figure 7 A schematic diagram of the structure of a data backup device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0049] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0050] Next, a flow chart of a data backup method provided by an embodiment of the present invention is described in detail. Figure 1 A data backup method is provided in an embodiment of the present invention, and the data backup method includes:

[0051] S101, performing a full backup of the data of the virtual machine to obtain full backup data, and storing the full backup data in a storage disk.

[0052] The executor of this embodiment is an electronic device. This embodiment does not limit the specific type of storage disk. For example, the storage disk in this embodiment may be a hard disk; or the storage disk in this embodiment may be a virtual disk. This embodiment does not limit the method of obtaining full backup data. For example, this embodiment may take a snapshot of the virtual machine to determine the full backup data; or this embodiment may perform a clone backup to determine the full backup data. The full backup data in this embodiment may be the data generated from the creation of the virtual machine to the full backup; or the full backup data may be the data corresponding to the creation of the virtual machine to the merging of the last real-time incremental data into the storage disk.

[0053] S102, backing up the real-time log incremental information and storing it in the log cache disk; wherein the real-time log incremental information is the incremental information corresponding to the full backup data.

[0054] The log cache disk in this embodiment is a special storage medium for storing logs generated by a virtual machine, and is usually used to record and save various log information generated during the operation of the virtual machine. The real-time log incremental information in this embodiment may include a timestamp and real-time incremental data.

[0055] It should be further explained that the above-mentioned log cache disk is an input and output log disk (IO log disk), and the input and output log disk includes first basic data and / or second basic data; wherein, the first basic data includes a timestamp and actual incremental data; the actual incremental data is the input and output data actually sent; the actual incremental data includes real-time incremental data; wherein, the second basic data includes at least one of log header data, virtual machine log data, quantity, length, first offset, second offset, third offset, data block size and sequence number; the log header data indicates the number of virtual machine log data recorded by the input and output log disk; the virtual machine log data indicates the input and output data recorded each time; the quantity indicates how much data is recorded in the log disk; the length indicates the current size of the log disk; the first offset indicates the offset of the first data in the input and output log disk; the second offset indicates The first offset indicates the offset of the last recorded data in the input-output log disk; the third offset indicates the offset of the input-output data in the storage disk; the data block size indicates the size of the input-output data in the storage disk; the sequence number indicates the number of the data in the input-output log disk; accordingly, after backing up the real-time log incremental information and storing it in the log cache disk, it also includes: sorting all the real-time log incremental information according to the timestamp, the first offset, the second offset and the sequence number in the input-output log disk; determining the space size of the current input-output log disk according to the length in the input-output log disk, and if the space size exceeds the preset space threshold, determining to create a new input-output log disk; analyzing the virtual machine log data in the input-output log disk to determine the cause of the fault of the faulty virtual machine; locating the log based on the log header data in the input-output log disk.

[0056] The first basic data in this embodiment is the real incremental data of the virtual machine (corresponding to the full backup data). The IO log disk in this embodiment contains two parts, superblock (log header data) and entry (virtual machine log data). Superblock can occupy 512 bytes. Entry represents each recorded IO, where count (number) represents how many entries are recorded in the IO log disk, lenth (length) represents the current size of the IO log disk, head (first offset) represents the offset of the first entry in the IO log disk, and tail (second offset) represents the offset of the last recorded entry in the IO log disk. The timestamp represents the time when this IO is generated, offset (third offset) and block size represent the offset and data block size of this IO in the virtual disk, respectively, IO sequence number (sequence number) represents the sequence number of the entry in the IO log disk, and data (real-time incremental data) is the real IO data sent. Each time the virtual machine sends data, a new entry will be added to the IO log disk, and the information of this IO will be recorded in the entry. At the same time, the count in the superblock is increased by 1, and the tail points to the position of the newly recorded entry. For easier understanding, please refer to Figure 2 , Figure 2 A schematic diagram of an input / output log disk provided by an embodiment of the present invention. In summary, the input / output log disk (IO log disk) is designed to include a format of first basic data and second basic data, which can be used to manage data and perform troubleshooting in the future, thereby improving the comprehensiveness of real-time backup data and the comprehensiveness of subsequent data processing.

[0057] It should be further explained that, in order to improve data processing speed and system performance, the number of the above-mentioned log cache disks is multiple, and the real-time log incremental information is backed up and stored in the log cache disk, which may include:

[0058] S1021, generating a new log cache disk according to the log cache disk generation cycle;

[0059] Accordingly, according to the data merging cycle and the timestamp in the real-time log incremental information, the real-time incremental data in the log cache disk is merged with the full backup data in the storage disk to obtain the target backup data, including:

[0060] S1022, according to the data merging cycle, merge the real-time incremental data in the real-time log incremental information in the log cache disk corresponding to the previous log cache disk generation cycle with the full backup data in the storage disk to obtain target backup data.

[0061] In this embodiment, a new log cache disk is generated according to the log cache disk generation cycle, so that when data is merged, the data in one log cache disk can be directly merged. By distributing data and requests on multiple cache disks, excessive load on a single disk can be effectively avoided, thereby speeding up data processing, and when one of the log cache disks is damaged, data at other time nodes will not be lost.

[0062] It should be further explained that, in order to improve the efficiency of data recovery, after generating a new log cache disk according to the log cache disk generation cycle, it can also include: merging data in all log cache disks according to timestamps to obtain a data backup chain including multiple data recovery areas; wherein the data in each data recovery area corresponds to a different data recovery cycle. In this embodiment, the data in the log cache disk is merged to form a data backup chain, and the data in the data backup chain is divided into multiple recovery points, so that when performing data recovery, it can be directly restored according to the recovery point.

[0063] It should be further explained that, in order to improve the data recovery efficiency, the above-mentioned multiple data recovery areas include a coarse-grained recovery area, a relatively coarse-grained recovery area and a fine-grained recovery area; wherein, the data recovery period of the coarse-grained recovery area is in months, the data recovery period of the relatively coarse-grained recovery area is in days, and the data recovery period of the fine-grained recovery area is in seconds. The coarse-grained recovery area in this embodiment is usually used to store data that is not frequently changed, and its recovery period is in months. The data update frequency of the relatively coarse-grained recovery area in this embodiment is higher than that of the coarse-grained area, and its recovery period is in days. The fine-grained recovery area in this embodiment is set for data that requires fast access and high-frequency updates, and its recovery period is in seconds. This embodiment sets different recovery areas, so that when performing data recovery, if the data is located in the log cache disk, the recovery area can be determined according to the recovery point, so that according to the recovery period, one recovery area is selected from the coarse-grained recovery area, the relatively coarse-grained recovery area and the fine-grained recovery area to recover the data, thereby reducing the amount of data required for data recovery and improving data recovery efficiency. It can be understood that in this embodiment, a data backup chain is formed for the backup data, and the data backup chain can be divided into three intervals according to the granularity of the recovery time point, namely, a coarse-grained recovery area, a relatively coarse-grained recovery area, and a fine-grained recovery area. For the coarse-grained area, each recovery point is in months, which can be called a base point. Each base point can be restored to a virtual machine individually. For the relatively fine-grained recovery area, each base point represents one day, and the fine-grained recovery area can be restored to the second level. Therefore, it cannot rely solely on the storage disk, and a log cache disk is also required. Each storage disk has a corresponding log cache disk, which can be restored to any time. During recovery, the corresponding log cache disk is found, and the data before the recovery time point is merged into the storage disk according to the information recorded in the log. Therefore, for the fine-grained recovery area, the closer the recovery time point is to the storage disk, the less data is merged, and the faster the recovery time is. For ease of understanding, please refer to Figure 3 , Figure 3 A schematic diagram of a data backup chain provided by an embodiment of the present invention, wherein each circle in the figure represents a log cache disk.

[0064] S103, according to the data merging cycle and the timestamp in the real-time log incremental information, merge the real-time incremental data in the log cache disk with the full backup data in the storage disk to obtain target backup data; wherein the storage location of the target backup data is the storage disk.

[0065] This embodiment does not limit the specific data merging cycle. For example, the data merging cycle of this embodiment may be daily; or the data merging cycle of this embodiment may also be hourly. In this embodiment, when the real-time incremental data in the log cache disk is merged with the full backup data, the data will be aligned according to the timestamp to improve the continuity of the backup data. This embodiment can subsequently perform data recovery based on the target backup data in the storage disk and the current real-time backup data in the log cache disk.

[0066] S104, deleting the real-time log increment information in the log cache disk according to the data deletion time period.

[0067] This embodiment does not limit the specific data deletion time period, as long as it is ensured that the real-time incremental data has been stored in the storage disk when deleting. For example, the data deletion time period in this embodiment can be 1 month; or the data deletion time period in this embodiment can be one week.

[0068] It is worth noting that high availability (HA) configuration is performed on the virtual machine cluster. When a virtual machine failure is detected, the service can be automatically switched to other nodes in the cluster, greatly reducing the service interruption time.

[0069] It should be further explained that in order to achieve the continuity of backup, the above-mentioned data backup method may also include: analyzing the virtual machine log data corresponding to the historical data loss time period to determine the fault parameters; determining the fault cause and fault result based on the fault parameters; establishing a fault prediction model based on the fault cause and fault result; wherein the fault prediction model is a model that predicts the fault result based on the fault parameters before the virtual machine fails. This embodiment can predict the virtual machine failure in advance based on the fault prediction model during the data backup process, thereby determining the corresponding response strategy for the predicted virtual machine failure. For example, when it is predicted that the target virtual machine will be forced to shut down (predicted virtual machine failure), the response strategy is determined to switch to other virtual machines to take over the work in advance to ensure that the business is not affected. The improved data backup method of the embodiment of the present invention may include: S101, performing a full backup of the data of the virtual machine to obtain full backup data, and storing the full backup data to a storage disk; S102, backing up the real-time log incremental information and storing it to a log cache disk; wherein the real-time log incremental information is incremental information corresponding to the full backup data; S103, merging the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the data merging cycle and the timestamp in the real-time log incremental information to obtain target backup data; wherein the storage location of the target backup data is the storage disk; S104, deleting the real-time log incremental information in the log cache disk according to the data deletion time period. Compared with the current data backup that has a delay and causes data loss, the present invention uses a log cache disk for real-time backup when performing data backup, so that the data can be restored to any time point. Moreover, when performing data merging, only the incremental data itself is copied to the storage disk, and other incremental information is not copied to the storage disk, so that the amount of data stored on the storage disk is reduced, and the incremental data is deleted according to the data deletion time period, so that real-time incremental backup can be performed without causing excessive data in the log cache disk. Furthermore, in this embodiment, the input and output log disk (IO log disk) is designed to include a format of first basic data and second basic data, so that the data can be managed and troubleshooted later, thereby improving the comprehensiveness of real-time backup data and the comprehensiveness of subsequent data processing; and, by distributing data and requests on multiple cache disks, this embodiment can effectively avoid excessive load on a single disk, thereby speeding up data processing, and when one of the log cache disks is damaged, data at other time nodes will not be lost; and, in this embodiment, the data in the log cache disk is merged to form a data backup chain, and the data in the data backup chain is divided into multiple recovery points, so that when performing data recovery, recovery can be performed directly according to the recovery point.

[0070] In order to make the present invention easier to understand, please refer to Figure 4 , Figure 4A flowchart of a data backup method provided by an embodiment of the present invention may specifically include:

[0071] S201, create a backup storage pool and a cache storage pool. The backup storage pool stores a base disk (storage disk), and the cache storage pool is an IO (input and output) log disk.

[0072] The IO log disk in this embodiment corresponds to the log cache disk mentioned above.

[0073] S202, suspend the virtual machine, take a snapshot of the virtual machine, enable IO redirection for the qemu process, and resume the virtual machine.

[0074] The qemu process in this embodiment refers to binary instruction translation. The IO redirection in this embodiment refers to storing incremental data to the IO log disk. For ease of understanding, please refer to Figure 5 , Figure 5 A schematic diagram of redirection provided by an embodiment of the present invention.

[0075] S203, copy the data in the snapshot disk to the storage disk, and then delete the snapshot disk.

[0076] S204, regenerate an IO log disk every hour, and perform a reflink operation on the storage disk to generate a new storage disk.

[0077] The reflink operation in this embodiment is a method for quickly copying files. It creates a reference link file instead of the traditional file copying method, thereby improving copying efficiency and saving storage space. Reflink uses the Copy-On-Write (COW) technology to copy only when the data block changes. Specifically, when using reflink to copy a file, a reference link file that shares the data block with the source file is created. This file has an independent inode node, but does not occupy storage space alone. Only when the data is modified will a data block be actually copied.

[0078] S205, copying the data in the old IO log disk to the newly generated storage disk.

[0079] S206, re-execute steps S204-S205 every hour.

[0080] S207: IO log disks older than the current day may be deleted, and only the last storage disk of each day is retained as a recovery point in the current month.

[0081] S208, for storage disks beyond the current month, only the last storage disk of each month is retained as a recovery point.

[0082] In this embodiment, when the last day of a month is reached, only the storage disk of the last day may be retained and placed in the coarse-grained recovery area.

[0083] S209, when recovering data, if the recovery point is in the coarse-grained recovery area and the relatively coarse-grained recovery area, the storage disk is directly used for recovery. The recovery is to directly copy the storage disk to the virtual disk of the virtual machine. For the disk in the fine-grained recovery area, it is necessary to first copy the storage disk close to the recovery point to the virtual disk of the virtual machine, and then copy the data in the IO log disk to the virtual disk of the virtual machine.

[0084] The present invention provides a virtual machine backup method based on continuous protection. The present invention combines IO redirection technology to back up virtual machine data, and reduces the base disk and IO log disk according to different time. The virtual machine is restored according to different time granularities. The time of the day can be restored by seconds, the time of this month can be restored by days, and the time beyond this month can be restored by months.

[0085] The data backup device provided by an embodiment of the present invention is introduced below. The data backup device described below and the data backup method described above can be referenced to each other.

[0086] Figure 6 A schematic diagram of the structure of a data backup device provided by an embodiment of the present invention, the data backup device may include:

[0087] The full backup module 100 is used to perform a full backup of the data of the virtual machine, obtain the full backup data, and store the full backup data in a storage disk;

[0088] The real-time incremental data backup module 200 is used to back up the real-time log incremental information and store it in the log cache disk; wherein the real-time log incremental information is the incremental information corresponding to the full backup data;

[0089] The backup data merging module 300 is used to merge the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle to obtain target backup data; wherein the storage location of the target backup data is the storage disk;

[0090] The data deletion module 400 is used to delete the real-time log increment information in the log cache disk according to the data deletion time period.

[0091] Further, based on the above embodiment, the full backup data is the data generated from the creation of the virtual machine to the full backup; or the full backup data is the data corresponding to the creation of the virtual machine to the merging of the last real-time incremental data into the storage disk.

[0092] Further based on any of the above embodiments, the log cache disk is an input / output log disk, and the input / output log disk includes the first basic data and / or the second basic data;

[0093] Wherein, the first basic data includes the timestamp and actual incremental data; the actual incremental data is the input and output data actually sent; the actual incremental data includes the real-time incremental data;

[0094] The second basic data includes at least one of log header data, virtual machine log data, quantity, length, first offset, second offset, third offset, data block size and sequence number; the log header data indicates the quantity of virtual machine log data recorded by the input / output log disk; the virtual machine log data indicates the input / output data recorded each time; the quantity indicates how much data is recorded in the log disk; the length indicates the current size of the log disk; the first offset indicates the offset of the first data in the input / output log disk; the second offset indicates the offset of the last recorded data in the input / output log disk; the third offset indicates the offset of the input / output data located in the storage disk; the data block size indicates the size of the input / output data in the storage disk; the sequence number indicates the number of the data in the input / output log disk;

[0095] Accordingly, the data backup device may further include:

[0096] A sorting processing module, used for sorting all real-time log incremental information according to the timestamp, the first offset, the second offset and the sequence number in the input and output log disk;

[0097] A space size determination module, used to determine the space size of the current input-output log disk according to the length in the input-output log disk, and if the space size exceeds a preset space threshold, determine to create a new input-output log disk;

[0098] A fault cause determination module, used to analyze the virtual machine log data in the input and output log disk to determine the fault cause of the faulty virtual machine;

[0099] A data positioning module is used to locate the log based on the log header data in the input and output log disk.

[0100] Further, based on any of the above embodiments, the real-time incremental data backup module 200 includes:

[0101] A generation unit, used to generate a new log cache disk according to a log cache disk generation cycle;

[0102] Accordingly, the backup data merging module 300 includes:

[0103] The data merging unit is used to merge the real-time incremental data in the real-time log incremental information in the log cache disk corresponding to the previous log cache disk generation cycle with the full backup data in the storage disk according to the data merging cycle to obtain the target backup data.

[0104] Further, based on any of the above embodiments, the above data backup device may further include:

[0105] The data backup chain determination module is used to merge the data of all the log cache disks according to the timestamp to obtain a data backup chain including multiple data recovery areas; wherein the data in each data recovery area corresponds to a different data recovery period.

[0106] Further, based on the above embodiment, the multiple data recovery areas include a coarse-grained recovery area, a relatively coarse-grained recovery area and a fine-grained recovery area; wherein, the data recovery period of the coarse-grained recovery area is in months, the data recovery period of the relatively coarse-grained recovery area is in days, and the data recovery period of the fine-grained recovery area is in seconds.

[0107] Further, based on any of the above embodiments, the above data backup device may further include:

[0108] The data recovery module is used to recover the target backup data in the storage disk and the current real-time backup data in the log cache disk.

[0109] It should be noted that the order of the modules and units in the above data backup device can be changed without affecting the logic.

[0110] Figure 6 The description of the features in the corresponding embodiments can be found in Figure 6 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.

[0111] The data backup device provided by the embodiment of the present invention may include: a full backup module 100, which is used to perform a full backup of the data of the virtual machine, obtain the full backup data, and store the full backup data to a storage disk; a real-time incremental data backup module 200, which is used to back up the real-time log incremental information and store it to a log cache disk; wherein the real-time log incremental information is the incremental information corresponding to the full backup data; a backup data merging module 300, which is used to merge the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging period, to obtain the target backup data; wherein the storage location of the target backup data is the storage disk; a data deletion module 400, which is used to delete the real-time log incremental information in the log cache disk according to the data deletion time period. Compared with the current data backup that has a delay and causes data loss, the present invention uses a log cache disk for real-time backup when performing data backup, so that the data can be restored to any time point. Moreover, when performing data merging, only the incremental data itself is copied to the storage disk, and other incremental information is not copied to the storage disk, so that the amount of data stored on the storage disk is reduced, and the incremental data is deleted according to the data deletion time period, so that real-time incremental backup can be performed without causing excessive data in the log cache disk. Furthermore, in this embodiment, the input and output log disk (IO log disk) is designed to include a format of first basic data and second basic data, so that the data can be managed and troubleshooted later, thereby improving the comprehensiveness of real-time backup data and the comprehensiveness of subsequent data processing; and, by distributing data and requests on multiple cache disks, this embodiment can effectively avoid excessive load on a single disk, thereby speeding up data processing, and when one of the log cache disks is damaged, data at other time nodes will not be lost; and, in this embodiment, the data in the log cache disk is merged to form a backup chain, and the data in the backup chain is divided into multiple recovery points, so that when data recovery is performed, it can be directly restored according to the recovery point.

[0112] A data backup device provided by an embodiment of the present invention is introduced below. The data backup device described below and the data backup method described above can refer to each other.

[0113] Figure 7 A schematic diagram of the structure of a data backup device provided by an embodiment of the present invention, such as Figure 7 As shown, the data backup device includes: a memory 60 for storing computer programs;

[0114] The processor 61 is used to implement the steps of the data backup method in the above embodiment when executing a computer program.

[0115] The data backup device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0116] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0117] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein, after the computer program is loaded and executed by the processor 61, it can implement the relevant steps of the data backup method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc. Data 603 may include but is not limited to data in the data backup process, etc.

[0118] In some embodiments, the data backup device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .

[0119] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the data backup device, and may include more or fewer components than those shown in the figure.

[0120] It is understandable that if the data backup method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the current technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk and other media that can store program codes.

[0121] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned data backup method are implemented.

[0122] Based on this, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned data backup method when executed by a processor.

[0123] The above is a detailed introduction to a data backup method, device, equipment and computer-readable storage medium provided by an embodiment of the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0124] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0125] The above is a detailed introduction to a data backup method, device, equipment and computer-readable storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A data backup method, characterized in that: include: Perform a full backup of the data of the virtual machine to obtain full backup data, and store the full backup data in a storage disk; Backing up the real-time log incremental information and storing it in the log cache disk; wherein the real-time log incremental information is the incremental information corresponding to the full backup data; Merging the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle to obtain target backup data; wherein the storage location of the target backup data is the storage disk; The real-time log incremental information in the log cache disk is deleted according to the data deletion time period.

2. The data backup method according to claim 1, characterized in that: The full backup data is data generated from the creation of the virtual machine to the full backup; or the full backup data is data corresponding to the creation of the virtual machine to the merging of the last real-time incremental data into the storage disk.

3. The data backup method according to claim 1, characterized in that: The log cache disk is an input / output log disk, and the input / output log disk includes the first basic data and / or the second basic data; Wherein, the first basic data includes the timestamp and actual incremental data; the actual incremental data is the input and output data actually sent; the actual incremental data includes the real-time incremental data; The second basic data includes at least one of log header data, virtual machine log data, quantity, length, first offset, second offset, third offset, data block size and sequence number; the log header data indicates the quantity of virtual machine log data recorded by the input / output log disk; the virtual machine log data indicates the input / output data recorded each time; the quantity indicates how much data is recorded in the log disk; the length indicates the current size of the log disk; the first offset indicates the offset of the first data in the input / output log disk; the second offset indicates the offset of the last recorded data in the input / output log disk; the third offset indicates the offset of the input / output data located in the storage disk; the data block size indicates the size of the input / output data in the storage disk; the sequence number indicates the number of the data in the input / output log disk; Accordingly, after backing up the real-time log incremental information and storing it in the log cache disk, it also includes: Sorting all real-time log incremental information according to the timestamp, the first offset, the second offset and the sequence number in the input and output log disk; Determine the space size of the current input-output log disk according to the length in the input-output log disk, and if the space size exceeds a preset space threshold, determine to create a new input-output log disk; Analyzing the virtual machine log data in the input and output log disk to determine the cause of the failure of the failed virtual machine; The log is located based on the log header data in the input and output log disk.

4. The data backup method according to any one of claims 1 to 3, characterized in that: The number of the log cache disks is multiple, and the real-time log incremental information is backed up and stored in the log cache disk, including: Generate a new log cache disk according to the log cache disk generation cycle; Correspondingly, the real-time incremental data in the log cache disk is merged with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle to obtain the target backup data, including: According to the data merging cycle, the real-time incremental data in the real-time log incremental information in the log cache disk corresponding to the previous log cache disk generation cycle is merged with the full backup data in the storage disk to obtain the target backup data.

5. The data backup method according to claim 4, characterized in that: After generating a new log cache disk according to the log cache disk generation cycle, the method further includes: All the log cache disks are merged according to the timestamps to obtain a data backup chain including multiple data recovery areas; wherein the data in each data recovery area corresponds to a different data recovery period.

6. The data backup method according to claim 5, characterized in that: The multiple data recovery areas include a coarse-grained recovery area, a relatively coarse-grained recovery area and a fine-grained recovery area; wherein the data recovery period of the coarse-grained recovery area is in months, the data recovery period of the relatively coarse-grained recovery area is in days, and the data recovery period of the fine-grained recovery area is in seconds.

7. The data backup method according to claim 1, characterized in that: After merging the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle to obtain the target backup data, the method further includes: Data recovery is performed based on the target backup data in the storage disk and the current real-time backup data in the log cache disk.

8. A data backup device, characterized in that: include: A full backup module, used for performing a full backup of the data of the virtual machine, obtaining full backup data, and storing the full backup data in a storage disk; A real-time incremental data backup module, used to back up real-time log incremental information and store it in a log cache disk; wherein the real-time log incremental information is incremental information corresponding to the full backup data; A backup data merging module, configured to merge the real-time incremental data in the log cache disk with the full backup data in the storage disk according to the timestamp in the real-time log incremental information according to the data merging cycle, to obtain target backup data; wherein the storage location of the target backup data is the storage disk; The data deletion module is used to delete the real-time log increment information in the log cache disk according to the data deletion time period.

9. A data backup device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the data backup method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data backup method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • System log management method and electronic equipment

    CN121029708A

  • System log management method and electronic device

    CN121029708B