Data backup method and device, electronic equipment and storage medium

Abstract

The invention discloses a data backup method and device, electronic equipment and a storage medium, and relates to the technical field of computers, the data backup method comprises the steps that a cloud hard disk backup interface of a block storage service is expanded, a local backup directory is configured, a snapshot reserved space can be dynamically calculated to optimize storage resource utilization, and the data backup efficiency is improved. In addition, unnecessary directories are eliminated through file system level incremental backup, redundant data transmission is reduced, backup file verification is calculated, data reliability is guaranteed, and therefore the technical problems that in an existing cloud hard disk backup method, due to the fact that a full disk backup strategy is adopted, backup efficiency is low, and storage space is seriously wasted can be solved. The technical effects of improving the backup efficiency of the cloud hard disk, reducing the waste of storage space and guaranteeing the consistency of backup data are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data backup method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the cloud computing environment, OpenStack, as one of the most widely used open source cloud platforms, has become the core architecture of enterprise-level cloud service deployment, and its block storage service (block storage) plays a key role in providing persistent storage volumes for virtual machine instances. Specifically, this technical system covers the whole process from storage volume creation, mounting, data reading and writing to backup and recovery, involving logical volume manager (LVM) storage management, partition table identification, file system mounting and data compression, and many other technical links.

[0003] In the existing cloud disk backup method, the full-disk backup strategy is usually directly used, resulting in low backup efficiency and serious storage space waste. SUMMARY

[0004] The present application provides a data backup method and device, electronic equipment and storage medium to at least solve the problem of low backup efficiency and serious storage space waste in related technologies.

[0005] The present application provides a data backup method, comprising: Extending the target manager cloud disk backup interface of the block storage service, and configuring the local directory of the backup backend; Creating a target manager snapshot, dynamically calculating the snapshot reserved space; Partitioning and mapping the snapshot reserved space, and mounting the mapped partition to the local backup directory; Performing file system level incremental backup on the data in the mounted directory, excluding unnecessary directories, compressing to generate a backup file, and calculating the checksum of the backup file to ensure data consistency.

[0006] Optionally, the creating a target manager snapshot, dynamically calculating the snapshot reserved space further comprises: Adjusting the safety factor dynamically according to the file system type and data change frequency of the cloud disk; Real-time acquisition of used space information in the file system metadata, and calculation of the snapshot reserved space according to the information.

[0007] Optionally, the partitioning and mapping the snapshot reserved space, and mounting the mapped partition to the local backup directory comprises: The partitioning and mapping operation is realized by executing a first preset command, and the path of the snapshot volume is specified in the first preset command. The partition mapping result is checked by executing a second preset command, if the partition table is not successfully identified, it is judged according to the error log whether the partition table format is incorrect or the device is not ready, and the first preset command is re-executed.

[0008] Optionally, the file system level incremental backup is performed on the data in the mounting directory, unnecessary directories are excluded, backup files are generated by compression, and checksums of the backup files are calculated to ensure data consistency, comprising: The unnecessary directories are excluded based on preconfigured exclusion rules; wherein, the recovery directory and the custom exclusion directory are included; The checksums are calculated, and the checksum results are written into a preset file for subsequent backup integrity verification.

[0009] Optionally, the method further comprises: Performing a database backup operation, connecting to a cloud host instance and logically backing up a database, and writing backup results into a backup directory; Judging whether to enable the database backup function according to parameters in a configuration file.

[0010] Optionally, the method further comprises: Receiving a cloud disk backup request issued by a user, and performing a backup feasibility check on the cloud disk through a block storage service; If the cloud disk does not meet the backup requirements, an error information is returned; If the cloud disk is in use, a computing service interface is called to write cache data of the cloud disk into a disk and create a target manager snapshot; If the cloud disk is in an offline state, a target manager snapshot is created.

[0011] Optionally, after the backup files are generated by compression and the checksums of the backup files are calculated, the method further comprises: Unmounting the partition mounted to the local backup directory; Deleting the partition table mapping relationship corresponding to the snapshot volume; Deleting the created target manager snapshot to release storage resources.

[0012] Optionally, the method further comprises: Configuring a periodic backup strategy, setting a backup execution period and a backup file retention period; After each backup task is completed, the historical backup files in the local backup directory are scanned, and the creation time of each backup file is compared with the retention period; Old backup files with a creation time exceeding the retention period are deleted to release redundant storage space.

[0013] Optionally, the method further comprises: generating a backup result report after each backup task is completed, the report containing backup task identification, backup start time, backup end time, backup object, backup file size, backup time consumption, backup status and failure cause; reading a preconfigured list of mail receiving addresses and sending the backup result report in the form of a mail to the addresses in the list.

[0014] Optionally, the mounting of the mapped partition to the local backup directory further comprises: firstly attempting to execute the mounting command, and if the mounting is successful, continuing to execute the subsequent backup steps; if the mounting fails, recording the failure cause to a log and re-executing the mounting command after waiting for a preset time length; if the number of reattempts of mounting exceeds a preset threshold, terminating the current backup task and returning error information of failure of mounting of the partition.

[0015] Optionally, the method further comprises: obtaining, through a cloud host management interface, a list of all cloud hard disks associated with the cloud host, the list containing system disks and data disks; traversing the list of cloud hard disks to determine a target manager and a storage backend corresponding to each cloud hard disk and verifying whether each cloud hard disk supports a backup process; generating a backup file and a checksum file corresponding to each cloud hard disk; associating backup information of all cloud hard disks to an identification of a whole-machine backup task of the cloud host to form a whole-machine backup set, the backup information including a backup file path, a checksum file path and a backup time.

[0016] The application further provides a backup device of data, comprising: an extension unit configured to extend a target manager cloud hard disk backup interface through a block storage service and configure a local directory of a backup backend; a creating unit configured to create a target manager snapshot and dynamically calculate a snapshot reserved space; a mapping unit configured to partition and map the snapshot reserved space and mount the mapped partition to a local backup directory; a compressing unit configured to perform file system level incremental backup on data in the mounted directory, exclude unnecessary directories, compress to generate a backup file and calculate a checksum of the backup file to ensure data consistency.

[0017] Optionally, the creating unit is further configured to: dynamically adjust a security coefficient according to a file system type of the cloud hard disk and a data change frequency; Real-time acquisition of used space information in file system metadata, and calculation of snapshot reserved space according to the information.

[0018] Optionally, the mapping unit is further configured to: The partitioning and mapping operation is implemented by executing a first preset command, and the first preset command specifies a path of the snapshot volume; The partition mapping result is checked by executing a second preset command, if the partition table is not successfully identified, it is determined according to an error log whether the partition table is in a format error or the device is not ready, and the first preset command is re-executed.

[0019] Optionally, the compression unit is further configured to: The unnecessary directories are excluded based on preconfigured exclusion rules, including recovery directories and custom exclusion directories; The checksum is calculated, and the checksum result is written into a preset file for subsequent backup integrity verification.

[0020] Optionally, the apparatus further comprises: The execution unit is configured to perform a database backup operation, connect to a cloud host instance, and perform a logical backup on a database, and write the backup result into a backup directory; The judgment unit is configured to determine whether to enable the database backup function according to parameters in a configuration file.

[0021] Optionally, the apparatus further comprises: The receiving unit is configured to receive a cloud disk backup request issued by a user, and perform a backup feasibility check on the cloud disk through a block storage service; The judgment unit is further configured to return error information when the cloud disk does not meet the backup requirements; The judgment unit is further configured to, when the cloud disk is in a use state, call a computing service interface to write cloud disk cache data into a disk and create a target manager snapshot; The judgment unit is further configured to, when the cloud disk is in an offline state, create a target manager snapshot.

[0022] Optionally, the apparatus further comprises: The uninstallation unit is configured to, after the compression unit compresses to generate a backup file and calculates the checksum of the backup file, unmount the partition mounted to the local backup directory; The uninstallation unit is further configured to delete the partition table mapping relationship corresponding to the snapshot volume; The uninstallation unit is further configured to delete the created target manager snapshot and release storage resources.

[0023] Optionally, the apparatus further comprises: The configuration unit is configured to configure a periodic backup strategy, set a backup execution period and backup file retention days; The configuration unit is further configured to scan historical backup files in a local backup directory after each backup task is completed, and compare the creation time of each backup file with the retention days; The configuration unit is further configured to delete old backup files whose creation time exceeds the retention days, and release redundant storage space.

[0024] Optionally, the apparatus further comprises: The first generation unit is configured to generate a backup result report after each backup task is completed, the report including a backup task identifier, a backup start time, a backup end time, a backup object, a backup file size, a backup time consumption, a backup state and a failure cause; The sending unit is configured to read a preconfigured mail receiving address list, and send the backup result report in the form of a mail to the addresses in the list.

[0025] Optionally, the mapping unit is further configured to: Firstly, attempt to execute a mounting command, if the mounting is successful, continue to execute a subsequent backup step; If the mounting fails, record the failure cause to a log, and wait for a preset time length and then re-execute the mounting command; If the number of times of reattempting the mounting exceeds a preset threshold, terminate the current backup task, and return an error information of a partition mounting failure.

[0026] Optionally, the apparatus further comprises: The acquisition unit is configured to acquire, through a cloud host management interface, a list of all cloud hard disks associated with a cloud host, the list including system disks and data disks; The determination unit is configured to traverse the list of cloud hard disks, determine a target manager and a storage backend corresponding to each cloud hard disk, and verify whether each cloud hard disk supports a backup process; The second generation unit is further configured to generate a backup file and a checksum file corresponding to each cloud hard disk; The association unit is configured to associate backup information of all cloud hard disks to an identifier of an entire machine backup task of the cloud host, form an entire machine backup set, and the backup information includes a backup file path, a checksum file path and a backup time.

[0027] The application further provides an electronic device, comprising a memory configured to store a computer program, and a processor configured to execute the computer program to implement the steps of the data backup method.

[0028] The application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0029] The application further provides a computer program product, which comprises a computer program.

[0030] According to the application, the cloud hard disk backup interface of the block storage service is expanded, the local backup directory is configured, the snapshot reserved space is dynamically calculated to optimize the storage resource utilization, the unnecessary directory is excluded through the file system level incremental backup to reduce the redundant data transmission, the checksum of the backup file is calculated to guarantee the data reliability, and therefore, the technical problems of low backup efficiency and serious storage space waste caused by the full disk backup strategy in the prior cloud hard disk backup method can be solved, the technical effects of improving the cloud hard disk backup efficiency, reducing the storage space waste and guaranteeing the consistency of the backup data are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 A flowchart of a data backup method provided by the embodiments of the application; Figure 2 A structural diagram of a data backup device provided by the embodiments of the application; Figure 3 A structural diagram of another data backup device provided by the embodiments of the application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0034] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0035] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] The embodiments of this application provide a data backup method, and the method is described in detail in conjunction with the execution flow of the data backup method. Figure 1 This is a flowchart illustrating a data backup method provided in an embodiment of this application.

[0037] like Figure 1 As shown, the method includes the following steps: Step 101: Extend the target manager cloud disk backup interface through block storage service and configure the local directory of the backup backend; To implement local disk backup functionality based on LVM, the LVM cloud disk backup interface first needs to be extended through the Cinder service. This is because the traditional Cinder service does not have a dedicated backup interface designed for LVM local disks. The extended interface can receive and process LVM cloud disk backup requests initiated by users, ensuring that subsequent backup operations can be recognized and executed by the system. For example, operations such as verifying whether the cloud disk meets the backup conditions and determining whether to return an error all rely on this extended interface.

[0038] Configure the local directory for the backup backend. This directory should be a local path on the backup server with sufficient storage space. Its function is to serve as the storage medium for LVM cloud disk backup data. Backup files, verification files, and other data generated during the backup process will be stored in this directory. Properly configuring the local directory can avoid problems such as backup data not being able to be written normally due to missing storage paths or insufficient space. It provides basic storage support for the entire LVM local disk backup process and ensures that backup tasks can be carried out in an orderly manner.

[0039] Step 102: Create a target manager snapshot and dynamically calculate the snapshot reserved space; When a user issues a cloud disk backup request and passes the verification of the Cinder-api layer, the operation of creating an LVM snapshot needs to be performed. The LVM snapshot can freeze the data state of the cloud disk during the backup process, prevent data from being tampered with or modified, and thus ensure the consistency of the subsequent backup data, avoiding data damage or inconsistency problems caused by data changes during the backup process.

[0040] When creating an LVM snapshot, the reserved space of the snapshot needs to be calculated dynamically, because the traditional fixed-size snapshot reserved space may be insufficient or wasted. Dynamic calculation can reasonably allocate space according to the actual usage of the cloud disk.

[0041] The specific calculation method follows the formula S_reserved = min(α×S_used, S_total), where α is a safety factor, with a value range of 1.5 to 2.0. This coefficient is set to address the possible sudden data write requirements of the cloud disk after the snapshot is created, avoiding snapshot failure due to insufficient reserved space. S_used refers to the actual used amount of the cloud disk, which needs to be obtained in real time through the dumpe2fs tool to ensure the authenticity and timeliness of the data, providing accurate basis for space calculation. S_total is the total capacity of the cloud disk, used to limit the reserved space not to exceed the total capacity of the cloud disk, preventing space allocation from exceeding the actual available range.

[0042] After completing the space calculation, the lvcreate command is used to create a snapshot, and a reasonable snapshot name is specified, such as combining the cloud disk ID and the snapshot identifier, to facilitate subsequent identification and management of the snapshot.

[0043] In step 103, the snapshot reserved space is partitioned and mapped, and the mapped partition is mounted to the local backup directory. The snapshot reserved space is the specific space calculated earlier according to the formula, used to carry snapshot data. It may contain partition tables corresponding to snapshot volumes, and only through partition mapping can these partition data be accurately identified and accessed. At this time, the kpartx tool is needed, which is a tool in Linux systems for discovering and mapping disk partition tables, and can effectively handle the partition structure in snapshot volumes. Specifically, the kpartx -av command is executed with the snapshot volume path as the parameter to trigger the tool to scan and map the partition table in the snapshot reserved space. After mapping is completed, the mapping result needs to be confirmed by checking the / dev / mapper / directory, which stores all mapped device information in the system. If a partition entry corresponding to the snapshot volume, such as an entry composed of a volume group, a snapshot name, and a dm device identifier, is found in the directory, it indicates that the partition mapping is successful.

[0044] After the partition mapping is successful, the mapped partition needs to be mounted to the local backup directory. The local backup directory is a local path configured in the backup backend before to store backup data. Mounting the partition to this directory can enable the system to directly access the file data in the partition, providing a data reading channel for subsequent backup operations. The mounting operation is implemented through the mount command, which needs to specify the mapped dm device path and the local backup directory path, for example, mounting the / dev / dm-x device to the preset local backup directory. After mounting, the operation result needs to be checked. If no error information is returned, it means that the mounting is successful. If the mounting fails, the partition mapping of the kpartx tool needs to be checked first, and then the mounting is reattempted. If the number of retries exceeds 5, the backup failure information needs to be returned to ensure that the subsequent backup process can be carried out based on the valid mounting state.

[0045] In step 104, file system level incremental backup is performed on the data in the mounting directory, unnecessary directories are excluded, backup files are compressed, and checksums of the backup files are calculated to ensure data consistency.

[0046] The file system level incremental backup is different from the traditional full disk backup. It only backs up the data actually used in the file system, without processing the empty sectors. This feature can significantly reduce the amount of backup data, save the storage space required for backup, and shorten the backup time, especially suitable for scenarios where the actual used space is smaller than the total capacity under LVM logical volume management.

[0047] When performing backup, unnecessary directories such as the lost+found directory need to be excluded. This directory is mainly used to store temporary data generated during file system repair, and does not contain valid information related to user business. Excluding such directories can further reduce the size of backup files and avoid occupying storage resources with invalid data. At the same time, a compression method is used to generate backup files. Usually, tar tools are used in combination with compression functions to package the filtered valid data into tar.gz format. Compression processing can further reduce the storage occupancy of backup files and improve the utilization efficiency of storage resources.

[0048] To ensure the consistency of backup data, the checksum of the backup file also needs to be calculated. The checksum is a fixed length numerical value obtained by calculating the content of the backup file through a specific algorithm. The sha1sum tool is commonly used to complete the calculation, and the calculated checksum is saved separately. Subsequently, the checksums of the backup file at different stages can be compared, such as the checksums before and after data recovery, to determine whether the backup file is damaged or tampered with. If the two checksums are consistent, it means that the backup data is complete and reliable. If they are not consistent, the problem needs to be investigated and the backup needs to be re-executed to ensure the effectiveness of the backup data.

[0049] In some embodiments, the creating a target manager snapshot and dynamically calculating a snapshot reservation space further comprises: Adjust the security coefficient dynamically according to the file system type and data change frequency of the cloud disk; Real-time acquisition of used space information in file system metadata, and calculation of the snapshot reserved space according to the information.

[0050] The file system types of cloud disks are commonly ext4, xfs, etc. Different types of file systems have differences in data storage and read-write mechanisms, and the data change frequency refers to the frequency of data writing, modifying, and deleting in daily use of cloud disks. For example, the data change frequency of business cloud disks used for high-frequency data interaction is high, and the data change frequency of cloud disks used for static data storage is low. Adjust the security coefficient according to different situations. If the cloud disk uses the ext4 file system and the data change frequency is high, in order to cope with sudden data writing requirements and avoid snapshot invalidity due to insufficient space, the security coefficient can be adjusted to a value close to 2.0; if the cloud disk uses the xfs file system and the data change frequency is low, the data writing amount is relatively stable, and the security coefficient can be adjusted to a value close to 1.5. In this way, the snapshot effectiveness can be ensured while avoiding excessive waste of reserved space.

[0051] The used space information in the file system metadata needs to be acquired in real time. The file system metadata records the key configuration and state information of the file system, and the used space information therein can accurately reflect the current actual occupied storage space of the cloud disk, which is more accurate and timely than other acquisition methods. After acquiring the used space information, the snapshot reserved space is calculated according to a preset formula combined with the adjusted security coefficient and the total capacity of the cloud disk, so as to ensure that the calculated reserved space meets the snapshot use requirements and reasonably utilizes the storage resources.

[0052] In some embodiments, the partitioning and mapping of the snapshot reserved space and the mounting of the mapped partition to the local backup directory include: The partitioning and mapping operation is implemented by executing a first preset command, and the path of the snapshot volume is specified in the first preset command; The partition mapping result is checked by executing a second preset command. If the partition table is not successfully identified, it is determined according to the error log whether the partition table format is incorrect or the device is not ready, and the first preset command is reattempted.

[0053] In the OpenStack-based LVM local disk backup process, the snapshot reserved space is a specific storage space used to carry the LVM snapshot data, which is dynamically calculated in advance. When performing partitioning and mapping operations on it, the first preset command needs to be executed. The first preset command is specifically the kpartx -av command, which is a tool command in the Linux system specially used to discover and map disk partition tables. It can deeply scan the snapshot volume corresponding to the snapshot reserved space and accurately identify the partition structure in the volume. When executing this command, the path of the snapshot volume must be specified. The snapshot volume path is usually located under the / dev / cinder-volumes directory, and the path name corresponds to the snapshot name. The snapshot name is generally generated in combination with the cloud disk ID and the snapshot identifier. Accurate specification of the path can ensure that the first preset command can accurately locate the target snapshot volume, avoiding the problem of unable to identify the partition table due to path deviation.

[0054] After the execution of the first preset command is completed, the second preset command needs to be executed to check the partition mapping result. The second preset command is specifically the ll / dev / mapper / command, which is used to view all mapped device information in the system. If the partition mapping is successful, there will be a partition entry corresponding to the snapshot volume under the / dev / mapper / directory. The entry is composed of the volume group snapshot name and the actual mapped dm device identifier. If the partition table is not successfully identified, the error log needs to be checked to determine the specific fault reason. Common reasons include partition table format error and device not ready. Partition table format error may be caused by damage to the partition table structure during the creation of the snapshot volume. Device not ready may be caused by the system not completing the loading process after the creation of the snapshot volume. After determining the reason, the first preset command is re-executed to achieve successful partition mapping as much as possible, and to prepare for the subsequent mounting of the mapped partition to the local backup directory.

[0055] In some embodiments, the file system level incremental backup of the data in the mounting directory is performed, unnecessary directories are excluded, backup files are compressed, and checksums of the backup files are calculated to ensure data consistency, including: Excluding the unnecessary directories based on pre-configured exclusion rules; wherein, including recovery directories and custom excluded directories; Calculating checksums and writing the checksum results into a preset file for subsequent backup integrity verification.

[0056] The recovery directory specifically refers to the lost+found directory, which is a directory used by the Linux file system to store temporary repair data during the repair process and does not contain valid data related to the user's actual business. If it is included in the backup, it will increase the size of the backup file and have no actual recovery value, so it needs to be excluded according to the rules. The custom exclusion directory is a directory that the user sets according to their own business needs and does not need to be backed up, such as a directory that stores temporary running logs and redundant test data. The data in this type of directory usually does not need to be backed up for a long time, and excluding it according to the preconfigured rules can further reduce the amount of backup data and save backup storage space and backup time.

[0057] After completing the exclusion of unnecessary directories, the valid data after screening is compressed to generate a backup file, and then the checksum of the backup file is calculated. The checksum is calculated by a specific algorithm on the entire content of the backup file, which can uniquely identify the content state of the backup file. The commonly used sha1sum tool performs the calculation operation. After the calculation is completed, the checksum result needs to be written into a preset file, which needs to be stored in association with the corresponding backup file in the local directory of the backup backend. Its core function is to provide a basis for subsequent backup integrity verification. When data recovery or backup effectiveness check is performed subsequently, only the checksum of the backup file needs to be recalculated and compared with the checksum result stored in the preset file. If they are consistent, it means that the backup file has not been damaged or tampered with, and the data integrity is guaranteed. If they are not consistent, backup abnormalities can be detected in time so that remedial measures such as re-backup can be taken to ensure the reliability of the backup data.

[0058] In some embodiments, the method further comprises: performing a database backup operation, connecting to a cloud host instance and logically backing up a database, and writing the backup result into a backup directory; determining whether to enable the database backup function according to the parameters in the configuration file.

[0059] The preconfigured exclusion rule is a directory screening mechanism set in advance in the backup system, which can accurately identify and filter directories that do not need to be backed up, ensuring that only valid business data is backed up. Among them, the recovery directory specifically refers to the lost+found directory, which is a directory used by the Linux file system to store temporary repair data during the repair process and does not contain valid data related to the user's actual business. If it is included in the backup, it will increase the size of the backup file and have no actual recovery value, so it needs to be excluded according to the rules. The custom exclusion directory is a directory that the user sets according to their own business needs and does not need to be backed up, such as a directory that stores temporary running logs and redundant test data. The data in this type of directory usually does not need to be backed up for a long time, and excluding it according to the preconfigured rules can further reduce the amount of backup data and save backup storage space and backup time.

[0060] After the unnecessary directory exclusion is completed, the effective data after the screening is compressed to generate a backup file, and then the checksum of the backup file is calculated. The checksum is calculated by a specific algorithm on the entire content of the backup file, which can uniquely identify the content state of the backup file, and the sha1sum tool is commonly used to perform the calculation operation. After the calculation is completed, the checksum result needs to be written into a preset file, which needs to be associated with the corresponding backup file and stored in the local directory of the backup backend. Its core function is to provide a basis for subsequent backup integrity verification. When subsequent data recovery or backup validity check is performed, only the checksum of the backup file needs to be recalculated and compared with the checksum result stored in the preset file. If they are consistent, it means that the backup file has not been damaged or tampered with, and the data integrity is guaranteed. If they are not consistent, backup abnormalities can be found in time so that remedial measures such as re-backup can be taken to ensure the reliability of the backup data.

[0061] In some embodiments, the method further comprises: receiving a cloud disk backup request issued by a user, and performing backup feasibility verification on the cloud disk through a block storage service; if the cloud disk does not meet the backup requirements, returning an error message; if the cloud disk is in a use state, calling a computing service interface to write cloud disk cache data to a disk and create a target manager snapshot; if the cloud disk is in an offline state, creating a target manager snapshot.

[0062] The block storage service, i.e., the Cinder service, performs backup feasibility verification on the cloud disk. The Cinder service is a core component in OpenStack responsible for providing block storage services. Its verification content covers basic state checking of the cloud disk, such as whether the cloud disk is in a normal available state, whether there is physical damage or logical error, whether it has been marked as inoperable by the system, etc. It also verifies whether the cloud disk meets the preconditions for backup, such as whether it exceeds the system's preset upper limit of single-disk backup times, whether it is associated with special configuration items that do not support backup, etc.

[0063] If the cloud hard disk does not meet the backup requirements, the Cinder service directly returns an error message to the user, which clearly indicates the specific reason for not meeting the backup requirements, so that the user can troubleshoot problems according to the prompt, such as repairing the cloud hard disk fault or adjusting the relevant configuration and then reinitiating the backup request. If the verification is passed and it is confirmed that the cloud hard disk can be backed up, the current use state of the cloud hard disk needs to be further judged: if the cloud hard disk is in use, that is, the cloud hard disk is being mounted and used as a system disk or data disk of a virtual machine, at this time, the interface of the computing service, that is, the Nova service, needs to be called, and the cache data currently stored in the memory of the cloud hard disk is written to the disk through the interface. This step operation can ensure that all data in the cloud hard disk is stored persistently, avoiding the loss or inconsistency of subsequent backup data due to the failure of cache data to be written to the disk in time; after the cache data writing is completed, the snapshot of the target manager, that is, the LVM, is created. If the cloud hard disk is in an offline state, that is, the cloud hard disk is not mounted and used by any virtual machine, the cache data writing operation does not need to be performed, and the LVM snapshot can be directly created to prepare for subsequent partition mapping, mounting, and data backup operations.

[0064] In some embodiments, after the backup file is generated by compression and the checksum of the backup file is calculated, the method further comprises: unmounting the partition mounted to the local backup directory; deleting the partition table mapping relationship corresponding to the snapshot volume; deleting the created target manager snapshot to release the storage resource.

[0065] After the compression to generate the backup file and the calculation of the checksum of the backup file are completed, to avoid the influence of temporary resource occupation on system storage efficiency and subsequent operations, the unmounting of the partition, the deletion of the partition table mapping relationship, and the deletion of the target manager snapshot need to be performed in sequence. The target manager, that is, the LVM, is the Logical Volume Manager. The partition mounted to the local backup directory is a partition for accessing snapshot data obtained through partition mapping. At this time, the backup has been completed, and there is no need to access data through the partition. The unmounting operation needs to be performed, which can be realized through the umount command. After unmounting, the system releases the mounting point resource corresponding to the partition, avoiding the long-term occupation of the mounting point, which causes the subsequent mounting operation to be unable to proceed normally.

[0066] After unloading the partition, the partition table mapping relationship corresponding to the snapshot volume needs to be deleted. The mapping relationship is established by the kpartx tool previously. The deletion operation can be completed by the kpartx -dv command. After execution, the mapping entries related to the snapshot volume under the / dev / mapper / directory can be cleaned up, preventing invalid mapping entries from occupying device identifier resources and ensuring the neatness of system device management. Finally, the created LVM snapshot needs to be deleted. The snapshot is temporarily created for backup. After the backup is completed, its mission is over. Continuing to retain it will occupy the storage space of the LVM volume group. The snapshot can be forcibly deleted by the lvremove -f command to release the storage resources occupied by the snapshot, so that these resources can be used for subsequent other LVM related operations or new backup tasks, ensuring the recycling of the entire storage system resources and avoiding resource waste.

[0067] In some embodiments, the method further comprises: configuring a periodic backup policy, setting a backup execution period and a backup file retention period; After each backup task is completed, scanning the historical backup files in the local backup directory, comparing the creation time of each backup file with the retention period; deleting old backup files created more than the retention period, releasing redundant storage space.

[0068] The periodic backup policy is an automatic execution rule set according to the time frequency requirement of user business data backup. Through the policy, LVM local disk backup can be completed periodically without manual intervention, improving the backup efficiency and guaranteeing regular data update. When configuring the policy, the backup execution period and the backup file retention period need to be set. The backup execution period needs to be determined in combination with the data change frequency of the business. In the scene of frequent data change, the period can be set to a shorter interval. In the scene of relatively stable data, the period can be appropriately extended. The backup file retention period is the longest time that the backup file can be retained in the local backup directory. When setting, the size of the local directory storage space and the retention requirement of the business for historical backup data need to be considered to avoid the directory space being occupied by a large number of old backup files due to the retention period being too long.

[0069] After each backup task is completed, the system automatically scans the historical backup files in the local backup directory, which is a previously configured local path for storing backup data. The scanning process traverses all backup files in the directory and obtains the creation time of each file. Then, the creation time of each backup file is compared with the preset retention period to calculate whether the interval between the file creation time and the current time exceeds the retention period. If the creation time of a backup file exceeds the retention period, it means that the file has exceeded the retention range required by the business, and the system will automatically delete such old backup files. By deleting redundant backup files, the system releases the storage space occupied by the redundant backup files, ensures that the local backup directory always has sufficient space to store new backup files, and ensures that the subsequent backup task can be normally executed, maintaining the stable operation of the entire backup process.

[0070] In some embodiments, the method further comprises: After each backup task is completed, a backup result report is generated, which includes backup task identification, backup start time, backup end time, backup object, backup file size, backup time consumption, backup status, and failure reason. The pre-configured email receiving address list is read, and the backup result report is sent to the addresses in the list in the form of an email.

[0071] The backup result report is a centralized record of key information for this backup task. The backup task identification is a unique identifier that distinguishes this task from other backup tasks. The backup start time and backup end time record the specific times when the task starts and completes, respectively, clearly presenting the time range of the task execution. The backup object specifies the cloud hard disk or cloud host that this backup is targeting, allowing relevant personnel to know the data source. The backup file size reflects the storage space occupied by the backup file, providing a basis for storage resource management. The backup time consumption is calculated by the difference between the end time and the start time, which can be used to evaluate the backup efficiency. The backup status directly displays whether the task is successful or failed, and if it fails, the failure reason will be detailed to indicate the problem, such as snapshot creation failure due to partition mounting exception, providing a direction for subsequent troubleshooting.

[0072] At the same time, the system reads a pre-configured email receiving address list, which is a set of email addresses of relevant personnel previously set in the backup system. According to business needs, the email addresses of operation and maintenance personnel, management personnel, and other roles can be added. After reading the list, the system sends the backup result report to each address in the list in the form of an email, ensuring that relevant personnel can timely grasp the backup task situation and quickly respond to handle backup failures, ensuring the monitorability and traceability of the backup process.

[0073] In some embodiments, the mounting of the mapped partition to the local backup directory further comprises: The mounting command is executed for the first time. If the mounting is successful, the subsequent backup steps are executed. If the mounting fails, the failure reason is recorded in the log, and the mounting command is re-executed after waiting for a preset time period. If the number of reattempts to mount exceeds a preset threshold, the current backup task is terminated, and error information indicating that the partition mounting fails is returned.

[0074] In the LVM local disk backup process based on OpenStack, when the mapped partition is mounted to the local backup directory, the mounting command is executed for the first time. The mapped partition is the partition obtained after the kpartx tool is used to discover and map the LVM snapshot volume. LVM is the Logical Volume Manager, which can realize flexible management of disk space. The local backup directory is a local path configured in advance by the backup backend to store backup data. The purpose of executing the mounting command is to enable the system to access the snapshot data in the mapped partition through the local backup directory, so as to prepare for subsequent extraction of valid data for backup. If the first mounting is successful, it indicates that the local backup directory and the mapped partition have established a stable data access channel, and the subsequent file system level incremental backup steps can be executed. If the first mounting fails, the system records the specific failure reason in the log. These failure reasons can include that the mapped partition is not correctly identified, the local backup directory has insufficient permissions, or the directory path does not exist. Recording the log facilitates subsequent maintenance personnel to troubleshoot problems.

[0075] The system re-executes the mounting command after waiting for a preset time period. The preset time period is set to avoid mounting failures caused by temporary problems such as transient system resource shortage or partition mapping delay, and to give the system a certain buffer time. If the number of reattempts to mount exceeds a preset threshold, it indicates that the mounting failure can be caused by a persistent problem. At this time, the current backup task is terminated, and error information indicating that the partition mounting fails is returned to the user, prompting relevant personnel to handle the problem in a timely manner, avoiding invalid retry operations that occupy system resources, and ensuring the rationality and efficiency of the entire backup process.

[0076] In some embodiments, the method further comprises: obtaining, through a cloud host management interface, a list of all cloud disks associated with the cloud host, the list including system disks and data disks; traversing the list of cloud disks to determine the target manager and storage backend corresponding to each cloud disk, and verifying whether each cloud disk supports the backup process; generating a backup file and a checksum file corresponding to each cloud disk; associating the backup information of all cloud disks to the whole-machine backup task identifier of the cloud host to form a whole-machine backup set, the backup information including a backup file path, a checksum file path, and a backup time.

[0077] Nova API as the core interface responsible for cloud host instance management in OpenStack can accurately call the storage resource information bound to the target cloud host, and the list contains both the system disk providing the operating system running environment for the cloud host and the data disk storing user business data, and complete acquisition of the list is the basis for realizing whole machine backup, which can avoid missing any associated key storage data.

[0078] Subsequently, the cloud disk list obtained is traversed, and the corresponding target manager and storage backend are determined for each cloud disk. The target manager specifically refers to LVM, i.e., Logical Volume Manager, which realizes flexible storage management by abstracting physical disks into logical volumes; the storage backend is the underlying storage carrier actually relied on by the cloud disk, and it is necessary to confirm whether the storage backend is of the LVM type, and to verify whether each cloud disk supports the backup process, the verification content including whether the cloud disk is in a normal available state, whether there is a configuration exception affecting backup, etc. If a cloud disk does not support backup, problems need to be promptly investigated to ensure the integrity of the whole machine backup.

[0079] After verification, the corresponding backup file and checksum file are generated for each cloud disk according to the previous backup process. The backup file is generated by file system level incremental backup and compression, and only contains actual used data to save space; the checksum file is generated by calculating the checksum of the backup file, and is used to ensure the consistency of the backup data.

[0080] The backup information of all cloud disks is associated to the whole machine backup task identifier of the cloud host, and the backup information covers the storage path of the backup file in the local backup directory, the storage path of the checksum file, and the backup time of each cloud disk. After binding these information with the whole machine backup task identifier, the whole machine backup set is formed, and the identifier can be used to quickly locate and manage all backup data of the cloud host, providing convenient support for whole machine data recovery.

[0081] Through the description of the above implementation manner, those skilled in the art can clearly understand that the method according to the above embodiment can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better implementation manner.

[0082] The embodiment of the application also provides a data backup device, as shown in Figure 2 The device comprises: An extension unit 21 is configured to extend a target manager cloud disk backup interface through a block storage service, and configure a local directory of a backup backend; A creation unit 22 is configured to create a target manager snapshot and dynamically calculate a snapshot reserved space. The mapping unit 23 is configured to partition and map the snapshot reserved space, and mount the mapped partition to a local backup directory. The compression unit 24 is configured to perform file system level incremental backup on data in the mounted directory, exclude unnecessary directories, compress to generate a backup file, and calculate a checksum of the backup file to ensure data consistency.

[0083] Further, in a possible implementation of the embodiment of the application, the creating unit 22 is further configured to: dynamically adjust the security coefficient according to the file system type and data change frequency of the cloud disk; real-time acquire used space information in the file system metadata, and calculate the snapshot reserved space according to the information.

[0084] Further, in a possible implementation of the embodiment of the application, the mapping unit 23 is further configured to: the partitioning and mapping operation is implemented by executing a first preset command, and the first preset command specifies a path of a snapshot volume; The partition mapping result is checked by executing a second preset command. If the partition table is not successfully identified, it is determined according to an error log whether the partition table is in a format error or the device is not ready, and the first preset command is reattempted to be executed.

[0085] Further, in a possible implementation of the embodiment of the application, the compression unit 24 is further configured to: exclude the unnecessary directories based on preconfigured exclusion rules, including a recovery directory and a custom excluded directory; calculate a checksum, and write the checksum result into a preset file, which is used for subsequent backup integrity verification.

[0086] Further, in a possible implementation of the embodiment of the application, as shown in Figure 3 the device further comprises: The executing unit 25 is configured to perform a database backup operation, connect to a cloud host instance, and perform a logical backup on a database, and write a backup result into a backup directory. The judging unit 26 is configured to determine whether to enable a database backup function according to a parameter in a configuration file.

[0087] Further, in a possible implementation of the embodiment of the application, as shown in Figure 3 the device further comprises: The receiving unit 27 is configured to receive a cloud disk backup request issued by a user, and perform a backup feasibility check on the cloud disk through a block storage service. The judging unit 26 is further configured to return error information when the cloud hard disk does not meet the backup requirement. The judging unit 26 is further configured to invoke a computing service interface to write the cloud hard disk cache data to a disk and create a target manager snapshot when the cloud hard disk is in a use state. The judging unit 26 is further configured to create a target manager snapshot when the cloud hard disk is in an offline state.

[0088] Further, in a possible implementation of the embodiment of the present application, as shown in Figure 3 The device further includes: The unloading unit 28 is configured to unload the partition mounted to the local backup directory after the compression unit 24 generates the backup file by compression and calculates the checksum of the backup file. The unloading unit 28 is further configured to delete the partition table mapping relationship corresponding to the snapshot volume. The unloading unit 28 is further configured to delete the created target manager snapshot and release the storage resource.

[0089] Further, in a possible implementation of the embodiment of the present application, as shown in Figure 3 The device further includes: The configuring unit 29 is configured to configure a periodic backup strategy, set a backup execution period and a backup file retention period. The configuring unit 29 is further configured to scan the historical backup files in the local backup directory after each backup task is completed, and compare the creation time of each backup file with the retention period. The configuring unit 29 is further configured to delete the old backup file whose creation time exceeds the retention period, and release the redundant storage space.

[0090] Further, in a possible implementation of the embodiment of the present application, as shown in Figure 3 The device further includes: The first generating unit 210 is configured to generate a backup result report after each backup task is completed, the report including a backup task identifier, a backup start time, a backup end time, a backup object, a backup file size, a backup time consumption, a backup state and a failure cause. The sending unit 211 is configured to read a preconfigured mail receiving address list, and send the backup result report in the form of a mail to the addresses in the list.

[0091] Further, in a possible implementation of the embodiment of the present application, the mapping unit 23 is further configured to: First attempt to execute a mounting command, if the mounting is successful, continue to execute the subsequent backup steps; If the mounting fails, record the failure reason to the log, and wait for a preset time length to re-execute the mounting command; If the number of re-attempted mounting exceeds a preset threshold, terminate the current backup task, and return error information of partition mounting failure.

[0092] Further, in a possible implementation manner of the embodiment of the application, as shown in Figure 3 The apparatus further includes: The acquisition unit 212 is configured to acquire, through a cloud host management interface, a list of all cloud hard disks associated with the cloud host, the list including system disks and data disks. The determination unit 213 is configured to traverse the list of cloud hard disks, determine a target manager and a storage backend corresponding to each cloud hard disk, and verify whether each cloud hard disk supports a backup process. The second generation unit 214 is further configured to generate a backup file and a checksum file corresponding to each cloud hard disk. The association unit 215 is configured to associate backup information of all cloud hard disks to an identification of a whole-machine backup task of the cloud host, to form a whole-machine backup set, the backup information including a backup file path, a checksum file path, and a backup time.

[0093] The features of the embodiment of the data backup apparatus can be referred to the related descriptions of the embodiment of the data backup method, which will not be repeated here.

[0094] The embodiment of the application further provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the data backup method embodiments.

[0095] The embodiment of the application further provides a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the steps in any one of the data backup method embodiments when running.

[0096] In an example embodiment, the computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0097] The embodiment of the application further provides a computer program product, the computer program product includes a computer program, and the computer program is executed by a processor to implement the steps in any one of the data backup method embodiments.

[0098] The embodiment of the present application further provides another computer program product comprising a nonvolatile computer readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above data backup method embodiments.

[0099] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, various components have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation does not depart from the scope of the present application.

[0100] The above provides a data backup method and device, an electronic device and a storage medium. The principles and implementation modes of the present application are described by applying specific examples. The above description of the embodiments is only for the purpose of helping to understand the method and its core idea. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A backup method of data, characterized by, The method comprises: Extending the target manager cloud hard disk backup interface through the block storage service, and configuring the local directory of the backup backend; Creating a target manager snapshot, and dynamically calculating a snapshot reserved space; Partitioning and mapping the snapshot reserved space, and mounting the mapped partition to the local backup directory; Performing a file system level incremental backup on the data in the mounted directory, excluding unnecessary directories, compressing to generate a backup file, and calculating a checksum of the backup file to ensure data consistency.

2. The method of claim 1, wherein, The creating a target manager snapshot and dynamically calculating a snapshot reserved space further comprises: Dynamically adjusting a safety coefficient according to the file system type and data change frequency of the cloud hard disk; Real-time acquisition of used space information in the file system metadata, and calculation of the snapshot reserved space according to the information.

3. The method of claim 1, wherein, The partitioning and mapping the snapshot reserved space, and mounting the mapped partition to the local backup directory further comprises: The partitioning and mapping operation is implemented by executing a first preset command, and the path of the snapshot volume is specified in the first preset command; The partition mapping result is checked by executing a second preset command, if the partition table is not successfully identified, it is judged whether the partition table format is incorrect or the device is not ready according to the error log, and the first preset command is reattempted to be executed.

4. The method of claim 1, wherein, The performing a file system level incremental backup on the data in the mounted directory, excluding unnecessary directories, compressing to generate a backup file, and calculating a checksum of the backup file to ensure data consistency further comprises: Excluding the unnecessary directories based on preconfigured exclusion rules; wherein, the recovery directory and the custom exclusion directory are included; Calculating the checksum, and writing the checksum result into a preset file for subsequent backup integrity verification.

5. The method of claim 1, wherein, The method further comprises: Performing a database backup operation, connecting to a cloud host instance and logically backing up a database, and writing the backup result into a backup directory; Judging whether to enable the database backup function according to the parameters in the configuration file.

6. The method of claim 1, wherein, The method further comprises: Receiving a cloud hard disk backup request issued by a user, and performing a backup feasibility check on the cloud hard disk through the block storage service; If the cloud hard disk does not meet the backup requirements, error information is returned; If the cloud hard disk is in use, calling a computing service interface to write the cloud hard disk cache data to the disk, and creating a target manager snapshot; If the cloud hard disk is in an offline state, creating a target manager snapshot.

7. The method of claim 1, wherein, After compressing to generate a backup file and calculating a checksum of the backup file, the method further comprises: Unmounting the partition mounted to the local backup directory; Deleting the partition table mapping relationship corresponding to the snapshot volume; Deleting the created target manager snapshot, and releasing the storage resource.

8. The method of claim 1, wherein, The method further comprises: Configuring a periodic backup strategy, setting a backup execution period and a backup file retention period; After each backup task is completed, scanning the historical backup files in the local backup directory, and comparing the creation time of each backup file with the retention period; Deleting the old backup files with a creation time exceeding the retention period, and releasing the redundant storage space.

9. The method of claim 1, wherein, The method further comprises: A backup result report is generated after each backup task is completed, and the report contains backup task identification, backup start time, backup end time, backup object, backup file size, backup time consumption, backup status, and failure cause; A pre-configured mail receiving address list is read, and the backup result report is sent to the addresses in the list in the form of a mail.

10. The method of claim 1, wherein, The mounting of the mapped partition to the local backup directory further includes: A mounting command is executed for the first time, and if the mounting is successful, subsequent backup steps are executed; If the mounting fails, the failure cause is recorded in a log, and the mounting command is re-executed after a preset time period; If the number of re-attempted mounting exceeds a preset threshold, the current backup task is terminated, and error information indicating that the partition mounting fails is returned.

11. The method of claim 1, wherein, It includes: A cloud host management interface is used to obtain a list of all cloud disks associated with the cloud host, and the list contains system disks and data disks; The cloud disk list is traversed to determine the target manager and storage backend corresponding to each cloud disk, and it is verified whether each cloud disk supports the backup process; Backup files and checksum files corresponding to each cloud disk are generated; Backup information of all cloud disks is associated with the whole-machine backup task identification of the cloud host to form a whole-machine backup set, and the backup information includes backup file path, checksum file path, and backup time.

12. A backup apparatus of data, characterized by comprising: It includes: An extension unit is configured to extend the target manager cloud disk backup interface through a block storage service and configure a local directory of the backup backend; A creation unit is configured to create a target manager snapshot and dynamically calculate a snapshot reserved space; A mapping unit is configured to partition and map the snapshot reserved space and mount the mapped partition to a local backup directory; A compression unit is configured to perform file system level incremental backup on data in the mounted directory, exclude unnecessary directories, compress to generate a backup file, and calculate a checksum of the backup file to ensure data consistency.

13. An electronic device, comprising: It includes: A memory is configured to store a computer program; A processor is configured to execute the computer program to implement the steps of the data backup method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the data backup method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the data backup method according to any one of claims 1 to 11.

Images