Backup method and device of data, electronic device and storage medium

Abstract

The application discloses a data backup method and device, an electronic device and a storage medium, relates to the technical field of computers, and comprises a cloud hard disk backup interface extended by a block storage service and a local backup directory configured, can dynamically calculate a snapshot reserved space to optimize storage resource utilization, and can also exclude unnecessary directories through a file system level incremental backup to reduce redundant data transmission, and calculate a backup file checksum to guarantee data reliability, so that the technical problem of low backup efficiency and serious storage space waste caused by a full disk backup strategy in the prior cloud hard disk backup method can be solved, and the technical effects of improving cloud hard disk backup efficiency, reducing storage space waste, and guaranteeing backup data consistency are achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to data backup methods and apparatus, electronic devices and storage media. Background Technology

[0002] In cloud computing environments, OpenStack, as one of the most widely adopted open-source cloud platforms, has become the core architecture for enterprise-level cloud service deployments. Its block storage service plays a crucial role in providing persistent storage volumes for virtual machine instances. Specifically, this technology system covers the entire process from storage volume creation, mounting, data reading and writing to backup and recovery, involving multiple technical aspects such as Logical Volume Manager (LVM) storage management, partition table identification, file system mounting, and data compression.

[0003] Existing cloud disk backup methods typically employ a full disk backup strategy, resulting in low backup efficiency and significant waste of storage space. Summary of the Invention

[0004] This application provides a data backup method and apparatus, electronic device and storage medium to at least solve the problems of low backup efficiency and serious waste of storage space in the related art.

[0005] This application provides a data backup method, including:

[0006] Extend the Target Manager cloud disk backup interface through block storage service and configure the local directory of the backup backend;

[0007] Create a target manager snapshot and dynamically calculate the snapshot reserved space;

[0008] The snapshot reserved space is partitioned and mapped, and the mapped partitions are mounted to the local backup directory;

[0009] Perform file system-level incremental backups of the data in the mounted directory, exclude unnecessary directories, compress the backup files, and calculate the checksum of the backup files to ensure data consistency.

[0010] Optionally, the creation of the target manager snapshot and the dynamic calculation of the snapshot reserved space further include:

[0011] The security level is dynamically adjusted based on the file system type and data change frequency of the cloud disk.

[0012] The system obtains used space information from the file system metadata in real time and calculates the reserved space for the snapshot based on this information.

[0013] Optionally, the step of partitioning and mapping the snapshot reserved space, and mounting the mapped partition to the local backup directory includes:

[0014] The partitioning and mapping operation is implemented by executing a first preset command, in which the path of the snapshot volume is specified;

[0015] The partition mapping result is checked by executing the second preset command. If the partition table is not successfully identified, the error log is used to determine whether the partition table format is incorrect or the device is not ready, and the first preset command is re-executed.

[0016] Optionally, performing file system-level incremental backups of the data in the mounted directory, excluding unnecessary directories, compressing the backup files, and calculating the checksum of the backup files to ensure data consistency includes:

[0017] The non-essential directories are excluded based on pre-configured exclusion rules; these include recovery directories and custom exclusion directories.

[0018] Calculate the checksum and write the checksum result to a preset file for subsequent backup integrity verification.

[0019] Optionally, the method further includes:

[0020] Perform a database backup operation, connect to the cloud host instance and perform a logical backup of the database, and write the backup results to the backup directory;

[0021] Determine whether to enable the database backup function based on the parameters in the configuration file.

[0022] Optionally, the method further includes:

[0023] Receive cloud disk backup requests from users and perform backup feasibility verification on the cloud disks through block storage services;

[0024] If the cloud disk does not meet the backup requirements, an error message will be returned;

[0025] If the cloud disk is in use, call the computing service interface to write the cloud disk cache data to the disk and create a target manager snapshot;

[0026] If the cloud disk is offline, create a Target Manager snapshot.

[0027] Optionally, after compressing and generating the backup file and calculating the checksum of the backup file, the method further includes:

[0028] Unmount the partition that has been mounted to the local backup directory;

[0029] Delete the partition table mapping relationship corresponding to the snapshot volume;

[0030] Delete the created target manager snapshot to free up storage resources.

[0031] Optionally, the method further includes:

[0032] Configure a periodic backup policy, setting the backup execution cycle and the number of days to retain backup files;

[0033] After each backup task is completed, scan the historical backup files in the local backup directory and compare the creation time of each backup file with the stated retention period;

[0034] Delete old backup files created more than the specified retention period to free up redundant storage space.

[0035] Optionally, the method further includes:

[0036] After each backup task is completed, a backup result report is generated. The report includes the backup task identifier, backup start time, backup end time, backup object, backup file size, backup time, backup status, and failure reason.

[0037] Read the pre-configured list of email receiving addresses and send the backup result report as an email to the addresses in the list.

[0038] Optionally, mounting the mapped partition to the local backup directory further includes:

[0039] The first attempt to execute the mount command will succeed. If the mount is successful, continue with the subsequent backup steps.

[0040] If mounting fails, record the reason for the failure in the log and wait for the preset time before re-executing the mount command;

[0041] If the number of attempts to mount exceeds the preset threshold, the backup task will be terminated, and an error message indicating that the partition mount failed will be returned.

[0042] Optional, including:

[0043] Obtain a list of all cloud disks associated with the cloud server through the cloud server management interface. The list includes system disks and data disks.

[0044] Traverse the cloud disk list to determine the target manager and storage backend for each cloud disk, and verify whether each cloud disk supports the backup process.

[0045] Generate backup files and checksum files for each cloud disk;

[0046] The backup information of all cloud disks is associated with the full machine backup task identifier of the cloud host to form a full machine backup set. The backup information includes the backup file path, checksum file path and backup time.

[0047] This application also provides a data backup device, including:

[0048] The extension unit is used to extend the target manager cloud disk backup interface through block storage services and configure the local directory of the backup backend;

[0049] Create a unit to create target manager snapshots and dynamically calculate snapshot reserved space;

[0050] The mapping unit is used to partition and map the snapshot reserved space, and mount the mapped partition to the local backup directory.

[0051] The compression unit is used to perform file system-level incremental backups of the data in the mounted directory, exclude unnecessary directories, compress the data to generate backup files, and calculate the checksum of the backup files to ensure data consistency.

[0052] Optionally, the creation unit is further configured to:

[0053] The security level is dynamically adjusted based on the file system type and data change frequency of the cloud disk.

[0054] The system obtains used space information from the file system metadata in real time and calculates the reserved space for the snapshot based on this information.

[0055] Optionally, the mapping unit is further configured to:

[0056] The partitioning and mapping operation is implemented by executing a first preset command, in which the path of the snapshot volume is specified;

[0057] The partition mapping result is checked by executing the second preset command. If the partition table is not successfully identified, the error log is used to determine whether the partition table format is incorrect or the device is not ready, and the first preset command is re-executed.

[0058] Optionally, the compression unit is further configured to:

[0059] The non-essential directories are excluded based on pre-configured exclusion rules; these include recovery directories and custom exclusion directories.

[0060] Calculate the checksum and write the checksum result to a preset file for subsequent backup integrity verification.

[0061] Optionally, the device further includes:

[0062] The execution unit is used to perform database backup operations, connect to the cloud host instance and perform logical backups of the database, and write the backup results to the backup directory;

[0063] The judgment unit is used to determine whether to enable the database backup function based on the parameters in the configuration file.

[0064] Optionally, the device further includes:

[0065] The receiving unit is used to receive cloud disk backup requests issued by users and perform backup feasibility verification on the cloud disk through block storage services;

[0066] The judgment unit is also used to return an error message when the cloud hard drive does not meet the backup requirements;

[0067] The judgment unit is also used to call the computing service interface to write the cloud disk cache data to the disk and create a target manager snapshot when the cloud disk is in use;

[0068] The judgment unit is also used to create a target manager snapshot when the cloud disk is offline.

[0069] Optionally, the device further includes:

[0070] The unloading unit is used to unload the partition that has been mounted to the local backup directory after compressing the backup file generated by the compression unit and calculating the checksum of the backup file.

[0071] The unloading unit is also used to delete the partition table mapping relationship corresponding to the snapshot volume;

[0072] The unloading unit is also used to delete created target manager snapshots and free up storage resources.

[0073] Optionally, the device further includes:

[0074] The configuration unit is used to configure periodic backup strategies, setting the backup execution cycle and the number of days to retain backup files;

[0075] The configuration unit is also used to scan historical backup files in the local backup directory after each backup task is completed, and compare the creation time of each backup file with the retention days.

[0076] The configuration unit is also used to delete old backup files whose creation time exceeds the retention period, thereby freeing up redundant storage space.

[0077] Optionally, the device further includes:

[0078] The first generation unit is used to generate a backup result report after each backup task is completed. The report includes the backup task identifier, backup start time, backup end time, backup object, backup file size, backup time, backup status, and failure reason.

[0079] The sending unit is used to read a pre-configured list of email receiving addresses and send the backup result report to the addresses in the list in the form of an email.

[0080] Optionally, the mapping unit is further configured to:

[0081] The first attempt to execute the mount command will succeed. If the mount is successful, continue with the subsequent backup steps.

[0082] If mounting fails, record the reason for the failure in the log and wait for the preset time before re-executing the mount command;

[0083] If the number of attempts to mount exceeds the preset threshold, the backup task will be terminated, and an error message indicating that the partition mount failed will be returned.

[0084] Optionally, the device further includes:

[0085] The acquisition unit is used to obtain a list of all cloud disks associated with a cloud server through the cloud server management interface. The list includes system disks and data disks.

[0086] The determination unit is used to traverse the list of cloud disks, determine the target manager and storage backend corresponding to each cloud disk, and verify whether each cloud disk supports the backup process.

[0087] The second generation unit is also used to generate backup files and checksum files corresponding to each cloud disk;

[0088] The association unit is used to associate the backup information of all cloud disks with the whole machine backup task identifier of the cloud host to form a whole machine backup set. The backup information includes the backup file path, checksum file path and backup time.

[0089] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the data backup method described above when executing the computer program.

[0090] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described data backup methods.

[0091] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described data backup methods.

[0092] This application extends the cloud disk backup interface of the block storage service and configures a local backup directory. It can dynamically calculate snapshot reserved space to optimize storage resource utilization. It also eliminates unnecessary directories and reduces redundant data transmission through file system-level incremental backup. At the same time, it calculates backup file checksums to ensure data reliability. Therefore, it can solve the technical problems of low backup efficiency and serious storage space waste caused by the use of full disk backup strategy in existing cloud disk backup methods. It achieves the technical effect of improving cloud disk backup efficiency, reducing storage space waste, and ensuring backup data consistency. Attached Figure Description

[0093] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0094] Figure 1 A schematic flowchart illustrating a data backup method provided in an embodiment of this application;

[0095] Figure 2 This is a schematic diagram of the structure of a data backup device provided in an embodiment of this application;

[0096] Figure 3 This is a schematic diagram of another data backup device provided in an embodiment of this application. Detailed Implementation

[0097] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0098] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0099] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0100] The embodiments of this application provide a data backup method, and the method is described in detail in conjunction with the execution flow of the data backup method. Figure 1 This is a flowchart illustrating a data backup method provided in an embodiment of this application.

[0101] like Figure 1 As shown, the method includes the following steps:

[0102] Step 101: Extend the target manager cloud disk backup interface through block storage service and configure the local directory of the backup backend;

[0103] To implement local disk backup functionality based on LVM, the LVM cloud disk backup interface first needs to be extended through the Cinder service. This is because the traditional Cinder service does not have a dedicated backup interface designed for LVM local disks. The extended interface can receive and process LVM cloud disk backup requests initiated by users, ensuring that subsequent backup operations can be recognized and executed by the system. For example, operations such as verifying whether the cloud disk meets the backup conditions and determining whether to return an error all rely on this extended interface.

[0104] Configure the local directory for the backup backend. This directory should be a local path on the backup server with sufficient storage space. Its function is to serve as the storage medium for LVM cloud disk backup data. Backup files, verification files, and other data generated during the backup process will be stored in this directory. Properly configuring the local directory can avoid problems such as backup data not being able to be written normally due to missing storage paths or insufficient space. It provides basic storage support for the entire LVM local disk backup process and ensures that backup tasks can be carried out in an orderly manner.

[0105] Step 102: Create a target manager snapshot and dynamically calculate the snapshot reserved space;

[0106] Once a user issues a cloud disk backup request and passes the verification by the Cinder API layer, an LVM snapshot needs to be created. The LVM snapshot can freeze the data state of the cloud disk during the backup process to prevent data from being tampered with or modified, thereby ensuring the consistency of subsequent backup data and avoiding data corruption or inconsistency caused by data changes during the backup process.

[0107] When creating an LVM snapshot, the snapshot reserved space needs to be dynamically calculated. This is because the traditional fixed-size snapshot reserved space may be insufficient or wasted. Dynamic calculation can reasonably allocate space according to the actual usage of the cloud disk.

[0108] The specific calculation method follows the formula S_reserved = min(α×S_used, S_total), where α is a safety factor, ranging from 1.5 to 2.0. This factor is set to cope with the sudden data writing demand that may occur after the cloud disk is created, and to avoid the snapshot from becoming invalid due to insufficient reserved space. S_used refers to the actual amount of cloud disk used, which needs to be obtained in real time through the dumpe2fs tool to ensure the authenticity and timeliness of the data and to provide an accurate basis for space calculation. S_total is the total capacity of the cloud disk, which is used to limit the reserved space from not exceeding the total capacity of the cloud disk to prevent the space allocation from exceeding the actual available range.

[0109] After completing the space calculation, create a snapshot using the lvcreate command, and specify a reasonable snapshot name, such as combining the cloud disk ID and the snapshot identifier, so that the snapshot can be identified and managed later.

[0110] Step 103: Partition and map the snapshot reserved space, and mount the mapped partition to the local backup directory;

[0111] Snapshot reserved space is a specific space calculated using a formula to hold snapshot data. It may contain the partition table corresponding to the snapshot volume. Only through partition mapping can this partition data be accurately identified and accessed. This requires the use of the kpartx tool, a Linux system tool for discovering and mapping disk partition tables. It effectively handles the partition structure within snapshot volumes. Specifically, executing the `kpartx -av` command with the snapshot volume path as the parameter triggers the tool to scan and map the partition table within the snapshot reserved space. After mapping, the mapping result needs to be confirmed by checking the ` / dev / mapper / ` directory. This directory stores information about all mapped devices in the system. If a partition entry corresponding to the snapshot volume can be found in it, such as an entry consisting of the volume group, snapshot name, and dm device identifier, it indicates that the partition mapping was successful.

[0112] After successful partition mapping, the mapped partition needs to be mounted to the local backup directory. The local backup directory is the local path configured in the backup backend for storing backup data. Mounting the partition to this directory allows the system to directly access the file data within the partition, providing a data read channel for subsequent backup operations. The mount operation is performed using the `mount` command, specifying both the mapped dm device path and the local backup directory path, for example, mounting the ` / dev / dm-x` device to the default local backup directory. After mounting, the operation result needs to be checked. If no error message is returned, the mount is successful. If the mount fails, first check if the partition mapping in the `kpartx` tool is correct, then try mounting again. If the number of retries exceeds 5, a backup failure message should be returned to ensure that subsequent backup processes can proceed based on a valid mount state.

[0113] Step 104: Perform file system-level incremental backups of the data in the mounted directory, exclude unnecessary directories, compress and generate backup files, and calculate the checksum of the backup files to ensure data consistency.

[0114] File system level incremental backup differs from traditional full disk backup. It only backs up the data actually used within the file system and does not require processing empty sectors. This feature can significantly reduce the amount of backup data, saving storage space required for backup and shortening backup time. It is especially suitable for scenarios where the actual space used is often less than the total capacity under LVM logical volume management.

[0115] When performing backups, unnecessary directories, such as the lost+found directories, must be excluded. These directories primarily store temporary data generated during file system repair and do not contain valid information related to user business. Excluding such directories further reduces the size of backup files and prevents invalid data from consuming storage resources. Simultaneously, backup files are generated using compression, typically employing the tar tool in conjunction with compression functionality to package the filtered valid data into a tar.gz format. Compression further reduces the storage footprint of backup files and improves the efficiency of storage resource utilization.

[0116] To ensure the consistency of backup data, a checksum of the backup file also needs to be calculated. The checksum is a fixed-length value calculated using a specific algorithm on the backup file content, commonly using the sha1sum tool. The calculated checksum is saved separately. Later, by comparing the checksums of the backup file at different stages, such as the checksum at the end of the backup and the checksum before data recovery, it can be determined whether the backup file has been corrupted or tampered with. If the two checksums match, it indicates that the backup data is complete and reliable; if they do not match, the problem needs to be investigated and the backup re-executed to ensure the validity of the backup data.

[0117] In some embodiments, the creation of a target manager snapshot and the dynamic calculation of snapshot reserved space further include:

[0118] The security level is dynamically adjusted based on the file system type and data change frequency of the cloud disk.

[0119] The system obtains used space information from the file system metadata in real time and calculates the reserved space for the snapshot based on this information.

[0120] Common file system types for cloud disks include ext4 and xfs. Different file systems differ in their data storage and read / write mechanisms. Data change frequency refers to how frequently data is written, modified, and deleted during daily use. For example, cloud disks used for high-frequency data interaction have a high data change frequency, while those used for static data storage have a low data change frequency. Adjusting the security factor according to different situations is crucial. If the cloud disk uses the ext4 file system and has a high data change frequency, to handle sudden data write demands and prevent snapshots from becoming invalid due to insufficient space, the security factor can be adjusted to a value close to 2.0. If the cloud disk uses the xfs file system and has a low data change frequency with relatively stable data write volume, the security factor can be adjusted to a value close to 1.5. This ensures the validity of snapshots while avoiding excessive waste of reserved space.

[0121] It is necessary to obtain the used space information from the file system metadata in real time. The file system metadata records the key configuration and status information of the file system, and the used space information can accurately reflect the actual storage space currently occupied by the cloud disk, which is more accurate and timely than other acquisition methods. After obtaining this used space information, combined with the adjusted security factor and the total capacity of the cloud disk, the snapshot reserved space is calculated according to the preset formula to ensure that the calculated reserved space meets the snapshot usage requirements and makes reasonable use of storage resources.

[0122] In some embodiments, partitioning and mapping the snapshot reserved space, and mounting the mapped partition to the local backup directory includes:

[0123] The partitioning and mapping operation is implemented by executing a first preset command, in which the path of the snapshot volume is specified;

[0124] The partition mapping result is checked by executing the second preset command. If the partition table is not successfully identified, the error log is used to determine whether the partition table format is incorrect or the device is not ready, and the first preset command is re-executed.

[0125] In the OpenStack-based LVM local disk backup process, the snapshot reserved space is a specific storage space dynamically calculated beforehand to hold LVM snapshot data. When performing partitioning and mapping operations on this space, it must be achieved by executing a first preset command. The first preset command is specifically the `kpartx -av` command, a utility command in Linux systems specifically used to discover and map disk partition tables. It can deeply scan the snapshot volume corresponding to the snapshot reserved space and accurately identify the partition structure within the volume. When executing this command, the path to the snapshot volume must be specified. The snapshot volume path is usually located in the ` / dev / cinder-volumes` directory. The path name corresponds to the snapshot name, which is generally generated by combining the cloud disk ID and the snapshot identifier. Accurately specifying the path ensures that the first preset command can accurately locate the target snapshot volume, avoiding problems caused by path deviations that prevent the partition table from being identified.

[0126] After executing the first preset command, the partition mapping result needs to be checked by executing the second preset command, specifically `ll / dev / mapper / `. This command is used to view the information of all mapped devices in the system. If the partition mapping is successful, a partition entry corresponding to the snapshot volume will appear in the ` / dev / mapper / ` directory. The entry consists of the volume group snapshot name and the actual mapped DM device identifier. If the partition table is not successfully identified, the error log needs to be checked to determine the specific cause of the failure. Common causes include incorrect partition table format and device insecurity. Incorrect partition table format may be caused by damage to the partition table structure during snapshot volume creation, while device insecurity may be due to the system not having completed the loading process after snapshot volume creation. After identifying the cause, try executing the first preset command again to try to achieve successful partition mapping as much as possible, preparing for mounting the mapped partition to the local backup directory later.

[0127] In some embodiments, performing file system-level incremental backups of the data in the mounted directory, excluding unnecessary directories, compressing the backup files, and calculating the checksum of the backup files to ensure data consistency includes:

[0128] The non-essential directories are excluded based on pre-configured exclusion rules; these include recovery directories and custom exclusion directories.

[0129] Calculate the checksum and write the checksum result to a preset file for subsequent backup integrity verification.

[0130] The recovery directory specifically refers to the lost+found directory. This directory is used by the Linux file system to store temporary repair data during the repair process. It does not contain valid data related to the user's actual business. Including it in the backup will increase the size of the backup file and has no actual recovery value. Therefore, it needs to be excluded according to the rules. Custom exclusion directories are directories that users set according to their own business needs and do not need to be backed up. For example, directories that store temporary running logs and redundant test data. Data in these directories usually does not need to be backed up for a long time. Excluding them according to the pre-configured rules can further reduce the amount of backup data and save backup storage space and backup time.

[0131] After excluding unnecessary directories, the filtered valid data is compressed to generate a backup file, and then a checksum is calculated for this backup file. The checksum is calculated using a specific algorithm on the entire content of the backup file, uniquely identifying the content status of the backup file. The sha1sum tool is commonly used to perform the calculation. After calculation, the checksum result must be written to a preset file, which must be associated with the corresponding backup file and stored in the local directory of the backup backend. Its core function is to provide a basis for subsequent backup integrity verification. When performing data recovery or backup validity checks, it is only necessary to recalculate the checksum of the backup file and compare it with the checksum result stored in the preset file. If they match, it indicates that the backup file has not been damaged or tampered with, and data integrity is guaranteed; if they do not match, backup anomalies can be detected in time, allowing remedial measures such as re-backup to ensure the reliability of the backup data.

[0132] In some embodiments, the method further includes:

[0133] Perform a database backup operation, connect to the cloud host instance and perform a logical backup of the database, and write the backup results to the backup directory;

[0134] Determine whether to enable the database backup function based on the parameters in the configuration file.

[0135] Pre-configured exclusion rules are directory filtering mechanisms set in advance in the backup system. They can accurately identify and filter directories that do not need to be backed up, ensuring that only valid business data is backed up. Specifically, the recovery directory refers to the lost+found directory. This directory is used by the Linux file system to store temporary repair data during the repair process and does not contain valid data related to the user's actual business. Including it in the backup would increase the backup file size and have no practical recovery value, so it must be excluded according to the rules. Custom exclusion directories are directories that users set according to their own business needs and do not need to be backed up, such as directories storing temporary running logs and redundant test data. Data in these directories usually does not require long-term backup, and excluding them according to pre-configured rules can further reduce the amount of backup data, saving backup storage space and backup time.

[0136] After excluding unnecessary directories, the filtered valid data is compressed to generate a backup file, and then a checksum is calculated for this backup file. The checksum is calculated using a specific algorithm on the entire content of the backup file, uniquely identifying the content status of the backup file. The sha1sum tool is commonly used to perform the calculation. After calculation, the checksum result must be written to a preset file, which must be associated with the corresponding backup file and stored in the local directory of the backup backend. Its core function is to provide a basis for subsequent backup integrity verification. When performing data recovery or backup validity checks, it is only necessary to recalculate the checksum of the backup file and compare it with the checksum result stored in the preset file. If they match, it indicates that the backup file has not been damaged or tampered with, and data integrity is guaranteed; if they do not match, backup anomalies can be detected in time, allowing remedial measures such as re-backup to ensure the reliability of the backup data.

[0137] In some embodiments, the method further includes:

[0138] Receive cloud disk backup requests from users and perform backup feasibility verification on the cloud disks through block storage services;

[0139] If the cloud disk does not meet the backup requirements, an error message will be returned;

[0140] If the cloud disk is in use, call the computing service interface to write the cloud disk cache data to the disk and create a target manager snapshot;

[0141] If the cloud disk is offline, create a Target Manager snapshot.

[0142] The backup feasibility is verified by using the block storage service, Cinder. As a core component of OpenStack responsible for providing block storage services, Cinder performs a verification that covers the basic status checks of the cloud disk, such as whether the cloud disk is in a normal and available state, whether there is physical damage or logical error, and whether it has been marked as inoperable by the system. It also verifies whether the cloud disk meets the prerequisites for backup, such as whether it exceeds the system's preset limit for the number of backups per disk, and whether it is associated with special configuration items that do not support backup.

[0143] If the verification determines that the cloud disk does not meet the backup requirements, the Cinder service will directly return an error message to the user. The error message will clearly indicate the specific reason why the backup is not met, allowing the user to troubleshoot the problem based on the prompts, such as fixing cloud disk malfunctions or adjusting relevant configurations before re-initiating the backup request. If the verification passes and the cloud disk is confirmed to be ready for backup, the current usage status of the cloud disk needs to be further determined: If the cloud disk is in use, meaning it is being mounted and used as the system disk or data disk of a virtual machine, the interface of the compute service, namely the Nova service, needs to be called. This interface writes the cached data currently temporarily stored in memory to the disk. This step ensures that all data in the cloud disk is persistently stored, avoiding subsequent backup data loss or inconsistency due to the failure to write cached data to disk in a timely manner. After the cached data is written, a snapshot of the target manager, namely LVM, is created. If the cloud disk is offline, meaning it is not mounted and used by any virtual machine, there is no need to perform the cached data writing operation. An LVM snapshot can be created directly to prepare for subsequent partition mapping, mounting, and data backup operations.

[0144] In some embodiments, after compressing and generating backup files and calculating the checksum of the backup files, the method further includes:

[0145] Unmount the partition that has been mounted to the local backup directory;

[0146] Delete the partition table mapping relationship corresponding to the snapshot volume;

[0147] Delete the created target manager snapshot to free up storage resources.

[0148] After compressing and generating the backup file and calculating its checksum, to avoid temporary resource consumption affecting system storage efficiency and subsequent operations, it is necessary to sequentially perform the operations of unmounting the partition, deleting the partition table mapping relationship, and deleting the target manager snapshot. The target manager is LVM, or Logical Volume Manager. The partition previously mounted to the local backup directory was obtained through partition mapping and used to access snapshot data. Now that the backup is complete, there is no need to access data through this partition again, so an unmount operation must be performed. This can be achieved using the `umount` command. After unmounting, the system will release the mount point resources corresponding to the partition, preventing the mount point from being occupied for a long time and causing subsequent mount operations to fail.

[0149] After unmounting the partition, the partition table mapping relationship corresponding to the snapshot volume needs to be deleted. This mapping relationship was previously established using the kpartx tool. The deletion operation can be completed using the `kpartx -dv` command. After execution, it will clean up the mapping entries related to the snapshot volume in the ` / dev / mapper / ` directory, preventing invalid mapping entries from occupying device identifier resources and ensuring the cleanliness of system device management. Finally, the created LVM snapshot needs to be deleted. This snapshot is temporarily created for backup. After the backup is completed, its purpose is over. Continuing to keep it will occupy the storage space of the LVM volume group. The `lvremove -f` command can be used to forcibly delete the snapshot, releasing the storage resources occupied by the snapshot so that these resources can be reused for other subsequent LVM-related operations or new backup tasks, ensuring the recycling of the entire storage system resources and avoiding resource waste.

[0150] In some embodiments, the method further includes:

[0151] Configure a periodic backup policy, setting the backup execution cycle and the number of days to retain backup files;

[0152] After each backup task is completed, scan the historical backup files in the local backup directory and compare the creation time of each backup file with the stated retention period;

[0153] Delete old backup files created more than the specified retention period to free up redundant storage space.

[0154] Periodic backup strategies are automated execution rules set according to the user's business requirements for data backup frequency. This strategy enables regular LVM local disk backups to be performed without manual intervention, improving backup efficiency and ensuring timely data updates. When configuring this strategy, both the backup execution cycle and the number of days to retain backup files must be set. The backup execution cycle should be determined based on the frequency of business data changes; for scenarios with frequent data changes, the cycle can be set to a shorter interval, while for scenarios with relatively stable data, the cycle can be appropriately extended. The number of days to retain backup files is the maximum time that backup files can remain in the local backup directory. When setting this, the local directory storage space size and the business's requirements for retaining historical backup data must be considered to avoid the directory space being occupied by a large number of old backup files due to excessively long retention periods.

[0155] After each backup task is completed, the system automatically scans historical backup files in the local backup directory. This local backup directory is the previously configured backend local path used to store backup data. The scan process traverses all backup files in the directory and obtains the creation time of each file. Then, it compares the creation time of each backup file with the preset retention period to calculate whether the interval between the file's creation time and the current time exceeds the retention period. If the creation time of a backup file exceeds the retention period, it indicates that the file has exceeded the retention range required by the business. The system automatically deletes this type of old backup file, freeing up storage space occupied by redundant backup files. This ensures that the local backup directory always has sufficient space to store new backup files, guaranteeing that subsequent backup tasks can execute normally and maintaining the stable operation of the entire backup process.

[0156] In some embodiments, the method further includes:

[0157] After each backup task is completed, a backup result report is generated. The report includes the backup task identifier, backup start time, backup end time, backup object, backup file size, backup time, backup status, and failure reason.

[0158] Read the pre-configured list of email receiving addresses and send the backup result report as an email to the addresses in the list.

[0159] The backup result report is a centralized record of key information for this backup task. The backup task identifier is the unique identifier that distinguishes this task, preventing confusion with other backup tasks. The backup start time and backup end time record the specific times of task initiation and completion, clearly presenting the time range of task execution. The backup object clearly identifies the cloud disk or cloud host targeted by this backup, letting relevant personnel know the data source. The backup file size reflects the storage space occupied by the backup file, providing a basis for storage resource management. The backup time is calculated by the difference between the end time and the start time, which can be used to evaluate backup efficiency. The backup status intuitively displays whether the task succeeded or failed. If it failed, the reason for failure will be explained in detail, such as abnormal partition mounting or snapshot creation failure, providing direction for subsequent troubleshooting.

[0160] Simultaneously, the system reads a pre-configured list of email addresses. This list is a collection of email addresses pre-set in the backup system for relevant personnel, and email addresses for roles such as operations and maintenance personnel and administrators can be added according to business needs. After reading the list, the system sends a backup result report as an email to each address in the list, ensuring that relevant personnel are promptly informed of the backup task status and can respond quickly in case of backup failure, thus guaranteeing the monitorability of the backup process and the traceability of problems.

[0161] In some embodiments, mounting the mapped partition to the local backup directory further includes:

[0162] The first attempt to execute the mount command will succeed. If the mount is successful, continue with the subsequent backup steps.

[0163] If mounting fails, record the reason for the failure in the log and wait for the preset time before re-executing the mount command;

[0164] If the number of attempts to mount exceeds the preset threshold, the backup task will be terminated, and an error message indicating that the partition mount failed will be returned.

[0165] In the OpenStack-based LVM local disk backup process, when mounting the mapped partition to the local backup directory, the system first attempts to execute the mount command. The mapped partition is the partition obtained after partition table discovery and mapping of the LVM snapshot volume using the kpartx tool. LVM, or Logical Volume Manager, enables flexible disk space management. The local backup directory is a pre-configured local path for storing backup data. Executing the mount command allows the system to access the snapshot data within the mapped partition through the local backup directory, preparing for subsequent data extraction and backup. If the initial mount is successful, it indicates that a stable data access channel has been established between the local backup directory and the mapped partition, and subsequent steps such as file system-level incremental backups can proceed. If the initial mount fails, the system logs the specific reasons for the failure. These reasons may include incorrect recognition of the mapped partition, insufficient permissions in the local backup directory, or a non-existent directory path. Logging facilitates troubleshooting by maintenance personnel.

[0166] The system will wait a preset time before re-executing the mount command. This preset time is to allow the system some buffer time to avoid mount failures caused by temporary issues such as momentary system resource shortages or partition mapping delays. If the number of re-attempts exceeds the preset threshold, it indicates that the mount failure may be caused by a persistent problem. In this case, the backup task will be terminated, and an error message indicating partition mount failure will be returned to the user to prompt relevant personnel to address the issue promptly. This avoids unnecessary retries consuming system resources and ensures the rationality and efficiency of the entire backup process.

[0167] In some embodiments, the method further includes:

[0168] Obtain a list of all cloud disks associated with the cloud server through the cloud server management interface. The list includes system disks and data disks.

[0169] Traverse the cloud disk list to determine the target manager and storage backend for each cloud disk, and verify whether each cloud disk supports the backup process.

[0170] Generate backup files and checksum files for each cloud disk;

[0171] The backup information of all cloud disks is associated with the full machine backup task identifier of the cloud host to form a full machine backup set. The backup information includes the backup file path, checksum file path and backup time.

[0172] The Nova API, as the core interface in OpenStack responsible for managing cloud host instances, can accurately retrieve storage resource information bound to the target cloud host. The list includes both system disks that provide the operating system environment for the cloud host and data disks that store user business data. Obtaining this complete list is the foundation for implementing full machine backup and can avoid missing any related critical storage data.

[0173] The obtained list of cloud disks is then iterated through to determine the corresponding target manager and storage backend for each cloud disk. The target manager specifically refers to LVM (Logical Volume Manager), which abstracts physical disks into logical volumes to achieve flexible storage management. The storage backend is the underlying storage medium on which the cloud disks actually rely. Here, it is necessary to confirm whether the storage backend is of type LVM and verify whether each cloud disk supports the backup process. Verification includes whether the cloud disks are in a normal and available state and whether there are any configuration anomalies affecting backups. If a cloud disk does not support backups, the problem must be investigated promptly to ensure the integrity of the entire machine backup.

[0174] After successful verification, a corresponding backup file and checksum file are generated for each cloud disk according to the previous backup process. The backup file is generated through incremental backup and compression at the file system level, containing only the data actually used to save space; the checksum file is generated by calculating the checksum of the backup file to ensure the consistency of the backup data.

[0175] The backup information of all cloud disks is associated with the full machine backup task identifier of the cloud host. The backup information includes the storage path of the backup files in the local backup directory, the storage path of the checksum file, and the backup time of each cloud disk. After this information is bound to the full machine backup task identifier, a full machine backup set is formed. Subsequently, all backup data of the cloud host can be quickly located and managed through this identifier, providing convenient support for full machine data recovery.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0177] Embodiments of this application also provide a data backup device, such as... Figure 2 As shown, it includes:

[0178] Extension unit 21 is used to extend the target manager cloud disk backup interface through block storage service and configure the local directory of the backup backend;

[0179] Create unit 22, which is used to create target manager snapshots and dynamically calculate snapshot reserved space;

[0180] Mapping unit 23 is used to partition and map the snapshot reserved space, and mount the mapped partition to the local backup directory;

[0181] Compression unit 24 is used to perform file system-level incremental backups of data in the mounted directory, exclude unnecessary directories, compress and generate backup files, and calculate the checksum of the backup files to ensure data consistency.

[0182] Furthermore, in one possible implementation of this application embodiment, the creation unit 22 is further configured to:

[0183] The security level is dynamically adjusted based on the file system type and data change frequency of the cloud disk.

[0184] The system obtains used space information from the file system metadata in real time and calculates the reserved space for the snapshot based on this information.

[0185] Furthermore, in one possible implementation of this application embodiment, the mapping unit 23 is further used for:

[0186] The partitioning and mapping operation is implemented by executing a first preset command, in which the path of the snapshot volume is specified;

[0187] The partition mapping result is checked by executing the second preset command. If the partition table is not successfully identified, the error log is used to determine whether the partition table format is incorrect or the device is not ready, and the first preset command is re-executed.

[0188] Furthermore, in one possible implementation of this application embodiment, the compression unit 24 is further configured to:

[0189] The non-essential directories are excluded based on pre-configured exclusion rules; these include recovery directories and custom exclusion directories.

[0190] Calculate the checksum and write the checksum result to a preset file for subsequent backup integrity verification.

[0191] Furthermore, in one possible implementation of the embodiments of this application, such as Figure 3 As shown, the device further includes:

[0192] Execution unit 25 is used to perform database backup operations, connect to the cloud host instance and perform logical backup of the database, and write the backup results to the backup directory;

[0193] Judgment unit 26 is used to determine whether to enable the database backup function based on the parameters in the configuration file.

[0194] Furthermore, in one possible implementation of the embodiments of this application, such as Figure 3 As shown, the device further includes:

[0195] Receiving unit 27 is used to receive cloud disk backup requests issued by users and perform backup feasibility verification on the cloud disk through block storage service.

[0196] The judgment unit 26 is also used to return an error message when the cloud hard drive does not meet the backup requirements;

[0197] The judgment unit 26 is also used to call the computing service interface to write the cloud disk cache data to the disk and create a target manager snapshot when the cloud disk is in use.

[0198] The determination unit 26 is also used to create a target manager snapshot when the cloud disk is offline.

[0199] Furthermore, in one possible implementation of the embodiments of this application, such as Figure 3 As shown, the device further includes:

[0200] The unloading unit 28 is used to compress the backup file generated by the compression unit 24, and after calculating the checksum of the backup file, unload the partition that has been mounted to the local backup directory.

[0201] The unloading unit 28 is also used to delete the partition table mapping relationship corresponding to the snapshot volume;

[0202] Unloading unit 28 is also used to delete created target manager snapshots and free up storage resources.

[0203] Furthermore, in one possible implementation of the embodiments of this application, such as Figure 3 As shown, the device further includes:

[0204] Configuration unit 29 is used to configure periodic backup strategies, setting the backup execution cycle and the number of days to retain backup files;

[0205] Configuration unit 29 is also used to scan historical backup files in the local backup directory after each backup task is completed, and compare the creation time of each backup file with the retention days;

[0206] Configuration unit 29 is also used to delete old backup files whose creation time exceeds the retention period, thereby freeing up redundant storage space.

[0207] Furthermore, in one possible implementation of the embodiments of this application, such as Figure 3 As shown, the device further includes:

[0208] The first generation unit 210 is used to generate a backup result report after each backup task is completed. The report includes the backup task identifier, backup start time, backup end time, backup object, backup file size, backup time, backup status, and failure reason.

[0209] Sending unit 211 is used to read a pre-configured list of email receiving addresses and send the backup result report to the addresses in the list in the form of an email.

[0210] Furthermore, in one possible implementation of this application embodiment, the mapping unit 23 is further used for:

[0211] The first attempt to execute the mount command will succeed. If the mount is successful, continue with the subsequent backup steps.

[0212] If mounting fails, record the reason for the failure in the log and wait for the preset time before re-executing the mount command;

[0213] If the number of attempts to mount exceeds the preset threshold, the backup task will be terminated, and an error message indicating that the partition mount failed will be returned.

[0214] Furthermore, in one possible implementation of the embodiments of this application, such as Figure 3 As shown, the device further includes:

[0215] Acquisition unit 212 is used to acquire a list of all cloud disks associated with a cloud server through a cloud server management interface, the list including system disks and data disks;

[0216] The determination unit 213 is used to traverse the list of cloud disks, determine the target manager and storage backend corresponding to each cloud disk, and verify whether each cloud disk supports the backup process.

[0217] The second generation unit 214 is also used to generate backup files and checksum files corresponding to each cloud disk;

[0218] The association unit 215 is used to associate the backup information of all cloud disks with the whole machine backup task identifier of the cloud host to form a whole machine backup set. The backup information includes the backup file path, checksum file path and backup time.

[0219] For a description of the features in the embodiment corresponding to the data backup device, please refer to the relevant description in the embodiment corresponding to the data backup method, which will not be repeated here.

[0220] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described data backup method embodiments.

[0221] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data backup method embodiments when running.

[0222] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0223] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data backup method embodiments.

[0224] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data backup method embodiments.

[0225] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0226] The foregoing has provided a detailed description of a data backup method, apparatus, electronic device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data backup method, characterized in that, include: Extend the Target Manager cloud disk backup interface through block storage service and configure the local directory of the backup backend; Create a target manager snapshot and dynamically calculate the snapshot reserved space; The snapshot reserved space is partitioned and mapped, and the mapped partitions are mounted to the local backup directory; Perform file system-level incremental backups of the data in the mounted directory, exclude unnecessary directories, compress the backup files, and calculate the checksum of the backup files to ensure data consistency. The creation of the target manager snapshot and the dynamic calculation of the snapshot reserved space also include: The security factor is dynamically adjusted based on the file system type and data change frequency of the cloud disk; whereby the data change frequency refers to the frequency of data writing, modification, and deletion in the cloud disk. The used space information in the file system metadata is obtained in real time. Combined with the adjusted security factor and the total capacity of the cloud disk, the snapshot reserved space is calculated according to S_reserved = min(α×S_used, S_total); where α is the security factor, S_used refers to the actual amount of cloud disk used, and S_total is the total capacity of cloud disk; the used space information reflects the actual amount of cloud disk used. The step of performing file system-level incremental backups of the data in the mounted directory, excluding unnecessary directories, compressing the backup files, and calculating the checksum of the backup files to ensure data consistency includes: The non-essential directories are excluded based on pre-configured exclusion rules; these include recovery directories and custom exclusion directories. Calculate the checksum and write the checksum result to a preset file for subsequent backup integrity verification.

2. The method according to claim 1, characterized in that, The step of partitioning and mapping the snapshot reserved space, and mounting the mapped partition to the local backup directory includes: The partitioning and mapping operation is implemented by executing a first preset command, in which the path of the snapshot volume is specified; The partition mapping result is checked by executing the second preset command. If the partition table is not successfully identified, the error log is used to determine whether the partition table format is incorrect or the device is not ready, and the first preset command is re-executed.

3. The method according to claim 1, characterized in that, The method further includes: Perform a database backup operation, connect to the cloud host instance and perform a logical backup of the database, and write the backup results to the backup directory; Determine whether to enable the database backup function based on the parameters in the configuration file.

4. The method according to claim 1, characterized in that, The method further includes: Receive cloud disk backup requests from users and perform backup feasibility verification on the cloud disks through block storage services; If the cloud disk does not meet the backup requirements, an error message will be returned; If the cloud disk is in use, call the computing service interface to write the cloud disk cache data to the disk and create a target manager snapshot; If the cloud disk is offline, create a Target Manager snapshot.

5. The method according to claim 1, characterized in that, After compressing and generating the backup file, and simultaneously calculating the checksum of the backup file, the method further includes: Unmount the partition that has been mounted to the local backup directory; Delete the partition table mapping relationship corresponding to the snapshot volume; Delete the created target manager snapshot to free up storage resources.

6. The method according to claim 1, characterized in that, The method further includes: Configure a periodic backup policy, setting the backup execution cycle and the number of days to retain backup files; After each backup task is completed, scan the historical backup files in the local backup directory and compare the creation time of each backup file with the stated retention period; Delete old backup files created more than the specified retention period to free up redundant storage space.

7. The method according to claim 1, characterized in that, The method further includes: After each backup task is completed, a backup result report is generated. The report includes the backup task identifier, backup start time, backup end time, backup object, backup file size, backup time, backup status, and failure reason. Read the pre-configured list of email receiving addresses and send the backup result report as an email to the addresses in the list.

8. The method according to claim 1, characterized in that, The step of mounting the mapped partition to the local backup directory also includes: The first attempt to execute the mount command will succeed. If the mount is successful, continue with the subsequent backup steps. If mounting fails, record the reason for the failure in the log and wait for the preset time before re-executing the mount command; If the number of attempts to mount exceeds the preset threshold, the backup task will be terminated, and an error message indicating that the partition mount failed will be returned.

9. The method according to claim 1, characterized in that, include: Obtain a list of all cloud disks associated with the cloud server through the cloud server management interface. The list includes system disks and data disks. Traverse the cloud disk list to determine the target manager and storage backend for each cloud disk, and verify whether each cloud disk supports the backup process. Generate backup files and checksum files for each cloud disk; The backup information of all cloud disks is associated with the full machine backup task identifier of the cloud host to form a full machine backup set. The backup information includes the backup file path, checksum file path and backup time.

10. A data backup device, characterized in that, include: The extension unit is used to extend the target manager cloud disk backup interface through block storage services and configure the local directory of the backup backend; Create a unit to create target manager snapshots and dynamically calculate snapshot reserved space; The mapping unit is used to partition and map the snapshot reserved space, and mount the mapped partition to the local backup directory. The compression unit is used to perform file system-level incremental backups of the data in the mounted directory, exclude unnecessary directories, compress and generate backup files, and calculate the checksum of the backup files to ensure data consistency. The creation unit is further configured to: The security factor is dynamically adjusted based on the file system type and data change frequency of the cloud disk; whereby the data change frequency refers to the frequency of data writing, modification, and deletion in the cloud disk. The used space information in the file system metadata is obtained in real time. Combined with the adjusted security factor and the total capacity of the cloud disk, the snapshot reserved space is calculated according to S_reserved = min(α×S_used, S_total); where α is the security factor, S_used refers to the actual amount of cloud disk used, and S_total is the total capacity of cloud disk; the used space information reflects the actual amount of cloud disk used. The compression unit is further used for: The non-essential directories are excluded based on pre-configured exclusion rules; these include recovery directories and custom exclusion directories. Calculate the checksum and write the checksum result to a preset file for subsequent backup integrity verification.

11. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data backup method as described in any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the data backup method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the data backup method as described in any one of claims 1 to 9.

Images