Virtual machine data backup method, device, program product, equipment and medium of cloud computing management platform

Abstract

The application discloses a virtual machine data backup method, device, program product, equipment and medium of a cloud computing management platform, relates to the technical field of computers, and comprises the following steps: generating a virtual machine snapshot; filtering out current change files from a disk image file of the virtual machine snapshot by scanning file system metadata of the disk image file, and determining current change data blocks that need to be backed up in an incremental backup mode by comparing target data blocks of the current change files with corresponding data blocks of reference backup data; the reference backup data is a basic backup data set used for incremental comparison; performing hierarchical compression processing on the current change data blocks; constructing a target storage directory in an object storage service according to virtual machine identifiers and backup task identifiers associated with the cloud computing management platform, and backing up the compressed data into the target storage directory. The application optimizes the backup data size and the backup time, and effectively improves the low-efficiency limitation of the backup function of the cloud computing management platform.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a virtual machine data backup method, apparatus, program product, device and medium for a cloud computing management platform. Background Technology

[0002] With the rapid development of cloud computing technology, virtual machines (VMs) have become an important component of data centers. However, backing up and restoring VMs remains a challenging issue. Traditional full backup methods consume significant storage space and time, resulting in inefficient backup and recovery. Currently, many enterprises and organizations are seeking more efficient and reliable VM backup solutions.

[0003] OpenStack (OpenStack Cloud Computing Platform), as an open-source cloud computing platform, is widely used for virtual machine management and deployment. However, its backup functionality still has limitations, such as excessively large backup data sizes and excessively long backup times; moreover, its backup function mainly relies on internal tools, which also have the aforementioned limitations. Therefore, these issues urgently need to be addressed by technical personnel. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a virtual machine data backup method, apparatus, program product, device, and medium for a cloud computing management platform, which optimizes the backup data size and backup time, and effectively improves the limitation of low efficiency in the backup function of the cloud computing management platform. The specific solution is as follows:

[0005] In a first aspect, this application discloses a method for backing up virtual machine data on a cloud computing management platform, including:

[0006] Call the target interface of the cloud computing management platform to generate a virtual machine snapshot;

[0007] By scanning the file system metadata of the disk image file of the virtual machine snapshot, the currently changed file is filtered out from the disk image file, and the target data block of the currently changed file is compared with the corresponding data block of the baseline backup data to determine the currently changed data block that needs to be backed up in an incremental backup manner; wherein, the baseline backup data is the basic backup data set used for incremental comparison.

[0008] Perform layered compression on the currently changed data block to obtain compressed data;

[0009] The target storage directory is constructed in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, and the compressed data is backed up to the target storage directory.

[0010] Optionally, the target interface of the cloud computing management platform is invoked to generate virtual machine snapshots, including:

[0011] Send a file system freeze command to the virtual machine to suspend write operations on the virtual machine's disk;

[0012] Determine whether a successful instruction execution result has been obtained. If a successful instruction execution result has been obtained, call the target interface of the cloud computing management platform to generate a virtual machine snapshot.

[0013] Optionally, the currently changed data block can be subjected to layered compression to obtain compressed data, including:

[0014] Based on the data type of the current changed data block, the current changed data block is subjected to the first level of compression processing to obtain the first processed data;

[0015] Based on the duplication of the current changed data block, the current changed data block is subjected to a second level of compression to obtain the second processed data;

[0016] The index file of the currently modified data block is compressed to obtain compressed metadata;

[0017] The first processed data, the second processed data, and the compressed metadata are identified as the compressed data.

[0018] Optionally, based on the data type of the currently changed data block, the current changed data block is subjected to a first level of compression processing to obtain the first processed data, including:

[0019] If the current changed data block is text data or configuration data, then obtain the compression dictionary that was pre-generated for the current changed data block, and compress the current changed data block by using the target standard compression algorithm and loading the compression dictionary to obtain the first processed data;

[0020] If the current changed data block is binary data, the target fast compression algorithm is used to compress the current changed data block to obtain the first processed data.

[0021] Optionally, based on the duplication of the current changed data block, a second level of compression is performed on the current changed data block to obtain second processed data, including:

[0022] Calculate the data fingerprint of the currently changed data block, determine the split point based on the data fingerprint, and then split the currently changed data block into multiple data blocks of dynamic length based on the split point;

[0023] For data blocks with content similarity less than a preset similarity threshold among multiple dynamically sized data blocks, a binary difference algorithm is used to generate corresponding difference patch data;

[0024] The difference patch data is integrated with data blocks whose content similarity is not less than a preset similarity threshold from multiple dynamic length data blocks to obtain the second processed data.

[0025] Optionally, calculate the data fingerprint of the currently changed data block and determine the split point based on the data fingerprint, including:

[0026] Use a preset sliding window to slide on the currently changed data block in byte sequence order;

[0027] Calculate the data fingerprint based on the portion of data currently covered by the preset sliding window;

[0028] If the numerical features of the data fingerprint meet the preset segmentation conditions, the current position of the preset sliding window will be determined as the segmentation point.

[0029] Optionally, the index file of the currently changed data block can be compressed to obtain compressed metadata, including:

[0030] The index file of the currently changed data block is serialized to generate serialized data;

[0031] The serialized data is compressed using the target standard compression algorithm to obtain compressed metadata.

[0032] Optionally, the target standard compression algorithm is the Z standard algorithm.

[0033] Optionally, a target storage directory is constructed in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, including:

[0034] In the object storage service, a first-level directory is created based on the virtual machine identifier associated with the cloud computing management platform, and a second-level directory is created based on the backup task identifier associated with the cloud computing management platform. Then, the first-level directory and the second-level directory are determined as the target storage directory.

[0035] Optionally, the compressed data can be backed up to the target storage directory, including:

[0036] The compressed data contains multiple data blocks that are backed up as independent storage objects to the target storage directory, and an index file corresponding to the backup task identifier is generated.

[0037] The index file records the mapping relationship between the storage path and the logical location. The storage path is the backup path of each independent storage object in the object storage service, and the logical location is the original location of each independent storage object in the disk image file.

[0038] Optionally, virtual machine data backup methods for cloud computing management platforms may also include:

[0039] Determine whether the backup data in the target storage directory meets the preset cold backup conditions; wherein, the preset cold backup conditions include the storage time of the backup data exceeding a preset duration threshold or the number of data accesses to the backup data within a preset statistical period being lower than a preset access threshold;

[0040] If the backup data in the target storage directory meets the preset cold backup conditions, erasure coding is used to store the corresponding backup data.

[0041] Optionally, virtual machine data backup methods for cloud computing management platforms may also include:

[0042] Monitor the number of consecutive incremental backups for the same virtual machine;

[0043] When the number of consecutive incremental backups reaches the preset merging threshold, the corresponding multiple incremental backups are merged into a new full backup, and the merged multiple incremental backups are deleted.

[0044] Optionally, virtual machine data backup methods for cloud computing management platforms may also include:

[0045] Parse the data recovery request to determine the target recovery time point and the corresponding target virtual machine identifier;

[0046] Based on the target recovery time point and the target virtual machine identifier, the full backup data and each incremental backup data that need to participate in the recovery are selected from the backup data in the target storage directory; the full backup data and each incremental backup data form a continuous backup sequence in the order of backup time;

[0047] The system restores the full backup data and updates the changed data blocks recorded in each incremental backup data to the virtual machine disk being restored in the order of backup time to achieve data recovery. During the recovery process, the system performs data decompression, data reconstruction, and disk write operations.

[0048] Optionally, the full backup data includes the initial full backup data generated during the first backup, as well as the new full backup data generated by merging multiple incremental backup data; the incremental backup data includes incremental backup data that has not reached the preset merging threshold, as well as incremental backup data used as merging input.

[0049] Optionally, before performing layered compression on the currently changed data block, the following steps are also included:

[0050] Determine the data hash value of the currently modified data block;

[0051] If no data record with the same data hash value exists in the preset database, proceed to the step of performing hierarchical compression on the currently changed data block and add the data hash value to the preset database; wherein, the preset database records the hash values ​​of the data blocks that have been stored.

[0052] Optionally, the currently changed data block is subjected to layered compression processing. After obtaining the compressed data, the following are also included:

[0053] Send a first request to the security module to obtain the data encryption key generated by the security module;

[0054] The compressed data is encrypted using a data encryption key to generate encrypted data.

[0055] A second request is sent to the security module so that the security module can encrypt the data encryption key based on the locally stored master key to obtain the encrypted data encryption key;

[0056] The encrypted data is associated with the encryption key and stored in the target storage directory.

[0057] Secondly, this application discloses a virtual machine data backup device for a cloud computing management platform, comprising:

[0058] The snapshot generation module is used to call the target interface of the cloud computing management platform to generate virtual machine snapshots;

[0059] The changed data determination module is used to filter out the currently changed files from the disk image file by scanning the file system metadata of the disk image file of the virtual machine snapshot, and to determine the currently changed data blocks that need to be backed up in an incremental backup manner by comparing the target data blocks of the current changed files with the corresponding data blocks of the baseline backup data; wherein, the baseline backup data is the basic backup data set used for incremental comparison.

[0060] The data compression module is used to perform layered compression on the currently changed data block to obtain compressed data;

[0061] The data backup module is used to build a target storage directory in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, and back up the compressed data to the target storage directory.

[0062] Thirdly, this application discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aforementioned virtual machine data backup method of the cloud computing management platform.

[0063] Fourthly, this application discloses an electronic device, comprising:

[0064] Memory, used to store computer programs;

[0065] A processor for executing computer programs to implement the virtual machine data backup method of the aforementioned disclosed cloud computing management platform.

[0066] Fifthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned virtual machine data backup method of the cloud computing management platform.

[0067] As can be seen, this application proposes a virtual machine data backup method for a cloud computing management platform, including: calling the target interface of the cloud computing management platform to generate a virtual machine snapshot; filtering out the currently changed file from the disk image file by scanning the file system metadata of the disk image file of the virtual machine snapshot, and determining the currently changed data block to be backed up in an incremental backup manner by comparing the target data block of the currently changed file with the corresponding data block of the baseline backup data; wherein, the baseline backup data is a basic backup data set used for incremental comparison; performing layered compression processing on the currently changed data block to obtain compressed data; constructing a target storage directory in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, and backing up the compressed data to the target storage directory.

[0068] Beneficial effects: On the one hand, this application adopts an incremental backup design, which filters changed files by scanning file system metadata and then determines the currently changed data blocks through block-level comparison. This allows backup only of the parts that differ from the baseline backup data, avoiding the repetitive processing of complete data required by traditional full backups, significantly reducing the amount of data to be backed up, thereby reducing the storage space occupied during backup and shortening backup time. On the other hand, this application further performs layered compression processing on the selected currently changed data blocks, which can further reduce the data volume on the basis of incremental backup, optimize the backup data size and backup time, and effectively improve the limitations of low efficiency in the backup function of cloud computing management platforms. Simultaneously, this application constructs a target storage directory based on virtual machine identifiers and backup task identifiers, realizing the orderly classification of backup data, facilitating subsequent management and retrieval, and meeting the needs of enterprises for efficient and reliable virtual machine backup solutions. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0070] Figure 1This application discloses a flowchart of a virtual machine data backup method for a cloud computing management platform.

[0071] Figure 2 This is a schematic diagram of a virtual machine data backup device for a cloud computing management platform disclosed in this application;

[0072] Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] As an open-source cloud computing platform, cloud computing platforms are widely used for virtual machine management and deployment. However, their backup functions still have limitations, such as excessively large backup data sizes and excessively long backup times; moreover, their backup functions mainly rely on internal tools, which also have the aforementioned limitations.

[0075] To address this, this application proposes a virtual machine data backup scheme for a cloud computing management platform, which optimizes the backup data size and backup time, effectively improving the limitations of low backup efficiency in cloud computing management platforms.

[0076] This application discloses a virtual machine data backup method for a cloud computing management platform. See [link to relevant documentation]. Figure 1 As shown, the method includes:

[0077] Step S11: Call the target interface of the cloud computing management platform to generate a virtual machine snapshot.

[0078] In this embodiment, a file system freeze command is sent to the virtual machine to suspend the writing operation of virtual machine disk data; it is determined whether a return result of successful command execution is obtained. If a return result of successful command execution is obtained, the target interface of the cloud computing management platform is called to generate a virtual machine snapshot.

[0079] Specifically, the Nova API (Nova Application Programming Interface) of the cloud computing management platform is called as the target interface to trigger virtual machine snapshot generation. Simultaneously, the Cinder (Cinder Volume Service) ensures data consistency during the snapshot process. Before generating the snapshot, a file system freeze command is sent to the virtual machine via the QEMU Guest Agent (a lightweight service program running inside the virtual machine) to suspend write operations on the virtual machine disk data, preventing disk data from being modified during the snapshot process, thus achieving a quiesced snapshot. Then, it checks if the command execution was successful. If so, subsequent snapshot generation calls are executed. A full snapshot (BaseSnapshot) is generated during the first backup as the basis for the baseline backup data. Subsequent backups generate incremental snapshots, using a QEMU Block Dirty Bitmap to record data block changes since the last snapshot, and simultaneously recording the snapshot timestamp and the virtual machine UUID (Universally Unique Device ID). Metadata information such as Identifier (universally unique identifier), parent snapshot ID, and data block verification is saved to a preset database (such as a MySQL database) to complete the generation and recording of virtual machine snapshots.

[0080] Step S12: By scanning the file system metadata of the disk image file of the virtual machine snapshot, the currently changed file is selected from the disk image file, and the target data block of the currently changed file is compared with the corresponding data block of the baseline backup data to determine the currently changed data block that needs to be backed up in an incremental backup manner; wherein, the baseline backup data is the basic backup data set used for incremental comparison.

[0081] First, using the Libguestfs (Library for Guest File Systems) tool, the disk image file corresponding to the virtual machine snapshot (mounting a full snapshot for the first backup, and incremental snapshots for subsequent backups) is mounted. By scanning the file system metadata (such as inode modification time, file size, etc.) of the disk image file, the currently changed files that have changed compared to the baseline backup data (i.e., the full snapshot data from the first backup or the data corresponding to the incremental snapshot generated from the previous backup) are quickly filtered out from the disk image file. In this way, the scope of incremental data can be initially narrowed down. Next, the Rsync rolling verification algorithm is used to divide the currently changed files into multiple target data blocks of 4KB size, and these target data blocks are compared one by one with the corresponding 4KB data blocks in the baseline backup data, while combining the above steps. The process involves cross-validating data block changes recorded in a persistent bitmap since the last snapshot to exclude data blocks that have not actually changed. Finally, incremental backups are used to determine the currently changed data blocks that need to be backed up. The baseline backup data is the basic set of backup data used for incremental comparisons; its specific value depends on the number of backups. During the first backup, the baseline backup data is empty (at this point, the full snapshot data is used directly as the basis for subsequent incremental comparisons). For subsequent backups, the baseline backup data is the set of backup data generated from the previous backup. Throughout the comparison process, if differences are found in data blocks, the offset, length, and other information of those data blocks are recorded synchronously, providing a basis for subsequent data compression and storage. This ensures that the selected currently changed data blocks accurately reflect the actual changes in virtual machine data, avoiding redundant data backups.

[0082] Step S13: Perform layered compression on the currently changed data block to obtain compressed data;

[0083] In this embodiment, to avoid reprocessing already stored data blocks and improve compression efficiency, a uniqueness check of the data blocks must be performed before performing layered compression: First, calculate the data hash value of the currently changed data block (using the SHA-256 algorithm, Secure Hash Algorithm 256); then, query the preset database (this database can be associated with the MySQL database that stores snapshot metadata in the aforementioned steps, or a separate global block data management database) to determine whether there is a data record with the same data hash value in the preset database. If it exists, it means that the data block has been stored and does not need to be compressed repeatedly; if it does not exist, jump to the layered compression processing step and add the data hash value to the preset database, thereby achieving block-level deduplication across virtual machines.

[0084] Furthermore, the current changed data block undergoes layered compression processing to obtain compressed data. Specifically, this includes: performing a first-level compression based on the data type of the current changed data block to obtain first-processed data; performing a second-level compression based on the duplication of the current changed data block to obtain second-processed data; compressing the index file of the current changed data block to obtain compressed metadata; and then determining the first-processed data, second-processed data, and compressed metadata as the compressed data. The following is a detailed analysis: Firstly, if the current changed data block is text or configuration data, a pre-generated compression dictionary is obtained. By employing a target standard compression algorithm and loading the compression dictionary, the current changed data block is compressed to obtain first-processed data. If the current changed data block is binary data, a target fast compression algorithm is used to compress the current changed data block to obtain first-processed data. Secondly, the data fingerprint of the currently changed data block is calculated. Based on the data fingerprint, a segmentation point is determined, and the current changed data block is segmented into multiple dynamically sized data blocks. For data blocks within these dynamically sized data blocks where the content similarity is less than a preset similarity threshold, a binary difference algorithm is used to generate corresponding difference patch data. The difference patch data is then integrated with data blocks within these dynamically sized data blocks where the content similarity is not less than the preset similarity threshold to obtain the second processed data. The segmentation point determination process involves: sliding a preset sliding window across the current changed data block in byte sequence order; calculating the data fingerprint based on the portion of data currently covered by the preset sliding window; and determining the segmentation point based on the numerical characteristics of the data fingerprint if it meets the preset segmentation conditions. Thirdly, the index file of the current changed data block is serialized to generate serialized data. The serialized data is then compressed using a target standard compression algorithm to obtain compressed metadata. The target standard compression algorithm is the Z-standard algorithm.

[0085] Specifically, the first level is general compression based on data type. The core is to match and adapt the compression according to the different types of the currently changed data block: If the currently changed data block is text data (such as log files, configuration scripts) or configuration data (such as system parameter configuration files), then a compression dictionary pre-generated for this type of data is first obtained (generated by training on historical data of the same type, which can improve compression efficiency and compression ratio). Then, the target standard compression algorithm (i.e., the Z standard algorithm, specifically Zstandard, with a compression level set to 19) is used and the compression dictionary is loaded to compress the data block, finally obtaining the first processed data. If the currently changed data block is binary data (such as application executable files), then the target fast compression algorithm (i.e., the LZ4 algorithm, specifically Lempel-Ziv 4, which can achieve a certain compression ratio while ensuring high compression speed, adapting to the processing speed requirements of binary data) is used to directly compress the currently changed data block, obtaining the first processed data.

[0086] Specifically, the second level is deduplication (re-deletion) based on data duplication, aiming to further eliminate redundant information within and between data blocks: First, a preset sliding window (the window size can be dynamically adjusted according to the average length of the data block) slides sequentially over the currently changed data block in byte order. For each portion of data covered by the sliding window, the Rabin fingerprint algorithm is used to calculate the data fingerprint. If the numerical characteristics of the calculated data fingerprint (e.g., the last N bits of the fingerprint value are preset values) meet the preset segmentation conditions, the current sliding window position is determined as the segmentation point. Based on all segmentation points, the currently changed data block is segmented into multiple dynamically sized data blocks (the average length is set to 8KB to adapt to the redundancy distribution characteristics of most virtual machine data). Next, the similarity between these dynamically sized data blocks and the already stored data blocks is compared. For data blocks with a similarity less than a preset similarity threshold (e.g., 70%), a binary difference algorithm (e.g., Binary) is used. The Diff algorithm generates difference patch data between the data and the most similar stored data block, retaining only the differences to reduce the amount of data. For data blocks with a similarity of not less than a preset similarity threshold, the original data block is directly retained. Finally, the difference patch data is integrated with the retained original data block to obtain the second processed data.

[0087] Specifically, the third level involves compressing the index file to reduce the size of the metadata storage: This involves obtaining information such as the offset, length, and hash value of the currently changed data block recorded in the preceding steps, along with compression parameters generated during the layered compression process (e.g., compression algorithm type and compression level). An index file for the currently changed data block is then constructed (using JSON format and JavaScript Object Notation). This index file is then serialized (using the Protobuf protocol, which results in small data size and high parsing efficiency, meeting the requirements for efficient metadata transmission and storage), generating serialized data. Finally, the serialized data is compressed using a target standard compression algorithm to obtain the compressed metadata. After completing these three levels of processing, the first processed data, the second processed data, and the compressed metadata are collectively defined as the compressed data.

[0088] In addition, to ensure the security of backup data, the compressed data needs to be encrypted before subsequent storage: A first request is sent to the security module (such as a hardware security module, which has high security and can prevent key leakage) to request the data encryption key (DEK). After receiving the request, the security module generates a DEK using the AES-256-GCM (Advanced Encryption Standard 256-bit Galois / counter mode) algorithm and returns it to the backup server. The backup server uses this DEK to encrypt the compressed data, generating encrypted data while ensuring that data integrity verification information is retained during the encryption process. Next, a second request is sent to the security module, sending the generated DEK to the security module. The security module encrypts the DEK based on the locally stored master key (KEK) to obtain the encrypted data encryption key. Finally, the encrypted data is associated with the encrypted data encryption key (e.g., by adding the same identification field to ensure accurate matching of the corresponding key during subsequent recovery), thus preparing for subsequent storage to the target storage directory.

[0089] Step S14: Build the target storage directory in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, and back up the compressed data to the target storage directory.

[0090] In this embodiment, based on the identification system and object storage service characteristics of the cloud computing management platform, the data directory is constructed and backup storage is completed to ensure the structured management and traceability of the backup data.

[0091] First, an object storage service (Ceph RadosGW, a distributed object storage solution adapted to the OpenStack ecosystem) is used. The virtual machine UUID is used as the virtual machine identifier associated with the cloud computing management platform. A first-level directory is created in Ceph RadosGW, and the directory name directly uses the virtual machine UUID (e.g., vm-8f4d7e9c-3b2a-11e9-b210-d663bd873d93) to ensure that the backup data of each virtual machine is stored independently. Then, the unique identifier of the current backup task (e.g., the backup_id automatically assigned by the system when the backup task is generated, the format can be referred to as backup-20250810-153000) is used as the backup task identifier associated with the cloud computing management platform. A second-level directory is created under the first-level directory. Finally, the hierarchical structure of the virtual machine UUID directory / backup task identifier directory is determined as the target storage directory. Next, the compressed data (including the encrypted first-processed data, second-processed data, compressed metadata, and the associated encrypted data encryption key) is split into multiple independent storage objects according to data block type. For example, the encrypted first-processed data block, second-processed data block, and encrypted data encryption key are each treated as independent objects. Each object is named in the format of data type-block sequence number-hash value fragment (e.g., first-processed-001-sha256-abc123). These independent storage objects are then uploaded to the target storage directory one by one. At the same time, an index file (in JSON format) corresponding to the current backup task identifier (backup_id) is generated. This index file must completely record two core mapping relationships: one is the storage path, i.e., the storage path of each independent storage object in Ceph. The backup path in RadosGW is required for two main components: First, the logical location, which is the position information of the original data block corresponding to each independent storage object within the virtual machine disk image file (such as block offset and length, in the format offset:102400, length:4096). Additionally, the encryption status identifier of the data block (such as encryption:AES-256-GCM) and hash value (SHA-256, a 256-bit secure hash algorithm) need to be added to ensure accurate data location and verification of data integrity during subsequent recovery. Finally, the index file is also uploaded as an independent object to the root directory of the target storage directory (named backup_id-index.json), and the storage path of the index file is synchronized and updated in the MySQL database.

[0092] In addition, it determines whether the backup data in the target storage directory meets the preset cold backup conditions. These preset cold backup conditions include the backup data's storage time exceeding a preset duration threshold or the backup data's access frequency within a preset statistical period being lower than a preset access threshold. If the backup data in the target storage directory meets the preset cold backup conditions, erasure coding is used to store the corresponding backup data. That is, for cold backup data in the target storage directory (such as data backed up for more than 30 days or data not frequently accessed), erasure coding technology is enabled, employing an 8+3 EC coding strategy (i.e., 8 data blocks corresponding to 3 check blocks), reducing storage costs by 50% while ensuring data reliability.

[0093] As can be seen, this application proposes a virtual machine data backup method for a cloud computing management platform, including: calling the target interface of the cloud computing management platform to generate a virtual machine snapshot; filtering out the currently changed file from the disk image file by scanning the file system metadata of the disk image file of the virtual machine snapshot, and determining the currently changed data block to be backed up in an incremental backup manner by comparing the target data block of the currently changed file with the corresponding data block of the baseline backup data; wherein, the baseline backup data is a basic backup data set used for incremental comparison; performing layered compression processing on the currently changed data block to obtain compressed data; constructing a target storage directory in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, and backing up the compressed data to the target storage directory.

[0094] Beneficial effects: On the one hand, this application adopts an incremental backup design, which filters changed files by scanning file system metadata and then determines the currently changed data blocks through block-level comparison. This allows backup only of the parts that differ from the baseline backup data, avoiding the repetitive processing of complete data required by traditional full backups, significantly reducing the amount of data to be backed up, thereby reducing the storage space occupied during backup and shortening backup time. On the other hand, this application further performs layered compression processing on the selected currently changed data blocks, which can further reduce the data volume on the basis of incremental backup, optimize the backup data size and backup time, and effectively improve the limitations of low efficiency in the backup function of cloud computing management platforms. Simultaneously, this application constructs a target storage directory based on virtual machine identifiers and backup task identifiers, realizing the orderly classification of backup data, facilitating subsequent management and retrieval, and meeting the needs of enterprises for efficient and reliable virtual machine backup solutions.

[0095] To avoid the reduction in efficiency caused by excessively long snapshot chains due to too many consecutive incremental backups, which necessitates the stacking of multiple incremental data blocks during recovery, this embodiment introduces an incremental backup data merging mechanism. The specific process is as follows:

[0096] Monitor the number of consecutive incremental backups for the same virtual machine; when the number of consecutive incremental backups reaches a preset merging threshold, merge the corresponding multiple incremental backups into a new full backup, and delete the merged multiple incremental backups. It should be noted that a snapshot chain refers to a continuous backup sequence formed by full backup data and each incremental backup in chronological order. This will be elaborated below:

[0097] The system monitors the number of consecutive incremental backups for the same virtual machine in real time. This count starts from the first full backup or the new full backup generated from the previous merge, and automatically increments after each incremental backup. Simultaneously, a preset merge threshold is set based on actual business needs. When the number of consecutive incremental backups reaches the preset merge threshold, the merge process is automatically triggered: All incremental backup data for the virtual machine since the last full backup, along with the corresponding baseline backup data (i.e., the last full backup data), are retrieved from the target storage directory. Multiple incremental backups are then sequentially added to the baseline backup data in chronological order, automatically verifying the hash value and logical position of each incremental data block to ensure the accuracy of the data merge. After the merge is complete, a new full backup is generated, containing complete virtual machine data up to the current merge node, and using the same metadata format as the first full backup (recording timestamp, virtual machine UUID, parent snapshot ID), and is synchronously updated to the MySQL database. Finally, after the new full backup data passes verification (by comparing the hash value of the merged data with the theoretical calculation value) and is successfully stored in the target storage directory, the merged multiple incremental backup data are automatically deleted. At the same time, the snapshot chain record is updated, and the baseline backup data for the next incremental backup is pointed to the newly generated full backup data. This shortens the snapshot chain length during subsequent recovery and avoids data accumulation that occupies too much storage space.

[0098] To achieve accurate and efficient data recovery, this embodiment refines the recovery process to ensure that backup data at any point in time can be located and recovered according to business needs. The specific steps are as follows:

[0099] The data recovery request is parsed to determine the target recovery time point and the corresponding target virtual machine identifier. Based on the target recovery time point and the target virtual machine identifier, the full backup data and each incremental backup data to be recovered are selected from the backup data in the target storage directory. The full backup data and each incremental backup data form a continuous backup sequence in backup time order. The full backup data is recovered, and according to the backup time order, the changed data blocks recorded in each incremental backup data are updated to the virtual machine disk being recovered to achieve data recovery. During the recovery process, data decompression, data reconstruction, and disk write operations are performed. The full backup data includes the initial full backup data generated by the first backup, and the new full backup data generated by merging multiple incremental backup data. The incremental backup data includes incremental backup data that has not reached the preset merging threshold, and incremental backup data used as merging input.

[0100] The analysis will proceed as follows:

[0101] First, based on the target virtual machine identifier, query the metadata records of all backup data for that virtual machine, and filter out candidate backup datasets with timestamps ≤ the target recovery time. If the target recovery time is between the first full backup and the first merge operation, the full backup is the initial full backup; if the target recovery time is after a merge, it is the new full backup data generated by that merge. Next, filter out all incremental backup data with timestamps between the full backup and the target recovery time. Sort the filtered full backup data and incremental backup data in backup time order to form a continuous backup sequence: full backup → incremental backup 1 → incremental backup 2 → ... → incremental backup at the target time.

[0102] Secondly, perform full data recovery and verification: After compressing the metadata of the full backup data, perform zstd decompression and Protobuf deserialization to obtain the logical location of data blocks and compression parameters; perform a layered compression process in reverse order of data type: text / configuration data is loaded with the corresponding compression dictionary and then decompressed using zstd; binary data is decompressed using LZ4; the difference patch data generated by similarity deduplication is restored using the Bsdiff algorithm, restoring the full data to a temporary disk partition; recalculate the hash value of the restored full data using the SHA-256 algorithm and compare it with the original checksum stored in the metadata: if they are the same, it means the full data has not been damaged or tampered with, and it can safely proceed to the subsequent incremental data recovery stage; if they are different, it indicates that there is an anomaly in the full data, so the recovery process is paused to investigate the cause of data corruption.

[0103] Then, incremental data recovery and verification are performed: each incremental backup data is processed in chronological order of the backup sequence, the disk location is located by the block offset and length recorded in the incremental backup index file, and decompression and reconstruction are performed; a parallel pipeline mechanism is adopted (decompression thread, deduplication and reconstruction thread, and disk write thread work synchronously) to update the incremental changed data blocks to the disk; after each incremental backup update is completed, the current disk data hash value is calculated and compared with the backup metadata checksum stored in the metadata.

[0104] Finally, after all incremental data updates are complete, the SHA-256 hash value of the restored virtual machine disk is calculated and compared with the metadata hash value of the backup data corresponding to the target recovery time point. Then, the virtual machine boot process is simulated, and the restored disk partition is mounted. This checks the integrity of the file system (e.g., ext4, xfs) and reads core application data (e.g., system configuration files, database files) to verify data availability. If both verifications pass, the temporary disk partition is switched to the virtual machine's official disk, a successful recovery result is returned to the user, and the recovery record (including recovery time, target time point, and recovery status) is updated in the MySQL database. If either verification fails, a rollback mechanism is immediately triggered, deleting the restored data from the temporary disk, restoring the virtual machine to its pre-recovery state, and generating an error log (recording failed steps, abnormal data block identifiers, and hash value differences).

[0105] Furthermore, in this embodiment of the application, before performing incremental detection, the core load indicators of the virtual machine, such as CPU (Central Processing Unit) utilization, memory usage, and disk I / O throughput, can be collected in real time by calling the target interface of the corresponding cloud computing management platform. Three load thresholds and corresponding incremental detection cycles are preset: the cycle is set to 1 hour for low load (CPU < 30%, disk I / O < 50MB / s) to ensure timely backup of changed data; the cycle is set to 2 hours for medium load (CPU 30%-70%, disk I / O 50-100MB / s); and the cycle is set to 4 hours for high load (CPU > 70%, disk I / O > 100MB / s) to reduce resource contention for virtual machine services during backup.

[0106] Accordingly, this application also discloses a virtual machine data backup device for a cloud computing management platform, see [link to relevant documentation]. Figure 2 As shown, the device includes:

[0107] Snapshot generation module 11 is used to call the target interface of the cloud computing management platform to generate virtual machine snapshots;

[0108] The changed data determination module 12 is used to filter out the currently changed file from the disk image file by scanning the file system metadata of the disk image file of the virtual machine snapshot, and to determine the currently changed data block that needs to be backed up in an incremental backup manner by comparing the target data block of the current changed file with the corresponding data block of the baseline backup data; wherein, the baseline backup data is the basic backup data set used for incremental comparison.

[0109] Data compression module 13 is used to perform layered compression processing on the currently changed data block to obtain compressed data;

[0110] The data backup module 14 is used to build a target storage directory in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, and back up the compressed data to the target storage directory.

[0111] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0112] Furthermore, embodiments of this application also provide an electronic device. Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0113] Figure 3 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the virtual machine data backup method of the cloud computing management platform disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0114] In this embodiment, the power supply 26 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 24 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0115] Furthermore, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon may include computer programs 221, and the storage method may be temporary storage or permanent storage. The computer programs 221 may include, in addition to computer programs capable of performing the virtual machine data backup method of the cloud computing management platform executed by the electronic device 20 as disclosed in any of the foregoing embodiments, computer programs capable of performing other specific tasks.

[0116] Furthermore, embodiments of this application also disclose a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned virtual machine data backup method of the cloud computing management platform.

[0117] For the specific steps of this method, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0118] The various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For the same or similar parts between the various embodiments, refer to each other. As for the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and relevant parts can be referred to in the method section.

[0119] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0121] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0122] The foregoing has provided a detailed description of the virtual machine data backup method, apparatus, program product, device, and medium of a cloud computing management platform provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for backing up virtual machine data on a cloud computing management platform, characterized in that, include: Call the target interface of the cloud computing management platform to generate a virtual machine snapshot; By scanning the file system metadata of the disk image file of the virtual machine snapshot, the currently changed file is filtered out from the disk image file, and the target data block of the currently changed file is compared with the corresponding data block of the baseline backup data to determine the currently changed data block that needs to be backed up in an incremental backup manner; wherein, the baseline backup data is the basic backup data set used for incremental comparison. The currently changed data block is subjected to layered compression processing to obtain compressed data; The target storage directory is constructed in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, and the compressed data is backed up to the target storage directory; The currently changed data block is subjected to layered compression processing to obtain compressed data, including: Based on the data type of the currently changed data block, the currently changed data block is subjected to a first level of compression processing to obtain the first processed data; Based on the duplication of the current changed data block, the current changed data block is subjected to a second level of compression processing to obtain the second processed data; The index file of the currently changed data block is compressed to obtain compressed metadata; The first processed data, the second processed data, and the compressed metadata are identified as the compressed data.

2. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, The process of calling the target interface of the cloud computing management platform to generate a virtual machine snapshot includes: Send a file system freeze command to the virtual machine to suspend write operations on the virtual machine's disk; Determine whether a successful instruction execution result has been obtained. If a successful instruction execution result has been obtained, then call the target interface of the cloud computing management platform to generate a virtual machine snapshot.

3. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, The first level of compression processing, based on the data type of the currently changed data block, is performed on the current changed data block to obtain first processed data, including: If the current changed data block is text data or configuration data, then a compressed dictionary pre-generated for the current changed data block is obtained. By using a target standard compression algorithm and loading the compressed dictionary, the current changed data block is compressed to obtain the first processed data. If the current changed data block is binary data, then the target fast compression algorithm is used to compress the current changed data block to obtain the first processed data.

4. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, Based on the duplication of the current changed data block, the second level of compression processing is performed on the current changed data block to obtain second processed data, including: Calculate the data fingerprint of the currently changed data block, determine the splitting point based on the data fingerprint, and then split the currently changed data block into multiple data blocks of dynamic length based on the splitting point; For data blocks with a content similarity less than a preset similarity threshold among the multiple dynamically lengthed data blocks, a binary difference algorithm is used to generate corresponding difference patch data; The difference patch data is integrated with the data blocks in the plurality of dynamically lengthed data blocks whose content similarity is not less than a preset similarity threshold to obtain the second processed data.

5. The virtual machine data backup method of the cloud computing management platform according to claim 4, characterized in that, The step of calculating the data fingerprint of the currently changed data block and determining the split point based on the data fingerprint includes: The preset sliding window is used to slide on the currently changed data block in byte sequence order; Calculate the data fingerprint based on the portion of data currently covered by the preset sliding window; If the numerical features of the data fingerprint satisfy the preset segmentation conditions, then the current position of the preset sliding window is determined as the segmentation point.

6. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, The compression process of the index file of the currently changed data block to obtain compressed metadata includes: The index file of the currently changed data block is serialized to generate serialized data; The serialized data is compressed using a target standard compression algorithm to obtain the compressed metadata.

7. The virtual machine data backup method for a cloud computing management platform according to claim 3 or 6, characterized in that, The target standard compression algorithm is the Z standard algorithm.

8. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, The step of constructing the target storage directory in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform includes: In the object storage service, a first-level directory is created based on the virtual machine identifier associated with the cloud computing management platform, and a second-level directory is created based on the backup task identifier associated with the cloud computing management platform. Then, the first-level directory and the second-level directory are determined as the target storage directory.

9. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, The step of backing up the compressed data to the target storage directory includes: The compressed data contains multiple data blocks that are stored as independent storage objects and backed up to the target storage directory, and an index file corresponding to the backup task identifier is generated. The index file records the mapping relationship between storage paths and logical locations. The storage path is the backup path of each independent storage object in the object storage service, and the logical location is the original location of each independent storage object in the disk image file.

10. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, Also includes: Determine whether the backup data in the target storage directory meets the preset cold backup conditions; wherein, the preset cold backup conditions include the storage time of the backup data exceeding a preset duration threshold or the number of data accesses to the backup data within a preset statistical period being lower than a preset access threshold; If the backup data in the target storage directory meets the preset cold backup conditions, then erasure coding is used to store the corresponding backup data.

11. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, Also includes: Monitor the number of consecutive incremental backups for the same virtual machine; When the number of consecutive incremental backups reaches a preset merging threshold, the corresponding multiple incremental backups are merged into a new full backup, and the merged multiple incremental backups are deleted.

12. The virtual machine data backup method of the cloud computing management platform according to claim 11, characterized in that, Also includes: Parse the data recovery request to determine the target recovery time point and the corresponding target virtual machine identifier; Based on the target recovery time point and the target virtual machine identifier, the full backup data and each incremental backup data that need to participate in the recovery are selected from the backup data of the target storage directory; the full backup data and each incremental backup data form a continuous backup sequence in the order of backup time; The full backup data is restored, and the data blocks that have been changed and recorded in each incremental backup data are updated to the virtual machine disk being restored in the order of the backup time to achieve data recovery. During the recovery process, data decompression, data reconstruction and disk writing operations are performed.

13. The virtual machine data backup method of the cloud computing management platform according to claim 12, characterized in that, The full backup data includes the initial full backup data generated during the first backup, and the new full backup data generated by merging multiple incremental backup data; the incremental backup data includes incremental backup data that has not reached the preset merging threshold, and incremental backup data used as merging input.

14. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, Before performing layered compression on the currently changed data block, the process also includes: Determine the data hash value of the currently changed data block; If no data record with the same data hash value exists in the preset database, the process proceeds to the step of performing hierarchical compression on the currently changed data block and adding the data hash value to the preset database; wherein, the preset database records the hash values ​​of the data blocks that have already been stored.

15. The virtual machine data backup method of the cloud computing management platform according to claim 1, characterized in that, After performing layered compression on the currently changed data block to obtain compressed data, the following steps are also included: Send a first request to the security module to obtain the data encryption key generated by the security module; The compressed data is encrypted using the data encryption key to generate encrypted data; A second request is sent to the security module so that the security module encrypts the data encryption key based on the locally stored master key to obtain the encrypted data encryption key; The encrypted data is associated with the encryption key of the encrypted data and stored in the target storage directory.

16. A virtual machine data backup device for a cloud computing management platform, characterized in that, include: The snapshot generation module is used to call the target interface of the cloud computing management platform to generate virtual machine snapshots; The changed data determination module is used to filter out the currently changed file from the disk image file by scanning the file system metadata of the disk image file of the virtual machine snapshot, and to determine the currently changed data block that needs to be backed up by comparing the target data block of the currently changed file with the corresponding data block of the baseline backup data in an incremental backup manner; wherein, the baseline backup data is a set of basic backup data used for incremental comparison. The data compression module is used to perform layered compression processing on the currently changed data block to obtain compressed data; The data backup module is used to construct a target storage directory in the object storage service according to the virtual machine identifier and backup task identifier associated with the cloud computing management platform, and back up the compressed data to the target storage directory; The currently changed data block is subjected to layered compression processing to obtain compressed data, including: Based on the data type of the currently changed data block, the currently changed data block is subjected to a first level of compression processing to obtain the first processed data; Based on the duplication of the current changed data block, the current changed data block is subjected to a second level of compression processing to obtain the second processed data; The index file of the currently changed data block is compressed to obtain compressed metadata; The first processed data, the second processed data, and the compressed metadata are identified as the compressed data.

17. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the virtual machine data backup method of the cloud computing management platform according to any one of claims 1 to 15.

18. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the virtual machine data backup method of the cloud computing management platform as described in any one of claims 1 to 15.

19. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the virtual machine data backup method of the cloud computing management platform as described in any one of claims 1 to 15.

Images