Virtual machine batch backup method, device, equipment, medium and product

Abstract

The application relates to a virtual machine batch backup method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: determining to-be-backed-up virtual machines associated with a batch backup task and a retry set corresponding to the batch backup task in response to an execution request of the batch backup task; the retry set is empty at the first time of initiation and contains virtual machine identifiers of historical backup failures at the time of retry initiation; and then, in the execution stage, the range of a backup operation is determined according to whether the retry set is empty, that is, the backup operation is performed on all to-be-backed-up virtual machines when the retry set is empty, and the backup operation is performed only on to-be-backed-up virtual machines whose identifiers are located in the retry set when the retry set is not empty. The method can effectively shorten the overall time consumption of batch backup task retries and save storage resources.

Description

Technical Field

[0001] This application relates to the field of data backup technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for batch backup of virtual machines. Background Technology

[0002] In current virtualization operation and maintenance scenarios, there is a common need to back up multiple virtual machines in batches.

[0003] Traditional batch backup methods typically execute backup operations sequentially or in parallel on all virtual machines defined in a backup task, according to a task list. When a batch backup task fails due to the failure of an individual virtual machine, the entire task is marked as "failed." After troubleshooting, the operations team needs to re-initiate a new batch backup task, which will again iterate through and back up all defined virtual machines in the task. This means that virtual machines that were successfully backed up in the previous round will be backed up again, wasting backend storage resources and increasing the overall time taken for batch backup tasks. Summary of the Invention

[0004] Therefore, it is necessary to provide a virtual machine batch backup method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can save storage resources and shorten the overall batch backup time, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for batch backup of virtual machines, the method comprising:

[0006] In response to the execution request of a batch backup task, the virtual machine to be backed up associated with the batch backup task and the retry set corresponding to the batch backup task are determined; wherein, the retry set is used to store the virtual machine identifiers of virtual machines that failed to be backed up in the historical backup records of the batch backup task, and the retry set is empty when the execution request of the batch backup task is initiated for the first time;

[0007] If the retry set is empty, perform backup operations on all virtual machines to be backed up;

[0008] If the retry set is not empty, the backup operation is performed on the virtual machine identified as being located in the retry set among all virtual machines to be backed up associated with the batch backup task.

[0009] In one embodiment, the retry set is generated by a preset backup management control module, and the steps for generating the retry set are as follows:

[0010] If it is determined that the execution request for the batch backup task is being initiated for the first time, a retry set with empty elements is generated.

[0011] If it is determined that the execution request of the batch backup task is not the first one, the historical backup records of the batch backup task are obtained, and the virtual machine identifiers of the virtual machines that failed to be backed up are extracted from the historical backup records. All extracted virtual machine identifiers are then aggregated to obtain a retry set.

[0012] In one embodiment, the virtual machines to be backed up associated with the batch backup task are deployed on multiple virtualization platforms; obtaining the historical backup records of the batch backup task includes:

[0013] Extract sub-historical records corresponding to the batch backup task from multiple virtualization platforms;

[0014] Standardize each sub-historical record to obtain a standardized sub-historical record;

[0015] All standardized sub-historical records are aggregated to obtain the standardized historical backup records for the batch backup task.

[0016] In one embodiment, after extracting the sub-historical records corresponding to the batch backup task, the method further includes:

[0017] All sub-historical records are aggregated to obtain the initial historical backup record of the batch backup task, and the initial historical backup record and the standardized historical backup record are associated and stored.

[0018] In one embodiment, after performing a backup operation on the virtual machines whose virtual machine identifiers are located within the retry set among all virtual machines to be backed up associated with the batch backup task, the method further includes:

[0019] Obtain the total number of virtual machines to be backed up associated with the batch backup task, and determine the virtual machines whose virtual machine identifiers are outside the retry set among all the virtual machines to be backed up associated with the batch backup task as virtual machines to be skipped during backup;

[0020] Based on the total number of virtual machines to be backed up and the total number of virtual machines skipped from backup, the total number of virtual machines actually performing backup operations is obtained; wherein, the total number of virtual machines actually performing backup operations is the difference between the total number of virtual machines to be backed up and the total number of skipped virtual machines.

[0021] The total number of successful backup operations for virtual machines and the total number of failed backup operations for virtual machines are obtained from the total number of executions.

[0022] The retry execution status of the batch backup task is obtained based on the total number of successful backups and the total number of executions.

[0023] In one embodiment, the retry execution status of the batch backup task includes a partially completed status, a fully completed status, and a completely failed status; wherein,

[0024] If the total number of successful attempts is greater than 0 and the total number of successful attempts is less than the total number of executions, then the retry execution status of the batch backup task is a partially completed status.

[0025] If the total number of successful attempts equals the total number of executions, then the retry execution status of the batch backup task is "complete".

[0026] If the total number of successful attempts is 0, then the retry execution status of the batch backup task is a complete failure status.

[0027] In one embodiment, when the retry execution status of the batch backup task is partially completed, the method further includes:

[0028] Generate retry execution alarm information; wherein, the retry execution alarm information includes the total number of failures.

[0029] In one embodiment, for any virtual machine among all virtual machines to be backed up, after performing a backup operation on any virtual machine to be backed up, if the backup operation of any virtual machine to be backed up fails, the method further includes:

[0030] Obtain the current retry count and the preset maximum retry count for any virtual machine to be backed up;

[0031] If the current number of retries is less than the maximum number of retries, the fault status of any virtual machine to be backed up is set to a temporary fault status, and the virtual machine identifier of any virtual machine to be backed up is updated to the retry set.

[0032] If the current number of retries is greater than or equal to the maximum number of retries, the fault status of any virtual machine to be backed up is set to a long-term fault status.

[0033] Secondly, this application also provides a virtual machine batch backup device, the device comprising:

[0034] A backup response module is used to respond to an execution request of a batch backup task by determining the virtual machine to be backed up associated with the batch backup task and the retry set corresponding to the batch backup task; wherein, the retry set is used to store the virtual machine identifiers of virtual machines that failed to be backed up in the historical backup records of the batch backup task, and the retry set is empty when the execution request of the batch backup task is initiated for the first time;

[0035] The backup execution module is configured to perform backup operations on all virtual machines to be backed up when the retry set is empty; and to perform backup operations on virtual machines whose virtual machine identifiers are located in the retry set among all virtual machines to be backed up associated with the batch backup task when the retry set is not empty.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the preceding claims.

[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0039] The aforementioned virtual machine batch backup method, apparatus, computer device, computer-readable storage medium, and computer program product, upon responding to a batch backup task execution request, determine the virtual machines to be backed up associated with the batch backup task and the retry set corresponding to the batch backup task. Specifically, the retry set is empty upon initial initiation, and contains identifiers of virtual machines that failed in previous backups upon retry initiation. During the execution phase, the scope of the backup operation is determined based on whether the retry set is empty; that is, if the retry set is empty, backup operations are performed on all virtual machines to be backed up; if the retry set is not empty, backup operations are performed only on virtual machines whose identifiers are within the retry set. Based on this, when a batch backup task needs to be retried due to individual virtual machine failures, instead of iterating through and backing up all virtual machines associated with the task, the retry set filter automatically skips virtual machines that have successfully completed backups in the previous round, performing backup operations only on the failed virtual machines. This avoids duplicate data transfer of successfully backed-up virtual machines and wastes storage space, effectively shortening the overall time of batch backup task retrying and saving storage resources. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is an application environment diagram of a virtual machine batch backup method in one embodiment;

[0042] Figure 2 This is a flowchart illustrating a virtual machine batch backup method in one embodiment;

[0043] Figure 3 This is a structural block diagram of a virtual machine batch backup device in one embodiment;

[0044] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] The virtual machine batch backup method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be integrated onto server 102 or placed on a cloud or other network server. In response to the batch backup task execution request issued by terminal 101, server 102 determines the virtual machines (VMs) to be backed up associated with the batch backup task and generates a retry set corresponding to the batch backup task. The retry set stores the VM identifiers of VMs that failed to be backed up in the historical backup records of the batch backup task, and is empty when the batch backup task execution request is initiated for the first time. When the retry set is empty, server 102 performs backup operations on all VMs to be backed up. When the retry set is not empty, server 102 performs backup operations on the VMs whose VM identifiers are located within the retry set among all VMs to be backed up associated with the batch backup task. Terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. Server 102 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0047] In one exemplary embodiment, such as Figure 2 As shown, a method for batch backup of virtual machines is provided, the method including:

[0048] Step 201: In response to the execution request of the batch backup task, determine the virtual machine to be backed up associated with the batch backup task, and the retry set corresponding to the batch backup task; wherein, the retry set is used to store the virtual machine identifiers of the virtual machines that failed to be backed up in the historical backup records of the batch backup task, and the retry set is empty when the execution request of the batch backup task is initiated for the first time.

[0049] Server 102 runs multiple virtual machines, which can be started, stopped, and managed. A backup task refers to a task that backs up the virtual machines on server 102, and can be issued by terminal 101. A batch backup task refers to a backup job that includes at least two virtual machines to be backed up. For example, if an operations and maintenance personnel create a backup job named "Daily Backup of Core Business Cluster" using the built-in backup management control module of terminal 101, which includes 100 virtual machines to be backed up, this job constitutes a batch backup task.

[0050] The execution request is the instruction that triggers the backup task to run. It can be initiated manually by operations and maintenance personnel through the backup management control module, or it can be triggered automatically according to a preset scheduling scheme. The execution request usually carries operation options for the batch backup task, which are used to pass the specific parameters for this execution to the backup agent built into server 102.

[0051] In this context, the virtual machine to be backed up is defined as the virtual machine that needs to be backed up in the batch backup task. Each virtual machine to be backed up has a unique virtual machine identifier in the batch backup task. This virtual machine identifier is used to uniquely identify the virtual machine throughout the entire backup process. It can be a unique ID (identification number) assigned to the virtual machine by the platform, a UUID (Universally Unique Identifier), or something similar.

[0052] The retry set, introduced in this embodiment, is a data set used to indicate which virtual machines need to be re-executed in a retry backup scenario. It can be viewed as a filter, and includes the identifiers of the virtual machines to be re-backed up. In this embodiment, the retry set is empty during the initial backup operation, indicating no filtering conditions and all virtual machines need to be backed up. During retry backup operations, the retry set contains the identifiers of virtual machines that failed in the previous execution. At this time, only virtual machines whose identifiers are located in this set among all virtual machines associated with the batch backup task are backed up, while other successfully backed-up virtual machines are skipped. This retry set can be implemented using various data structures, such as JSON (JavaScript Object Notation, a lightweight data interchange format), arrays, bitmaps, or custom serialization structures. For example, the retry set is a JSON-formatted collection obtained by serializing a list of virtual machine IDs that failed in previous backups, and is embedded in the operation option field of the batch backup task and sent to the backup agent.

[0053] Among them, historical backup records refer to the execution information generated after one or more backup operations of batch backup tasks and their included virtual machines. They can reflect the final status of each virtual machine in a certain execution result (backup success or failure). Historical backup records can be stored in a preset historical record database.

[0054] For example, when server 102 receives an execution request for a batch backup task from the backup management control module in terminal 101, the backup agent in server 102 parses the operation options in the request to obtain the list of virtual machines that need to be backed up in this task. Assume that the batch backup task is associated with 100 virtual machines, whose virtual machine identifiers are vm_id_1 to vm_id_100. Simultaneously, the backup management control module or the backup agent determines whether this execution request is the first time it is initiated or a retry based on the relevant information carried in the execution request or the task's own status flags. If the task is being executed for the first time, an empty retry set is generated, such as an empty JSON object {} or an empty list []. If the task is a retry execution of a previously partially failed job, the list of virtual machine identifiers that failed to be backed up in the previous execution is extracted from the task's historical backup records. For example, if the backup of virtual machines vm_id_1, vm_id_2, and vm_id_3 failed in the previous execution, the retry set would be {"retry_entries": {"vm_id_1": {}, "vm_id_2": {}, "vm_id_3": {}}}, indicating that only these three virtual machines need to be backed up again. If the retry set is generated by the backup management module, it can be embedded in the operation options of the execution request and sent to the backup agent.

[0055] In this embodiment, the associated virtual machine to be backed up and the retry set corresponding to the task are determined when responding to the execution request, so that the batch backup task can be executed for the first time or retried based on the retry set. Specifically, when the retry set is empty, it corresponds to the normal mode of full backup; when the retry set is not empty, it corresponds to a fine-grained partial retry mode. The two modes are switched according to the content of the retry set. In this way, the retry set acts as a filter for this execution, enabling fine-grained control over the backup operation.

[0056] Step 202: If the retry set is empty, perform backup operations on all virtual machines to be backed up.

[0057] Backup operations refer to the steps of data protection processing for the virtual machine to be backed up, used to ensure data security and system stability. Depending on the backup method, they can be divided into snapshot backup, clone backup, and exporting the virtual machine. Taking snapshot backup as an example, the specific steps of the backup operation are as follows: Create a snapshot for the target virtual machine through the virtualization platform interface to obtain the data consistency status at a certain point in time; read the virtual machine's disk data from the source storage location and transfer it to the target storage area using a preset transfer mode; after the transfer is complete, retain or delete the snapshot.

[0058] For example, when the backup agent determines that the retry set is empty, it indicates that this backup is being executed for the first time, and all 100 virtual machines associated with this batch backup task need to be backed up. The backup executor in the backup agent traverses the list of virtual machines to be backed up, which consists of all virtual machines associated with the batch backup task, and creates a snapshot for each virtual machine sequentially or in parallel, and transfers the virtual disk data to the target storage area through a pre-configured transfer mode. After the backup operation for each virtual machine is completed, the backup agent records the execution result for that virtual machine, including success, failure, related error codes, and data volume statistics, providing a basis for possible subsequent retry backups.

[0059] Step 203: If the retry set is not empty, perform a backup operation on the virtual machines whose virtual machine identifier is located in the retry set among all virtual machines to be backed up that are associated with the batch backup task.

[0060] For example, when the retry set is not empty, it indicates that the current backup is being retried. While traversing the list of virtual machines to be backed up, the backup agent checks whether the virtual machine identifier of the current virtual machine exists in the retry set. If it exists, the backup operation is performed; if it does not exist, the virtual machine is skipped. Specifically, the skipping operation includes: not performing any data reading or transmission actions on the virtual machine, directly marking its execution result as "skipped" (STATE_SKIPPED), and updating a skipped virtual machine counter. In an example scenario, a batch backup task containing 100 virtual machines failed in the previous backup operation, and the corresponding retry set is {"retry_entries": {"vm_id_1": {}, "vm_id_2": {}, "vm_id_3": {}}}. During backup retry, if the batch backup task still includes the 100 virtual machines, the backup agent will skip the 97 successfully backed-up virtual machines and only back up the 3 virtual machines in the retry set. However, if the administrator edited or deleted the task before initiating the retry because virtual machine vm_id_3 had been migrated or deprecated, meaning the batch backup task includes 99 virtual machines other than vm_id_3, the backup agent will skip the 97 virtual machines whose virtual machine identifiers are outside the retry set and only back up the two virtual machines vm_id_1 and vm_id_2 whose virtual machine identifiers are in the retry set. Therefore, during batch backup retry, only virtual machines that failed in the previous backup are backed up, skipping successfully backed-up virtual machines. This avoids wasting resources from successfully backed-up machines, significantly saving backup time and storage space. Based on the above example, approximately 97% of the repeated transfer time and storage space can be saved.

[0061] Furthermore, during the virtual machine backup operation, skipped virtual machines report a "skipped" (STATE_SKIPPED) status to the management control module via the progress reporter, and the VM_HIST_STATUS_SKIPPED flag is recorded in their corresponding virtual machine history entry. In the final job statistics, skipped virtual machines are included in the total number of completed jobs, but are not included in the number of successful or failed jobs.

[0062] In the aforementioned batch backup method for virtual machines, when responding to the execution request of a batch backup task, the virtual machines to be backed up associated with the batch backup task and the retry set corresponding to the batch backup task are determined. The retry set is empty upon initial initiation, and contains identifiers of virtual machines that failed in previous backups upon retry initiation. During the execution phase, the scope of the backup operation is determined based on whether the retry set is empty. That is, if the retry set is empty, the backup operation is performed on all virtual machines to be backed up; if the retry set is not empty, the backup operation is performed only on virtual machines whose identifiers are within the retry set. Based on this, when a batch backup task needs to be retried due to individual virtual machine failures, instead of iterating through and backing up all virtual machines associated with the task, the retry set filter automatically skips virtual machines that have already successfully completed backups in the previous round, and only performs backup operations on the failed virtual machines. This avoids duplicate data transfer of successfully backed-up virtual machines and wastes storage space, effectively shortening the overall time of batch backup task retrying and saving storage resources.

[0063] In one embodiment, the retry set is generated by a preset backup management control module, and the steps for generating the retry set are as follows:

[0064] Step S2011: If it is determined that the execution request of the batch backup task is initiated for the first time, a retry set with empty elements is generated.

[0065] In step S2012, if it is determined that the execution request of the batch backup task is not the first time it is initiated, that is, it belongs to the retry initiation, the historical backup record of the batch backup task is obtained, the virtual machine identifier of the virtual machine that failed to be backed up is extracted from the historical backup record, and all the extracted virtual machine identifiers are summarized to obtain the retry set.

[0066] Extracting the virtual machine identifier of a failed backup from historical backup records involves iterating through and filtering all entries in the historical backup records. The filtering condition is that the execution status of the entry is equal to "failed". The extracted virtual machine identifier can be a virtual machine ID, UUID, or other string or numerical value that can uniquely identify the identity of the virtual machine in this task.

[0067] For example, when the backup management control module determines that the current execution request is a retry, it queries the historical record database for the most recent execution history of the batch backup task. The database returns the execution information of all virtual machines in this execution, including fields such as vm_id, vm_name, status, and end_time. The backup management control module iterates through these records, filters out entries where the status field value is "FAILED" (failure status), and extracts their corresponding vm_id. Assuming vm_id_1, vm_id_2, and vm_id_3 are filtered out, these three identifiers are aggregated to construct a retry set. The retry set can be constructed by directly adding these identifiers to a list or set data structure and then serializing it into a JSON string.

[0068] In this embodiment, the steps for generating the retry set are disclosed. Specifically, an empty set is generated during the first execution, and a non-empty set is generated by extracting failure identifiers from the historical records during retry execution. Based on this, the retry set is generated according to the historical backup records, which can ensure the accuracy of the retry set content.

[0069] Currently, in real-world production environments, the virtual machines to be backed up for the same batch backup task may be deployed on multiple different virtualization platforms, such as H3C CAS (a self-developed virtualization platform for data centers), CloudOS (an enterprise cloud-native digital innovation platform), TStack (a cloud computing management platform developed by Tencent), VMware (a virtualization platform developed by VMware), OpenStack (a cloud computing platform jointly developed by Rackspace and NASA), and FusionCompute (a cloud operating system developed by Huawei). These virtualization platforms each have independent historical record storage formats, resulting in inconsistent formats and fragmented storage of virtual machine backup history records in a cross-platform batch backup task. To address this issue, this embodiment further makes the following improvements:

[0070] In one embodiment, the virtual machines to be backed up associated with the batch backup task are deployed on multiple virtualization platforms; correspondingly, obtaining the historical backup records of the batch backup task includes:

[0071] Step a1: Extract the sub-historical records corresponding to the batch backup task from multiple virtualization platforms.

[0072] Sub-historical records refer to portions of historical execution information extracted from a single virtualization platform that conform to that platform's proprietary data format. For example, sub-historical records extracted from a VMware platform might be task event data in XML (Extensible Markup Language) format; sub-historical records extracted from an OpenStack platform might be instance operation logs in JSON format. Although these sub-historical records semantically describe the backup execution status of virtual machines, their field names, data types, and structural organization methods differ.

[0073] Step a2: Standardize each sub-historical record to obtain standardized sub-historical records.

[0074] The standardization process refers to the process of uniformly converting sub-historical records with different formats extracted from different virtualization platforms into a cross-virtualization platform universal virtual machine historical entry data structure defined in this embodiment. The standardization process may include steps such as field mapping, data type conversion, and filling missing fields with default values, which are not limited here.

[0075] Specifically, in this embodiment, a unified virtual machine history entry data structure is defined across different virtualization platforms to standardize the recording of historical execution details for each virtual machine and to create backup records based on this structure. The fields in the virtual machine history entry may include virtual machine identifier, execution start time, execution end time, source data size, backup set size, storage size, status bitmask, transmission mode, and whether it is a retry execution, as shown in Table 1 below:

[0076] Table 1. Data Structure Definition Table for Virtual Machine History Entries

[0077] vm_id string (string) Virtual machine unique identifier vm_uuid string Virtual machine UUID vm_name string Virtual machine name vm_display_name string Virtual machine display name cluster_id string Cluster ID host_id string Host ID host_name string Hostname ip_addrs vector <string> < / string> Virtual machine IP address list start_time time_t Start time end_time time_t End time source_size uint64_t Source data size set_size uint64_t Backup set size storaged_size uint64_t Storage size status bitmask Status bitmask (COMPLETED / RETRY / FAILED / SKIPPED) transport_mode enum (enumeration) Transmission modes (SAN / LAN / NBD, etc.) retry bool (Boolean) Should we retry the execution?

[0078] The historical execution state of the virtual machine is represented by binary bits, with each state occupying an independent binary bit. As an example, the bitmask representation of the historical execution state of the virtual machine is as follows:

[0079] Stateless: VM_HIST_STATUS_NONE = 0b0000

[0080] Completion status: VM_HIST_STATUS_COMPLETED = 0b0001

[0081] Retry status: VM_HIST_STATUS_RETRY = 0b0010

[0082] Failure status: VM_HIST_STATUS_FAILED = 0b0100

[0083] Skip status: VM_HIST_STATUS_SKIPPED = 0b1000

[0084] Based on this, a single historical backup record can simultaneously represent multiple concurrent states through bitwise OR operations (e.g., a backup attempt that was performed but ultimately failed can be marked as 0b0110). There is no need to define enumeration values ​​for each state combination separately, which helps save storage space and facilitates subsequent quick filtering and statistical analysis through bitwise AND operations.

[0085] The above data structure defines a set of predefined standardized fields, including, for example: vm_id (unique identifier of the virtual machine), vm_uuid (virtual machine UUID), vm_name (virtual machine name), vm_display_name (virtual machine display name), cluster_id (cluster ID), host_id (host ID), host_name (host name), ip_addrs (list of virtual machine IP addresses), start_time (start time), end_time (end time), source_size (source data size), set_size (backup set size), stored_size (storage size), status (status bitmask), transport_mode (transport mode), retry (whether to retry execution), etc.

[0086] For example, a batch backup task is associated with two virtual machines to be backed up, deployed on VMware and OpenStack platforms respectively. The sub-historical records of these two virtualization platforms are standardized by performing field standardization mapping through the aforementioned virtual machine historical entry data structure, as well as unified conversion of time format, etc. After processing, standardized sub-historical records with the same field structure and the same semantics can be obtained.

[0087] Step a3: Summarize all standardized sub-historical records to obtain standardized historical backup records for the batch backup task.

[0088] In this embodiment, a predefined virtual machine history entry data structure is used to standardize the processing of each sub-historical record, enabling unified historical querying and analysis across virtualization platforms and meeting the backup requirements across virtualization platforms. Furthermore, after obtaining the standardized historical backup records, the virtual machine identifiers of virtual machines that failed to be backed up can be extracted, and all extracted virtual machine identifiers are aggregated to obtain the final retry set. Based on this, the retry backup scheme of this embodiment can be applied to complex and heterogeneous virtualization environments, thus having wider applicability.

[0089] Furthermore, after obtaining standardized historical backup records for batch backup tasks, this embodiment adopts a batch migration strategy to write the standardized historical backup records into a unified job history table in batches, thereby avoiding the performance overhead of inserting a single record.

[0090] In one embodiment, after extracting the sub-historical records corresponding to the batch backup task, the method further includes:

[0091] Step a4: Summarize all sub-historical records to obtain the initial historical backup records for the batch backup task, and then store the initial historical backup records and the standardized historical backup records together.

[0092] Initial historical backup records refer to the raw historical record set that is directly aggregated from the sub-historical records in the proprietary formats of each platform and has not undergone standardization. These records retain the original data format and all original fields of each platform. Standardized historical backup records, on the other hand, are records that have undergone the aforementioned transformation using a unified data structure.

[0093] The process of associating and storing initial historical backup records and standardized historical backup records refers to persistently saving historical records in two formats using a dual-write model and establishing a logical association between them. This association can be achieved by including a field in the database's job history table that points to the storage location of the old format record and another that points to the storage location of the new standardized format record, or by linking the two records using a common job identifier.

[0094] For example, all raw, unstandardized sub-historical records are aggregated to obtain the initial historical backup records for the batch backup task. Subsequently, the initial historical backup records and standardized historical backup records are associated and stored. For example, the job history table contains both a field pointing to the old format record (legacy_record_link) and a field pointing to the new standardized format record (unified_record_link).

[0095] In this embodiment, by introducing a dual-write mechanism that associates and stores initial historical backup records with standardized historical backup records, a smooth migration of historical records from multi-platform proprietary formats to a unified standardized format can be achieved. The dual-write mechanism ensures the normal operation of online services such as queries relying on the old format during the upgrade process of historical record formats from old to new, and the old and new systems can run in parallel, ensuring backward compatibility and a smooth transition, thus reducing the risk of system upgrades.

[0096] In one embodiment, after performing a backup operation on all virtual machines to be backed up among all virtual machines associated with the batch backup task, where the virtual machine identifier is located within the retry set, the method further includes:

[0097] Step S204: Obtain the total number of virtual machines to be backed up associated with the batch backup task, and determine the virtual machines whose virtual machine identifiers are outside the retry set among all the virtual machines to be backed up associated with the batch backup task as virtual machines to be skipped during backup.

[0098] The total number of virtual machines to be backed up refers to the total number of virtual machines associated with the current batch backup task, which can be represented by `total_vm`. This value is determined during task definition and can be obtained directly from the task's operation options.

[0099] Step S205: Based on the total number of virtual machines to be backed up and the total number of virtual machines skipped from backup, obtain the total number of virtual machines actually performing backup operations; wherein, the total number of executions is the difference between the total number of virtual machines to be backed up and the total number of skipped virtual machines.

[0100] The total number of skipped virtual machines (VMs) refers to the total number of VMs skipped during the backup operation because their identifiers were not in the retry set. It is denoted by `skipped_vm`. This value can be obtained by the backup executor in the backup agent during execution by accumulating a skipped VM counter.

[0101] The total number of virtual machines actually backed up (including successful and failed backups) during this backup operation is denoted by `executed_vm`. It is calculated as: `executed_vm = max(total_vm - skipped_vm, 0)`, which is the total number of virtual machines to be backed up minus the total number of virtual machines that skipped backups.

[0102] Step S206: Obtain the total number of successful backup operations for virtual machines whose backup operation results were successful and the total number of failed backup operations for virtual machines whose backup operation results were unsuccessful.

[0103] The total number of successful backups is the total number of virtual machines that were successfully backed up, denoted by succeeded_vm, while the total number of failed backups is the total number of virtual machines that were failed to be backed up, denoted by failed_vm.

[0104] Step S207: Based on the total number of successful backups and the total number of backups executed, obtain the retry execution status of the batch backup task.

[0105] Among them, the retry execution status is a status label used to describe the final result of the batch backup task after the backup execution is retried.

[0106] For example, after a batch backup task involving 100 virtual machines completes its retry execution, the backup agent performs status statistics. Assuming the job was not canceled, and total_vm=100, among all the virtual machines to be backed up associated with the batch backup task, the retry set contains 3 virtual machines (vm_id_1, vm_id_2, vm_id_3), therefore skipped_vm=100-3=97. Of the 3 virtual machines that actually performed the backup operation, assuming vm_id_2 and vm_id_3 were backed up successfully, and vm_id_1 failed, then succeeded_vm=2, failed_vm=1, and executed_vm=max(total_vm-skipped_vm,0)=3. The backup agent compares succeeded_vm=2 with executed_vm=3 to determine the final status of the job.

[0107] In this embodiment, by introducing the total number of virtual machines that are skipped during backup, and by determining the retry execution status of batch backup tasks based on the total number of executions and the total number of successes, the status of the job can truly reflect the intermediate execution results of "partial success and partial failure".

[0108] In one embodiment, the retry execution status of a batch backup task includes a partially completed status, a fully completed status, and a completely failed status; wherein,

[0109] If the total number of successful attempts is greater than 0 and the total number of successful attempts is less than the total number of executions, then the retry execution status of the batch backup task is partially completed.

[0110] If the total number of successful backups equals the total number of backups executed, then the retry execution status of the batch backup task will be "all completed".

[0111] If the total number of successful attempts is 0, the retry execution status of the batch backup task will be all failures.

[0112] Wherein, the partially completed status (JOB_STATE_PARTIALLY_COMPLETED) is a new final job execution status introduced in this embodiment. Different from the traditional statuses with only "success" or "failure", it can describe the intermediate execution result status that "some virtual machines are successfully backed up while other virtual machines fail to be backed up", so as to more accurately reflect the actual completion of the job. By introducing the partially completed status, operation and maintenance personnel can quickly identify which jobs need attention (partially completed) and which jobs are completely successful or completely failed, without checking one by one among a large number of successful or failed jobs.

[0113] Wherein, the fully completed status (JOB_STATE_COMPLETED) indicates that all virtual machines that actually perform the backup operation are successfully backed up, and the fully failed status (JOB_STATE_FAILED) indicates that all virtual machines that actually perform the backup operation fail to be backed up.

[0114] By way of example, according to the foregoing example, succeeded_vm=2 and executed_vm=3. Since succeeded_vm>0 and succeeded_vm<executed_vm, the determination condition for the partially completed status is satisfied, and the final status of the batch backup task is marked as the partially completed status. If all the 3 retried virtual machines are successfully backed up, then succeeded_vm=3=executed_vm, and the job status is the fully completed status at this time. If all the 3 retried virtual machines fail to be backed up, then succeeded_vm=0, and the job status is the fully failed status at this time.

[0115] In this embodiment, by introducing the partially completed status, the problem in the prior art that the backup operation lacks the intermediate status of partial completion and cannot accurately reflect the actual execution situation is solved. Operation and maintenance personnel can quickly filter out jobs in the partially completed status through the status field, and can solve problematic jobs through manual intervention, thereby effectively improving operation and maintenance efficiency.

[0116] It should be understood that, in the normal mode of full backup, the completion status of the backup job can also be automatically determined based on the processing logic of the foregoing steps S204 to S206. In this case, the total number of skipped virtual machines that are skipped for backup is directly set to 0, and the total number of executed virtual machines that actually perform the backup operation is exactly the total number of virtual machines to be backed up. Then the total number of successfully backed up virtual machines is obtained according to the backup operation results, and the initial execution status of the batch backup task is obtained according to the total number of successful virtual machines and the total number of executed virtual machines; correspondingly, the initial execution status can also include the partially completed status, the fully completed status and the fully failed status.

[0117] Furthermore, in practical applications, the criteria for determining the partial completion status can be extended to be based on a success rate threshold (e.g., if the success rate is below 80%, it is marked as a failure). Compared with this implementation method, the determination process in this embodiment is simpler and faster.

[0118] In one embodiment, when the retry execution status of the batch backup task is partially completed, the method further includes:

[0119] Generate retry execution alarm information; the retry execution alarm information includes the total number of failures. The total number of failures is the number of virtual machines that actually performed backup operations but ultimately failed in this execution, i.e., failed_vm. This value directly reflects the scale of failed virtual machines that need to be monitored in the job.

[0120] The retry execution alarm is an automatic notification generated by the system when it detects that a job has entered a partially completed state. This alarm proactively informs operations and maintenance personnel that some virtual machine backups have failed, eliminating the need for them to manually check the job status periodically. The alarm message can include the alarm level (e.g., WARNING), error code (e.g., COMMON_CODE_PARTIAL_VM_BACKUP_FAILED), context information (e.g., total number of failures), and the alarm message body (e.g., "This job has N virtual machine backup failures. Please log in to the management console > history interface to view the job logs and virtual machine details."). It can be sent to operations and maintenance personnel via email, SMS, system push notifications, and other methods.

[0121] For example, when the aforementioned job is determined to be partially completed, the system automatically generates an alarm message, including the following: alarm level is WARNING; error code is COMMON_CODE_PARTIAL_VM_BACKUP_FAILED; the context information records the number of failed virtual machines, failed_vm=1; the alarm message body is: "One virtual machine backup failed in this job. Please log in to the management console > history interface to view the job log and virtual machine details." After receiving the alarm, the operations and maintenance personnel can directly click the alarm link to jump to the history interface of the management control module to view which specific virtual machine failed in the job, the reason for the failure, etc., so as to quickly take targeted repair measures, such as manually retrying the single virtual machine or checking its storage status.

[0122] In this embodiment, by automatically generating alarm information when the retry execution status of the batch backup task is in a partially completed state, the system automatically pushes the jobs that need attention to the maintenance personnel without the need for the maintenance personnel to passively check the job status one by one in the case of partial failure. This shortens the fault response time and further improves the maintenance efficiency.

[0123] In one embodiment, for any virtual machine among all virtual machines to be backed up, if the backup operation fails after performing a backup operation on any virtual machine, the method further includes:

[0124] Get the current number of retries and the preset maximum number of retries for any virtual machine to be backed up;

[0125] If the current number of retries is less than the maximum number of retries, set the fault status of any virtual machine to be backed up to a temporary fault status, and update the virtual machine identifier of any virtual machine to be backed up to the retry set.

[0126] If the current number of retries is greater than or equal to the maximum number of retries, set the fault status of any virtual machine to be backed up to a long-term fault status.

[0127] The current retry count refers to the cumulative number of times the virtual machine has failed to back up within the current job execution cycle. In this embodiment, an independent retry counter is configured for each virtual machine, using the VM_Job_Progress data structure, which includes a retry_count field to record the current retry count for that virtual machine.

[0128] The preset maximum number of retries is a configurable threshold parameter, represented by `max_retry_count`. This threshold can be set by the administrator based on the importance of the business, the virtual machine type, or the type of failure. For example, the maximum number of retries for a core business virtual machine can be set to 5, while the maximum number of retries for a regular test virtual machine can be set to 1.

[0129] Temporary failures refer to backup failures caused by occasional reasons such as network interruptions or momentary resource shortages. These failures can usually be resolved by retrying. Long-term failures refer to backup failures caused by insufficient storage space, abnormal virtual machine status, or permission errors. These failures cannot be resolved by simple retries and require manual troubleshooting and intervention.

[0130] Updating the virtual machine identifier to the retry set means that when a virtual machine is determined to be retryable due to a temporary failure, its identifier is added to the retry set of the current job, and the backup agent's task scheduler will reschedule and execute it at an appropriate time.

[0131] For example, suppose a virtual machine is configured with a `max_retry_count` of 3. After the virtual machine's first backup fails, its `retry_count` is 0, which is less than `max_retry_count`, so the system determines it's a temporary failure. The task scheduler resets the virtual machine's state to `STATE_PENDING` (pending processing), adds it back to the execution queue, and records the retry interval (e.g., 30 seconds) before scheduling it for execution again. If the second execution also fails, `retry_count` is incremented to 1, still less than 3, and it's retried again. If the third execution fails, `retry_count` is incremented to 2, still less than 3, and it continues to retry. If the fourth execution fails, `retry_count` is 3, equal to `max_retry_count`, and the system determines the failure to be a long-term failure. The task scheduler stops retries for the virtual machine, finally marks its state as `STATE_FAILED`, returns the error code `CODE_FAILED`, and no longer adds it to the execution queue. At this point, the automatic retry process for the virtual machine ends, and it needs to wait for intervention from operations personnel.

[0132] In this embodiment, a retry counter is set for each virtual machine to count retries, and a maximum retry threshold is set to distinguish between temporary and long-term faults. For temporary faults, a limited number of retries can be performed automatically to maximize backup completion and reduce the need for manual intervention; for long-term faults, retries can be terminated promptly after the maximum number of retries is reached, reducing unnecessary waiting time.

[0133] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0134] Based on the same inventive concept, this application also provides a virtual machine batch backup apparatus for implementing the virtual machine batch backup method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more virtual machine batch backup apparatus embodiments provided below can be found in the limitations of the virtual machine batch backup method described above, and will not be repeated here.

[0135] In one exemplary embodiment, such as Figure 3 As shown, a virtual machine batch backup device is provided, comprising:

[0136] The backup response module is used to respond to the execution request of the batch backup task, determine the virtual machine to be backed up associated with the batch backup task, and the retry set corresponding to the batch backup task; wherein, the retry set is used to store the virtual machine identifiers of the virtual machines that failed to be backed up in the historical backup records of the batch backup task, and the retry set is empty when the execution request of the batch backup task is initiated for the first time.

[0137] The backup execution module is used to perform backup operations on all virtual machines to be backed up when the retry set is empty; it is also used to perform backup operations on virtual machines whose virtual machine identifiers are located in the retry set among all virtual machines to be backed up associated with the batch backup task when the retry set is not empty.

[0138] Each module in the aforementioned virtual machine batch backup device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0139] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores data related to batch backups. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a virtual machine batch backup method.

[0140] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0141] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the steps of the method as described in any of the preceding embodiments.

[0142] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method as described in any of the preceding embodiments.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0144] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0145] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for batch backup of virtual machines, characterized in that, The method includes: In response to the execution request of a batch backup task, the virtual machine to be backed up associated with the batch backup task and the retry set corresponding to the batch backup task are determined; wherein, the retry set is used to store the virtual machine identifiers of virtual machines that failed to be backed up in the historical backup records of the batch backup task, and the retry set is empty when the execution request of the batch backup task is initiated for the first time; If the retry set is empty, perform backup operations on all virtual machines to be backed up; If the retry set is not empty, the backup operation is performed on the virtual machine identified as being located in the retry set among all virtual machines to be backed up associated with the batch backup task.

2. The method according to claim 1, characterized in that, The retry set is generated by a preset backup management control module, and the steps for generating the retry set are as follows: If it is determined that the execution request for the batch backup task is being initiated for the first time, a retry set with empty elements is generated. If it is determined that the execution request of the batch backup task is not the first one, the historical backup records of the batch backup task are obtained, and the virtual machine identifiers of the virtual machines that failed to be backed up are extracted from the historical backup records. All extracted virtual machine identifiers are then aggregated to obtain a retry set.

3. The method according to claim 2, characterized in that, The virtual machines to be backed up, associated with the batch backup task, are deployed on multiple virtualization platforms; obtaining the historical backup records of the batch backup task includes: Extract sub-historical records corresponding to the batch backup task from multiple virtualization platforms; Standardize each sub-historical record to obtain a standardized sub-historical record; All standardized sub-historical records are aggregated to obtain the standardized historical backup records for the batch backup task.

4. The method according to claim 3, characterized in that, After extracting the sub-historical records corresponding to the batch backup task, the method further includes: All sub-historical records are aggregated to obtain the initial historical backup record of the batch backup task, and the initial historical backup record and the standardized historical backup record are associated and stored.

5. The method according to claim 1, characterized in that, After performing backup operations on the virtual machines whose virtual machine identifiers are located within the retry set among all virtual machines to be backed up associated with the batch backup task, the method further includes: Obtain the total number of virtual machines to be backed up associated with the batch backup task, and determine the virtual machines whose virtual machine identifiers are outside the retry set among all the virtual machines to be backed up associated with the batch backup task as virtual machines to be skipped during backup; Based on the total number of virtual machines to be backed up and the total number of virtual machines skipped from backup, the total number of virtual machines actually performing backup operations is obtained; wherein, the total number of virtual machines actually performing backup operations is the difference between the total number of virtual machines to be backed up and the total number of skipped virtual machines. The total number of successful backup operations for virtual machines and the total number of failed backup operations for virtual machines are obtained from the total number of executions. The retry execution status of the batch backup task is obtained based on the total number of successful backups and the total number of executions.

6. The method according to claim 5, characterized in that, The retry execution status of the batch backup task includes partially completed, fully completed, and completely failed status; among which, If the total number of successful attempts is greater than 0 and the total number of successful attempts is less than the total number of executions, then the retry execution status of the batch backup task is a partially completed status. If the total number of successful attempts equals the total number of executions, then the retry execution status of the batch backup task is "complete". If the total number of successful attempts is 0, then the retry execution status of the batch backup task is a complete failure status.

7. The method according to claim 6, characterized in that, When the retry execution status of the batch backup task is partially completed, the method further includes: Generate retry execution alarm information; wherein, the retry execution alarm information includes the total number of failures.

8. The method according to claim 1, characterized in that, For any virtual machine among all virtual machines to be backed up, after performing a backup operation on any virtual machine to be backed up, if the backup operation fails, the method further includes: Obtain the current retry count and the preset maximum retry count for any virtual machine to be backed up; If the current number of retries is less than the maximum number of retries, the fault status of any virtual machine to be backed up is set to a temporary fault status, and the virtual machine identifier of any virtual machine to be backed up is updated to the retry set. If the current number of retries is greater than or equal to the maximum number of retries, the fault status of any virtual machine to be backed up is set to a long-term fault status.

9. A virtual machine batch backup device, characterized in that, The device includes: A backup response module is used to respond to an execution request of a batch backup task by determining the virtual machine to be backed up associated with the batch backup task and the retry set corresponding to the batch backup task; wherein, the retry set is used to store the virtual machine identifiers of virtual machines that failed to be backed up in the historical backup records of the batch backup task, and the retry set is empty when the execution request of the batch backup task is initiated for the first time; The backup execution module is configured to perform backup operations on all virtual machines to be backed up when the retry set is empty; and to perform backup operations on virtual machines whose virtual machine identifiers are located in the retry set among all virtual machines to be backed up associated with the batch backup task when the retry set is not empty.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images