Virtual Machine Backup via Storage System Change Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual machine (VM) backup and restore processes in data storage systems are resource-intensive and inefficient, particularly due to the challenges of managing server resources, identifying storage containers, and tracking changes across system shutdowns, which complicates incremental backups and restores in virtual machine environments.
Innovation Solution
The solution involves using a VM API to establish a consistent state of virtual machines, taking snapshots, and copying these snapshots to a target backup storage system, where metadata is interpreted to deduplicate and store data efficiently, allowing for both full and incremental backups and restores, while minimizing resource consumption by only transmitting changed data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If backup agents are installed and run in each VM to handle backup as physical machines, then backup capability is provided, but server resource consumption increases and management becomes unwieldy
Solution Approach 1:
The backup agent functionality is extracted from individual VMs and consolidated into a single backup application server. The backup application communicates directly with storage systems to perform backups without requiring agents in each VM, thereby reducing server resource consumption while maintaining backup capability.
Solution Approach 2:
The backup application server performs universal backup functions for multiple VMs through direct storage system communication. This multi-functional approach eliminates the need for separate backup agents in each VM, reducing overall resource consumption while providing comprehensive backup coverage.
2Reliability
If backup agents are installed and run in each VM to handle backup as physical machines, then backup capability is provided, but management complexity increases as VM count grows
Solution Approach 1:
Multiple backup agent functions are merged into a single backup application server that communicates directly with storage systems. This consolidation simplifies management by eliminating the need to individually configure and maintain backup agents in each VM, thereby reducing management complexity as VM counts grow.
3Productivity
If storage level backup is used by copying storage containers containing VM, then backup is performed, but identifying exact storage containers and achieving consistent state becomes challenging
Solution Approach 1:
The backup application serves as an intermediary between the virtualization environment and storage systems. It receives backup requests, identifies the specific storage containers containing VMs, and coordinates with storage systems to achieve consistent states before copying data, thereby simplifying the complex tasks of storage container identification and consistency management.
4Loss of substance
If incremental backup walks the file system to find changed files, then incremental backup is achieved, but the process becomes slow and resource intensive
Solution Approach 1:
The mechanical file system walking process is replaced with storage system-based change tracking mechanisms. The backup application leverages storage system capabilities to identify changed data blocks without traversing the file system, thereby maintaining incremental backup efficiency while significantly improving backup speed and reducing resource consumption.
5Productivity
If CBT feature is used to track data blocks changed since last backup, then incremental backup efficiency improves, but overhead on VM operation increases and changes across system shutdowns are not tracked
Solution Approach 1:
The backup application acts as an intermediary that receives change tracking information from storage systems rather than relying solely on VM-level CBT features. This approach maintains incremental backup efficiency while improving reliability by capturing changes across system shutdowns through storage system persistence.
6Reliability
If VM API is used to pull data out of virtual infrastructure onto backup system, then backup is achieved, but routing through proxy server consumes precious IT resources
Solution Approach 1:
The data routing function is extracted from the proxy server and directly connected between the backup application and storage systems. This eliminates the need for proxy server intermediation, reducing IT resource consumption while maintaining reliable backup capability through direct storage system access.
Data Source
AI summary
Techniques for virtual machine incremental backup are described herein. According to one embodiment, a request for an incremental backing up a virtual machine (VM) is received at a storage system, the request identifying a requested VM disk image associated with a consistent state of the VM. The storage system determines a difference between the requested VM disk image and a previous VM disk image representing a previous VM backup. The changes between the requested VM disk image and a previous VM disk image are then transmitted to a target backup storage system.