Backup Integrity Validation via Chunk Signatures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data backup methods for large storage systems are resource-intensive and may not efficiently detect errors in backup storage objects until a restore operation, particularly when using direct push techniques with large storage objects like those found in storage area networks (SANs).
Innovation Solution
A method where a storage array selects chunks of data from a production device, calculates chunk signatures, and sends both the chunks and signatures to a remote storage system, creating a snap pair for efficient backup and integrity verification, which can be used for data recovery without requiring the initiator storage array to read the target storage array's data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the production storage array reads the backup storage object data from the backup storage array to verify data integrity, then data integrity can be verified, but significant resources are required because the storage objects are typically large
Solution Approach 1:
The patent extracts only the essential verification element (signature) from the complete backup data. Instead of reading and comparing entire backup storage objects, the system calculates and transmits only signatures (hash values) of the backup data, which are minimal in size yet sufficient for integrity verification. This dramatically reduces the resources required for verification while maintaining reliability.
Solution Approach 2:
The patent transforms the verification approach by changing the parameter being verified from the entire data set to a derived parameter (signature). By calculating cryptographic signatures or hash values of the backup storage object and comparing these compact representations, the system achieves integrity verification with minimal resource consumption instead of comparing the full large-scale data sets.
2Productivity
If the production storage array pushes data directly to the backup storage array, then backup efficiency is improved, but errors in the backup storage object may remain unknown until a restore operation is attempted
Solution Approach 1:
The patent implements feedback by calculating and transmitting signatures of the backup data from the production storage array to the backup storage array. The backup array can then verify these signatures against the stored backup data, providing immediate feedback on data integrity. This allows error detection to occur during or immediately after the backup process rather than waiting for restore operations.
Solution Approach 2:
The patent performs preliminary integrity verification by calculating signatures during the backup process itself. Rather than waiting for restore operations to detect errors, the system proactively computes and validates signatures as data is being backed up, ensuring data integrity is established before the backup is considered complete.
3Reliability
If a host server reads storage object data from a production storage array and writes it to a backup storage object, then backup can be created, but significant host server resources are required
Solution Approach 1:
The patent enables the storage array itself to perform the backup operation without requiring host server intervention. The production storage array directly pushes data to the backup storage array and autonomously calculates and transmits signatures for verification. This self-service approach eliminates the need for host server resources to be consumed for backup operations while maintaining reliable backup creation.
Data Source
AI summary
A primary storage array calculates signatures of chunks of production device data that are sent to a target device on a secondary storage array. The chunk signatures are sent to a signature device on the secondary storage array, where the chunk signatures are stored within the same LBA range on the signature device as their corresponding chunks are stored on the target device. Snaps of the target and signature device are created and associated as a snap pair. Later, the primary storage array calculates signatures of changed chunks of production device data that are sent to the target device. The changed chunk signatures are sent to the signature device. New snaps of the target and signature device are created and associated as a new snap pair. Chunk data is validated by calculating signatures of the chunks from the target device and comparing those signatures with the chunk signatures from the signature device.