Data Deduplication via Digital Signatures for Backup Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for retrieving and managing data from large quantities of backup tapes are costly, inefficient, and prone to duplicating data, especially as systems change over time, making it difficult to recreate the original environment for data restoration and storage.
Innovation Solution
Systems and methods for extracting files from various media without duplicating identical components, using digital signatures generated by algorithms like SHA1 hashing to compare and store content and metadata, ensuring that only unique data is stored, thereby reducing storage needs and costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is restored from backup tapes using conventional methods, then data can be retrieved, but data duplication occurs and storage costs increase
Solution Approach 1:
The patent uses digital signatures (hash values) as unique identifiers for data components, analogous to color coding. Each data component is assigned a digital signature that serves as its unique identifier, allowing the system to recognize and track duplicate components across different backup sets without storing redundant data.
Solution Approach 2:
Instead of copying actual data components, the patent creates and stores only unique digital signatures (hash values) of data components. When data is restored, the system uses these stored signatures to identify and retrieve the original data components from backup media, avoiding duplication of the actual data while maintaining retrieval capability.
2Reliability
If backup environment is replicated to restore data, then data restoration is possible, but system complexity increases
Solution Approach 1:
The patent extracts the essential identification information (digital signatures) from the backup environment and stores it separately in a database. This separation allows data restoration without needing to replicate the entire backup environment, as the stored signatures serve as keys to retrieve data components from the original backup media.
Solution Approach 2:
The patent introduces a database as an intermediary layer between the backup media and the restoration process. The database stores digital signatures that mediate the restoration operation, allowing the system to identify and retrieve data components without directly replicating the original backup environment or software configuration.
3Reliability
If all data records are examined during backup, then complete data coverage is achieved, but processing time increases
Solution Approach 1:
The patent performs preliminary processing during the backup operation by generating and storing digital signatures for all data components as they are backed up. This preliminary action creates an index of unique data components that enables rapid identification and retrieval during restoration, eliminating the need to examine all data records during subsequent backup or restoration operations.
Data Source
AI summary
A method and system can be used to read and obtain data from a variety of media, regardless of the application used to generate the backup media. The component parts of a file may be read from a medium, including content and metadata pertaining to a file. These pieces of content and metadata may then be stored and associated. To avoid duplication of data, pieces of content and metadata may be compared to previously stored content and metadata. Furthermore, using these same methods and systems the content and metadata of a file may be associated with a location where the file resided. A database which stores these components and allows linking between the various stored components may be particularly useful in implementing embodiments of these methods and systems.