Deduplication Storage Reverse Mapping for Efficient Data Location
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication storage systems face inefficiencies in storing and processing redundant data copies, as comparing actual data units for duplicates is time and resource-intensive, prompting the need for a method to efficiently locate and process data in a deduplication storage system.
Innovation Solution
A deduplication storage system that maps logical volumes to physical block locations, using reverse-mapping information to associate logical block locations with files, allowing for efficient identification of duplicate files and data processing across multiple filesystems without directly accessing each file, and employing techniques like hash generation for data deduplication and storage virtualization to minimize storage of redundant data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored on a deduplication storage system without reverse-mapping information, then storage space is saved by eliminating redundant copies, but locating and processing specific data becomes time-consuming and resource-intensive
Solution Approach 1:
The patent introduces reverse-mapping information as an intermediary data structure that connects logical block locations to physical block locations and file names. This intermediary enables efficient data location by allowing the system to directly query which files contain specific data blocks without scanning entire files, thus resolving the contradiction between space savings and location speed.
Solution Approach 2:
The system performs preliminary actions by pre-establishing reverse-mapping information when data is first stored or modified. This preliminary indexing of the relationship between logical blocks, physical blocks, and file names enables rapid subsequent queries about data location and file associations without requiring time-consuming scans during actual data retrieval operations.
2Reliability
If the system scans all files to locate duplicates or process data, then complete data processing is achieved, but processing time and system resources increase significantly
Solution Approach 1:
The patent extracts the essential information needed for data processing from the full file contents by utilizing reverse-mapping information. Instead of scanning entire files to find duplicates or process data, the system extracts and processes only the metadata (logical block addresses, physical block addresses, and file name relationships) stored in the reverse-mapping structure, achieving complete processing without examining all file data.
Solution Approach 2:
Reverse-mapping information is built in advance during data storage operations, creating a pre-organized index that maps logical block locations to physical locations and associated file names. This preliminary action enables subsequent processing operations to work directly with this indexed metadata rather than scanning files sequentially, dramatically improving processing speed while maintaining completeness.
3Ease of operation
If multiple filesystems are processed independently, then filesystem operations remain simple and isolated, but locating and processing data across filesystems becomes complex and inefficient
Solution Approach 1:
The reverse-mapping information structure serves multiple functions simultaneously: it tracks relationships within individual filesystems and enables cross-filesystem queries. By maintaining a unified reverse-mapping structure that records logical block addresses, physical block addresses, and file names across all filesystems, the system achieves universal data location capability without requiring separate processing mechanisms for each filesystem.
Solution Approach 2:
The patent merges the reverse-mapping information from multiple filesystems into a single integrated structure. This consolidation allows the system to treat all filesystems uniformly when locating and processing data, combining what would otherwise be separate complex operations into a single simplified query process that works across filesystem boundaries.
Data Source
AI summary
A system and method for efficiently locating and processing data on a deduplication storage system are provided. A logical volume is mapped to a deduplication storage area. The logical volume includes a collection of logical block locations and the deduplication storage area includes a collection of physical block locations. The collection of logical block locations is mapped to the collection of physical block locations. A filesystem organizes the logical volume by organizing the logical block locations into a collection of files. Reverse-mapping information is generated for a logical block location that associates the logical block location with at least one file. In response to receiving a logical block address of a logical block location, at least one file is returned.