Mapping Layer for Block-Level Data Reduction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data storage systems face inefficiencies due to the overhead of deduplication methods, which increase latency and overhead in write and read operations, especially in primary data storage, as they require location updates and metadata management, leading to suboptimal performance.

Innovation Solution

A system with a virtual logic unit (vLUN) layer and a mapped logical unit (LUN) layer, along with a mapping layer that optimizes data access by reserving specific regions for direct and shared data, reducing the need for location allocation during writes and minimizing metadata access during reads, utilizing a data reduction engine for deduplication and compression.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If conventional deduplication methods are used, then data storage efficiency is improved, but read and write performance deteriorates due to location updates and metadata management overhead

Engineering Contradiction:

Improvedata storage efficiencyVSAvoidread and write performance

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The storage space is segmented into direct regions and shared regions. Direct regions store unique data blocks while shared regions store deduplicated data blocks. This segmentation allows the system to efficiently manage both unique and duplicate data, reducing metadata overhead by clearly delineating where different types of data reside.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system pre-allocates direct regions for each virtual LUN before deduplication operations. This preliminary action eliminates the need for dynamic location allocation during write operations, reducing write latency. The direct regions are reserved in advance, so when data needs to be written, the location is already determined.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If block-level deduplication is implemented, then redundant data is eliminated, but write latency increases due to the need to find unallocated space and update maps

Engineering Contradiction:

Improveredundant data eliminationVSAvoidwrite latency

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

Direct regions are pre-allocated for each virtual LUN before any write operations occur. This eliminates the need to search for unallocated space during writes, as the system already knows where direct regions are located. The map of direct region locations is established in advance, reducing write latency significantly.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system applies different storage qualities to different types of data blocks. Unique data blocks are stored in direct regions with direct access characteristics, while duplicate blocks are stored in shared regions with deduplication characteristics. This local differentiation optimizes write performance for unique data while maintaining space efficiency for duplicate data.

Inventive Principle:

Principle #3Local quality

3Quantity of substance

If deduplication is used, then storage capacity is optimized, but read overhead increases due to metadata lookups and data reconstitution

Engineering Contradiction:

Improvestorage capacity optimizationVSAvoidread overhead

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The storage space is segmented into direct regions and shared regions, with each type optimized for different access patterns. Direct regions provide fast access for unique data, while shared regions efficiently store duplicate data. This segmentation reduces read overhead by allowing the system to directly access data based on its location type without extensive metadata lookups.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system introduces an intermediary layer (the mapping structure) that caches location information for both direct and shared regions. This intermediary allows read operations to quickly determine whether data is in a direct or shared region and retrieve it accordingly, minimizing the overhead of metadata lookups and data reconstitution.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Productivity

If sequential writes are performed on deduplicated storage, then data is stored in chunks, but data order is disrupted requiring traversal of multiple locations

Engineering Contradiction:

Improvewrite efficiencyVSAvoiddata location management complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The storage space is segmented into direct regions and shared regions with clear boundaries. Sequential writes to direct regions maintain data order within each region, while the mapping structure tracks the location of each virtual LUN's direct region. This segmentation simplifies location management by providing a clear, hierarchical structure rather than requiring traversal of multiple scattered locations.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS8140821B1Efficient read/write algorithms and associated mapping for block-level data reduction processes

Publication Date: 2012.03.20 EMC IP HLDG CO LLC

AI summary

A system configured to optimize access to stored chunks of data is provided. The system comprises a vLUN layer, a mapped LUN layer, and a mapping layer disposed between the vLUN and the mapped LUN. The vLUN provides a plurality of logical chunk addresses (LCAs) and the mapped LUN provides a plurality of physical chunk addresses (PCAs), where each LCA or PCA stores a respective chunk of data. The mapping layer defines a layout of the mapped LUN that facilitates efficient read and write access to the mapped LUN.