ECC-Based Data De-Duplication for NAND Memory Endurance

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current data de-duplication methods in solid state storage systems and virtual machines are inefficient, leading to software overhead and reduced performance due to the need for additional storage and increased energy consumption, especially in NAND memory devices where write operations degrade the memory over time.

Innovation Solution

Implementing a data de-duplication system that uses error-correcting code (ECC) check bytes for both error correction and de-duplication purposes, employing a concatenated coding scheme to reduce the need for additional storage and improve the endurance of NAND devices by minimizing redundant data writes and extending the device's lifespan.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional hash algorithms are used for data de-duplication, then data duplication can be detected, but storage overhead increases due to additional hash digest storage

Engineering Contradiction:

Improvedata de-duplication detectionVSAvoidstorage overhead

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent applies multi-functionality by making ECC check bytes serve dual purposes: traditional error correction and data de-duplication detection. By comparing ECC check bytes from different data blocks, the system can identify duplicates without requiring separate hash digest storage, thereby eliminating the 7% overhead associated with conventional hash-based approaches while maintaining reliable duplicate detection

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent merges the error correction function and de-duplication detection function into a single ECC check byte mechanism. Instead of implementing separate hash calculation and storage systems, the invention combines these functions by utilizing the existing ECC check bytes for both error detection and duplicate data identification, reducing overall system complexity and storage requirements

Inventive Principle:

Principle #5Merging (Combining)

2Quantity of substance

If data de-duplication is implemented in NAND memory systems, then storage efficiency improves, but write operations increase which accelerates memory degradation

Engineering Contradiction:

Improvestorage efficiencyVSAvoidNAND memory lifespan

Core Design Contradiction:

Quantity of substanceVSDuration of action of stationary object

Solution Approach 1:

The patent applies preliminary action by performing de-duplication detection during the write operation itself, before data is permanently committed to NAND memory. By using ECC check bytes to identify duplicate data blocks during the write process, the system can skip writing redundant data, thereby improving storage efficiency while preventing unnecessary write operations that would accelerate NAND memory degradation

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent enables self-service by allowing the ECC mechanism to automatically perform both error correction and de-duplication detection without requiring separate processing systems. The ECC check bytes inherently contain information about data blocks, and by comparing these check bytes, the system can autonomously identify duplicates and avoid redundant writes, extending NAND memory lifespan while maintaining storage efficiency

Inventive Principle:

Principle #25Self-service

3Quantity of substance

If software-based memory sharing is used in virtual machines, then memory capacity needs are reduced, but software overhead increases reducing performance

Engineering Contradiction:

Improvememory capacity utilizationVSAvoidsoftware performance

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The patent applies mechanics substitution by replacing software-based de-duplication mechanisms with hardware-level ECC check byte comparisons. Instead of using software processes to detect and manage duplicate memory blocks in virtual machines, the system leverages the hardware-implemented ECC mechanism to automatically identify and share memory blocks, eliminating software overhead and improving performance while still reducing memory capacity requirements through effective de-duplication

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS9229853B2Method and system for data de-duplication

Publication Date: 2016.01.05 INTEL CORP

AI summary

An apparatus may comprise a non-volatile random access memory to store data and a processor coupled to the non-volatile random access memory. The apparatus may further include a data de-duplication module operable on the processor to read a signature of incoming data, compare the signature to first data in the non-volatile random access memory, and flag the incoming data for discard when the signature indicates a match to the first data. Other embodiments are disclosed and claimed.