Storage Controller Entropy Analysis for Data Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data reduction techniques in storage systems often waste processing time and energy by attempting to reduce data units that are not viable for reduction, such as those with random or non-repeating patterns, leading to inefficient performance.
Innovation Solution
A storage system controller analyzes incoming data units for entropy values and only attempts reduction if they meet specific thresholds, using deduplication and compression logic to reduce viable data units while storing non-viable units in their original form, thereby optimizing processing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data reduction techniques are applied to all incoming data units, then data storage efficiency is improved, but processing time and energy consumption increase due to attempting reduction on non-viable data units
Solution Approach 1:
The patent applies preliminary action by calculating entropy values for incoming data units before attempting data reduction. The controller computes entropy metrics (such as Shannon entropy or max-min entropy) to assess the viability of reduction operations in advance. This preliminary entropy analysis allows the system to identify data units with repeating patterns that are suitable for reduction, while skipping data units with random patterns that would waste processing resources. By performing this preliminary assessment, the system avoids unnecessary reduction attempts on non-viable data, thereby reducing overall processing time while maintaining storage efficiency.
2Quantity of substance
If data reduction techniques are applied to all incoming data units, then data storage efficiency is improved, but energy consumption increases due to unproductive reduction attempts
Solution Approach 1:
The patent applies preliminary action by calculating entropy values for incoming data units before attempting data reduction. The controller computes entropy metrics (such as Shannon entropy or max-min entropy) to assess the viability of reduction operations in advance. This preliminary entropy analysis allows the system to identify data units with repeating patterns that are suitable for reduction, while skipping data units with random patterns that would waste processing resources. By performing this preliminary assessment, the system avoids unnecessary reduction attempts on non-viable data, thereby reducing overall processing time while maintaining storage efficiency.
3Measurement precision
If entropy analysis is performed on each data unit, then reduction viability is accurately determined, but processing complexity increases
Solution Approach 1:
The patent applies parameter changes by utilizing entropy values as a key parameter to determine data reduction viability. The controller calculates entropy metrics (such as Shannon entropy or max-min entropy) for incoming data units, transforming the complex problem of reduction viability assessment into a quantitative parameter-based decision. By changing the approach from qualitative judgment to quantitative entropy measurement, the system achieves accurate viability determination while managing processing complexity through established entropy calculation methods.
Data Source
AI summary
Example implementations relate to storing data in a storage system. An example includes receiving, by a storage controller of a storage system, a data unit to be stored in persistent storage of the storage system. The storage controller determines maximum and minimum entropy values for the received data unit. The storage controller determines, based on at least the minimum entropy value and the maximum entropy value, whether the received data unit is viable for data reduction. In response to a determination that the received data unit is viable for data reduction, The storage controller performs at least one reduction operation on the received data unit.