Storage Module Deduplication via Local Controller Decision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale storage systems formed by combining multiple storage modules, the performance degradation due to the need to read and transfer data between modules for deduplication leads to reduced read performance and throughput.
Innovation Solution
Implementing a system where each storage module determines whether data already exists in another module before writing or reading, thereby minimizing the need for inter-module data transfer and optimizing data placement to reduce other-system input/output operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is stored in a distributed manner across multiple storage modules to improve storage capacity and scalability, then storage scalability is improved, but read performance deteriorates due to the need for inter-module data transfer
Solution Approach 1:
The patent applies local quality by making each storage module autonomous in performing deduplication operations. Each module maintains its own deduplication database and can independently determine whether data already exists within its own storage, eliminating the need to transfer data requests across the network to other modules. This local decision-making capability resolves the contradiction by enabling both distributed storage and fast local read operations.
Solution Approach 2:
The patent segments the deduplication functionality from the storage modules, creating independent deduplication databases within each module. This segmentation allows each storage module to operate independently for deduplication decisions while maintaining distributed storage architecture, thus preserving both scalability and read performance.
2Loss of substance
If deduplication is performed across all storage modules to improve data compression ratio, then storage efficiency is improved, but system complexity increases due to inter-module coordination
Solution Approach 1:
The patent segments the deduplication system into independent modular units, where each storage module maintains its own deduplication database and performs deduplication operations autonomously. This segmentation eliminates the need for complex inter-module coordination while maintaining storage efficiency through local deduplication.
Solution Approach 2:
Each storage module performs self-service deduplication by maintaining its own deduplication database and independently determining whether data already exists within its module. This self-service approach eliminates the need for external coordination, reducing system complexity while preserving deduplication effectiveness.
3Loss of substance
If data is transferred between storage modules for deduplication to improve data deduplication ratio, then deduplication effectiveness is improved, but throughput decreases due to network transfer overhead
Solution Approach 1:
The patent enables each storage module to perform deduplication operations locally using its own deduplication database, eliminating network transfer overhead. This local deduplication approach maintains effective data compression while preserving high throughput by avoiding inter-module data transfers.
Solution Approach 2:
Each storage module independently performs deduplication checks using its own deduplication database, making self-service decisions without requiring network communication with other modules. This approach maintains high deduplication ratios while preserving system throughput.
Data Source
AI summary
In a large-scale storage system configured by combining a plurality of storage modules, it is possible to improve a read performance for deduplicated data. A large-scale storage system includes a first storage module and a second storage module each connected to a computing machine, the first storage module and the second storage module being connected to each other by a network, the first controller determines whether second data that is same as first data requested to be written is already stored in the second storage module when the first storage module receives a write request from the computing machine, and the first controller determines whether to store the first data in the first storage medium or to refer to the second data in the second storage module in a case in which the second data is already stored in the second storage module.