File Segmentation and Metadata for Scalable Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in efficiently managing large volumes of electronic data with extended retention periods, requiring cost-effective, energy-efficient solutions that are resilient to partial failures and allow for incremental growth and data migration while maintaining accessibility.
Innovation Solution
A storage system that breaks files into segments, deduplicates them across storage units, and uses metadata to enable access independently of storage units, allowing for incremental capacity expansion, energy conservation, and fault tolerance by transferring segments and metadata between units based on usage and capacity thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a single large storage system, then accessibility and management are simplified, but the system becomes vulnerable to partial failures and harder to expand incrementally
Solution Approach 1:
The patent divides the storage system into multiple independent storage units, each capable of storing segments of files. This segmentation allows the system to maintain reliability through distribution while managing complexity through standardized interfaces. Each storage unit operates independently, so failure of one unit does not compromise the entire system.
2Quantity of substance
If storage capacity is expanded by adding more storage units, then capacity grows, but data migration and system upgrades become more complex
Solution Approach 1:
The patent creates universal storage units with standardized interfaces and capabilities that can perform multiple functions. Each storage unit can store file segments, metadata, and can be independently accessed. This universality allows seamless expansion by adding identical units without complex integration, and enables easy migration by moving segments between equivalent units.
Solution Approach 2:
The system employs dynamic segment migration capabilities where file segments can be moved between storage units based on capacity needs, performance requirements, or maintenance schedules. This dynamic approach allows the system to adapt to changing storage requirements without rigid structural constraints, facilitating incremental expansion and simplified upgrades.
3Ease of operation
If all storage units remain powered on for immediate access, then data accessibility is maximized, but energy consumption increases
Solution Approach 1:
The patent implements periodic access patterns where storage units are powered on only when needed based on access requirements. The system can determine which storage units contain actively accessed data and keep only those powered on, while powering down units with inactive data. This periodic activation strategy maintains accessibility for needed data while significantly reducing overall energy consumption compared to keeping all units continuously powered.
Data Source
AI summary
A system for storing files comprises a processor and a memory. The processor is configured to break a file into one or more segments; store the one or more segments in a first storage unit; and add metadata to the first storage unit so that the file can be accessed independent of a second storage unit, wherein a single namespace enables access for files stored in the first storage unit and the second storage unit. The memory is coupled to the processor and configured to provide the processor with instructions.