AI Data Storage Tiering Using Vector Similarity for Lower Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems consume more power than necessary due to inefficient tiering, as data often remains in higher power tiers despite being accessed, which is not aligned with current computational needs.
Innovation Solution
A sustainable storage system that uses machine learning to tag and vectorize data based on context, clustering data types, and continuously updates the vector database to optimize storage tier assignment, ensuring data is stored in the most efficient tier based on similarity measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in higher power tiers for faster access, then data access speed is improved, but power consumption increases
Solution Approach 1:
The system dynamically adjusts data tier placement based on real-time access patterns and computational needs. Data is not statically assigned to tiers but is continuously repositioned according to usage metrics, allowing the system to optimize between speed and power consumption adaptively
Solution Approach 2:
The system changes the parameter of data tier assignment based on access statistics and computational context. By vectorizing data characteristics and comparing them against usage models, the system transforms static tiering into a dynamic parameter optimization problem that balances performance and energy efficiency
2Ease of operation
If data remains in higher tiers longer, then data accessibility is improved, but energy waste increases
Solution Approach 1:
The system implements continuous feedback loops that monitor data access patterns, computational needs, and power consumption metrics. This feedback drives automated tier adjustment decisions, ensuring data remains accessible when needed while being moved to lower-power tiers when not required, thereby eliminating energy waste from unnecessary high-tier retention
Solution Approach 2:
The system autonomously manages data tier placement without requiring manual intervention. The automated tiering mechanism continuously evaluates data usage patterns and self-adjusts tier assignments, enabling the system to serve itself in optimizing the balance between accessibility and energy efficiency
3Device complexity
If traditional tiering methods are used, then system simplicity is maintained, but power optimization capability is reduced
Solution Approach 1:
The system introduces an intermediary layer between raw data and storage tiers that handles the complexity of power optimization. This intermediary uses usage models and vector databases to translate complex power optimization requirements into automated tiering decisions, shielding users from complexity while achieving power efficiency
Data Source
AI summary
A method, computer system, and a computer program product for sustainable data storage is provided. A sustainable storage program receives raw data comprising data access statistics and power consumption of data currently in storage tiers. The received data is tagged by a usage model based on data context. The tagged data is vectorized, whereby the vectorizing includes clustering data types, and identifying a storage tier for each data type. The vectorized and tagged data is stored in a vector database. Incoming data is assigned to a storage tier based on a similarity search of the vector database, whereby the similarity measures the proximity or distance of two vectors in the vector database. The usage model and the vector database are continuously updated and monitored.