AI-Based Data Importance Scoring for Storage Zone Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems lack the ability to differentiate and prioritize data based on importance, leading to potential loss of critical files during storage medium failures, as they store all data indiscriminately, resulting in frustration and loss when important data is unrecoverable.
Innovation Solution
A system utilizing an AI model to generate a data file importance score based on content and metadata, which determines the storage zone for files, using a controller to store data in higher or regular reliability zones depending on the score, ensuring more critical data is stored in more reliable mediums.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data is stored indiscriminately in the same storage zone, then storage space utilization is maximized, but data loss risk increases when storage medium fails
Solution Approach 1:
The storage medium is divided into multiple storage zones with different reliability levels. The controller segments data files into important and non-important categories based on file extensions and metadata, then stores them in appropriate zones. This segmentation allows the system to protect critical data while maintaining efficient storage utilization across the entire medium.
Solution Approach 2:
Different regions of the storage medium are assigned different quality characteristics - some zones are designated for high-reliability storage of important data, while other zones can use standard storage. This local differentiation of storage quality allows the system to optimize reliability where needed without compromising overall storage efficiency.
2Reliability
If important data is stored in higher reliability storage zones, then data protection is improved, but storage cost increases
Solution Approach 1:
Instead of applying high-reliability storage to all data, the system applies it partially only to important data files identified by their extensions and metadata. This partial action approach ensures that critical data receives enhanced protection while non-critical data uses standard storage, optimizing the balance between protection and cost.
Solution Approach 2:
The system changes the storage parameter (reliability level) based on the data classification. By dynamically adjusting the storage zone assignment according to file importance parameters, the system achieves differentiated protection levels that match actual data value, avoiding unnecessary costs for non-critical files.
3Measurement precision
If data files are classified by importance using AI models, then data prioritization accuracy is improved, but processing time increases
Solution Approach 1:
The system performs preliminary classification of data files based on easily identifiable characteristics such as file extensions and metadata before storing them. This preliminary action allows the majority of files to be quickly categorized without requiring time-consuming AI analysis, while still achieving accurate classification for most common file types.
Solution Approach 2:
The system uses simple classification rules and metadata as a copy or approximation of the more complex AI-based importance assessment. By creating this simplified version of the classification process, the system achieves fast processing for routine files while maintaining the option to use full AI analysis when needed.
Data Source
AI summary
An example system and method may provide an importance score for a data file based on the content of the data file. An importance score may be used to determine whether to store the data file in a regular reliability storage media or in a higher reliability storage media. A controller generates a document vector for a data file based on content processed from a data file. The data file includes metadata and the content. The controller generates, using an artificial intelligence (AI) model and the document vector, a data file importance score for the data file. The controller then stores the data file in one of the first data storage zone and the second data storage zone based on the data file importance score.