Data Storage Transformation System for Semi-Structured Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems are inefficient in handling semi-structured and unstructured data, requiring subsequent data transformation for processing, which extends processing time and is not optimally stored using common file-based storage systems.
Innovation Solution
An Information Handling System (IHS) with a data storage management engine that receives data, predicts processing operations, determines an optimal data format, transforms the data, and transmits it for storage, utilizing AI and ML techniques to classify data types and predict processing operations, thereby optimizing storage and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in common file-based storage subsystems close to where data is received, then storage speed is improved, but data processing efficiency deteriorates due to subsequent data transformation requirements
Solution Approach 1:
The system performs preliminary data format transformation during the storage process itself, rather than after retrieval. The storage subsystem automatically converts data to compute system-optimized formats before writing to storage, eliminating the need for subsequent transformation operations and enabling immediate processing upon retrieval.
Solution Approach 2:
An intermediary data format conversion layer is introduced within the storage subsystem that acts as a bridge between the incoming data format and the compute system's required format. This intermediary transformation capability is embedded in the storage path, allowing data to be stored in optimized formats without requiring post-retrieval conversion.
2Adaptability or versatility
If data is stored in common file-based storage subsystems, then storage compatibility is improved, but processing time increases due to required data transformation
Solution Approach 1:
The system performs preliminary data format transformation during the storage process itself, rather than after retrieval. The storage subsystem automatically converts data to compute system-optimized formats before writing to storage, eliminating the need for subsequent transformation operations and enabling immediate processing upon retrieval.
3Adaptability or versatility
If data transformation is performed after data retrieval, then storage flexibility is maintained, but overall system efficiency deteriorates
Solution Approach 1:
The system performs preliminary data format transformation during the storage process itself, rather than after retrieval. The storage subsystem automatically converts data to compute system-optimized formats before writing to storage, eliminating the need for subsequent transformation operations and enabling immediate processing upon retrieval.
Data Source
AI summary
A data storage transformation system includes a data storage management device that is coupled to a data provisioning device and a storage system. The data storage management device receives first format data that includes a first data format from the data provisioning device, and predicts at least one processing operation that will be performed on the first format data. The data storage management device then determines a second data format for performing the at least one processing operation, and transforms the first format data to second format data that includes the second data format. The data storage management device then transmits the second format data for storage in the storage system.