Storage Data Reorganization for Heterogeneous Format Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data processing systems with heterogeneous data structures face inefficiencies in data access operations due to the complexity of handling disparate data formats, leading to degraded performance and increased power consumption.
Innovation Solution
A storage device with a data reorganizing unit that adapts the structure of data from a source format to a destination format compatible with the computing resource, enabling efficient data transfer by rearranging source addresses and converting data structures to match the requirements of the computing resource device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is stored and processed using heterogeneous data structures (e.g., AoS vs. SoA), then the storage device can maintain flexibility in data organization, but data access operations become complex and inefficient
Solution Approach 1:
A data reorganizing unit is introduced as an intermediary component between the storage medium and the processing device. This unit automatically converts data between different structures (e.g., Array of Structure to Structure of Array) based on the processing device's requirements, eliminating the need for complex host-level data reorganization while maintaining storage flexibility.
Solution Approach 2:
The data reorganizing unit performs data structure conversion in advance, before data is transferred to the processing device. By pre-converting data to the required format, the system avoids inefficient runtime conversions and ensures optimal data access performance without sacrificing storage adaptability.
2Productivity
If data reorganization is performed to match computing resource requirements, then data processing efficiency is improved, but device complexity increases
Solution Approach 1:
The data reorganizing unit is integrated within the storage device controller, merging data conversion functionality with existing storage management operations. This consolidation allows the system to perform data reorganization using existing hardware resources without adding significant structural complexity, while still achieving improved data processing efficiency.
3Extent of automation
If direct data communication between storage device and processing device is implemented, then host intervention is eliminated, but heterogeneous data structures cause performance degradation
Solution Approach 1:
The data reorganizing unit serves as an automated intermediary that handles data structure conversion during direct communication between the storage device and processing device. This eliminates the need for host intervention while simultaneously addressing the performance degradation caused by heterogeneous data structures through automatic format conversion.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
An electronic system includes; a host providing a command and source data having a source data structure, a computing resource device operating in relation to destination data having a destination data structure different from the source data structure, and a storage device. The storage device receives the command from the host and includes a storage medium storing the source data, and a data reorganizing unit converting the source data into reorganized source data, wherein the storage device outputs the reorganized source data to the computing resource device as destination data.