Storage-Assisted Data Compaction Using Memory Access Techniques
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data structures containing invalid data consume additional memory and storage space, and existing cleaning operations increase memory usage, IO operations, and power consumption.
Innovation Solution
Implement a memory access technique to load and merge valid data from storage device blocks into a more compact form, either at the host processor or storage device, reducing memory usage and IO operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a cleaning operation is performed to merge valid data from one or more portions of the data structure into a more compact form, then the space occupied by invalid data is reduced, but memory usage and IO operations increase
Solution Approach 1:
The storage device performs the cleaning operation autonomously using its own processing resources. The storage device receives information about valid and invalid data blocks, identifies merge opportunities, and executes the merging operation internally without requiring the host system to load all data into host memory. This self-service approach reduces the burden on host memory and IO operations while achieving compact data structure organization.
2Ease of operation
If valid data is loaded from storage device to host memory for processing, then data modification operations can be performed, but memory usage and power consumption increase
Solution Approach 1:
The cleaning operation is segmented into distinct phases: information gathering phase where the host sends metadata about valid and invalid blocks to the storage device, and execution phase where the storage device performs the actual merging operation using its internal resources. This segmentation allows data modification operations to be performed without loading entire data structures into host memory, thereby reducing power consumption while maintaining operational capability.
3Quantity of substance
If cleaning operation merges valid data into compact form, then storage space utilization is optimized, but IO operations increase
Solution Approach 1:
Metadata structures serve as intermediaries between the host system and the storage device's cleaning operation. The host sends compact metadata descriptions of valid and invalid blocks rather than the actual data blocks themselves. The storage device uses this metadata information to perform the merging operation, minimizing IO operations while achieving optimized storage space utilization. The metadata acts as a mediator that enables the cleaning operation without requiring extensive data movement.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method may include operating, by at least one processor, a data structure, receiving, by the at least one processor, from a storage device, using a memory access technique, information for a modification operation for the data structure, and performing, by the at least one processor, using the information, at least a portion of the modification operation. A system may include a storage device, and a host comprising at least one processor configured to operate a data structure, receive, from the storage device, using a memory access technique, information for a modification operation for the data structure, and perform, using the information, at least a portion of the modification operation. A device may include a storage medium, a memory, and a device controller configured to receive, using a memory access technique, information for a modification operation for a data structure and perform at least a portion of the modification operation.