Cache Memory Chunking for Large Data Object I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face inefficiencies and reliability issues when handling large data objects, as they often require reading or writing entire data objects to cache memory, leading to increased latency and potential system crashes due to cache memory limitations.
Innovation Solution
A system that includes a non-volatile cache memory device and a storage controller, which divides large data objects into smaller chunks to manage input/output operations, allowing for efficient storage and retrieval by writing or reading only the necessary data chunks, thereby optimizing cache memory usage and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire data object is read to the storage buffer for read requests, then the data can be accessed in cache memory, but the read latency increases and cache memory is wasted when only a subset of data is needed
Solution Approach 1:
The patent divides large data objects into smaller chunks or blocks that can be independently managed in cache memory. When a read request arrives, only the specific chunk containing the requested data is read to the storage buffer, rather than reading the entire data object. This segmentation enables precise data retrieval, reducing read latency and avoiding the waste of reading unnecessary data while maintaining reliable data access through proper chunk management and tracking mechanisms.
2Reliability
If the entire data object is read and written to long-term storage for read-write requests, then data modification can be performed, but the operation time increases significantly when only a subset of data needs modification
Solution Approach 1:
The patent segments data objects into manageable chunks that can be independently loaded, modified, and written back to storage. When a read-write request targets a specific portion of data, only the relevant chunk is read to the storage buffer, modified in place, and then written back to long-term storage. This eliminates the need to read and write entire data objects, significantly reducing modification operation duration while maintaining data integrity through proper chunk tracking and management.
Solution Approach 2:
The patent extracts only the necessary portion of data from the larger data object and loads it to the storage buffer for modification. Instead of bringing the entire data object into the buffer, the system identifies and extracts only the specific chunk containing the data to be modified, performs the modification operation, and writes back only that chunk. This extraction approach minimizes I/O operations and buffer usage while ensuring reliable data modification.
3Reliability
If large data objects are written to the storage buffer in their entirety, then the data can be stored in cache, but the cache memory size requirements increase and system reliability decreases when data object size exceeds cache capacity
Solution Approach 1:
The patent divides large data objects into smaller chunks that fit within the available cache memory capacity. Each chunk can be independently managed, loaded, and stored in the storage buffer as needed. This segmentation allows the system to handle data objects larger than the cache memory by processing them in manageable pieces, eliminating the need for excessively large cache memory while maintaining system reliability through proper chunk tracking, metadata management, and recovery mechanisms.
Data Source
AI summary
Various aspects for managing input/output (I/O) operations for data objects (e.g., large data objects (LOBs)), in a storage system are provided. For instance, a method may include receiving an I/O request for executing an I/O operation on a data object, determining the size of the data object, and determining the data object to be a non-conforming data object relative to a storage threshold of a cache memory device. The method may further include dividing the non-conforming data object into multiple data chunks, each of the data chunks having a size that is less than the storage threshold, moving one or more data chunks stored in one or more memory spaces of the cache memory device to a long-term memory device to free up the one or more memory spaces, and storing one or more of the data chunks of the non-conforming data object in the freed-up memory space(s).