User Space File System Prefetching for Data Reading Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data reading methods are inefficient due to prolonged data transfer times and repeated reading of the same file from storage devices, especially when multiple data reading requests are made for the same file.
Innovation Solution
A data system and method that utilize a file system in user space to prefetch data of a target file into memory buffer blocks, dynamically adding and recycling these blocks based on reading requests to optimize data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is read directly from storage device for each reading request, then data reading operations can be performed, but data transfer time is prolonged and reading efficiency is reduced
Solution Approach 1:
The patent implements a prefetching mechanism that proactively loads data from the storage device into the memory buffer before actual reading requests are processed. The file system in user space monitors reading patterns and pre-fetches data blocks into the memory buffer, so that when reading requests arrive, the data is already available in fast memory, eliminating the time-consuming storage-to-memory transfer operation.
Solution Approach 2:
The patent introduces a memory buffer as an intermediary layer between the storage device and the file system. This buffer acts as a mediator that caches frequently accessed or pre-fetched data, allowing the file system to read from the fast memory buffer rather than directly from the slow storage device, thereby reducing data transfer time and improving reading speed.
2Productivity
If the same file data is read repeatedly from storage device for multiple reading requests, then all requests can be served, but reading time is increased due to repeated transfers
Solution Approach 1:
The patent merges multiple reading requests for the same or overlapping data ranges into a single prefetching operation. When the file system detects multiple reading requests targeting the same file or overlapping data blocks, it combines these requests and performs a single prefetch operation to load all required data into the memory buffer, eliminating redundant storage device accesses and reducing total reading time.
Solution Approach 2:
The system performs preliminary prefetching of file data into the memory buffer based on anticipated reading patterns. By analyzing reading requests and pre-loading data that is likely to be requested, the system ensures that subsequent reading operations can be served from the memory buffer without repeated storage device accesses, significantly reducing the time spent on repeated reading operations.
3Productivity
If memory buffer blocks are statically allocated, then memory management is simple, but data reading speed is limited by fixed buffer capacity
Solution Approach 1:
The patent implements dynamic buffer block management where the number and size of buffer blocks are adjusted based on actual reading patterns and system conditions. The file system in user space monitors the usage of buffer blocks and dynamically allocates or releases buffer blocks as needed, allowing the buffer capacity to adapt to varying data reading demands. This dynamic approach enables faster data reading when needed while maintaining simpler management during low-demand periods.
Data Source
AI summary
A data system and a data reading method are provided in the present disclosure. The data system includes a processor, a memory, and a storage device. The processor includes a file system in user space. The memory includes a memory buffer. The user file space system receives a reading request sent by an application end. The file system in user space reads prefetch data of a target file pre-stored in at least one buffer block of the memory buffer according to the reading request. In response to the file system in user space determining that a total reading amount of a last buffer block with stored data from the at least one buffer block being currently read exceeds a preset total amount, the file system in user space prefetches a next batch of prefetch data of the target file from the storage device and stores it in another buffer block. Therefore a data reading operation of high efficiency is achieved.