DHT Storage Node Prefetching via Local Buffer Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing data prefetching methods for distributed hash table (DHT) storage systems improve communication efficiency but fail to enhance data read efficiency due to the reliance on client-dependent prefetching operations, resulting in low data read efficiency in peer-to-peer storage systems.
Innovation Solution
A data prefetching method where storage nodes proactively identify and prefetch subsequent data blocks based on a threshold, storing them in local buffers to enable direct client access, thereby reducing network transmission time and improving access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is partitioned into several blocks and stored in several nodes in the system, then the system has high extendibility and can support massive data storage, but the client cannot consecutively read data from the system, resulting in low data read efficiency
Solution Approach 1:
The system performs preliminary actions by prefetching subsequent data blocks before they are actually requested by the client. When a client requests a data block, the system not only retrieves the requested block but also proactively fetches next sequential data blocks from the network and stores them in local buffers, so that when the client needs to read consecutive data, it can do so without additional network latency.
2Loss of time
If the client performs data prefetching operations, then communication efficiency from the client to each node is improved, but the data read efficiency of the system is still low because the prefetching depends on the client
Solution Approach 1:
The system inverts the traditional client-dependent prefetching approach by making the storage nodes perform prefetching operations autonomously. Instead of waiting for the client to initiate prefetching requests, the storage nodes automatically detect sequential access patterns and trigger prefetching of subsequent data blocks, thereby improving system-wide data read efficiency rather than just client-node communication efficiency.
3Loss of time
If subsequent data blocks are prefetched and stored in local buffers, then network transmission time is reduced, but system resources are consumed for buffer storage and data management
Solution Approach 1:
The system applies partial prefetching by using a threshold mechanism. When a data block is accessed, the system determines whether to prefetch subsequent blocks based on whether the access count meets a predefined threshold. This selective approach prefetches only when beneficial, avoiding unnecessary network transmissions and buffer allocations for accesses that wouldn't benefit from prefetching, thus balancing resource consumption with performance improvement.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Embodiments of the present invention provide a data prefetching method for a distributed hash table DHT storage system, a node, and a system. The method includes the following: A first storage node receives a read request sent by a client, determines a to-be-prefetched data block and a second storage node where the to-be-prefetched data block resides according to a read data block and a set to-be-prefetched data block threshold, and sends a prefetching request to the second storage node, in which the first storage node and the second storage node are the same or different storage nodes, the prefetching request includes identification information of the to-be-prefetched data block, and the identification information is used to uniquely identify the to-be-prefetched data block; and the second storage node reads the to-be-prefetched data block from a disk according to the prefetching request, and stores the to-be-prefetched data block in a local buffer, so that the client reads the to-be-prefetched data block from the local buffer of the second storage node. In the embodiments of the present invention, the to-be-prefetched data block can be directly read from the buffer, thereby improving the access efficiency of data in each storage node in the system.