A multi-ndp device read-write acceleration method and system based on hierarchical learning index
By introducing hierarchical learning indexes and write hotness recognition mechanisms into the multi-NDP architecture, the read and write performance of the key-value storage system is optimized, the read performance bottleneck in the multi-NDP environment is solved, efficient data location and cache management are achieved, and the overall performance and storage efficiency of the system are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANHUI UNIV
- Filing Date
- 2026-02-24
- Publication Date
- 2026-07-14
AI Technical Summary
In a multi-NDP architecture, the read performance of a key-value storage system based on LSM-tree is limited by the random read and probe overhead caused by data sharding, making it difficult to meet the needs of large-scale data processing.
A hierarchical learning index mechanism is introduced, which identifies write frequency through LRU caching, separates data into hot value log areas or cold value log areas, and builds a hierarchical learning index with a hybrid memory and flash memory on the NDP device side. Combined with multi-level cache collaborative management, the efficiency of read path location is optimized.
It significantly reduces the overhead of random reads and probes for point queries, improves read performance, reduces write amplification, increases cache hit rate and garbage collection efficiency, and optimizes overall system performance.
Smart Images
Figure CN122387376A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of key-value storage accelerated read and write technology, and in particular to a method and system for accelerating read and write of multiple NDP devices based on hierarchical learning index. Background Technology
[0002] With the rapid growth of global data volume, especially with unstructured data gradually becoming dominant, how to efficiently store and process massive amounts of data has become a core issue in the field of information technology. According to predictions from the International Data Corporation (IDC), the global data scale will increase significantly in the coming years. How to address this challenge, especially in data-intensive application scenarios, has become a focal point for both academia and industry. Traditional relational databases, due to their complex storage structures and performance bottlenecks, struggle to meet the demands of unstructured data processing. Therefore, key-value storage systems based on LSM-tree (LogStructuredMergeTree) have become mainstream solutions, such as LevelDB, RocksDB, and Cassandra, which are widely used in big data scenarios due to their high write performance and flexible scalability.
[0003] However, as data volume and read request volume continue to grow, the read / write performance of LSM-tree-based key-value storage systems is constrained by two factors: data transport overhead and location overhead. To reduce end-to-end latency caused by frequent transmissions between the host and storage, researchers have introduced the Near Data Processing (NDP) architecture, offloading some computation to the vicinity of data storage. Related work has applied NDP to key-value storage systems (such as PStore, nKV, and ProckStore), effectively reducing data transmission between storage devices and the CPU, thereby significantly reducing data transport overhead and effectively optimizing compaction performance. However, most of these optimization schemes are based on a single NDP, which has limited computational power. Therefore, researchers have further proposed multi-NDP architectures for large-scale key-value storage systems to expand capacity and parallel computing capabilities (such as MStore and JMSStore). However, the aforementioned proximity computation at the architectural level cannot automatically resolve the core localization bottleneck of LSM-tree read paths: LSM-tree-based key-value stores typically employ a hierarchical localization approach of "range metadata + Bloom filter + block-level sparse index." Bloom filters can only provide negative filtering, sparse indexes often only locate data blocks, and point queries still require cross-layer / cross-file verification and trigger multiple random reads. In a multi-NDP environment, data sharding further disperses data, making the localization process more prone to additional probing and random read overhead, thus increasing read latency and becoming a key factor limiting read performance improvements in multi-NDP systems.
[0004] Therefore, a new index organization mechanism is needed that can provide more accurate address positioning on the device side, and a hierarchical learning index is introduced to reduce the random reading and probing overhead of point queries. Summary of the Invention
[0005] To address the aforementioned issues, this invention discloses a method and system for accelerating read and write operations on multiple NDP devices based on hierarchical learning indexes. By introducing a hierarchical learning index mechanism, the read path positioning efficiency of key-value storage systems in multi-NDP environments is optimized, and the random reading and probing overhead during point query processes is reduced, thereby improving overall read performance.
[0006] The objective of this invention can be achieved through the following technical solution: a method for accelerating read and write operations of multiple NDP devices based on hierarchical learning indexes, comprising:
[0007] On the host side, write key-value pairs are identified based on LRU cache and marked as hot write data or cold write data.
[0008] The key-value pair data is distributed to the corresponding NDP device, and key-value separation is performed on the NDP device. The value data is written to the hot value log area or cold value log area according to the write hotness mark.
[0009] A hierarchical learning index is built and maintained on the NDP device side to store the mapping relationship between key and value addresses;
[0010] In response to a read request, the value address of the target key is located by querying the hierarchical learning index, and the query is performed in the multi-level cache collaborative management system;
[0011] The hierarchical learning index collects invalid addresses during data updates, maintains the proportion of invalid data at the value log file granularity, and triggers a garbage collection (GC) task when the proportion reaches a threshold. The background thread executes the rewriting of valid data and index updates in parallel.
[0012] As a further embodiment of the present invention, the write heat identification includes the following steps:
[0013] Maintain a fixed-size LRU key cache on the host side to record recently written keys and their update counts;
[0014] When a key exists in the LRU cache and the number of updates exceeds a preset threshold, the corresponding key-value pair is marked as hot write data; otherwise, it is marked as cold write data.
[0015] The hot and cold tags are downloaded to the NDP device along with the key-value pair data.
[0016] As a further embodiment of the present invention, the hierarchical learning index adopts a hybrid memory and flash storage structure, including an incremental index, a segment mapping table and a segment model table residing in memory, and leaf nodes stored in flash memory.
[0017] As a further embodiment of the present invention, the NDP device-side hierarchical learning index query step includes:
[0018] Determine the data segment to which the query key belongs and the corresponding segment model table based on the segment mapping table;
[0019] The linear model in the segment model table is used to predict the position of the query key in the leaf node, and the candidate search interval is determined by combining the error boundary.
[0020] The leaf node data pages corresponding to the candidate interval are read from flash memory into memory, and a binary search is performed to obtain the precise value address.
[0021] As a further embodiment of the present invention, the step of maintaining the hierarchical learning index includes:
[0022] Maintain an incremental index in memory to cache the addresses of recently written or updated key-value pairs;
[0023] When the incremental index data volume of a certain data segment reaches the threshold, segment merging and model retraining are triggered.
[0024] The host machine performs model training, generates a new version of the model table, and sends it to the NDP device.
[0025] The NDP device updates the segment model table version to ensure query consistency.
[0026] As a further embodiment of the present invention, the multi-level cache collaborative management forms a read heat identification mechanism, the steps of which include:
[0027] Using the data segments of the hierarchical learning index as the granularity, the access frequency, recent access time, and average operation time of read requests are statistically analyzed to calculate segment popularity and identify hot segments.
[0028] Maintain a Top-K hotkey table within the hotspot segment, recording the most frequently accessed keys and their addresses within the segment;
[0029] Calculate a migration benefit score for the keys in the Top-K hotkey table. This score takes into account access popularity, cross-device access cost, migration cost, and cache hit benefit.
[0030] The migration benefit score drives the admission, replacement, and cross-layer migration decisions of key-value data between the host-side cache and the NDP device-side cache.
[0031] As a further embodiment of the present invention, in the multi-level cache collaborative management, the NDP device performs a hierarchical learning index query to obtain the key-value pair address, and searches in the host cache and device cache in sequence according to the hierarchy. If the host cache is hit, it is returned directly; if the host cache is not hit, the device cache is queried, and if it is hit, it is also returned directly; if neither cache is hit, the value log file of the target NDP device is located and read according to the address.
[0032] During the query process, the segment heat container and the Top-K table of hot segments are updated synchronously. If the key is a hot key, the migration benefit score is calculated / updated based on its final service location, and the admission, replacement and cross-layer migration (including promotion / demotion) in the host and device cache are determined accordingly.
[0033] As a further embodiment of the present invention, the waste recycling step includes:
[0034] Based on the invalid addresses recorded during the hierarchical learning index update process, the proportion of invalid data in each value log file is maintained in real time;
[0035] When the proportion of invalid data in the value log file reaches a threshold, it is added to the background garbage collection task queue, and the following operations are performed in parallel by multiple background threads:
[0036] The target value log file is read sequentially to verify the validity of key-value pairs. Valid data is rewritten to the new value log file according to its original hot and cold attributes, and the address mapping in the hierarchical learning index is updated.
[0037] A multi-NDP device read / write acceleration system, comprising:
[0038] The host-side write processing module is configured to perform write activity identification and data distribution.
[0039] Multiple NDP devices, each NDP device includes:
[0040] The key-value separation module is used to write value data to the hot value log area or the cold value log area based on the write popularity flag.
[0041] The hierarchical learning index module is used to build and maintain a mapping index from key to value address;
[0042] The multi-level cache management module is configured to manage the collaborative work between the host-side cache and the caches of each NDP device, including heat identification and data migration.
[0043] As a further aspect of the present invention, it also includes:
[0044] The garbage collection module is configured to trigger and execute background garbage collection tasks based on hierarchical learning index information.
[0045] The beneficial effects of this invention are:
[0046] 1. This invention introduces a write hotness identification mechanism based on a fixed-size LRU key cache on the host side. Key-value pairs are marked as hot or cold writes according to the recent write frequency and are flushed to the device side with the data. When the NDP device separates the key and value, it writes the values to the hot value log area and the cold value log area respectively according to the mark, so that the invalid data of the frequently updated data is more concentrated, thereby reducing write amplification and improving the space reclamation efficiency of subsequent garbage collection.
[0047] 2. This invention proposes a hierarchical learning index structure with hybrid on-device memory and flash memory residing in NDP (Network DP), consisting of an incremental index, a segment mapping table, a segment model table, and leaf nodes. During a query, the device first performs linear model prediction on the target key based on the segment mapping table and the segment model table, and then converges the candidate positions into a continuous leaf node interval by combining the model error bound. Subsequently, only a small number of data pages within the leaf nodes corresponding to this small range need to be read, and the precise key-value pair address is obtained within the read data pages through binary search. By transforming "random reads of global multi-layer probing" into "small-range, approximately sequential reads guided by prediction," this invention significantly reduces the number of random flash reads triggered by point queries and lowers tail latency.
[0048] 3. This invention constructs a master-slave multi-level collaborative caching system consisting of host-side caching and multiple NDP device-side caching. The system uses segments maintained by the hierarchical learning index on the NDP devices as the basic statistical granularity to perform online statistics on read requests. At the end of the observation window, segment popularity is calculated and sorted to select hot segments. To reduce the overhead of popularity filtering and maintenance, a Top-K hot key table is maintained only for segments identified as hot, storing the set of most frequently accessed keys within the segment. Furthermore, the system calculates migration benefit scores for candidate hot key-value pairs. This score comprehensively considers data access popularity, the time cost of cross-device access, data migration costs, and the potential benefits brought by hits at different cache levels. Based on this, the system uniformly drives cache admission, replacement, and cross-level migration decisions, ensuring that high-benefit hot data resides preferentially at more suitable cache levels, thereby improving the overall hit rate and significantly reducing tail latency.
[0049] 4. This invention leverages the characteristic of hierarchical learning indexes to obtain "new and old address replacement" information during update paths: Each time an index entry is updated, the replaced old address is synchronously recorded as invalid, and the proportion of invalid data is maintained at the value log file level, thus forming real-time, fine-grained tracking of garbage data distribution. When the proportion of invalid data in a certain value log file reaches a threshold, the system accurately selects objects to be reclaimed and submits a GC task. The GC task is executed in parallel by multiple threads in the background, performing validity verification, rewriting valid values to new hot / cold value logs, and updating index addresses, improving reclamation throughput and reducing the impact on foreground read / write requests. Attached Figure Description
[0050] Figure 1 Write a principle flowchart for the method of this invention;
[0051] Figure 2 This is a flowchart illustrating the principle of hierarchical learning indexing on the NDP device side in this invention.
[0052] Figure 3 This is a flowchart illustrating the incremental indexing principle of the method of the present invention;
[0053] Figure 4 This is a flowchart illustrating the principle of the reading process of the method of the present invention;
[0054] Figure 5 This is a flowchart illustrating the principle of the multi-NDP collaborative caching system of the present invention.
[0055] Figure 6 This is a flowchart illustrating the waste recycling principle of the method of the present invention. Detailed Implementation
[0056] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0057] Example 1:
[0058] This embodiment discloses a multi-NDP device read / write acceleration system based on hierarchical learning index. The system adopts a distributed collaborative architecture of host and multiple NDP devices. Through the division of labor and cooperation of each functional module, the read / write performance of the key-value storage system is improved, write amplification is reduced, and garbage collection efficiency is optimized.
[0059] The system includes a host-side write processing module, as well as a key-value separation module, a hierarchical learning index module, a multi-level cache management module, and a garbage collection module configured on each NDP device. Each module completes the entire process of read and write acceleration and storage optimization through data interaction and command coordination.
[0060] The host-side write processing module is responsible for receiving write requests, identifying write activity, and distributing data in a targeted manner, laying the foundation for subsequent value log partitioning, index building, and cache collaboration.
[0061] Write frequency identification: An identification mechanism is built based on a fixed-size LRU key cache. When processing a key-value pair write request, it is determined whether the current key already exists in the LRU key cache and whether its update count is greater than a preset threshold (default value is 2). If both conditions are met, the key-value pair is marked as hot write data, and the corresponding flag bit is set to 1; otherwise, it is marked as cold write data, and the corresponding flag bit is set to 0, thus achieving accurate classification of hot and cold data based on write frequency.
[0062] Data distribution: A hash operation is performed on the key-value pairs that have completed cold and hot tagging. The target NDP device for storage is determined based on the hash result. The key-value pairs are then written into the MemTable (organized in a skip list) in the buffer of that NDP device. When the number of key-value pairs accumulated in the MemTable of an NDP device reaches a predetermined threshold, a Flush operation is triggered, converting the MemTable into an immutable MemTable and flushing the ordered key-value pairs to the corresponding NDP device, completing the directed data transfer from the host to the device.
[0063] The NDP device-side modules include a key-value separation module, a hierarchical learning index module, and a multi-level cache management module.
[0064] Key-value separation module: Receives ordered key-value pairs with hot and cold tags from the host, performs key-value separation processing, and implements partitioned storage of value data.
[0065] The key-value separation module parses the hot and cold flag bits in the key-value pair records and writes the value data accordingly. If the flag bit is 1 (hot write data), the corresponding value is written to the hot value log area on the NDP device; if the flag bit is 0 (cold write data), it is written to the cold value log area. At the same time, the "key → value address" mapping relationship obtained after key-value separation is synchronously output to the hierarchical learning index module for index update and maintenance.
[0066] Hierarchical Learning Index Module: Constructs and maintains a high-precision mapping index of "key → value address" to achieve fast location for point queries, reduce random read overhead, and lower tail latency. It adopts a hierarchical structure with hybrid "memory + flash" resident architecture, specifically including:
[0067] Incremental index: Residing in the memory of the NDP device, it adopts a segmented B+ tree structure to cache the addresses of recently written or updated key-value pairs, avoiding random rewriting of leaf nodes in flash memory with each write, thus improving write efficiency.
[0068] Segment mapping table: Residing in the memory of the NDP device, it divides the full key space into multiple independent data segments according to the value range of the key, and maintains a one-to-one correspondence between "segment → segment model table", allocating an independent model table and lookup boundary for each segment.
[0069] Segment Model Table: Residing in the memory of the NDP device, it stores a set of piecewise linear models trained based on all leaf node data within the corresponding segment. Each model entry contains the starting key, linear model slope, intercept, error boundary, and corresponding leaf node address, which is used to map the target key to an approximate position in the leaf node and limit the search range.
[0070] Leaf nodes: Stored in flash memory, they are arranged in order of key and corresponding value address, and are the final data carrier for precise address lookup.
[0071] Query process: First, locate the data segment to which the target key belongs and the corresponding segment model table through the segment mapping table. Use the linear model in the segment model table to predict the location of the target key. Combine the convergence of the error boundary to obtain a continuous leaf node search interval. Then, read a small number of leaf node data pages corresponding to this interval and obtain the precise key-value pair address in memory through binary search, transforming "global multi-level random read" into "small-range approximate sequential read".
[0072] Maintenance process: When the accumulated data in a certain incremental index reaches a preset threshold, an intra-segment maintenance operation is triggered. The incremental index and the corresponding leaf node data are merged in batches, and training information such as intra-segment data, error statistics, and popularity statistics are extracted and uploaded to the host along with the segment metadata. The host executes model training based on the training information and allocates differentiated error boundaries according to segment popularity, generating a new version of the segment model table and sending it to the NDP device. The device adopts version management and completes the atomic replacement of the segment model table through atomic pointer switching. During the replacement, the old version is retained to handle online requests and ensure query consistency.
[0073] Multi-level cache management module: Constructs a master-slave multi-level cache system that coordinates host-side cache and multi-NDP device-side cache. Through hot identification and intelligent migration, it improves cache hit rate and reduces cross-device access and flash random read overhead.
[0074] Read popularity identification unit: Using the "segment" of the hierarchical learning index as the basic statistical granularity, a segment popularity container is maintained for each segment, recording the segment's read count, recent access time, and average operation time. At the end of each observation window, segment popularity is calculated based on the three-dimensional indicators of "frequency-time-cost": read count reflects activity level, recent access time reflects novelty through monotonically decreasing time weights such as exponential decay, and average operation time reflects access cost (the higher the cost, the greater the weight). Based on this, a set of "high-frequency and high-cost" hot segments is selected.
[0075] Top-K Hotkey Maintenance Unit: Maintains the Top-K hotkey table only within the hotspot segment, storing the K most frequently accessed keys within the segment and their address information. The maintenance rule is "update the count if there is a hit, insert if there is no hit and the table is not full, and replace the smallest count item if there is no hit and the table is full," accurately locking key-level hotspot data while controlling memory overhead.
[0076] Migration Benefit Evaluation Unit: Calculates migration benefit scores only for hot keys in the Top-K hot key table. This score comprehensively considers data access frequency, cross-device access time cost, data migration cost, and potential benefits from different cache levels, quantifying migration value using a "benefit-cost" model. Cache Collaboration Unit: Drives cache admission, replacement, and cross-layer migration decisions based on migration benefit scores. It prioritizes high-benefit hot data for resident in host-side caches or NDP device-side caches closer to the accessing end, achieving hierarchical placement and dynamic migration of hot data. Simultaneously, it feeds back the popularity identification results to the hierarchical learning index module, allocating lower training error boundaries to hot segments and improving the query and location accuracy of hot data.
[0077] When a query request is received, the key-value pair address is first obtained through the hierarchical learning index, and then the query is performed in the order of "host-side cache → target NDP device-side cache → value log". During the search, the segment heat container and the Top-K hot key table are updated synchronously. If the query key is a hot key, its migration benefit score is updated according to the final service location (host hit, device hit or value log read) and the cache placement strategy is dynamically adjusted.
[0078] Garbage collection module: Based on the address tracking capability of hierarchical learning index, it achieves accurate identification and efficient reclamation of invalid data, reduces garbage collection overhead, frees up storage space, and ensures system storage efficiency. Specifically, it includes:
[0079] Invalid Data Tracking Unit: Utilizing the characteristic of obtaining "new and old address replacement" information during the hierarchical learning index update process, it synchronously records the replaced old key-value pair addresses as invalid addresses, and maintains the proportion of invalid data in real time at the granularity of value log files, thereby achieving fine-grained tracking of the distribution of garbage data.
[0080] Garbage collection trigger unit: When the proportion of invalid data in a certain value log file reaches a preset threshold, garbage collection is automatically triggered, and the value log file is added to the background GC task queue.
[0081] The background parallel data reclamation unit employs multi-threaded parallel execution of reclamation tasks. First, it determines the reclamation timing based on the current system load (waiting for idle periods when load is high) to avoid impacting foreground read / write requests. During reclamation, it sequentially reads the target value log file and verifies the validity of each key-value pair using a hierarchical learning index. Valid key-value pairs are rewritten to new hot or cold value log files based on the original partition attributes (hot / cold). After reclamation, the new key-value pair addresses are updated to the hierarchical learning index, while simultaneously releasing the storage space occupied by the original log files. By separating hot and cold data for storage, invalid data is centrally distributed, significantly improving reclamation efficiency and space utilization.
[0082] Example 2:
[0083] Applying the system architecture of Example 1, this example discloses a method for accelerating read / write operations of multiple NDP devices based on hierarchical learning indexes, including the following steps:
[0084] On the host side, write key-value pairs are identified based on LRU cache for write hotness and marked as hot write data or cold write data.
[0085] After user data is written to the batch processing area, the host first inserts the key-value pairs to be written into the host buffer, and introduces a write hotness identification module on the write path to classify the key-value pairs as hot or cold, in order to support subsequent partition placement and write amplification control.
[0086] The write hotness identification module is implemented based on a fixed-size LRU key cache on the host side to identify write hotness:
[0087] When processing a write operation, the system determines whether the current key already exists in the write LRU cache and whether its update count is greater than a preset threshold (default is 2).
[0088] If the above conditions are met, the key-value pair is marked as frequently updated "hot write" data; otherwise, it is marked as "cold write" data with a lower update frequency. Subsequently, the system updates the hot / cold flag bits in the key-value pair record: a flag bit of 1 indicates a hot write, and a flag bit of 0 indicates a cold write.
[0089] like Figure 1 As shown, after completing the hot and cold tagging, the host performs a hash operation on the key to determine the target NDP device for storage, and writes the key-value pairs into the MemTable in the corresponding NDP device's buffer, organizing them in a skip list manner. For each NDP device, when the number of key-value pairs accumulated in its MemTable reaches a predetermined threshold, the host converts the MemTable into an immutable MemTable and triggers a Flush operation, flushing the ordered key-value pair data in the immutable MemTable to the corresponding NDP device.
[0090] The key-value pair data is distributed to the corresponding NDP device, and key-value separation is performed on the NDP device. The value data is written to the hot value log area or cold value log area according to the write hotness mark.
[0091] like Figure 1 As shown, after the refreshed data arrives at the NDP device, the NDP device performs key-value separation on the ordered key-value pairs: it parses the hot and cold flag bits in the records and writes the values to different value log areas according to the hot and cold attributes. Specifically, if the flag bit is 1, the value is written to the hot value log area; if the flag bit is 0, the value is written to the cold value log area.
[0092] While writing value data to the hot value log area or cold value log area respectively, the key-value pair addresses after key-value separation are inserted into the hierarchical learning index. During the insertion process, the structure and model of the hierarchical learning index are adaptively maintained according to the insertion results, including triggering internal merging and model retraining, to ensure a balance between index accuracy and structure. Simultaneously, during the generation and updating of the hierarchical learning index, the system collects invalid key-value pair addresses and maintains information on the proportion of invalid data in the value log file.
[0093] Specifically, a hierarchical learning index is built and maintained on the NDP device side to store the mapping relationship between key and value addresses.
[0094] On the NDP device side, the hierarchical learning index adopts a hierarchical structure with "memory + flash" hybrid storage, and consists of four parts: the incremental index residing in NDP memory, the segment mapping table, the segment model table, and the leaf nodes stored in flash memory.
[0095] The segment mapping table is used to segment the full key space by range: it divides the data into multiple segments according to the value range of the key and maintains the correspondence between "segment → segment model table", so that each segment has an independent model table and lookup boundaries.
[0096] For any given segment, the segment model table stores a set of piecewise linear models trained based on all leaf node data within that segment.
[0097] Each entry in the segment model table contains: the starting key, the linear model slope, the linear model intercept, the error boundary, and the address of the corresponding leaf node. These are used to map the key to an approximate position in the leaf node and to limit the search range.
[0098] Leaf nodes reside in flash memory and are used to store keys and their corresponding value addresses arranged in order of key.
[0099] like Figure 2As shown, during the query process, the system first locates the target segment and its segment model table based on the segment mapping table; then it queries the segment model table to obtain the linear model used in the region where the query key is located, and then uses the linear model to predict the location of the target key, and calculates the candidate search interval within the leaf node based on the error boundary; finally, it reads the leaf node data page of the interval into memory to perform a binary search to obtain the accurate key-value pair address, thereby reducing the search cost while ensuring correctness.
[0100] This invention sets up an incremental index in NDP memory to cache the addresses of newly generated key-value pairs or updated addresses from recent writes, avoiding random rewriting of leaf nodes in flash memory with each write. The incremental index adopts a segmented B+ tree structure: when the accumulated incremental data in a segment reaches a preset threshold, the NDP device triggers an intra-segment maintenance process.
[0101] like Figure 3 As shown, the process includes: batch merging the incremental index and internal leaf node data of the segment, generating merged segment metadata, and extracting training information for model training (including intra-segment data, error statistics, and popularity statistics). The device uploads the training information and segment metadata to the host, which then performs model training calculations for the segment, allocates training error boundaries based on segment popularity, generates a new version of the segment model table, and sends it to the device. After receiving the new version of the segment model table, the NDP device atomically replaces the local segment model table in a versioned manner. To ensure consistency of online queries during model updates, this invention adopts versioned management for the segment model table: the new segment model table sent by the host carries a version number; after the NDP device completes the writing and verification of the new version of the segment model table in the background, it completes the version replacement through atomic pointer switching. During the replacement, the device retains the old version of the segment model table for processing requests; it is then recycled after confirming that the old version has no related references, thereby ensuring that model updates do not interrupt or cause errors in front-end read / write requests.
[0102] The processing of the read request includes: performing a hash operation on the target key to determine the target NDP device to which it belongs; sequentially querying the MemTable and the immutable MemTable of the target NDP device, and if a match is found, returning the corresponding value data; if no match is found, entering the read path processing flow based on hierarchical learning index and multi-level cache collaborative management.
[0103] In response to a read request, such as Figure 4 As shown, the value address of the target key is located by hierarchical learning index query and then queried in the multi-level cache collaborative management system.
[0104] A multi-level collaborative caching module is used, which consists of host-side cache and multiple NDP device-side caches to form a multi-level cache structure, thereby reducing cross-device access and flash random read overhead in multiple NDP key-value systems.
[0105] To enable multi-level caching to more effectively handle high-return, frequently accessed data, a read-hotness identification mechanism is integrated into the collaborative caching module. This mechanism classifies data access into different levels and uses the identification results for two key decision-making processes:
[0106] (1) Drive the admission, replacement and cross-layer migration (including promotion / demotion) of cache on the host side and device side.
[0107] (2) Guide the allocation of error boundaries during segment merging and retraining of the hierarchical learning index, so that hotspot segments can obtain more accurate positioning capabilities, thereby further reducing the random flash reading and tail latency of point queries.
[0108] The read popularity identification module uses segments of the hierarchical learning index on the NDP device as the main statistical granularity. The system maintains a segment popularity container for each segment, recording read counts, recent access time, and average operation time.
[0109] like Figure 5 As shown, at the end of each observation window, the system calculates the segment read heat based on three types of information: frequency, time, and cost, and completes the segment-level sorting: the read count reflects the activity level within the window; the recent access time reflects the recentity through a monotonically decreasing time weight (such as exponential decay); the average operation time indicates the impact of the segment access on the read latency. When the average operation time of a segment is higher, it indicates that it is more likely to dominate the overall tail latency. Therefore, its cost weight is increased in the heat calculation to prioritize the identification of "high-frequency and high-cost" hot segments.
[0110] Based on this, the system selects the top-ranked segments to form a hotspot segment set. When performing intra-segment merging and retraining during hierarchical learning indexing, a lower training error boundary is configured for hotspot segments to reduce the range of leaf nodes that need to be read after prediction, thereby reducing the number of flash reads triggered by point queries. A relatively loose error boundary is configured for non-hotspot segments to reduce index maintenance and resource overhead, achieving an adaptive balance between accuracy and cost.
[0111] Regarding key-level hotspot maintenance, the system maintains a Top-K table only within hotspot segments to store the most frequently accessed keys and their address information within those segments. This limits key-level maintenance overhead to a very small number of hotspot segments and ensures manageable memory consumption. The Top-K table maintenance rules are as follows: if a key is hit, the count is updated; if a key misses and the table is not full, it is inserted; if a key misses and the table is full, the current smallest count item is replaced and the count is updated. The system calculates or updates a migration benefit score only when a key enters the Top-K table of a hotspot segment and is identified as a hotspot key. This score drives the admission, replacement, and cross-level migration of multi-level caches. The migration benefit score comprehensively considers data access frequency, the time cost of cross-device access, data migration costs, and the potential benefits brought by hits at different cache levels. Therefore, it prioritizes ensuring that "high-frequency and high-benefit" data resides at more suitable cache levels.
[0112] On the query path, the NDP device first performs a hierarchical learning index query to obtain the key-value pair address. Then, the system searches in the host cache and the device cache in order of hierarchy: if the host cache is hit, it is returned directly; if the host cache is not hit, the device cache is queried, and if it is hit, it is also returned; if neither cache is hit, the value log file of the target NDP device is located and read according to the address.
[0113] During the above process, the system synchronously updates the segment heat container and the Top-K table of hot segments; if the key is a hot key, the migration benefit score is calculated / updated based on its final service location (host hit, device hit, or value log read), and the system determines its admission, replacement, and cross-layer migration (including promotion / demotion) in the host and device caches accordingly.
[0114] like Figure 6 As shown, based on hierarchical learning index, invalid addresses are collected during data update, the proportion of invalid data is maintained at the value log file granularity, and a garbage collection task is triggered when the proportion reaches a threshold, with the background performing valid data rewriting and index update in parallel.
[0115] During the storage of key-value pairs on an NDP device, the value data is stored in a value log file. However, with system operation and frequent data updates, a large amount of data in the value log file gradually becomes invalid. This invalid data occupies a significant amount of storage space, which may severely impact the device's storage efficiency over time. The garbage collection strategy based on hierarchical learning indexes proposed in this invention can efficiently reclaim the storage space occupied by invalid data in the NDP device's value log file, thereby significantly improving the system's storage efficiency and overall performance.
[0116] This invention maintains invalid data information and triggers garbage collection through a hierarchical learning index. During the update process of the hierarchical learning index, the system collects the address information of invalid old key-value pairs and uses this information to maintain the proportion of invalid data in each value log file. When the proportion of invalid data in a value log file reaches a preset threshold, the system automatically triggers the garbage collection process, thereby cleaning up unnecessary data and freeing up storage space. To further improve the efficiency of garbage collection, this invention introduces a hot data and cold data partitioning strategy. Specifically, the system classifies data according to the update frequency of each value data using a write heat identification module. Data with high write heat is assigned to the hot value log area. These data are frequently updated, so their invalid data is more concentrated, making them easier to process.
[0117] When garbage collection is triggered, the system places value log files that have reached the invalid data ratio threshold into a background task queue. The background thread determines the timing of garbage collection based on the current system load, ensuring that the collection operation does not affect foreground read / write requests, thereby maintaining the overall system throughput and response speed. During garbage collection, the system first sequentially reads the target value log files from the collection task into memory. During reading, the system verifies the validity of the read key-value pairs using a hierarchical learning index. Valid key-value pairs are rewritten back to the new value log file in the cold or hot value log partition, depending on where the target value log file is located. After collection is complete, the system updates the addresses of the newly generated key-value pairs back to the hierarchical learning index and ends the garbage collection of the target value log files. If there are still value log files to be collected in the task queue, the system repeats this process.
[0118] Through this design for identifying, classifying, and triggering invalid data based on hierarchical learning indexes, as well as a background processing mechanism, this invention effectively improves the efficiency of garbage collection, reduces system resource consumption and latency, and provides strong support for the application of NDP devices in high-load, large-scale data storage environments.
[0119] Based on the aforementioned system and method, this invention introduces a hierarchical learning index on the NDP device side to efficiently organize and accurately locate key-value pair addresses, thereby reducing the probing and random read overhead of point queries and significantly improving read performance. Key-value separation is used to reduce write amplification, while key-value separation-related processing is offloaded to the NDP device side. A hierarchical learning index is used on the device side to maintain the mapping relationship between "key" and "value" addresses, enabling queries to complete address lookups with lower location overhead, thus accelerating point queries and reducing read amplification caused by cross-layer / cross-file access.
[0120] A multi-level caching mechanism is designed in collaboration between the host and various NDP devices. The system maintains a set of hotkeys based on a hierarchical learning index and calculates migration benefit scores for hotkey-value pairs. This score comprehensively considers data access frequency, the time cost of cross-device access, data migration cost, and the potential benefits brought by different cache levels, evaluating migration value in a "benefit-cost" manner. Based on this, the system drives the hierarchical placement and dynamic migration of hot data between the host cache and device cache, improving cache hit rate and reducing the overhead of cross-device access and random reads. In addition, this invention proposes a garbage collection module driven by a hierarchical learning index: it uses the hierarchical learning index to collect invalid old addresses during the update process, continuously maintains the proportion of invalid data in the value log, and triggers garbage collection when a threshold is reached, thereby improving the accuracy and efficiency of garbage collection; at the same time, write frequency identification is introduced into the write path to achieve separation of hot and cold data, and high-frequency updated data is stored centrally, further reducing garbage collection overhead and improving the overall system performance.
[0121] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for accelerating read / write operations of multiple NDP devices based on hierarchical learning indexes, characterized in that, include: On the host side, write key-value pairs are identified based on LRU cache and marked as hot write data or cold write data. The key-value pair data is distributed to the corresponding NDP device, and key-value separation is performed on the NDP device. The value data is written to the hot value log area or cold value log area according to the write hotness mark. A hierarchical learning index is built and maintained on the NDP device side to store the mapping relationship between key and value addresses; In response to a read request, the value address of the target key is located by querying the hierarchical learning index, and the query is performed in the multi-level cache collaborative management system; The hierarchical learning index collects invalid addresses during data updates, maintains the proportion of invalid data at the value log file granularity, and triggers a garbage collection task when the proportion reaches a threshold. The background thread executes the rewriting of valid data and index updates in parallel.
2. The method according to claim 1, characterized in that, The write heat identification includes the following steps: Maintain a fixed-size LRU key cache on the host side to record recently written keys and their update counts; When a key exists in the LRU cache and the number of updates exceeds a preset threshold, the corresponding key-value pair is marked as hot write data; otherwise, it is marked as cold write data. The hot and cold tags are flushed to the NDP device along with the key-value pair data.
3. The method according to claim 1, characterized in that, The hierarchical learning index adopts a hybrid memory and flash storage structure, including an incremental index, segment mapping table and segment model table residing in memory, and leaf nodes stored in flash memory.
4. The method according to claim 3, characterized in that, The NDP device-side hierarchical learning index query steps include: Determine the data segment to which the query key belongs and the corresponding segment model table based on the segment mapping table; The linear model in the segment model table is used to predict the position of the query key in the leaf node, and the candidate search interval is determined by combining the error boundary. The leaf node data pages corresponding to the candidate interval are read from flash memory into memory, and a binary search is performed to obtain the precise value address.
5. The method according to claim 1, characterized in that, The steps for maintaining the hierarchical learning index include: Maintain an incremental index in memory to cache the addresses of recently written or updated key-value pairs; When the incremental index data volume of a certain data segment reaches the threshold, segment merging and model retraining are triggered. The host machine performs model training, generates a new version of the model table, and sends it to the NDP device. The NDP device updates the segment model table version to ensure query consistency.
6. The method according to claim 1, characterized in that, The multi-level cache collaborative management forms a read heat identification mechanism, the steps of which include: Using the data segments of the hierarchical learning index as the granularity, the access frequency, recent access time, and average operation time of read requests are statistically analyzed to calculate segment popularity and identify hot segments. Maintain a Top-K hotkey table within the hotspot segment, recording the most frequently accessed keys and their addresses within the segment; Calculate a migration benefit score for the keys in the Top-K hotkey table. This score takes into account access popularity, cross-device access cost, migration cost, and cache hit benefit. The migration benefit score drives the admission, replacement, and cross-layer migration decisions of key-value data between the host-side cache and the NDP device-side cache.
7. The method according to claim 6, characterized in that, In the multi-level cache collaborative management, the NDP device performs a hierarchical learning index query to obtain the key-value pair address, and searches in the host cache and device cache in sequence according to the level. If the host cache is hit, it is returned directly; if the host cache is not hit, the device cache is queried, and if it is hit, it is also returned directly; if neither cache is hit, the value log file of the target NDP device is located and read according to the address. During the query process, the segment heat container and the hot segment Top-K table are updated synchronously. If the key is a hot key, the migration benefit score is calculated / updated based on its final service location, and the admission, replacement and cross-layer migration in the host and device cache are determined accordingly.
8. The method according to claim 1, characterized in that, The waste recycling steps include: Based on the invalid addresses recorded during the hierarchical learning index update process, the proportion of invalid data in each value log file is maintained in real time; When the proportion of invalid data in the value log file reaches a threshold, it is added to the background garbage collection task queue, and the following operations are performed in parallel by multiple background threads: The target value log file is read sequentially to verify the validity of key-value pairs. Valid data is rewritten to the new value log file according to its original hot and cold attributes, and the address mapping in the hierarchical learning index is updated.
9. A multi-NDP device read / write acceleration system for implementing the method of any one of claims 1-8, characterized in that, include: The host-side write processing module is configured to perform write activity identification and data distribution. Multiple NDP devices, each NDP device includes: The key-value separation module is used to write value data to the hot value log area or the cold value log area based on the write popularity flag. The hierarchical learning index module is used to build and maintain a mapping index from key to value address; The multi-level cache management module is configured to manage the collaborative work between the host-side cache and the caches of each NDP device, including heat identification and data migration.
10. The system according to claim 9, characterized in that, Also includes: The garbage collection module is configured to trigger and execute background garbage collection tasks based on hierarchical learning index information.