An NVMe all-flash storage method based on a data buffering mechanism and a system thereof

Abstract

The application discloses a kind of NVMe full flash storage method and system based on data buffering mechanism, its characteristics are that the method adopts through hash linear extension chain table to find data information, such as data in buffer layer, directly through cache reading or rewriting, otherwise through CPU to memory application virtual block device hardware, using DMA and NVMe hard disk directly data interaction, such as write or rewrite data request, write or rewrite metadata information to metadatabase, periodically data in buffer layer is written into NVMe hard disk by virtual block device using DMA, using RDB to solidify and clean up memory, system includes: NVMe hard disk read-write, metadata management, read-write data cache and memory cleaning and so on module.The application has the advantages of high scalability, low read-write delay, high concurrent write throughput compared with prior art, so that the performance of CPU is quite liberated, can adapt to the application demand of big data.

Description

Technical Field

[0001] This invention relates to the field of computer storage technology, and in particular to an NVMe all-flash storage method and system based on a data buffering mechanism. Background Technology

[0002] With the development of the Internet of Things (IoT), spatiotemporal data is growing exponentially. To address the challenges brought by this data increase, a scalable and efficient storage architecture is needed to store, query, analyze, and visualize spatiotemporal big data. Thanks to emerging technologies such as solid-state drives (SSDs) and the Non-Volatile Fast Memory Host Controller Interface (NVMe), NVMe-based SSDs offer unprecedented performance in terms of latency and peak bandwidth. Considering the characteristics of IoT services, data storage methods must meet requirements such as extremely high scalability, low-latency data access, high-concurrency write throughput, and efficient spatiotemporal data querying. However, there is still a significant gap between existing storage systems and the storage needs of IoT big data.

[0003] Currently mainstream data storage systems, such as Ceph and HDFS, often encounter severe read / write throughput bottlenecks when faced with the pressure of rapidly accessing massive amounts of small files. When processing massive amounts of small file data, the size of metadata increases dramatically with the data volume. Traditional metadata architectures, metadata backup management, and dynamic metadata load balancing become increasingly inadequate for the demands of big data applications, leading to a series of performance bottlenecks. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by designing an NVMe all-flash storage method and system based on a data buffering mechanism. The all-flash storage system, constructed using an NVMe hard drive writing module with added virtual blocks implemented in SPDK and a metadata management module, data buffering module, and memory clearing strategy selection module implemented in Redis, achieves high scalability, low read / write latency, and high concurrent write throughput in the context of spatiotemporal big data. This significantly improves the efficiency of data storage operations (CRUD operations), effectively alleviates the I / O pressure on NVMe solid-state drives under high-concurrency write conditions, and considerably liberates CPU performance. It is well-suited to the application needs of big data and has excellent application prospects and promotional value.

[0005] The specific technical solution for achieving the objective of this invention is: an NVMe all-flash storage method based on a data buffering mechanism, characterized by the following steps:

[0006] Step 1: When a read / write request is received from a user, the data information is retrieved from the metadata database by using a hash-based linearly expanded linked list.

[0007] Step 2: If the data is in the buffer layer, it is read or rewritten directly through the cache; if the data is not in the buffer layer, the CPU requests virtual block device hardware from memory and maps it to a specific physical block device on the NVMe hard drive, and uses DMA to directly interact with the NVMe hard drive for data exchange.

[0008] Step 3: After the DMA transfer is completed, determine whether to read the data into the buffer layer (read request) or update the corresponding data in the NVMe hard drive (write request) based on the type of data request.

[0009] Step 4: If it is a write or modify data request, write or modify the metadata information to the metadata database, and use AOF to write the operation to the log file.

[0010] Step 5: Periodically write the data in the buffer layer directly to the NVMe disk using DMA via the virtual block device, use RDB to solidify the metadata, and use deletion strategies (random deletion, LRU algorithm deletion, or deletion based on read frequency) to clean up the memory.

[0011] An all-flash storage system based on a data buffering mechanism and an NVMe all-flash storage architecture is disclosed. The system specifically includes: an NVMe hard drive read / write module with added virtual blocks, a metadata management module, a read / write data cache module, and a memory clearing strategy selection module. The NVMe hard drive read / write module with added virtual blocks receives read / write requests and writes or reads data from the solid-state drive (SSD) according to the requests. The metadata management module stores and manages metadata, including storage path, storage capacity, changes, additions, deletions, SSD information, data layout, and UUID information. The read / write data cache module stores data from the NVMe SSD in a cache and processes read / write requests directly in the cache. The memory clearing strategy selection module selects an appropriate strategy to clear the cache based on the read / write request distribution of the previous time period.

[0012] The NVMe hard drive read / write module that adds virtual blocks requests a larger virtual block in the I / O stack via CPU requests to serve as a read / write unit. Users can customize the size of the virtual block to meet the write requirements of their programs. The virtual block is treated as a hardware device by the system, therefore it can be written directly to the corresponding block device of the NVMe hard drive via DMA without CPU interrupt operations, enabling high-speed writing of pages and data larger than the device itself.

[0013] The metadata management module uses a cache database to manage metadata, uses memory to accelerate the reading and writing speed of metadata, uses a hash linear expansion linked list to perform add, delete, modify and query operations on metadata, and uses master-slave synchronization function to synchronize the metadata database of each node.

[0014] The read / write data caching module uses the idle portion of the cache database to construct a data buffer layer, which reads frequently accessed data from the NVMe SSD. When the data in a read / write request exists in the buffer layer, it is read or written directly from the buffer layer. At regular intervals, the data in the buffer layer is written back to the NVMe SSD through the NVMe SSD read / write module with added virtual blocks.

[0015] The memory clearing strategy selection module selects one of three deletion methods—LRU algorithm deletion, random deletion, and deletion by read frequency—based on the distribution of read and write requests and the number of times the buffer layer is hit in the previous time period to periodically delete data in the buffer layer, ensuring that the buffer layer has sufficient space.

[0016] Compared with existing technologies, this invention improves the read and write I / O speed of NVMe SSDs, fully utilizing their functionality. It also eliminates reliance on CPU interrupts for read and write operations, freeing up CPU performance. In terms of metadata, a buffer layer reduces the read and write load on the NVMe SSD, significantly accelerating the processing of read and write requests within the buffer layer and effectively solving the low-case write problem. It also provides three deletion strategies (random deletion, LRU algorithm deletion, and read frequency deletion) to ensure available space in the buffer layer. The method is simple and effective, achieving high scalability, low read and write latency, and high concurrent write throughput in the context of spatiotemporal big data. This greatly improves the efficiency of data storage operations (CRUD operations), effectively alleviates the I / O pressure on NVMe SSDs under high-concurrency write conditions, and significantly releases CPU performance. It is well-suited to the application needs of big data and has excellent application prospects and promotional value. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the NVMe all-flash storage system architecture of the present invention;

[0018] Figure 2 This is a flowchart of the NVMe all-flash storage system architecture of this invention;

[0019] Figure 3 Diagram of the read / write module for an NVMe SSD with added virtual blocks;

[0020] Figure 4 A module diagram for managing metadata using a hash-linearly expanding chain;

[0021] Figure 5 For adding a cache layer to the read module;

[0022] Figure 6 For writing modules that add a caching layer. Detailed Implementation

[0023] This invention employs a data buffering mechanism for NVMe all-flash storage, specifically including the following steps:

[0024] Step 1: Request virtual block device hardware from the CPU memory, which corresponds to a specific physical block device on the NVMe hard drive;

[0025] Step 2: If it is a read or write request, retrieve the data information from the Redis metadata database by using a hash-based linearly expanded linked list;

[0026] Step 3: Based on the read / write data request, if the data is in the buffer, it is directly read or rewritten through the cache and transferred to the specified virtual block device. If the data is not in the buffer, it interacts directly with the NVMe hard drive using DMA through the virtual block device and determines whether to read the data into the cache.

[0027] Step 4: If it is a write or modify request, write or modify the metadata information into the Redis metadata database, and use AOF to write the operation to the log file;

[0028] Step 5: Periodically write the data in the buffer directly to the NVMe hard drive using DMA via the virtual block device, use RDB to solidify the metadata, and use read frequency deletion to clean up the memory.

[0029] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0030] See Figure 1 The NVMe all-flash storage system employing a data buffering mechanism of this invention includes: an NVMe hard drive read / write module with added virtual blocks, a metadata management module, a read / write data cache module, and a memory clearing strategy selection module. The NVMe hard drive read / write module with added virtual blocks is used to receive read / write requests and write data to or read data from the solid-state drive (SSD) according to the requests. The metadata management module is used to store and manage metadata, including storage path, storage capacity, changes, additions, deletions, SSD information, data layout, and UUID information. The read / write data cache module is used to store data from the NVMe hard drive in a buffer layer and process read / write requests directly in the buffer layer. The memory clearing strategy selection module is used to select an appropriate strategy to clear the buffer layer based on the read / write request distribution of the previous time period.

[0031] See Figure 2The workflow of the NVMe all-flash storage system employing a data buffering mechanism in this invention is as follows: A user issues a read / write request to the Redis database. If the data is not present in the Redis database, the server where the data resides is determined through distributed processing, and the request is distributed to the Redis database of that server. The system checks the Redis database to see if the data exists in the cache layer. If it does, it is modified directly in the data cache layer. If it does not exist, based on the metadata information, it uses SPDK to read / write to the NVMe SSD, and determines whether the data needs to be read back to the data cache layer. Data in the data cache layer is periodically written back to the hard drive via SPDK, and metadata is periodically persisted to the hard drive.

[0032] See Figure 3 The SPDK implementation of the NVMe hard drive read / write (write) module for adding virtual blocks works as follows: The user program passes the read / write request to the virtual file system, which allocates inodes such as ext2, ext3, and NFS, and allocates virtual blocks (vBdev) of the appropriate size according to the read / write request size. Then, it requests memory from the CPU and recognizes it as a hardware device. Afterward, read / write operations can bypass CPU interrupts, and vBdev can directly read and write to the block device in the corresponding NVMe hard drive via DMA.

[0033] See Figure 4 The Redis metadata management module stores metadata information in Redis and uses a hash-based linear linked list for searching, which greatly improves the speed of adding, deleting, modifying, and querying data. Each hash bucket in the hash-based linked list stores key-value pairs. The key points to the address of the metadata descriptor to be queried, and the value points to the address of the metadata structure. The metadata information at this address includes storage path, storage size, SSD information, data layout, and UUID, etc.

[0034] See Figure 5 The read / write data caching module operates as follows: A data buffer layer is constructed, storing frequently accessed data from the previous time period. Subsequent read requests first check if the data exists in the metadata cache layer. If it does, it is read directly from the cache layer; otherwise, SPDK is used to read the data from the hard drive, and a determination is made as to whether the read data needs to be stored in the cache layer.

[0035] See Figure 6The read / write data caching module operates as follows during writing: A data buffer layer is constructed, and frequently accessed data from the previous time period is stored in the buffer layer. Subsequent write requests first check if the data exists in the metadata cache layer. If it does, it is modified directly in the cache layer, and a determination is made as to whether to write it to the hard drive via SPDK; if not, it is written to the hard drive via SPDK, and a determination is made as to whether the written data needs to be stored in the cache layer.

[0036] Finally, at regular intervals, the data in the cache is written to the hard drive via the NVMe solid-state drive writing module, and the metadata is also solidified to the hard drive. Based on the access situation in the previous period, if the high-frequency access is very concentrated, the LRU algorithm is used for deletion, and the amount of data deleted is relatively small; if the high-frequency access is not concentrated, the read frequency deletion is used, and the amount of data deleted can be slightly larger.

[0037] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the invention. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the invention as defined in the claims should be included within the protection scope of the present invention.

Claims

1. An NVMe all-flash storage method based on a data buffering mechanism, characterized in that, The method includes the following steps: Step 1: Upon receiving a read / write request from a user, retrieve the data information from the metadata database using a hash-based linearly expanded linked list; Step 2: If the data is in the buffer layer, it is read or rewritten directly through the cache; if the data is not in the buffer layer, the CPU requests virtual block device hardware from memory and maps it to a specific physical block device on the NVMe hard drive, and uses DMA to directly interact with the NVMe hard drive for data exchange. Step 3: After the DMA transfer is completed, determine whether to read the data into the buffer layer or update the corresponding data in the NVMe hard drive based on the type of data request. Step 4: If it is a write or modify data request, write or modify the metadata information to the metadata database, and use AOF to write the operation to the log file; Step 5: Periodically write the data in the buffer layer directly to the NVMe hard drive using DMA via virtual block device, use RDB to solidify the metadata, and use random deletion, LRU algorithm deletion, or read frequency deletion to clean up the buffer layer.

2. An NVMe all-flash storage system constructed using the data buffering mechanism-based NVMe all-flash storage method as described in claim 1, characterized in that, The NVMe all-flash storage system includes: an NVMe hard drive read / write module with added virtual blocks, a metadata management module, a read / write data caching module, and a memory clearing strategy selection module. The NVMe hard drive read / write module with added virtual blocks is used to receive read / write requests and write data to or read data from the solid-state drive according to the requests. The metadata management module is used to store and manage metadata, including storage path, storage amount, changes, additions, deletions, SSD information, data layout, and UUID. The read / write data caching module is used to store data in the NVMe hard drive into a buffer layer and process read / write requests directly in the buffer layer. The memory clearing strategy selection module is used to select an appropriate strategy to clear the buffer layer based on the read / write request distribution of the previous time period.

3. The NVMe all-flash storage system according to claim 2, characterized in that, The NVMe hard drive read / write module that adds virtual blocks creates a new virtual block device layer, located above the physical block device layer, and directly reads and writes data to the NVMe hard drive via DMA, bypassing the CPU.

4. The NVMe all-flash storage system according to claim 2, characterized in that, The metadata management module uses a distributed cache database for management, and achieves fast addition, deletion, modification and query of metadata through hash linear expansion linked lists, and uses RDB and AOF operations to persist the metadata.

5. The NVMe all-flash storage system according to claim 2, characterized in that, The read / write data caching module uses memory to build a buffer layer, which reads some data from the NVMe hard drive into the buffer layer. When read / write requests are made for this part of the data, it can be processed directly in memory.

6. The NVMe all-flash storage system according to claim 2, characterized in that, The memory cleanup strategy selection module selects different deletion strategies to clean up the memory based on the distribution of read and write requests in the previous time unit.

Images