Data reading method for storage device and storage device

Abstract

The invention discloses a data reading method for a storage device and the storage device, and particularly provides a data reading method for a storage device, which comprises the following steps: the storage device receives a search request from a host, and the search request comprises an address of a file to be searched; loading the file into a computing unit included in the storage device based on the address; obtaining a search result by searching the file with the data to be read by the storage device; and returning the search result to the host.

Description

Technical Field

[0001] The present disclosure relates to the field of storage technology, and in particular to a method for reading data from a storage device and a storage device. Background Art

[0002] Log-Structured Merge-Tree (LSM-Tree) can be a hierarchical ordered data storage structure designed for the characteristics of block storage devices. When performing a read operation on a database based on LSM-Tree, if the data to be read is not found in the memory table (MemTable) and the immutable memory table (Immutable MemTable), a sorted string table (SST) file stored in a hierarchical form inside the disk of the storage device can be loaded into the host memory for search. This search not only has a large read amplification, which may reduce the efficiency of the read operation, but also increases the occupancy rate of the central processing unit (CPU) and the power consumption of the host. Summary of the invention

[0003] A method for reading data from a storage device and a storage device are provided to solve part or all of the above problems.

[0004] Additional aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented embodiments.

[0005] According to aspects of the present disclosure, a method for reading data from a storage device includes: receiving a search request by the storage device from a host, wherein the search request includes an address of a file to be searched; based on the address, loading the file into a computing unit included in the storage device; obtaining a search result by the storage device searching for the data to be read in the file; and returning the search result to the host.

[0006] Optionally, the file includes an ordered character table SST file stored hierarchically in a database based on a log structure merge tree LSM-Tree.

[0007] Optionally, loading the file includes: loading an index block and a filter block included in the SST file into the computing unit, wherein the index block includes index information about the data block in the SST file, and wherein searching for the data in the file includes: determining a candidate data block based on the index block; determining whether the data exists in the candidate data block based on the filter block; loading the candidate data block into the computing unit based on determining that the data exists in the candidate data block; and searching for the data in the candidate data block.

[0008] Optionally, the candidate data block is determined using a binary search in the index block, and wherein determining whether the data exists in the candidate data block comprises: based on the filter block, using a Bloom filter to check whether the data exists in the candidate data block.

[0009] Optionally, checking whether the data exists in the candidate data block includes: calculating multiple hash values ​​corresponding to the key Key of the data using multiple hash functions; and matching the multiple hash values ​​with the data structure in the filter block.

[0010] Optionally, returning the search result to the host includes: based on finding the data in the file, returning the address of the data to the host; based on not finding the data in the file, returning a message indicating that the data was not found to the host.

[0011] Optionally, after returning the message to the host, receiving a new search request from the host, wherein the new search request includes a new address of a new file to be searched; and based on the new address, loading the new file into the computing unit.

[0012] Optionally, based on the message being received by the host and the file having been traversed, or based on the address of the data being received by the host, the search for the data is terminated.

[0013] Optionally, the search request is sent by the host by calling a read interface in the database.

[0014] Optionally, the computing unit comprises a dynamic random access memory DRAM, and wherein loading the file into the storage device comprises: loading the file into the DRAM.

[0015] According to aspects of the present disclosure, a storage device includes: at least one processor configured to implement a computing unit, wherein the computing unit is configured to: obtain a search request received by the storage device from a host, wherein the search request includes an address of a file to be searched, load the file into the computing unit based on the address, obtain a search result by searching for data to be read in the file, and return the search result to the host.

[0016] Optionally, the file includes an ordered character table SST file stored hierarchically in a database based on a log structure merge tree LSM-Tree.

[0017] Optionally, the computing unit is also configured to: load the index block and filter block included in the SST file into the computing unit, wherein the index block includes index information about the data block in the SST file; determine the candidate data block based on the index block; determine whether the data exists in the candidate data block based on the filter block; load the candidate data block into the computing unit based on determining that the data exists in the candidate data block; and search for the data in the candidate data block.

[0018] Optionally, the computing unit is further configured to: use binary search to search for the candidate data block in the index block, and use a Bloom filter based on the filter block to check whether the data exists in the candidate data block.

[0019] Optionally, the calculation unit is further configured to: calculate a plurality of hash values ​​corresponding to the key Key of the data using a plurality of hash functions; and match the plurality of hash values ​​with the data structure in the filter block.

[0020] Optionally, the computing unit is further configured to: based on finding the data in the file, return the address of the data to the host; based on not finding the data in the file, return a message indicating that the data was not found to the host.

[0021] Optionally, the computing unit is further configured to: after returning the message, receive a new search request from the host, wherein the new search request includes a new address of a new file to be searched, and based on the new address, load the new file into the computing unit in the storage device.

[0022] Optionally, based on the message being received by the host and the file having been traversed, or based on the address of the data being received by the host, the search for the data is terminated.

[0023] Optionally, the search request is sent by the host by calling a read interface in the database.

[0024] Optionally, the computing unit comprises a dynamic random access memory DRAM, and wherein the computing unit is further configured to: load the file into the DRAM.

[0025] According to aspects of the present disclosure, a storage system includes: a main processor; a memory; and a storage device, wherein the storage device is configured to: receive a search request from a host, wherein the search request includes an address of a file to be searched; based on the address, load the file into a computing unit included in the storage device; obtain a search result by searching for data to be read in the file; and return the search result to the host.

[0026] According to aspects of the present disclosure, a host storage system includes: a host; and a storage device, wherein the storage device is configured to: receive a search request from the host, wherein the search request includes an address of a file to be searched; based on the address, load the file into a computing unit included in the storage device; obtain a search result by searching for data to be read in the file; and return the search result to the host.

[0027] According to aspects of the present disclosure, a data center system includes: multiple application servers; and multiple storage servers, wherein each storage server includes a storage device, and wherein the storage device is configured to receive a search request from a host, wherein the search request includes an address of a file to be searched; based on the address, load the file into a computing unit included in the storage device; obtain a search result by searching for data to be read in the file; and return the search result to the host.

[0028] According to an aspect of the present disclosure, a computer-readable storage medium stores computer program instructions, wherein, when the computer program instructions are executed by at least one processor of a storage device, the storage device is caused to perform the following operations: receiving a search request from a host, wherein the search request includes an address of a file to be searched; based on the address, loading the file into a computing unit included in the storage device; obtaining a search result by searching for data to be read in the file; and returning the search result to the host. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0030] Figure 1 The read process of RocksDB according to the embodiment is shown;

[0031] Figure 2 A block diagram showing a comparison between a computational storage device (CSD) and a solid state drive (SSD) according to an embodiment;

[0032] Figure 3 A diagram showing the interaction between a host and a storage device regarding a read operation according to an embodiment;

[0033] Figure 4 A comparative flow chart showing a read operation according to an embodiment;

[0034] Figure 5 A flow chart showing a method for reading data from a storage device according to an embodiment;

[0035] Figure 6 A detailed flow chart showing data reading according to an embodiment;

[0036] Figure 7 A data flow comparison diagram according to an embodiment is shown;

[0037] Figure 8 A block diagram of the internal modules of a storage device according to an embodiment is shown;

[0038] Fig. 9 is a schematic diagram of a system to which a storage device is applied according to an embodiment;

[0039] Fig.10 is a block diagram of a host storage system according to an embodiment; and

[0040] Fig.11 is a diagram of a data center to which a storage device is applied according to an embodiment. DETAILED DESCRIPTION

[0041] In order to enable ordinary persons in the art to better understand the technical solution of the present disclosure, the embodiments of the present disclosure will be described below in conjunction with the accompanying drawings.

[0042] It should be noted that the terms "first", "second", etc. are used herein to distinguish similar objects or elements, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0043] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of operation 1 and operation 2" which means the following three parallel situations: (1) performing operation 1; (2) performing operation 2; (3) performing operation 1 and operation 2.

[0044] Log-Structured Merge-Tree (LSM-Tree) is a hierarchical ordered data storage structure designed for block storage devices and can be applied to key-value (KV) storage systems. LSM-Tree may include a memory table (MemTable), an immutable memory table (Immutable MemTable), and a sorted string table (SST). MemTable can be a data structure of LSM-Tree, which can save the most recently updated data and organize the data according to the key. When MemTable reaches a certain size, it will be converted into ImmutableMemTable. Immutable MemTable can be an intermediate state for converting MemTable into SST file. Write operations can be handled by the new MemTable, and data update operations can be unblocked during the transfer process. SST, as a set of ordered key-value pairs, is the data structure of LSM-Tree in a storage device (e.g., a disk).

[0045] RocksDB can be a database based on LSM-Tree. In the following description, the RocksDB database can be used as an example to illustrate the read operation of the LSM-Tree-based database, but the embodiment is not limited to this. RocksDB data can be stored in layers in the form of SST files inside a storage device (e.g., a disk). According to an embodiment, there can be 6 layers by default, and the size of each layer of SST files can be ten times that of the previous layer. Figure 1 An example of a read process of RocksDB according to an embodiment is shown, as shown in Figure 1 As shown, the read process of RocksDB may correspond to the search (Get()) process of RocksDB.

[0046] For example, in operation S101, process 100 may include searching for a target Key (e.g., a Key of data to be read) in a MemTable of a host memory;

[0047] In operation S102 , the process 100 may include searching for a target Key in an Immutable MemTable in a host memory;

[0048] Because the keys of the SST files of the L0 layer may overlap, in step S103, the process 100 may include sequentially loading each SST file of the L0 layer from the storage device to the host for binary search (BS) and Bloom filter (BF) check;

[0049] In operation S104, for the L1 layer and the layers below L1, only one SST file that may contain the search content (for example, the target Key) can be searched each time, and one candidate file (i.e., the file to be searched) of each layer can be loaded into the host in turn for BS and BF checks.

[0050] In order to improve the read performance of LSM-Tree, an embodiment may involve a method of changing a serial search to a parallel search. For example, according to some methods, one thread may be used to search an SST file in one layer at a time, and then search an SST file in the next layer after the search is completed. However, according to an embodiment, two threads may be used to search two different SST files in two layers at the same time.

[0051] In addition, in order to improve the read performance of LSM-Tree, the embodiment may involve XRP (eXpress Resubmission Path). XRP uses eBPF (extended Berkeley Packet Filter) technology to put some read operations in the driver layer, thereby reducing the input and output (IO) stack in the request kernel to improve the read performance.

[0052] According to an embodiment, a potential problem in the above-mentioned read process may be that the read amplification may be too large. As an example, the LSM-Tree may include 8 SST files in the L0 layer, and there may also be 6 layers (for example, layers L1 to L6) of SST files. In addition, at least an index block (which may have a size of, for example, 16 kilobytes (KB)), a filter block (for example, a Bloom filter block) (which may have a size of, for example, 4KB), and a data block (which may have a size of, for example, 4KB) may be read in each SST file to complete the search. According to this example, for a user reading 1KB of data, the amount of data actually read by a processor (such as a central processing unit (CPU)) may be: (8+6)*(16+4+4)KB=336KB. Therefore, the read amplification factor can be expressed as: 336KB / 1KB=336. This read amplification may be too large, which may not only reduce the efficiency of the read operation, but also increase the CPU occupancy rate and the power consumption of the host.

[0053] The parallel search method can provide reduced latency for read operations, but the amount of data moved from the storage device to the host may not be reduced, so the read amplification may not be reduced. In addition, because an extra layer of SST files may be read, the read amplification may be even higher. For example, in a serial search, if the target key is found in the previous layer of SST files, the reading of the next layer of SST files can be stopped, but in a parallel search, the next layer of SST files may be read in parallel for search, so the read amplification may be even higher. In addition, because additional thread operations are introduced, the CPU occupancy and host power consumption may increase.

[0054] In addition, XRP can reduce the IO stack above the driver layer, but the files will still be moved from the storage device to the host's driver layer, so the problem of excessive read amplification cannot be solved.

[0055] Therefore, the embodiments can provide a storage device and a method for reading data from the storage device, which can offload some search processes to the storage device to reduce data movement between the storage device and the host, greatly reduce read amplification, improve read performance, and reduce CPU occupancy and host power consumption. Figures 2 to 11 An example of a method for reading data from a storage device and a storage device according to an embodiment of the present invention will be described in detail.

[0056] Figure 2 A block diagram showing a comparison between a solid state drive (SSD) and a computational storage drive (CSD) according to an embodiment of the present disclosure. A CSD may be a customizable programmable storage device, such as Samsung's SmartSSD. Figure 2 As shown, the Field Programmable Gate Array (FPGA) in the CSD can provide software engineers with the ability to program one or more intellectual property (IP) cores or IP blocks that can be used to accelerate a variety of applications. Compared with SSDs, CSDs according to embodiments can provide several advantages, such as the ability to offload the CPU, the ability to use hardware for acceleration, and the ability to use near-storage computing to reduce data movement.

[0057] Figure 3An interaction diagram of a host 310 and a storage device (such as CSD 320) regarding a read operation according to an embodiment of the present disclosure is shown. According to an embodiment, CSD 320 may include a computing unit 321, which includes a search module 322, and the search module 322 includes a BS module 323 configured to perform a binary search (BS) algorithm and a Bloom filter (BF) module 324. In some embodiments, since the computing unit 321 can be an FPGA, the BS module 323 and the BF module 324 can be implemented as an IP that can provide two functions for the computing unit 321. The computing unit 321 may also include an ARM core 325, a computing engine 326, and a random access memory (RAM) (such as a dynamic random access memory (DRAM) 327). The function of the BS module 323 may include binary searching for a data block where the data to be read is located in the index block of the SST file, and searching for the data to be read in the data block. In addition, CSD 320 may also include a controller 328 and one or more NAND devices 329 that control the overall operation of CSD 320.

[0058] exist Figure 3 In the interaction diagram shown, after initiating a read request, it may not be necessary to move the SST file to the host 310 for BS and BF, because the CSD 320 can use the high bandwidth within the CSD 320 to load the SST file into the DRAM 327 for BF and BS. Therefore, only the result can be returned to the host 310. Figure 3 , taking RocksDB as an example, the interaction between the host 310 and the CSD 320 in a read operation may be performed as follows:

[0059] In operation S301 , a new read interface (eg, Smart_get()) may be added to RocksDB to use the read process of the present disclosure.

[0060] In operation S302, the read interface may first search for a target key (eg, a key of data to be read) in the memtable and immutable memTable of the memory in the host 310. If not found, the following operations may be continued.

[0061] In operation S303 , the host 310 may send a search instruction (which may be referred to as a search request) with an address of a file to be searched (which may be referred to as a candidate file) to the search module 322 .

[0062] In operation S304, the lookup module 322 may convert the address and load the corresponding file into the DRAM 327, and then the BS module 323 and the BF module 324 may perform BS and BF on the file;

[0063] In operation S305 , the search module 322 may return the search result, a message indicating that the to-be-read data is not found or the address of the found target value (Value), to the host 310 .

[0064] If operation S305 returns a message indicating that the data to be read is not found, the host 310 will again send a search instruction (or search request) with the address of the new file to be found (or new candidate file), and the CSD 320 may continue to search until the target Key-Value (for example, the data to be read) is found, or until all the files to be found at each layer are traversed.

[0065] Although the computing unit 321 is described as being implemented by an FPGA, the embodiments are not limited thereto. For example, in some embodiments, the computing unit 321 may also be implemented by other software, hardware, or a combination of software and hardware. In addition, the internal modules of the computing unit 321 (e.g., FPGA) and the internal modules of the CSD 320 are only examples, and some modules may be added or deleted, and the embodiments are not limited thereto.

[0066] Figure 4 A flow chart comparing a comparative example read operation (shown as flow 400A) and a read operation according to an embodiment of the present disclosure (shown as flow 400B) is shown. Figure 4 , taking RocksDB database as an example, the read operation of the database based on LSM-Tree is explained. The read operation disclosed in the present invention and the existing one mainly have three differences as shown in the figure.

[0067] In operation S401A, the process 400A may use a Get() interface. According to an embodiment, in operation S401B, the process 400B may, for example, add a new read interface (eg, Smart_get()) in RocksDB to use a read method according to an embodiment.

[0068] According to an embodiment, in operations S402A and S402B, processes 400A and 400B may search in the MemTable and the Immutable MemTable in the host, and in operations S403A and S403B, processes 400A and 400B may determine whether the target Key is found. If the target Key is found, processes 400A and 400B may end, and if the data to be read is not found, processes 400A and 400B may enter operations S404A and S404B, respectively.

[0069] In operation S404A, process 400A needs to frequently load the file to be found from the storage device to the host memory. In contrast, in operation S404B, the host can only send the address of the file to be found to the CSD, and the CSD can load the file to be found into the DRAM according to the address. Therefore, it is not necessary to load the file into the host memory.

[0070] At operation S405A, the process 400A may include the host searching for the loaded file (e.g., by executing BS and BF). In contrast, at operation S405B, the process 400B according to an embodiment of the present disclosure may use a newly added BS module (e.g., BS module 323) and a BF module (e.g., BF module 324) included in the CSD to search for the loaded file (e.g., by executing BS and BF).

[0071] According to an embodiment, in operations S406A and S406B, processes 400A and 400B may include determining whether a target Key is found. If the target Key is found, processes 400A and 400B may end, and if the data to be read is not found, processes 400A and 400B may return to operations S404A and S404B, respectively.

[0072] Figure 5 A flow chart showing a data reading process for a storage device according to an embodiment of the present disclosure is shown. According to an embodiment, Figure 5 The illustrated process 500 may be performed by any of the above elements, for example, a host (such as host 310 ) and a storage device (such as CSD 320 ), but the embodiments are not limited thereto.

[0073] Reference Figure 5 In operation S510, based on the address of the file to be found included in the search request received from the host, the file to be found is loaded into the computing unit in the storage device. In an embodiment, the host may correspond to the above-mentioned host 310, and the storage device may correspond to the above-mentioned CSD 320, but the embodiment is not limited thereto.

[0074] LSM-Tree-based databases (e.g., RocksDB) can add a read interface (e.g., Smart_get()). After initiating a read request, for example, the read interface can be used (e.g., called by the host) to initiate a read request. The read interface can first search for the data to be read (e.g., the target Key) in the memtable and immutable memTable of the host memory. If not found, the data to be read can be searched in the SST file.

[0075] According to an embodiment of the present disclosure, the search request may be sent by the host by calling a read interface in the database.

[0076] The read interface (e.g., Smart_get()) called by the host may send a search request including the address of the file to be searched to the storage device to instruct the storage device to search for the data to be read in the corresponding file. The storage device may load the file to be searched into the computing unit in the storage device based on the address of the file to be searched included in the search request received from the host. Specifically, the storage device may convert the address of the file to be searched included in the search request (e.g., conversion of the logical address to the physical address), and may then load the file to be searched into the computing unit in the storage device based on the converted address. Here, the search request may also include other parameters, such as the Key of the data to be read (e.g., the target Key), etc., to instruct the storage device to search for the data to be read (e.g., the target Key) in the file to be searched.

[0077] According to an embodiment of the present disclosure, the computing unit may include a dynamic random access memory DRAM, and the file to be found may be loaded into the DRAM included in the computing unit.

[0078] In an embodiment of the present disclosure, the computing unit may be, for example, Figure 3 The CSD 320 shown includes a computing unit 321. The computing unit 321 may be, for example, an FPGA, and the computing unit 321 may include a DRAM 327, to which the file to be found may be loaded from, for example, a flash memory NAND in the CSD 320.

[0079] According to an embodiment of the present disclosure, the file to be found may be an SST file stored hierarchically in a database based on LSM-Tree.

[0080] For example, the file to be searched may be all SST files in the L0 layer, or may be an SST file in L1-L6 that may contain the data to be read (e.g., the target key). The search may start from L0 and proceed to L6. For example, there may be 8 SST files in the L0 layer, and there may also be 6 layers (e.g., layers L1 to L6) of SST files. Therefore, the total number of files to be searched may be 8+6=14. In an embodiment of the present disclosure, the read interface called by the host may send the address of an SST file to be searched to the storage device each time.

[0081] In operation S520 , the to-be-read data may be searched in the to-be-searched file loaded into the computing unit.

[0082] According to an embodiment of the present disclosure, an index block and a filter block in an SST file to be searched may be loaded into a computing unit. In an embodiment, the index block may include index information about a data block in the SST file to be searched. Based on the index block loaded into the computing unit, a data block where the data to be read may be located may be determined. In an embodiment, a data block where the data to be read may be located may be referred to as a candidate data block. In an embodiment, operation S520 may also include determining whether the data to be read exists in the data block based on the filter block loaded into the computing unit. Based on determining that the data to be read exists in the data block, the data block in the SST file to be searched is loaded into the computing unit; and the data to be read is searched for in the data block loaded into the computing unit.

[0083] In an embodiment of the present disclosure, an SST file may include an index block, a filter block, and a data block, wherein the index block includes index information of the data block, and the filter block includes filter data corresponding to the data block. The filter block may be, for example, a Bloom filter block, the filter data may be a bit array of the Bloom filter, and the number of bit arrays of the Bloom filter may be the same as the number of data blocks. The record of the index block may, for example, be in the form of "Key: offset: size", wherein the Key may indicate the minimum Key (or maximum Key) of each data block, the offset may indicate the starting position of the data block, and the size may indicate the size of each data block. The data blocks may be stored in a sequential manner, and it may be convenient to perform a binary search on the index block to find the corresponding data block. In order to speed up the efficiency of data query in the SST file, before directly querying the content in the data block, it may be determined whether the data to be queried exists in the data block based on the filter data in the filter block. If it is determined that it does not exist, it is not necessary to search the entire data block.

[0084] According to an embodiment of the present disclosure, a data block where the data to be read may be located can be found in an index block through binary search, and a Bloom filter can be used to determine whether the data to be read exists in the data block based on a filter block loaded into a computing unit.

[0085] In an embodiment of the present disclosure, the index block and filter block of the file to be searched (for example, an SST file) may be first loaded into a computing unit. Since the index block may include index information of the data block, the candidate data block may be found in the index block through a binary search. For example, the Key is used for search and comparison to find the offset and size of the data block where the data to be read (for example, the target Key) may be located. In order to speed up the efficiency of data search in the file to be searched (for example, an SST file), before directly searching for the content in the data block, the filter data corresponding to the data block in the filter block may be obtained based on the candidate data block. For example, the bit array of the Bloom filter corresponding to the data block in the filter block may be obtained, and the Bloom filter is used to check whether the data to be read exists in the data block. In some embodiments, a method such as Figure 3 The BS module included in the calculation unit 321 included in the illustrated CSD 320 performs a binary search, and a Bloom filter may be performed using the BF module 324 included in the calculation unit 321 .

[0086] According to an embodiment of the present disclosure, multiple hash functions can be used to calculate multiple hash values ​​of the key Key (for example, the target Key) of the data to be read, and whether the data to be read exists in the data block can be determined by matching the multiple hash values ​​with the data structure in the filter block.

[0087] A Bloom filter may refer to a data structure used to check whether an element is in a set. A Bloom filter may use a bit array (e.g., a bitmap or a bitmap) and a set of hash functions to complete element checking. When adding an element, the added element may be mapped to k positions in a bit array using k (where k is a positive integer) hash functions. In an embodiment, the bit array is initialized to 0, and the values ​​of the above k positions may be set to 1. For example, the string "hello world" may be hashed three times using three hash functions, and the three hash values ​​obtained may be 2, 5, and 8, respectively, then the values ​​of the three positions in the bit array (e.g., positions 2, 5, and 8) may all be set to 1. When checking the existence of an element, the element to be checked may also be mapped to k positions in the bit array through the k hash functions, and whether the values ​​of the k positions in the bit array are all 1 may be used to determine whether the queried element exists in the set. According to an embodiment, if the value of any of these positions is 0, the element being checked must not exist; and if the values ​​of these positions are all 1, the retrieved element may exist. For example, checking the existence of "hello world" includes applying a hash function to the element to be checked, obtaining hash values ​​2, 5, and 8, and checking whether the values ​​of the corresponding positions in the bit array are all 1.

[0088] In an embodiment of the present disclosure, multiple hash functions of a Bloom filter can be used to respectively calculate multiple hash values ​​of the Key of the data to be read, and then the calculated multiple hash values ​​can be matched with the data structure in the filter block. The data structure here can be a bit array of a Bloom filter corresponding to the data block in the filter block, and the data block can be a data block where the data to be read may be located. For example, it can be determined whether the values ​​of the positions of multiple hash values ​​in the bit array are all 1 to determine whether the data to be read exists in the data block. For example, based on the fact that the values ​​of the positions of multiple hash values ​​in the bit array are all 1 (which can indicate a match), it can be determined that the data to be read exists in the data block. If any of the values ​​of these positions is 0 (which can indicate a mismatch), it can be determined that the read data does not exist in the data block.

[0089] If it is determined through the Bloom filter that the data to be read exists in the data block, the data block in the SST file to be searched can be loaded into the computing unit. For example, according to the offset and size of the data block obtained above, the data block in the SST file to be searched can be loaded from the flash memory NAND 329 in the CSD 320 to the DRAM 327 in the computing unit 321, and the data to be read can be searched in the data block loaded into the computing unit 321. Here, the search method for searching the data to be read in the data block is not limited to the above example, and can also be performed by, for example, Figure 3 The BS module 323 in the computing unit 321 in the CSD 320 is shown to execute. If it is determined through the Bloom filter that the data to be read does not exist in the data block, the search for the SST file to be found can be terminated, and the storage device can return a message to the host indicating that the data was not found.

[0090] In operation S530, the search result may be returned to the host.

[0091] According to an embodiment of the present disclosure, in response to finding the data to be read in the file to be found, the address of the data to be read is returned to the host, and in response to not finding the data to be read in the file to be found, a message indicating that the data was not found can be returned to the host.

[0092] In an embodiment of the present disclosure, if the data to be read is found in the file to be found, the storage device may return the address of the data to be read to the host. According to an embodiment, the address may be a target value address, for example, an address in a NAND device. After the host receives the address of the data to be read returned by the storage device, the search for the data to be read may end, and the data may be read by the host. If the data to be read is not found in the file to be found, the storage device may send a message to the host indicating that the data is not found.

[0093] According to an embodiment of the present disclosure, after returning a message indicating that the data is not found to the host, in response to receiving a search request including the address of a file to be found from the host, process 500 may include returning an operation of loading the file to be found into a computing unit in a storage device based on the address of the file to be found included in the search request received from the host, wherein the address of the file to be found is the address of a new file to be found.

[0094] According to an embodiment of the present disclosure, when a message indicating that the data is not found is received by the host and the file to be found has been traversed, or when the address of the data to be read is received by the host, the search for the data to be read is completed.

[0095] In an embodiment of the present disclosure, after receiving a message indicating that the data has not been found, if the file to be found has not been traversed, the host may continue by sending a search request including the address of the new file to be found to the storage device. In the example of using RocksDB, if the SST file to be found has not been traversed to a possible SST file in the last layer (for example, layer L6), the host may send a search request (for example, a new search request) including the address of the next SST file to be found to the storage device, instructing the storage device to continue searching. After receiving the new search request, the storage device may repeat the above operations S510 to S530. If the file to be found has been traversed, for example, it has been traversed to a possible SST file in the last layer of RocksDB (for example, layer L6), the search for the data to be read may be terminated, and the search request for the data to be read may no longer be sent to the storage device.

[0096] According to the method for reading data from a storage device disclosed in the present invention, the high bandwidth in the storage device can be fully utilized to load the file to be searched into the storage device, which can greatly reduce the data movement between the storage device and the host, and can significantly reduce the read amplification. In addition, the binary search module and the Bloom filter module implemented in the storage device can be used to search for the file to be searched, which can not only improve the search efficiency, but also reduce the CPU occupancy rate and the power consumption of the host.

[0097] Figure 6 A detailed flow chart of a data reading process 600 according to an embodiment of the present disclosure is shown. According to an embodiment, the process 600 may be performed by any of the above elements, for example, a host (such as the host 310) and a storage device (such as the CSD 320), but the embodiment is not limited thereto.

[0098] According to the data reading process 600 of the present disclosure, in operation S601, the read interface may be called by the host to initiate a read request for the data to be read. The read interface may be, for example, the read interface Smart_get() added by RocksDB. In operation S602, the read interface may search for the target Key (e.g., the Key of the data to be read) in the memtable and immutable memTable included in the host memory. In operation S603, if the target Key is found ("Yes" in operation S603), the process 600 ends. If it is not found ("No" in operation S603), the process 600 goes to operation S604. In operation S604, the read interface may send a search request including the address of the file to be found to the storage device (e.g., CSD 320).

[0099] In an embodiment, the following operations S605 to S614 may be performed by a storage device:

[0100] In operation S605, the storage device may load the index block and filter block (e.g., Bloom filter block) of the file to be searched into the DRAM in the storage device (e.g., the DRAM in the computing unit) according to the address of the file to be searched (e.g., the SST file).

[0101] In operation S606 , a binary search may be performed in the index blocks loaded into the DRAM to find the data blocks where the data to be read may be located.

[0102] In operation S607 , a plurality of hash values ​​of the target Key may be calculated through a plurality of hash functions of the Bloom filter.

[0103] In operation S608 , a plurality of hash values ​​of the target Key may be checked in the Bloom filter block to determine whether the data to be read exists in the data block.

[0104] In operation S609, if the multiple hash values ​​do not match the bit array in the Bloom filter block ("No" in operation S609), the search for the file to be found can be ended, and process 600 can go to operation S614 to return a message to the host indicating that the data was not found. If the multiple hash values ​​match the bit array of the Bloom filter ("Yes" in operation S609), process 600 goes to operation S610.

[0105] In operation S610, the data block of the file to be found may be loaded into a DRAM in a storage device (eg, a DRAM in a computing unit).

[0106] In operation S611 , a target Key (eg, a Key of data to be read) may be searched in a data block loaded into the DRAM.

[0107] In operation S612, if the target Key is found ("Yes" in operation S612), the process 600 proceeds to operation S613, and if not ("No" in operation S612), the process 600 proceeds to operation S614.

[0108] In operation S613 , the storage device may return an address of a target value (eg, an address of Value of data to be read) to the host.

[0109] And in operation S614, the storage device may return a message to the host indicating that the data is not found.

[0110] After operation S614, process 600 may return to operation S604. For example, after the host receives a message indicating that the data is not found, if the file to be found has not been traversed, the read interface in operation S604 may send a search request including the address of the new file to be found to the storage device, so that the storage device may repeatedly perform operations S605 to S614 to complete the search for the new file to be found, until the data to be read is found or the file to be found is traversed.

[0111] Figure 7 A data flow comparison diagram of the present disclosure and a comparative example according to an embodiment of the present disclosure is shown.

[0112] In a comparative example, in the SST file search process (taking RocksDB as an example, using the Get() interface), the SST file to be searched can be loaded from a storage device (for example, SSD) to a host, the amount of data moved may be large, and BS and BF operations can be performed on the loaded SST file on the host.

[0113] In contrast, according to an embodiment of the present disclosure, in the search process for SST files (taking RocksDB as an example, using the newly added Smart_Get() interface), the host may send a search request including the address of the SST file to be searched to the storage device (e.g., CSD). The storage device may load the SST file to be searched into the storage device (e.g., DRAM) according to the address of the SST file to be searched after address conversion, and perform BS and BF operations on the loaded SST file in the storage device. Therefore, the BS and BF operations can be unloaded to the storage device. If the data to be read is not found, the storage device returns a message (a small amount of data) to the host indicating that the data to be read is not found, and after receiving the message, the host may send a search request including the address of a new SST file to be searched to the storage device. The storage device may load the new SST file to be searched into the storage device according to the address of the new SST file to be searched after address conversion, and perform BS and BF operations on the loaded SST file in the storage device. If found, the storage device returns the address of the target value to the host.

[0114] Although an example in which the storage device is a CSD is described above, embodiments are not limited thereto.

[0115] An example comparison between the comparative example and the embodiment according to the present disclosure is shown in Table 1 below:

[0116] Table 1

[0117] Comparison Examples Embodiments of the present disclosure Large amount of data movement Reduced data movement Not fully utilizing the high bandwidth on the disk Better use of the high bandwidth on the disk High CPU usage Reduced CPU usage High power consumption Reduced power consumption

[0118] As described above, the read process according to the comparative example may involve moving at least 24K of data for each SST file, while the read process according to the embodiment of the present disclosure may only return one result after each SST file search. According to the calculation that the size of a completed command of a non-volatile memory express (NVMe) SSD can be 16B, the read amplification according to the embodiment of the present disclosure can be: ((8+6)*16B+1KB) / 1KB=1.22 times. Compared with the 336-fold read amplification according to the comparative example, the embodiment can provide greatly reduced read amplification.

[0119] Figure 8 A block diagram of the internal modules of a storage device according to an embodiment of the present disclosure is shown.

[0120] Reference Figure 8, the storage device 800 may include a computing unit 810, which may be configured to: load the file to be found into the computing unit 810 in the storage device 800 based on the address of the file to be found included in the search request received from the host; obtain the result of the search by searching for the data to be read in the file to be found loaded into the computing unit 810; and return the result of the search to the host. The storage device 800 may include at least one processor, and the at least one processor is configured to implement the computing unit 810. For example, the processor may be an embedded processor, such as an FPGA embedded in the storage device. In an embodiment, the storage device 800 may correspond to the CSD 320 discussed above, and the computing unit 810 may correspond to the computing unit 321 discussed above, but the embodiment is not limited thereto.

[0121] According to an embodiment of the present disclosure, the file to be searched may be an ordered character table SST file stored hierarchically in a database based on LSM-Tree.

[0122] According to an embodiment of the present disclosure, the computing unit 810 may also be configured to: load the index block and the filter block in the SST file to be searched into the computing unit 810, wherein the index block includes the index information of the data block in the SST file to be searched; determine the data block where the data to be read may be located based on the index block loaded into the computing unit 810; determine whether the data to be read exists in the data block based on the filter block loaded into the computing unit 810; in response to the data to be read existing in the data block, load the data block in the SST file to be searched into the computing unit 810; and search for the data to be read in the data block loaded into the computing unit 810.

[0123] According to an embodiment of the present disclosure, the computing unit 810 may also be configured to: find the data block where the data to be read may be located in the index block through binary search, and based on the filter block loaded into the computing unit 810, check whether the data to be read exists in the data block through a Bloom filter.

[0124] In an embodiment of the present disclosure, the computing unit 810 may include a binary search module and a Bloom filter module. The binary search module may be configured to find a data block where the to-be-read data may be located in the index block through a binary search, and to find the to-be-read data in the data block loaded into the computing unit 810. The Bloom filter module may be configured to check whether the to-be-read data exists in the data block through a Bloom filter based on the filter block loaded into the computing unit 810.

[0125] According to an embodiment of the present disclosure, the calculation unit 810 may also be configured to: calculate multiple hash values ​​of the key Key of the data to be read using multiple hash functions; and check whether the data to be read exists in the data block by matching the multiple hash values ​​with the data structure in the filter block.

[0126] According to an embodiment of the present disclosure, the computing unit 810 may also be configured to: in response to finding the data to be read in the file to be found, return the address of the data to be read to the host; in response to not finding the data to be read in the file to be found, return a message indicating that the data was not found to the host.

[0127] According to an embodiment of the present disclosure, the computing unit 810 may also be configured to: after returning a message indicating that the data has not been found to the host, in response to receiving a search request from the host including the address of the file to be found, return an operation of loading the file to be found into the computing unit 810 based on the address of the file to be found included in the search request received from the host, wherein the address of the file to be found is the address of a new file to be found.

[0128] According to an embodiment of the present disclosure, when a message indicating that the data is not found is received by the host and the file to be found has been traversed, or when the address of the data to be read is received by the host, the search for the data to be read can be ended.

[0129] According to an embodiment of the present disclosure, the search request may be sent by the host by calling a read interface in the database.

[0130] According to an embodiment of the present disclosure, the computing unit 810 may include a dynamic random access memory DRAM, and the computing unit 810 may also be configured to: load the file to be searched into the DRAM.

[0131] Therefore, according to the embodiments of the present disclosure, the high bandwidth in the storage device 800 can be fully utilized to load the file to be searched into the storage device, which can greatly reduce the data movement between the storage device 800 and the host, and can significantly reduce the read amplification. In addition, the binary search module and the Bloom filter module implemented in the storage device 800 can be used to search for the file to be searched, which can improve the search efficiency and reduce the CPU occupancy and the power consumption of the host.

[0132] Fig. 9 A schematic diagram of a system 1000 to which a storage device is applied according to an exemplary embodiment of the present disclosure.

[0133] Fig. 9The system 1000 may basically be a mobile system, such as a portable communication terminal (e.g., a mobile phone), a smart phone, a tablet personal computer (PC), a wearable device, a healthcare device, or an Internet of Things (IOT) device. However, Fig. 9 The system 1000 is not necessarily limited to a mobile system, and may be a PC, a laptop, a server, a media player, or an automotive device (eg, a navigation device).

[0134] Reference Fig. 9 , the system 1000 may include a main processor 1100, a memory (e.g., 1200a and 1200b), and a storage device (e.g., 1300a and 1300b). In addition, the system 1000 may include at least one of an image capture device 1410, a user input device 1420, a sensor 1430, a communication device 1440, a display 1450, a speaker 1460, a power supply device 1470, and a connection interface 1480.

[0135] The main processor 1100 may control all operations of the system 1000, and more specifically, may control operations of other components included in the system 1000. The main processor 1100 may be implemented as a general-purpose processor, a dedicated processor, an application processor, or the like.

[0136] The main processor 1100 may include at least one central processing unit (CPU) core 1110, and also include a controller 1120 for controlling memories 1200a and 1200b and / or storage devices 1300a and 1300b. In some exemplary embodiments, the main processor 1100 may further include an accelerator 1130, which may be a dedicated circuit for high-speed data operations such as artificial intelligence (AI) data operations. The accelerator 1130 may include a graphics processing unit (GPU), a neural processing unit (NPU), and / or a data processing unit (DPU), etc., and is implemented as a chip physically separated from other components of the main processor 1100.

[0137] The memories 1200a and 1200b may be used as main storage devices of the system 1000. Although the memories 1200a and 1200b may respectively include volatile memories such as static random access memory (SRAM) and / or DRAM, etc., the memories 1200a and 1200b may respectively include non-volatile memories such as flash memory, phase change random access memory (PRAM) and / or resistive random access memory (RRAM), etc. The memories 1200a and 1200b may be implemented in the same package as the main processor 1100.

[0138] The storage devices 1300a and 1300b may be used as non-volatile storage devices, which are configured to store data regardless of whether they are powered on and have a larger storage capacity than the memories 1200a and 1200b. The storage devices 1300a and 1300b may include memory controllers (shown as STRG CTRL) 1310a and 1310b, respectively, and non-volatile memories (NVMs) 1320a and 1320b, which are configured to store data via the control of the memory controllers 1310a and 1310b. Although the NVMs 1320a and 1320b may include vertical NAND (V-NAND) flash memories having a two-dimensional (2D) or three-dimensional (3D) structure, the NVMs 1320a and 1320b may include other types of NVMs, such as PRAM and / or RRAM, etc.

[0139] The storage devices 1300a and 1300b may be physically separated from the main processor 1100 and included in the system 1000, or may be implemented in the same package as the main processor 1100. In addition, the storage devices 1300a and 1300b may have a type of solid-state devices (SSDs) or memory cards, and may be removably combined with other components of the system 1000 through an interface such as a connection interface 1480 to be described later. The storage devices 1300a and 1300b may be devices to which a standard protocol such as Universal Flash Storage (UFS), Embedded Multimedia Card (eMMC), or NVMe may be applied, but are not limited thereto.

[0140] The image capture device 1410 can capture still images or moving images. The image capture device 1410 may include a camera, a camcorder, and / or a webcam, etc.

[0141] The user input device 1420 may receive various types of data input by a user of the system 1000 and includes a touch pad, a keypad, a keyboard, a mouse, a microphone, and the like.

[0142] The sensor 1430 may detect various types of physical quantities that may be obtained from the outside of the system 1000 and convert the detected physical quantities into electrical signals. The sensor 1430 may include a temperature sensor, a pressure sensor, an illumination sensor, a position sensor, an acceleration sensor, a biosensor, and / or a gyro sensor, etc.

[0143] The communication device 1440 may transmit and receive signals between other devices outside the system 1000 according to various communication protocols. The communication device 1440 may include an antenna, a transceiver, a modem, or the like.

[0144] Display 1450 and speaker 1460 may be used as output devices configured to output visual information and auditory information, respectively, to a user of system 1000 .

[0145] The power supply device 1470 may appropriately convert power supplied from a battery (not shown) embedded in the system 1000 and / or an external power source, and supply the converted power to each component of the system 1000 .

[0146] The connection interface 1480 may provide a connection between the system 1000 and an external device that is connected to the system 1000 and is capable of transmitting and receiving data to and from the system 1000. The connection interface 1480 may be implemented by using various interface schemes, for example, Advanced Technology Attachment (ATA), Serial ATA (SATA), External Serial ATA (e-SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Peripheral Component Interconnect (PCI), PCI express (PCIe), NVMe, IEEE 1394, Universal Serial Bus (USB) interface, Secure Digital (SD) card interface, Multimedia Card (MMC) interface, Embedded Multimedia Card (eMMC) interface, UFS interface, Embedded UFS (eUFS) interface, Compact Flash (CF) card interface, and the like.

[0147] According to an embodiment of the present disclosure, a system (e.g., system 1000) using a storage device is provided, including: a main processor (e.g., main processor 1100); a memory (e.g., memory 1200a and 1200b); and a storage device (e.g., storage devices 1300a and 1300b), wherein the storage device is configured to execute the method for reading data from the storage device as described above.

[0148] Fig.10 is a block diagram of a host storage system 10 according to an exemplary embodiment.

[0149] The host storage system 10 may include a host 101 and a storage device 200. In addition, the storage device 200 may include a memory controller 210 and an NVM 220. According to an exemplary embodiment of the present invention, the host 101 may include a host controller 110 and a host memory 120. The host memory 120 may be used as a buffer memory configured to temporarily store data to be transmitted to or received from the storage device 200.

[0150] The storage device 200 may include a storage medium configured to store data in response to a request from the host 101. As an example, the storage device 200 may include at least one of an SSD, an embedded memory, and a removable external memory. When the storage device 200 is an SSD, the storage device 200 may be a device that complies with the NVMe standard. When the storage device 200 is an embedded memory or an external memory, the storage device 200 may be a device that complies with the UFS standard or the eMMC standard. Both the host 101 and the storage device 200 may generate a packet and send the packet according to the adopted standard protocol.

[0151] When the NVM 220 of the storage device 200 includes a flash memory, the flash memory may include a 2D NAND memory array or a 3D NAND (or V-NAND) memory array. As another example, the storage device 200 may include various other types of NVM. For example, the storage device 200 may include a magnetic random access memory (MRAM), a spin transfer torque MRAM, a conductive bridge RAM (CBRAM), a ferroelectric RAM (FRAM), a PRAM, a RRAM, and various other types of memory.

[0152] According to an exemplary embodiment, the host controller 110 and the host memory 120 may be implemented as separate semiconductor chips. In some exemplary embodiments, the host controller 110 and the host memory 120 may be integrated in the same semiconductor chip. As an example, the host controller 110 may be any one of a plurality of modules included in an application processor (AP). The AP may be implemented as a system on chip (SoC). In addition, the host memory 120 may be an embedded memory included in the AP or a memory module outside the AP.

[0153] The host controller 110 may manage an operation of storing data (eg, write data) of the buffer area of ​​the host memory 120 in the NVM 220 or an operation of storing data (eg, read data) of the NVM 220 in the buffer area.

[0154] The memory controller 210 may include a host interface 211, a memory interface 212, and a CPU 213. In addition, the memory controller 210 may further include a flash translation layer (FTL), a packet manager 215, a buffer memory 216, an error correction code (ECC) engine 217, and an advanced encryption standard (AES) engine 218. The memory controller 210 may further include a working memory (not shown) in which the FTL 214 is loaded. The CPU 213 may execute the FTL 214 to control data write and read operations on the NVM 220.

[0155] The host interface 211 may transmit and receive packets to and from the host 101. The packets transmitted from the host 101 to the host interface 211 may include commands or data to be written to the NVM 220, etc. The packets transmitted from the host interface 211 to the host 101 may include responses to commands or data read from the NVM 220, etc. The memory interface 212 may transmit data to be written to the NVM 220 to the NVM 220 or receive data read from the NVM 220. The memory interface 212 may be configured to comply with a standard protocol such as Toggle or Open NAND Flash Interface (ONFI).

[0156] The FTL 214 may perform various functions, such as an address mapping operation, a wear leveling operation, and a garbage collection operation. The address mapping operation may be an operation of converting a logical address received from the host 101 into a physical address for actually storing data in the NVM 220. The wear leveling operation may be a technique for preventing excessive degradation of a specific block by allowing the blocks of the NVM 220 to be used evenly. As an example, the wear leveling operation may be implemented by using a firmware technique that balances the erase counts of physical blocks. The garbage collection operation may be a technique for ensuring available capacity in the NVM 220 by erasing an existing block after copying valid data of the existing block to a new block.

[0157] The packet manager 215 may generate a packet according to a protocol agreed to an interface of the host 101, or parse various types of information from a packet received from the host 101. In addition, the buffer memory 216 may temporarily store data to be written to or read from the NVM 220. Although the buffer memory 216 may be a component included in the memory controller 210, the buffer memory 216 may be outside the memory controller 210.

[0158] The ECC engine 217 may perform error detection and correction operations on read data read from the NVM 220. More specifically, the ECC engine 217 may generate parity bits for write data to be written to the NVM 220, and the generated parity bits may be stored together with the write data in the NVM 220. During reading of data from the NVM 220, the ECC engine 217 may correct errors in the read data by using the read data and the parity bits read from the NVM 220, and output the error-corrected read data.

[0159] The AES engine 218 may perform at least one of an encryption operation and a decryption operation on data input to the memory controller 210 by using a symmetric key algorithm.

[0160] According to an embodiment of the present disclosure, a host storage system (e.g., host storage system 10) is provided, including: a host (e.g., 100); and a storage device (e.g., storage device 200), wherein the storage device is configured to execute the method for reading data for the storage device as described above.

[0161] Fig.11 FIG. 3 is a diagram of a data center 3000 to which a storage device may be applied according to an exemplary embodiment of the present invention.

[0162] Reference Fig.11 , the data center 3000 may be a facility that collects various types of data and provides services, and is referred to as a data storage center. The data center 3000 may be a system for operating a search engine and a database, and may be a computing system used by a company (such as a bank) or a government agency. The data center 3000 may include application servers 3100 to 3100n and storage servers 3200 to 3200m. According to an exemplary embodiment, the number of applications 3100 to 3100n and the number of storage servers 3200 to 3200m may be selected differently. The number of application servers 3100 to 3100n and the number of storage servers 3200 to 3200m may be different from each other.

[0163] The application server 3100 or the storage server 3200 may include at least one of the processors 3110 and 3210 and the memories 3120 and 3220. Now, the storage server 3200 will be described as an example. The processor 3210 may control all operations of the storage server 3200, access the memory 3220, and execute instructions and / or data loaded into the memory 3220. The memory 3220 may be a double data rate synchronous DRAM (DDR SDRAM), a high bandwidth memory (HBM), a hybrid memory cube (HMC), a dual in-line memory module (DIMM), an Optane DIMM, or a non-volatile DIMM (NVMDIMM). In some exemplary embodiments, the number of processors 3210 and memories 3220 included in the storage server 3200 may be selected differently. In an exemplary embodiment, the processor 3210 and the memory 3220 may provide a processor-memory pair. In an exemplary embodiment, the number of processors 3210 and the number of memories 3220 may be different from each other. The processor 3210 may include a single-core processor or a multi-core processor. The above description of the storage server 3200 may be similarly applied to the application server 3100. In some embodiments, the application server 3100 may not include the storage device 3150. The storage server 3200 may include at least one storage device 3250. According to the embodiment, the number of storage devices 3250 included in the storage server 3200 may be selected differently.

[0164] The application servers 3100 to 3100n can communicate with the storage servers 3200 to 3200m through the network 3300. The network 3300 can be implemented by using a fiber channel (FC) or Ethernet. In this case, FC can be a medium for relatively high-speed data transmission, and an optical switch with high performance and high availability can be used. According to the access method of the network 3300, the storage servers 3200 to 3200m can be set to file storage, block storage, or object storage.

[0165] In one embodiment, the network 3300 may be a network dedicated to storage, such as a storage area network (SAN). For example, the SAN may be an FC-SAN, which uses an FC network and is implemented according to the FC protocol (FCP). As another example, the SAN may be an Internet Protocol (IP)-SAN, which uses a transmission control protocol (TCP) / IP network and is implemented according to the SCSI or Internet SCSI (iSCSI) protocol over TCP / IP. In another embodiment, the network 3300 may be a general network, such as a TCP / IP network. For example, the network 3300 may be implemented according to protocols such as FC over Ethernet (FCoE), network attached storage (NAS), and NVMe over fabric (NVMe-oF).

[0166] Hereinafter, examples of the application server 3100 and the storage server 3200 are described. The description of the application server 3100 may be applied to another application server 3100n, and the description of the storage server 3200 may be applied to another storage server 3200m.

[0167] The application server 3100 may store data requested to be stored by a user or client in one of the storage servers 3200 to 3200m through the network 3300. In addition, the application server 3100 may obtain data requested to be read by a user or client from one of the storage servers 3200 to 3200m through the network 3300. For example, the application server 3100 may be implemented as a network server or a database management system (DBMS).

[0168] The application server 3100 may access a memory 3120n or a storage device 3150n included in another application server 3100n through the network 3300. Alternatively, the application server 3100 may access a memory 3220 to 3220m or a storage device 3250 to 3250m included in the storage servers 3200 to 3200m through the network 3300. Therefore, the application server 3100 may perform various operations on the data stored in the application servers 3100 to 3100n and / or the storage servers 3200 to 3200m. For example, the application server 3100 may execute an instruction for moving or copying data between the application servers 3100 to 3100n and / or the storage servers 3200 to 3200m. In this case, data may be moved from the storage devices 3250 to 3250m of the storage servers 3200 to 3200m through the storage 3220 to 3220m of the storage servers 3200 to 3200m or directly to the storage 3120 to 3120n of the application servers 3100 to 3100n. The data moved through the network 3300 may be data encrypted for security or privacy.

[0169] The storage server 3200 is described below. The interface 3254 can provide a physical connection between the processor 3210 and the controller 3251 and a physical connection between the network interface card (NIC) 3240 and the controller 3251. For example, the interface 3254 can be implemented using a direct attached storage (DAS) solution, in which the storage device 3250 is directly connected to a dedicated cable. For example, the interface 3254 can be implemented using various interface solutions, such as ATA, SATA, e-SATA, SCSI, SAS, PCI, PCIe, NVMe, IEEE 1394, USB interface, SD card interface, MMC interface, eMMC interface, UFS interface, eUFS interface, and CF card interface.

[0170] The storage server 3200 may further include a switch 3230 and a network interconnect (NIC) 3240. The switch 3230 may selectively connect the processor 3210 to the storage device 3250 or selectively connect the NIC 3240 to the storage device 3250 via the control of the processor 3210.

[0171] In one embodiment, the NIC 3240 may include a network interface card and a network adapter. The NIC 3240 may be connected to the network 3300 through a wired interface, a wireless interface, a Bluetooth interface, or an optical interface. The NIC 3240 may include an internal memory, a digital signal processor (DSP), and a host bus interface, and is connected to the processor 3210 and / or the switch 3230 through the host bus interface. The host bus interface may be implemented as one of the above examples of the interface 3254. In one embodiment, the NIC 3240 may be integrated with at least one of the processor 3210, the switch 3230, and the storage device 3250.

[0172] In the storage servers 3200 to 3200m or the application servers 3100 to 3100n, the processor may send a command to the storage devices 3150 to 3150n and 3250 to 3250m or the memories 3120 to 3120n and 3220 to 3220m and program or read data. In this case, the data may be data in which an error is corrected by an ECC engine. The data may be data on which a data bus inversion (DBI) operation or a data masking (DM) operation is performed, and may include cyclic redundancy coding (CRC) information. The data may be data encrypted for security or privacy.

[0173] The storage devices 3150 to 3150n and 3250 to 3250m may send control signals and command / address signals to the NAND flash memory devices 3252 to 3252m in response to a read command received from the processor. Therefore, when data is read from the NAND flash memory devices 3252 to 3252m, a read enable (RE) signal may be input as a data output control signal, and thus, the data may be output to the DQ bus. The data strobe signal DQS may be generated using the RE signal. Depending on the rising edge or falling edge of the write enable (WE) signal, the command and address signals may be latched in the page buffer.

[0174] The controller 3251 may control all operations of the storage device 3250. In one embodiment, the controller 3251 may include an SRAM. The controller 3251 may write data to the NAND flash memory device 3252 in response to a write command, or read data from the NAND flash memory device 3252 in response to a read command. For example, a write command and / or a read command may be provided from a processor 3210 of the storage server 3200, a processor 3210m of another storage server 3200m, or processors 3110 and 3110n of application servers 3100 and 3100n. The DRAM 3253 may temporarily store (or buffer) data to be written to the NAND flash memory device 3252 or data to be read from the NAND flash memory device 3252. And, the DRAM 3253 may store metadata. Here, the metadata may be user data or data generated by the controller 3251 for managing the NAND flash memory device 3252. The storage device 3250 may include a secure element (SE) for security or privacy.

[0175] According to an exemplary embodiment of the present disclosure, a data center system (e.g., data center system 3000) is provided, comprising: a plurality of application servers (e.g., application servers 3100 to 3100n); and a plurality of storage servers (e.g., storage servers 3200 to 3200m), wherein each storage server comprises a storage device, wherein the storage device is configured to execute the method for reading data of the storage device as described above.

[0176] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the method for reading data from a storage device as described above is implemented. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), SSD, card storage (such as, for example, multimedia card, secure digital (SD) card and / or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and / or any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-temporary manner and provide the computer programs and any associated data, data files and / or data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In one example, the computer program and any associated data, data files and / or data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0177] In the storage device and the method for reading data from the storage device according to the embodiment of the present disclosure, the high bandwidth in the storage device can be fully utilized to load the file to be searched into the storage device, which can greatly reduce the data movement between the storage device and the host, and can significantly reduce the read amplification. In addition, the BS module and the BF module implemented in the storage device can be used to search for the file to be searched, which can not only improve the search efficiency, but also reduce the CPU occupancy rate and the power consumption of the host.

[0178] While some embodiments of the present disclosure have been particularly shown and described above, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A method for reading data from a storage device, the method comprising: receiving a search request from a host, the search request including an address of a file to be searched; Based on the address, loading the file into a computing unit included in the storage device; Obtaining the search result by searching the file for the data to be read; as well as The result of the search is returned to the host.

2. The method of claim 1, wherein: The file includes an ordered character table SST file stored hierarchically in a database based on a log structure merge tree LSM-Tree.

3. The method of claim 2, wherein: Load the file, including: Loading the index block and filter block included in the SST file into the computing unit, The index block includes index information about the data block in the SST file, and Wherein, searching for the data in the file includes: Based on the index block, determining a candidate data block; Based on the filter block, determining whether the data exists in the candidate data block; Based on determining that the data exists in the candidate data block, loading the candidate data block into the computing unit; and The data is searched in the candidate data blocks.

4. The method of claim 3, wherein: The candidate data blocks are determined using a binary search in the index block, and Wherein, determining whether the data exists in the candidate data block includes: Based on the filter block, a Bloom filter is used to check whether the data exists in the candidate data block.

5. The method of claim 4, wherein: Checking whether the data exists in the candidate data block includes: Calculating multiple hash values ​​corresponding to the key Key of the data using multiple hash functions; and The plurality of hash values ​​are matched to a data structure in the filter block.

6. The method of claim 1, wherein: The search result is returned to the host, including: Based on finding the data in the file, returning the address of the data to the host; Based on not finding the data in the file, a message indicating that the data was not found is returned to the host.

7. The method of claim 6, further comprising: After returning the message to the host, receiving a new search request from the host, wherein the new search request includes a new address of a new file to be searched; and Based on the new address, the new file is loaded into the computing unit.

8. The method of claim 7, wherein: Based on the message being received by the host and the file having been traversed, or based on the address of the data being received by the host, the search for the data is completed.

9. The method of claim 2, wherein: The search request is sent by the host by calling a read interface in the database.

10. The method of claim 1, wherein: The computing unit includes a dynamic random access memory DRAM, and Wherein, loading the file into the storage device includes: loading the file into the DRAM.

11. A storage device comprising: At least one processor is configured to implement a computing unit, Wherein, the computing unit is configured as follows: Obtaining a search request received by the storage device from a host, wherein the search request includes an address of a file to be searched, Based on the address, loading the file into the computing unit, Obtaining the search result by searching the file for the data to be read, and The result of the search is returned to the host.

12. A storage system comprising: Main processor; Memory; as well as Storage devices, Wherein, the storage device is configured as: Receiving a search request from a host, wherein the search request includes an address of a file to be searched; Based on the address, loading the file into a computing unit included in the storage device; Obtaining a search result by searching the file for the data to be read; and The result of the search is returned to the host.

13. A host storage system, comprising: Host; as well as Storage devices, Wherein, the storage device is configured as: Receiving a search request from a host, wherein the search request includes an address of a file to be searched; Based on the address, loading the file into a computing unit included in the storage device; Obtaining a search result by searching the file for the data to be read; and The result of the search is returned to the host.

14. A data center system, comprising: Multiple application servers; as well as Multiple storage servers, Each storage server includes a storage device, and Wherein, the storage device is configured as: Receiving a search request from a host, wherein the search request includes an address of a file to be searched; Based on the address, loading the file into a computing unit included in the storage device; Obtaining a search result by searching the file for the data to be read; and The result of the search is returned to the host.

15. A computer-readable storage medium storing computer program instructions, wherein: When the computer program instructions are executed by at least one processor of the storage device, the storage device is caused to perform the following operations: Receiving a search request from a host, wherein the search request includes an address of a file to be searched; Based on the address, loading the file into a computing unit included in the storage device; Obtaining a search result by searching the file for the data to be read; and The result of the search is returned to the host.

Images