Storage device and storage system

Abstract

The present disclosure relates to a storage device and a storage system, the device includes a memory, a storage controller, a control module, and a cache module, wherein the control module is configured to: upon receiving a data read / write instruction, update the data read / write instruction using a virtual address; apply for K cache units in the cache module, and perform write operations on the K cache units using the data to be read / written; and directly start the data transmission of the cache unit that is filled when any of the K cache units is full. The embodiments of the present disclosure use virtual storage addresses and data block management to achieve the update and forwarding of NVMe I / O commands without applying for storage space in advance, start the transmission of data blocks without checking the completion message, optimize the data processing flow, reduce the storage time of data in the local cache, reduce the memory requirements of the storage server, and reduce the completion delay of NVMe I / O commands.

Description

Technical Field

[0001] The present disclosure relates to the field of storage technology, and in particular to a storage device and a storage system. Background Art

[0002] With the improvement of PCIe performance and the decrease of the cost of solid-state drive particles, the NVMe standard based on PCIe architecture is becoming more and more mainstream in the consumer market. Now more and more companies expect to use hard disks based on the NVMe standard in data centers and enterprise fields. However, due to the large amount of data in data centers, the flexible access of multiple hosts and the need for remote access, the industry has introduced the NVMe over Fabric standard, which allows the NVMe protocol to run on network protocols, including Ethernet, Fiber Channel networks, etc., thereby expanding the flexibility of the NVMe protocol.

[0003] In the related technical solutions, in order to ensure efficient transmission performance, the storage server requires a huge memory space. In addition, in the related technology, the completion delay of the read and write commands is relatively high, and if the storage server uses a processor to complete command conversion and interrupt processing, the processor utilization rate will be very high. Summary of the invention

[0004] According to one aspect of the present disclosure, a storage device is provided, the device being applied to a storage server based on an NVMe overFabric architecture, the storage device comprising a memory, a storage controller, a control module, and a cache module, the memory being used to store data to be read and written by a host in the architecture, the storage controller being used to implement read and write control of the memory, the control module being used to receive data read and write instructions, the cache module being divided into a plurality of cache units, wherein the control module is configured as follows:

[0005] When receiving a data read / write instruction, updating the data read / write instruction using a virtual address;

[0006] Applying for K cache units in the cache module according to the updated data read and write instructions, and performing write operations on the K cache units using the data to be read and written, wherein K is less than the total number of cache units in the cache module and is an integer;

[0007] When any cache unit among the K cache units is full, data transmission of the full cache unit is directly started, and when the data transmission is completed, the full cache unit is released.

[0008] In a possible implementation manner, the control module is further configured to:

[0009] Establishing a mapping relationship between the virtual address and the cache unit;

[0010] When the data transmission of the full cache unit is completed, the mapping relationship between the virtual address and the full cache unit is released.

[0011] In a possible implementation manner, the updating the data read / write instruction by using the virtual address includes: replacing the host memory address in the data pointer in the data read / write instruction by using the virtual address;

[0012] The control module is further configured to establish a mapping relationship between the virtual address and the data read and write instructions.

[0013] In a possible implementation manner, the control module is further configured to:

[0014] When the i-th cache unit among the K cache units is full, writing is performed on the i+m-th cache unit among the K cache units, wherein i and m are less than K and are integers, and i+m≤K;

[0015] When the (i+m)th cache unit is full and the (i)th cache unit completes data transmission and is released, writing is performed on the (i)th cache unit.

[0016] In a possible implementation manner, the control module is further configured to:

[0017] When receiving the read / write completion message from the storage controller, the read / write completion message is transmitted to the host.

[0018] In a possible implementation manner, the data read and write instruction is a data read instruction, and the data read instruction is used to read the to-be-read data from the memory, and the control module is further configured to:

[0019] Sending an updated data read instruction to the storage controller;

[0020] In case of receiving a write operation of the memory in response to the updated data read instruction, applying for K cache units in the cache module, and writing the to-be-read data in the memory into the K cache units in sequence;

[0021] When any cache unit among the K cache units is full, the data transmission of the full cache unit is directly started, the data of the full cache unit is transmitted to the host, and when the data transmission is completed, the full cache unit is released, and the mapping relationship between the virtual address and the full cache unit is released.

[0022] In a possible implementation, the cache module further includes a cache manager, the cache manager is used to manage each cache unit, the data read and write instruction is a data write instruction, the data write instruction is used to write the data to be written in the memory, and the control module is further configured to:

[0023] Sending updated data write instructions to the storage controller and the cache manager;

[0024] Controlling the cache manager to predict a target data request sequence of the storage controller in response to the updated data write instruction according to the sequence of the updated data write instruction and / or the sequence of data requests received from the storage controller;

[0025] Control the cache manager to read the data to be written from the host according to the target data request sequence and apply for K cache units, and write the data to be written in the host into the K cache units in sequence;

[0026] When any cache unit among the K cache units is full, data transmission of the full cache unit is directly initiated in response to a read request of the storage controller, the data of the full cache unit is transferred to the memory for storage, and when the data transmission is completed, the full cache unit is released, and the mapping relationship between the virtual address and the full cache unit is released.

[0027] In a possible implementation manner, the control module is further configured to:

[0028] If the data corresponding to the read request of the storage controller in response to the updated data write instruction is not written into the corresponding cache unit, the data is read from the host to the cache unit using a preset channel.

[0029] In a possible implementation manner, the control module is further configured to:

[0030] If the number of free cache units of the cache module is reduced to a preset number of free units, data transmission of a preset number of cache units being written is directly started, so that the number of free cache units of the cache module reaches the preset number of free units.

[0031] In a possible implementation, the storage sizes of the cache units in the cache module are the same and are integer multiples of a preset size.

[0032] In a possible implementation manner, the preset size is 4096 bytes.

[0033] According to one aspect of the present disclosure, a storage system is provided, the system comprising a host and a storage device, the system is implemented based on the NVMe over Fabric architecture, the device is applied to a storage server of the architecture, the storage device comprises a memory, a storage controller, a control module, and a cache module, the host reads and writes data in the storage device through data read and write instructions, the memory is used to store data to be read and written by the host, the storage controller is used to implement read and write control of the memory, the control module is used to receive data read and write instructions, and the cache module is divided into a plurality of cache units, wherein the control module is configured as follows:

[0034] When receiving a data read / write instruction, the data read / write instruction is updated using a virtual address; according to the updated data read / write instruction, K cache units are applied for in the cache module, and a write operation is performed on the K cache units using the data to be read / written, wherein K is less than the total number of cache units in the cache module and is an integer;

[0035] When any cache unit among the K cache units is full, data transmission of the full cache unit is directly started, and when the data transmission is completed, the full cache unit is released.

[0036] A storage device proposed in an embodiment of the present disclosure is applied to a storage server based on an NVMe over Fabric architecture. The storage device includes a memory, a storage controller, a control module, and a cache module. The memory is used to store data to be read and written by a host in the architecture. The storage controller is used to implement read and write control of the memory. The control module is used to receive data read and write instructions. The cache module is divided into multiple cache units, wherein the control module is configured to: upon receiving a data read and write instruction, update the data read and write instruction using a virtual address; apply for K cache units in the cache module according to the updated data read and write instruction, and perform write operations on the K cache units using the data to be read and written, wherein K is less than the total number of cache units in the cache module and is an integer; when any cache unit among the K cache units is full, directly start data transmission of the full cache unit, and release the full cache unit when the data transmission is completed.

[0037] The disclosed embodiment uses a virtual storage address and divides the data into blocks for management by dividing the data into multiple cache units, thereby realizing that the NVMe I / O command can be updated and forwarded without applying for storage space in advance, and the transmission of the data blocks can be started without checking the completion message, thereby optimizing the data processing flow and reducing the storage time of the data in the local cache, thereby reducing the memory demand on the storage server side, and also reducing the completion delay of the NVMe I / O command.

[0038] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0040] Figure 1 A schematic diagram of the NVMe over Fabric architecture in the related technical solution is shown.

[0041] Figure 2 A block diagram of a storage device according to an embodiment of the present disclosure is shown.

[0042] Figure 3 A schematic diagram of a read and write process of a storage device according to an embodiment of the present disclosure is shown.

[0043] Figure 4 A schematic diagram of partitioning a cache unit according to an embodiment of the present disclosure is shown.

[0044] Figure 5 A schematic diagram of an NVMe over Fabric architecture according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0045] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0046] In the description of the present disclosure, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure.

[0047] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0048] In the present disclosure, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.

[0049] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0050] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0051] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0052] First, Table 1 is used to exemplarily introduce the definitions of various abbreviations and key terms in the present disclosure.

[0053] Table 1

[0054]

[0055] See also Figure 1 , Figure 1 A schematic diagram of the NVMe over Fabric architecture in the related technical solution is shown.

[0056] Among the related technical solutions, the current NVMe (Non-Volatile Memory Express, non-volatile memory host controller interface specification) over Fabric architecture has both software architecture solutions and software-hardware combination solutions.

[0057] like Figure 1 As shown, the processor 101 of the host 100 sends an NVMe write I / O command to the processor 201 of the storage server 200 through the network adapter card 103. The processor 201 receives the NVMe write I / O command and applies for local storage space in the memory 202 according to the data length of the I / O command. After that, the processor 201 reads the data to the local storage space in the memory 202 of the storage server 200 through the network adapter card 203. Then the processor 201 will update the Data Pointer related fields in the NVMe write I / O command, change the data address pointed to by the field from the host memory address to the address of the local memory, and send the updated NVMe I / O command to the target NVMe controller. The NVMe controller receives the corresponding NVMe write I / O command, reads the data from the memory 202 of the storage server 200 and writes the data to the solid-state drive. After the NVMe controller completes the storage of the data, it will return a completion message to the completion queue. The processor 201 of the storage server 200 then transmits the entire completion message to the processor 101 of the host 100, and the entire NVMe write I / O command process is completed.

[0058] For the NVMe read I / O command process of the host 100, the processor 101 of the host 100 sends the NVMe read I / O command to the processor 201 of the storage server 200 through the network adapter card 103. The processor 201 receives the NVMe read I / O command and applies for local storage space in the memory 202 according to the data length of the I / O command. Then the processor 201 updates the Data Pointer related fields in the NVMe read I / O command, changes the data address pointed to by the field from the host memory address to the local memory address, and sends the updated NVMe I / O command to the target NVMe controller. The NVMe controller reads the corresponding data from the solid-state drive and writes the data to the storage space in the memory 202 of the storage server 200. When the NVMe controller completes the writing of the data, it sends a completion message to the processor 201. The processor 201 of the storage server 200 sends the data in the memory 202 to the memory 102 of the host 100 through the network adapter card 203. After completing the data transmission, the processor 201 of the storage server 200 transmits the completion message to the processor 101 of the host 100, and a complete NVMe read I / O command is completed.

[0059] It can be seen that in the relevant technical scheme, in order to ensure efficient transmission performance, the storage server 200 requires a huge memory space. If the storage server 200 uses the processor 201 to complete command conversion and interrupt processing, the utilization rate of the processor 201 will be very high. In addition, for NVMe write I / O commands, if the storage server 200 receives an NVMe I / O write command, it is necessary to first apply for storage space for the data of the NVMe I / O command in the memory 202, read all or part of the data from the memory 102 of the host 100 into the local memory 202, and then send the updated NVMe I / O write command to the target NVMe controller; for NVMe read I / O commands, if the processor 201 of the storage server 200 receives an NVMe I / O read command, it is necessary to first apply for storage space for the data of the NVMe I / O command in the local memory 202, and then send the updated NVMe I / O read command to the target NVMe controller. The related art migrates the data in the memory 202 to the memory 102 of the host 100 only after the NVMe controller writes all the data into the memory 202 of the storage server 200 and the storage server 200 receives the completion message of the corresponding NVMe read I / O command sent by the NVMe controller. However, such an operation mode of the related art will cause the NVMe I / O command and data to be temporarily stored in the memory 202 of the storage server 200 for a long time, which greatly increases the completion delay of the NVMe I / O command and cannot meet the transmission requirements of applications that are sensitive to data delay.

[0060] In the related art, there is also an NVMe controller that splits the received NVMe I / O command into multiple sub-NVMe I / O commands and sends them to the solid-state drive. However, in the related art, since the split command may cause atomic operations of some NVMe commands, some additional designs are required. At the same time, the split I / O command and the completion messages of multiple sub-I / O commands need to be temporarily stored, which increases the complexity of the design, and this solution will not reduce the operation delay of the I / O command.

[0061] The present disclosure provides a storage device, which is applied to a storage server based on an NVMe over Fabric architecture. The storage device includes a memory, a storage controller, a control module, and a cache module. The memory is used to store data to be read and written by a host in the architecture, the storage controller is used to implement read and write control of the memory, the control module is used to receive data read and write instructions, and the cache module is divided into multiple cache units, wherein the control module is configured to: upon receiving a data read and write instruction, update the data read and write instruction using a virtual address; apply for K cache units in the cache module according to the updated data read and write instruction, and perform write operations on the K cache units using the data to be read and written, wherein K is less than the total number of cache units in the cache module and is an integer; when any cache unit among the K cache units is full, directly start data transmission of the full cache unit, and release the full cache unit when the data transmission is completed.

[0062] The disclosed embodiment uses a virtual storage address and divides the data into multiple cache units for data block management, thereby achieving the need to update and forward NVMe I / O commands without applying for storage space in advance, and starting the transmission of data blocks without checking the completion message, thereby optimizing the data processing flow, reducing the storage time of data in the local cache, thereby reducing the memory demand on the storage server side, and also reducing the completion delay of NVMe I / O commands, and reducing the usage frequency and load of the processor.

[0063] Figure 2 A block diagram of a storage device according to an embodiment of the present disclosure is shown.

[0064] Figure 3 A schematic diagram of a read and write process of a storage device according to an embodiment of the present disclosure is shown.

[0065] The host 10 may be a terminal device, and the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a handheld device, a computing device or a vehicle-mounted device, etc. For example, some examples of terminals are: mobile phones, tablet computers, laptop computers, PDAs, mobile Internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control (Industrial Control), wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids (Smart Grid), wireless terminals in transportation safety (Transportation Safety), wireless terminals in smart cities (Smart City), wireless terminals in smart homes (Smart Home), wireless terminals in Internet of Vehicles, etc.

[0066] In some possible implementations, the control module 240 can be implemented by a processing component. In one example, the processing component includes but is not limited to a separate processor, or a discrete component, or a combination of a processor and a discrete component. The processor may include a controller having an execution instruction function in an electronic device, and the processor may be implemented in any appropriate manner, for example, by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components. Inside the processor, the executable instructions may be executed by hardware circuits such as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers.

[0067] The storage device 20 can be applied to a storage server based on NVMe over Fabric architecture, such as Figure 2As shown, the storage device 20 includes a memory 210, a storage controller 220, a control module 240, and a cache module 230. The memory 210 is used to store the data to be read and written by the host 10 in the architecture. The storage controller 220 is used to implement the read and write control of the memory 210. The control module 240 is used to receive data read and write instructions. The cache module 230 is divided into a plurality of cache units 2310, wherein Figure 3 The control module 240 is configured as follows:

[0068] Step S11, when a data read / write instruction is received, the data read / write instruction is updated using a virtual address;

[0069] Step S12, applying for K cache units 2310 in the cache module 230 according to the updated data read and write instructions, and performing write operations on the K cache units 2310 using the data to be read and written, wherein K is less than the total number of cache units 2310 in the cache module 230 and is an integer;

[0070] Step S13, when any cache unit 2310 among the K cache units 2310 is full, directly start the data transmission of the full cache unit 2310, and release the full cache unit 2310 when the data transmission is completed.

[0071] In a possible implementation, the storage device 20 may include multiple memories 210 and corresponding multiple storage controllers 220. The memory 210 of the embodiment of the present disclosure may be a solid state disk (Solid State Disk or Solid State Drive, referred to as SSD), etc., and the storage controller 220 may be a storage controller 220 of the solid state disk. In the NVMe overFabric architecture, the storage controller 220 is also an NVMe controller that controls the reading and writing of the memory 210. The embodiment of the present disclosure does not limit the specific implementation of the storage controller 220. Those skilled in the art may use the controller implementation in the relevant technology or use the aforementioned processing components to implement it.

[0072] In a possible implementation, the data read and write instructions of the embodiment of the present disclosure may be NVMe read I / O commands and NVMe write I / O commands in the NVMe overFabric architecture.

[0073] In a possible implementation, the control module 240 may also be configured as:

[0074] Establishing a mapping relationship between the virtual address and the cache unit 2310;

[0075] When the data transmission of the full cache unit 2310 is completed, the mapping relationship between the virtual address and the full cache unit 2310 is released.

[0076] By establishing a mapping relationship between the virtual address and the cache unit 2310, the disclosed embodiment can obtain the cache unit 2310 through virtual address mapping, thereby realizing efficient management of the cache unit 2310, and when the data transmission of the full cache unit 2310 is completed, the mapping relationship between the virtual address and the full cache unit 2310 is released, thereby improving the utilization rate of the cache unit 2310, effectively reducing the demand for the cache module 230, and realizing data reading and writing operations efficiently, quickly and accurately using a smaller storage space.

[0077] In a possible implementation, step S11 uses a virtual address to update the data read / write instruction, which may include: using the virtual address to replace the host 10 memory address in the data pointer in the data read / write instruction.

[0078] In a possible implementation manner, the control module 240 is further configured to: establish a mapping relationship between the virtual address and the data read / write instruction.

[0079] In the embodiment of the present disclosure, when receiving an NVMe I / O command, the control module 240 does not split the NVMe I / O command, does not need to apply for local storage space for the NVMe I / O command, and does not need to immediately read all the data in the host 10 memory to the local storage space for the NVMe write I / O command. A virtual address is used to replace the host 10 memory address in the DataPointer related field in the NVMe I / O command (for example, it can be the memory address to be written in the host 10 for the data to be read from the host 10, or the memory address in the host 10 for the data to be written to the storage device 20 in the host 10), and the updated NVMe I / O command is quickly forwarded to the storage controller 220.

[0080] In the embodiment of the present disclosure, the virtual address may be a base address pointing to a virtual local storage space required for storing data in the NVMe I / O command. This storage space is virtual and does not correspond to the actual local storage space. Although the virtual address does not represent the actual address, each virtual address can also be represented by a base address + offset. For example, virtual address 1 can be represented by base address 1 + offset, virtual address 2 can be represented by base address 2 + offset, or base address 1 + 2 times offset, that is, the virtual address can be represented or distinguished by offset, and so on. Alternatively, the virtual address can also be represented in other ways, as long as the virtual addresses can be distinguished from each other. When the NVMe controller (storage controller 220) uses a virtual address to read or write data, the embodiment of the present disclosure will apply for a real storage space (cache unit 2310) and establish a temporary mapping relationship between the virtual storage space and the real storage space. In addition, the virtual address is associated with the NVMe I / O command, that is, the virtual address that replaces the host memory address in the NVMe I / O command is bound to the replaced NVMe I / O command, and the corresponding NVMe I / O command can be quickly queried through the virtual address. In the disclosed embodiment, a virtual address is used to quickly update and forward NVMe I / O commands without considering the status of local storage space, thereby reducing the storage time of NVMe I / O commands in the NVMe over Fabric implementation framework.

[0081] In the embodiment of the present disclosure, when the control module 240 receives a data read or write instruction, the data read or write instruction is updated through a virtual address. This is different from the related art in which the data pointer in the data read or write instruction directly points to the actual storage space. The embodiment of the present disclosure can update and forward the data read or write instruction without applying for storage space in advance, thereby optimizing the data processing flow.

[0082] In a possible implementation, the control module 240 is further configured to:

[0083] When the i-th cache unit 2310 among the K cache units 2310 is full, writing is performed to the i+m-th cache unit 2310 among the K cache units 2310, wherein i and m are less than K and are integers, i+m≤K;

[0084] When the (i+m)th cache unit 2310 is full and the (i)th cache unit 2310 is released after completing data transmission, writing is performed to the (i+m)th cache unit 2310 .

[0085] The disclosed embodiment writes to the i+mth cache unit 2310 among the K cache units 2310 when the i-th cache unit 2310 among the K cache units 2310 is full, and writes to the i-th cache unit 2310 when the i+mth cache unit 2310 is full and the i-th cache unit 2310 completes data transmission and is released. By dividing the multiple cache units 2310 for data block management and starting the transmission of data blocks without checking the completion message of the entire data, the data processing flow is optimized, the storage time of data in the local cache is reduced, thereby reducing the memory demand on the storage server side, and also reducing the completion delay of NVMe I / O commands.

[0086] The embodiment of the present disclosure does not limit the value of m. For example, the value of m may be 1, that is, each cache unit is written in sequence. Of course, m may also take other values ​​besides 1 as needed.

[0087] In a possible implementation, the control module 240 is further configured to:

[0088] When receiving the read / write completion message from the storage controller 220 , the read / write completion message is transmitted to the host 10 .

[0089] The embodiment of the present disclosure can control the progress of the data reading and writing process and notify the host 10 that the data transmission is completed by transmitting the reading and writing completion message to the host 10 upon receiving the reading and writing completion message from the storage controller 220.

[0090] In a possible implementation, the control module 240 of the disclosed embodiment does not immediately apply for storage space in the cache module 230 when receiving an NVMe read I / O command, but uses a virtual address to replace the host 10 memory address in the Data Pointer related field in the NVMe read I / O command, and then sends the updated NVMe read I / O command to the target NVMe controller. When the control module 240 receives the NVMe controller's write to the virtual address, it immediately applies for a cache unit 2310 in the cache module 230 (local memory or other storage), establishes a mapping relationship between the virtual address and the cache unit 2310, and writes the data to the local cache unit 2310. That is to say, during the execution of the NVMe read I / O command, a mapping relationship is established between the virtual address in the command and the requested cache unit 2310, so that according to the virtual address, it can be mapped to the cache unit 2310, so that data is written in the cache unit 2310, and if the virtual address in other NVMe read I / O commands does not establish a mapping relationship with the cache unit 2310, data will not be written to the cache unit 2310. When the cache unit 2310 is full, there is no need to wait for the completion message (Completion Queue, CQ), and the data of the cache unit 2310 is immediately transferred to the memory space of the corresponding host 10, and the mapping relationship between the cache unit 2310 and the virtual address is released, and the cache unit 2310 is immediately released. Repeat the above data relocation process until the data of the NVMe read I / O command is completely transmitted.

[0091] In a possible implementation, when the control module 240 of the disclosed embodiment receives an NVMe write I / O command, it will not immediately apply for storage space for the I / O command, but will use a virtual address to replace the host 10 memory address in the Data Pointer related field in the NVMe write I / O command, and then send the updated NVMe write I / O command to the target NVMe controller, and then start the hardware control module 240 to apply for one or more cache units 2310, and temporarily map them with the virtual address, and read part of the data from the memory of the host 10 to the local cache unit 2310. When the NVMe controller finishes reading the data in the cache unit 2310, it immediately releases the mapping relationship between the virtual address and the cache unit 2310, and immediately releases the cache unit 2310. Repeat the above data relocation process until the data of the NVMe write I / O command is completely transmitted.

[0092] The disclosed embodiment introduces virtual addresses and a block management mechanism based on the cache unit 2310 to implement NVMe I / O commands and data transmission channels through hardware. There is no need to wait until the data of the entire NVMe I / O command is moved to the local storage space before forwarding the NVMe I / O command to the target NVMe controller. By managing and quickly transmitting data according to the cache unit 2310 and efficiently utilizing and releasing the cache unit 2310, the local storage requirements of the storage server can be significantly reduced, and the completion delay of the NVMe I / O command can also be greatly reduced.

[0093] In a possible implementation, the embodiment of the present disclosure may divide the cache resources of the cache module 230 into units in advance, so as to divide the cache module 230 into a plurality of cache units 2310. The embodiment of the present disclosure does not limit the unit division method of the cache resources, and those skilled in the art may divide according to needs and actual conditions. In addition, the cache module 230 in the embodiment of the present disclosure may be a memory or other storage resources.

[0094] In a possible implementation, the storage sizes of the cache units 2310 in the cache module 230 are the same and are integer multiples of a preset size.

[0095] In a possible implementation, the preset size is 4096 bytes (4K). The disclosed embodiment can improve the efficiency of data reading and writing by setting the preset size to 4K.

[0096] In one example, the embodiment of the present disclosure can divide the cache module 230 into a plurality of cache units 2310 of equal size, and the size of the cache unit 2310 can be a multiple of 512 bytes, such as 1K, 2K, 4K, 8K or 16K. It is understandable that the selection of the size of the cache unit 2310 will have an impact on the efficiency of data transmission. The larger the cache unit 2310 is, the lower the reuse efficiency of the cache unit 2310 is. The smaller the cache unit 2310 is, the higher the storage space utilization rate is, but the transmission will be more frequent, which will reduce the efficiency of the transmission network, so it is necessary to select according to the transmission conditions of different networks.

[0097] In the disclosed embodiment, the cache unit 2310 divides the local storage space into a plurality of fixed small storage spaces, and each small storage space is a cache unit 2310. Assuming that the size of each cache unit 2310 is set to 4K bytes, then the local 1M byte storage can be divided into 256 cache units 2310. In the disclosed embodiment, the reading and writing of data in the memory of the host 10 is performed according to the size of the cache unit 2310.

[0098] In the disclosed embodiment, the cache unit 2310 is recycled, and the release of the cache unit 2310 does not need to wait for the completion message of the corresponding NVMe I / O command. The rapid release of the cache unit 2310 can improve the utilization efficiency of the cache unit 2310, thereby reducing the local storage space and the cache time of data in the local storage space.

[0099] For NVMe read I / O commands, when the control module 240 detects that the NVMe controller uses a virtual address to write data, it immediately applies for a cache unit 2310, establishes a temporary mapping relationship between the cache unit 2310 and the address space pointed to by the virtual address, and writes the data into the cache unit 2310 in sequence. When the control module 240 detects that the cache unit 2310 is full, the data in the cache unit 2310 is immediately moved to the corresponding address space in the memory of the host 10. When the control module 240 detects that the data in the cache unit 2310 has been transferred, the mapping relationship between the cache unit 2310 and the address space pointed to by the storage virtual address is released, and the cache unit 2310 is released.

[0100] For NVMe write I / O commands, the control module 240 will apply for one or more cache units 2310, and establish a mapping relationship for the virtual address space of the corresponding I / O, and then pre-read part of the data of the NVMe write I / O command into the cache unit 2310. When the control module 240 detects that the NVMe controller uses a virtual address to read data, it directly reads data from the corresponding cache unit 2310. When the data in a cache unit 2310 is read, the mapping relationship between the cache unit 2310 and the address space pointed to by the storage virtual address can be released, and the cache unit 2310 is released.

[0101] As an example, see Figure 4 , Figure 4 A schematic diagram of the division of the cache unit 2310 according to an embodiment of the present disclosure is shown.

[0102] In one example, if Figure 4 As shown, the embodiment of the present disclosure can divide the cache resources of the cache module 230 into N consecutive cache units 2310, and the cache units 2310 include cache units 2310 being written, cache units 2310 that have been fully written (full), free cache units 2310, etc.

[0103] The data reading process and data writing process are respectively introduced as examples below.

[0104] In a possible implementation manner, the data read and write instruction is a data read instruction, and the data read instruction is used to read the to-be-read data from the memory 210. The control module 240 may also be configured as follows:

[0105] Sending an updated data read instruction to the storage controller 220;

[0106] In case of receiving a write operation of the memory 210 in response to the updated data read instruction, applying for K cache units 2310 in the cache module 230, and writing the to-be-read data in the memory 210 into the K cache units 2310 in sequence;

[0107] When any cache unit 2310 among the K cache units 2310 is full, the data transmission of the full cache unit 2310 is directly started, and the data of the full cache unit 2310 is transmitted to the host 10. When the data transmission is completed, the full cache unit 2310 is released, and the mapping relationship between the virtual address and the full cache unit 2310 is released.

[0108] Through the above configuration, the embodiment of the present disclosure can send an updated data read instruction to the storage controller 220 when receiving a data read instruction. After receiving a data write request using a virtual address from the storage controller 220, K cache units 2310 are applied for in the cache module 230, and the data to be read in the memory 210 are written into the K cache units 2310 in sequence. When any cache unit 2310 among the K cache units 2310 is full, the data transmission of the full cache unit 2310 is directly started, and the data of the full cache unit 2310 is transmitted to the host 10. When the data transmission is completed, the full cache unit 2310 is released, and the mapping relationship between the virtual address and the full cache unit 2310 is released. By managing and quickly transmitting the data to be read according to the cache unit 2310, the cache unit 2310 is efficiently utilized and released, the local storage demand in the storage server is reduced, and the completion delay of the I / O instruction is greatly reduced.

[0109] In one example, K can be 1, that is, for a data read instruction, one cache unit is applied for writing input each time, and when the cache unit is full, another cache unit is applied for data writing. Of course, K can also be other, for example, K can be 2, and two cache units are applied for at a time, and the first cache unit is written first. When the first cache unit is full, the second cache unit is directly written. When the second cache unit is full and the data transmission of the first cache unit is completed and released, the first cache unit is written again. In this way, the data read operation is completed by this cyclic working mode, which can reduce the size of the storage space and reduce the instruction delay. Of course, the size of K can also be estimated in advance according to parameters such as the write speed, read speed, and release time of the cache unit, and the cache unit is applied according to the estimated size of K. This is not limited in the embodiments of the present disclosure.

[0110] The following is an exemplary introduction to the host 10 reading data in the memory 210 with reference to a specific example.

[0111] See also Figure 5 , Figure 5 A schematic diagram of an NVMe over Fabric architecture according to an embodiment of the present disclosure is shown.

[0112] In one example, if Figure 5 As shown, the host 10 may include a processor 101, a memory 102, and a network adapter card 103. The host 10 is connected to a storage server via a network. The memory 210 in the storage device 20 may be a non-volatile memory 210, the storage controller 220 may be an NVMe controller, the control module 240 may include an NVMe-oF controller 2410 and a processor 2420, the cache module 230 may include a storage manager 2320 and a storage resource pool 2330, the storage resource pool 2330 includes a plurality of storage resources divided into cache units 2310, the storage manager 2320 may be used to manage the storage resource pool 2330 and interact with the control module 240, and the storage server may communicate with the network adapter card 103 of the host 10 via a network interface controller 250.

[0113] The following is an exemplary introduction to the processing steps of an NVMe read I / O command request for 128K data. In one example, the storage device 20 can process the data read instruction of the host 10 through the above steps 1 to 6. It should be noted that in the embodiment of the present disclosure, if the NVMe-oF controller is implemented using hardware, steps 2, 3 and 4 occur simultaneously.

[0114] Step 1, when the NVMe-oF controller 2410 receives the NVMe read I / O command (data read instruction), it is not necessary to apply for any cache unit 2310 for the NVMe read I / O command. It only needs to use the virtual address to replace the host 10 memory address in the DataPointer related field in the NVMe I / O, and send the updated NVMe read I / O command to the NVMe controller of the target solid state drive (non-volatile memory 210).

[0115] Step 2. When the NVMe-oF controller 2410 monitors that the NVMe controller of the solid-state drive uses a virtual address to write data (the storage controller 220 writes to the cache module 230), the NVMe-oF controller 2410 immediately applies for a cache unit 2310 from the storage resource pool 2330, completes the mapping relationship between the 4K bytes (a cache unit 2310) of virtual address space and the cache unit 2310, and writes the received data into the cache unit 2310 in sequence.

[0116] Step 3, when the NVMe-oF controller 2410 monitors that a local cache unit 2310 is filled with 4K bytes, there is no need to wait for the completion message of the NVMe I / O command corresponding to the cache unit 2310, and the 4K data of the cache unit 2310 is immediately transmitted to the corresponding storage space in the memory of the host 10 through the network interface controller 250.

[0117] Step 4, when the NVMe-oF controller 2410 receives the data of a 4K cache unit 2310 written to the host 10 by the network interface controller 250, it immediately releases the mapping relationship between the 4K virtual address space and the cache unit 2310, and releases the cache unit 2310 to the storage resource pool 2330.

[0118] Step 5. Repeat steps 2 to 4 to complete the migration of the remaining data.

[0119] Step 6, wait for the NVMe-oF controller 2410 to receive the NVMe I / O completion message from the NVMe controller, check and confirm that all 128K data has been sent to the host 10, and then transmit the completion information to the host 10 through the network interface controller 250 to complete the entire process of an NVMe read I / O command.

[0120] In the disclosed embodiment, when the NVMe-oF controller 2410 receives a request to write data using a virtual address from the NVMe controller, it will not apply for 32 cache units 2310 for all 128K data at once. For example, it will only apply for one cache unit 2310 (of course, the number of cache units 2310 applied for can be set according to actual needs). Because on the PCIe IP interface, data is transmitted byte by byte. If the virtual address is represented by an offset, the NVMe-oF controller 2410 will write the data to the corresponding cache unit 2310 according to the offset of the virtual address of the data request. If the offset of the virtual address corresponding to the data request has established a mapping relationship with the cache unit 2310, the data is directly written to the corresponding position of the cache unit 2310. If the offset of the virtual address corresponding to the data request is the second 4K space and has not been mapped to any cache unit 2310, a cache unit 2310 is immediately applied for, and after the mapping relationship is established, the data is directly written to the cache unit 2310.

[0121] In a possible implementation, the cache module 230 further includes a cache manager 2320, the cache manager 2320 is used to manage each cache unit 2310, the data read and write instruction is a data write instruction, the data write instruction is used to write the data to be written in the memory 210, and the control module 240 is further configured to:

[0122] Sending updated data write instructions to the storage controller 220 and the cache manager 2320;

[0123] Control the cache manager 2320 to predict the target data request sequence of the storage controller 220 in response to the updated data write instructions according to the sequence of the updated data write instructions and / or the sequence of data requests received from the storage controller 220; in one example, the prediction can be achieved through relevant technologies, and the present application does not limit the specific prediction method.

[0124] Control the cache manager 2320 to read the data to be written from the host 10 according to the target data request sequence and apply for K cache units 2310, and write the data to be written in the host 10 into the K cache units 2310 in sequence;

[0125] When any cache unit 2310 among the K cache units 2310 is full, the data transmission of the full cache unit 2310 is directly initiated in response to the read request of the storage controller 220, and the data of the full cache unit 2310 is transferred to the memory 210 for storage. When the data transmission is completed, the full cache unit 2310 is released, and the mapping relationship between the virtual address and the full cache unit 2310 is released.

[0126] Through the above configuration, the control module 240 of the disclosed embodiment sends the updated data write instruction to the storage controller 220 and the cache manager when receiving the data write instruction; controls the cache manager to predict the target data request sequence of the storage controller 220 in response to the updated data write instruction according to the sequence of the updated data write instructions and / or the received data request sequence of the storage controller 220; controls the cache manager to read the data to be written from the host 10 according to the target data request sequence and apply for K cache units 2310, and writes the data to be written in the host 10 into the K cache units 2310 in sequence; When any cache unit 2310 in the cache unit 2310 is full, the data transmission of the full cache unit 2310 is directly started in response to the read request of the storage controller 220, and the data of the full cache unit 2310 is transmitted to the memory 210 for storage. When the data transmission is completed, the full cache unit 2310 is released, and the mapping relationship between the virtual address and the full cache unit 2310 is released. In this way, the data to be written is managed and quickly transmitted according to the cache unit 2310, and the cache unit 2310 is efficiently utilized and released, thereby reducing the local storage requirements in the storage server and greatly reducing the completion delay of the I / 0 instructions.

[0127] The following is an exemplary introduction to the processing steps of an NVMe write I / O command request for 128K data. In one example, the storage device 20 can process the data write instruction of the host 10 through the above steps a to e. In the embodiment of the present disclosure, if the NVMe-oF controller is implemented by hardware, steps b, c and d can occur simultaneously.

[0128] Step a, when the NVMe-oF controller 2410 receives an NVMe write I / O command, it does not need to apply for any cache unit 2310 for the NVMe write I / O command, nor does it need to immediately read the data in the host 10 memory. It only needs to use a virtual address to replace the host 10 memory address in the Data Pointer related field in the NVMe write I / O, and send the updated NVMe write I / O command to the target NVMe controller, and send the NVMe write I / O information to the storage manager 2320.

[0129] In step b, the NVMe-oF controller 2410 predicts the data request sequence of the NVMe controller through the storage manager 2320 according to the sequence of the forwarded NVMe write I / O commands and the sequence of the data requests received from the NVMe controller, and reads part of the data in the memory of the host 10. The specific implementation method of predicting the data request sequence of the NVMe controller is not limited in the embodiment of the present disclosure, and those skilled in the art can implement it according to relevant technologies. The storage manager 2320 will apply for a cache unit 2310 in the storage resource pool 2330 for the data involved in the data request in turn according to the predicted data request sequence, and establish a mapping relationship between the cache unit 2310 and a 4K virtual address space corresponding to the NVMe write I / O, and start to read the corresponding 4K data from the memory of the host 10 to the cache unit 2310 in advance. In order to avoid the situation where the data request of the NVMe controller waits for reading the memory data of the host 10, in the embodiment of the present disclosure, the storage manager 2320 can apply for multiple cache units 2310 in sequence and read multiple corresponding 4K data from the memory of the host 10 in advance.

[0130] Step c, when the NVMe-oF controller 2410 monitors that the NVMe controller has completed reading the data of a cache unit 2310, the NVMe-oF controller 2410 will release the cache unit 2310 and establish a mapping relationship between the cache unit 2310 and a 4K virtual address space corresponding to the NVMe write I / O, and immediately release the cache unit 2310, and the storage manager 2320 will use the cache unit 2310 in a circular manner.

[0131] Step d: Repeat steps b, c, and d to complete the migration of the remaining data until the NVMe controller reads all 128K data of the NVMe I / O command.

[0132] Step e, after the NVMe-oF controller 2410 receives the I / O completion message from the NVMe controller, it directly transmits the completion information to the host 10 through the network interface controller 250, completing the entire process of an NVMe write I / O command.

[0133] In the disclosed embodiment, during the data migration process of the NVMe write I / O command, the storage manager 2320 does not apply for 32 cache units 2310 at a time to store 128K data, which can save the use of the cache unit 2310, while the related art directly applies for 128K storage space, which obviously wastes multiple cache units 2310. The storage manager 2320 will apply for several cache units 2310 in sequence, and read the corresponding data from the host 10 memory into the cache unit 2310. After the NVMe controller starts to read the data of the NVMe I / O and releases a cache unit 2310, the storage manager 2320 applies for another cache unit 2310 to start reading the next 4K data. Exemplarily, the storage manager 2320 only needs to ensure that the data reading of the NVMe controller and the data reading from the host 10 memory are approximately balanced. When the NVMe controller needs to read data, the storage manager 2320 has already read the data in the host 10 memory into the cache unit 2310.

[0134] In the disclosed embodiment, the NVMe I / O command does not need to wait for the application of storage space, and the data does not need to wait for the completion message of the NVMe I / O command. This will significantly reduce the storage time of the NVMe I / O command and data in the cache module 230. At the same time, the data migration of the host 10 memory is completed according to the size of the cache unit 2310, which can also significantly save local storage resources.

[0135] In order to complete the NVMe I / O command and data transmission process more efficiently, the control module 240 (taking the NVMe-oF controller 2410 as an example) also needs to consider some abnormal reading and writing situations, and also needs to make some special situation processing mechanisms.

[0136] In a possible implementation, the control module 240 is further configured to:

[0137] If the data corresponding to the read request of the updated data write instruction in response to the storage controller 220 is not written into the corresponding cache unit 2310 , the data is read from the host 10 to the cache unit 2310 using a preset channel.

[0138] Exemplarily, for the NVMe write I / O command request on the host 10 side, the NVMe-oF controller 2410 needs to adjust the number of data read from different channels to the host 10 memory in real time according to the resources of the local cache unit 2310 and the data request of the NVMe controller, and try to complete the accuracy of the data request of the NVMe controller. Since the data read request of the NVMe controller passes through the PCIe switching structure, there will be disorder. The data request may come from different NVMe controllers, and the NVMe-oF controller 2410 only reads part of the data in advance. When the NVMe-oF controller 2410 finds that the data in the memory has not been read to the local cache unit 2310, the NVMe-oF controller 2410 can enable a dedicated emergency channel resource to read the required data from the host 10 end memory to the corresponding cache unit 2310. Emergency channel resources can be, for example, communication resources of any available network card, and emergency channel resources can be configured in advance. The specific configuration method of emergency channel resources is not limited in the embodiment of the present disclosure.

[0139] In a possible implementation, the control module 240 may also be configured as:

[0140] If the number of free cache units 2310 of the cache module 230 is reduced to a preset number of free units, data transmission of a preset number of cache units being written is directly started, so that the number of free cache units 2310 of the cache module 230 reaches the preset number of free units.

[0141] Exemplarily, for the NVMe read I / O command request of the host 10, the NVMe-oF controller 2410 needs to adjust the number of I / Os forwarded to different target NVMe controllers in real time according to the resources of the local cache unit 2310, the data writing of the NVMe controller and the processing speed of the network interface control 250. Since the data writing of the NVMe controller passes through the PCIe switching structure, there will be disorder, and even multiple retransmissions. In order to cope with extreme situations, the NVMe-oF controller 2410 needs to ensure sufficient free cache units 2310. When the free cache unit 2310 reaches a certain threshold, even if some cache units 2310 are not filled, it is necessary to start the emergency channel resources to write the cache unit 2310 data into the corresponding address space in the memory of the host 10, so as to ensure the number of free cache units 2310.

[0142] The embodiment of the present disclosure does not limit the specific size of the preset number of free units, and those skilled in the art can set it according to needs and actual conditions.

[0143] In the disclosed embodiment, the required storage is divided into two categories, one is to record and store some information of NVMe I / O command conversion (such as SGL, Command ID and cache unit 2310 index, etc.), the ability to store NVMe I / O command information here will affect the number of forwarded NVMe I / O commands, and further affect the overall I / O throughput performance, but this part does not occupy much storage space. Another category is the cache unit 2310 for storing data, which is used to cache the data of NVMe I / O commands. The size of this type of storage space can be set according to the network interface controller transmission data delay. In order to ensure the transmission performance of I / O commands, different network transmission protocols will have their own optimal storage space size. In the disclosed embodiment, due to the division and efficient management of the cache unit 2310, the size of the local storage space will be greatly reduced. For the calculation of the storage space for local storage data, it is assumed that 2048 NVMe I / O commands can be processed simultaneously, and the maximum transmission length of each NVMe I / O command is 512K bytes. In the current mainstream technology, 1G byte storage space will be required. In the disclosed embodiment, due to the use of virtual addresses and efficient cache unit 2310 management technology, the used storage space will be much smaller than 1G bytes. Assuming that RDMA transmission on Ethernet is used, the required storage space is about 8M bytes.

[0144] The disclosed embodiment implements the data transmission channel of NVMe over Fabric through hardware, and by introducing virtual storage addresses and data block management, it is possible to update and forward NVMe I / O commands without applying for storage space in advance. At the same time, by transmitting in blocks without checking the completion message, the data processing flow is optimized, the storage time of data in the cache is reduced, and thus the completion delay of NVMe I / O commands is reduced, which can better meet the application with strict requirements on transmission delay. In addition, the efficient use of storage space for NVMe I / O command transmission data by the disclosed embodiment also reduces the storage demand of the server side. If the internal storage of the chip is used in the implementation, the design complexity of the board can be simplified.

[0145] According to one aspect of the present disclosure, a storage system is provided. Figure 2As shown, the system includes a host 10 and a storage device 20, the system is implemented based on the NVMe over Fabric architecture, the device is applied to the storage server of the architecture, the storage device 20 includes a memory 210, a storage controller 220, a control module 240, and a cache module 230, the host 10 reads and writes data in the storage device 20 through data read and write instructions, the memory 210 is used to store the data to be read and written by the host 10, the storage controller 220 is used to implement read and write control of the memory 210, the control module 240 is used to receive data read and write instructions, and the cache module 230 is divided into a plurality of cache units 2310, wherein the control module 240 is configured as follows:

[0146] When receiving a data read / write instruction, the data read / write instruction is updated using a virtual address; according to the updated data read / write instruction, K cache units 2310 are applied for in the cache module 230, and a write operation is performed on the K cache units 2310 using the data to be read / written, wherein K is less than the total number of cache units 2310 in the cache module 230 and is an integer;

[0147] When any cache unit 2310 among the K cache units 2310 is full, data transmission of the full cache unit 2310 is directly started, and when the data transmission is completed, the full cache unit 2310 is released.

[0148] It should be noted that the storage device in the storage system is the aforementioned storage device 20.

[0149] The disclosed embodiment uses a virtual storage address and divides the data into multiple cache units 2310 for data block management, thereby realizing that the NVMe I / O command can be updated and forwarded without applying for storage space in advance, and the transmission of data blocks can be started without checking the completion message, thereby optimizing the data processing flow and reducing the storage time of data in the local cache, thereby reducing the memory requirements of the storage server, and also reducing the completion delay of the NVMe I / O command.

[0150] In a possible implementation, the control module 240 is further configured to:

[0151] Establishing a mapping relationship between the virtual address and the cache unit 2310;

[0152] When the data transmission of the full cache unit 2310 is completed, the mapping relationship between the virtual address and the full cache unit 2310 is released.

[0153] In a possible implementation, the updating of the data read / write instruction by using the virtual address includes: replacing the host 10 memory address in the data pointer in the data read / write instruction by using the virtual address;

[0154] The control module 240 is further configured to establish a mapping relationship between the virtual address and the data read and write instruction.

[0155] In a possible implementation, the control module 240 is further configured to:

[0156] When the i-th cache unit 2310 among the K cache units 2310 is full, writing is performed to the i+m-th cache unit 2310 among the K cache units 2310, wherein i and m are less than K and are integers, i+m≤K;

[0157] When the (i+m)th cache unit 2310 is full and the (i)th cache unit 2310 is released after completing data transmission, writing is performed to the (i+m)th cache unit 2310 .

[0158] In a possible implementation, the control module 240 is further configured to:

[0159] When receiving the read / write completion message from the storage controller 220 , the read / write completion message is transmitted to the host 10 .

[0160] In a possible implementation manner, the data read and write instruction is a data read instruction, and the data read instruction is used to read the to-be-read data from the memory 210, and the control module 240 is further configured to:

[0161] Sending an updated data read instruction to the storage controller 220;

[0162] In case of receiving a write operation of the memory 210 in response to the updated data read instruction, applying for K cache units 2310 in the cache module 230, and writing the to-be-read data in the memory 210 into the K cache units 2310 in sequence;

[0163] When any cache unit 2310 among the K cache units 2310 is full, the data transmission of the full cache unit 2310 is directly started, and the data of the full cache unit 2310 is transmitted to the host 10. When the data transmission is completed, the full cache unit 2310 is released, and the mapping relationship between the virtual address and the full cache unit 2310 is released.

[0164] In a possible implementation, the cache module 230 further includes a cache manager, which is used to manage each cache unit 2310, the data read and write instruction is a data write instruction, and the data write instruction is used to write the data to be written in the memory 210, and the control module 240 is further configured to:

[0165] Sending updated data write instructions to the storage controller 220 and the cache manager;

[0166] Controlling the cache manager to predict a target data request sequence of the storage controller 220 in response to the updated data write instruction according to the updated data write instruction sequence and / or the received data request sequence of the storage controller 220;

[0167] Control the cache manager to read the data to be written from the host 10 according to the target data request sequence and apply for K cache units 2310, and write the data to be written in the host 10 into the K cache units 2310 in sequence;

[0168] When any cache unit 2310 among the K cache units 2310 is full, the data transmission of the full cache unit 2310 is directly initiated in response to the read request of the storage controller 220, and the data of the full cache unit 2310 is transferred to the memory 210 for storage. When the data transmission is completed, the full cache unit 2310 is released, and the mapping relationship between the virtual address and the full cache unit 2310 is released.

[0169] In a possible implementation, the control module 240 is further configured to:

[0170] If the data corresponding to the read request of the updated data write instruction in response to the storage controller 220 is not written into the corresponding cache unit 2310 , the data is read from the host 10 to the cache unit 2310 using a preset channel.

[0171] In a possible implementation, the control module 240 is further configured to:

[0172] If the number of free cache units 2310 of the cache module 230 is reduced to a preset number of free units, data transmission of a preset number of cache units being written is directly started, so that the number of free cache units 2310 of the cache module 230 reaches the preset number of free units.

[0173] In a possible implementation, the storage sizes of the cache units 2310 in the cache module 230 are the same and are integer multiples of a preset size.

[0174] In a possible implementation manner, the preset size is 4096 bytes.

[0175] It should be noted that the storage device in the system corresponds to the aforementioned storage device. For a detailed introduction, please refer to the previous description and will not be repeated here.

[0176] In various aspects of the embodiments of the present disclosure, the local storage space is pre-divided and managed, and the storage space is divided into cache units 2310 of the same size according to a pre-set data block size. Each cache unit 2310 is independent and can be used and released separately.

[0177] In various aspects of the embodiments of the present disclosure, when the NVMe over Fabric controller receives an NVMe I / O command, it does not need to immediately apply for local storage space. After using a virtual address to replace the host 10 memory address in the Data Pointer related field in the NVMe I / O command, the updated NVMe I / O command can be forwarded to the NVMe controller of the target solid state drive.

[0178] In various aspects of the embodiments of the present disclosure, when the NVMe over Fabric controller receives a data write request using a virtual address from the NVMe controller, it applies for a cache unit 2310 in the local storage space, establishes a temporary mapping relationship between the cache unit 2310 and the virtual address, and writes the data into the cache unit 2310.

[0179] In various aspects of the embodiments of the present disclosure, when the NVMe over Fabric controller detects that a cache unit 2310 is full, it does not need to check the completion message of the NVMe I / O command, and immediately starts the transmission of the data in the cache unit 2310, and transmits the data in the cache unit 2310. When the NVMe over Fabric controller detects that the data in the cache unit 2310 has been transmitted, the mapping relationship between the virtual address and the cache unit 2310 is released, and the cache unit 2310 is immediately released. Other NVMe I / O commands or other data blocks of the NVMe I / O can recycle the released cache unit 2310.

[0180] In various aspects of the embodiments of the present disclosure, the NVMe over Fabric controller can pre-read data from the memory of the host 10, apply for the required number of cache units 2310 in the local storage space according to the size of the pre-read data, and establish a mapping relationship between the cache unit 2310 and the virtual address of the NVMe I / O.

[0181] In various aspects of the embodiments of the present disclosure, when the NVMe over Fabric controller detects that the data in a cache unit 2310 has been read by the NVMe controller, it immediately releases the mapping relationship between the virtual address of the NVMe I / O and the cache unit 2310 and releases the cache unit 2310.

[0182] The various aspects of the embodiments of the present disclosure, the flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to the multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A storage device, characterized in that: The device is applied to a storage server based on the NVMe over Fabric architecture. The storage device includes a memory, a storage controller, a control module, and a cache module. The memory is used to store data to be read and written by a host in the architecture. The storage controller is used to implement read and write control of the memory. The control module is used to receive data read and write instructions. The cache module is divided into a plurality of cache units, wherein the control module is configured as follows: When receiving a data read / write instruction, the data read / write instruction is updated using a virtual address, including: replacing a host memory address in a data pointer in the data read / write instruction with a virtual address; the control module is further configured to: establish a mapping relationship between the virtual address and the data read / write instruction; Applying for K cache units in the cache module according to the updated data read and write instructions, and performing write operations on the K cache units using the data to be read and written, wherein K is less than the total number of cache units in the cache module and is an integer; When any cache unit among the K cache units is full, data transmission of the full cache unit is directly started, and when the data transmission is completed, the full cache unit is released. The control module is further configured to: Establishing a mapping relationship between the virtual address and the cache unit; When the data transmission of the full cache unit is completed, the mapping relationship between the virtual address and the full cache unit is released.

2. The device according to claim 1, characterized in that The control module is further configured to: When the i-th cache unit among the K cache units is full, writing is performed on the i+m-th cache unit among the K cache units, wherein i and m are less than K and are integers, and i+m≤K; When the (i+m)th cache unit is full and the (i)th cache unit completes data transmission and is released, writing is performed on the (i)th cache unit.

3. The device according to claim 1, characterized in that The control module is further configured to: When receiving the read / write completion message from the storage controller, the read / write completion message is transmitted to the host.

4. The device according to claim 1, characterized in that The data read and write instruction is a data read instruction, and the data read instruction is used to read the to-be-read data from the memory. The control module is further configured as follows: Sending an updated data read instruction to the storage controller; In case of receiving a write operation of the memory in response to the updated data read instruction, applying for K cache units in the cache module, and writing the to-be-read data in the memory into the K cache units in sequence; When any cache unit among the K cache units is full, the data transmission of the full cache unit is directly started, the data of the full cache unit is transmitted to the host, and when the data transmission is completed, the full cache unit is released, and the mapping relationship between the virtual address and the full cache unit is released.

5. The device according to claim 1, characterized in that The cache module further includes a cache manager, which is used to manage each cache unit. The data read and write instructions are data write instructions, which are used to write the data to be written in the memory. The control module is further configured as follows: Sending updated data write instructions to the storage controller and the cache manager; Controlling the cache manager to predict a target data request sequence of the storage controller in response to the updated data write instruction according to the sequence of the updated data write instruction and / or the sequence of data requests received from the storage controller; Control the cache manager to read the data to be written from the host according to the target data request sequence and apply for K cache units, and write the data to be written in the host into the K cache units in sequence; When any cache unit among the K cache units is full, data transmission of the full cache unit is directly initiated in response to a read request of the storage controller, the data of the full cache unit is transferred to the memory for storage, and when the data transmission is completed, the full cache unit is released, and the mapping relationship between the virtual address and the full cache unit is released.

6. The device according to claim 1, characterized in that The control module is further configured to: If the data corresponding to the read request of the storage controller in response to the updated data write instruction is not written into the corresponding cache unit, the data is read from the host to the cache unit using a preset channel.

7. The device according to claim 1, characterized in that The control module is further configured to: If the number of free cache units of the cache module is reduced to a preset number of free units, data transmission of a preset number of cache units being written is directly started, so that the number of free cache units of the cache module reaches the preset number of free units.

8. The device according to claim 1, characterized in that The storage size of each cache unit in the cache module is the same and is an integer multiple of a preset size.

9. The device according to claim 8, wherein the preset size is 4096 bytes.

10. A storage system, characterized in that: The system includes a host and a storage device according to any one of claims 1 to 9. The system is implemented based on the NVMe over Fabric architecture. The device is applied to a storage server of the architecture. The storage device includes a memory, a storage controller, a control module, and a cache module. The host reads and writes data in the storage device through data read and write instructions. The memory is used to store data to be read and written by the host. The storage controller is used to implement read and write control of the memory. The control module is used to receive data read and write instructions. The cache module is divided into multiple cache units, wherein the control module is configured as follows: When receiving a data read / write instruction, the data read / write instruction is updated using a virtual address, including: replacing a host memory address in a data pointer in the data read / write instruction with a virtual address; the control module is further configured to: establish a mapping relationship between the virtual address and the data read / write instruction; Applying for K cache units in the cache module according to the updated data read and write instructions, and performing write operations on the K cache units using the data to be read and written, wherein K is less than the total number of cache units in the cache module and is an integer; When any cache unit among the K cache units is full, data transmission of the full cache unit is directly started, and when the data transmission is completed, the full cache unit is released. The control module is further configured to: establish a mapping relationship between the virtual address and the cache unit; and release the mapping relationship between the virtual address and the cache unit when the data transmission of the full cache unit is completed.

Images