Data transmission method and system between NVMe host, hard disk and memory

Abstract

The present application provides a method and system for data transmission between an NVMe host, a hard disk and a memory. The NVMe host is used to receive user commands sent by a microprocessor based on the NVMe protocol outside the CPU, write the user commands into a user submission queue and trigger a doorbell after writing, so that an external root complex extracts the user submission queue and performs data transmission between the hard disk and the memory according to each user command therein, and then returns the write completion data corresponding to each user command; the NVMe host is also used to receive and store each write completion data, so that the microprocessor can obtain the completion status information corresponding to the user command according to each write completion data. The present application uses hardware to replace the CPU to realize data transmission between the hard disk and the memory based on the NVMe protocol, which can effectively reduce the occupancy rate and resource consumption of the host CPU, and can effectively improve the processing efficiency of NVMe user commands and increase throughput.

Description

Technical Field

[0001] The present application relates to the field of high-speed storage technology, and in particular to a method and system for data transmission between an NVMe host, a hard disk, and a memory. Background Art

[0002] Solid State Disk (SSD) is a hard disk made of solid-state electronic storage chip arrays. At present, the traditional storage architecture is that the solid-state drive is connected to the root complex (RC) through PCIe as a PCIe Endpoint (EP), and then the root complex connects the CPU and memory. The processing of the NVMe protocol is completed by the CPU software.

[0003] However, as the network transmission speed increases, the storage rate also increases, and the storage occupancy rate and memory consumption of the CPU are also increasing dramatically. In addition, the CPU's use of software to complete the reading and writing of the solid-state drive will also be affected by CPU performance, software optimization, etc. Therefore, it is urgent to design a hardware device that can replace the CPU to realize data transmission between the solid-state drive and the memory based on the NVMe protocol. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a data transmission method and system between an NVMe host, a hard disk and a memory to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present application provides an NVMe host, the NVMe host is used to receive a user command sent by a microprocessor based on the NVMe protocol outside a CPU, write the user command into a local user submission queue, and trigger a doorbell after the user submission queue is written, so that an external root complex extracts the user submission queue and performs data transmission between a solid-state drive and a memory according to each user command in the user submission queue, and then returns write completion data corresponding to each user command;

[0006] The NVMe host is also used to receive and store the write completion data corresponding to each of the user commands sent by the root complex, so that the microprocessor can obtain the completion status information corresponding to the user command according to the write completion data corresponding to each of the user commands.

[0007] In some embodiments of the present application, the following are included: a register, a user submission queue state machine, and a user submission queue cache that are sequentially communicatively connected;

[0008] The register is used to receive and store user commands and hardware status data sent by the microprocessor based on the NVMe protocol outside the CPU, and send the user command to the user submission queue state machine;

[0009] The user submission queue state machine is used to forward the user command to the user submission queue cache and record the state data of the user submission queue in the user submission queue cache;

[0010] The user submission queue buffer is used to store the user submission queue.

[0011] In some embodiments of the present application, it further includes: a doorbell state machine communicatively connected to the user submission queue state machine;

[0012] The user submission queue state machine is further used to determine whether the user submission queue is currently written according to the recorded state data of the user submission queue, and if so, send a first trigger instruction to the doorbell state machine;

[0013] The doorbell state machine is used to trigger the doorbell according to the first trigger instruction sent by the user submission queue state machine, so that the external root complex extracts the user submission queue and performs data transmission between the solid state drive and the memory according to each user command in the user submission queue, and then returns the write completion data corresponding to each of the user commands.

[0014] In some embodiments of the present application, it further includes: a user completion queue cache and a user completion queue state machine for the communication connection;

[0015] The user completion queue buffer is communicatively connected to the root complex, and the user completion queue state machine is communicatively connected to the register and the doorbell state machine respectively;

[0016] The user completion queue buffer is used to receive the write completion data corresponding to each of the user commands sent by the root complex, and store the write completion data corresponding to each of the user commands in a local user completion queue;

[0017] The user completion queue state machine is used to record the state data of the user completion queue in the user completion queue cache, and determine whether the user completion queue is currently written according to the recorded state data of the user completion queue, and if so, send a second trigger instruction to the doorbell state machine;

[0018] The doorbell state machine is further used to trigger the doorbell according to the second trigger instruction sent by the user completion queue state machine, so that the register obtains the state data of the user completion queue from the user completion queue state machine;

[0019] The register is further used to forward the status data of the user completion queue obtained from the user completion queue state machine to the microprocessor, so that the microprocessor can obtain the completion status information corresponding to the user command based on the status data of the user completion queue.

[0020] In some embodiments of the present application, further comprising: a manager queue state machine and a manager submission queue cache in communication connection;

[0021] The manager queue state machine is communicatively connected to the register;

[0022] The register is also used to receive and store manager commands and hardware status data sent by the microprocessor based on the NVMe protocol outside the CPU, and send the manager command to the manager submission queue state machine;

[0023] The manager submission queue state machine is used to forward the manager command to the manager submission queue cache and record the state data of the manager submission queue in the manager submission queue cache;

[0024] The manager submission queue buffer is used to store the manager submission queue.

[0025] In some embodiments of the present application, it further includes: a manager queue storage space communicatively connected to the root complex;

[0026] The manager queue state machine is also in communication connection with the doorbell state machine;

[0027] The manager submission queue state machine is further used to determine whether the manager submission queue is currently written according to the recorded state data of the manager submission queue, and if so, send a third trigger instruction to the doorbell state machine;

[0028] The doorbell state machine is used to trigger the doorbell according to the third trigger instruction sent by the manager submission queue state machine, so that the external root complex extracts the manager submission queue and writes the manager queue data corresponding to the manager submission queue into the manager queue storage space, and then returns the write completion data corresponding to each of the manager commands;

[0029] The manager queue storage space is used to store manager queue data corresponding to the manager submission queue.

[0030] In some embodiments of the present application, it further includes: a manager completion queue cache that is respectively communicatively connected to the manager queue state machine and the root complex;

[0031] The manager completion queue buffer is used to receive the write completion data corresponding to each manager command sent by the root complex, and store the write completion data corresponding to each manager command in a local manager completion queue;

[0032] The manager completion queue state machine is used to record the state data of the manager completion queue in the manager completion queue cache, and determine whether the manager completion queue is currently written according to the recorded state data of the manager completion queue, and if so, send a fourth trigger instruction to the doorbell state machine;

[0033] The doorbell state machine is further used to trigger the doorbell according to the fourth trigger instruction sent by the manager completion queue state machine, so that the register obtains the state data of the manager completion queue from the manager completion queue state machine;

[0034] The register is further used to forward the state data of the manager completion queue obtained from the manager completion queue state machine to the microprocessor, so that the microprocessor can obtain the completion state information corresponding to the manager command based on the state data of the manager completion queue.

[0035] In some embodiments of the present application, it further includes: an initialization state machine that is communicatively connected to the register, the root complex, and the manager queue state machine respectively;

[0036] The register is also used to receive and store the manager command and hardware status data for initializing the NVMe system after reset sent by the microprocessor based on the NVMe protocol outside the CPU, and send the manager command for initializing the NVMe system after reset to the initialization state machine;

[0037] The initialization state machine is used to write the management command for initializing the NVMe system after reset into the manager submission queue cache.

[0038] Another aspect of the present application provides a method for data transmission between a hard disk and a memory, comprising:

[0039] The NVMe host is used to receive user commands sent by the microprocessor based on the NVMe protocol outside the CPU, write the user commands into a local user submission queue, and trigger a doorbell after the user submission queue is written, so that the external root complex extracts the user submission queue and performs data transmission between the solid-state drive and the memory according to each user command in the user submission queue, and then returns the write completion data corresponding to each user command;

[0040] The NVMe host is used to receive and store the write completion data corresponding to each of the user commands sent by the root complex, so that the microprocessor can obtain the completion status information corresponding to the user command according to the write completion data corresponding to each of the user commands.

[0041] A third aspect of the present application provides a data transmission system between a hard disk and a memory, comprising: a microprocessor, a root complex, and the NVMe host;

[0042] The NVMe host, the microprocessor and the CPU are sequentially communicated and connected;

[0043] The NVMe host is communicatively connected to the root complex;

[0044] The root complex is also connected to the solid state drive and the memory for communication.

[0045] The NVMe host provided by the present application is used to receive user commands sent by a microprocessor based on the NVMe protocol outside the CPU, write the user commands into a local user submission queue, and trigger a doorbell after the user submission queue is written, so that the external root complex extracts the user submission queue and performs data transmission between the solid-state hard disk and the memory according to each user command in the user submission queue, and then returns the write completion data corresponding to each user command; the NVMe host is also used to receive and store the write completion data corresponding to each user command sent by the root complex, so that the microprocessor can obtain the completion status information corresponding to the user command according to the write completion data corresponding to each user command, and use hardware to replace the CPU to realize the data transmission between the hard disk and the memory based on the NVMe protocol, without the need for the CPU to process operations related to the NVMe protocol, and the CPU only needs to configure the hardware and check the hardware status, which can effectively reduce the occupancy rate and resource consumption of the host CPU; at the same time, the hardware implementation has greater advantages in processing speed and latency, which can effectively improve the processing efficiency of NVMe user commands and increase throughput; and can improve the parallelism of solid-state hard disk access, and can make full use of the advantages of NVMe multi-queues.

[0046] Additional advantages, purposes, and features of the present application will be partially described in the following description, and will become partially apparent to those skilled in the art after studying the following, or may be learned from the practice of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0047] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present application. In the drawings:

[0049] Figure 1 Schematic diagram of the connection relationship between the NVMe host and other components in one embodiment of the present application.

[0050] Figure 2 This is a schematic diagram of the first structure of the NVMe host in one embodiment of the present application.

[0051] Figure 3 This is a second structural diagram of the NVMe host in one embodiment of the present application.

[0052] Figure 4 This is a third structural diagram of the NVMe host in one embodiment of the present application.

[0053] Figure 5 Schematic diagram of a data transmission method between a hard disk and a memory in an embodiment of the present application.

[0054] Figure 6 Schematic diagram of the architecture of a data transmission system between a hard disk and a memory in one embodiment of the present application.

[0055] Figure 7 A schematic diagram comparing a traditional storage architecture and a storage architecture implemented by hardware in an application example of the present application.

[0056] Figure 8 This is an example schematic diagram of a data transmission system between a hard disk and a memory in an application example of the present application.

[0057] Fig. 9 This is a schematic diagram showing an example of an internal module of an NVMe host in an application example of the present application.

[0058] Fig.10 This is a schematic diagram of the initialization workflow in an application example of the present application.

[0059] Fig.11This is a schematic diagram of the manager queue workflow in an application example of the present application.

[0060] Fig.12 This is a schematic diagram of the user queue workflow in an application example of the present application.

[0061] Reference numerals:

[0062] 100. NVMe host;

[0063] 101. Register;

[0064] 102. User submission queue state machine;

[0065] 103. User submission queue cache;

[0066] 104. Doorbell state machine;

[0067] 105. User completes queue caching;

[0068] 106. User completion queue state machine;

[0069] 107. Manager queue state machine;

[0070] 108. Manager submits queue cache;

[0071] 109. Manager queue storage space;

[0072] 110. The manager completes the queue cache;

[0073] 111. Initialize state machine;

[0074] 200, microprocessor;

[0075] 300, root complex;

[0076] 400, NVMe oF unit;

[0077] 500, RDMA unit;

[0078] 600, CPU;

[0079] 700, solid state drive;

[0080] 800. Memory. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the implementation modes and the accompanying drawings. Here, the illustrative implementation modes and descriptions of the present application are used to explain the present application, but are not intended to limit the present application.

[0082] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the scheme according to the present application are shown in the accompanying drawings, while other details that are not very relevant to the present application are omitted.

[0083] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0084] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0085] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0086] Today is a real era of data explosion, and all this data is stored on cloud servers. The core of cloud servers is storage media. With the explosive growth of data, there are higher requirements for data storage media in terms of speed and capacity. Traditional storage media are no longer competent, and the emergence of solid-state drives can just meet this demand. Solid-state drives use electronic chips to store data and do not have the mechanical parts of mechanical hard disks (HDDs). Therefore, they have overwhelming advantages over mechanical hard disks in terms of speed, latency, power consumption, and shock resistance.

[0087] Most mechanical hard disks and early solid-state disks use the SATA (Serial ATA) hard disk interface and run the Advanced Host Controller Interface (AHCI). AHCI supports the native command sequence NCQ (Native Command Queuing) function and hot-swap technology. The maximum depth of NCQ is 32, that is, the host can send up to 32 commands to the mechanical hard disk or solid-state disk for execution. Compared with the previous hard disk that could only execute commands one by one, the hard disk performance has been greatly improved. In the era of mechanical hard disks or the early days of solid-state disks, the AHCI protocol and SATA interface were sufficient to meet the system performance requirements.

[0088] With the rapid development of SSD technology, the performance of SSDs has soared, the bandwidth of underlying flash memory has become wider and wider, and the media access latency has become lower and lower. The system performance bottleneck has shifted from the bottom to the interface and protocol above. AHCI and SATA can no longer meet the requirements of high performance and low latency, so SSDs are in urgent need of faster and more efficient protocols and interfaces. Against this background, the non-volatile memory host controller interface specification NVMe (NVM Express) was born. Compared with traditional SATA and other forms of SSDs, NVMe has the characteristics of multiple queues (64K), large depth (64K), and no need for locking. It fully utilizes the characteristics of non-volatile storage media NVM (Non Volatile Memory) and has lower latency and higher bandwidth.

[0089] However, the existing data transmission method between the hard disk and the memory has increased the CPU occupancy rate and memory consumption dramatically, and also affected the read and write efficiency of the solid-state drive. Although other existing methods have proposed to use a method including an NVMe command subsystem and multiple NVM subsystems to implement the NVMe controller to execute commands without the need to interact with the ARM core, this method is not suitable for data transmission between the hard disk and the memory. There are other existing methods that use FPGA embedded PCIE hard cores, implement NVMe host controllers with RTL logic, and build an overall system to achieve high-speed reading and writing of data streams and SSD management and control, but this method inevitably requires the participation of the operating system, which will increase the complexity of the processing process and the consumption of FPGA resources.

[0090] Based on this, in order to solve the problem that the existing data transmission method between the hard disk and the memory has a dramatic increase in the CPU occupancy rate and memory consumption, and also affects the read and write efficiency of the solid-state hard disk, and at the same time can avoid the participation of the operating system, the embodiments of the present application respectively provide an NVMe host, a data transmission method between the hard disk and the memory implemented by the NVMe host, and a data transmission system between the hard disk and the memory including the NVMe host, which can be applicable to the data transmission scenario between the hard disk and the memory, and use hardware to replace the CPU to implement the data transmission between the hard disk and the memory based on the NVMe protocol, which can effectively reduce the host CPU occupancy rate and resource consumption, and can effectively improve the processing efficiency of NVMe user commands and increase throughput; the function of NVMe HOST can be fully implemented using FPGA, without the participation of the operating system, and only simple configuration registers are required to implement NVMe's ADMIN queue and IO queue, which can be more efficient and save FPGA resources.

[0091] The details are described in detail through the following examples.

[0092] Based on this, the present application embodiment provides an NVMe host, see Figure 1 , the NVMe host 100, the microprocessor 200 and the CPU 600 are sequentially connected in communication, and the NVMe host 100 is respectively connected in communication with the solid state drive 700 and the memory 800 based on the root complex 300;

[0093] The NVMe host 100 is used to process the user command sent by the microprocessor 200 based on the NVMe protocol, so that the root complex 300 is used to realize data transmission between the solid state drive 700 and the memory 800 based on the user command processing result of the NVMe host 100.

[0094] Specifically, the NVMe host 100 is used to receive the user command sent by the microprocessor 200 based on the NVMe protocol outside the CPU 600, write the user command into the local user submission queue, and trigger the doorbell after the user submission queue is written, so that the external root complex 300 extracts the user submission queue and performs data transmission between the solid state drive 700 and the memory 800 according to each user command in the user submission queue, and then returns the write completion data corresponding to each user command;

[0095] The NVMe host 100 is also used to receive and store the write completion data corresponding to each of the user commands sent by the root complex 300, so that the microprocessor 200 can obtain the completion status information corresponding to the user command according to the write completion data corresponding to each of the user commands.

[0096] In one or more embodiments of the present application, the memory may adopt a double data rate synchronous dynamic random access memory, which may be abbreviated as DDR.

[0097] In one or more embodiments of the present application, the user instruction can be written as an NVMe user instruction, which is a specific user command proposed by a user according to his own needs in the NVMe instructions mentioned in subsequent embodiments.

[0098] In one or more embodiments of the present application, the microcontroller may adopt a system on chip SOC, which is mainly responsible for inputting user instructions and acquiring hardware status.

[0099] From the above description, it can be seen that the NVMe host provided in the embodiment of the present application uses hardware instead of the CPU to realize data transmission between the hard disk and the memory based on the NVMe protocol. The CPU does not need to process operations related to the NVMe protocol. The CPU only needs to configure the hardware and check the hardware status, which can effectively reduce the occupancy rate and resource consumption of the host CPU; at the same time, the hardware implementation has greater advantages in processing speed and latency, which can effectively improve the processing efficiency of NVMe user commands and increase throughput; and it can improve the parallelism of solid-state hard disk access and make full use of the advantages of NVMe multi-queues.

[0100] In order to further improve the reliability and efficiency of data transmission between the solid-state drive and the memory based on the NVMe protocol implemented by the NVMe host, in the embodiment of the NVMe host provided in this application, see Figure 2 , the NVMe host specifically includes the following contents:

[0101] Register 101, user submission queue state machine 102 and user submission queue buffer 103 which are sequentially connected in communication;

[0102] The register 101 is used to receive and store user commands and hardware status data sent by the microprocessor 200 based on the NVMe protocol outside the CPU 600, and send the user commands to the user submission queue state machine 102;

[0103] The user submission queue state machine 102 is used to forward the user command to the user submission queue cache 103 and record the state data of the user submission queue in the user submission queue cache 103;

[0104] The user submission queue cache 103 is used to store the user submission queue.

[0105] That is, the register 101 is mainly responsible for storing user instructions and saving hardware status; the user submission queue state machine 102 is mainly used to process the status of the user submission queue; and the user submission queue cache 103 is mainly used to store the user submission queue.

[0106] In order to further improve the reliability and efficiency of data transmission between the solid-state drive and the memory based on the NVMe protocol implemented by the NVMe host, in the embodiment of the NVMe host provided in this application, see Figure 2 , the NVMe host further specifically includes the following contents:

[0107] A doorbell state machine 104 communicatively connected to the user submission queue state machine 102;

[0108] The user submission queue state machine 102 is further used to determine whether the user submission queue is currently written according to the recorded state data of the user submission queue, and if so, send a first trigger instruction to the doorbell state machine 104;

[0109] The doorbell state machine 104 is used to trigger the doorbell according to the first trigger instruction sent by the user submission queue state machine 102, so that the external root complex 300 extracts the user submission queue and performs data transmission between the solid state drive 700 and the memory 800 according to each user command in the user submission queue, and then returns the write completion data corresponding to each of the user commands.

[0110] That is, the doorbell state machine 104 is mainly used to process the doorbell.

[0111] In order to further improve the reliability and efficiency of data transmission between the solid-state drive and the memory based on the NVMe protocol implemented by the NVMe host, in the embodiment of the NVMe host provided in this application, see Figure 2 , the NVMe host further specifically includes the following contents:

[0112] A user completion queue cache 105 and a user completion queue state machine 106 for communication connections;

[0113] The user completion queue cache 105 is communicatively connected to the root complex 300, and the user completion queue state machine 106 is communicatively connected to the register 101 and the doorbell state machine 104 respectively;

[0114] The user completion queue buffer 105 is used to receive the write completion data corresponding to each of the user commands sent by the root complex 300, and store the write completion data corresponding to each of the user commands in a local user completion queue;

[0115] The user completion queue state machine 106 is used to record the state data of the user completion queue in the user completion queue cache 105, and determine whether the user completion queue is currently written according to the recorded state data of the user completion queue, and if so, send a second trigger instruction to the doorbell state machine 104;

[0116] The doorbell state machine 104 is further used to trigger the doorbell according to the second trigger instruction sent by the user completion queue state machine 106, so that the register 101 obtains the status data of the user completion queue from the user completion queue state machine 106;

[0117] The register 101 is further used to forward the status data of the user completion queue obtained from the user completion queue state machine 106 to the microprocessor 200, so that the microprocessor 200 can obtain the completion status information corresponding to the user command based on the status data of the user completion queue.

[0118] That is, the user completion queue buffer 105 is mainly used to store the user completion queue, and the user completion queue state machine 106 is mainly used to process the state of the user completion queue.

[0119] Based on the above-mentioned register 101, user submission queue state machine 102, user submission queue cache 103, doorbell state machine 104, user completion queue cache 105, and user completion queue state machine 106, the user queue workflow can be implemented: the user writes a specific user command into the user submission queue cache according to his own needs; after the user submission queue is written, the user submission queue state machine rings the doorbell to notify the root complex 300 to take the user submission queue; the root complex 300 takes the submission queue from the user submission queue cache; the root complex 300 takes the user data from the solid state drive and writes it into the memory according to the content of the user submission queue, or writes the data in the memory into the solid state drive; after the root complex 300 writes the data, it writes the completion queue into the user completion queue cache; the user completion queue state machine writes the user completion queue status into the register module, and the microprocessor 200 can obtain the user queue completion status in real time through the register.

[0120] In order to further improve the effectiveness and reliability of NVMe protocol to realize data transmission between solid state drive and memory, in the embodiment of NVMe host provided in this application, see Figure 3 , the NVMe host further specifically includes the following contents:

[0121] A communication-connected manager queue state machine 107 and a manager submission queue buffer 108;

[0122] The manager queue state machine 107 is in communication connection with the register 101;

[0123] The register 101 is also used to receive and store the manager command and hardware status data sent by the microprocessor 200 based on the NVMe protocol outside the CPU 600, and send the manager command to the manager submission queue state machine;

[0124] The manager submission queue state machine is used to forward the manager command to the manager submission queue cache 108 and record the state data of the manager submission queue in the manager submission queue cache 108;

[0125] The manager submission queue cache 108 is used to store the manager submission queue.

[0126] That is, the manager queue state machine 107 is mainly responsible for processing the states of the manager submission queue and the manager completion queue; the manager submission queue cache 108 is mainly used to store the manager submission queue.

[0127] In order to further improve the effectiveness and reliability of NVMe protocol to realize data transmission between solid state drive and memory, in the embodiment of NVMe host provided in this application, see Figure 3 , the NVMe host further specifically includes the following contents:

[0128] A manager queue storage space 109 in communication with the root complex 300;

[0129] The manager queue state machine 107 is also in communication connection with the doorbell state machine 104;

[0130] The manager submission queue state machine is further used to determine whether the manager submission queue is currently written according to the recorded state data of the manager submission queue, and if so, send a third trigger instruction to the doorbell state machine 104;

[0131] The doorbell state machine 104 is used to trigger the doorbell according to the third trigger instruction sent by the manager submission queue state machine, so that the external root complex 300 extracts the manager submission queue and writes the manager queue data corresponding to the manager submission queue into the manager queue storage space 109, and then returns the write completion data corresponding to each of the manager commands;

[0132] The manager queue storage space 109 is used to store manager queue data corresponding to the manager submission queue.

[0133] That is, the manager queue storage space 109 is used to store manager queue data. It is understandable that in the embodiment of the present application, a user queue storage space may also be provided, and the function of the user queue storage space is similar to that of the manager queue storage space 109, and is used to store user queue data.

[0134] In order to further improve the effectiveness and reliability of NVMe protocol to realize data transmission between solid state drive and memory, in the embodiment of NVMe host provided in this application, see Figure 3 , the NVMe host further specifically includes the following contents:

[0135] A manager completion queue cache 110 that is communicatively coupled to the manager queue state machine 107 and the root complex 300;

[0136] The manager completion queue buffer 110 is used to receive the write completion data corresponding to each manager command sent by the root complex 300, and store the write completion data corresponding to each manager command in a local manager completion queue;

[0137] The manager completion queue state machine is used to record the state data of the manager completion queue in the manager completion queue cache 110, and determine whether the manager completion queue is currently written according to the recorded state data of the manager completion queue, and if so, send a fourth trigger instruction to the doorbell state machine 104;

[0138] The doorbell state machine 104 is further used to trigger the doorbell according to the fourth trigger instruction sent by the manager completion queue state machine, so that the register 101 obtains the state data of the manager completion queue from the manager completion queue state machine;

[0139] The register 101 is further used to forward the state data of the manager completion queue obtained from the manager completion queue state machine to the microprocessor 200, so that the microprocessor 200 can obtain the completion state information corresponding to the manager command based on the state data of the manager completion queue.

[0140] Based on the above-mentioned register 101, doorbell state machine 104, manager queue state machine 107, manager submission queue cache 108, manager queue storage space 109 and manager completion queue cache 110, the manager queue workflow can be implemented: the manager writes a specific manager command into the manager submission queue cache according to his own needs; after the manager submission queue is written, the manager queue state machine rings the doorbell to notify the root complex 300 to take the manager submission queue; the root complex 300 takes the submission queue from the manager submission queue cache; the root complex 300 writes the manager queue data into the manager queue storage space according to the content of the manager submission queue; after the root complex 300 writes the data, it writes the completion queue into the manager completion queue cache; the manager queue state machine writes the manager completion queue status into the register module, and the microprocessor 200 can obtain the manager queue completion status in real time through the register.

[0141] In order to further improve the effectiveness and reliability of NVMe protocol to realize data transmission between solid state drive and memory, in the embodiment of NVMe host provided in this application, see Figure 4 , the NVMe host further specifically includes the following contents:

[0142] an initialization state machine 111 that is communicatively connected to the register 101, the root complex 300, and the manager queue state machine 107;

[0143] The register 101 is also used to receive and store the manager command and hardware status data for initializing the NVMe system after reset sent by the microprocessor 200 based on the NVMe protocol outside the CPU 600, and send the manager command for initializing the NVMe system after reset to the initialization state machine 111;

[0144] The initialization state machine 111 is used to write the management command for initializing the NVMe system after reset into the manager submission queue cache 108.

[0145] Based on the above-mentioned register 101, doorbell state machine 104, manager queue state machine 107, manager submission queue cache 108, manager queue storage space 109, manager completion queue cache 110 and initialization state machine 111, the initialization workflow can be implemented: after the system is reset, the initialization state machine writes the manager command required by the NVMe system into the manager submission queue cache; after the manager submission queue is written, the manager queue state machine rings the doorbell to notify the root complex 300 to take the manager submission queue; the root complex 300 takes the submission queue from the manager submission queue cache; the root complex 300 writes the manager queue data into the manager queue storage space according to the content of the manager submission queue; after the root complex 300 writes the data, it writes the completion queue into the manager completion queue cache; the manager queue state machine writes the manager completion queue status into the register module, and the SOC can obtain the manager queue completion status in real time through the register.

[0146] Based on the above-mentioned NVMe host embodiment, the present application also provides an embodiment of a method for data transmission between a hard disk and a memory, see Figure 5 The data transmission method between the hard disk and the memory specifically includes the following contents:

[0147] Step 10: Using the NVMe host to receive the user command sent by the microprocessor based on the NVMe protocol outside the CPU, write the user command into a local user submission queue, and trigger a doorbell after the user submission queue is written, so that the external root complex extracts the user submission queue and performs data transmission between the solid-state drive and the memory according to each user command in the user submission queue, and then returns the write completion data corresponding to each user command;

[0148] Step 20: Use the NVMe host to receive and store the write completion data corresponding to each of the user commands sent by the root complex, so that the microprocessor can obtain the completion status information corresponding to the user command according to the write completion data corresponding to each of the user commands.

[0149] It can be understood that the specific structure and function of the NVMe host mentioned in this embodiment refer to the aforementioned NVMe host embodiment and will not be repeated here.

[0150] From the above description, it can be seen that the data transmission method between the hard disk and the memory provided in the embodiment of the present application uses hardware instead of the CPU to realize the data transmission between the hard disk and the memory based on the NVMe protocol. The CPU does not need to process the operations related to the NVMe protocol. The CPU only needs to configure the hardware and check the hardware status, which can effectively reduce the occupancy rate and resource consumption of the host CPU; at the same time, the hardware implementation has greater advantages in processing speed and latency, which can effectively improve the processing efficiency of NVMe user commands and increase throughput; and it can improve the parallelism of solid-state hard disk access and make full use of the advantages of NVMe multi-queues.

[0151] Based on the above-mentioned NVMe host and / or hard disk and memory data transmission method embodiment, the present application also provides a hard disk and memory data transmission system embodiment, see Figure 6 , the data transmission system between the hard disk and the memory specifically includes the following contents:

[0152] Microprocessor 200, root complex 300 and NVMe host 100;

[0153] The NVMe host 100, the microprocessor 200 and the CPU 600 are sequentially connected in communication;

[0154] The NVMe host 100 is communicatively connected to the root complex 300;

[0155] The root complex 300 is also connected to the solid state drive 700 and the memory 800 for communication.

[0156] Furthermore, the NVMe host 100 in the data transmission system between the hard disk and the memory is also connected to an NVMe oF unit 400, and the NVMe oF unit 400 is connected to an RDMA unit 500. The RDMA unit 500 is respectively communicated with the NVMe oF unit 400 and the memory 800 to realize the mutual conversion between NVMe commands and RDMA commands, as well as the submission of the user extraction queue and the acquisition of the user completion queue in the NVMe host 100.

[0157] It can be understood that the specific structure and function of the NVMe host mentioned in this embodiment refer to the aforementioned NVMe host embodiment and will not be repeated here.

[0158] From the above description, it can be seen that the data transmission system between the hard disk and the memory provided in the embodiment of the present application uses hardware to replace the CPU to realize the data transmission between the hard disk and the memory based on the NVMe protocol. The CPU does not need to process the operations related to the NVMe protocol. The CPU only needs to configure the hardware and check the hardware status, which can effectively reduce the occupancy rate and resource consumption of the host CPU; at the same time, the hardware implementation has greater advantages in processing speed and latency, which can effectively improve the processing efficiency of NVMe user commands and increase throughput; and it can improve the parallelism of solid-state hard disk access and make full use of the advantages of NVMe multi-queues.

[0159] In order to further illustrate the above-mentioned NVMe host, data transmission method between hard disk and memory, and data transmission system between hard disk and memory, this application also provides an application example of hardware implementation of NVMe disk. Figure 7 As shown in the figure, the implementation scheme of SSD access is implemented by hardware. The CPU does not need to process NVMe protocol-related operations on the data storage side. The CPU only needs to configure the hardware and check the hardware status, which greatly reduces the CPU performance consumption. At the same time, hardware implementation has greater advantages in processing speed and latency.

[0160] An example of a data transmission system between hard disk and memory is Figure 8 As shown, it mainly includes 7 modules, namely, SOC as the system's microprocessor, RDMA, NVMe of, SSD controller (the SSD controller is the root complex, which can be abbreviated as RC), NVMe host (which can be written as NVMe HOST) that processes the NVMe protocol, memory for data sharing (which can be written as DDR), and solid-state drive (which can be written as SSD) as a data storage medium.

[0161] Among them, the main function of the SOC part is to obtain the system status, monitor the hardware, and issue and process NVMe admin instructions.

[0162] The RDMA network part mainly processes network data.

[0163] SSD is used to store data.

[0164] RC connects the NVMe HOST, SSD and DDR, and finally moves data between SSD and DDR through PCIE.

[0165] DDR is used to store temporary data from RDMA, SOC, and SSD to achieve data transmission and sharing.

[0166] NVMe of implements the mutual conversion between NVMe commands and RDMA commands, as well as the submission of SQ and acquisition of CQ in some IO commands of NVMe HOST.

[0167] The NVMe HOST that processes the NVMe protocol implements the NVMe protocol through hardware.

[0168] In the data transmission system between the hard disk and the memory, an internal module of the NVMe host is exemplified by Fig. 9 , the functions and specific workflows of each module are as follows:

[0169] (1) SOC is mainly responsible for inputting user instructions and obtaining hardware status.

[0170] (2) The register module (also written as register or REGIDTERS) is mainly responsible for storing user instructions and saving hardware status.

[0171] (3) The initialization state machine (also written as INITIALIZATION STATE MACHINE) is mainly responsible for the execution of administrator commands during system initialization.

[0172] (4) The administrator submission queue buffer (also written as ADMIN SQ FIFO) is mainly used to store the administrator submission queue.

[0173] (5) The administrator completion queue buffer (also written as ADMIN CQ FIFO) is mainly used to store the administrator completion queue.

[0174] (6) The administrator queue state machine (also written as ADMIN STATE MACHINE) is mainly responsible for processing the status of the administrator submission queue and completion queue.

[0175] (7) Not in Fig. 9 The manager queue storage space (also written as ADMIN MEM) shown in the figure is used to store manager queue data; and the user queue storage space is used to store user queues. Fig. 9 The physical region page list (PRP LIST) in.

[0176] (8) Doorbell state machine (also written as DOORBELL STATE MACHINE) Module 8 is the doorbell state machine module, which is mainly used to process the doorbell.

[0177] (9) User submission queue cache (also written as NVMe SQ FIFO) is mainly used to store user submission queues.

[0178] (10) The user completion queue buffer (also written as NVMe CQ FIFO) is mainly used to store the user completion queue.

[0179] (11) The user submission queue state machine (also written as SQ STATE MACHINE) is mainly used to process the status of the user submission queue.

[0180] (12) The user completion queue state machine (also written as CQ STATE MACHINE) is mainly used to process the status of the user completion queue.

[0181] (13) DDR is the memory, which is mainly responsible for the data cache movement of the user queue.

[0182] (14) Root Complex: It is mainly responsible for queue interaction with the NVMe host and reading and writing SSDs.

[0183] (15) The hard disk (SSD) is mainly responsible for the permanent storage of data. It is understandable that the hard disk mentioned in one or more embodiments of the present application is the abbreviation of the solid state drive.

[0184] Among them, AXl_LITE and AX14 are both interfaces in the AXI work of FPGA. Data can be transmitted bidirectionally between the host and the slave. AXl_LITE does not support burst reading and writing; AX14 supports a maximum of 256 burst reading and writing.

[0185] Based on the above content, this application example provides initialization workflow, management queue workflow and user queue workflow respectively, which are described as follows:

[0186] First, see Fig.10 The initialization workflow specifically includes the following contents:

[0187] S1: Write ADMIN SQ: After the system is reset, the initialization state machine writes the administrator commands required by the NVMe system into the administrator submission queue cache.

[0188] S2: Ringing the doorbell: After the manager submits the queue, the manager queue state machine rings the doorbell to notify RC to take the manager's submitted queue.

[0189] S3: Get ADMIN SQ: RC takes the submission queue from the administrator submission queue cache.

[0190] S4: Write ADMIN data: RC writes the administrator queue data into the administrator queue storage space according to the contents of the administrator submission queue.

[0191] S5: Write ADMIN CQ: After RC finishes writing the data, it writes the completion queue into the administrator completion queue cache.

[0192] S6: Read ADMIN CQ status: The administrator queue state machine writes the administrator completion queue status into the register module, and the SOC can obtain the administrator queue completion status in real time through the register.

[0193] Second, see Fig.11 The management queue workflow specifically includes the following contents:

[0194] S1: Write ADMIN SQ: The user writes specific administrator commands to the administrator submission queue cache according to their own needs.

[0195] S2: Ringing the doorbell: After the manager submits the queue, the manager queue state machine rings the doorbell to notify RC to take the manager's submitted queue.

[0196] S3: Get ADMIN SQ: RC takes the submission queue from the administrator submission queue cache.

[0197] S4: Write ADMIN data: RC writes the administrator queue data into the administrator queue storage space according to the contents of the administrator submission queue.

[0198] S5: Write ADMIN CQ: After RC finishes writing the data, it writes the completion queue into the administrator completion queue cache.

[0199] S6: Read ADMIN CQ status: The administrator queue state machine writes the administrator completion queue status into the register module, and the SOC can obtain the administrator queue completion status in real time through the register.

[0200] Third, see Fig.12 , the user queue workflow specifically includes the following contents:

[0201] S1: Write IO SQ: The user writes specific user commands to the user submission queue cache according to their own needs; it is understandable that the IO SQ can also be written as NVMe SQ.

[0202] S2: Ringing the doorbell: After the user submission queue is written, the user submission queue state machine rings the doorbell to notify RC to take the user submission queue.

[0203] S3: Get IO SQ: RC takes the submission queue from the user submission queue cache.

[0204] S4: Write IO data: RC takes the user queue data from DDR and writes it to SSD or writes the SSD data to DDR according to the content of the user submission queue.

[0205] S5: Write IO CQ: After RC writes the data, it writes the completion queue into the user completion queue cache; IO CQ can also be written as NVMe CQ.

[0206] S6: Read IO CQ status: The user completion queue state machine writes the user completion queue status into the register module, and the SOC can obtain the user queue completion status in real time through the register.

[0207] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0208] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0209] In the present application, features described and / or illustrated for one embodiment may be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0210] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An NVMe host, characterized in that: The NVMe host is used to receive user commands sent by the microprocessor based on the NVMe protocol outside the CPU, write the user commands into a local user submission queue, and trigger a doorbell after the user submission queue is written, so that the external root complex extracts the user submission queue and performs data transmission between the hard disk and the memory according to each user command in the user submission queue, and then returns the write completion data corresponding to each user command; The NVMe host is also used to receive and store the write completion data corresponding to each of the user commands sent by the root complex, so that the microprocessor can obtain the completion status information corresponding to the user command according to the write completion data corresponding to each of the user commands; The NVMe host includes: a register, a user submission queue state machine and a user submission queue cache connected in sequence; The register is used to receive and store user commands and hardware status data sent by the microprocessor based on the NVMe protocol outside the CPU, and send the user command to the user submission queue state machine; The user submission queue state machine is used to forward the user command to the user submission queue cache and record the state data of the user submission queue in the user submission queue cache; The user submission queue cache is used to store the user submission queue; The NVMe host also includes: a doorbell state machine that is communicatively connected to the user submission queue state machine; The user submission queue state machine is further used to determine whether the user submission queue is currently written according to the recorded state data of the user submission queue, and if so, send a first trigger instruction to the doorbell state machine; The doorbell state machine is used to trigger the doorbell according to the first trigger instruction sent by the user submission queue state machine, so that the external root complex extracts the user submission queue and performs data transmission between the hard disk and the memory according to each user command in the user submission queue, and then returns the write completion data corresponding to each user command; The NVMe host also includes: a user completion queue cache and a user completion queue state machine for communication connection; The user completion queue buffer is communicatively connected to the root complex, and the user completion queue state machine is communicatively connected to the register and the doorbell state machine respectively; The user completion queue buffer is used to receive the write completion data corresponding to each of the user commands sent by the root complex, and store the write completion data corresponding to each of the user commands to a local user completion queue; The user completion queue state machine is used to record the state data of the user completion queue in the user completion queue cache, and determine whether the user completion queue is currently written according to the recorded state data of the user completion queue, and if so, send a second trigger instruction to the doorbell state machine; The doorbell state machine is further used to trigger the doorbell according to the second trigger instruction sent by the user completion queue state machine, so that the register obtains the state data of the user completion queue from the user completion queue state machine; The register is further used to forward the status data of the user completion queue obtained from the user completion queue state machine to the microprocessor, so that the microprocessor can obtain the completion status information corresponding to the user command based on the status data of the user completion queue.

2. The NVMe host according to claim 1, characterized in that: Also includes: The manager queue state machine and the manager submission queue cache for communication connections; The manager queue state machine is communicatively connected to the register; The register is also used to receive and store manager commands and hardware status data sent by the microprocessor based on the NVMe protocol outside the CPU, and send the manager command to the manager submission queue state machine; The manager submission queue state machine is used to forward the manager command to the manager submission queue cache and record the state data of the manager submission queue in the manager submission queue cache; The manager submission queue buffer is used to store the manager submission queue.

3. The NVMe host according to claim 2, characterized in that: Also includes: A manager queue storage space communicatively coupled to the root complex; The manager queue state machine is also in communication connection with the doorbell state machine; The manager submission queue state machine is further used to determine whether the manager submission queue is currently written according to the recorded state data of the manager submission queue, and if so, send a third trigger instruction to the doorbell state machine; The doorbell state machine is used to trigger the doorbell according to the third trigger instruction sent by the manager submission queue state machine, so that the external root complex extracts the manager submission queue and writes the manager queue data corresponding to the manager submission queue into the manager queue storage space, and then returns the write completion data corresponding to each of the manager commands; The manager queue storage space is used to store manager queue data corresponding to the manager submission queue.

4. The NVMe host according to claim 3, characterized in that: Also includes: A manager completion queue cache is respectively connected to the manager queue state machine and the root complex in communication; The manager completion queue buffer is used to receive the write completion data corresponding to each manager command sent by the root complex, and store the write completion data corresponding to each manager command in a local manager completion queue; The manager completion queue state machine is used to record the state data of the manager completion queue in the manager completion queue cache, and determine whether the manager completion queue is currently written according to the recorded state data of the manager completion queue, and if so, send a fourth trigger instruction to the doorbell state machine; The doorbell state machine is further used to trigger the doorbell according to the fourth trigger instruction sent by the manager completion queue state machine, so that the register obtains the state data of the manager completion queue from the manager completion queue state machine; The register is further used to forward the state data of the manager completion queue obtained from the manager completion queue state machine to the microprocessor, so that the microprocessor can obtain the completion state information corresponding to the manager command based on the state data of the manager completion queue.

5. The NVMe host according to claim 4, characterized in that: Also includes: an initialization state machine communicatively coupled to the register, the root complex, and the manager queue state machine, respectively; The register is also used to receive and store the manager command and hardware status data for initializing the NVMe system after reset sent by the microprocessor based on the NVMe protocol outside the CPU, and send the manager command for initializing the NVMe system after reset to the initialization state machine; The initialization state machine is used to write the management command for initializing the NVMe system after reset into the manager submission queue cache.

6. A method for data transmission between a hard disk and a memory, characterized in that: include: The NVMe host according to any one of claims 1 to 5 is used to receive a user command sent by a microprocessor based on the NVMe protocol outside the CPU, write the user command into a local user submission queue, and trigger a doorbell after the user submission queue is written, so that an external root complex extracts the user submission queue and performs data transmission between a hard disk and a memory according to each user command in the user submission queue, and then returns the write completion data corresponding to each user command; The NVMe host is used to receive and store the write completion data corresponding to each of the user commands sent by the root complex, so that the microprocessor can obtain the completion status information corresponding to the user command according to the write completion data corresponding to each of the user commands.

7. A data transmission system between a hard disk and a memory, characterized in that: include: A microprocessor, a root complex, and an NVMe host as claimed in any one of claims 1 to 5; The NVMe host, the microprocessor and the CPU are sequentially communicated and connected; The NVMe host is communicatively connected to the root complex; The root complex is also connected to the solid state drive and the memory for communication.

Images