An Embedded System Based on NVMe oF RDMA

Abstract

The application provides an NVME oF RDMA-based embedded system, which comprises an initiator for initiating a data reading request or a writing request to a storage target of the target, a target for performing a corresponding data operation according to the request of the initiator, and an NVMe-oF Target unit for controlling information interaction with the initiator; the initiator is connected with the target through an optical fiber; the NVMe-oF Target unit is arranged in the target; the system constructs a storage access architecture with software and hardware cooperation, adopts a kernel protocol stack bypass mechanism, effectively reduces the processing delay in the data transmission path, meets the real-time requirement in the embedded scene, and solves the high-latency bottleneck of data transmission on a standard Linux platform.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to an embedded system based on NVMe oF RDMA. Background Technology

[0002] NVMe (Non-Volatile Memory Express) is a high-performance storage protocol designed specifically for high-speed flash memory (such as SSDs), aiming to replace the traditional AHCI (SATA interface protocol) and utilize the low latency and high bandwidth characteristics of PCIe. NVMe-oF (NVMe over Fabrics) is an extension of the NVMe protocol, allowing access to remote NVMe storage devices over networks (such as TCP and InfiniBand), not limited to local PCIe SSDs. NVMe over RDMA (NVMe-oF RDMA) is a high-performance implementation based on RDMA (Remote Direct Memory Access). In the current fields of edge computing, Industrial Internet of Things (IIoT), and real-time embedded systems, the demand for high-performance storage access is surging. However, traditional embedded systems are limited by low-speed storage interfaces (such as SPI Flash and SDIO) and limited computing resources, making it difficult to meet the requirements of low-latency, high-throughput storage access.

[0003] In existing technologies, NVMe-oF RDMA, as a high-performance remote storage protocol, runs on standard Linux servers. However, its complex protocol stack and hardware dependencies make it difficult to directly apply to embedded environments, such as deploying MCUs that do not support operating systems. Furthermore, the real-time performance of the NVMe-oF RDMA protocol running on standard Linux servers is affected by kernel scheduling. The data I / O process is handled by the kernel protocol stack, which consumes CPU resources, resulting in significant data transmission latency and response time on standard Linux systems. The aforementioned methods for implementing the NVMe-oF RDMA protocol on standard Linux platforms rely on RDMA network card hardware, and data processing is subject to kernel protocol stack intervention. Therefore, in embedded application scenarios, the initiator's access to SSD memory data on the target device cannot meet real-time requirements, resulting in high latency issues for remote storage access on standard Linux platforms.

[0004] Therefore, providing an NVME oF RDMA-based embedded system that can reduce processing latency in the data transmission path while meeting the real-time requirements of embedded scenarios is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention aims to provide an embedded system based on NVME oF RDMA. By constructing a hardware-software co-operated storage access architecture and adopting a kernel protocol stack bypass mechanism, the system effectively reduces processing latency in the data transmission path, meets the real-time requirements of embedded scenarios, and solves the high latency bottleneck of data transmission on the standard Linux platform.

[0006] The first objective of this invention is to provide an embedded system based on NVMe oF RDMA;

[0007] The technical solution provided by this invention is as follows:

[0008] An embedded system based on NVMe-oF RDMA includes: an initiator, a target, and an NVMe-oF Target unit;

[0009] The initiating end is connected to the target end via an optical fiber;

[0010] The NVMe-oF Target unit is disposed in the target end;

[0011] The initiating end is used to initiate a data read request or write request to the storage target of the target end;

[0012] The target terminal is used to perform corresponding data operations according to the request from the initiating terminal;

[0013] The NVMe-oF Target unit is used to control information interaction with the initiating end.

[0014] Preferably, the target terminal further includes: a main control chip;

[0015] The main control chip is connected to the NVMe-oF Target unit via a PCIe interface.

[0016] Preferably, the NVMe-oF Target unit is also used to complete the configuration of the main control chip.

[0017] Preferably, the NVMe-oF Target unit specifically includes: a hardware resource initialization module, a software resource initialization module, a CM link establishment and transmission module, and an admin command processing module;

[0018] The software resource initialization module is connected to the hardware resource initialization module and the CM link establishment and transmission module, respectively.

[0019] The CM link establishment and transmission module is connected to the admin command processing module;

[0020] The hardware resource initialization module is used to complete the configuration of the main control chip and enable the main control chip to enter the working state.

[0021] The software resource initialization module is used to complete the initialization of the NVMe-oF subsystem and the configuration of the NVMe controller;

[0022] The CM connection establishment transmission module is used to parse the CM connection establishment request message submitted by the initiating end, and to construct a CM connection establishment response message and send it to the initiating end.

[0023] The admin command processing module is used to parse the admin command submitted by the initiating end and construct a completion message to respond to the initiating end.

[0024] Preferably, the hardware resource initialization module is specifically used to configure the CMAC IP core, the PCIe interface, the ERNIC IP core, the NVMEOF IP core, the interrupt controller, and the UART serial port.

[0025] Preferably, the hardware resource initialization module is used to complete the configuration of the CMAC IP core, specifically including:

[0026] Configure the interface receiving and sending functions of the CMAC IP core through the hardware resource initialization module.

[0027] Preferably, the hardware resource initialization module is used to complete the configuration of the PCIe interface, specifically including:

[0028] The hardware resource initialization module sets the link width and speed, configures the BAR space size, and configures the AXI4 interface address mapping of the DMA engine.

[0029] Preferably, the hardware resource initialization module is used to complete the configuration of the ERNIC IP core, specifically including:

[0030] The hardware resource initialization module configures the starting addresses of the registers and buffers required for communication between the ERNIC and other IP cores.

[0031] Preferably, the hardware resource initialization module is used to complete the configuration of the NVMEOF IP core, specifically including:

[0032] Configure the SSD controller and SSD namespace information through the hardware resource initialization module.

[0033] Preferably, the hardware resource initialization module is used to complete the configuration of the interrupt controller, specifically including:

[0034] The interrupt controller is initialized through the hardware resource initialization module, the interrupt service routine of the RNIC is registered, and the interrupt controller and processor exception handling are enabled.

[0035] Preferably, the hardware resource initialization module is used to complete the configuration of the UART serial port, specifically including:

[0036] The hardware resource initialization module initializes the UART Lite hardware, registers the UART interrupt service routine uart_handler, and enables UART interrupts.

[0037] Compared with existing technologies, this invention provides an embedded system based on NVMe oF RDMA, comprising: an initiating end for initiating data read or write requests to a storage target at the target end; a target end for performing corresponding data operations according to the request from the initiating end; and an NVMe-oFTarget unit for controlling information interaction with the initiating end; the initiating end is connected to the target end via optical fiber; the NVMe-oF Target unit is disposed in the target end; this system effectively reduces processing latency in the data transmission path by constructing a hardware-software co-working storage access architecture and adopting a kernel protocol stack bypass mechanism, thereby meeting the real-time requirements of embedded scenarios and solving the high latency bottleneck of data transmission on the standard Linux platform. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A schematic diagram of the structure of an embedded system based on NVME oF RDMA is provided for one embodiment;

[0040] Figure 2 A schematic diagram of the target end provided in one embodiment;

[0041] Figure 3 This is a schematic diagram of communication between the initiator and the target end, provided as an embodiment. Detailed Implementation

[0042] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] It should be noted that when a component is referred to as being "fixed to" or "set on" another component, it can be directly on or indirectly set on the other component; when a component is referred to as being "connected to" another component, it can be directly connected to or indirectly connected to the other component.

[0044] It should be noted that the structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which this application can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size should still fall within the scope of the technical content disclosed in this application, provided that they do not affect the effects and purposes that this application can produce.

[0045] like Figures 1 to 2 As shown, this embodiment provides an embedded system based on NVMe-oF RDMA, including: an initiator 1, a target 2, and an NVMe-oF Target unit 3;

[0046] The initiating end 1 is connected to the target end 2 via an optical fiber;

[0047] The NVMe-oF Target unit 3 is disposed in the target end 2;

[0048] The initiating end 1 is used to initiate a data read request or write request to the storage target of the target end 2;

[0049] The target terminal 2 is used to perform corresponding data operations according to the request of the initiating terminal 1;

[0050] The NVMe-oF Target unit 3 is used to control information interaction with the initiating end 1.

[0051] In practical applications, to address the high latency bottleneck of data transmission on standard Linux platforms, effectively reduce processing latency in the data transmission path, and meet the real-time requirements of embedded scenarios, an embedded system based on NVMeoF RDMA was designed. This system includes an initiator, a target, and an NVMe-oF Target unit. The initiator is connected to the target via fiber optic cable, and the NVMe-oF Target unit is located within the target. The initiator is a high-performance server, computer, or other type of host running a standard Linux system and equipped with a standard RDMA network card supporting 100G interfaces, such as the Mellanox ConnectX-6, which supports the RoCEv2 protocol. The initiator initiates remote storage requests, reading or writing data to the storage target (such as a storage array or disk) on the remote target, using the NVMe CLI tool for management and data reading / writing to the remote NVMe device. The target processes the read or write requests from the initiator and performs the corresponding data operations. The target FPGA integrates an embedded MCU, which runs a lightweight NVMe-oF Target unit to directly manage the main controller hardware.

[0052] Specifically, such as Figure 3 The diagram illustrates the communication flow between the initiator and target in this invention, using the Property Get management command as an example. The specific flow is as follows:

[0053] 1. Connection Establishment Phase

[0054] Step 301: The initiator sends an NVMe-oF Connect request to the target to establish an NVMe-oF session, pass parameters such as the host NQN (NVMe Qualified Name) and subsystem NQN, and uses RC Send Only mode without carrying data payload.

[0055] Step 302: The target sends an RRoCE ACK to the initiator. This acknowledges receipt of the connection request and includes a protocol layer or link layer ACK.

[0056] Step 303: The target sends an RDMA Read Request to the initiator to actively read connection context data, such as authentication information, from the initiator's memory.

[0057] Step 304: The initiator returns an RDMA Read Response to the target, providing the data requested by the target, such as NVMe-oF connection parameters, with offset 0 indicating the starting position.

[0058] Step 305: The target sends an NVMe-oF CQE to the initiator to notify the initiator that the connection has been successfully established.

[0059] Step 306: The initiator replies with an RRoCE ACK to the target, confirming receipt of the CQE and completing the connection handshake.

[0060] 2. Controller Attribute Query Phase

[0061] Step 307: The initiator sends a Property Get Controller Capabilities request to the target; the purpose is to query the target's controller capabilities (such as supported NVMe versions, maximum number of queues).

[0062] Step 308: The target sends an RRoCE ACK to the initiator, acknowledging receipt of the attribute query request.

[0063] Step 309: The target sends NVMe-oF CQE Property Get ControllerCapabilities to the initiator, along with controller capability data (such as Max Queue Count and NVMeVersion).

[0064] Step 310: The initiator replies with an RRoCE ACK to the target. This confirms receipt of the attribute response and completes the query process.

[0065] By employing an operating system-free embedded system architecture at the target end, the FPGA main control chip is directly managed and controlled, eliminating the performance interference caused by traditional operating system task scheduling. This optimization scheme achieves efficient operation of the NVMe-of-Target firmware by bypassing kernel-mode overhead, significantly reducing I / O processing latency and improving the determinism of memory access.

[0066] Preferably, the target terminal 2 further includes: a main control chip 21;

[0067] The main control chip 21 is connected to the NVMe-oF Target unit 3 via a PCIe interface.

[0068] In practical applications, the target also includes an FPGA main controller chip. The FPGA main controller chip connects to the NVMe-oF Target unit via a PCIe interface, allowing the NVMe-oF Target unit to access the FPGA main controller chip through the PCIe interface. Embedded systems based on NVMe-oF RDMA can directly read and write to the FPGA main controller chip, reducing resource overhead caused by intermediate buffers and significantly reducing latency. In this embodiment, the FPGA main controller chip used is the Xilinx XCVU13P-2FHGB2104I FPGA, belonging to the Ultra Scale+ series of high-performance FPGAs. It has powerful logic resources and high-speed interface capabilities, suitable for high-performance computing, storage acceleration, and network processing scenarios. The FPGA main controller chip provides high logic density and DSP / BRAM resources, supports PCIe Gen4x16, and provides bandwidth up to 32GT / s to meet high-speed data transmission requirements and possess high-performance computing capabilities. The FPGA main controller chip has an external Micron DDR4 memory interface, supporting high-bandwidth memory access, suitable for caching and data processing. In addition, this module supports NVMe-oF and can be combined with the RoCEv2 protocol of RDMA to achieve low-latency storage access.

[0069] Preferably, the NVMe-oF Target unit 3 is also used to complete the configuration of the main control chip 21.

[0070] In practical applications, the NVMe-oF Target unit has two main functions: in addition to controlling and exchanging information with the initiator, it also manages and configures the FPGA main controller chip hardware. By adopting kernel bypass mode to communicate efficiently with the FPGA main controller chip, it achieves precise control of the FPGA main controller chip.

[0071] Preferably, the NVMe-oF Target unit 3 specifically includes: a hardware resource initialization module 31, a software resource initialization module 32, a CM link establishment and transmission module 33, and an admin command processing module 34;

[0072] The software resource initialization module 32 is connected to the hardware resource initialization module 31 and the CM link establishment and transmission module 33, respectively.

[0073] The CM link establishment and transmission module 33 is connected to the admin command processing module 34;

[0074] The hardware resource initialization module 31 is used to complete the configuration of the main control chip 21 and enable the main control chip 21 to enter the working state.

[0075] The software resource initialization module 32 is used to complete the initialization of the NVMe-oF subsystem and the configuration of the NVMe controller;

[0076] The CM connection establishment transmission module 33 is used to parse the CM connection establishment request message submitted by the initiating end 1, and to construct a CM connection establishment response message and send it to the initiating end 1.

[0077] The admin command processing module 34 is used to parse the admin command submitted by the initiating end 1 and to construct completion information to respond to the initiating end 1.

[0078] In practical applications, the NVMe-oF Target unit includes: a hardware resource initialization module, a software resource initialization module, a CM connection establishment and transmission module, and an admin command processing module; the software resource initialization module is connected to both the hardware resource initialization module and the CM connection establishment and transmission module; the CM connection establishment and transmission module is connected to the admin command processing module.

[0079] The hardware resource initialization module configures all modules of the FPGA main controller chip, including the CMAC IP core, PCIe interface, ERNIC IP core, NVMeOF IP core, interrupt controller, and UART serial port, putting the FPGA main controller chip into operation. This module configures the CMAC core's interface receive and transmit functions. PCIe interface configuration includes setting the link width (x1 / x4 / x8 / x16) and speed (Gen1 / 2 / 3 / 4), configuring the BAR (BaseAddressRegister) space size, and configuring the DMA engine's AXI4 interface address mapping, etc. The ERNIC IP core configures the registers (doorbell register, enable register) and buffer start addresses (receive queue buffer, transmit queue buffer, and completion queue buffer) required for communication between ERNIC and other IP cores. The NVMe-oF IP core configures the SSD controller and SSD namespace information. The interrupt controller binds PCIe interrupt vectors to different interrupt types, supporting six interrupt types. Packet reception interrupts notify the host of new data arrival, requiring triggering receive queue processing. Specifically, RNR NACK generation interrupts are triggered when reception is not ready, necessitating retransmission or window adjustment. WQE completion interrupts indicate the completion of a work queue element (e.g., DMA transfer finished). Illegal opcode interrupts detect invalid RDMA opcodes (e.g., error messages), requiring logging and discarding. QP packet reception interrupts signal the arrival of a specific QP, used for precise notification in multi-QP scenarios. Fatal error interrupts indicate hardware malfunctions (e.g., PCIe link disconnection), requiring an emergency device reset.

[0080] The software resource initialization module initializes the NVMe-oF subsystem and configures the NVMe controller. It also records and backs up key hardware parameters of the FPGA main control chip after configuring the hardware resource initialization module, ensuring the software layer can correctly manage and control the underlying hardware resources. Core information is stored in a local custom management structure, which includes: FPGA IP core registers, ERNIC IP core related registers, the number of SSD controllers, the number of namespaces, and interrupt configuration.

[0081] After configuring the hardware and software resource initialization modules, the CM connection establishment module polls and parses the CM connection establishment request messages sent by the initiator, and constructs a CM connection establishment response message. The complete connection establishment process consists of three steps: The initiator sends a Connect Request (REQ) message, carrying local QP information, resource configuration, and other parameters. The target replies with a ConnectReply (REP) message, carrying its own QP information and parameters. The initiator sends a Ready To Use (RTU) message, indicating that the parameters are confirmed and the connection is ready. The RTU message is the final confirmation signal for RC QP connection establishment, after which both parties can reliably transmit data through this connection.

[0082] The admin command processing module is responsible for processing admin commands sent by the initiator after the QP connection is established between the initiator and the target. It performs different response processing based on the opcode type of the admin command. For example, if the initiator sends a Get Property command, the admin command processing module replies with controller capability data; if the initiator sends an Identify Namespace command, the admin command processing module replies with namespace parameter information.

[0083] A capsule is the basic unit for command transmission in the NVMe protocol, used to encapsulate NVMe commands (including Admin and I / O commands) and their related data (such as metadata or user data). The capsule is the transmission carrier, and the Admin and I / O commands are the specific content it carries. Both Admin and I / O commands are transmitted after being encapsulated in the capsule format. In this embodiment, its main functions include: encapsulating command information (encapsulating opcodes, parameters, etc., into a standard format), controlling data transmission (indicating the data buffer location via PRP or SGL), and providing command identification (uniquely identifying each command via CID), as shown in Table 1.

[0084] Table 1. Information on each field of the Capsule

[0085]

[0086] The NVMe-oF Target unit supports six admin commands: Get Log Page, Identify, SetFeatures, Get Features, Synchronous Event Request, and Keep Alive. These admin commands constitute the basic management framework of the NVMe-oF Target, as shown in Table 2, covering device information acquisition, dynamic configuration, health monitoring, and connection maintenance, and are the core guarantee for the stable operation of the NVMe storage network.

[0087] Table 2. Admin command set supported by NVMe-oF Target unit

[0088]

[0089] Preferably, the hardware resource initialization module 31 is used to complete the configuration of the CMAC IP core, specifically including:

[0090] The hardware resource initialization module 31 configures the interface receiving and sending functions of the CMAC IP core.

[0091] In practical applications, the CMAC IP core is configured to enable the receive channel, enable the transmit channel, enable RS-FEC correction at the receiver, enable RS-FEC error indication at the receiver, enable standard error indication, enable RS-FEC bidirectional transmit and receive function, and finally read the link status register cyclically until the status changes to the ready state.

[0092] Preferably, the hardware resource initialization module 31 is used to complete the configuration of the PCIe interface, specifically including:

[0093] The hardware resource initialization module 31 sets the link width and speed, configures the BAR space size, and configures the AXI4 interface address mapping of the DMA engine.

[0094] In practical applications, configuring the PCIe interface involves traversing all PCIe entities, configuring devices on the PCIe bus, such as the size of the BAR space, and performing basic configuration on the PCIe RC side, including enabling the AXI interface bridge and configuring the registers of the PCIe configuration space (vendor ID and device ID, RC mode, etc.).

[0095] Preferably, the hardware resource initialization module 31 is used to complete the configuration of the ERNIC IP core, specifically including:

[0096] The hardware resource initialization module 31 configures the starting addresses of the registers and buffers required for communication between the ERNIC and other IP cores.

[0097] In practical applications, configuring the ERNIC IP core includes QP attributes, data buffer address, CM disconnection and recovery timeout for each QP, MAC address and IP address of the network card, etc.

[0098] Preferably, the hardware resource initialization module 31 is used to complete the configuration of the NVMEOF IP core, specifically including:

[0099] The hardware resource initialization module 31 configures the information of the SSD controller and SSD namespace.

[0100] In practical applications, the configuration of NVMe-oF cores includes the number of controllers, the number of SSD controller namespaces, and the number of SSD IO queue pairs.

[0101] Preferably, the hardware resource initialization module 31 is used to complete the configuration of the interrupt controller, specifically including:

[0102] The interrupt controller is initialized through the hardware resource initialization module 31, the interrupt service routine of the RNIC is registered, and the interrupt controller and processor exception handling are enabled.

[0103] In practical applications, configuring the interrupt controller involves initializing the interrupt controller, registering the interrupt service routine of the RNIC, enabling the interrupt controller and processor exception handling, and configuring this information to ensure that interrupts can be correctly identified and handled.

[0104] Preferably, the hardware resource initialization module 31 is used to complete the configuration of the UART serial port, specifically including:

[0105] The hardware resource initialization module 31 initializes the UART Lite hardware, registers the UART interrupt service routine uart_handler, and enables UART interrupts.

[0106] In practical applications, configuring the UART serial port involves initializing the UART Lite hardware, registering the UART interrupt service routine (uart_handler), and enabling UART interrupts. The uart_handler implements the echo function by immediately sending the same data upon receiving it, verifying hardware connectivity through serial port echo.

[0107] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical, or other forms.

[0108] Furthermore, in the various embodiments of the present invention, each functional module can be fully integrated into a processor, or each module can be a separate device, or two or more modules can be integrated into a device; each functional module in the various embodiments of the present invention can be implemented in hardware or in the form of hardware plus software functional units.

[0109] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0110] It should be understood that the use of terms such as "system," "device," "unit," and / or "module" in this application is merely one method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0111] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0112] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0113] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0114] The above description is merely an embodiment of the present invention. It should be noted that those skilled in the art can make improvements without departing from the inventive concept of the present invention, but these improvements all fall within the protection scope of the present invention.

Claims

1. An NVME oF RDMA based embedded system, characterized in that, The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end.

2. The NVME oF RDMA based embedded system of claim 1, wherein, The application relates to a method for realizing NVMe-oF Target in a target end.

3. The NVME oF RDMA based embedded system of claim 1, wherein, The application relates to a method for realizing NVMe-oF Target in a target end.

4. The NVME oF RDMA based embedded system of claim 3, wherein, The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end.

5. The NVME oF RDMA based embedded system of claim 3, wherein, The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end.

6. The NVME oF RDMA based embedded system of claim 3, wherein, The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end.

7. The NVME oF RDMA based embedded system of claim 3, wherein, The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in a target end. The application relates to a method for realizing NVMe-oF Target in 8. The NVME oF RDMA based embedded system of claim 3, wherein, The hardware resource initialization module is configured to complete the configuration of an interrupt controller, specifically including: The hardware resource initialization module is configured to initialize the interrupt controller, register the interrupt service routine of the RNIC, enable the interrupt controller, and handle processor exceptions.

9. The NVME oF RDMA based embedded system of claim 3, wherein, The hardware resource initialization module is configured to complete the configuration of a UART serial port, specifically including: The hardware resource initialization module is configured to initialize the UART Lite hardware, register the UART interrupt service program uart_handler, and enable the UART interrupt.

Images