A method, system, device, and storage medium for front-end and back-end simulation of RDMA network card
By combining a front-end QEMU simulator, a back-end QEMU simulator, and an APP component, front-end and back-end simulation of RDMA network cards is achieved, solving the problem of insufficient hardware resource simulation in existing technologies and reducing chip design and verification costs.
Patent Information
- Application Number
- CN202411586798.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-07
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2044-11-07
AI Technical Summary
Existing RDMA simulation solutions fail to realistically simulate hardware resources such as queue pairs, work queue entries, and doorbells, resulting in a large gap between the simulation results and the actual device, which increases the cost of chip design and verification.
By combining a front-end QEMU simulator and a back-end QEMU simulator with APP components, the control plane and data plane data flow of the RDMA network card are simulated through the Socket mechanism. This simulates the front-end and back-end parts of the RDMA network card, including the processes of parsing hardware entries, constructing sent packets, and processing received packets.
It reduces the resource costs of real hardware simulation, saves chip design and verification costs, and improves the accuracy of simulation results.
Smart Images

Figure CN119621502B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of network card emulation, and in particular to a method, system, device and storage medium for front-end and back-end emulation of a network card with RDMA. Background Technology
[0002] Remote Direct Memory Access (RDMA) emulation technology aims to simulate RDMA operations without dedicated hardware to support development, testing, and verification. RDMA technology effectively bypasses the operating system kernel by directly transferring data between applications, significantly reducing latency and greatly improving data transfer efficiency. However, due to the high cost and technical complexity of RDMA hardware (such as InfiniBand network interface cards), testing and developing RDMA operations in a real-world hardware environment often faces numerous challenges.
[0003] To improve testing and development efficiency, several RDMA simulation solutions have emerged. However, in existing technologies, most RDMA simulation solutions do not include a dedicated RDMA hardware environment, but only simulate the...
[0004] The RDMA application programming interface (API) is used for simulation. This simulation method is significantly insufficient for hardware-based RDMA performance verification. It cannot realistically simulate hardware resources, such as the hardware layout and implementation of queue pairs (QPs), work queue entries (WQEs), and doorbells, resulting in a large gap between the simulation results and the actual device. Summary of the Invention
[0005] The technical problem to be solved by this application is to provide a method, system, device and storage medium for front-end and back-end simulation of RDMA network card, which can reduce the resource cost of real hardware simulation and greatly save chip design and verification costs.
[0006] To address the aforementioned technical problems, as one aspect of this application, a front-end and back-end simulation implementation system with an RDMA network card is provided, comprising:
[0007] The front-end QEMU simulator is used to simulate the front-end part of the RDMA network card and realize the interaction with the front-end driver.
[0008] The backend QEMU simulator is used to simulate the backend part of the RDMA network card and implement backend control path processing.
[0009] The APP component is used to simulate the RDMA message processing engine and simulate the RDMA message processing process, including at least: parsing hardware entries, constructing sent messages, processing received messages, and updating the completion queue.
[0010] The front-end QEMU simulator, the back-end QEMU simulator, and the APP component communicate with each other via a Socket mechanism to simulate the control plane data flow and data plane data flow of the RDMA network card.
[0011] The front-end QEMU simulator further includes:
[0012] The RDMA application component is used to generate RDMA requests and interact with the underlying hardware through the RDMA driver component.
[0013] The RDMA driver component is used to receive requests from the RDMA application component, convert them into appropriate instructions or data packets, and then send them to the auxiliary device component or endpoint driver component. It is also responsible for handling responses and interrupts from the underlying hardware.
[0014] Auxiliary device components are used to receive and process instructions or data packets from the RDMA driver components and interact with the endpoint driver components to coordinate the execution of RDMA operations.
[0015] The endpoint driver component is used to receive requests from the RDMA driver, forward them to the hardware endpoint of the RDMA network card, and handle interrupts and responses from the hardware endpoint, returning them to the RDMA driver or upper-layer application.
[0016] Memory components are used to interact with RDMA drivers, auxiliary devices, and endpoint drivers. In RDMA operations, they serve as source or destination memory regions for reading and writing data.
[0017] The PCIE root complex is used to simulate the PCIE bus and communicate with the endpoint driver component, and is responsible for forwarding requests from the endpoint driver component to the PCIE ep.
[0018] PCIe endpoint devices are used to emulate virtual endpoint devices, receive requests from the PCIe root complex, and perform corresponding RDMA operations.
[0019] The PCIE endpoint device further includes: a multi-message signal interrupt component, a doorbell component, a first DMA component, a TLP component, and a first socket; it at least enables read and write operations on the memory component through the first socket, as well as the injection of interrupts into the front-end OS.
[0020] The backend QEMU simulator further includes:
[0021] The second socket is used to communicate with the front-end QEMU simulator and the APP component;
[0022] Registers are used to store virtual machine status information and instructions;
[0023] TLP ring components are used to store and transmit TLP data packets;
[0024] DMA ring component, simulating data transfer between the DMA controller and memory;
[0025] The Mbox component is used to receive Mbox requests and obtain the corresponding email communication data from the memory component of the front-end QEMU simulator;
[0026] The RTOS component communicates with the register, TLP ring component, DMA ring component, and Mbox component respectively; it is used to process TLP messages, initiate front-end and back-end memory read / write requests, and initiate control plane SDMA requests and Mbox requests.
[0027] The APP component further includes:
[0028] The third socket is used to communicate with the front-end QEMU simulator and the back-end QEMU simulator;
[0029] Registers are used to store intermediate results and status information;
[0030] The second DMA component is used to simulate the RDMA message processing engine;
[0031] A thread pool component is used to manage multi-task threads during the RDMA network card emulation process. The multi-task threads include at least: request threads, completion threads, and response threads.
[0032] The aforementioned system is used to implement this method, which includes the following steps:
[0033] A front-end QEMU simulator is provided to simulate the front-end part of the RDMA network card and realize the interaction with the front-end driver;
[0034] A backend QEMU simulator is provided to simulate the backend part of the RDMA network card and realize backend control path processing;
[0035] Provide an APP component to simulate an RDMA message processing engine, and simulate the RDMA message processing process, including parsing hardware entries, constructing sent messages, processing received messages, and updating the completion queue.
[0036] The front-end QEMU simulator, the back-end QEMU simulator, and the APP component interact with each other via a Socket mechanism to simulate the control plane data flow and data plane data flow of the RDMA network card.
[0037] This further includes the step of simulating the control plane data flow of the RDMA network card, including:
[0038] The RDMA application component of the front-end QEMU simulator issues a command to request the creation of a new RDMA entry;
[0039] After receiving the command, the RDMA driver component generates an RDMA table entry and stores it in the memory component;
[0040] The RDMA driver component constructs the command to create the table entry, and notifies the RTOS component of the backend QEMU emulator through the first socket and the second socket, instructing it to obtain the table entry information and the address and size information of the memory component through the second DMA component;
[0041] After receiving the notification, the RTOS component uses DMA technology to retrieve table entry information from the memory component of the front-end QEMU simulator;
[0042] The RTOS component parses the obtained commands, extracts the table entry content from the memory component address of the front-end QEMU simulator, writes it to the register component, and notifies the APP component via Socket to cache the table entry content in the APP component's local buffer.
[0043] This further includes the step of simulating the data plane data flow of the RDMA network card, the data transmission process of which includes:
[0044] The RDMA application component of the front-end QEMU simulator populates the Send Work Queue (Send WQE) entry and stores it in the memory component;
[0045] The RDMA application component sends a doorbell signal, which is then sent to the APP component. The APP component puts the received request (req) into the thread pool and reads the work queue entry from the memory component for processing.
[0046] The APP component obtains basic information from the registers of the backend QEMU module through the request thread;
[0047] The APP component reads work queue entries (WQE) from the memory component DDR of the front-end QEMU module and processes them in the local cache through the request thread;
[0048] The APP component encapsulates the opcode in the work queue entry into an RCOEv2 message and sends it to the other end via Socket;
[0049] Add a completion (comp) task thread to perform retransmission and exception handling, populate the completion queue (CQE) entry, and write it back to the completion queue entry of the front-end QEMU simulator via Socket;
[0050] If the front-end QEMU emulator is configured to report events, the back-end QEMU emulator sends an interruption message to the front-end QEMU emulator; otherwise, only the CQE index is updated, and the front-end QEMU emulator obtains the completion message by checking the CQE update.
[0051] This further includes the step of simulating the data plane data stream of the RDMA network card, and the data reception process includes:
[0052] The RDMA application of the front-end QEMU simulator issues a receive work queue entry (Recv WQE) for receiving data, and the work queue entry is stored in the memory component;
[0053] ROCEv2 messages received from the outside are stored in the local buffer of the APP component through the third socket, and a response (resp) task is added to the thread pool to process the received messages.
[0054] The APP component reads basic information from the registers of the backend QEMU emulator through the response task thread;
[0055] The APP component reads work queue entry information from the memory component of the front-end QEMU simulator through the response task thread, parses the opcode and global identifier (GID) information in the received message, and verifies it with the local table entry; after successful verification, the payload in the message is written to the buffer specified by the received work queue entry through the third socket.
[0056] The APP component fills in the completion queue entry (CQE) information and writes it to the completion queue entry buffer of the front-end QEMU simulator via Socket; if the RDMA application component of the front-end QEMU simulator is configured to report events, it sends an interrupt message to the front-end QEMU simulator through the response task thread; otherwise, it only updates the index of the completion queue entry.
[0057] As another aspect of this application, a computer-readable storage medium is also provided, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the method described above.
[0058] As another aspect of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the functions of the aforementioned system.
[0059] Implementing this embodiment has the following beneficial effects:
[0060] This application provides a method, system, device, and storage medium for front-end and back-end simulation of RDMA network cards. By using two QEMU simulators and one APP component to simulate the hardware design and overall operation logic of RDMA network cards, the resource cost of real hardware simulation can be reduced, and the time cost of building software architecture can be reduced. It can also verify real hardware solutions, greatly saving chip design verification costs. Attached Figure Description
[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of this application.
[0062] Figure 1 A schematic diagram of the structure of an embodiment of a front-end and back-end simulation implementation system with RDMA network card provided in this application;
[0063] Figure 2 This is a schematic diagram of the main flow of an embodiment of a front-end and back-end simulation implementation method with RDMA network card provided in this application;
[0064] Figure 3 for Figure 2 A schematic diagram illustrating the principle of the simulation process of the control plane data flow involved;
[0065] Figure 4 for Figure 2 A schematic diagram illustrating the principle of the simulation process for sending data streams in the data plane involved;
[0066] Figure 5 for Figure 2 A schematic diagram illustrating the principle of the simulation process for receiving data streams in the data plane involved. Detailed Implementation
[0067] To make the objectives, technical solutions, and advantages of this application clearer, the application will be described in further detail below with reference to the accompanying drawings.
[0068] like Figure 1The diagram shown illustrates a structural schematic of an embodiment of a front-end and back-end simulation implementation system with an RDMA network card provided in this application.
[0069] In this embodiment, the system includes two QEMU simulators and a user-mode APP component.
[0070] More specifically, the front-end and back-end simulation implementation system with RDMA network card includes:
[0071] The front-end QEMU simulator 1 is used to simulate the front-end part of the RDMA network card and realize the interaction with the front-end driver; the guest OS can be a Linux operating system.
[0072] The backend QEMU simulator 2 is used to simulate the backend part of the RDMA network card and implement backend control path processing. In a specific example, its CPU can be a Cortex-M3 and the guest OS can be a real-time operating system (RTOS). The backend QEMU simulator 2 can simulate some core components of real network card devices, such as TLP ring, DMA controller, Mbox and registers.
[0073] APP component 3 is used to simulate the RDMA message processing engine and simulate the RDMA message processing process, including at least: parsing hardware entries, constructing sent messages, processing received messages, and updating the completion queue; thereby achieving the purpose of verifying the chip design.
[0074] The front-end QEMU simulator, the back-end QEMU simulator, and the APP component interact with each other via a Socket mechanism to simulate the control plane data flow and data plane data flow of the RDMA network card.
[0075] More specifically, the front-end QEMU simulator 1 further includes:
[0076] The RDMA application component is used to generate RDMA requests and interact with the underlying hardware through the RDMA driver component.
[0077] The RDMA driver component is used to receive requests from the RDMA application component, convert them into appropriate instructions or data packets, and then send them to the auxiliary device component or endpoint driver component. It is also responsible for handling responses and interrupts from the underlying hardware.
[0078] An auxiliary device (aux device) is simulated hardware in the QEMU simulator used to assist RDMA operations. It receives and processes instructions or data packets from the RDMA driver component and interacts with the endpoint driver component to coordinate the execution of RDMA operations. In specific examples, it may include additional logic and registers to support specific functions of RDMA operations, such as flow control and error handling.
[0079] The endpoint driver component is the simulated RDMA network card endpoint driver in the QEMU emulator. It is used to receive requests from the RDMA driver, forward them to the hardware endpoint of the RDMA network card, and handle interrupts and responses from the hardware endpoint, returning them to the RDMA driver or upper-layer applications.
[0080] A memory component is used to interact with the RDMA driver, auxiliary devices, and endpoint drivers. During RDMA operations, it serves as a source or destination memory region for reading and writing data. In this application, the memory component is a DDR module.
[0081] The PCIE root complex (PCIE RC) is used to simulate the PCIE bus and communicate with the endpoint driver component, and is responsible for forwarding requests from the endpoint driver component to the PCIE endpoint device (PCIE ep).
[0082] PCIe endpoint devices are used to emulate virtual endpoint devices, receive requests from the PCIe root complex, and perform corresponding RDMA operations.
[0083] The PCIe endpoint device further includes: a Multi-Message Signal Interrupt (MSI-X) component, a doorbell component, a first DMA component, a Transport Layer Protocol (TLP) component, and a first socket; it at least enables read and write operations on the memory component through the first socket, as well as the injection of interrupts into the front-end OS.
[0084] In this application, the following functions can be implemented in the front-end QEMU simulator:
[0085] The first socket provides read and write functionality to the memory component (DDR), and also allows the first socket to inject interrupt functionality into the front-end OS. This functionality can intercept the PCIe device's read bar address and send the bar-based offset and data to the back-end QEMU module via the first socket to construct TLP data.
[0086] More specifically, the backend QEMU simulator 2 further includes:
[0087] The second socket is used to communicate with the front-end QEMU simulator and the APP component;
[0088] Registers are used to store virtual machine status information and instructions;
[0089] The TLP ring component is used to store and transmit TLP packets. It is similar to a circular buffer for storing and transmitting TLP packets. The TLP ring component receives front-end information and assembles TLP messages. It interacts with the RTOS through ring head and ring tail pointers, which are simulated through registers.
[0090] A DMA ring is a data structure used for DMA transfers to simulate data transfer between the DMA controller and memory. Specifically, the DMA ring consists of a request ring (req ring) and a completion ring (cmpl ring). The req ring receives read and write requests from the RTOS to the front-end memory, while the cmpl ring notifies the RTOS of completed read and write requests.
[0091] The Mbox component is used to receive Mbox requests and obtain the corresponding email communication data from the memory component of the front-end QEMU simulator. Specifically, the MBOX component parses the table entry type of the command issued by the RTOS, stores the address and size of the front-end memory component (DDR), reads the data from the front-end DDR and caches it in the local register and the local cache of the APP component.
[0092] The RTOS component is a real-time operating system that communicates with the register, TLP ring component, DMA ring component, and Mbox component respectively; it is used to process TLP messages, initiate front-end and back-end memory read / write requests, and initiate control plane SDMA requests and Mbox requests.
[0093] In this application, the following functions can be implemented in the backend QEMU simulator:
[0094] For TLP data received and sent via the second Socket, during reception, TLP messages are constructed according to the TLP format and interact with the RTOS via the TLP ring. The RTOS processes the TLP messages. During transmission, the TLP messages sent by the RTOS via the tlpring are parsed and sent to the front end via the second Socket. This enables the RTOS to receive memory read / write requests from the front end driver via the req ring and send them to the front end via the socket. After transmission is complete, the backend QEMU emulator notifies the RTOS to complete the operation via the cmplring.
[0095] To implement the hardware functionality of Mbox, when the backend RTOS issues an Mbox request, the backend QEMU simulator will parse the address and size of the specified hardware entry stored in the frontend DDR, read it from the frontend through the second socket, and then store it in the register component.
[0096] More specifically, the APP component 3 further includes:
[0097] The third socket is used to communicate with the front-end QEMU simulator and the back-end QEMU simulator;
[0098] Registers are used to store intermediate results and status information;
[0099] The second DMA component is used to simulate the RDMA message processing engine;
[0100] The thread pool component is used to manage multi-task threads during the RDMA network card emulation process. The multi-task threads include at least: request thread (requester, req), completion thread (completer, comp), and response thread (responser, resp).
[0101] In the embodiments of this application, the APP component is used to simulate the RDMA message processing engine, parse the entries of the hardware design, obtain the entry information, construct and send messages through the request task (req) thread and the completion task (comp) thread, and process the received messages through the response task (resp) thread to implement a thread pool for the execution of req, comp and resp tasks.
[0102] like Figure 2 The diagram shown illustrates the main flow of an embodiment of a front-end and back-end simulation implementation method for an RDMA network card provided in this application. (In conjunction with...) Figures 3 to 5 As shown, in this embodiment, the method employs the aforementioned... Figure 1 The system described is implemented, and the method includes the following steps:
[0103] Step S10: Provide a front-end QEMU simulator to simulate the front-end part of the RDMA network card and realize the interaction with the front-end driver;
[0104] Step S11: Provide a backend QEMU simulator to simulate the backend part of the RDMA network card and realize backend control path processing.
[0105] Step S12: Provide an APP component to simulate an RDMA message processing engine and simulate the RDMA message processing process, including parsing hardware entries, constructing sent messages, processing received messages, and updating the completion queue.
[0106] Step S13: The front-end QEMU simulator, the back-end QEMU simulator, and the APP component interact with each other through the Socket mechanism to simulate the control plane data flow and data plane data flow of the RDMA network card.
[0107] More specifically, such as Figure 3 As shown, the method provided in this application further includes: a step of simulating the control plane data flow of the RDMA network card, including:
[0108] The RDMA application component of the front-end QEMU simulator issues a command requesting the creation of a new RDMA table entry. Upon receiving this command, the RDMA driver component generates the RDMA table entry and stores it in the DDR memory component (see...). Figure 3 Step 1);
[0109] The RDMA driver component constructs the command to create the table entry, and notifies the RTOS component of the backend QEMU emulator through the first and second sockets, instructing it to obtain the table entry information and the address and size information of the memory component through the second DMA component (see...). Figure 3 Step 2);
[0110] After receiving the notification through the second DMA component, the RTOS component uses DMA technology to retrieve table entry information from the memory component of the front-end QEMU emulator (see...). Figure 3 Step 3);
[0111] The RTOS component parses the obtained commands, extracts the table entries from the memory component address of the front-end QEMU emulator, and writes them to the register component (see...). Figure 3 In step 4), notify the APP component via Socket to cache the table entry content in the APP component's local buffer (see...). Figure 3 Step 5).
[0112] More specifically, such as Figure 4 As shown, the method provided in this application further includes: a step of simulating the data plane data flow of the RDMA network card, wherein the data transmission process includes:
[0113] The RDMA application component of the front-end QEMU simulator populates the Send Work Queue (Send WQE) entry and stores it in the DDR memory component (see...). Figure 4 Step 1);
[0114] The RDMA application component sends a doorbell signal, which is then sent to the APP component. The APP component places the received request (req) into a thread pool and reads the work queue entry (WQE) from the memory component for processing (see...). Figure 4 Step 2);
[0115] The APP component obtains basic information from the registers of the backend QEMU module via a request thread (see...). Figure 4 Step 3) is used to compare and verify the information with the information in the read work queue entries;
[0116] The APP component reads work queue entries (WQE) from the front-end memory component DDR via req and processes them in the local cache, such as performing sge permission verification (see...). Figure 4 Step 4);
[0117] After successful verification, the APP component encapsulates the opcode in the Work Queue Entry (WQE) into an RCOEv2 message and sends it to the peer via Socket (see...). Figure 4 Step 5)
[0118] Add a completion (comp) task thread to handle retransmission and exception handling, populate the completion queue (CQE) entry, and write it back to the completion queue entry of the front-end QEMU emulator via the third socket (see...). Figure 4 Step 6);
[0119] If the front-end QEMU emulator is configured to report events, the back-end QEMU emulator sends an interruption message to the front-end QEMU emulator; otherwise, only the CQE index is updated, and the front-end QEMU emulator obtains the completion message by checking the CQE update (see [link to QEMU emulator]). Figure 4 Step 7).
[0120] More specifically, such as Figure 5 As shown, the method provided in this application further includes: a step of simulating the data plane data stream of the RDMA network card, wherein the data receiving process includes:
[0121] The front-end QEMU simulator's RDMA application issues receive work queue entries (Recv WQE) for receiving data; these work queue entries are stored in a memory component (see...). Figure 5 Step 1);
[0122] ROCEv2 messages received from the outside are stored in the local buffer of the APP component via a third socket, and a response (resp) task is added to the thread pool to process the received messages (see...). Figure 5 Step 2);
[0123] The APP component reads basic information from the registers of the backend QEMU emulator via the response task thread (see...). Figure 5Step 3) is used to compare the information in the wqe and received messages;
[0124] The APP component reads work queue entry information from the memory component of the front-end QEMU simulator through the response task (resp) thread, parses the opcode and global identifier (GID) information in the received message, and verifies it against the local table entry; after successful verification, the payload in the message is written to the buffer specified by the receive work queue entry (WQE) through the third socket (see...). Figure 5 Step 4);
[0125] The APP component fills in the completion queue entry (CQE) information and writes it to the completion queue entry buffer (CQE buffer) of the front-end QEMU emulator via Socket; if the RDMA application component of the front-end QEMU emulator is configured for event reporting, it sends an interrupt message to the front-end QEMU emulator via the response task (resp) thread; otherwise, it only updates the index of the completion queue entry (see...). Figure 5 Step 5).
[0126] As another aspect of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement, as follows: Figure 1 The system's functions are described. For more details, please refer to and combine with the foregoing descriptions. Figure 1 The description of that will not be repeated here.
[0127] As another aspect of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements as follows: Figures 2 to 5 The steps of the described method. For more details, please refer to and combine with the foregoing descriptions. Figures 2 to 5 The description of that will not be repeated here.
[0128] As another aspect of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the aforementioned... Figures 3 to 4 The steps of the described method. For more details, please refer to and combine with the foregoing descriptions. Figures 3 to 4 The description of that will not be repeated here.
[0129] Implementing this embodiment has the following beneficial effects:
[0130] This application provides a method, system, device, and storage medium for front-end and back-end simulation of RDMA network cards. By using two QEMU simulators and one APP component to simulate the hardware design and overall operation logic of RDMA network cards, the resource cost of real hardware simulation can be reduced, and the time cost of building software architecture can be reduced. It can also verify real hardware solutions, greatly saving chip design verification costs.
[0131] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0132] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0133] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.
Claims
1. A front-end and back-end simulation implementation system with RDMA network card, characterized in that, include: A front-end QEMU simulator used to simulate the front-end portion of an RDMA network card; A backend QEMU simulator used to simulate the backend portion of an RDMA network card; The APP component is used to simulate the RDMA message processing engine and simulate the RDMA message processing process, including at least: parsing hardware entries, constructing sent messages, processing received messages, and updating the completion queue. The front-end QEMU simulator, the back-end QEMU simulator, and the APP component interact with each other via a Socket mechanism to simulate the control plane data flow and data plane data flow of the RDMA network card. The APP component further includes: The third socket is used to communicate with the front-end QEMU simulator and the back-end QEMU simulator; Registers are used to store intermediate results and status information; The second DMA component is used to simulate the RDMA message processing engine; A thread pool component is used to manage multi-task threads during the RDMA network card emulation process. The multi-task threads include at least: request threads, completion threads, and response threads.
2. The system as described in claim 1, characterized in that, The front-end QEMU simulator further includes: The RDMA application component is used to generate RDMA requests and interact with the underlying hardware through the RDMA driver component. The RDMA driver component is used to receive requests from the RDMA application component, convert them into appropriate instructions or data packets, and then send them to the auxiliary device component or endpoint driver component. It is also responsible for handling responses and interrupts from the underlying hardware. Auxiliary device components are used to receive and process instructions or data packets from the RDMA driver components and interact with the endpoint driver components to coordinate the execution of RDMA operations. The endpoint driver component is used to receive requests from the RDMA driver, forward them to the hardware endpoint of the RDMA network card, and handle interrupts and responses from the hardware endpoint, returning them to the RDMA driver or upper-layer application. Memory components are used to interact with RDMA drivers, auxiliary devices, and endpoint drivers. In RDMA operations, they serve as source or destination memory regions for reading and writing data. The PCIE root complex is used to simulate the PCIE bus and communicate with the endpoint driver component, and is responsible for forwarding requests from the endpoint driver component to the PCIE endpoint device. PCIe endpoint devices are used to emulate virtual endpoint devices, receive requests from the PCIe root complex, and perform corresponding RDMA operations.
3. The system as described in claim 2, characterized in that, The PCIE endpoint device further includes: a multi-message signal interrupt component, a doorbell component, a first DMA component, a TLP component, and a first socket; it at least enables read and write operations on the memory component through the first socket, as well as the injection of interrupts into the front-end OS.
4. The system as described in claim 3, characterized in that, The backend QEMU simulator further includes: The second socket is used to communicate with the front-end QEMU simulator and the APP component; Registers are used to store virtual machine status information and instructions; TLP ring components are used to store and transmit TLP data packets; A DMA ring component is used to simulate data transfer between the DMA controller and memory; The Mbox component is used to receive Mbox requests and obtain the corresponding email communication data from the memory component of the front-end QEMU simulator; The RTOS component communicates with the register, TLP ring component, DMA ring component, and Mbox component respectively; it is used to process TLP messages, initiate front-end and back-end memory read / write requests, and initiate control plane SDMA requests and Mbox requests.
5. A method for front-end and back-end simulation implementation with RDMA network card, characterized in that, The method is implemented using the system described in any one of claims 1 to 4, and includes the steps of: A front-end QEMU simulator is provided to simulate the front-end part of the RDMA network card and realize the interaction with the front-end driver; A backend QEMU simulator is provided to simulate the backend part of the RDMA network card and realize backend control path processing; Provide an APP component to simulate an RDMA message processing engine, and simulate the RDMA message processing process, including parsing hardware entries, constructing sent messages, processing received messages, and updating the completion queue. The front-end QEMU simulator, the back-end QEMU simulator, and the APP component interact with each other via a Socket mechanism to simulate the control plane data flow and data plane data flow of the RDMA network card. The APP component further includes: The third socket is used to communicate with the front-end QEMU simulator and the back-end QEMU simulator; Registers are used to store intermediate results and status information; The second DMA component is used to simulate the RDMA message processing engine; A thread pool component is used to manage multi-task threads during the RDMA network card emulation process. The multi-task threads include at least: request threads, completion threads, and response threads.
6. The method as described in claim 5, characterized in that, Further, it includes the step of simulating the control plane data flow of the RDMA network card, including: The RDMA application component of the front-end QEMU simulator issues a command to request the creation of a new RDMA entry; After receiving the instruction, the RDMA driver component generates an RDMA table entry and stores it in the memory component; The RDMA driver component constructs the command to create the table entry, and notifies the RTOS component of the backend QEMU emulator through the first socket and the second socket, instructing it to obtain the table entry information and the address and size information of the memory component through the second DMA component; After receiving the notification, the RTOS component uses DMA technology to retrieve table entry information from the memory component of the front-end QEMU simulator; The RTOS component parses the obtained commands, extracts the table entry content from the memory component address of the front-end QEMU simulator, writes it to the register component, and notifies the APP component via Socket to cache the table entry content in the APP component's local buffer.
7. The method as described in claim 6, characterized in that, Further, it includes the step of simulating the data plane data flow of the RDMA network card, the data transmission process of which includes: The RDMA application component of the front-end QEMU simulator populates the Send WQE work queue entry and stores it in the memory component; The RDMA application component sends a doorbell signal, which is then sent to the APP component. The APP component puts the received request task req into the thread pool and reads the work queue entry from the memory component for processing. The APP component obtains basic information from the registers of the backend QEMU module through the request thread; The APP component reads the work queue entry WQE from the memory component DDR of the front-end QEMU module through the request thread and processes it in the local cache; The APP component encapsulates the opcode in the work queue entry into an RCOEv2 message and sends it to the other end via Socket. Add a completion task thread (comp) to handle retransmission and exceptions, populate the completion queue entry (CQE), and write it back to the completion queue entry of the front-end QEMU simulator via Socket; If the front-end QEMU emulator is configured to report events, the back-end QEMU emulator sends an interruption message to the front-end QEMU emulator; otherwise, only the CQE index is updated, and the front-end QEMU emulator obtains the completion message by checking the CQE update.
8. The method as described in claim 6, characterized in that, Further, it includes the step of simulating the data plane data flow of the RDMA network card, and its data reception process includes: The RDMA application of the front-end QEMU simulator issues a receive work queue entry Recv WQE for receiving data, and the work queue entry is stored in the memory component; ROCEv2 messages received from the outside are stored in the local buffer of the APP component through the third socket, and a response task resp is added to the thread pool to process the received messages. The APP component reads basic information from the registers of the backend QEMU emulator through the response task thread; The APP component reads work queue entry information from the memory component of the front-end QEMU simulator through the response task thread, parses the opcode and global identifier GID information in the received message, and verifies it with the local table entry; after successful verification, the payload in the message is written into the buffer specified by the received work queue entry through the third socket. The APP component fills in the completion queue entry CQE information and writes it to the completion queue entry buffer of the front-end QEMU simulator via Socket; if the RDMA application component of the front-end QEMU simulator is configured to report events, it sends an interrupt message to the front-end QEMU simulator through the response task thread; otherwise, it only updates the index of the completion queue entry.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 5 to 8.
10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the functions of the system as described in any one of claims 1 to 4.
Citation Information
Patent Citations
Network card equipment simulation system and method, electronic equipment and storage medium
CN117971400A